What is pretraining?

The early, broad training that gives an AI model many of its patterns and abilities before more focused training.

Also called Pre-training.

For example: A language model learns from a wide mix of writing before being trained for a specific job.