Pre-training is the heavy, costly stage of building an LLM: the model absorbs trillions of tokens of public text (web, books, code) until language structure and general knowledge settle into its parameters. The output is a raw generalist — smart but unspecialized and occasionally unpolite.
This stage’s price is the main reason the industry fixates on open models and distillation: building a frontier model costs millions in GPU time, so re-running it is rarely rational — you reuse what exists.
Key point: pre-training builds capability, not persona. Turning that capability into a useful assistant is the job of the next stages: fine-tuning and RLHF.
