Technical Notes on Modern LLM Training Methods: DPO, RLVR/GRPO, PEFT, MoE, Distillation, RLAIF, Long-Context, Synthetic Data, Model Merging, and Curriculum Learning

This collection gathers ten short technical notes covering modern methods used to train and adapt large language models: direct preference optimization, RLVR with GRPO, parameter-efficient fine-tuning (LoRA, QLoRA, DoRA), sparse mixture-of-experts and upcycling, knowledge and self-distillation, RLAIF and constitutional AI, long-context extrapolation, synthetic data for self-improvement, model merging and capability composition, and curriculum learning for data efficiency. Each note summarizes the core ideas, trade-offs, and practical considerations of the corresponding technique.