2.2: Types of Learning

Different Types of Learning

ML involves teaching computers to learn patterns and make decisions from data, primarily through different approaches based on the availability and type of data.

With the rise of (industrial scale) LLMs, various learning approaches have been developed and integrated, leading to new variations and hybrid methods. These techniques are applied at different stages of the LLM training and development process. The goal is to enhance the models’ efficiency, intelligence, and accuracy. See the “LLM context” below.

Here are some key learning paradigms:

• Supervised Learning

  • Learn from labeled input-output pairs.

  • Most common method for classification (e.g., is the given email a spam?) and regression (e.g., predicting house pricing)

  • In the LLM context. After initial self-supervised learning, models are fine-tuned using labeled data via various different methods.

• Unsupervised Learning

  • Learn patterns from unlabeled data. It focuses on discovering hidden patterns and structures within unlabeled data. Unlike supervised learning, which relies on pre-labeled datasets with known outputs, unsupervised learning algorithms are given raw, unclassified data and are tasked with finding inherent relationships and organizations without explicit guidance.

  • Examples: Group customers based on similar buying habits—like “budget shoppers”—without knowing the groups beforehand, and uncover connections between items in the data, such as finding that customers who buy bread often also buy butter.

  • In the LLM context. A variation of unsupervised learning—self-supervised learning—is the backbone of LLM (pre-)training. See below.

• Self-Supervised Learning

  • When (pre-)training large-scale LLMs in recent years, often trillions of tokens are required. It is clear that relying on human-annotated labeled data is simply practically impossible. This is where self-supervised learning method comes in: it is a method where the model trains itself by generating its own labels from unlabeled data. We say the models create “pretext tasks” that generate their own supervisory signals. Simply put, the models is to learn or predict the masked parts. It is cost- and time-efficient compared to supervised learning since it doesn’t require labeled data.

  • Computer vision: Learning visual representations by predicting parts of images or video, solving pretext tasks like colorization, patch order prediction, motion and depth estimation, and image reconstruction.

  • In the LLM context. It allows models to train on massive unlabeled text corpora to predict missing or next tokens, leveraging the structure of language to learn rich representations implicitly from the data itself.

• Semi-Supervised Learning

  • Combine small labeled datasets with large unlabeled data. It leverages the unlabeled data to improve the accuracy of a model trained with limited labeled data, making it particularly useful when labeling data is costly or time-consuming.

  • Workflow. First, the model is trained on (small) labeled data. Next, the model predicts labels on the larger unlabeled dataset, creating pseudo-labels. Then, the model is retrained with both the original labeled dataset and the pseudo-labeled data to refine the model for accuracy. This iterative cycle repeats until the model performance stabilizes or meets desired criteria. See, e.g., [1, 7, 12, 15].

  • For fine-tuning LLMs, see [11]: it improves LLMs alignment with knowledge propagation (from the small labeled dataset to the larger unlabeled dataset) and collaborative learning (selecting higher-quality pseudo-response samples from the generated unlabeled data).

  • See also: LESS (Large Language Model Enhanced Semi-supervised Learning) from NVIDIA [4] and applications with VoxPopuli unlabeled data (a large-scale multilingual speech corpus developed by Facebook; “VoxPopuli” means “voice of the people” in Latin) [16].

• Reinforcement Learning (including RLHF)

  • Optimize model behavior using human feedback as reward signals.

  • Central for alignment and helpfulness (GPT-3.5+, GPT-4, Gemini, DeepSeek).

• Continuous/Online Learning

  • Update models continuously with fresh or domain-specific data post-release.

  • Keeps models up to date and relevant (Gemini, LLaMA).

• Curriculum Learning

  • Train progressively with data/tasks of increasing complexity.

  • Enhances learning efficiency and robustness (Gemini, DeepSeek).

• Other types of learning: Transfer learning, in-context learning, multi-task learning, multitask learning (MTL), meta learning, federated learning, active learning, contrastive learning, neuro symbolic AI, collaborative Learning, and more.

Further Exploration

We want to briefly discuss on how different types of learning are used/deployed in training large-scale LLMs (some would overlap with above discussion).

1. Training large-scale LLMs: the main phases: see, e.g., [14]. A recent review on LLM alignment: see [17].

2. See also

   • Instruction Tuning: After initial self-supervised training, models are fine-tuned using labeled data to follow instructions and improve task performance. See, e.g., [3, 18].

   • Instruction Pretraining: Improving LLMs efficiency by including synthetic instruction-response pairs instead of just raw text. This is done via an “instruction synthesizer,” an LLM specifically finetuned for this task. See [2].

   • Rejection sampling (RS): filter out less desirable responses from a model’s output and only use the high-quality ones for further training; e.g., in Llama 3 (see [10]).

   • Parameter-Efficient Fine-Tuning (PEFT). Instead of fine-tuning all the parameters of a pre-trained LLM, it focuses on adjusting a small subset of parameters (a cost-efficient method scaling to large models with smaller compute). Use task-specific datasets containing labeled data (input-output pairs) that guide the model in learning the desired task. See, e.g., [5].

   • On Prompt Tuning (a technique that focuses on training or adjusting a small set of parameters called “soft prompts” or “prompt parameters” alongside the input text to improve the model’s performance on specific tasks): see, e.g., [9].

   • Direct Preference Optimization (DPO)

     • Directly optimize for human preferences without traditional RL loops.

     • Emerging alternative to PPO-based RLHF used in Gemini, GPT-4.5, DeepSeek.

   • Synthetic Data Training

     • Use model- or tool-generated data, especially for instruction tuning.

     • Enables large-scale generation of QA and reasoning datasets (LLaMA 3, Gemini).

   • Instruction Pretraining

     • Incorporate instruction-like data directly in base pretraining.

     • Models gain native instruction-following ability (Phi-3, LLaMA 3, Gemini).

   • Constitutional AI

     • Use AI-generated or rule-based feedback alongside or instead of human labeling.

     • Aims to improve alignment and safety with less human annotation (Gemini, GPT-4).

   • Multimodal Joint Pretraining

     • Train jointly on multiple data types: text, images, audio, code.

     • Supports versatile vision-language capabilities (Gemini, LLaVA, Qwen-VL).

   • Mixture of Experts (MoE)

     • Sparse activation of specialized subnetworks (experts) selected dynamically.

     • Scales model capacity efficiently (LLaMA 3, GPT-4 reported, Gemini).

   • Knowledge-Enhanced Pretraining

     • Integrate structured external knowledge (e.g., knowledge graphs) during training.

     • Improves reasoning and interpretability (some Gemini variants).

   • Weight-Generative Fine-Tuning (WeGeFT)

     • Dynamically generate fine-tuning weights for efficient task adaptation.

     • Cutting-edge research advancing fine-tuning (2025).

   • Zero-shot/Few-shot Prompting Zero-shot, One-shot, and Few-shot Learning

     • Use natural language prompts with zero or few examples to induce new behavior.

     • Core capability of all modern LLMs (GPT, Gemini, etc.).

   • Retrieval-Augmented Generation (RAG)

     • Combine retrieval of external documents with generation for more factual output.

     • Used in Gemini, GPT-4, DeepSeek-VL.

   • Long Context/Chunked Training

     • Specialized training for extremely long input sequences (100K+ tokens).

     • Enables handling very large documents (Gemini, LLaMA 3).

   • Knowledge Distillation/Model Compression

     • Train smaller “student“ models to mimic larger “teacher” models.

     • Reduces inference cost while preserving performance (Gemini Nano, DistilGPT).

3. Some recent surveys on LLMs: see [6, 8, 13, 19].

References:

1. Chapelle, O., Schölkopf, B., & Zien, A. (Eds.). (2006). Semi-supervised learning. MIT Press.
2. Cheng, D., Gu, Y., Huang, S., Bi, J., Huang, M., & Wei, F. (2024). Instruction pre-training: Language models are supervised multitask learners. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (pp. 2529–2550). Association for Computational Linguistics.
3. DeepLearningAI. (2023, May 31). Building with Instruction-Tuned LLMs: A Step-by-Step Guide [Video]. YouTube.
4. Ding, W., & Qian, F. (2025). LESS: Large language model enhanced semi-supervised learning for speech foundational models. arXiv:2506.04586v3.
5. Han, Z., Gao, C., Liu, J., Zhang, J., & Zhang, S. Q. (2024). Parameter-efficient fine-tuning for large models: A comprehensive survey. Transactions on Machine Learning Research (TMLR). arXiv:2403.14608.
6. Kalyan, K. S. (2024). A survey of GPT-3 family large language models including ChatGPT and GPT-4. Natural Language Processing Journal, 6.
7. Kingma, D. P., Rezende, D. J., Mohamed, S., & Welling, M. (2014). Semi-supervised learning with deep generative models. Advances in Neural Information Processing Systems, 27, NeurIPS 2014, 3581–3589.
8. Kumar, P. (2024). Large language models (LLMs): Survey, technical frameworks, and future challenges. Artificial Intelligence Review, 57(10), Article 260.
9. Lester, B., Al-Rfou, R., & Constant, N. (2021). The power of scale for parameter-efficient prompt tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 3045–3059). Association for Computational Linguistics.
10. Llama Team, AI @ Meta. (2024, July). The Llama 3 Herd of Models. arXiv:2407.21783.
11. Luo, J., Luo, X., Chen, X., Xiao, Z., Ju, W., & Zhang, M. (2025). Semi-supervised fine-tuning for large language models. Findings of the North American Chapter of the Association for Computational Linguistics (NAACL). arXiv:2410.14745.
12. Ouali, Y., Hudelot, C., & Tami, M. (2020). An overview of deep semi-supervised learning. arXiv:2006.05278.
13. Shahzad, T., Mazhar, T., Tariq, M. U., Ahmad, W., Ouahada, K., & Hamam, H. (2024). A comprehensive review of large language models: Issues and solutions in learning environments. Sustainable Computing: Informatics and Systems.
14. Snorkel AI. (2023, April 20). Large language model training: Three phases that shape LLM training. Snorkel AI Blog.
15. Van Engelen, B., & Hoos, H. (2019). A survey on semi-supervised learning. Machine Learning, 109(2), 373–440.
16. Wang, C., Rivière, M., Lee, A., Wu, A., Talnikar, C., Haziza, D., Williamson, M., Pino, J., & Dupoux, E. (2021). Voxpopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation. arXiv:2101.00390.
17. Wang, Z., Bi, B., Pentyala, S. K., Ramnath, K., Chaudhuri, S., Mehrotra, S., Zhu, Z. J., Mao, X.-B., Asur, S., & Cheng, N. (2024). A comprehensive survey of LLM alignment techniques: RLHF, RLAIF, PPO, DPO and more. arXiv:2407.16216.
18. Zhang, S., Dong, L., Li, X., Zhang, S., Sun, X., Wang, S., Li, J., Hu, R., Zhang, T., Wu, F., & Wang, G. (2023). Instruction tuning for large language models: A survey. arXiv:2308.10792.
19. Zhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., Du, Y., Yang, C., Chen, Y., Chen, Z., Jiang, J., Ren, R., Li, Y., Tang, X., Liu, Z., Liu, P., Nie, J.-Y., & Wen, J.-R. (2023). A survey of large language models. arXiv:2303.18223.