Summary book: Hands-On Large Language Models

This article synthesizes the core themes from “Hands-On Large Language Models,” providing a comprehensive overview of the current landscape of Language AI. The field has undergone a monumental shift, moving from statistical methods to complex neural networks, culminating in the highly capable Transformer architecture that powers today’s Large Language Models (LLMs). A critical distinction is made between representation models (e.g., BERT), which excel at tasks like classification and search by creating rich numerical representations of text, and generative models (e.g., GPT, Phi-3), which are trained to generate human-like text in response to prompts.

The ecosystem is divided into proprietary models (like GPT-4), accessed via APIs, and open models (like Llama 2), which offer greater control and transparency but require significant local hardware. The practical application of these models spans a wide range of use cases, including text classification, semantic search, summarization, and multimodal reasoning.

Mastery of LLMs involves several key techniques. Prompt engineering is crucial for guiding generative models to produce desired outputs. More advanced applications leverage Retrieval-Augmented Generation (RAG) to ground models in external, up-to-date information, reducing hallucinations and enhancing factuality. For domain-specific tasks, fine-tuning is employed to adapt pretrained models, with parameter-efficient methods like QLoRA making this process accessible without requiring massive computational resources. Evaluating these complex systems remains a challenge, relying on a combination of automated benchmarks, leaderboards, and essential human-preference-based assessments.

I. Foundational Concepts of Language AI

A. Defining Language Artificial Intelligence (AI)

Language AI is a field focused on developing systems capable of understanding and generating human language. It is a subfield of artificial intelligence, which is broadly defined as the science and engineering of creating intelligent machines and computer programs. The evolution of AI has been rapid, with the term often applied to a wide variety of systems, from simple rule-based logic in video games to the sophisticated deep neural networks that underpin modern LLMs.

A pivotal moment for Language AI was the 2022 release of ChatGPT, which demonstrated the technology’s potential to revolutionize human interaction with information and technology, reaching one hundred million active users in just two months. This event spurred massive investment and research into the underlying technology: Large Language Models (LLMs).

B. The Evolution of Language Models

The journey to modern LLMs involved several key architectural and conceptual shifts, each building upon the last to better capture the meaning and structure of human language.

C. Core Building Blocks: Tokens and Embeddings

1. Tokenization Before a model can process text, the text must be converted into a numerical format. This process begins with tokenization.

• Process: Tokenization splits a piece of text into smaller units called tokens. While simple whitespace splitting is a basic method, modern models use more advanced techniques to handle different languages and sub-word units.
• Vocabulary: After tokenization, all unique tokens are collected to create a vocabulary. Each unique token is assigned a numerical ID.
• Significance: The tokenizer is inextricably linked to its language model. A pretrained model must be used with the specific tokenizer it was trained with.

2. Embeddings Embeddings are numerical, vector representations of tokens, sentences, or documents that capture their semantic meaning.

• Token Embeddings: The language model maintains an embedding matrix, which holds a vector for each token in its tokenizer’s vocabulary.
• Contextualized Embeddings: Unlike static embeddings from models like word2vec, Transformer-based LLMs produce contextualized embeddings. The vector for a word changes based on the surrounding words in the sentence, allowing the model to disambiguate meaning.
• Text Embeddings: Specialized models can produce a single vector to represent an entire sentence, paragraph, or document. These are crucial for applications like semantic search and text clustering. The word2vec algorithm, which relies on the core ideas of skip-gram (predicting context words from a target word) and negative sampling (contrasting neighboring words with non-neighboring words), laid the groundwork for modern embedding techniques.

II. A Taxonomy of Language Models

A. Representation vs. Generative Models

The book makes a clear and continuous distinction between two primary types of language models based on their core function.

• Representation Models:
  • Architecture: Typically use an encoder-only architecture (e.g., BERT, RoBERTa).
  • Function: Designed to create rich, contextualized numerical representations (embeddings) of text. They do not generate new text.
  • Use Cases: Excel at language understanding tasks such as text classification, named-entity recognition, and as the foundation for semantic search systems (embedding models). They are often fine-tuned for a specific task.
• Generative Models:
  • Architecture: Typically use a decoder-only architecture (e.g., GPT series, Phi-3, Llama).
  • Function: Designed to predict the next token in a sequence, allowing them to generate new, human-like text in response to a prompt.
  • Use Cases: Power applications like chatbots, copywriting, summarization, and code generation. Their capabilities are often unlocked through prompt engineering.

B. Proprietary vs. Open Models

The LLM ecosystem is broadly divided into two categories based on accessibility and control.

C. The LLM Ecosystem

• Foundation Models: These are large, pretrained models (e.g., Llama, Falcon) that serve as the base for further specialization. They are often released in various sizes (e.g., 7B, 13B, 70B parameters) and can be fine-tuned for specific tasks like instruction following.
• Open Source Frameworks: A rich ecosystem of software facilitates working with open models.
  • Hugging Face: A central platform and company. The Hugging Face Hub is the main repository for finding and downloading models (over 800,000 at the time of writing), while the Transformers library is the core package for loading and running these models in code.
  • Execution Frameworks: Libraries like llama.cpp are created for efficiently loading and running LLMs, especially compressed (quantized) versions.
  • Application Frameworks: Libraries like LangChain provide tools to build complex applications by “chaining” LLMs with other components like memory, prompt templates, and external tools.

III. Applying Pretrained Language Models

A. Text Classification

Text classification involves assigning a category or label to a piece of text. LLMs can perform this task with high accuracy using several methods.

1. Using a Task-Specific Representation Model: This involves using a model (like BERT) that has already been fine-tuned for a specific classification task (e.g., sentiment analysis). This is a straightforward approach that offers strong performance if a suitable pretrained model exists.
2. Using an Embedding Model: This approach separates feature extraction from classification.
   • An embedding model (like all-mpnet-base-v2) converts all text documents into numerical embeddings. This model is kept “frozen.”
   • A traditional, lightweight classifier (e.g., Logistic Regression) is then trained on these embeddings. This is computationally cheaper than fine-tuning an entire LLM.
3. Zero-Shot Classification: This technique allows for classification without any labeled training examples.
   • It works by embedding both the input document and the text descriptions of the candidate labels (e.g., “a positive review,” “a negative review”).
   • Cosine similarity is used to measure the “distance” between the document embedding and each label embedding. The document is assigned the label with the highest similarity score.
4. Using Generative Models: Generative models like Flan-T5 or GPT-3.5 can be instructed to perform classification through prompting. By providing a prompt like "Is the following sentence positive or negative? [DOCUMENT]", the model will generate the classification as text.

B. Text Clustering and Topic Modeling

Text clustering is an unsupervised task that groups similar documents together without predefined labels, allowing for the discovery of topics within a corpus.

• Common Pipeline:
  1. Embed Documents: Use an embedding model to convert each document into a vector.
  2. Reduce Dimensionality: High-dimensional embeddings can be difficult for clustering algorithms. Techniques like UMAP are used to reduce the embedding dimensions (e.g., from 768 to 5) while preserving the global structure.
  3. Cluster Embeddings: A clustering algorithm like HDBSCAN is applied to the reduced-dimension embeddings to group them into clusters.
• BERTopic Framework: A modular framework that implements this pipeline. It uses a class-based TF-IDF (c-TF-IDF) variant to extract topic representations (keywords) for each cluster. Its modular design allows for swapping components (e.g., using different embedding or clustering models) and adding “Lego blocks” for advanced features like fine-tuning topic representations.

C. Semantic Search and Retrieval-Augmented Generation (RAG)

Semantic search goes beyond keyword matching to find documents based on conceptual meaning. This capability is the foundation of RAG.

1. Dense Retrieval (Semantic Search):
   • Indexing: All documents in a corpus are chunked and embedded into vectors, which are stored in a specialized vector index (e.g., using libraries like FAISS).
   • Querying: A user’s query is also embedded. The system then searches the index to find the document vectors that are closest to the query vector (nearest neighbors).
   • Chunking: Since LLMs have context limits, long documents must be split into smaller chunks before embedding. Strategies include splitting by sentence or paragraph, with overlapping chunks used to preserve context between them.
2. Rerankers: Search results from an initial retrieval step (either keyword-based like BM25 or dense) can be improved. A reranker model takes the query and a list of candidate documents and re-orders them based on relevance, significantly improving search quality.
3. Retrieval-Augmented Generation (RAG):
   • Purpose: RAG enhances generative models by grounding them in external information, reducing factual errors (hallucinations) and allowing them to answer questions about data not seen during their training.
   • Process:
     1. Retrieve: When a user asks a question, the system first performs a semantic search on a relevant knowledge base to find the most relevant text chunks.
     2. Augment: The retrieved chunks are added as context to the user’s original prompt.
     3. Generate: The augmented prompt is fed to a generative LLM, which then generates an answer based on both the original question and the provided context.

IV. Mastering Generative Models

A. Prompt Engineering

Prompt engineering is the art of designing inputs (prompts) to guide a generative model toward producing a high-quality, desired output.

• Basic Ingredients of a Prompt: While a simple question can work, structured prompts perform better. Components can include:
  • Persona: “You are an expert in…”
  • Instruction: The specific task to perform.
  • Context: Additional information to guide the model.
  • Data: The input text or information to be processed.
  • Format: Instructions on how to structure the output (e.g., “Create a bullet-point summary”).
  • Audience/Tone: “The summary is for busy researchers,” “The tone should be professional.”
• Controlling Model Output: Parameters can be adjusted to control the randomness of the generated text:
  • Temperature: Controls creativity. A value of 0 is deterministic (greedy decoding), while higher values increase randomness by allowing less probable tokens to be selected.
  • Top-p (Nucleus Sampling): Considers only the smallest set of most probable tokens whose cumulative probability exceeds the top_p value.
• Advanced Prompting Techniques:
  • Few-Shot Learning: Providing one or more examples of the task within the prompt to show the model the desired input-output pattern.
  • Chain Prompting: Breaking a complex task into a sequence of simpler prompts, where the output of one step becomes the input for the next.
  • Chain-of-Thought (CoT): Encouraging the model to “think step-by-step” by providing examples that include a reasoning process before the final answer. This improves performance on complex reasoning tasks.
  • Self-Consistency: Generating multiple reasoning paths (thoughts) for the same problem and choosing the most common answer via a majority vote.

B. Output Verification and Control

To build robust applications, the output of LLMs must be controlled and verified.

• Reasons for Verification:
  • Structured Output: Ensuring the output conforms to a specific format like JSON.
  • Validity: Preventing the model from generating invalid choices (e.g., a third option when asked for one of two).
  • Ethics & Safety: Filtering for profanity, bias, or personally identifiable information (PII).
  • Accuracy: Checking for factual correctness and reducing hallucinations.
• Methods for Control:
  • Prompting: Using few-shot examples to demonstrate the desired structure.
  • Constrained Generation (Grammars): Applying a grammar (e.g., a JSON schema) during the generation process to force the model to only produce tokens that conform to the required format. Libraries like llama-cpp-python support this natively.

C. Multimodal LLMs

Multimodal models can process and reason about information from multiple data types (modalities), such as text and images.

• CLIP (Contrastive Language–Image Pre-training): An embedding model that learns to place corresponding image and text embeddings close together in the same vector space. This allows for tasks like zero-shot image classification and text-to-image search.
• BLIP-2: An architecture that efficiently bridges the modality gap between a frozen vision model (like ViT) and a frozen LLM. It uses a lightweight, trainable Q-Former module to extract visual features that are then translated into “soft visual prompts” that the LLM can understand. This enables complex visual reasoning and chat-based interactions with images.

V. Training and Fine-Tuning Language Models

A. Creating Text Embedding Models with Contrastive Learning

While pretrained embedding models are widely available, they can be created or fine-tuned for specific needs using contrastive learning.

• Concept: The model is trained to distinguish between similar and dissimilar pairs of documents. It learns to pull embeddings of similar pairs closer together while pushing embeddings of dissimilar pairs further apart.
• Data Generation: Contrastive examples can be generated from datasets like Natural Language Inference (NLI) collections, where “entailment” pairs serve as positive examples and “contradiction” pairs serve as negative examples.
• SBERT (Sentence-BERT): A modification of the BERT architecture that uses a Siamese network structure to efficiently generate sentence embeddings suitable for comparison tasks. It is trained using contrastive objectives.
• Loss Functions: Different loss functions guide the training process, including CosineSimilarityLoss (for tasks with graded similarity scores) and MultipleNegativesRankingLoss (efficient for training with positive pairs and in-batch negative examples).
• Domain Adaptation: To adapt a model to a new domain (e.g., scientific papers), one can first perform unsupervised pretraining on domain text (e.g., using TSDAE) and then fine-tune on a small amount of labeled data.

B. Fine-Tuning Representation Models

Fine-tuning adapts a general-purpose pretrained model to a specific downstream task like classification or named-entity recognition.

• Full Fine-Tuning: All weights of the pretrained model are updated during training. This typically yields the best performance but is the most computationally expensive.
• Freezing Layers: To reduce computational cost, parts of the model can be “frozen” so their weights are not updated. One can train only the final classification head or unfreeze the top few layers of the model. Performance often stabilizes after unfreezing just a few of the top layers, offering a good trade-off between performance and efficiency.
• SetFit (Few-Shot Classification): An efficient framework for fine-tuning an embedding model with very few labeled examples. It first fine-tunes the model contrastively on sentence pairs generated from the few labels and then trains a classification head on the resulting embeddings.

C. Fine-Tuning Generative Models

• Instruction Tuning (Supervised Fine-Tuning): This process fine-tunes a base generative model on a dataset of instruction-response pairs. This teaches the model to follow instructions and act as a helpful assistant rather than just completing text.
• Parameter-Efficient Fine-Tuning (PEFT): Methods that avoid updating all of the model’s billions of parameters. Instead, they freeze the original model and train a small number of additional parameters.
  • LoRA (Low-Rank Adaptation): This popular PEFT method freezes the massive weight matrices of an LLM and injects small, trainable “low-rank” matrices alongside them. The outputs of the frozen and new matrices are combined. This dramatically reduces the number of trainable parameters (e.g., from 150 million to ~200k for a single matrix).
• Quantization: A compression technique that reduces the memory footprint and increases the speed of LLMs by representing their weights with lower precision (e.g., 16-bit or 8-bit, down to 4-bit). This makes it possible to run large models on consumer-grade hardware.
• QLoRA: A highly efficient technique that combines Quantization and LoRA. It involves loading a base model in a quantized 4-bit format and then performing LoRA fine-tuning on top of it. This makes fine-tuning billion-parameter models accessible on a single GPU.

VI. Evaluating Language Model Performance

Evaluating the quality of LLMs is a complex and multifaceted challenge.

A. Metrics for Classification and Search

• Classification: Standard metrics derived from a confusion matrix are used:
  • Precision: Of the items predicted as positive, how many were actually positive?
  • Recall: Of all actual positive items, how many were correctly identified?
  • F1-Score: The harmonic mean of precision and recall, providing a balanced measure.
  • Accuracy: The overall percentage of correct predictions.
• Search:
  • Mean Average Precision (MAP): A common metric for evaluating ranked search results across a suite of test queries. It rewards systems that not only find relevant documents but also place them at the top of the results list.

B. Evaluating Generative Models

Evaluating open-ended text generation is notoriously difficult.

• Automated Metrics: Traditional metrics like Perplexity (how well a model predicts a sequence of text), BLEU, and ROUGE (which measure n-gram overlap with reference texts) are used but are often poor proxies for human judgment.
• Benchmarks: Standardized test suites are used to measure model capabilities across various domains:
  • Examples include MMLU (multitask language understanding), GSM8k (grade school math), and HumanEval (code generation).
• Leaderboards: Platforms like the Open LLM Leaderboard aggregate results from multiple benchmarks to provide a comparative ranking of models. However, there is a risk of models “overfitting” to the public test sets.
• LLM-as-a-Judge: Using a powerful LLM to evaluate the output of another model, often through pairwise comparison (“Which response is better?”). This allows for automated evaluation of open-ended questions.
• Human Evaluation: This is considered the gold standard. Platforms like the Chatbot Arena collect crowdsourced human preference data by having users interact with two anonymous models and vote for the better one. Ultimately, the best evaluation method is tailored to the specific use case of the model.

Get the Book