Introduction to LLMs in Python

What Makes a Language Model “Large”

A Large Language Model is a deep learning model, almost always built on the transformer architecture, trained on massive volumes of text so it can understand and generate human language. The word “large” refers to parameter count: millions to billions of numerical weights learned during training. Training one from scratch costs millions of dollars, but thanks to the Hugging Face Hub you can download pre-trained models and use them immediately via the transformers library. The simplest entry point is the pipeline function, which bundles a model, a tokenizer, and task-specific logic into a single callable.

The pipeline API

pipeline lets you run an NLP task in two to three lines. You specify what you want to do (task) and which model should do it (model).

from transformers import pipeline
summarizer = pipeline(
    task="summarization",
    model="facebook/bart-large-cnn"
)
text = "Long article text goes here..."
summary = summarizer(text, max_length=50, clean_up_tokenization_spaces=True)
print(summary[0]["summary_text"])

Think of pipeline like ordering from a restaurant menu. You say what you want (“summarization”) and name the chef (BART), and the kitchen handles everything: sourcing ingredients, cooking, and plating. You never touch the oven. task="summarization" tells the wrapper which post-processing logic to apply, model="facebook/bart-large-cnn" specifies the checkpoint to download, and max_length=50 caps the output at 50 tokens. Pipelines always return a list of dictionaries, and summary[0]["summary_text"] pulls the actual string from the first result.

Common Tasks at a Glance

Text Generation

Decoder-style models like GPT-2 continue a prompt one token at a time.

generator = pipeline(task="text-generation", model="gpt2")
prompt = "The Gion neighborhood in Kyoto is famous for"
output = generator(
    prompt,
    max_length=100,
    pad_token_id=generator.tokenizer.eos_token_id
)
print(output[0]["generated_text"])

GPT-2 is like an autocomplete engine that never stops. You give it the start of a sentence and it keeps predicting “what word comes next?” one token at a time until it hits the length limit. max_length=100 sets the total token budget for both the prompt and the generated continuation. pad_token_id is housekeeping: GPT-2 was never given an explicit padding token during training, so you borrow the end-of-sequence token for that role to prevent a warning.

Translation

The Helsinki NLP group released hundreds of small encoder-decoder models, one per language pair.

translator = pipeline(
    task="translation_es_to_en",
    model="Helsinki-NLP/opus-mt-es-en"
)
source_text = "Este curso sobre LLMs se está poniendo muy interesante"
translations = translator(source_text, clean_up_tokenization_spaces=True)
print(translations[0]["translation_text"])

The direction is encoded directly in the task string: es_to_en means Spanish to English. The encoder reads the source text and compresses it into a dense representation; the decoder uses that representation to write the target language word by word. Notice that translation pipelines return results under translation_text, distinct from the generated_text key used in generation and the summary_text key used in summarization.

Transformer Architectures

The transformer family splits into three flavors, and choosing the right one comes down to matching architecture to task shape.

Encoder-only models (BERT, DistilBERT, RoBERTa) are built for understanding input. They read the full sequence bidirectionally and produce rich contextual representations, making them ideal for classification, named-entity recognition, and extractive question answering. Decoder-only models (GPT-2, GPT-3, LLaMA) are built for generating output. They read text left to right and predict the next token, which is why they dominate text generation and conversational AI. Encoder-decoder models (T5, BART, mBART) handle tasks where both input and output are sequences of different shapes, such as translation, summarization, and paraphrasing.

You can inspect any loaded pipeline’s internals directly.

llm = pipeline(model="bert-base-uncased")
print(llm.model)
print(llm.model.config)

.model exposes the full neural network as a PyTorch module tree, showing every layer and sub-module. .model.config is the spec sheet: hidden layer size, number of attention heads, vocabulary size, and every hyperparameter used during pre-training. You would consult these the same way an engineer reads a circuit diagram — not to change anything, just to understand what is inside.

Fine-tuning: Teaching a Model Your Task

Fine-tuning takes a pre-trained model and continues training it on your own labeled data. It almost always outperforms zero-shot prompting when you have task-specific examples, and it lets the model adapt general language knowledge to the patterns that matter for your use case.

Loading data

from datasets import load_dataset
train_data = load_dataset("imdb", split="train")
train_data = train_data.shard(num_shards=4, index=0)
test_data = load_dataset("imdb", split="test")
test_data = test_data.shard(num_shards=4, index=0)

The IMDB dataset ships with 25,000 training reviews. .shard(num_shards=4, index=0) cuts the dataset into four equal parts and picks only the first, giving you a representative 25% in a fraction of the time. Think of it as cutting a deck of cards into four piles and working with just one. In production you would use all shards; during experimentation, one is enough to verify the pipeline works.

Loading model and tokenizer

from transformers import AutoModelForSequenceClassification, AutoTokenizer
sentiment_model = AutoModelForSequenceClassification.from_pretrained("distilbert-base-uncased")
sentiment_tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased")

Auto* classes work like universal adapters: you give them a model name and they figure out which specific architecture class to instantiate. AutoModelForSequenceClassification loads the base model and attaches a classification head on top, replacing the vocabulary-sized output layer with one output slot per label. The tokenizer must come from the same checkpoint as the model since each pre-trained model uses a different vocabulary and set of special tokens.

Tokenization

Neural networks cannot read words. Tokenization translates each word, or sub-word piece, into an integer ID from the model’s vocabulary.

tokenized_training_data = sentiment_tokenizer(
    train_data["text"],
    return_tensors="pt",
    padding=True,
    truncation=True,
    max_length=64
)

padding=True adds dummy tokens to short sequences so every row in a batch reaches the same length — like packing a box with foam so nothing rattles. truncation=True cuts anything longer than max_length. Together they ensure every input is a uniform 64-token rectangle the model can process. return_tensors="pt" outputs PyTorch tensors rather than plain Python lists.

For datasets too large to fit in memory at once, .map() applies tokenization lazily, one row or batch at a time.

def tokenize_function(data):
    return sentiment_tokenizer(
        data["text"],
        return_tensors="pt",
        padding=True,
        truncation=True,
        max_length=64
    )
tokenized_by_row = train_data.map(tokenize_function, batched=False)

.map() works like a conveyor belt, feeding rows through your function and writing results back to disk incrementally so you never need the whole dataset in memory. batched=True is faster for large datasets since it passes groups of rows at once rather than one at a time.

The Trainer API

from transformers import Trainer, TrainingArguments
training_args = TrainingArguments(
    output_dir="./results",
    evaluation_strategy="epoch",
    num_train_epochs=3,
    learning_rate=2e-5,
    per_device_train_batch_size=8,
    per_device_eval_batch_size=8,
    weight_decay=0.01,
)
trainer = Trainer(
    model=sentiment_model,
    args=training_args,
    train_dataset=tokenized_training_data,
    eval_dataset=tokenized_test_data,
    tokenizer=sentiment_tokenizer,
)
trainer.train()

TrainingArguments is the recipe card: how many times to read through the data (num_train_epochs), how large each weight update should be (learning_rate), how many examples to process at once (per_device_train_batch_size), and how strongly to penalize overconfident weights (weight_decay). Trainer follows that recipe without you writing a single training loop. evaluation_strategy="epoch" runs validation at the end of each pass, and 2e-5 is the standard fine-tuning learning rate for transformer models.

Running inference after fine-tuning

import torch
new_reviews = ["This film was genuinely moving and beautifully acted."]
new_input = sentiment_tokenizer(
    new_reviews,
    return_tensors="pt",
    padding=True,
    truncation=True,
    max_length=64
)
with torch.no_grad():
    outputs = sentiment_model(**new_input)
predicted_labels = torch.argmax(outputs.logits, dim=1).tolist()
label_map = {0: "NEGATIVE", 1: "POSITIVE"}
for i, predicted_label in enumerate(predicted_labels):
    print(new_reviews[i])
    print(label_map[predicted_label])
sentiment_model.save_pretrained("sentiment_model")
sentiment_tokenizer.save_pretrained("sentiment_tokenizer")

During training, PyTorch tracks every calculation so it can compute gradients and update weights. At inference that bookkeeping is unnecessary, so torch.no_grad() switches it off: same arithmetic, less memory, faster execution. The model outputs raw class scores called logits; torch.argmax(..., dim=1) picks the highest-scoring class per row, like choosing the tallest bar on a chart, and label_map translates the winning index back into a readable word. save_pretrained writes both the weights and the tokenizer vocabulary to disk so you can reload everything later without retraining.

Fine-tuning Approaches Compared

Few-shot prompting deserves a closer look because it requires no training data and no gradient updates. You write one worked example directly into the prompt and the model pattern-matches it to answer the next input.

input_text = """
Text: "The dinner we had was great and the service too."
Classify the sentiment of this sentence as either positive or negative.
Example:
Text: "The food was delicious"
Sentiment: Positive
Text: "The dinner we had was great and the service too."
Sentiment:
"""
result = model(input_text, max_length=100)
print(result[0]["label"])

It is the equivalent of showing a student one solved problem on a test and asking them to solve the next one the same way. No datasets, no compute, no cost — just a prompt that teaches by example.

Evaluating LLM Performance

Different tasks need different metrics. The evaluate library is the standard tool in the Hugging Face ecosystem.

Classification metrics

import evaluate
accuracy  = evaluate.load("accuracy")
precision = evaluate.load("precision")
recall    = evaluate.load("recall")
f1        = evaluate.load("f1")
print(accuracy.description)
print(f"Required data types: {accuracy.features}")

Each evaluate.load call downloads the corresponding metric module from the Hub. .description explains what the metric measures in plain English; .features shows the expected input schema. You compute each metric by passing the ground-truth labels alongside your model’s predictions.

predicted_labels = torch.argmax(outputs.logits, dim=1).tolist()
print(accuracy.compute(references=true_labels, predictions=predicted_labels))
print(precision.compute(references=true_labels, predictions=predicted_labels))
print(recall.compute(references=true_labels, predictions=predicted_labels))
print(f1.compute(references=true_labels, predictions=predicted_labels))

Think of references as the answer key and predictions as the student’s answers. Accuracy counts how many are exactly right. Precision asks “of the inputs you labeled positive, how many actually were?” Recall asks “of the inputs that were actually positive, how many did you catch?” F1 is the harmonic mean of those two, balancing both concerns in a single number.

Perplexity

input_text = "Current trends show that by 2030 "
input_ids = tokenizer(input_text, return_tensors="pt").input_ids
output = model.generate(input_ids, max_length=20)
generated_text = tokenizer.decode(output[0], skip_special_tokens=True)
perplexity = evaluate.load("perplexity", module_type="metric")
results = perplexity.compute(model_id="gpt2", predictions=generated_text)
print("Perplexity:", results["mean_perplexity"])

Perplexity measures how surprised a reference language model is by a piece of text. Lower surprise means the text reads as more natural and fluent under that model’s understanding of language. A perplexity of 10 means the text was very predictable; a perplexity of 1000 means it was baffling. The metric says nothing about factual correctness — only linguistic fluency as judged by the reference model (GPT-2 here). skip_special_tokens=True strips any <bos> or <eos> markers from the decoded string before scoring.

BLEU

translator = pipeline("translation", model="Helsinki-NLP/opus-mt-es-en")
bleu = evaluate.load("bleu")
source_sentences = ["Hola, ¿cómo estás?", "Estoy genial, gracias."]
references = [
    ["Hello, how are you?", "Hi, how are you?"],
    ["I'm great, thanks.", "I'm great, thank you."],
]
translated_outputs = translator(source_sentences)
predictions = [t["translation_text"] for t in translated_outputs]
results = bleu.compute(predictions=predictions, references=references)
print(results)

BLEU compares your translation against one or more human references by counting how many short word sequences they share. Each input can have multiple valid references because “Hi, how are you?” and “Hello, how are you?” are both correct translations. BLEU picks whichever reference is most generous for each phrase and scores accordingly. A precision of 1.0 means every n-gram you produced appears in a reference; 0.0 means none did.

ROUGE

rouge = evaluate.load("rouge")
predictions = ["Pluto is a dwarf planet in our solar system..."]
references  = ["Pluto is a dwarf planet in the solar system..."]
results = rouge.compute(predictions=predictions, references=references)
print("ROUGE results:", results)

ROUGE is BLEU’s recall-oriented counterpart, designed for summarization. Where BLEU asks “of the words you wrote, how many appear in the reference?”, ROUGE asks “of the words in the reference, how many did you include?” For summaries you care more about not missing important content than about avoiding a few extra words, which is why recall is the primary concern. ROUGE-1 checks single words, ROUGE-2 checks word pairs, and ROUGE-L checks the longest common subsequence.

METEOR

meteor = evaluate.load("meteor")
generated = ["The burrow stretched forward like a narrow corridor..."]
reference = ["The rabbit-hole went straight on like a tunnel..."]
results = meteor.compute(predictions=generated, references=reference)
print("METEOR:", results["meteor"])

BLEU and ROUGE only count exact word matches, so “burrow” and “rabbit-hole” score zero overlap even though they mean the same thing. METEOR consults a thesaurus to recognize synonyms and also handles different word forms (run, ran, running). This makes its scores better correlated with human judgment, at the cost of the WordNet lookup dependency it brings along.

Exact Match

exact_match = evaluate.load("exact_match")
predictions = ["It's a wonderful day", "I love dogs", "The answer is forty-two"]
references  = ["What a wonderful day",  "I love cats", "The answer is forty-two"]
results = exact_match.compute(predictions=predictions, references=references)
print("EM results:", results)

Exact Match is the strictest possible grader: a prediction is either a character-for-character copy of the reference or it scores zero. One extra space, one different word, zero credit. It is harsh but appropriate for extractive QA, where the model should be lifting the exact span from a document rather than paraphrasing. Only the third example here matches exactly, giving an EM score of 1/3.

Toxicity

toxicity_metric = evaluate.load("toxicity")
toxicity_scores = toxicity_metric.compute(predictions=comments_a)
print("Per-input toxicities:", toxicity_scores["toxicity"])
max_toxicity = toxicity_metric.compute(predictions=comments_a, aggregation="maximum")
print("Worst score:", max_toxicity["max_toxicity"])
toxicity_ratio = toxicity_metric.compute(predictions=comments_a, aggregation="ratio")
print("Fraction flagged:", toxicity_ratio["toxicity_ratio"])

The toxicity metric runs each string through a classifier trained on human-labeled harmful content and returns a probability between 0 and 1. The three aggregation modes answer different safety questions. Without aggregation you get per-input probabilities, useful for inspecting individual outputs. aggregation="maximum" returns the worst score in the batch, useful for flagging outliers. aggregation="ratio" returns the fraction of inputs that exceeded the harmful threshold, useful for monitoring output quality at scale.

Regard

regard      = evaluate.load("regard")
regard_comp = evaluate.load("regard", "compare")
group_a = ["Members of this group are described as loyal", "They are honest but kind"]
group_b = ["Members of this group are known for being confrontational", "They are very blunt"]
print(regard.compute(data=group_a))
print(regard.compute(data=group_b))
print(regard_comp.compute(data=group_a, references=group_b))

Regard measures whether a model uses the same tone when writing about different demographic groups. You replace the group name with a neutral placeholder and compare the polarity distribution across groups. If the model consistently produces warmer language for one group than another, that gap is the bias signal. The compare variant runs both groups in a single call and surfaces the polarity difference directly, making the disparity easier to read than two separate calls.

See you soon.