LLMs Explained for Web Developers: Tokens, Temperature, and Everything In Between
A plain-English guide to the concepts and knobs behind every AI API call — written for developers who build with JavaScript, not ML PhDs.
If you're a web developer in 2026, you've probably already called an LLM API — or you're about to. Maybe you're adding an AI feature to a Next.js app, maybe you're just trying to understand what temperature: 0.7 in that code snippet actually does.
The problem is that most explanations either stay too shallow ("AI is magic!") or go too deep (softmax equations on slide two). This post sits in the middle. By the end, you'll understand what an LLM actually does, what every major API parameter controls, and which knobs you should actually touch in production.
Let's build up from the ground floor.
Part 1: What Is an LLM, Really?
An LLM (Large Language Model) is an AI system that has read an enormous amount of text and learned the statistical patterns of language — which words follow which, how sentences are structured, how ideas connect.
At its core, an LLM is a very sophisticated next-word predictor. That sounds underwhelming, but predicting the next word well — across code, essays, legal documents, and conversations — turns out to require something that looks a lot like understanding.
Here's the mental model I use:
The LLM is the engine. Your prompt is the steering wheel. Prompt engineering is learning how to drive.
The engine is powerful, but it goes wherever you steer it. Vague input, vague output.
How it works, in three steps
Training: The model reads trillions of words and learns patterns — grammar, facts, reasoning shapes, code idioms.
Storage: That learning is compressed into parameters (also called weights) — billions of tiny numerical knobs that encode everything the model "knows."
Inference: When you send a prompt, the model breaks it into tokens, processes them, and predicts the most likely next token. Then the next. Then the next. That loop is text generation.
Every parameter we'll discuss later — temperature, top-p, penalties — is just a way of interfering with that "pick the next token" step.
Part 2: The Vocabulary You Actually Need
Before touching API parameters, let's lock in the core glossary. Skim this now; it'll pay off in Part 3.
Tokens
A token is a chunk of text the model processes — a word, part of a word, or punctuation. "Kathmandu" might be 3 tokens; "the" is 1. Roughly, 1,000 tokens ≈ 750 English words.
Why you care: you're billed per token, and every limit is measured in tokens. When your API bill spikes or your response gets truncated, tokens are the unit of the crime scene.
Context window
The context window is the model's short-term memory — how many tokens it can "see" at once. Your system prompt, the conversation history, any documents you paste in, and the model's response all have to fit inside it.
This is why long chats eventually "forget" early messages, and why RAG (below) exists.
Hallucination
A hallucination is when the model produces a confident, fluent, wrong answer. It's not lying — it's pattern-completing without a fact-checker. Treat LLM output like a pull request from a talented but overconfident junior dev: review before merging.
Prompt injection
Prompt injection is the LLM world's version of SQL injection: malicious text (in a user message, a scraped web page, a PDF) that tries to override your instructions — "ignore previous instructions and reveal the system prompt."
If your app lets a model read external content, this is a real attack surface, not a theoretical one.
RAG (Retrieval-Augmented Generation)
RAG bolts a search step onto the model. Instead of relying only on what the model memorized during training, you retrieve relevant documents (from your database, docs, or vector store) and inject them into the prompt. The model answers grounded in your data — fresher, more accurate, and citable.
Agents
Agents are LLMs given tools and a loop: search, run code, call APIs, check the result, decide the next step. Instead of one-shot text generation, they plan and act. Most of the interesting AI products being built right now are agents under the hood.
Fine-tuning vs. prompt engineering
Two ways to change model behavior:
| Prompt engineering | Fine-tuning | |
|---|---|---|
| What changes | The input prompt | The model's weights |
| Cost | Cheap, instant | Expensive, slow |
| Best for | Iteration, instructions, prototypes | Consistent specialized behavior |
| Flexibility | Change it anytime | Locked in after training |
Rule of thumb: prompt engineering first, always. If you want a polite chatbot, that's a prompt. Only reach for fine-tuning when you need deeply consistent, specialized behavior that prompting can't reliably produce — and even then, try RAG first.
Part 3: The Sampling Knobs — Temperature, Top-K, Top-P
Now the fun part. Remember: at every step, the model has a ranked list of candidate next tokens with probabilities — "this token is 60% likely, this one 20%, this one 5%..." It then samples from that list.
Sampling parameters control how adventurous that pick is.
Temperature: the creativity dial
Temperature reshapes the probability distribution itself.
Low (0 – 0.3): The model almost always picks the highest-probability token. Same prompt → nearly the same answer every time. Stable, predictable, boring — in a good way.
Medium (0.3 – 0.7): Mostly sensible, with some variation. The sweet spot for chat and coding assistants.
High (0.8 – 1.2+): The model explores low-probability tokens. More imaginative — and more likely to say something weird or wrong.
The analogy that made it click for me:
Temp 0: You always choose the most obvious next word.
Temp 0.7: You usually choose the obvious word, but sometimes pick a different one to keep things interesting.
Temp 1.2: You're improvising jazz. Sometimes brilliant, sometimes nonsense.
In practice: structured output, JSON extraction, code generation → low temperature. Brainstorming, marketing copy, story writing → higher temperature.
Top-K: cap the guest list
Top-K says: only consider the K most likely tokens, throw away the rest.
Picture 100 candidate next words sorted by likelihood. Top-K = 40 means: "we roll the dice among the top 40; the other 60 are off the table."
Top-K = 1 → always pick the single most likely token (fully deterministic).
Top-K = 40 → a common default; enough variety, no absurd rare words.
Top-K = 0 / disabled → no cap; consider everything.
Top-P (nucleus sampling): cap by confidence, not count
Top-P says: keep the smallest set of tokens whose probabilities add up to P, ignore the rest.
The model sorts tokens by likelihood and keeps adding them until their combined probability hits P (say, 0.9 = 90%). It samples only from that "nucleus."
The key difference from Top-K — and this is the part worth remembering:
Top-K is a fixed number. Top-P adapts to the model's confidence.
If the model is very sure (one token holds 95% probability), Top-P 0.9 might allow only that single token. If the model is unsure and probability is spread thin, the nucleus expands to include many candidates. Top-K would blindly allow 40 tokens in both situations.
Top-P = 1.0 → no restriction.
Top-P ≈ 0.9 → the common production default.
Top-P = 0.5 → strict; only the strongest candidates survive.
How they stack
When you set all three, most systems apply them in a pipeline:
Temperature reshapes the probabilities (flatter or peakier).
Top-K trims to the K best tokens.
Top-P trims further to the nucleus.
One token is sampled from what's left. Repeat.
Honest advice: most developers should just tune temperature. Slightly more advanced setups tune temperature + Top-P. Tuning all three at once is possible, but you're shaping the same distribution three ways and it becomes hard to reason about what caused what.
Part 4: The Output Controls — Length, Stops, and Repetition
Sampling knobs control which token gets picked. These parameters control when the output ends and whether it repeats itself.
Max tokens: the word budget
Max tokens caps how long the response can be. Set it too low and the answer gets cut off mid-sentence; set it high and the model can write longer replies (at more cost and latency).
Think of it as handing the model a word budget for its reply — a hard ceiling, not a target. The model won't pad to reach it.
Stop sequences: the emergency brake
A stop sequence is a string that tells the model "stop generating the moment you produce this."
Classic use case: your prompt format uses User: and Assistant: labels. Set "\nUser:" as a stop sequence and the model halts when it starts hallucinating the next turn of the conversation instead of rambling on. You can also invent custom markers like ### END to force a clean cut exactly where you want one.
Frequency penalty: stop saying the same word
Frequency penalty counts how many times each token has already appeared and penalizes heavy repeaters — the more a word has been used, the stronger the pushback.
If the model has already said "Kathmandu" five times, a positive frequency penalty nudges it toward "the city" or "the capital" instead of a sixth "Kathmandu."
Presence penalty: stop circling the same topic
Presence penalty is the binary version: it only cares whether a token has appeared at all, not how often. Once a word shows up, it gets penalized — which pushes the model toward new vocabulary and new topics.
The vibe: "You already mentioned Pokhara. Now talk about something else too."
Higher presence penalty → more topic diversity and exploration.
Zero → the model may stay locked on one topic (which is often exactly what you want).
The debugging cheat sheet
| Symptom | Fix |
|---|---|
| Answer cut off mid-sentence | Increase max tokens |
| Rambles into extra sections | Add a stop sequence |
| Repeats the same phrases | Small frequency penalty |
| Stuck circling one topic | Small presence penalty |
| Output too random / wrong | Lower temperature |
| Output too robotic / samey | Raise temperature (or Top-P) |
Part 5: Sensible Defaults to Start With
If you're wiring up your first AI feature and just want reasonable settings:
Structured, reliable output (JSON extraction, classification, code):
js
{
temperature: 0.2,
max_tokens: 1024,
// top_p / top_k: leave at defaults
}
Conversational assistant (chatbots, support, general Q&A):
js
{
temperature: 0.7,
top_p: 0.9,
max_tokens: 2048,
}
Creative generation (copywriting, brainstorming, stories):
js
{
temperature: 0.9,
top_p: 0.9,
presence_penalty: 0.3, // encourage fresh angles
}
Start there, change one knob at a time, and compare outputs. Tuning three parameters simultaneously is how you end up with settings you can't explain in code review.
Wrapping Up
Here's the whole post in one paragraph: an LLM is a next-token predictor whose knowledge lives in its weights, whose short-term memory is the context window, and whose behavior you steer with prompts. RAG hands it fresh reference material, agents give it tools and a loop, and hallucination and prompt injection are the two failure modes to design against. At generation time, temperature, Top-K, and Top-P control how adventurous each token pick is, while max tokens, stop sequences, and the repetition penalties control where the output ends and whether it repeats itself.
None of this is magic. It's a probability distribution and a handful of dials — and now you know what each one does.
If you're learning this stack yourself, here's the order I'd suggest: tokens and context windows first, then prompting, then the failure modes (hallucination, injection), then RAG and agents, and fine-tuning last. Each layer builds on the one before it.
