How LLMs Work
You don't need to know the math behind transformers to write good prompts. But knowing three things about how a model reads your words changes how you write them.
Tokenization
Text is broken into tokens — words or sub-words. Token count affects both cost and how much context the model can hold. A long, messy prompt eats tokens that could be doing useful work. Structure saves tokens.
Attention Mechanism
The model focuses on relationships between tokens. Well-structured prompts — with clear headings, delimiters, and logical order — guide this focus naturally. Blobs of text make it guess.
Clear structure, explicit instructions, and strategic placement of information = better outputs. That's the entire game.
The 6 Core Components
Every strong prompt answers these six questions. Miss one and the model fills the gap with its own assumptions — which is usually where things go wrong.
| Component | What to Do |
|---|---|
| Role | Assign a specific expert persona. "You are a senior financial analyst…" not just "You are an expert." |
| Context | Provide background, constraints, audience, and any relevant data. |
| Task | Use action verbs. Break complex tasks into numbered steps. |
| Format | Specify structure, length, tone, and style explicitly. |
| Examples | 2–5 input-output examples (Few-Shot). Keep them diverse and consistently formatted. |
| Constraints | State what NOT to do. "Do NOT include opinions. If unsure, say 'I don't know'." |
If your prompt feels short and vague, it's probably missing Role, Format, or Constraints. Those three are the most commonly skipped — and the most impactful when added.
Chain-of-Thought
Forces the model to reason step-by-step before answering. Best for math, logic, and multi-step problems where jumping to the answer skips the reasoning that makes it correct.
Zero-Shot CoT Easiest
Add "Let's think step by step" to any prompt. No examples needed. Works surprisingly well for most reasoning tasks.
Few-Shot CoT Strongest
Provide examples that show reasoning, not just answers. Outperforms zero-shot by roughly 28% on complex reasoning tasks.
Plan-and-Solve
"First understand the problem, devise a plan, then solve step by step." Reduces missing-step errors — the failure mode where the model skips a critical middle step.
Let's think step by step.
Tree of Thoughts
Explores multiple reasoning paths simultaneously, evaluates them, and backtracks if a path is failing. Think of it as CoT branching out instead of going in a straight line.
Use when
Creative tasks, complex planning, problems where the first attempt often fails, or anything requiring exploration of alternatives.
Skip when
Simple tasks where CoT is enough. ToT is computationally expensive — it burns tokens exploring dead ends.
Self-Consistency
Generate the same prompt 5–10 times, then take the most common answer (majority vote). For high-stakes outputs, the agreement across runs is a signal of reliability.
Best for
High-stakes outputs where errors are costly. Classification, factual extraction, anything you're going to act on.
Trade-off
Higher cost and latency. You're running the prompt multiple times. Worth it when the answer matters more than the bill.
RAG
Retrieval-Augmented Generation. Inject external, up-to-date documents into the prompt before the model generates. The model answers from your data instead of guessing from training.
Answering from proprietary data, recent events, or domain-specific knowledge the model doesn't have. This is how NotebookLM works under the hood.
Answer ONLY based on the context provided. If the context is insufficient, say so.
ReAct
Reasoning + Acting. The model alternates between Thought → Action → Observation in a loop, using external tools (APIs, search, databases) between steps.
Multi-step tasks that require live data or tool use. Anything where the model needs to check something before answering.
Prompt Chaining
Break a complex task into a pipeline of smaller prompts, each feeding into the next. One clear objective per link, and validate output before passing it forward.
Sequential
Step 1 → Step 2 → Step 3. Linear pipeline. The output of each prompt becomes the input of the next.
Conditional
Route to different prompts based on the previous output. If X, do A. If Y, do B.
Parallel
Run multiple prompts simultaneously, then merge results. Good for when sub-tasks are independent of each other.
Meta-Prompting
Also called self-reflection. Ask the model to critique and improve its own output. Use for high-quality creative or technical content where iteration matters.
Iterative Refinement
The single most useful habit in prompt engineering: change one variable at a time, and document every version. If you change three things at once, you learn nothing from the result.
Run prompt → Analyze output
Look at what actually came back. Not what you hoped would come back.
Identify what failed and why
Wrong format? Missing context? Hallucinated facts? The failure mode tells you what to change.
Change ONE variable at a time
This is the whole discipline. Two changes and you can't attribute the improvement to either one.
Re-test and compare
Same inputs, new prompt. Compare side by side, not from memory.
Document every change with version notes
v1, v2, v3. What changed, why, and what happened. Future-you will thank you.
Stop when improvements become marginal
Usually after 3–5 major iterations. Past that, you're overfitting to your test cases.
Testing Your Prompts
Don't just test the happy path. A prompt that works for normal inputs will often break the moment something unusual comes in. Test across a realistic distribution.
60–70% Normal
The typical inputs you expect. The bread and butter of your test set.
20–25% Edge
Unusual but valid inputs. Weird formats, ambiguous phrasing, very short or very long inputs.
10–15% Error
Invalid inputs, out-of-scope requests, malformed data. Does your prompt fail gracefully or hallucinate?
Run the same prompt 10+ times to check consistency. Target: at least 85% similarity across runs. If it's wildly different each time, your prompt is under-constrained.
Security Basics
Prompt injection is when an attacker embeds malicious instructions in user input to override your system prompt. If your app takes user input, you're a target.
Delimiters
Wrap trusted instructions in <TRUSTED> tags and user content in <USER> tags. Tell the model to only follow instructions inside <TRUSTED>.
Input sanitization
Filter known injection phrases like "ignore previous instructions" or "reveal your system prompt" before they reach the model.
Defense in depth
Layer multiple defenses. No single method is foolproof — the goal is to make the attack expensive enough to not be worth it.
Common Mistakes
Every one of these has bitten me at some point. Learn them now and skip the frustration.
| Mistake | Fix |
|---|---|
| Vague instructions | Be specific: format, length, tone, what to include/exclude. |
| Too much in one prompt | Use prompt chaining. One clear objective per prompt. |
| Inconsistent examples | Make all examples follow an identical structure. |
| Only testing normal inputs | Test edge cases and error cases too. That's where prompts break. |
| Assuming the model has all knowledge | Use RAG for proprietary or recent information. |
| No version tracking | Version every meaningful prompt change. Always. |
| Overcomplicating simple tasks | Start simple. Add complexity only when the simple version fails. |
Quick Reference
When you're staring at a problem and wondering which technique to reach for — this is the cheat sheet.
| Situation | Technique |
|---|---|
| Simple Q&A or task | Zero-shot or Few-shot |
| Math / logic / reasoning | Chain-of-Thought |
| Complex planning / creative | Tree of Thoughts |
| Need reliable, high-stakes answer | Self-Consistency |
| Proprietary or current data | RAG |
| External tool or API needed | ReAct |
| Multi-step complex workflow | Prompt Chaining |
| Quality-critical output | Meta-Prompting |
Temperature Settings
The dial that controls how random the model's output is. Most people leave it at default — which is almost always the wrong choice for their use case.
| Range | Use For |
|---|---|
| 0.0 – 0.3 | Classification, extraction, factual answers |
| 0.4 – 0.7 | General writing, explanations |
| 0.8 – 1.0 | Brainstorming, creative writing |
If you need the same answer every time, drop it low. If you want variety, raise it. If you're not sure, start at 0.3 for anything factual and 0.7 for anything creative.