Weights vs context vs tools

A useful split for builders: the model has four places knowledge can come from. Mix them up and you get a confident wrong number from a blog that was in the pretraining data.

Where knowledge lives

Instruction tuning makes refusals and tool use more likely when the model was trained to do so. It does not install your wiki. Chat models follow instructions. They do not "know" private or fresh facts unless you provide them.

Base models continue text and may ignore "you are a helpful assistant." Instruction / chat models follow roles, tools and safety policies — almost all products use these. Reasoning / thinking models spend extra inference-time compute on hard problems such as math, coding and multi-step planning.

Walking through parental leave

"What is our parental leave?" against weights → a confident wrong number from some public blog in pretraining. Same question with the HR policy chunk in context → a correct citation. Same question with a get_policy(name) tool → correct and auditable.

weights only     → fluent, wrong, undated
RAG chunk        → fluent, cited, as fresh as the index
get_policy tool  → fluent, cited, as fresh as the API

The transformer did not become more honest. You changed where the bits came from. Hallucinations thrive when the model is forced to answer from weights alone about something that belongs in another row.

Two levers, not one

Prompting steers the residual stream with tokens you insert this request. Fine-tuning changes the weights. Use prompting, RAG and tools for facts that move. Use fine-tuning for style, format and tasks that stay stable. See fine-tuning vs prompting vs RAG.

Everything in context can affect the answer — including leaked PII in history and malicious instructions in a retrieved page. Architecture permits the collision. Guardrails are how you refuse it.

Common mistakes

• Asking the model for live account state without a tool. You will get a fluent guess.
• Fine-tuning on this week's refund policy. Next week's policy will still be wrong.
• Treating an instruction-tuned model as a truth database. Helpfulness is not grounding.

Check yourself

1. "What is our parental leave?" should use which row of the table? Context / RAG or a policy tool. The number is private and can change.
2. Why does instruction tuning not fix that question? It teaches the model to be helpful, not to store your HR wiki.

Key takeaways

• Weights hold general skill. Context holds the docs you retrieved. Tools hold live state.
• Hallucinations are the default when the model must recall instead of read.
• Change the source of the bits, not the hope that the model will "know."
• Prompting and fine-tuning are different levers. Facts that move belong in context or tools.

Further learning

RAG, tool use, fine-tuning. Back to the map: how transformers work.