TECHA working developer's toolkit for Claude CodeTRENDSThe quiet convergence: when agent frameworks start looking alikeTECHRAG is dead, long live RAG — why retrieval isn't going anywhereINFOWhy Claude's extended thinking changes the economics of complex tasksTECHA field guide to 2026's LLM benchmarks (and the ones you should ignore)TRENDSFine-tuning vs. long context: a practical decision matrixTRENDSWhat Anthropic's memory API tells us about the next year of agentsTECHPrompt caching is the closest thing to free compute we haveTECHA working developer's toolkit for Claude CodeTRENDSThe quiet convergence: when agent frameworks start looking alikeTECHRAG is dead, long live RAG — why retrieval isn't going anywhereINFOWhy Claude's extended thinking changes the economics of complex tasksTECHA field guide to 2026's LLM benchmarks (and the ones you should ignore)TRENDSFine-tuning vs. long context: a practical decision matrixTRENDSWhat Anthropic's memory API tells us about the next year of agentsTECHPrompt caching is the closest thing to free compute we have
AI × Forward
Subscribe
Trends

Fine-tuning vs. long context: a practical decision matrix

A simple four-box framework for when to reach for fine-tuning and when to trust the prompt. With receipts from three teams who tried both.

Initial Editor·2026-04-01·3min read·205 words

I get this question weekly: should we fine-tune?

The answer is almost always "not yet." Here's the framework I've settled on.

The two axes

  • Domain distance: how far is your task from the model's training distribution?
  • Throughput: how many calls per day?

The four boxes

Low distance, low throughput → Prompt engineering. Don't fine-tune, don't even spend on RAG.

Low distance, high throughput → Prompt caching. You're paying for the same tokens over and over.

High distance, low throughput → Few-shot with retrieved examples. Fine-tuning rarely pays at low volume.

High distance, high throughput → Now you fine-tune. And only then.

The receipts

Three teams I've worked with ran this decision:

  • Legal summarization SaaS: high distance, high throughput → fine-tuned a 7B model, cut costs 60%, latency 4×.
  • Internal code-review bot: low distance, low throughput → stayed on frontier model with a well-engineered prompt. No fine-tune.
  • Customer support assistant: high distance, low throughput → retrieved few-shot examples from ticket history. Beat the fine-tuned baseline they tried six months earlier.

The rule

Fine-tune when you have both a real distribution gap and enough calls to amortize the training cost. Otherwise you're buying a pet, not a tool.

// more in trends

see all →