Chain-of-Thought Prompting
Chain-of-Thought (CoT) prompting is a simple but powerful technique: instead of asking for a direct answer, you prompt the model to explain its reasoning step-by-step. Adding 'Let's think step by step' to prompts can improve accuracy on complex reasoning tasks by 50-80%. CoT works because it forces models to break down problems, catch errors in intermediate steps, and use working memory effectively. The technique is especially powerful for math, logic, common sense reasoning, and multi-step workflows. Advanced variants include Zero-Shot CoT (no examples needed), Few-Shot CoT (provide reasoning examples), and Self-Consistency (sample multiple reasoning paths and vote).
Overview
Chain-of-Thought prompting transforms how we interact with large language models for complex tasks. Standard prompting often fails on multi-step problems—the model jumps to an answer without showing work, making errors along the way. CoT prompting adds intermediate reasoning steps, dramatically improving accuracy. A model that gets 18% accuracy on math word problems can reach 90%+ with CoT prompting. The technique emerged from research showing that models trained on vast text corpora already have reasoning capabilities—they just need the right prompt format to access them.
Types of Chain-of-Thought Prompting
- **Zero-Shot CoT**: Simply add 'Let's think step by step' or 'Let's solve this carefully' to prompt—no examples needed
- **Few-Shot CoT**: Provide 2-8 examples with detailed reasoning chains in prompt
- **Self-Consistency CoT**: Generate multiple reasoning paths (temperature > 0), take majority vote on final answer
- **Least-to-Most Prompting**: Break complex problems into simpler subproblems, solve sequentially
- **Tree of Thoughts**: Explore multiple reasoning branches, backtrack and prune bad paths
Why It Works
CoT works because language models use next-token prediction. When generating an answer directly, the model must compress all reasoning into a single forward pass. With CoT, intermediate reasoning tokens provide 'working memory'—the model can reference its own reasoning in subsequent tokens. This is analogous to how humans solve complex problems: we don't calculate 347 × 28 instantly, we break it into steps and use intermediate results.
Business Integration
CoT prompting immediately improves accuracy on business-critical reasoning tasks with zero additional training. Financial analysts using AI for due diligence can prompt models to show reasoning chains when evaluating investment opportunities—catching logical errors before they impact decisions. Customer support bots using CoT provide transparent explanations for policy decisions, building trust. Legal document review systems using CoT can explain why a clause is problematic, making AI assistants truly helpful rather than black boxes.
Real-World Example: Financial Analysis
A private equity firm analyzes acquisition targets using AI. Standard prompting: 'Is this company a good investment?' produces vague answers. With CoT prompting: 'Analyze this company's financial health step-by-step: 1) Revenue trends, 2) Profit margins, 3) Debt levels, 4) Market position, 5) Risk factors, then provide final recommendation.' The model now catches red flags (e.g., declining margins masked by revenue growth) and provides auditable reasoning. Investment quality improves 23%.
Implementation Example
Technical Specifications
- **Accuracy Improvement**: 20-80% on reasoning tasks depending on complexity
- **Token Cost**: Increases input/output tokens by 2-5× due to reasoning chains
- **Latency Impact**: Increases response time by 2-4× due to longer outputs
- **Model Requirements**: Works best with models >10B parameters (GPT-4, Claude, PaLM, Gemini)
- **Optimal Temperature**: 0.0-0.3 for single-path CoT, 0.7-1.0 for self-consistency
- **Examples Needed (Few-Shot)**: 2-8 examples with reasoning chains, diminishing returns after 8
Best Practices
- Start with Zero-Shot CoT ('Let's think step by step')—simplest and often sufficient
- Use Few-Shot CoT when zero-shot fails—provide 3-5 diverse reasoning examples
- Apply Self-Consistency for high-stakes decisions—accuracy improvement worth the cost
- Combine with output format constraints (JSON, numbered steps) for structured reasoning
- Monitor reasoning quality—if model takes wrong path early, final answer will be wrong
- Use CoT for complex problems (multi-step math, logic, planning)—overkill for simple lookups
When NOT to Use CoT
- Simple factual lookups ('What is the capital of France?')—direct answer is faster and cheaper
- Creative writing—reasoning chains constrain creativity
- Very small models (<1B parameters)—lack reasoning capability regardless of prompting
- Latency-critical applications—CoT increases response time significantly