Skip to content
AI Coding GuildAI Coding Guild
Sign InGet Started Free

Chain-of-Thought — Making AI Think Step by Step

Use chain-of-thought prompting to get AI to reason through complex coding problems.

10 min readai-tools, prompt-engineering, chain-of-thought, reasoning

When you ask AI to solve a complex problem in one step, it often gets the answer wrong. When you ask it to think through the problem step by step, it often gets it right.

This isn't just an observation — it's a well-documented technique called chain-of-thought prompting. It works because complex problems require intermediate reasoning that the AI skips when you ask for a direct answer.

For vibe coders, this means the difference between AI that produces working solutions to hard problems and AI that produces plausible-looking code that doesn't actually work.

The Problem With Direct Prompts

Here's a prompt that often produces bad code:

Write a function that takes a list of financial transactions and
calculates the running balance, accounting for pending transactions
that haven't cleared yet, recurring transactions that repeat monthly,
and refunds that should be applied to the original transaction date.

This is a complex problem with multiple interacting rules. When you ask for the solution directly, the AI tries to write everything at once. It might handle two of the three rules correctly but get the third wrong, or it might produce code where the rules interact in unexpected ways.

What Chain-of-Thought Looks Like

The same problem, with chain-of-thought prompting:

I need a function that calculates running balances for financial transactions.
Before writing any code, think through this step by step:
 
1. What data structure should each transaction have?
2. How should pending transactions be handled differently from cleared ones?
3. How should recurring transactions be expanded into individual entries?
4. How should refunds modify the balance at the original transaction date?
5. What's the algorithm for calculating the running balance once all
   transactions are normalized?
 
Walk me through your reasoning for each step, then write the code.

By asking the AI to reason through each step before coding, you get:

  • A clear explanation of its approach (which you can correct before it writes code)
  • Better handling of edge cases (because it thought about them explicitly)
  • Code that's more likely to work correctly on the first try

How to Trigger Chain-of-Thought

There are several phrases that reliably activate step-by-step reasoning:

Explicit Step-by-Step

Think through this step by step before writing code.

Planning First

Before implementing anything:
1. Explain your approach
2. List the edge cases you need to handle
3. Describe the data flow
Then write the implementation.

Reasoning Out Loud

Walk me through your reasoning, then write the solution.

Problem Decomposition

Break this problem into smaller sub-problems. Solve each one,
then combine them into the final solution.

All of these achieve the same thing: they force the AI to show its work, which improves the quality of that work.

When Chain-of-Thought Helps Most

Complex Business Logic

Think step by step: how should we calculate shipping costs?
Rules:
- Free shipping over $50
- Standard rate: $5.99 for items under 2 lbs, $9.99 for 2-5 lbs,
  $14.99 for 5+ lbs
- Express doubles the rate
- Alaska and Hawaii add $10 flat fee
- Multiple items: ship each item separately, but cap total at $29.99

Without chain-of-thought, the AI might miss the interaction between the weight-based pricing, the multi-item cap, and the free shipping threshold. With it, the AI reasons through each rule and how they interact.

Debugging

This function is producing wrong results. Before suggesting a fix,
analyze it step by step:
 
1. What is this function supposed to do?
2. Trace through it with this input: [example input]
3. At what point does the actual behavior diverge from expected?
4. What's causing the divergence?
5. What's the minimal fix?
 
[paste the buggy code]
 
Expected output: [what you expected]
Actual output: [what you got]

Architecture Decisions

I need to decide how to structure the data layer for my app.
Before recommending an approach, think through:
 
1. What data entities do we have and how do they relate?
2. What are the most common queries we'll need to run?
3. What are the performance implications of different approaches?
4. What's the simplest solution that handles our requirements?
 
Here are the requirements: [describe them]

Algorithm Implementation

I need to implement a scheduling algorithm that assigns tasks to
team members based on availability and skill match.
 
Before writing code:
1. Define the inputs and outputs clearly
2. Describe the algorithm in plain English
3. Walk through an example with 3 team members and 5 tasks
4. Identify the edge cases (no match, all busy, etc.)
5. Then implement it

A Practical Example

Let's say you're building a feature that lets users filter products by multiple criteria. Here's the chain-of-thought approach:

I need to build a multi-filter system for a product catalog.
Filters: category, price range, rating, in-stock status, brand.
 
Before writing the filter logic, think through:
 
1. Should filters combine with AND or OR logic? (I want AND — narrowing)
2. How should the URL reflect active filters? (I want query params so users
   can share filtered views)
3. What happens when a filter produces zero results?
4. How should the filter counts work? (Show how many products match each
   option, considering other active filters)
5. What's the most performant way to do this client-side with ~500 products?
 
Explain your approach for each point, then implement the filter system.

The AI's response will walk through each consideration, and you can correct its approach before it writes a single line of code. Maybe you disagree with point 4 — you'd rather show total counts regardless of active filters. You adjust that one point, and the AI writes the implementation with your preference baked in.

The "Think, Plan, Code" Framework

Here's a reliable three-phase prompt structure:

Phase 1: Think

I want to build [feature]. Before writing any code, analyze the
requirements and identify:
- What data structures are needed
- What the main functions/components should be
- What edge cases exist
- What could go wrong

Phase 2: Plan

Based on your analysis, propose an implementation plan:
- What files to create or modify
- What the function signatures should look like
- What the data flow looks like
- Which parts should be built first

Phase 3: Code

Now implement the plan. Start with [the first piece].

You can do all three phases in one prompt, or you can break them into separate messages. Separate messages give you a chance to adjust the plan between phases.

When NOT to Use Chain-of-Thought

Chain-of-thought adds overhead. For simple tasks, it's overkill:

| Task | Chain-of-Thought? | |------|-------------------| | "Add a button that logs out the user" | No — straightforward | | "Change the background color to blue" | No — trivial | | "Fix this typo in the heading" | No — obvious | | "Build a permission system with roles" | Yes — complex logic | | "Implement search with fuzzy matching" | Yes — algorithmic | | "Debug why payments are double-charging" | Yes — needs analysis | | "Design the database schema for a new feature" | Yes — needs reasoning |

Use chain-of-thought when the problem requires reasoning about multiple interacting pieces. Skip it when the task is direct and unambiguous.

Combining With Other Techniques

Chain-of-thought + Decomposition: Use chain-of-thought to plan the decomposition. "Think through how to break this feature into implementation steps, then we'll build each one."

Chain-of-thought + Few-shot: "Here's how I solved a similar problem [example]. Think through how this approach applies to my new problem, then adapt it."

Chain-of-thought + Constraints: "Think through the approach, but keep these constraints in mind: no external dependencies, client-side only, must work offline."

Try this now

Take one risky task from your backlog and rewrite it into a three-phase request:

  1. Think
  2. Plan
  3. Code

Do not let the agent skip to phase three until you agree with phases one and two.

Prompt to give your agent

Use this for tasks with real logic, architecture, or debugging complexity: "I need help with [problem]. Before writing code, work in three phases.

Phase 1: Think

  • Explain the problem in plain English
  • Identify the main rules, dependencies, and edge cases
  • Point out anything ambiguous or risky

Phase 2: Plan

  • Propose the implementation approach
  • List files or systems likely to change
  • Explain how you will verify correctness

Phase 3: Code

  • Do not start this phase until I approve phases 1 and 2

Keep these constraints in mind: [constraints]. Tell me what I must test manually after implementation."

What you must review yourself

  • Whether the reasoning actually addresses the real business rules instead of paraphrasing them
  • Whether the plan changes the right files for the right reasons
  • Whether the agent identified edge cases that matter in production
  • Whether the final implementation still needs manual testing for bad inputs, race conditions, or partial failures

Common mistakes to avoid

  • Using chain-of-thought on trivial work. It slows you down when the task is obvious.
  • Approving the plan without reading it. The whole point is to catch bad direction before code is written.
  • Treating plausible reasoning as proof. A good explanation still needs testing.
  • Letting the agent collapse the phases. If analysis, planning, and coding happen in one blur, you lose the review checkpoint.

Key takeaways

  • Chain-of-thought is most useful when the task has interacting rules or real uncertainty
  • "Think, Plan, Code" gives you a natural approval gate before implementation
  • Reasoning improves results because it exposes assumptions and edge cases early
  • The technique is a review tool, not a substitute for testing

What's Next

Next up: The Anti-Patterns — Prompts That Produce Bad Code. You now know what good prompting looks like. The next lesson shows the failure patterns that quietly undo all of that discipline.