Advanced Prompt Engineering: CoT, ReAct, and Tree-of-Thoughts

Introduction

Prompt engineering has evolved from simple instruction crafting to sophisticated reasoning frameworks. As large language models (LLMs) become more capable, techniques like chain-of-thought (CoT), ReAct, and tree-of-thoughts (ToT) unlock deeper reasoning, multi-step problem-solving, and task decomposition. In this post, we’ll dissect these methods, compare them, and provide practical code examples.

Chain-of-Thought (CoT)

CoT encourages an LLM to produce intermediate reasoning steps before arriving at a final answer. Instead of a direct answer, the model outputs a step-by-step explanation.

How It Works

Approach	Description
Zero-shot CoT	Append "Let's think step by step" to the prompt.
Few-shot CoT	Provide examples with reasoning steps, then ask the question.

Example: Zero-shot CoT

Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 balls. How many does he have now?
A: Let's think step by step.

typically produces:

Roger starts with 5 balls. He buys 2 cans, each with 3 balls, so 2*3=6. Total = 5+6 = 11. Answer: 11.

Code Example (OpenAI API)

import openai

response = openai.ChatCompletion.create(
    model="gpt-4",
    messages=[
        {"role": "user", "content": "Q: A bat and a ball cost $1.10 total. The bat costs $1.00 more than the ball. How much does the ball cost?\nA: Let's think step by step."}
    ],
    max_tokens=200
)
print(response.choices[0].message.content)

Output:

The bat costs $1.00 more than the ball, so if the ball costs x, the bat costs x+1.00. Total: x + (x+1.00) = 1.10 => 2x+1.00=1.10 => 2x=0.10 => x=0.05. Answer: $0.05.

ReAct

ReAct combines reasoning and acting: the model interleaves thoughts (reasoning) with actions (e.g., calling APIs, searching the web) and observes results to inform subsequent steps.

Structure

• Thought: Internal reasoning about next steps.
• Action: A tool call (e.g., search, calculator).
• Observation: The output from the tool.

Example: Multi-step Question

Question: What is the population of the capital of France?
Thought: I need to find the capital of France (Paris) and then its population.
Action: Search[capital of France]
Observation: Paris
Action: Search[population of Paris]
Observation: 2.15 million
Thought: The population is 2.15 million.
Answer: 2.15 million.

Code Example (Lite)

import requests

def search(query):
    return f"Result for {query}"  # placeholder

tools = {"search": search}

prompt = """Question: Who is the current CEO of OpenAI?
Thought: I need to find the current CEO.
Action: search[CEO of OpenAI]
Observation:"""

# In practice, generate with LLM and parse thought/action/observation.

For full implementations, see LangChain ReAct.

Tree-of-Thoughts (ToT)

ToT generalizes CoT by exploring multiple reasoning paths simultaneously as a tree. At each step, the model generates several “thoughts,” evaluates them (e.g., via self-consistency or a value function), and selects the most promising branch for expansion.

Algorithm

1. Generate candidate next thoughts.
2. Evaluate each candidate’s likelihood to lead to a correct answer.
3. Select the best thought(s) and repeat.

Example: Creative Writing

Prompt: Write a short story about a robot learning to paint.
Thought branches:
- Branch A: Robot discovers old paintbrush.
- Branch B: Robot reads a manual on painting.
- Branch C: Robot meets a painter.

Evaluation: Judge coherence, creativity, etc. Expand the best.

Code Snippet (Conceptual)

import itertools

def generate_thoughts(context):
    # Use LLM to produce N candidate thoughts
    return ["think A", "think B", "think C"]

def evaluate_thought(thought):
    # Use LLM to assign a score (1-10)
    return 7.5

best_thought = max(generate_thoughts("..."), key=evaluate_thought)

For deeper reading, see "Tree of Thoughts: Deliberate Problem Solving with Large Language Models".

Comparison

Technique	Strengths	Weaknesses
CoT	Simple, effective for arithmetic and common sense	Can propagate errors; single path
ReAct	Handles tools, dynamic information retrieval	Requires tool integration; more tokens
ToT	Explores multiple paths, better for complex planning	Computationally expensive; needs evaluation

Practical Tips

• Choose CoT for math/logic problems with clear steps.
• Choose ReAct when the LLM needs external data (e.g., web search, database queries).
• Choose ToT for open-ended tasks like writing a novel or solving puzzles.

Conclusion

Advanced prompt engineering goes beyond single-shot queries. By structuring reasoning into chains, agents, or trees, we extract more accurate and insightful responses from LLMs. Experiment with these techniques in your next project to see improved performance on complex tasks.

References

• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
• ReAct: Synergizing Reasoning and Acting in Language Models