Advanced Prompt Engineering: Chain-of-Thought, ReAct, and Tree-of-Thoughts

Introduction

Prompt engineering has evolved beyond simple instruction-giving. As large language models (LLMs) become more powerful, advanced techniques like Chain-of-Thought (CoT), ReAct, and Tree-of-Thoughts (ToT) have emerged to unlock deeper reasoning, factual accuracy, and multi-step problem-solving. In this post, we'll dive into each method, with practical examples and code snippets to help you apply them immediately.

Chain-of-Thought (CoT) Prompting

Chain-of-Thought prompting encourages the model to produce intermediate reasoning steps before arriving at a final answer. This mimics human thought processes and significantly improves performance on arithmetic, commonsense, and symbolic reasoning tasks.

How It Works

Instead of asking for a direct answer, you provide a few examples that include step-by-step reasoning. The model learns to generate similar reasoning chains.

Practical Example

Problem: "If a ball costs $10 and a bat costs $20 more than the ball, what is the total cost?"

Prompt:

Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 balls. How many tennis balls does he have now?
A: Roger starts with 5 balls. 2 cans × 3 balls each = 6 balls. 5 + 6 = 11. The answer is 11.

Q: A ball costs $10. A bat costs $20 more than the ball. What is the total cost?
A:

Model Output:

The ball costs $10. The bat costs $20 more than the ball, so the bat costs $10 + $20 = $30. Total cost = $10 + $30 = $40. The answer is $40.

Benefits & Limitations

CoT is simple to implement and works well with few-shot examples. However, it can be brittle for unfamiliar tasks and may generate plausible but incorrect reasoning.

ReAct: Synergizing Reasoning and Acting

ReAct (Reasoning + Acting) combines reasoning traces with action steps (e.g., calling external tools or APIs). This allows the model to gather information, verify facts, and update its reasoning dynamically.

How It Works

The model outputs interleaved "Thought" and "Act" lines. A thought contains reasoning, and an act specifies an action (like a search query or calculator call). The environment returns observations that feed into the next thought.

Practical Example

Task: "What is the current population of Tokyo?"

Prompt:

You are a helpful assistant with access to a search tool. Answer the following question by reasoning step by step and using the search tool when needed.

Question: What is the current population of Tokyo?
Thought 1: I need to find the latest population estimate for Tokyo. Let me search for it.
Act 1: search("Tokyo population 2024")
Observation 1: "Tokyo population 2024 estimate: 14,000,000"
Thought 2: The search returned 14,000,000. That seems plausible. I'll answer.
Answer: The current population of Tokyo is approximately 14 million.

To implement ReAct, you need to define a loop that parses the model's output, executes actions, and feeds observations back. Here's a simplified Python pseudocode:

import openai

def react_loop(prompt, max_steps=5):
    messages = [{"role": "user", "content": prompt}]
    for step in range(max_steps):
        response = openai.ChatCompletion.create(model="gpt-4", messages=messages)
        content = response.choices[0].message.content
        messages.append({"role": "assistant", "content": content})
        if "Answer:" in content:
            return content.split("Answer:")[-1].strip()
        if "Act:" in content:
            # Parse action and execute (e.g., call search API)
            observation = execute_action(content)
            messages.append({"role": "user", "content": f"Observation: {observation}"})
    return "Max steps reached"

Benefits & Limitations

ReAct enables real-time fact-checking and tool use, making it ideal for dynamic queries. The main challenge is orchestrating the action-execution loop and handling errors.

Tree-of-Thoughts (ToT)

Tree-of-Thoughts extends CoT by exploring multiple reasoning paths simultaneously. It uses a tree search (like BFS or DFS) where each node is a "thought" step, and the model evaluates branches to choose the most promising.

How It Works

1. Thought Decomposition: Break the problem into intermediate steps.
2. Thought Generation: For each step, generate several candidate next thoughts.
3. State Evaluation: Use the model to evaluate the promise of each branch (e.g., "sure/maybe/impossible").
4. Search: Use BFS or DFS to explore the tree, guided by evaluations.

Practical Example

Problem: "24 game: use 4, 7, 8, 8 to make 24 using +, -, *, /."

Prompt for evaluation:

Evaluate if the current expression can lead to 24 using the remaining numbers.
Expression: (4 + 7) = 11. Remaining numbers: 8, 8. Is it promising? [sure/maybe/impossible]

The model might output "maybe", and then another branch like "(8 - 7) = 1" might be "sure". The search algorithm picks the best path.

Implementation Sketch

import itertools

def tree_of_thoughts(problem, max_depth=3, beam_width=2):
    # BFS-like search
    initial_state = (problem, [])
    frontier = [initial_state]
    best_solutions = []
    depth = 0
    while frontier and depth < max_depth:
        new_frontier = []
        for state in frontier:
            current_problem, steps = state
            # Generate possible next steps
            candidates = generate_candidates(current_problem)
            # Evaluate each candidate with LLM
            evaluations = [evaluate_candidate(c) for c in candidates]
            # Keep top beam_width
            top_candidates = sorted(zip(candidates, evaluations), key=lambda x: x[1], reverse=True)[:beam_width]
            for cand, eval_score in top_candidates:
                new_problem = apply_step(current_problem, cand)
                if is_solved(new_problem):
                    best_solutions.append(steps + [cand])
                else:
                    new_frontier.append((new_problem, steps + [cand]))
        frontier = new_frontier
        depth += 1
    return best_solutions

Benefits & Limitations

ToT excels at tasks requiring strategic planning, like puzzles or creative writing. However, it is computationally expensive due to multiple LLM calls per step.

Comparison Table

Technique	Mechanism	Use Case	Cost
CoT	Linear reasoning chain	Arithmetic, logic	Low
ReAct	Interleaved reasoning & actions	Fact-checking, tool use	Medium
ToT	Tree search with evaluation	Puzzles, planning	High

Conclusion

Chain-of-Thought, ReAct, and Tree-of-Thoughts represent a spectrum of advanced prompting strategies. CoT is great for straightforward reasoning, ReAct adds dynamic information retrieval, and ToT enables deliberate exploration. By combining these techniques, you can build robust AI agents that reason, act, and plan effectively.

For further reading, check out the original Chain-of-Thought paper and the ReAct paper. Experiment with these methods in your projects to see dramatic improvements in LLM performance!