LLM Prompter: ReAct

Rodrigo Baron,Tue Jan 16 2024•prompt-engineer llm openchat langchain

This is the second article of the "LLM Prompter" series! In our last article, we covered the fundamentals of prompt engineering. Today, we'll dive into Chain of Thoughts (CoT) and ReAct (Reasoning + Acting) prompts and explore how they can help us tackle more complicated tasks requiring reasoning and information extraction.

For this discussion, we'll continue working with OpenChat 3.5 (opens in a new tab) and LangChain (opens in a new tab), but please note that these posts are not meant to serve as tutorials. Instead, use them to help understand the core concepts.

Let's begin! 🚀

Chain of Thoughts Prompting

If you've been following from the previous article, you might have encountered "Let's think step by step" – a popular prompt technique used to guide language models through problems by breaking their responses into smaller, manageable steps.

Here's an example using OpenChat 3.5 (opens in a new tab):

**prompt:** 8 + 2 / 2 * 2?  
**output:** To solve this expression, we need to follow the order of operations (PEMDAS/BODMAS).
1. Parentheses/Brackets
2. Exponents/Orders
3. Multiplication and Division (from left to right)
4. Addition and Subtraction (from left to right)
 
The expression does not have any parentheses or exponents, so we move on to multiplication and division.
8 + 2 / 2 * 2
First, we perform the multiplication:
8 + 2 / 4
Next, we perform the division:
8 + 0.5
Finally, we perform the addition:
8 + 0.5 = 8.5
So, the result of the expression is 8.5.

While the model has broken down the problem, its decision-making could benefit from further guidance. To address this, consider asking the model to think about each step before moving forward:

**prompt:** how much is 8 + 2 / 2 * 2? Think each step before solving
**output:** To solve this expression, we need to follow the order of operations (PEMDAS/BODMAS). This means we need to perform the operations inside the parentheses first, then exponents, then multiplication and division from left to right, and finally addition and subtraction from left to right.
The given expression is: 8 + 2 / 2 * 2
Step 1: Perform multiplication and division from left to right.
8 + (2 / 2) * 2
8 + 1 * 2
Step 2: Perform the remaining operations from left to right.
8 + 2
The result is 10.

By doing so, we encourage the model to carefully evaluate each stage, improving results. The key concept behind CoT prompting is applying consistent templates across multiple steps chain of iterations. This allows users to monitor progress and extract valuable insights throughout the process.

Here's a typical CoT prompt structure, however we can use any format:

**prompt: You are an math expert which solve questions best as you can.**
 
Use the following format:
 
Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take
Action Input: the input to the action
OBSERVATION: the result of the action
... (this Thought/Action/Action Input/OBSERVATION can be repeated zero or more times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question.
 
Question: how much is 8 + 2 / 2 * 2?**
**output**: Thought: I need to follow the order of operations (PEMDAS/BODMAS)
Action: Perform the multiplication and division operations first, then addition
Action Input: 2 / 2 * 2
OBSERVATION: The result of 2 / 2 is 1, and 1 * 2 is 2
Thought: Now I need to perform the addition operation
Action Input: 8 + 2
OBSERVATION: The result of 8 + 2 is 10
Thought: I now know the final answer
Final Answer: 10

Some models respond particularly well to common CoT templates, thas is because they exist in the training samples. If possible review the training data and use the template accordingly. Now, since large language models sometimes struggle with math, why not enhance CoT prompting with embedded functionalities?

ReAct

ReAct (Reasoning + Acting) merges reasoning (CoT) and action capabilities to simplify intricate tasks and enable controlled action trajectories via reasoning traces. Implementing ReAct helps reduce errors and hallucination during CoT's intermediate steps by offering extra resources like external knowledge or specialized tools designed for particular tasks.

LLM Training

Consider this basic LangChain example:

from langchain import LLMMathChain
from langchain.agents import initialize_agent, Tool
from langchain.agents import AgentType
from langchain.llms import HuggingFacePipeline
 
llm = HuggingFacePipeline(pipeline=generate_text)
 
llm_math_chain = LLMMathChain.from_llm(llm=llm, verbose=True)
tools = [
    Tool(
        name="Calculator",
        func=llm_math_chain.run,
        description="useful for when you need to calculate numbers."
    )
]
agent = initialize_agent(tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True)

To build upon our earlier CoT example, let's incorporate a ZERO_SHOT_REACT_DESCRIPTION agent along with llm_math_chain tool. Our updated prompt template now looks like this:

print(agent.agent.llm_chain.prompt.template)
...
Answer the following questions as best you can. You have access to the following tools:
 
Calculator: useful for when you need to calculate numbers.
 
Use the following format:
 
Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [Calculator]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question
 
Begin!
 
Question: {input}
Thought:{agent_scratchpad}

As we can see now the LLM have access to Calculato tool, if run this:

agent.run('8 + 2 / 2 * 2?')
...
Entering new AgentExecutor chain...
First, we need to follow the order of operations, which is Parentheses, Exponents, Multiplication and Division (from left to right), Addition and Subtraction (from left to right).
Action: Calculator
Action Input: 8 + 2 / 2 * 2
 
> Entering new LLMMathChain chain...
8 + 2 / 2 * 2
...numexpr.evaluate("8 + 2 / 2 * 2")...
 
Answer: 10.0
> Finished chain.
 
Observation: Answer: 10.0
Thought: The answer is 10.0
Final Answer: 10.0

Our straightforward example demonstrates the potential of an ReAct agent, we can leverage language models as powerful reasoning engines and grant them access to tools. We can make agents interact with databases, documents, and other systems to quey relevant information. That we call RAG (Retrieval Augmented Generation) agent, which we'll discuss in the next article.

Ending

In this article we dive into prompts which allow the model plan the steps to solve complex tasks. At same time we have control of the steps during the iteration. I hope the article was helpful you can alse reproduce the results running this Notebook (opens in a new tab). See you in RAG article..

LLM Prompter: ReAct

Chain of Thoughts Prompting

ReAct

Ending

References