LangChain in Chains #23: ReACT
Reasoning And Acting
One of the key papers on directing a large language model (LLM) to achieve desired outcomes is titled “ReAct: Synergizing Reasoning and Acting in Language.”
It is a concept that aims to enhance language models by integrating reasoning and action capabilities. The idea is to create models that can not only understand and generate natural language but also perform reasoning and execute actions based on that understanding.
Previous Chapter:
The core of REACT is the seamless integration of reasoning and acting.
Reasoning Only
Reasoning refers to the model’s ability to process and analyze information, make decisions, solve problems, and infer new information from existing knowledge.
“Reasoning Only” means that the model relies on its internal representation to generate thoughts, without incorporating information from the external world.
Once a model is trained, the parameters are fixed, and its internal representations do not change in response to new data. This makes their reasoning process static. This makes them static black boxes.
Models in “Reasoning Only” systems lack the ability to interact with the external world. This limits their understanding to the scope of their training data, and they cannot incorporate new information that they have not explicitly trained on.
If an early step in the reasoning chain is incorrect, it can lead the entire process astray and produce hallucinations because the system cannot use referenced facts from the external world.
Act Only
Acting refers to the capacity of a system to execute actions that impact its environment.
“Act Only” systems focus on taking actions based on predefined rules or directives, without incorporating a significant reasoning process.
In the example depicted above:
- Reasoning Only (CoT): This approach takes a step-by-step reasoning process, concluding that Apple TV, is controlled by iPhone, iPad, and iPod Touch. The answer includes multiple devices and is more comprehensive but is incorrect.
- Act Only: This part details a sequence of actions without explicit reasoning for each step. The actions involve searching for information about the Apple Remote and related software or services (like Front Row), leading to the conclusion that no more steps are required (Finish[yes]), leading to an incorrect answer.
ReACT
ReACT essentially employs both of these techniques.
ReACT = Reason + Act
In this cycle, the system has reasoning upfront, takes actions based on reasoning, receives feedback, and refines the reasoning.
- The system starts by engaging in a reasoning process. It understands the task at hand, and makes decisions on how to proceed.
- Based on reasoning, the system then takes a specific action. This action could involve using a tool or executing a command that affects the system’s environment. For example, querying a database, interacting with a user, or manipulating some digital content.
- After taking action, the system gathers feedback from the environment.
- The feedback is fed back into the system, where it is used to refine and improve the initial reasoning.
Imagine you are using a smart assistant (like Siri or Alexa) integrated with this kind of system. First, the assistant would think about what you asked for, such as setting up a meeting. It would then take action by accessing your calender to add the event. After the action, it might observe that the time you asked for is already booked. With this new information, it can refine its reasoning by suggesting an alternative time.
In ReACT’s solution to the aforementioned question, it begins with a thought through multiple-step reasoning. It then takes an action and gathers observations. Using this new feedback or information, it forms another logical step. After taking another action and observing new feedback, it generates another thought. Ultimately, it successfully accesses the correct answer.
ReACT in LangChain
import os
os.environ["OPENAI_API_KEY"] = "your-key"
from langchain import OpenAI, Wikipedia
from langchain.agents import initialize_agent, Tool
from langchain.agents import AgentType
from langchain.agents.react.base import DocstoreExplorerdocstore=DocstoreExplorer(Wikipedia())
tools = [
Tool(
name="Search",
func=docstore.search,
description="useful for when you need to ask with search"
),
Tool(
name="Lookup",
func=docstore.lookup,
description="useful for when you need to ask with lookup"
)
]
llm = OpenAI()
react = initialize_agent(tools, llm, agent=AgentType.REACT_DOCSTORE,
verbose=True, handle_parsing_errors=True)
react
"""
AgentExecutor(verbose=True, tags=['react-docstore'], agent=ReActDocstoreAgent(llm_chain=LLMChain(prompt=PromptTemplate(input_variables=['agent_scratchpad', 'input'], template='\n\nQuestion: What is the elevation range for the area that the eastern sector of the Colorado orogeny extends into?\nThought: I need to search Colorado orogeny, find the area that the eastern sector of the Colorado orogeny extends into, then find the elevation range of the area.\nAction: Search[Colorado orogeny]\nObservation: The Colorado orogeny was an episode of mountain building (an orogeny) in Colorado and surrounding areas.\nThought: It does not mention the eastern sector. So I need to look up eastern sector.\nAction: Lookup[eastern sector]\nObservation: (Result 1 / 1) The eastern sector extends into the High Plains and is called the Central Plains orogeny.\nThought: The eastern sector of Colorado orogeny extends into the High Plains. So I need to search High Plains and find its elevation range.\nAction: Search[High Plains]\nObservation: High Plains refers to one of two distinct land regions\nThought: I need to instead search High Plains (United States).\nAction: Search[High Plains (United States)]\nObservation: The High Plains are a subregion of the Great Plains. From east to west, the High Plains rise in elevation from around 1,800 to 7,00
"""
print(react.agent.llm_chain.prompt.template)
"""
Question: What is the elevation range for the area that the eastern sector of the Colorado orogeny extends into?
Thought: I need to search Colorado orogeny, find the area that the eastern sector of the Colorado orogeny extends into, then find the elevation range of the area.
Action: Search[Colorado orogeny]
Observation: The Colorado orogeny was an episode of mountain building (an orogeny) in Colorado and surrounding areas.
Thought: It does not mention the eastern sector. So I need to look up eastern sector.
Action: Lookup[eastern sector]
Observation: (Result 1 / 1) The eastern sector extends into the High Plains and is called the Central Plains orogeny.
Thought: The eastern sector of Colorado orogeny extends into the High Plains. So I need to search High Plains and find its elevation range.
Action: Search[High Plains]
Observation: High Plains refers to one of two distinct land regions
Thought: I need to instead search High Plains (United States).
Action: Search[High Plains (United States)]
Observation: The High Plains are a subregion of the Great Plains. From east to west, the High Plains rise in elevation from around 1,800 to 7,000 ft (550 to 2,130 m).[3]
Thought: High Plains rise in elevation from around 1,800 to 7,000 ft, so the answer is 1,800 to 7,000 ft.
Action: Finish[1,800 to 7,000 ft]
Question: Musician and satirist Allie Goertz wrote a song about the "The Simpsons" character Milhouse, who Matt Groening named after who?
Thought: The question simplifies to "The Simpsons" character Milhouse is named after who. I only need to search Milhouse and find who it is named after.
Action: Search[Milhouse]
Observation: Milhouse Mussolini Van Houten is a recurring character in the Fox animated television series The Simpsons voiced by Pamela Hayden and created by Matt Groening.
Thought: The paragraph does not tell who Milhouse is named after, maybe I can look up "named after".
Action: Lookup[named after]
Observation: (Result 1 / 1) Milhouse was named after U.S. president Richard Nixon, whose middle name was Milhous.
...
Question: {input}
{agent_scratchpad}
"""
DocstoreExplorer uses Wikipedia as the document source. This setup enables the agent to perform document-based searches and lookups in Wikipedia.
tools is a list of Tool objects, each configured to perform specific actions:
Search: Uses thesearchmethod of thedocstoreto perform search queries.Lookup: Uses thelookupmethod of thedocstorefor direct data lookup.
initialize_agent sets up the agent with specified tools, language model, and configurations. agent=AgentType.REACT_DOCSTORE specifies that the agent type is REACT_DOCSTORE, indicating that it uses document store explorations within the REACT framework.
question = "How old is the CEO of Apple?"
react.run(question)
"""
> Entering new AgentExecutor chain...
Thought: I need to search for Apple's CEO and find their age.
Action: Search[Apple CEO]
Observation: Could not find [Apple CEO]. Similar: ['Apple Inc.', 'Tim Cook', 'IPhone', 'IPhone 11', 'Steve Jobs', 'ITunes', 'Apple car project', 'Apple TV', 'IPhone 7', 'History of Apple Inc.']
Thought: Since Tim Cook is the current CEO of Apple, I need to search for his age.
Action: Search[Tim Cook age]
Observation: Timothy Donald Cook (born November 1, 1960) is an American business executive who is the current chief executive officer of Apple Inc. Cook had previously been the company's chief operating officer under its co-founder Steve Jobs. Cook joined Apple in March 1998 as a senior vice president for worldwide operations, and then as executive vice president for worldwide sales and operations. He was appointed chief executive on August 24, 2011 after Jobs, who was ill and died that October, resigned. During his tenure as the chief executive, he has advocated for the political reform of international and domestic surveillance, cybersecurity, American manufacturing, and environmental preservation.
Since 2011 when he took over Apple, to 2020, Cook doubled the company's revenue and profit, and the company's market value increased from $348 billion to $1.9 trillion. Cook is also on the boards of directors of Nike, Inc. and the National Football Foundation; he is a trustee of Duke University, his alma mater. Outside of Apple, Cook engages in philanthropy; in March 2015 he said he planned to donate his fortune to charity. In 2014, Cook became the first chief executive of a Fortune 500 company to publicly come out as gay.
Thought: Tim Cook was born on November 1, 1960, so he is currently 60 years old.
Action: Finish[60 years old]
> Finished chain.
"""
As you can notice, ReACT consume a lot of tokens primarily due to the complexity and continuous nature of their operations.
- High token consumption translates into higher computational demands. This can increase the operational costs, especially for systems requiring real-time or near-real-time processing capabilities.
- As more tokens need to be processed, the response time can increase, which might impact user experience in time-sensitive applications.
- Managing and scaling a high-token-consuming system can be challenging, especially when dealing with a large number of simultaneous interactions or when operating within limited resource environments.
Next:
Read More
Sources
https://research.google/blog/react-synergizing-reasoning-and-acting-in-language-models/
https://python.langchain.com/docs/modules/agents/agent_types/react/
https://www.youtube.com/watch?v=Eug2clsLtFs
https://arxiv.org/pdf/2210.03629
In Plain English 🚀
Thank you for being a part of the In Plain English community! Before you go:
- Be sure to clap and follow the writer ️👏️️
- Follow us: X | LinkedIn | YouTube | Discord | Newsletter
- Visit our other platforms: Stackademic | CoFeed | Venture | Cubed
- More content at PlainEnglish.io
