Architecture of a Hybrid LLM-Agent System (ReAct + Orchestration)

System Architecture Overview

A hybrid LLM-agent system combines a powerful central language model with specialized tool agents to tackle complex tasks. The architecture follows the ReAct (Reason + Act) paradigm, meaning the LLM interleaves logical reasoning steps with concrete actions (tool calls) in an iterative loop (LLM Powered Autonomous Agents | Lil’Log). A central LLM Coordinator (e.g. GPT-4) serves as the “brain” (LLM Powered Autonomous Agents | Lil’Log), orchestrating various expert agents and deciding which tasks to delegate. The design is highly modular – new specialist agents (for web search, text analysis, planning, math, browsing, etc.) can be added or swapped out without changing the core logic. All reasoning steps and actions are made transparent and traceable in a structured format (e.g. clearly logging each thought, tool invocation, and result), ensuring the chain of thought is understandable. A long-term memory module stores facts and context for persistent knowledge, allowing the system to recall information across sessions. This memory, combined with the ReAct loop, enables the agent to update its plan dynamically whenever new facts emerge during execution. Finally, the system supports external tools and APIs (search engines, calculators, web browsers, etc.) as first-class components – the LLM can call these tools to gather information or perform computations beyond its built-in knowledge (LLM Powered Autonomous Agents | Lil’Log). The result is a flexible, transparent, and scalable agent architecture that can reason, act, learn, and adjust on the fly.

(LLM Powered Autonomous Agents | Lil’Log) Figure: High-level overview of the hybrid LLM-agent architecture. The central Agent (LLM coordinator, pink) uses a Planning module (for subtask decomposition, self-critique, reflection) and a Memory module (with short-term and long-term memory) to decide on actions. It can invoke various Tools/Agents (left) such as search, calculators, code interpreters, etc., to execute specific tasks beyond its internal knowledge. Solid arrows indicate data or control flow (e.g. agent issuing an action to a tool or querying memory), and dashed arrows indicate influence on planning (e.g. new information from memory or reflections feeding back into the agent’s reasoning).

Key Components of the System

The hybrid system is composed of several interacting components, each with a clear responsibility. The table below summarizes the core modules:

Component	Description
Central LLM Coordinator	A powerful language model (e.g. GPT-4) acting as the brain ([LLM Powered Autonomous Agents
Specialized Tool Agents	Modular expert modules for specific functions. Examples: a Web Search Agent (queries the internet or knowledge base), Text Analysis Agent (summarizes or extracts info from text), Planning Agent (for complex task decomposition or external planning algorithms), Math/Calculation Agent (performs arithmetic or calls a Python interpreter), Web Browser Agent (navigates web pages or APIs). Each agent has a well-defined interface (e.g. a `run()` method) and can be added or replaced independently.
Memory Module (Long-Term)	A knowledge base that stores facts, previous observations, and context beyond the current session. Often implemented as a database or vector store for semantic retrieval ([LLM Powered Autonomous Agents
Working Memory (Short-Term)	The transient context of the current conversation or task – essentially the LLM’s prompt context. This includes the running log of thoughts, actions, and observations (the ReAct chain) and any intermediate conclusions. It’s how the system “remembers” what it’s doing within a single task.
Reasoning & Planning Module	The internal logic by which the LLM Coordinator plans its next steps. In practice this is part of the LLM’s prompting (e.g. chain-of-thought reasoning or few-shot exemplars that encourage stepwise thinking). It may also involve an external planner agent for certain tasks. Planning includes breaking the user query into sub-tasks, deciding the order of execution, and refining the plan if needed. The LLM uses this to generate the “Thought” steps in ReAct, possibly informed by retrieved memory or by self-reflection feedback loops.
Tool Interface/Orchestrator	The mechanism that enables the LLM to invoke tools/agents. This could be a predefined API for actions or a function-calling interface. The coordinator sends a command (e.g. “Action: Search[query]”) which the orchestrator intercepts, matches to the correct agent (SearchAgent), executes it, and returns the result as an Observation back to the LLM. This interface abstracts the details of each tool so that the LLM just needs to specify the action in a standard format.
Logging & Transparency Layer	Although not a separate physical component, the system includes comprehensive logging of each reasoning step and action. Every “Thought”, “Action”, and “Observation” in the ReAct loop is recorded. This makes the process transparent for debugging and for users who want to trace the solution. The logs can be formatted or even presented in a user-readable way if needed, ensuring the logic is explainable.

Modularity and Scalability: The above components are loosely coupled. New specialized agents can be plugged into the Tool interface easily – for example, if you want to add a Database Query Agent or a Translation Agent, you implement it and register it with the coordinator. The central LLM can be updated or swapped (e.g. replace GPT-4 with a local LLM) as long as it follows the same prompting interface. This modular design makes the system scalable: you can distribute agents across servers, run multiple tool calls in parallel when tasks allow, and upgrade parts independently. The memory store can be scaled up (using a more powerful database or vector index) without affecting how the planner works, etc. The transparency/logging means as the system grows more complex, developers can still follow the reasoning flow.

Central LLM Coordinator (Core Reasoner)

The central coordinator is an LLM (Large Language Model) that drives the problem-solving process. It receives the user’s query or goal and is responsible for figuring out how to solve it by possibly breaking it into sub-tasks. Acting as a controller or manager, it uses its prompt-based intelligence to determine which specialized agent or tool is needed at each step (LLM Powered Autonomous Agents | Lil’Log). This is very similar to the role of ChatGPT in the HuggingGPT framework, where the LLM plans tasks and selects expert models to execute them (LLM Powered Autonomous Agents | Lil’Log). In our system, instead of calling HuggingFace models, the coordinator will call our modular agents in a similar fashion.

To perform its role, the coordinator uses a ReAct reasoning loop. Rather than one-shot answering, it iteratively alternates between reasoning (thought) and acting (tool calls). For example, the LLM might think: “I need more information on X” – that’s a reasoning step. It then produces an action: Search for X. The orchestrator executes the Search agent and returns results, which the LLM sees as an observation. Then it reasons again: “The search results suggest Y; next I should analyze Y or perhaps calculate Z”, and so on. This process continues until the LLM believes it has enough information to answer the user or complete the task. Finally, the coordinator produces a final answer or outcome (this could be a text answer to the user, a file generated, etc., depending on the task). Throughout this process the coordinator ensures each step’s outcome is checked against the goal, and if new information changes the approach, it can adjust the plan.

The LLM Coordinator is prompted in a way to encourage this behavior. A typical prompt template might include instructions and examples following the format:

Thought: [the LLM’s thought process reasoning here]  
Action: [the chosen action command]  
Observation: [the result of the action]

This ReAct format forces the LLM to explicitly output its thinking and intended action (LLM Powered Autonomous Agents | Lil’Log). The system captures these and executes as needed. Notably, the coordinator might also have a stop condition or a maximum number of thought-action iterations to avoid infinite loops. In practice, frameworks like LangChain implement this loop under the hood, but one can also code it directly using Python with API calls to the LLM.

Modular Specialized Agents (Tools)

Specialized agents are analogous to plugins or tools that the LLM can use to gather information or perform specific operations. Each agent is designed to be independent and focused on a certain domain of expertise or a type of action. This modular design means you can maintain or scale each tool separately and add new ones as needed without altering the others.

Some examples of specialized agents and their roles:

Web Search Agent: Takes a search query (text) and returns relevant results from the internet or a document corpus. Implementation could use an API (e.g. Google Custom Search or Bing API) or a local indexed knowledge base. The coordinator uses this when it needs up-to-date information or facts not in its training data.
Text Analysis Agent: Performs NLP tasks on text. For instance, it could summarize a document, extract entities or keywords, classify sentiment, or analyze the content in other ways. This agent might be powered by another smaller language model or a rule-based system or library. For example, it could use a HuggingFace transformer pipeline for summarization.
Math/Calculation Agent: Executes mathematical computations or code. This could be as simple as evaluating arithmetic or as powerful as running a Python interpreter for complex code (similar to OpenAI’s Code Interpreter). The LLM will delegate to this agent when exact calculation or code execution is required (to avoid the LLM’s known tendency to make arithmetic mistakes).
Planning Agent: Although the central LLM handles most planning, in some cases a dedicated planning module might be used. For example, for very complex multi-step problems or when integrating with formal planning frameworks (like PDDL planners or task schedulers), a Planning Agent can take a high-level goal and return a sequence of actions (a plan) which the LLM can then execute or verify. This agent could be a classical AI planner or even another LLM specialized via prompt for planning.
Web Browser Agent: Goes beyond simple API calls to actually browse web pages or interact with a web environment. It might fetch a page’s content given a URL, click links, or scrape information. This is useful for tasks like “find the latest news on X” where you might need to navigate a website. Implementation could use tools like Selenium or an API-based browser.
Other Tools: The architecture can support many others – e.g. a Database Query Agent (for retrieving structured data from a database), Image Analysis Agent (if the system handles images, it could call an image captioning or OCR model), etc. As long as the agent can be invoked via the Tool interface (taking some input and returning an output), the coordinator can incorporate it into the reasoning loop. HuggingGPT demonstrated this extensibility by allowing an LLM to choose from dozens of HuggingFace models as tools (LLM Powered Autonomous Agents | Lil’Log).

Each agent typically has a standard interface in code. For example, one can define a base class ToolAgent with a method execute(tool_input) that each agent implements. The Tool Interface/Orchestrator component will call agent.execute() and pass the result back. The agent should also return results in a consistent format (e.g. a text snippet or data structure) that the LLM can read as an observation. Keeping the interface consistent (like input/output schemas) is important for maintainability.

To add a new agent, you register it with the system (perhaps in a dictionary mapping tool names to agent instances). The LLM’s prompt can be updated to include a description of the new tool’s capabilities so it knows it’s available. This modularity allows the system to grow in functionality over time. For instance, if you later add a Translation Agent, you’d describe its use (e.g. “Tool Translate: translates text to a specified language”) in the prompt, and the LLM could then choose to invoke Action: Translate["Hello", "French"] when needed. This approach follows the principle of tools like Toolformer and HuggingGPT which extend an LLM by connecting it to many external models/APIs (AI agents: Capabilities, working, use cases, architecture, benefits …) (LLM Powered Autonomous Agents | Lil’Log).

Transparent Reasoning with ReAct

Transparency is a key design principle of this system. We want each step the AI takes to be traceable and logically justified. The ReAct framework naturally enforces a form of transparency by separating reasoning and acting steps in the LLM’s output (LLM Powered Autonomous Agents | Lil’Log). The system logs might look like:

Thought: I need to find current population of Paris.  
Action: Search["current population of Paris"]  
Observation: [Search results show Paris population ~2,161,000 (2023)...]  

Thought: Now I have the population. The question also asks for population of London, so I should find that too.  
Action: Search["current population of London"]  
Observation: [Search results show London population ~8,982,000 (2023)...]  

Thought: I will compare these numbers now.  
Action: Calculate["8982000 - 2161000"]  
Observation: [Result: 6,821,000]  

Thought: The difference in population is about 6.82 million. Now I can answer the question.  
Action: FinalAnswer["London has about 6.8 million more people than Paris, based on recent population figures."]  
Observation: [Final answer delivered to user]

In this illustrative log, every reasoning step (“Thought”) and tool use (“Action” with its result) is clearly shown. In an implementation, the “FinalAnswer” action denotes that the agent is done and the content in brackets is sent as the answer. By inspecting such a trace, a developer or user can follow why the agent took each step. If the agent makes a mistake or an incorrect assumption, it’s easy to pinpoint where it went wrong. This greatly aids debugging and building trust, as the process is not a black box – it’s closer to watching the agent think out loud.

To ensure transparency, the system should not suppress the chain-of-thought. The coordinator LLM is instructed to output the Thought/Action/Observation labels explicitly. (In a user-facing deployment, one might filter out or post-process the raw chain-of-thought so that the user only sees the final answer, but during development or in a “verbose mode”, this full trace would be available.) Each specialized agent can also be designed to provide explanatory output. For example, a math agent might not only return the number but also the formula or steps it took, if that’s useful for transparency. However, generally the coordinator’s reasoning is sufficient to understand the overall logic.

In summary, the ReAct approach intrinsically provides a transparent logical flow (LLM Powered Autonomous Agents | Lil’Log). The key is to capture and present these steps. In Python, this might mean accumulating a list of steps (as strings or a structured format) as the LLM produces them, and perhaps printing them to console or storing them in a log file. Each entry would include the type of step, its content, and timestamp or step number. When something goes wrong (e.g., the LLM tries an invalid action or loops), this log is the first place to check.

Long-Term Memory and Knowledge Base

A long-term memory module allows the system to accumulate knowledge over multiple queries or a prolonged session, addressing the requirement for remembering facts that influence planning and decisions. While the LLM has a context window for short-term memory (the prompt), anything outside of that window or beyond one conversation turn would normally be forgotten. The long-term memory component solves this by providing an external storage for facts, results, and other information that the agent can query as needed (LLM Powered Autonomous Agents | Lil’Log).

Implementation: This is often realized with a vector database or embedding store. For example, when the agent learns a new fact (say, the user at some point told the agent their birthday), the system can encode that fact into a vector and store it with a key or metadata. Later, if the agent needs relevant info, it can embed the current query or context and do a similarity search in the vector store to retrieve stored knowledge. Alternatively, memory can be stored as key-value pairs or documents that the LLM can search or be fed as context when relevant. The memory could also be a curated knowledge graph or any database that’s queryable via a tool agent (for instance, a SQL database of facts that the LLM can query through a SQL Agent).

The coordinator is responsible for using this memory when appropriate. One pattern is to have a pre-step before the main reasoning where the system searches memory for any entries related to the user’s request. If something is found, it can prepend a summary of those facts into the LLM’s prompt (so the LLM is aware of them during planning). Another pattern is on-demand memory access: at any point in the chain-of-thought, the LLM can decide to call a Recall action (a tool that queries the knowledge base). For example: “Thought: Perhaps I’ve encountered this user’s issue before; Action: Recall[“user_issue”]” which would search the memory. The memory agent would return any matches as an observation that the LLM can then integrate into its reasoning.

For long-term memory to influence planning (requirement 4), the agent’s prompt can include instructions like “Remember to use the knowledge base for any relevant facts.” The presence of relevant retrieved facts can change the LLM’s next thought. For instance, if the user asks a question and the memory already contains the answer from an earlier conversation, the LLM might skip a web search and use the stored answer directly. Or if the memory contains a context (like user preferences), the LLM will plan actions taking that into account (e.g. using a specific data source the user prefers).

Example: Suppose the user previously asked “What is the capital of X country?” and that was stored. Later the user asks a related question that requires that info. The memory could provide “Capital of X is Y (from previous interaction on 2025-03-01)”. The LLM then uses that without re-searching, making the system more efficient and personalized.

From an engineering perspective, implementing the knowledge base can be done using libraries like FAISS or ChromaDB for vector search, or even a simple Python dictionary for small-scale memory. The data stored could be text snippets or structured JSON. The agent should also have some strategy to prevent memory from growing indefinitely – perhaps summarizing old entries or expiring those that are no longer relevant, to maintain scalability.

Dynamic Plan Adaptation (Self-Reflection and Re-planning)

One of the powerful aspects of the ReAct loop and the modular design is the ability to adjust the plan on the fly. The system is not locked into a rigid sequence of steps; after each action, the coordinator LLM observes the result and can change course if needed. This fulfills requirement 5: the ability to rebuild or adjust the action plan based on new facts obtained during execution.

In practice, the LLM’s chain-of-thought may include conditional or corrective logic. For example, the LLM might initially reason “If the search results mention a date, I should use the calculator agent to compute the time difference”. If an observation comes in that triggers a new realization (say the search returned a surprising piece of info), the LLM can incorporate that. This is akin to a human solving a problem: you might have a plan, but if halfway through you discover something new, you alter your approach.

The system can further enhance this adaptability by explicitly incorporating self-reflection steps (LLM Powered Autonomous Agents | Lil’Log). After a series of Thought/Action iterations, the agent might pause and think at a higher level: “Am I approaching this the right way? Did I get the result I expected? If not, should I try a different strategy?”. This could be prompted by the orchestrator if it detects the agent is stuck in a loop or not making progress, or it can be a natural part of the prompt (some frameworks include a “reflect” tool the LLM can call). For instance, if the agent has tried two searches and both returned irrelevant info, the LLM could decide to reformulate the query or try a different agent (maybe a specialized database agent) – effectively re-planning its approach to the task. In extreme cases, it might even start over on the task with a new plan (like “Let me break down the problem differently”).

HuggingGPT’s multi-stage design demonstrates structured re-planning: the initial stage is parsing into tasks, but if something fails, the system could in principle revise that task list (LLM Powered Autonomous Agents | Lil’Log) (LLM Powered Autonomous Agents | Lil’Log). In our simpler single-coordinator loop, re-planning is more fluid – it happens whenever the LLM’s reasoning decides on a new direction. Because each Thought considers the latest observations, the plan is naturally responsive. There’s no separate static plan that could go stale; the plan is continuously updated in the LLM’s “mind” as it works through the problem.

To support this in implementation, one could include checkpoints or heuristics. For example, after each action’s result comes in, the system might append a special token asking the LLM “Do you need to adjust your plan? If so, think step-by-step how.” However, often the LLM will do this on its own if it’s been instructed to be reflective. It’s important that the prompt encourages the model to be flexible and critical of its progress. Including a few-shot example where the model encounters a dead-end and then backtracks can teach the LLM to do the same. Some research works like Self-Refine or Reflexion have explored this idea of letting the model critique and correct itself mid-task (LLM Powered Autonomous Agents | Lil’Log) (LLM Powered Autonomous Agents | Lil’Log).

From a software perspective, the architecture might incorporate a feedback loop: if a certain number of steps have passed with little progress, the orchestrator can trigger a reflection. The memory can also log failures or successful strategies, so next time a similar task is attempted, the agent might recall what worked or didn’t work previously. All these contribute to a more resilient, adaptive agent that doesn’t blindly follow an initial plan when circumstances change.

Tool Integration and External API Support

Supporting external tools (requirement 6) is at the heart of this system. The LLM on its own is powerful in reasoning but limited to its trained knowledge cutoff and unable to perform actions like actual web searches, API calls, or heavy computations. By integrating tools, we give the LLM actuators and sensors to interact with the world beyond its neural weights (LLM Powered Autonomous Agents | Lil’Log).

Mechanism: The integration is typically done through a function-calling API or an action dispatcher. There are two common approaches to implement this:

ReAct with textual prompts: The LLM is prompted to output a special format for actions. For example, if the LLM says Action: Search["query"], the orchestrator code parses this string and knows it should call the SearchAgent with the argument "query". Similarly Action: Calculate["expression"] routes to the Math agent. This approach relies on careful prompt design to ensure the LLM sticks to the format (so it’s machine-readable). It’s a straightforward approach and is model-agnostic (works with any LLM that can follow the prompt pattern). The downside is the parser might break if the LLM deviates or hallucinates an unknown tool name, but with few-shot examples and guardrails in the prompt this can be minimized.
Function Calling / API schema: Newer LLM interfaces (like OpenAI’s function calling or JSON-formatted outputs) allow the developer to define functions that the LLM can call. In this setup, you define each tool agent as a function with a name and parameters. During conversation, if the LLM decides that function is needed, it will output a JSON object like {"function": "search", "args": {"query": "..."}} (the exact format depends on the API). The orchestrator (client library) sees this and calls the actual Python function for search. The result is then fed back to the LLM. This approach has the benefit of being more structured — less chance for misunderstandings, since the LLM is effectively constrained to valid function calls. It’s very much aligned with the ReAct idea but more formalized (the LLM doesn’t literally print “Observation: …”; instead the system knows to take the function result as the observation to pass in the next prompt internally).

No matter which mechanism is used, the design of the Tool Interface is crucial. Each tool needs a clear contract: what inputs it expects and what output it returns. In Python, one might define a Tool dataclass or simple dict that includes the tool name, a reference to the function to execute, and optionally a description for the LLM. For example:

class Tool:
    def __init__(self, name, func, description):
        self.name = name
        self.func = func        # Python callable implementing the tool
        self.description = description  # For the LLM prompt
    
# Example tool registration
tools = []
tools.append(Tool("Search", web_search_function, "Use this to search the web for information."))
tools.append(Tool("Calculate", calculator_function, "Use this for math calculations or code execution."))
...

The coordinator would be initialized with this list of tools. In the LLM prompt, we might provide a concise list of available tool names and descriptions (so the model knows what it can do). During runtime, when the LLM outputs an action, we look up the corresponding Tool by name and call its func. The result (which could be any Python object, but we convert to a string or text summary) is then inserted back into the LLM’s context as the observation.

For external APIs (say a weather API, or a custom business database API), you would wrap the API call in a function and register it as a tool in the same way. The agent could then do Action: Weather["Seattle"] for example, which your code knows to map to calling the weather API function with argument “Seattle”. The response might be JSON which you convert to a readable sentence for the LLM.

Error Handling: It’s important to handle errors gracefully. If a tool fails (e.g. network error on search, or the math agent throws an exception on invalid input), the orchestrator can catch that and feed an error message back as the observation (so the LLM knows the action didn’t succeed and can try something else). Likewise, if the LLM requests an undefined tool, the system should return an observation like “Error: Unknown tool” or have the coordinator handle it by correcting the LLM’s choice.

By supporting a wide range of tools, the system becomes very powerful – it’s essentially an extensible AI agent platform. HuggingGPT showed that an LLM can coordinate dozens of models by just reading their descriptions (LLM Powered Autonomous Agents | Lil’Log), and similarly our system can coordinate diverse tools from web services to local computations. The key is the orchestrator code in Python that mediates between the LLM and these external functions.

Example Task Flow

To illustrate how all components come together, consider an example user query and walk through the internal flow of the system:

User request: “Find out the average GDP of France and Germany, then tell me which country had a higher GDP growth rate in the last year, and open the latest related news article.”

This complex request involves multiple sub-tasks (searching for data, doing a comparison, and browsing a news article). Here’s how the hybrid system might handle it:

Initial Understanding (Coordinator): The central LLM receives the query. In its prompt, it has the tool list (search, calculator, browser, etc.) and possibly some memory context. It begins reasoning: “The user asks for average GDP of France and Germany – likely need economic data, then compare growth rates from last year, and then find a news article about it. This requires web search for data and news, and maybe calculation for growth rates.”
- The LLM might output a plan (implicitly via chain-of-thought): Thought: “First, search for France GDP and Germany GDP data.”
- It then issues an Action: Search["France Germany GDP average last year"].
Web Search (Search Agent): The orchestrator receives the search action and invokes the Search agent with the query “France Germany GDP average last year”. The search agent (using an API or local knowledge) returns some results – e.g., an article or data source that has the GDP values and growth rates for France and Germany.
- Observation: The result might be a snippet: “France GDP (2024): $3.0 t r i ll i o n, g ro wt h 1.5$ 4.2 trillion, growth 1.2%.” This observation is fed into the LLM.
Reasoning on Data (Coordinator): The LLM now has data. It thinks: “The search result gives GDPs and growth rates. The question asks for average GDP and which had higher growth. I should calculate the average GDP of the two countries, and compare growth percentages.”
- It might parse the numbers from the text. Possibly it calls a Math agent if needed: Action: Calculate["(3.0 + 4.2)/2"] to compute the average GDP in trillions. Or the LLM could do that simple math itself – depending on prompt design, it might offload any math to be sure. Let’s say it does call the math tool.
Calculation (Math Agent): The math agent executes (3.0 + 4.2)/2 and returns 3.6.
- Observation: “3.6” (meaning average GDP is $3.6 trillion).
Continue Reasoning (Coordinator): The LLM now knows: average GDP ~3.6 trillion, France’s growth 1.5%, Germany’s 1.2%. It formulates partial answer in its head: “The average GDP is 3.6 trillion, and France had higher growth (1.5% vs 1.2%).” The last part of the user’s query: “open the latest related news article.” The LLM decides it needs to find a news article about GDP or economic growth for those countries.
- Thought: “Now I should find the latest news article related to France and Germany GDP growth.”
- Action: Search["latest news France Germany GDP growth rate"].
News Search (Search Agent): The search agent is invoked again with that query. It returns a result, say a news article titled “IMF Releases Latest Economic Outlook: France and Germany GDP Growth Updates”, with a URL.
- Observation: Perhaps the search agent returns the URL or a snippet of the article and a URL.
Web Browsing (Browser Agent): The LLM sees a URL in the observation and decides to use the Browser tool to fetch the full content.
- Action: Browse["<URL_of_article>"].
Fetch Article (Browser Agent): The Browser agent fetches the webpage content of the news article. Suppose it returns the first paragraph or a summary of the article (to avoid overloading the context).
- Observation: “Article Title: IMF Releases Latest Economic Outlook… Summary: The IMF report indicates France’s economy grew by 1.5% last year, slightly outpacing Germany’s 1.2% growth. However, both countries face challenges… (etc).”
Finalize Answer (Coordinator): The LLM now has all pieces: the numerical answer and some context from the news. It can compose the final response to the user.
- It forms the answer: “Answer: The average GDP of France and Germany is about $3.6 trillion. Last year, France had a higher GDP growth rate (around 1.5%) compared to Germany (about 1.2%). For example, a recent IMF report noted that France’s growth slightly outpaced Germany’s. [It then might provide or open the article as requested.]”
- Action: FinalAnswer["The average GDP ... (complete answer)..."].
Output: The system delivers this answer to the user. If this were an interactive setting (like a chat), the assistant would also perhaps say, “I have opened the latest related news article for you.” If integrated in a UI, maybe the article content is displayed.

Throughout this flow, the memory module could store the retrieved data (GDP figures, etc.) for future reference. If the user later asks something related (“What was France’s GDP growth again?”), the agent can recall it from memory instead of searching again. Also, every step was logged, so one can review how the agent reached its conclusion.

This example shows the system handling a multi-part query by dividing it into sub-tasks (data search, calculation, comparative reasoning, news lookup) and using different tools for each, under the direction of the central LLM. If at any point a sub-task returned an unexpected result (imagine the search didn’t find the data initially), the LLM could reformulate the query or try a different approach (for instance, query one country at a time). That would be an example of dynamic re-planning in action.

Recommendations for Python Implementation

Building this system in Python can be done using a combination of existing AI libraries and custom orchestration code. Here are some recommendations and best practices:

LLM Integration: Choose an LLM API or library for the coordinator. This could be OpenAI’s GPT-4 API, which is powerful and supports function calling, or a local open-source model via HuggingFace Transformers (like GPT-4All, Llama 2, etc. if running locally). Ensure you have a wrapper that can send the prompt (with the accumulating conversation + ReAct format) and receive the model’s output. For open-source models, libraries like transformers or LangChain’s LLM wrappers can be used. If using OpenAI and function calling, you can define each tool as a function in the API call so the model can directly invoke them (making the ReAct loop a bit more automatic).
Tool Agent Classes: Define each specialized agent as a Python class or function. It’s wise to give each a clear run or execute method that accepts input (maybe as strings or dict) and returns a result. Keep them stateless if possible (they take input, consult an API or perform computation, then return output). Use existing libraries when available: e.g., use the requests library for web APIs, selenium or playwright for complex browsing (if needed), numpy or built-in Python for math, etc. For search, you might use Bing Web Search API or Google API – you’ll need API keys and use their Python SDK or REST calls via requests. Wrap these details inside the agent so the orchestrator just calls SearchAgent.run(query).

Orchestrator Loop: The core of the implementation is the loop that alternates between the LLM and the tools. In pseudocode, it would look like:

conversation = []  # to hold the dialogue including thoughts/actions
conversation.append({"role": "user", "content": user_query})
while True:
    # Send conversation to LLM and get response
    response = llm.generate(conversation)  
    conversation.append({"role": "assistant", "content": response})
    if response.startswith("Action:"):
        # Parse the action and parameters
        action_name, action_arg = parse_action(response)
        if action_name == "FinalAnswer":
            # The assistant indicates it's done
            final_answer = action_arg
            break
        # Find the corresponding tool agent
        tool = tools_registry[action_name]
        result = tool.execute(action_arg)
        # Record the observation
        observation = f"Observation: {result}"
        conversation.append({"role": "assistant", "content": observation})
        # Loop continues, the observation will be in the next LLM prompt
    else:
        # If the LLM didn't output an action (which it should in ReAct until final), handle gracefully
        break

This outline uses a conversation list that holds the messages (for an OpenAI API, roles “user” and “assistant” are used; for a local model, you might just build a single prompt string). The parse_action function should interpret the LLM’s output. A simple approach: the LLM outputs Action: ToolName[arguments]. We can regex match ^Action:\s*(\w+)\[(.*)\]. The arguments might be a string or JSON depending on complexity; keep it simple, often it’s just a string or a few parameters. Then we look up tools_registry, which is a dict mapping tool name to the agent instance.

Each iteration appends the LLM’s thought/action and the observation back into the conversation. This is important – we keep the full chain as context so the LLM remembers what it has done so far. With GPT-4 or similar, it should handle quite a few turns (but be mindful of token limits; if the chain grows large, consider summarizing or pruning older parts that are no longer needed, though usually each task’s chain is short-lived).

Memory Storage: Implement a simple memory class that can save and retrieve information. For example, a KnowledgeBase class with methods store(key, info) and query(query_text). On each user query, you might do something like relevant_facts = kb.query(user_query) which returns a list of related facts (strings). You can then prepend those as a system message or as part of the user prompt. Storing can be done whenever a new fact is confirmed by the agent (e.g., after a successful search or a final answer, you might store it for later). For vector search, you can use faiss (Facebook AI Similarity Search) by storing embeddings. Libraries like sentence-transformers can turn text into vectors. Alternatively, for simplicity, you could just store recent Q&A pairs and do a substring match or embedding match.
Prompt Design: Crafting the prompt for the LLM coordinator is crucial. You’ll likely start the conversation with a system message like: “You are an AI coordinator that can solve tasks by reasoning and using tools. Follow the format: Thought → Action → Observation. You have the following tools: [list names and descriptions]. Use them when needed to find information or calculate. Think step by step and provide the final answer when ready.” Also include one or two few-shot examples in the prompt illustrating the ReAct process with your tools. This significantly helps the LLM produce correct format outputs. For instance, show an example conversation where a question is answered by using a Search action then a Calculate action, so the model sees how it should behave.
Testing and Refinement: As you implement, test with various queries. Start with simple ones that require one tool, then chain tools. Examine the logs (the chain-of-thought) to see if the reasoning is sound. If the LLM is doing unnecessary steps or making errors (e.g., calling the wrong tool), you may need to refine the prompt or the few-shot examples. It’s an iterative process to get the right balance of freedom and guidance in the prompt.
Parallelism and Scaling: In a straightforward ReAct loop, each action is decided sequentially. However, if the coordinator outputs a plan with multiple independent tasks (it could theoretically do something like “Action: Search[query1] and Search[query2] in parallel” though most models won’t unless explicitly allowed), you could run those in parallel threads and feed both results back. A simpler approach: if you know the task can be parallelized (say two searches that don’t depend on each other), you could modify the plan to do so outside the LLM. But generally, LLM will handle one step at a time. For scaling, consider using asynchronous I/O in Python (asyncio) for tools like web requests, so the LLM isn’t blocked unnecessarily. Also, you could deploy each agent as a microservice if building a large system (the coordinator calls an API endpoint for the Search agent instead of a local function, which allows that service to scale independently). The architecture is flexible to such distribution.
Use of Frameworks: While one can write this from scratch, frameworks like LangChain or Haystack provide abstractions for tool use and memory. LangChain, for example, has the concept of Agents with Tools and can manage the prompting for ReAct style chains. It can save time to leverage such a library – you’d define your tools and let the framework handle the loop and parsing. However, understanding the basics as we’ve outlined is important because it allows customization. Given the emphasis on transparency, you might prefer a custom implementation where you have full control over logging and decision-making.
Error Handling & Security: Ensure the system has safeguards. An open-ended LLM that can execute code or call APIs needs constraints. For instance, validate inputs before passing to a shell or calculator to avoid executing dangerous code. Limit the search queries to avoid very long or malicious queries. Also, monitor for the LLM going off-script – if it outputs something that isn’t following the format, your parser should handle it (maybe by reminding the model of the format or re-issuing the prompt with more constraints). Logging every action with timestamps and tool outputs is useful not just for transparency but for auditing what the agent did.

In summary, implementing this in Python involves managing prompts and context for the LLM, setting up a loop to handle the ReAct pattern, and writing/connecting various tool functions. Emphasize clean modular design: separate the concerns (LLM interaction vs tool implementations vs memory store). This makes the system easier to maintain and scale. With careful engineering, you’ll have a system where the LLM does the high-level reasoning and delegates specialized tasks to Python functions – harnessing the strengths of both AI and traditional software. By following the ReAct principle for clarity and HuggingGPT’s orchestration ideas for tool use, the resulting system will be robust, extensible, and capable of addressing complex multi-step queries with transparency and efficiency.

🪴 Quartz 4.0

Explorer

Архитектура решения