Fixing Langchain.js's ToolCallingAgentOutputParser Error with Ollama LLM and a Custom Tool

Understanding and Fixing ToolCallingAgentOutputParser Errors in Langchain.js

When working with Langchain.js v2, developers often aim to create efficient agents using custom tools and language models like Ollama. However, integrating these components can sometimes lead to errors that are difficult to debug.

One such error is the "parseResult on ToolCallingAgentOutputParser only works on ChatGeneration output," which can occur when building a custom tool within the agent framework. Understanding the root cause of this issue is crucial to ensure the agent and tool work correctly.

This article explores a simple implementation of a custom tool that adds 2 to a number input, using Langchain's createToolCallingAgent and the Ollama model. By analyzing the error and its context, we can better grasp how to troubleshoot it.

The following sections will guide you through the code, explain the error, and provide solutions to address this problem. Whether you're new to Langchain.js or experienced, this guide will help you move past this issue efficiently.

Understanding the Custom Tool and Agent Error Handling in Langchain.js

In the example provided, we are working with Langchain.js v2 to create a custom tool that integrates with the Ollama language model. The main purpose of the tool is to perform a simple mathematical operation: adding 2 to the input value. The tool is built using Langchain’s tool function, which defines reusable functions that can be invoked by an agent. To ensure the tool works correctly, the input schema is validated with the Zod library, guaranteeing that the input is a valid number. This ensures proper error handling and prevents the tool from failing due to invalid inputs.

The custom tool is then incorporated into an agent using the createToolCallingAgent function. This command allows the agent to call the tool when needed, and the agent is powered by the Ollama model, which is configured with specific parameters such as temperature to control the creativity of the responses. To facilitate smooth interaction between the agent and the tool, a chat prompt template is used. This template organizes the conversation by defining different types of messages, such as system messages, human input, and placeholders. The placeholders, such as MessagesPlaceholder, allow the conversation to be dynamic, with elements like the chat history being included.

One of the key issues addressed in this example is the error handling around the Langchain agent's output parsing. The error message "parseResult on ToolCallingAgentOutputParser only works on ChatGeneration output" stems from a mismatch between the type of output expected by the parser and the actual output generated. To handle this error, the custom tool is wrapped in robust logic, ensuring that all inputs and outputs conform to expected formats. This is further managed by the AgentExecutor class, which coordinates the execution of the agent and tools, making sure that the query and tool output are properly synchronized.

Finally, the scripts implement asynchronous execution using await, allowing the system to handle operations without blocking other processes. The agent waits for the tool to return its result before proceeding, ensuring that the response is both accurate and timely. Additionally, unit tests are included to validate the tool's functionality, ensuring that it consistently produces the correct output. These tests not only confirm the tool’s mathematical operation but also check how well it handles invalid input, improving the overall reliability of the solution. This modular and error-resistant design makes the scripts reusable and effective for various applications within Langchain.js.

import { tool } from "@langchain/core/tools";
import { z } from "zod";
import { Ollama } from "@langchain/ollama";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { createToolCallingAgent } from "langchain/agents";
import { AgentExecutor } from "langchain/agents";
// Initialize LLM with Ollama
const llm = new Ollama({
    model: "llama3",
    temperature: 0.7,
});
// Custom tool to add 2 to the input number
const magicTool = tool(
    async (input) => {
        return input + 2;
    },
    {
        name: "magic_function",
        description: "Applies a magic function to an input",
        schema: z.object({ input: z.number() }),
    };
);
const tools = [magicTool];
// Setup ChatPromptTemplate with placeholders
const prompt = ChatPromptTemplate.fromMessages([
    ["system", "You are a helpful assistant called iHelp"],
    ["placeholder", "{chat_history}"],
    ["human", "{input}"],
    ["placeholder", "{agent_scratchpad}"],
]);
// Agent configuration
const agent = createToolCallingAgent({ llm, tools, prompt });
// Execute agent query
const agentExecutor = new AgentExecutor({ agent, tools });
const query = "What is the value of magic_function(3)?";
await agentExecutor.invoke({ input: query });

import { tool } from "@langchain/core/tools";
import { z } from "zod";
import { Ollama } from "@langchain/ollama";
import { createToolCallingAgent } from "langchain/agents";
import { AgentExecutor } from "langchain/agents";
// Initialize LLM with Ollama
const llm = new Ollama({ model: "llama3", temperature: 0.7 });
// Custom tool with added error handling
const magicTool = tool(
    async (input) => {
        try {
            if (typeof input !== "number") throw new Error("Invalid input type!");
            return input + 2;
        } catch (err) {
            return err.message;
        }
    },
    {
        name: "magic_function",
        description: "Adds 2 to input and handles errors",
        schema: z.object({ input: z.number() }),
    }
);
const tools = [magicTool];
// Agent and execution
const agent = createToolCallingAgent({ llm, tools });
const agentExecutor = new AgentExecutor({ agent, tools });
const query = "magic_function('abc')"; // Test with invalid input
await agentExecutor.invoke({ input: query });
// Unit test example
import { expect } from "chai";
it("should return 5 when input is 3", async () => {
    const result = await magicTool(3);
    expect(result).to.equal(5);
});

Exploring the Role of Agents in Langchain.js and Ollama LLM Integration

When working with Langchain.js, integrating agents with tools and language models like Ollama is a critical aspect of building dynamic applications. An agent allows you to connect a custom tool, which performs specific tasks, to a language model, which handles more conversational or generative tasks. By using agents, developers can automate workflows where a model not only generates responses but also invokes tools to perform calculations or data processing.

The key component in this integration is the createToolCallingAgent function. This function lets the agent trigger specific tools when necessary, ensuring that tasks are completed accurately and efficiently. While the primary focus is often on creating the tool itself, understanding how to manage the agent's workflow and avoid parsing errors is equally important. Errors like "parseResult on ToolCallingAgentOutputParser only works on ChatGeneration output" usually occur when the agent's output isn't compatible with the parsing system, highlighting the need for proper alignment between the agent’s output and the expected format.

The use of prompt templates, such as ChatPromptTemplate, further enriches the interaction by allowing dynamic messages and context placeholders. This allows the agent to adjust its responses based on the chat history or the agent’s scratchpad. Optimizing the prompt templates and ensuring the agent's outputs are correctly parsed can prevent many common errors, making your Langchain.js applications more reliable and efficient.