Langchain, Vercel and Waiters

Introduction
Language models like OpenAI’s GPT series have revolutionized natural language processing tasks. However, effectively integrating these models requires robust frameworks and tools. LangChain offers a suite of components for building applications with large language models, while the Vercel AI SDK provides tools for seamless integration with frontend applications, including support for streaming responses. In this guide, we’ll build a chat application that leverages LangChain to process and generate responses based on user input, using the Vercel AI SDK to handle real-time communication between the server and client.

But first, a fun analogy, Imagine you’re throwing a grand party. LangChain is like your master chef, capable of creating complex and delightful dishes (AI functionalities) by chaining together recipes (modules). The Vercel AI SDK is your efficient waiter, ensuring these dishes are served to your guests (users) promptly and elegantly.

Let’s get into the code.

Prerequisites

Node.js and npm installed

Basic understanding of Next.js or a similar framework

An OpenAI API key

Project Setup
Initialize a new Next.js project:

npx create-next-app@latest my-langchain-app
cd my-langchain-app

Install the required dependencies

npm install openai langchain ai

Setting Up Environment Variables
Create a .env.local file in the root of your project and add your OpenAI API key:

OPENAI_API_KEY = your - openai - api - key - here;

Ensure this file is included in your .gitignore to prevent sensitive information from being committed to version control. (I learnt this in a very embarassing way)

Implementing the API Route
We’ll create an API route that handles incoming chat messages and generates responses using LangChain.

// pages/api/chat.js
import { ChatOpenAI } from "langchain/chat_models/openai";
import { PromptTemplate } from "langchain/prompts";
import { LLMChain } from "langchain/chains";
import { StreamingTextResponse } from "ai";

export default async function handler(req, res) {
  try {
    const { messages } = req.body;

    const chat = new ChatOpenAI({
      openAIApiKey: process.env.OPENAI_API_KEY,
      modelName: "gpt-3.5-turbo",
      temperature: 0.7,
      streaming: true,
    });

    const prompt = PromptTemplate.fromTemplate(`
      The following is a conversation between a user and an assistant. The assistant is helpful, creative, and friendly.
      {conversationHistory}
      User: {userMessage}
      Assistant:
    `);

    const chain = new LLMChain({
      llm: chat,
      prompt,
      verbose: true,
    });

    const conversationHistory = messages
      .slice(0, -1)
      .map(
        (msg) => `${msg.role === "user" ? "User" : "Assistant"}: ${msg.content}`
      )
      .join("\n");

    const userMessage = messages[messages.length - 1].content;

    const response = await chain.call({
      conversationHistory,
      userMessage,
    });

    return res.send(new StreamingTextResponse(response.text));
  } catch (error) {
    console.error("Error handling chat request:", error);
    res.status(500).json({ error: error.message });
  }
}

Explanation:
Import necessary modules from LangChain and the Vercel AI SDK. Create an instance of ChatOpenAI, configured for streaming responses. Define a PromptTemplate that structures how the assistant should respond, considering conversation history. Use an LLMChain to process the input and generate a response. Compile the conversation history to maintain context in the conversation.

Creating the Frontend Component
Next, we’ll create a React component that interacts with our API using the Vercel AI SDK.

// components/Chat.js
import { useChat } from "ai/react";

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit } = useChat({
    api: "/api/chat",
  });

  return (
    <div>
      <h2>Chat with AI Assistant</h2>
      <div>
        {messages.map((m) => (
          <div key={m.id}>
            <strong>{m.role === "user" ? "User" : "Assistant"}:</strong>{" "}
            {m.content}
          </div>
        ))}
      </div>
      <form onSubmit={handleSubmit}>
        <input
          value={input}
          onChange={handleInputChange}
          placeholder="Type your message..."
          autoComplete="off"
        />
        <button type="submit">Send</button>
      </form>
    </div>
  );
}

Explanation:
Use the useChat hook from the Vercel AI SDK to manage chat state and interactions. Render the conversation history, displaying messages from both the user and the assistant. Provide an input field for the user to send new messages.

Integrating the Chat Component
Include the Chat component in your main application page, such as pages/index.js:

// pages/index.js
import Chat from "../components/Chat";

export default function Home() {
  return (
    <main>
      <Chat />
    </main>
  );
}

How It Works

LangChain Components ChatOpenAI: Wraps around OpenAI’s chat models, providing methods to interact with the API. PromptTemplate: Allows for dynamic creation of prompts, inserting variables where needed. LLMChain: A chain that connects the language model with the prompt, handling the input and output flow. Vercel AI SDK useChat Hook: Simplifies the implementation of chat functionality, handling state management for messages and input. StreamingTextResponse: Enables server-side streaming of responses to the client, allowing messages to appear in real-time.

Or if you prefer the kitchen analogy;

LangChain as the Master Chef: It knows how to combine ingredients (data, prompts, models) to create an exquisite dish.
Vercel AI SDK as the Waiter: It ensures the prepared dishes are delivered to the guests smoothly and interactively.
Chains as Recipes: Predefined steps to achieve a delicious outcome.

Testing the Application Run your development server:

npm run dev

Open your browser and navigate to http://localhost:3000. You should be able to interact with the AI assistant, with responses appearing in real-time.

Conclusion
By integrating LangChain with the Vercel AI SDK, we’ve created a powerful yet straightforward chat application that leverages the capabilities of large language models. This approach abstracts much of the complexity involved in handling prompt engineering, API communication, and real-time data streaming.

And remember, you can’t learn everything at once, one line of code at a time!