LangChain Memory: Conversations That Remember

By default, a language model has no memory between calls. Each time you send a message, the model starts fresh. For one-shot tasks this is fine, but for a chatbot it is a disaster. Users expect the model to remember what they said three turns ago. Without memory, every conversation degenerates into "what did I just tell you?"

LangChain solves this with memory modules that manage conversation history and inject it into each new prompt automatically. This post covers the two most useful options and shows you how to wire them into a chain.

Why Memory Matters More Than You Think

Consider a user who says "I run a SaaS company with 50 employees" and then later asks "what is a good churn rate for my business?" Without memory, the model does not know it is talking to a SaaS founder. With memory, it can give a targeted answer: enterprise SaaS benchmarks are different from SMB, which are different from consumer.

Memory also prevents the friction of re-explaining context at every step. A good assistant remembers what you told it and builds on it. That is the baseline users now expect from AI products.

ConversationBufferMemory: Simple and Reliable

ConversationBufferMemory stores every message in the conversation verbatim and appends the full history to each new prompt. It is the simplest option and works well for short to medium conversations.

from langchain_anthropic import ChatAnthropic
from langchain.memory import ConversationBufferMemory
from langchain.chains import ConversationChain

model = ChatAnthropic(model='claude-3-5-sonnet-20241022')

memory = ConversationBufferMemory(return_messages=True)

conversation = ConversationChain(
    llm=model,
    memory=memory,
    verbose=False
)

# First turn
response_1 = conversation.predict(
    input='I run a SaaS company with 50 employees in the HR tech space.'
)
print(response_1)

# Second turn - the model remembers the context from turn 1
response_2 = conversation.predict(
    input='What is a healthy monthly churn rate for my business?'
)
print(response_2)

The memory object accumulates messages on every predict call. Pass the same memory instance to every call in a session and the model will have full context for the entire conversation.

ConversationSummaryMemory: Handle Long Conversations

Buffer memory has a problem: long conversations exceed the model's context window. ConversationSummaryMemory solves this by using the LLM itself to compress older parts of the conversation into a rolling summary. Only the summary and the most recent messages are sent with each new request.

from langchain.memory import ConversationSummaryMemory
from langchain_anthropic import ChatAnthropic
from langchain.chains import ConversationChain

model = ChatAnthropic(model='claude-3-5-sonnet-20241022')

# This memory uses the model to summarize old turns
summary_memory = ConversationSummaryMemory(
    llm=model,
    return_messages=True
)

long_conversation = ConversationChain(
    llm=model,
    memory=summary_memory
)

# Works the same way as buffer memory from the outside
response = long_conversation.predict(input='Tell me about our product roadmap.')
print(response)

Use buffer memory for short conversations (under 20 turns). Switch to summary memory when conversations can run long or when users are likely to stay in a session for more than a few minutes.

Persisting Memory Across Sessions

In-memory storage disappears when your server restarts. For production chatbots, persist the conversation history to a database. LangChain has built-in support for Redis, PostgreSQL via SQLAlchemy, MongoDB, and others through the langchain-community package.

Set this as the system prompt for your ConversationChain and the model will actively leverage the memory it has, not just passively carry it along.