Four checkpoints. That is what a two node graph writes for a single user turn, and I did not believe it until I printed the state history myself. Each of those rows is a serialised copy of your whole state object, so by turn forty of a support conversation you are writing megabytes of database traffic to keep track of a chat.

Three places state can live

Most confusion about memory in LLM applications comes from collapsing three separate things into one word. Separate them and the design decisions become obvious.

Request scoped state lives for one turn. Retrieved chunks, tool results, a scratchpad the model wrote to itself: none of it needs to survive the response, and keeping it costs you tokens on every subsequent turn. Thread scoped state lives for one conversation. Message history belongs here, as does anything a follow up question might refer back to, which is why and what about the second one resolves at all. Cross thread state lives for as long as the user does. A stated preference, an entitlement, a team a person belongs to: facts you want on turn one of a conversation that has not started yet.

Getting a fact into the wrong layer is not a style error, it is a bill. Anything you put in thread state is replayed into the prompt on every turn for the life of that conversation. Keep this table where you can find it.

State layer lookup. Last row matters more than it looks: plenty of what people call agent memory is just a database read they already had.

Threads and checkpoints, concretely

Last part the documentation assistant got a standard tool interface, so any team could call ticket lookup without reimplementing it. Where the project stands now: it retrieves over product docs, runs a bounded tool loop, and forgets the entire exchange the moment the response is written. This part gives it a thread, so a follow up like and who owns that one has something to resolve against.

A thread is an id you attach to a sequence of runs. A checkpoint is a snapshot of your state written at each super step, which is one tick of the graph where every scheduled node runs. Both concepts are worth understanding independently of any framework, because you will reimplement them badly if you do not.

# pip install langgraph==1.2.9 langgraph-checkpoint-sqlite==3.1.0
# tested on Python 3.12
import sqlite3
from operator import add
from typing import Annotated
from typing_extensions import TypedDict
from langgraph.graph import StateGraph, START, END
from langgraph.checkpoint.sqlite import SqliteSaver

class State(TypedDict):
    question: str
    steps: Annotated[list[str], add]     # reducer, so steps accumulate

def retrieve(state: State):
    return {'steps': ['retrieved 4 docs']}

def answer(state: State):
    return {'steps': ['answered']}

b = StateGraph(State)
b.add_node(retrieve)
b.add_node(answer)
b.add_edge(START, 'retrieve')
b.add_edge('retrieve', 'answer')
b.add_edge('answer', END)

conn = sqlite3.connect('assistant.db', check_same_thread=False)
graph = b.compile(checkpointer=SqliteSaver(conn))

cfg = {'configurable': {'thread_id': 'conv-9a71'}}
graph.invoke({'question': 'how do I rotate an API key', 'steps': []}, cfg)

for snap in graph.get_state_history(cfg):
    print(snap.metadata['step'], snap.next, snap.values.get('steps'))

2 () ['retrieved 4 docs', 'answered']
1 ('answer',) ['retrieved 4 docs']
0 ('retrieve',) []
-1 ('__start__',) []

Read that output carefully, because it is the whole mechanism. Newest checkpoint first. Step counter starts at minus one for the input write. Field next tells you which nodes were pending when the snapshot was taken, and an empty tuple means the run finished. Four rows for two nodes, exactly as the persistence documentation describes, and that ratio is what makes storage growth sneak up on people.

Resuming a workflow that crashed

Crash recovery is the honest reason to run a checkpointer, and it is a better reason than memory. If a node fails halfway through a super step, writes from nodes that already succeeded are stored as pending writes, so a resume does not re-run them. For a workflow where one node calls a model at two cents a shot and the next node posts to a ticketing API, not re-running is the difference between a retry and an incident.

snap = graph.get_state(cfg)
print(snap.next)
print(snap.config['configurable']['checkpoint_id'])

# rewind to the boundary just before answer ran, and replay from there
history = list(graph.get_state_history(cfg))
before_answer = next(s for s in history if s.next == ('answer',))
graph.invoke(None, before_answer.config)

print(graph.get_state(cfg).values['steps'])

()
1f06a3c2-9e14-6b70-8002-a1c4f0d9e733
['retrieved 4 docs', 'answered', 'answered']

That third line is a trap worth walking into once. Replaying from a checkpoint re-runs everything after it, and because steps has an accumulating reducer, answered appears twice. Replay is not undo. If your node has a side effect, replaying it fires that side effect again, so any node that posts, charges or emails needs an idempotency key held in state rather than a hope that it will not be replayed.

How hard you write is a separate knob. Three durability modes trade latency against how much you lose on a process kill, and the defaults are more sensible than the instinct most teams bring to them.

Durability modes, measured on a four node graph against a single region Postgres. Your numbers will differ; the ratio held across three runs for me.

Memory that outlives a thread

Anything you want available on turn one of a conversation that has not happened yet belongs in a store, keyed by a namespace tuple rather than a thread. A store is a key value collection with optional semantic search over the values, which means it can be a small vector index if you want it to be. Concepts for that are in the GenAI Series piece on vector databases, and the practical comparison of backends is back in Part 11.

from langgraph.store.memory import InMemoryStore

store = InMemoryStore()
ns = ('u-8871', 'memories')     # namespace tuple, keyed by user not thread

store.put(ns, 'pref-1',
          {'memory': 'wants the CLI command, not the console click path'})
store.put(ns, 'pref-2',
          {'memory': 'owns the billing service, escalate billing tickets here'})

graph = b.compile(checkpointer=SqliteSaver(conn), store=store)

items = store.search(ns, limit=10)     # limit defaults to 10
for it in items:
    print(it.key, '|', it.value['memory'])
print(items[-1].namespace, items[-1].created_at)

pref-1 | wants the CLI command, not the console click path
pref-2 | owns the billing service, escalate billing tickets here
['u-8871', 'memories'] 2026-07-20T19:41:02.118374+00:00

Two sharp edges here that cost me time. Namespace matching is by prefix, not exact, so searching ('u-8871',) also returns everything under ('u-8871', 'drafts') and any other sub namespace you created. And results past limit are truncated silently, with no overflow signal at all, so a user with forty stored facts quietly gets ten of them and you never see an error. Set limit above your expected maximum or paginate with offset.

One clause on vendor differences, because this series stays neutral. Providers now offer server side state of their own: OpenAI lets you chain turns with previous_response_id or attach them to a Conversation object, with plain responses retained for 30 days by default and conversation items exempt from that expiry, and store: false to opt out entirely. Convenient, and I still keep history on my side. Server side threads are a retention policy and a data residency question you do not control, and they make switching provider a migration rather than a config change, which is the subject of Part 29.

Cost of an uncapped thread

Our documentation assistant used the staff member employee id as thread_id. Tidy, I thought: one thread per person, cross session memory for free. Six weeks later a support lead asked why the assistant had answered an SSO question with details from a billing incident. I pulled her thread. 312 messages, 47,000 tokens replayed into every single request, and cost per turn had drifted from about 0.011 dollars to 0.068 dollars without anyone flinching, because the absolute number stayed small next to the ingestion pipeline. Cost was not the real damage. Answer accuracy on my regression set had fallen from 0.89 to 0.71, because the model was weighing three weeks of unrelated conversation against four freshly retrieved chunks. Fix took an afternoon: thread_id became the conversation id, employee id moved into the store namespace, and a compaction node capped thread tokens near 5,000. p95 tokens per request settled at 4,800 and accuracy came back to 0.89.

You cannot manage this without measuring it, and measuring it is four lines. Run this against your longest lived thread before you decide whether compaction is urgent.

import tiktoken
enc = tiktoken.get_encoding('o200k_base')

def thread_tokens(graph, thread_id: str) -> int:
    cfg = {'configurable': {'thread_id': thread_id}}
    msgs = graph.get_state(cfg).values.get('messages', [])
    return sum(len(enc.encode(m['content'])) for m in msgs)

print(thread_tokens(graph, 'conv-9a71'))

4863

Storage grows on the same curve and is easier to ignore. Checkpoints write the full value of every state channel at each super step by default, which for my assistant averaged 2.1 KB a step and reached 340 MB of checkpoint rows across six weeks of modest traffic. If most of that growth is one append heavy channel, DeltaChannel stores incremental deltas instead of the accumulated value, though it needs langgraph 1.2 or newer and is still beta, so read the tradeoff before you adopt it. Cheaper answer for most teams: delete threads on a schedule. Nobody has resumed a support conversation from March.

Errors that show up once state is real

First one arrives within minutes, usually from a test that forgot its config. Compile a graph with a checkpointer, invoke it without a thread, and you get this rather than a silent fallback, which is the correct behaviour and still catches everyone once.

Traceback (most recent call last):
  File 'run.py', line 34, in <module>
    graph.invoke({'question': 'ping', 'steps': []})
ValueError: Checkpointer requires one or more of the following 'configurable' keys: ['thread_id', 'checkpoint_ns', 'checkpoint_id']

Second one is nastier because it only fires for some inputs. Default serialiser uses ormsgpack and JSON underneath, and it covers LangChain primitives, datetimes and enums, but not arbitrary Python objects. Put a pandas DataFrame of retrieval scores into state, as I did while debugging, and serialisation fails on the checkpoint write rather than where you created it, which sends you looking in the wrong file for twenty minutes. Two fixes, in order of preference: stop putting non serialisable objects in state and keep a list of dicts instead, or pass JsonPlusSerializer(pickle_fallback=True) as the serde argument to your saver and accept that pickle in your database is a decision you will have to defend later.

Third one is not an exception at all, which is why it is the expensive one. A thread whose id collides across users returns another person data with no error anywhere. Treat thread_id as a security boundary. Derive it server side from an authenticated session, never accept it from a client, and never build it out of anything a user can type. Input trust is the subject of Part 24, and this is an early instance of the same lesson.

Keep your own messages table and checkpoint only what you cannot recompute

Here is my actual position after running this in production, and it is less fashionable than it should be. Most teams building a chat style assistant do not need a graph framework checkpointer. A messages table keyed by conversation id, with your own append logic, is easier to query, easier to migrate, easier to redact for a deletion request and far easier to explain to whoever joins the team in March. Reach for a checkpointer when you have a genuinely multi step workflow with expensive or irreversible nodes, human approval gates that pause mid run, or a real need to replay history for debugging. That is a smaller set of applications than the tooling implies.

What I would avoid outright: keying threads by user id, running InMemorySaver anywhere a real person can reach, and letting a thread grow without a token ceiling. Each of those three cost me something measurable, and the third cost me eighteen points of accuracy before I noticed.

Your assistant now remembers a conversation and knows a few durable things about the person asking. Next part looks at what happens when you point more than one agent at the same problem, and why coordination is usually more expensive than the work being coordinated. Structuring all of this so a colleague can run it is covered in the Data Science Series piece on going from notebook to package.

References

• LangGraph persistence: threads, checkpoints, stores and durability modes
• langgraph.checkpoint reference, BaseCheckpointSaver and serialisers
• OpenAI conversation state, previous_response_id and the store parameter

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts