A user types cancel order 4471 and the assistant answers with three tidy paragraphs about the returns policy. It parsed the sentence perfectly, then did the one thing a bare model can do when it has nowhere to send an intent. It talked.
Key takeaways
watsonx Assistant is the conversational front end for the models and agents you have already built. It answers with two mechanisms: actions, which are scripted task flows you control step by step, and conversational search, built-in RAG that handles the questions no action covers. Route deterministic tasks to actions, the long tail of document questions to conversational search, and anything needing tools or multi-step reasoning to the Orchestrate agent from Part 15. Ship on web chat first. Watch the per-conversation cost, because MAU and message pricing climb faster than traffic does.
Part 15 ended with a promise: the agent you built would become something people actually talk to. This is that part. In earlier parts you called Granite over the inference API in Part 8, grounded it with retrieval in Part 9, and gave it tools and routing in Part 15. None of that is a product a customer can use on its own. watsonx Assistant is the layer that turns it into one, and the mistake at the top, a bot that explains instead of acting, is exactly the mistake this layer exists to prevent.
What watsonx Assistant does, and where it stops
watsonx Assistant is IBM’s platform for conversational applications, meaning applications a person interacts with by typing or speaking rather than clicking. It owns the conversation itself: the chat widget or phone channel, the turn taking, the memory of what was said two messages ago, and the analytics that tell you where users got stuck. That is a different job from the two layers below it. watsonx.ai holds the models and the tuning. watsonx Orchestrate, from Part 15, holds the agents that reason and call tools. Assistant is the front door those two stand behind.
Knowing where it stops matters as much as knowing what it does. Assistant does not train models, and by itself it is not the reasoning engine for a complex multi-step task. It is very good at scripted conversation and at retrieval-based answers, and it hands off everything harder. IBM has been drawing Assistant and Orchestrate closer together under one builder, but the capability you are reaching for is unchanged: build a thing people talk to, keep the conversation coherent, and connect it to the logic that does the work.
Assistant answers a request one of three ways, and picking the right one for each kind of request is the design work of this part. A scripted task is an action. A question against your documents is conversational search. Anything that needs tools, live data, or several reasoning steps goes to an agent. The rest of this part is how to build each and how to decide between them.
How an action turns a message into an answer
An action is a single conversation that completes one task, such as booking a return or checking an order. You build it from steps. A step is one exchange: the assistant says something, the customer responds, and a condition decides which step runs next. Values the user gives you along the way, an order number, a date, get stored in session variables, which are named slots that persist for the length of the conversation so a later step can use what an earlier step collected.
The assistant decides which action a message belongs to from training phrases, the example sentences you attach to each action. When a message comes in, it is matched against those phrases, the best matching action starts, its steps run in order, and if a step needs a fact from a real system it calls a tool or a custom extension, which is a connection to an external API. When no action matches, the request falls through to conversational search or to an agent. That fall-through is the safety net that stops the bot from guessing.
The diagram below is one user message travelling through that decision.
flowchart TD
U[User message on a channel] --> M{Matches an action?}
M -->|Yes| S[Run steps, fill session variables]
S --> T{Step needs a fact?}
T -->|Yes| X[Call a tool or extension]
T -->|No| R[Compose reply]
X --> R
M -->|No, a doc question| CS[Conversational search over knowledge base]
M -->|No, needs reasoning| AG[Route to Orchestrate agent]
CS --> R
AG --> R
R --> U2[Reply to user]
Gotcha
Training phrases are routing, not decoration, the same lesson as tool descriptions in Part 15. Give an action three near-identical phrases and it will miss the fourth way a user phrases the same intent, then start on the wrong action or fall through. Write five to ten genuinely varied phrases per action, including the blunt ones real users type, and re-check them against the analytics after launch. Most misroutes I see are thin training data, not a weak model.
Build a first action and call it from the API
You author actions in the no-code Actions editor in the browser: create the action, add training phrases, lay out the steps, and preview it in the panel on the right. That is the fast way to get the conversation right. But an assistant only earns its keep once it is embedded in something, so the piece worth showing here is how your application talks to it. The runtime API is one POST per user message. Below I send a message to a published assistant with curl, using the stateless message endpoint so there is no session to open and close.
# 1. Exchange your API key for a short-lived IAM token
export IAM_TOKEN=$(curl -s -X POST 'https://iam.cloud.ibm.com/identity/token'
-H 'Content-Type: application/x-www-form-urlencoded'
-d 'grant_type=urn:ibm:params:oauth:grant-type:apikey&apikey=YOUR_API_KEY'
| python3 -c 'import sys,json; print(json.load(sys.stdin)["access_token"])')
# 2. Send one message, stateless, no session to manage
curl -s -X POST
"$ASSISTANT_URL/v2/assistants/$ASSISTANT_ID/message_stateless?version=2024-08-25"
-H 'Content-Type: application/json'
-H "Authorization: Bearer $IAM_TOKEN"
-d '{"input":{"message_type":"text","text":"where is order 4471?"}}'
Expected: a JSON body whose output.generic array holds the reply, for example {"output":{"generic":[{"response_type":"text","text":"Your order 4471 shipped on Tuesday."}]}}. Your application reads output.generic and renders each entry in the chat window. The version query parameter is a dated API version.
Failure mode: a 401 means the IAM token step failed, usually a wrong or expired API key, so re-run step 1 and confirm the token is non-empty. A 404 on the message call means the ASSISTANT_ID or the service URL points at the wrong environment or region. Fix the URL before retrying, because a wrong region returns 404 on every call, not a transient error to retry through.
Conversational search for the questions actions miss
You cannot script an action for every question a user might ask, and you should not try. Conversational search is the built-in retrieval-augmented generation, or RAG, that catches the rest. It is generally available and powered by watsonx.ai. When a message matches no action, the assistant retrieves relevant passages from a knowledge base, then sends them to a Granite or other watsonx.ai model to compose a grounded answer, the same retrieval pattern you wired by hand in Part 9, now managed inside the assistant.
The knowledge base can be an Elasticsearch index, watsonx Discovery, a Milvus vector store, or your own retrieval service through the custom retrieval option, so you can point it at the index you already run. A few controls decide how it behaves. Streaming returns the answer token by token so the user is not staring at a blank box. Response length is set to concise, moderate, or verbose. Contextual awareness switches between reading only the current message, Single Turn, or the whole conversation, Entire conversation, which matters for follow-up questions like what about the refund. And a Search info banner in the preview lets you inspect which passages were retrieved, which is the first place to look when an answer is wrong.
The three-way split, action, search, or agent, is the decision to internalise. Actions for tasks with a known shape. Conversational search for open questions against documents. An agent for anything that needs tools or several reasoning steps.
| Mechanism | What it handles | Example request | Backed by |
|---|---|---|---|
| Action | A task with a known shape | Track order 4471 | Steps plus a tool or extension |
| Conversational search | Open questions on your documents | What is the warranty on this model | Retrieval plus a watsonx.ai model |
| Agent | Reasoning across tools and steps | Rebook my flight and expense it | watsonx Orchestrate, Part 15 |
Response length is not a cosmetic choice, it is a cost and latency choice. A verbose answer is more tokens generated per reply, which is more money and more wait, multiplied across every conversation. The chart below is the rough shape of that difference.
Put the Orchestrate agent behind the assistant
Actions and conversational search cover scripted tasks and document questions. The agent from Part 15 covers everything that needs to reason across tools, and connecting it is the point this whole series has been building toward. The clean division of labour is this: the assistant owns the conversation and the channel, and the agent owns the reasoning and the tool calls. A request that needs to check three systems and decide what to do next is not an action you script, it is a job you hand to the agent, which routes among its tools exactly as it did in Part 15.
The trap is duplicating logic. If you rebuild the payslip lookup as an action here and also as a tool on the agent, you now maintain the same rule in two places and they will drift. Keep each piece of logic in one home. Simple, fixed flows live as actions in the assistant. Reasoning and real tool use live in the agent. The assistant decides which to invoke, and neither reimplements the other.
My take
Do not put the agent behind the assistant on day one. Ship actions and conversational search first, watch the transcripts, and you will find that a surprising share of real traffic is scripted tasks and document questions that never needed an agent at all. Add the agent when the transcripts show requests that genuinely need multi-step reasoning. Wiring the most expensive path in before you know you need it is how a simple support bot ends up slow and pricey for questions a single action would have closed.
Which channels are worth turning on first?
The same assistant can serve many channels, a channel being a surface users reach it through. The built-in web chat drops into a website with a script tag. Phone and voice connect it to a real telephone number. There are integrations for SMS, WhatsApp, Slack, and Microsoft Teams, and a custom channel through the same runtime API you saw above for anything else. One assistant definition, many front ends.
The temptation is to turn on several at launch. Do not. Each channel has its own quirks, voice needs speech handling and shorter replies, WhatsApp has message templates and approval rules, and every extra surface multiplies what you test and monitor. Ship one, learn from real transcripts, then add the next.
| Channel | How it connects | Reach for it when |
|---|---|---|
| Web chat | Script tag on your site | Almost always, start here |
| Phone and voice | A telephony number and voice config | Deflecting call-centre volume |
| WhatsApp and SMS | Messaging integration and templates | Customers already reach you there |
| Slack and Teams | Workspace app integration | An internal, employee-facing bot |
| Custom | The runtime message API | An app or surface with no prebuilt integration |
What a conversation actually costs
Pricing is where conversational apps surprise teams, because the meter runs on usage, not on the number of assistants you build. The Lite plan is free for a small cap, roughly 1,000 monthly active users and 10,000 messages, which is fine for a prototype. The Plus plan starts around 140 dollars a month, includes about 1,000 monthly active users, and then bills per additional message at a fraction of a cent, on the order of 0.0014 dollars each. Enterprise pricing is custom. A monthly active user is one distinct person who interacts in the month, and a message is a single exchange, so both climb with adoption.
Worked example
Take a support assistant on the Plus plan. At a light 10,000 messages a month you are near the 140 dollar base. Grow to 100,000 messages and the additional messages, roughly 90,000 at about 0.0014 dollars, add around 126 dollars, so call it about 280 dollars a month. Push to 500,000 messages and you are near 840 dollars. The base barely moves; the message volume is what drives the bill.
The lesson: model cost per conversation early, not after the invoice. A verbose conversational search reply, a chatty action that takes five turns where two would do, each multiplies across every user. Figures are the reported public rates and should be re-checked against current pricing.
Web chat first, the rest when traffic demands it
Here is the recommendation, and it is a real one. Build a handful of actions for your highest-volume tasks, turn on conversational search over the documents you already have indexed, embed it in web chat, and ship that. Do not open five channels, do not wire the agent, and do not set responses to verbose. That version answers most of what real users ask, costs the least to run, and gives you transcripts to learn from within a week.
Grow it from evidence, not from the architecture diagram. Add the Orchestrate agent when transcripts show requests that need real reasoning. Add voice when call-centre deflection justifies the extra tuning. Add verbose only where users ask for detail. Every one of those is a cost, in money, latency, and testing, so let the traffic tell you it is worth paying. Part 17 turns to model evaluation and benchmarking, how you prove one of these assistants is actually good before and after it ships. Before you read it, open your assistant preview, type the three questions your users ask most, and see which of the three mechanisms answers each.
References
- IBM Cloud Docs, watsonx Assistant conversational search
- IBM, watsonx Assistant and conversational search
- IBM, watsonx Assistant v2 API reference
- Cross-series, Microsoft Copilot Studio on Azure AI Foundry
About The Author
DrJha