The first prompt I ran in watsonx.ai came back as a wall of text when I wanted a single word. Nothing was broken. I had left the output length cap high, the decoding on sampling, and no stop sequence, so the model kept talking. Prompt Lab is where you catch that in a minute instead of in production, and this part is about living in it.

In Part 1 I mapped the three watsonx products at the platform level. Now I open the first box. This is the hands on part, so have an account ready and follow along.

What the studio actually is

The watsonx.ai studio is the web workbench you log into. It is a browser front end over the watsonx.ai service, and it is where every hands on task in this series happens: building prompts, running notebooks, tuning models, and promoting them to endpoints. When people say they are in watsonx.ai, they usually mean they are in this studio.

One thing trips up almost everyone on day one. The studio on its own does not run models. It needs a compute service behind it, and on IBM Cloud that service is called the watsonx.ai Runtime, which used to carry the name Watson Machine Learning. You provision that service once, then attach it to your workspace. Skip that step and every prompt you submit fails with an error about a missing associated service, which is confusing because the interface looks ready to go.

So the studio is the room, and the Runtime service is the power supply. You wire them together through a project, which is the next thing to understand.

Projects, and why nothing works until you make one

A project is the container for everything you do in the studio. Prompts, notebooks, data assets, and connections all live inside a project, and access is granted per project. Think of it as a folder with permissions and an attached engine. You create one from the studio home, give it a name, and associate the watsonx.ai Runtime service with it. That association is the step that turns the folder from a static bucket into something that can actually call a model.

There is a second container you meet later, the deployment space. A project is where you build. A deployment space is where you promote a finished asset to run as a served endpoint, with its own access control so a prototype cannot accidentally take production traffic. You do not need a deployment space to use Prompt Lab, but you need one the moment you want an application to call your prompt. I cover promotion at the end of this part; the deep serving material is in a later part.

Here is how the pieces nest, from the studio down to a served endpoint.

Inside Prompt Lab

Prompt Lab is the visual workspace for engineering prompts. You reach it from inside a project: open the Assets tab, click New asset, and pick the tile for chatting and building prompts with foundation models. The screen has three regions. The centre is where you type the prompt and read the response. A right hand panel picks the model and sets generation parameters. A prompt variables area lets you insert named placeholders so one prompt can run against many inputs.

Model choice sits at the top of the panel. Click to view all foundation models, filter by task, and read the card for context window, cost tier, and provider before you commit. Granite is the default family for the reasons I gave in Part 1, but the same Prompt Lab drives the third party and open models in the catalog, so nothing here locks you to one vendor.

A word on modes, because the old guidance is now wrong. Prompt Lab used to offer three modes: Chat, Structured, and Freeform. IBM deprecated Structured and Freeform and removed them on 2 April 2026. Chat mode is the interface you get today. If you are following an older tutorial that tells you to switch to Freeform for raw prompt control, that button is gone. The capability did not vanish, it moved into Chat mode and the API, which is where you keep full control over the prompt text anyway.

How decoding parameters change the output

The right hand panel is where a prompt goes from almost right to right. The first choice is the decoding method. Greedy decoding picks the single highest probability token at each step, so the same prompt gives the same answer every time. Sampling decoding draws from the probability distribution, so you get variety, controlled by temperature and the top settings. My rule is simple: greedy for anything you extract or classify, sampling for anything you draft or brainstorm. If two people run your extraction prompt and get different answers, you almost certainly left it on sampling.

The rest of the panel shapes length and repetition. Max new tokens caps how long the answer can get, which is your main lever on both cost and latency. Stop sequences end generation as soon as the model emits a string you name, which is how you stop a one word classifier from writing a paragraph. Repetition penalty pushes the model off loops. The table lists what each control does and when I actually touch it.

My back to basics advice: change one parameter at a time and reread the output. It is tempting to drop temperature, lower top P, and add a stop sequence in one go, then you cannot tell which one fixed it. Prompt engineering here is the same discipline covered in the vendor neutral prompt engineering guide, just with IBM controls in front of you.

Prompt variables and the guardrails toggle

Two features in the panel do more work than their size suggests. The first is prompt variables. Instead of hardcoding the ticket text into the prompt, you define a named placeholder, then feed values into it. That turns a single prompt into a reusable template you can test across a batch of inputs without retyping the instruction each time, and it is the same variable that later becomes the request payload when you call the saved template from code. If you find yourself pasting a new example over the old one to test, you want a variable there instead.

The second is the AI guardrails toggle. Switched on, it screens both the prompt you send and the text the model returns, and it flags or blocks content that trips a harm filter, replacing flagged spans so they do not reach the user. It is a blunt first layer, not the full governed control set that watsonx.governance gives you, and it is off by default in some setups. Turn it on for anything user facing while you prototype, so you see early how often your prompts trip it. The deeper Granite Guardian models and policy controls come later in the series; the toggle is the one you can flip today.

Neither feature changes your token math, but both change how safely and how quickly you can iterate. I turn guardrails on and wire a variable before I share a Prompt Lab session with anyone else on the team.

From Prompt Lab to code

A prompt that only lives in the browser is a demo. To make it an asset you can version and call, save it. Prompt Lab lets you save the current prompt and its parameters as a prompt template, a first class asset in the project, or export the setup as a notebook that reproduces the call in Python. The parameter names in the panel map straight to the SDK, so once you have a template you like, moving to code is mechanical rather than a rewrite.

The lifecycle below is the path every prompt takes, from the first keystroke in Chat mode to an endpoint an application calls, with the governance factsheet created when you promote.

flowchart LR
  A[Open project] --> B[Prompt Lab Chat mode]
  B --> C[Tune model and params]
  C --> D[Save as prompt template]
  D --> E[Promote to deployment space]
  E --> F[Online endpoint]
  F --> G[App via REST or SDK]
  E --> H[Governance factsheet]

Here is the same prompt you would build in Chat mode, expressed in the ibm-watsonx-ai Python SDK. It authenticates, picks a Granite model, sets the parameters you would set in the panel, and sends a system plus user message. Note that greedy decoding is expressed as temperature 0 on the chat call.

from ibm_watsonx_ai import Credentials
from ibm_watsonx_ai.foundation_models import ModelInference

creds = Credentials(
    url='https://us-south.ml.cloud.ibm.com',
    api_key='YOUR_IBM_CLOUD_API_KEY',
)

model = ModelInference(
    model_id='ibm/granite-4-1-8b-instruct',   # confirm the live id under View all foundation models
    credentials=creds,
    project_id='YOUR_PROJECT_ID',
    params={'temperature': 0, 'max_tokens': 40},
)

messages = [
    {'role': 'system', 'content': 'Classify the ticket as billing, bug, or other. Answer with one word.'},
    {'role': 'user', 'content': 'I was charged twice for my subscription this month.'},
]

resp = model.chat(messages=messages)
print(resp['choices'][0]['message']['content'])

Expected output: the single word billing. Failure modes: a 404 model_not_supported means the model_id or region is wrong; a 401 means the api_key or project_id do not match; a full sentence instead of one word means the instruction was ignored or temperature is too high, so tighten the system message or add a stop sequence. The model_id above follows the Granite 4.1 naming pattern, so verify the exact id in your region before running.

What a prompt costs, and where the tokens go

Prompt Lab hides a cost decision that most people do not see until the bill arrives. As covered in Part 1, watsonx.ai meters inference in Resource Units, where one Resource Unit is 1,000 tokens counted across input and output together. The input side is the part you control in Prompt Lab, and it is usually the larger half. A long instruction block that you paste into every call is charged on every call. Trim it once and you cut the cost of every future request.

The reason trimming input pays off so well is that for most enterprise tasks the input dwarfs the output. A classifier reads a long instruction and a ticket, then answers in one word. A summarizer reads a long document and returns a short summary. Only open ended chat pushes real length onto the output side. The next chart shows that split for three common task types, and it is why I profile the input first when a token bill looks high.

Prototype in Chat mode, then freeze it in the SDK

Here is how I would actually work in the studio. Create one project, associate the Runtime service, and open Prompt Lab in Chat mode. Get the prompt right there, changing one parameter at a time, on greedy decoding if the task is extraction or classification. Watch the token count while you do it, and trim the input the moment the boilerplate outweighs the real content. Only once the output is stable do you save it as a template and move to the SDK, where the same parameters live and the behaviour no longer drifts because a teammate nudged a slider.

The studio is built to make that loop fast, and Prompt Lab is the part of it you will spend the most time in early on. Open a project now, run one Granite prompt in Chat mode, and check the token count on your first response. In the next part I go deep on the Granite family itself, the model ids, the licensing, and how to choose a size, so you know exactly which model to load into the panel you just learned.

References

IBM docs: Prompt Lab
IBM: Chat with watsonx models
ibm-watsonx-ai SDK: ModelInference
watsonx.ai foundation models
Companion: prompt engineering that works

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts