The request came in a two-line email. List every AI model in production, name who signed off on each one, and state what each is permitted to do, then attach the evidence. It took the team three weeks of chasing spreadsheets, stale wiki pages, and half-remembered approvals to answer, and the answer still had holes. That is the exact job watsonx.governance removes, by turning governance from a scramble at audit time into a record that fills itself while you build. This part is about how that record gets made and how the monitors keep it honest once a model is live.
Key takeaways
watsonx.governance is one layer that tracks, evaluates, and monitors models from any vendor, not only watsonx.ai. It records the whole lifecycle in AI factsheets, automatic nutritional labels that log how a model was built, evaluated, and approved. The Risk Atlas gives you a catalog of known AI risks to attach to each use case, including agentic risks added through 2025. Production monitors watch drift, bias, quality, and toxic output against thresholds you set, and raise alerts when a metric crosses. Pre-built frameworks for the EU AI Act, ISO 42001, NIST AI RMF, and GDPR turn those records into the documentation an auditor asks for. Start with a use-case inventory, not a policy PDF.
What watsonx.governance is, in one layer
watsonx.governance is the governance component of the watsonx platform, sitting alongside watsonx.ai for building and watsonx.data for data, the split introduced in Part 1. It does three jobs. It manages the AI lifecycle, tracking every use case and model from request to retirement. It handles risk and compliance, mapping known risks to each use case, enforcing approval policy, and generating the documentation regulators expect. And it monitors models and agents in production, watching quality and fairness against thresholds. The workflow and reporting side inherits IBM OpenPages heritage, which is why the approval and audit machinery feels like a governance, risk, and compliance tool rather than a data science notebook.
The reach matters as much as the features. watsonx.governance evaluates and monitors models wherever they run, watsonx.ai as well as models hosted on Amazon Bedrock, Amazon SageMaker, Google Vertex AI, and Microsoft Azure. So a bank running Granite on watsonx and a fraud model on SageMaker gets one inventory, not two. Keep one distinction clear from the start. Evaluation, the offline scoring covered in Part 17, tells you whether a candidate is good enough. Governance is the system of record wrapped around that decision, before the model ships and long after, and an AI factsheet is where the two meet.
A factsheet is the automatically maintained record of a model, sometimes called a nutritional label because it collects, in one place, how the model was trained or which base model it uses, what data it saw, how it scored on evaluation, which risks it carries, and who approved it for which purpose. You do not write it by hand. The platform accumulates it as work happens, which is the whole point.
Objects you actually govern: use cases, models, agents
Governance in this tool hangs off three object types, and getting them straight saves a lot of confusion later. An AI Use Case is the business problem and its approved solution, a container that holds the candidate models, the risks you have identified, the approvals, and the documentation for one purpose such as automating a loan-eligibility summary. A Model entry is a single model or prompt template tracked in the inventory, with its factsheet attached. An AI Agent object type, added in 2025 for agentic applications, extends the same tracking to tool-using agents like the ones built in Part 15, because an agent that can call tools carries risks a single prompt does not.
These objects move through a lifecycle with role-based gates. Someone requests a use case, a data scientist develops candidate models against it, a validator reviews the evaluation evidence, an approver signs off for a defined purpose, the model deploys, and monitoring runs from that point on. Each transition writes to the factsheet, so the approval chain the auditor asked about is a byproduct of doing the work rather than a document someone reconstructs afterward.
flowchart LR
A[Use case requested] --> B[Model developed]
B --> C[Risks identified]
C --> D{Validate}
D -->|approve| E[Deploy]
D -->|reject| B
E --> F[Monitor in production]
F -->|drift or bias alert| D
F --> G[Factsheet updated]
Log a factsheet from your own pipeline
Factsheets fill themselves when you wire the facts client into training and evaluation. The ibm-aigov-facts-client Python SDK captures parameters, metrics, and the model asset from a run and attaches them to the model entry in the inventory. Here is the shape of initializing it against a deployment space and turning on autolog, so subsequent runs record themselves. Install with pip install ibm-aigov-facts-client.
# pip install ibm-aigov-facts-client
from ibm_aigov_facts_client import AIGovFactsClient
# point the client at a deployment space and open an experiment
facts_client = AIGovFactsClient(
api_key='YOUR_IBM_CLOUD_APIKEY',
experiment_name='support-rag-v3',
container_type='space',
container_id='YOUR_DEPLOYMENT_SPACE_ID',
set_as_current_experiment=True,
enable_autolog=True,
)
# from here, training and evaluation runs auto-log to the factsheet:
# parameters, metrics, and the model asset attach to the model entry
print('logging runs into:', facts_client.experiment_name)Expected output: logging runs into: support-rag-v3, and each later run shows up under the model factsheet in the Governance Console with its parameters and metrics recorded.
Failure mode: a 401 or an ApiRequestFailure means the API key or the container_id is wrong, not your model, so check the space id in the watsonx UI before you touch the code. For a model that runs outside watsonx, on Bedrock or Vertex, you register it as an external model with the external_model flag and save it into the same inventory, so its facts land next to the native ones. The exact method names track the installed SDK version, so pin it.
In practice
The clean provenance you kept during ingestion pays off right here. If your chunks carried page and coordinate metadata from the Docling front-end in Part 18, the factsheet can trace a production answer back to the source document without a reconstruction project. Governance is easy when the data underneath it was recorded properly, and miserable when it was not.
Which risks does the Risk Atlas track?
The Risk Atlas is IBM’s catalog of known AI risks, and it ships inside the Governance Console out of the box so you are not inventing a risk taxonomy from scratch. It covers the familiar model risks, accuracy and reliability, bias and fairness, data drift, data privacy and personal data leakage, and it covers generative-specific ones like hallucination, prompt injection, toxic or harmful output, and intellectual property exposure in training data. As of mid-2025 it also carries agentic AI risks, the failure modes that only appear once a model can act, such as goal misalignment, unsafe tool use, and cascading errors across a chain of calls.
You do not read the Atlas like a book. You attach its risks to your use cases and models through out-of-box risk identification assessments, which come in three flavors: an AI Use Case assessment, a Model Onboarding assessment, and a combined one. Answer the assessment and the relevant risks link automatically, each carrying suggested controls. The value is not the list itself. It is that every use case now names the risks it accepted, which is precisely the column an auditor scans first, and it maps onto the same categories that runtime tools like Granite Guardian in Part 10 enforce at inference time.
Monitoring drift, bias, quality, and safety in production
Approval is not the finish line. A model that passed evaluation in March can quietly degrade by June because the world it sees has shifted, and that shift is called drift, a change in the distribution of the inputs or outputs relative to what the model was validated on. watsonx.governance runs monitors that catch it. Drift monitoring includes embedding drift detection to flag behavioral shifts even when raw inputs look similar. Fairness monitoring uses disparate impact, the ratio of favorable outcomes between a monitored group and a reference group, to surface bias. Quality monitoring keeps scoring the generative metrics from Part 17, faithfulness and answer relevance among them, but now on live traffic. Model health tracks the operational picture, data volume, latency, and throughput. And safety monitors flag toxic language through HAP, hate, abuse, and profanity detection, and catch leaked identifiers through PII detection.
| Monitor | What it measures | Example threshold | What it catches |
|---|---|---|---|
| Drift | Input and embedding distribution shift | Drop over 10 percent | A model going stale on new data |
| Fairness | Disparate impact ratio | Below 0.8 | Biased outcomes across groups |
| Quality | Faithfulness on live traffic | Below 0.80 | Hallucination creeping in |
| Health | Latency and throughput | p95 over 2s | Cost and performance regressions |
| Safety | HAP and PII in output | Any hit | Toxic text or leaked identifiers |
Two 2026 additions are worth naming. Agent Monitoring and Insights extends production monitoring to agentic applications, scoring whole tool-using flows rather than single calls. And the Model Risk Evaluation Engine adds a faster path to risk-scoring a model against a policy. Both matter more as your apps become agents, but the mechanics below are the same: a metric, a threshold, an alert.
Worked example
A support RAG assistant passed validation at faithfulness 0.91. A quality monitor scored it weekly on live traffic: 0.91, 0.90, 0.88, 0.85, 0.83, 0.79, 0.75, 0.72 across eight weeks, as a new product line filled the knowledge base with documents the retriever handled poorly. The alert threshold was 0.80, breached in week 6 at 0.79. Because the monitor fired, the team caught a slow hallucination drift six weeks in instead of from an angry customer, traced it to stale retrieval, and re-indexed. These are illustrative numbers from one model, not a benchmark; the shape is the lesson, a gradual slide that no single spot check would have caught.
Mapping controls to the EU AI Act
The EU AI Act sorts systems by risk, unacceptable, high, limited, and minimal, and piles most obligations onto the high-risk tier: a risk management system, data governance, technical documentation, human oversight, record keeping, and demonstrated accuracy and robustness. Meeting those on your own means assembling evidence from a dozen places, which is the three-week scramble from the opening. watsonx.governance ships pre-configured frameworks for the EU AI Act, ISO 42001, the NIST AI Risk Management Framework, and GDPR, and it produces the technical documentation the Act requires, intended purpose statements, risk management records, data governance notes, and human oversight protocols, from the facts it already tracked.
| EU AI Act obligation | watsonx.governance control |
|---|---|
| Risk management system | Risk Atlas assessments linked to each use case |
| Technical documentation | Auto-generated factsheet and framework report |
| Record keeping and logging | Payload logging and monitoring datamart |
| Human oversight | Role-based approval gates in the lifecycle |
| Accuracy and robustness | Quality, drift, and fairness monitors with thresholds |
The payoff is measured in evidence-assembly time, which is where governance quietly costs a team the most. When each factsheet is complete and every approval is on record, producing a framework report is closer to an export than a project. The illustrative split below is what teams tend to report after wiring governance in properly: what once ate weeks of manual collation becomes hours of review. For how another platform frames the same responsible-AI reporting, the Azure Responsible AI dashboard and scorecard is worth a look from the Microsoft side.
Wire governance in at build time, not audit time
If you take one thing from this part, wire the facts client into your pipeline on day one and open an AI Use Case before you train, not the week the auditor emails. Do that and the record builds itself: every run logs to a factsheet, every risk you accept is named against a use case, every approval is a gated transition, and every production monitor keeps scoring after launch. The framework report for the EU AI Act or ISO 42001 then becomes an export rather than an archaeology dig. Retrofitting governance onto a model already in production is possible, but it is the expensive path, and it is the one that leaves holes.
Start this week by creating one AI Use Case for a model you already run and turning on a single quality monitor with payload logging. The first alert it raises, or the first factsheet it hands you on request, is the moment governance stops being paperwork and starts being infrastructure. The next part turns to cost governance and FinOps for watsonx, because monitoring and evaluation are themselves workloads with a bill attached.
References
- IBM watsonx.governance product overview
- Governing assets with watsonx.governance, IBM Documentation
- IBM, Model Risk Evaluation Engine for watsonx.governance
- Preparing for the EU AI Act, IBM Think
About The Author
DrJha