flowchart LR
  A[Seed examples in taxonomy] --> B[Teacher model generates data]
  B --> C{Critic filters samples}
  C -->|weak| B
  C -->|kept| D[Phase one, knowledge]
  D --> E[Phase two, skills]
  E --> F[Aligned Granite model]

Run it end to end with the ilab CLI

The open source path is a command line tool called ilab. You can run the whole loop on a laptop at low fidelity, which is the right way to learn it before you spend real compute. Here is the sequence, start to finish.

# 1. create the config and pick a student (base) model
ilab config init

# 2. add your leaf under the taxonomy, then validate it
#    a knowledge leaf needs a qna.yaml with 5 or more Q and A pairs
ilab taxonomy diff

# 3. generate synthetic data from your seed examples
#    a teacher model expands 5 seeds into thousands of samples
ilab data generate

# 4. train in phases on the generated data
ilab model train

# 5. serve the tuned model and sanity check it
ilab model serve
ilab model chat

Expected output: ilab taxonomy diff lists the qna.yaml files you changed and prints that the taxonomy is valid. ilab data generate writes a synthetic dataset to your local output directory. Failure mode to plan for: a qna.yaml with fewer than five pairs, or a stray space in a folder name, or broken indentation, makes ilab taxonomy diff mark the leaf invalid and ilab data generate refuse to run. Fix the leaf before you spend compute, because a generate run against a broken taxonomy fails late and wastes the cycle.

InstructLab on watsonx.ai versus your laptop

The laptop run teaches you the mechanics, but it does not give you a production model. Full scale SDG and phased training need real GPUs, a data pipeline, and a place to evaluate and track what you built. That is the case IBM makes for running InstructLab inside watsonx.ai rather than on your own hardware: the same method, with the compute attached and the governance the rest of the platform already gives you, data lineage, evaluation, and the factsheets that a regulated shop needs anyway. The trade is the usual one. The laptop is free and local and small. The platform costs capacity and gives you scale and a paper trail.

When InstructLab beats prompt tuning and RAG

Three tools solve three different problems, and mixing them up wastes money. Retrieval, which I covered in Part 9 on RAG, is for facts that change or are too many to train in. Prompt tuning, from Part 11, steers behavior the model can already almost do. InstructLab is for a skill the model genuinely lacks and cannot be steered into, or a body of knowledge you want baked into the weights rather than retrieved every call. If a better prompt or a retrieval step closes your gap, stop there. Both are cheaper to build and far cheaper to keep running.

Reach for InstructLab when the model keeps failing a task in a way no prompt fixes, and you can write a clean handful of examples of the task done right. The teacher and critic loop is the same teacher and student idea other platforms use for distillation, which my Google Cloud Series part on Vertex AI distillation walks through in Gemini terms. And the underlying lesson, that the data you feed a model matters more than the size of the model, is the one I argued in the vendor neutral Generative AI Series part on data versus model size.

My call on InstructLab in a governed shop

InstructLab is the right tool for a narrow, real problem: teaching a base model a skill or a body of knowledge it does not have, from examples your own experts can write in an afternoon. In a governed environment its taxonomy gives you something most tuning approaches do not, a readable record of exactly what you taught the model and why, which is the kind of artifact an auditor can follow. That alone is worth a lot in a regulated shop. What I would not do is treat it as the default answer to every gap. Most gaps are prompt or retrieval gaps wearing a training costume.

For your first run this week, pick one skill the model keeps getting wrong, write five clean examples in a single leaf, and run the open source ilab loop end to end on a laptop just to feel the shape of it. Do not aim for a production model yet. Aim to understand the flow before you spend platform compute on it. Once the seed data is real and the taxonomy is clean, the next question is where that data comes from and how you prepare it at scale, which is exactly where Part 13 on data prep and synthetic data with watsonx.data picks up.

References

• About the InstructLab Taxonomy, InstructLab documentation
• Democratizing LLM development with InstructLab support in watsonx.ai, IBM
• Aligning models for enterprise use cases with InstructLab in watsonx.ai, IBM
• LAB: Large-Scale Alignment for ChatBots, arXiv 2403.01081

Phased training and why the model does not forget

Now the generated data has to go into the model. A naive fine tune on a fresh batch of task data risks catastrophic forgetting, where the model learns the new task and quietly loses ground on things it used to do well. InstructLab handles that with phased training. Knowledge goes in first, in one phase. Skills go in second, in another. The run does not slam everything through at once.

Two details keep the phases stable. A replay buffer mixes in older data alongside the new, so each phase reminds the model of what it already knew while it learns. And a checkpoint is saved at each stage, so you can evaluate and pick the best one rather than trusting the final weights blindly. The result is a model that gains the skill you taught it and keeps its general ability. The flow below is the shape of a full run.

flowchart LR
  A[Seed examples in taxonomy] --> B[Teacher model generates data]
  B --> C{Critic filters samples}
  C -->|weak| B
  C -->|kept| D[Phase one, knowledge]
  D --> E[Phase two, skills]
  E --> F[Aligned Granite model]

Run it end to end with the ilab CLI

The open source path is a command line tool called ilab. You can run the whole loop on a laptop at low fidelity, which is the right way to learn it before you spend real compute. Here is the sequence, start to finish.

# 1. create the config and pick a student (base) model
ilab config init

# 2. add your leaf under the taxonomy, then validate it
#    a knowledge leaf needs a qna.yaml with 5 or more Q and A pairs
ilab taxonomy diff

# 3. generate synthetic data from your seed examples
#    a teacher model expands 5 seeds into thousands of samples
ilab data generate

# 4. train in phases on the generated data
ilab model train

# 5. serve the tuned model and sanity check it
ilab model serve
ilab model chat

Expected output: ilab taxonomy diff lists the qna.yaml files you changed and prints that the taxonomy is valid. ilab data generate writes a synthetic dataset to your local output directory. Failure mode to plan for: a qna.yaml with fewer than five pairs, or a stray space in a folder name, or broken indentation, makes ilab taxonomy diff mark the leaf invalid and ilab data generate refuse to run. Fix the leaf before you spend compute, because a generate run against a broken taxonomy fails late and wastes the cycle.

InstructLab on watsonx.ai versus your laptop

The laptop run teaches you the mechanics, but it does not give you a production model. Full scale SDG and phased training need real GPUs, a data pipeline, and a place to evaluate and track what you built. That is the case IBM makes for running InstructLab inside watsonx.ai rather than on your own hardware: the same method, with the compute attached and the governance the rest of the platform already gives you, data lineage, evaluation, and the factsheets that a regulated shop needs anyway. The trade is the usual one. The laptop is free and local and small. The platform costs capacity and gives you scale and a paper trail.

When InstructLab beats prompt tuning and RAG

Three tools solve three different problems, and mixing them up wastes money. Retrieval, which I covered in Part 9 on RAG, is for facts that change or are too many to train in. Prompt tuning, from Part 11, steers behavior the model can already almost do. InstructLab is for a skill the model genuinely lacks and cannot be steered into, or a body of knowledge you want baked into the weights rather than retrieved every call. If a better prompt or a retrieval step closes your gap, stop there. Both are cheaper to build and far cheaper to keep running.

Reach for InstructLab when the model keeps failing a task in a way no prompt fixes, and you can write a clean handful of examples of the task done right. The teacher and critic loop is the same teacher and student idea other platforms use for distillation, which my Google Cloud Series part on Vertex AI distillation walks through in Gemini terms. And the underlying lesson, that the data you feed a model matters more than the size of the model, is the one I argued in the vendor neutral Generative AI Series part on data versus model size.

My call on InstructLab in a governed shop

InstructLab is the right tool for a narrow, real problem: teaching a base model a skill or a body of knowledge it does not have, from examples your own experts can write in an afternoon. In a governed environment its taxonomy gives you something most tuning approaches do not, a readable record of exactly what you taught the model and why, which is the kind of artifact an auditor can follow. That alone is worth a lot in a regulated shop. What I would not do is treat it as the default answer to every gap. Most gaps are prompt or retrieval gaps wearing a training costume.

For your first run this week, pick one skill the model keeps getting wrong, write five clean examples in a single leaf, and run the open source ilab loop end to end on a laptop just to feel the shape of it. Do not aim for a production model yet. Aim to understand the flow before you spend platform compute on it. Once the seed data is real and the taxonomy is clean, the next question is where that data comes from and how you prepare it at scale, which is exactly where Part 13 on data prep and synthetic data with watsonx.data picks up.

References

• About the InstructLab Taxonomy, InstructLab documentation
• Democratizing LLM development with InstructLab support in watsonx.ai, IBM
• Aligning models for enterprise use cases with InstructLab in watsonx.ai, IBM
• LAB: Large-Scale Alignment for ChatBots, arXiv 2403.01081

flowchart TD
  R[Root taxonomy] --> K[knowledge directory]
  R --> C[compositional skills directory]
  K --> KL[Leaf, qna dot yaml]
  C --> CL[Leaf, qna dot yaml]

Knowledge and skills are handled differently, and the distinction decides where a leaf goes and what it needs inside. Knowledge is grounded in a source document you supply, so the generator can tie its output to real text. Skills split again: a grounded skill comes with the context it needs in each example, while an ungrounded skill is a pure instruction the model should follow with no attachment. The table sums up the three shapes you will actually create.

From five examples to thousands

This is the step that makes the whole method worth the trouble. You wrote five examples in a leaf. Synthetic data generation, the process InstructLab abbreviates as SDG, reads them and produces orders of magnitude more. A teacher model, a strong general model, generates candidate instructions and answers that follow the pattern of your seeds. For a knowledge leaf, it grounds each candidate in chunks of your source document so the output stays tied to real text instead of drifting into invention.

Left there, you would drown in mediocre samples. So a second model, the critic, rates each generated pair and filters the weak ones out before any of it reaches training. That teacher plus critic loop is the quality gate. It is why five careful seed examples can become a few thousand usable ones without a human reading each. The chart below puts the jump on a log scale, because on a linear axis the seed bar would be invisible next to the generated one.

Phased training and why the model does not forget

Now the generated data has to go into the model. A naive fine tune on a fresh batch of task data risks catastrophic forgetting, where the model learns the new task and quietly loses ground on things it used to do well. InstructLab handles that with phased training. Knowledge goes in first, in one phase. Skills go in second, in another. The run does not slam everything through at once.

Two details keep the phases stable. A replay buffer mixes in older data alongside the new, so each phase reminds the model of what it already knew while it learns. And a checkpoint is saved at each stage, so you can evaluate and pick the best one rather than trusting the final weights blindly. The result is a model that gains the skill you taught it and keeps its general ability. The flow below is the shape of a full run.

flowchart LR
  A[Seed examples in taxonomy] --> B[Teacher model generates data]
  B --> C{Critic filters samples}
  C -->|weak| B
  C -->|kept| D[Phase one, knowledge]
  D --> E[Phase two, skills]
  E --> F[Aligned Granite model]

Run it end to end with the ilab CLI

The open source path is a command line tool called ilab. You can run the whole loop on a laptop at low fidelity, which is the right way to learn it before you spend real compute. Here is the sequence, start to finish.

# 1. create the config and pick a student (base) model
ilab config init

# 2. add your leaf under the taxonomy, then validate it
#    a knowledge leaf needs a qna.yaml with 5 or more Q and A pairs
ilab taxonomy diff

# 3. generate synthetic data from your seed examples
#    a teacher model expands 5 seeds into thousands of samples
ilab data generate

# 4. train in phases on the generated data
ilab model train

# 5. serve the tuned model and sanity check it
ilab model serve
ilab model chat

Expected output: ilab taxonomy diff lists the qna.yaml files you changed and prints that the taxonomy is valid. ilab data generate writes a synthetic dataset to your local output directory. Failure mode to plan for: a qna.yaml with fewer than five pairs, or a stray space in a folder name, or broken indentation, makes ilab taxonomy diff mark the leaf invalid and ilab data generate refuse to run. Fix the leaf before you spend compute, because a generate run against a broken taxonomy fails late and wastes the cycle.

InstructLab on watsonx.ai versus your laptop

The laptop run teaches you the mechanics, but it does not give you a production model. Full scale SDG and phased training need real GPUs, a data pipeline, and a place to evaluate and track what you built. That is the case IBM makes for running InstructLab inside watsonx.ai rather than on your own hardware: the same method, with the compute attached and the governance the rest of the platform already gives you, data lineage, evaluation, and the factsheets that a regulated shop needs anyway. The trade is the usual one. The laptop is free and local and small. The platform costs capacity and gives you scale and a paper trail.

When InstructLab beats prompt tuning and RAG

Three tools solve three different problems, and mixing them up wastes money. Retrieval, which I covered in Part 9 on RAG, is for facts that change or are too many to train in. Prompt tuning, from Part 11, steers behavior the model can already almost do. InstructLab is for a skill the model genuinely lacks and cannot be steered into, or a body of knowledge you want baked into the weights rather than retrieved every call. If a better prompt or a retrieval step closes your gap, stop there. Both are cheaper to build and far cheaper to keep running.

Reach for InstructLab when the model keeps failing a task in a way no prompt fixes, and you can write a clean handful of examples of the task done right. The teacher and critic loop is the same teacher and student idea other platforms use for distillation, which my Google Cloud Series part on Vertex AI distillation walks through in Gemini terms. And the underlying lesson, that the data you feed a model matters more than the size of the model, is the one I argued in the vendor neutral Generative AI Series part on data versus model size.

My call on InstructLab in a governed shop

InstructLab is the right tool for a narrow, real problem: teaching a base model a skill or a body of knowledge it does not have, from examples your own experts can write in an afternoon. In a governed environment its taxonomy gives you something most tuning approaches do not, a readable record of exactly what you taught the model and why, which is the kind of artifact an auditor can follow. That alone is worth a lot in a regulated shop. What I would not do is treat it as the default answer to every gap. Most gaps are prompt or retrieval gaps wearing a training costume.

For your first run this week, pick one skill the model keeps getting wrong, write five clean examples in a single leaf, and run the open source ilab loop end to end on a laptop just to feel the shape of it. Do not aim for a production model yet. Aim to understand the flow before you spend platform compute on it. Once the seed data is real and the taxonomy is clean, the next question is where that data comes from and how you prepare it at scale, which is exactly where Part 13 on data prep and synthetic data with watsonx.data picks up.

References

• About the InstructLab Taxonomy, InstructLab documentation
• Democratizing LLM development with InstructLab support in watsonx.ai, IBM
• Aligning models for enterprise use cases with InstructLab in watsonx.ai, IBM
• LAB: Large-Scale Alignment for ChatBots, arXiv 2403.01081

Prompt tuning could not fix it. The model kept mislabeling a class of insurance claims that my customer’s own adjusters recognized on sight, because the distinction lived in a decade of internal practice that no public model had ever read. That is the wall prompt tuning hits. It steers what the model already knows, and it cannot teach a skill the base model never learned. InstructLab is built for that gap. It takes a handful of examples you write by hand, multiplies them into thousands of synthetic training samples, and trains the model in phases so it picks up the new skill without dropping the old ones. This part is how that works, and when it earns its cost.

What InstructLab actually does

InstructLab is a project from IBM Research and Red Hat that builds on a technique called LAB, short for Large-Scale Alignment for ChatBots. The problem it solves is old and expensive. Teaching a model a new skill usually means writing thousands of high quality instruction examples by hand, which almost nobody has the time or budget to do well. LAB flips that around. You write a few examples, five or so per skill, and a larger teacher model reads them and generates the thousands of training samples you would otherwise have written yourself.

Three pieces make that safe rather than reckless. First, a taxonomy, a tree of folders that organizes what you are teaching so the generator knows how each example relates to the others. Second, synthetic data generation driven by a teacher model and checked by a critic model that rates each generated sample and drops the ones that do not hold up. Third, phased training, which folds the kept data into the base model in stages instead of one blunt pass, so the model gains the new behavior without overwriting what it already did well. Take those three in order.

Why a taxonomy instead of a pile of examples

A taxonomy is a cascading directory tree. The top folder is the root. Below it, InstructLab expects two branches: a knowledge directory for facts the model should be able to recall, and a compositional_skills directory for tasks the model should be able to perform. Each branch narrows through subfolders until it reaches a leaf node, and the leaf holds a single file named qna.yaml with your example question and answer pairs. That file is the only thing InstructLab strictly requires in a leaf.

The tree is not bureaucracy. The synthetic data generation process uses the structure to relate chunks of knowledge to each other, so a leaf about one type of insurance claim does not bleed into a leaf about another. The rule from the InstructLab docs is one skill or one piece of knowledge per leaf, never several stuffed into one file. Folder names use underscores, not spaces. And every qna.yaml needs a minimum of five question and answer pairs, because the generator needs enough of a pattern to imitate before it can expand it.

flowchart TD
  R[Root taxonomy] --> K[knowledge directory]
  R --> C[compositional skills directory]
  K --> KL[Leaf, qna dot yaml]
  C --> CL[Leaf, qna dot yaml]

Knowledge and skills are handled differently, and the distinction decides where a leaf goes and what it needs inside. Knowledge is grounded in a source document you supply, so the generator can tie its output to real text. Skills split again: a grounded skill comes with the context it needs in each example, while an ungrounded skill is a pure instruction the model should follow with no attachment. The table sums up the three shapes you will actually create.

From five examples to thousands

This is the step that makes the whole method worth the trouble. You wrote five examples in a leaf. Synthetic data generation, the process InstructLab abbreviates as SDG, reads them and produces orders of magnitude more. A teacher model, a strong general model, generates candidate instructions and answers that follow the pattern of your seeds. For a knowledge leaf, it grounds each candidate in chunks of your source document so the output stays tied to real text instead of drifting into invention.

Left there, you would drown in mediocre samples. So a second model, the critic, rates each generated pair and filters the weak ones out before any of it reaches training. That teacher plus critic loop is the quality gate. It is why five careful seed examples can become a few thousand usable ones without a human reading each. The chart below puts the jump on a log scale, because on a linear axis the seed bar would be invisible next to the generated one.

Phased training and why the model does not forget

Now the generated data has to go into the model. A naive fine tune on a fresh batch of task data risks catastrophic forgetting, where the model learns the new task and quietly loses ground on things it used to do well. InstructLab handles that with phased training. Knowledge goes in first, in one phase. Skills go in second, in another. The run does not slam everything through at once.

Two details keep the phases stable. A replay buffer mixes in older data alongside the new, so each phase reminds the model of what it already knew while it learns. And a checkpoint is saved at each stage, so you can evaluate and pick the best one rather than trusting the final weights blindly. The result is a model that gains the skill you taught it and keeps its general ability. The flow below is the shape of a full run.

flowchart LR
  A[Seed examples in taxonomy] --> B[Teacher model generates data]
  B --> C{Critic filters samples}
  C -->|weak| B
  C -->|kept| D[Phase one, knowledge]
  D --> E[Phase two, skills]
  E --> F[Aligned Granite model]

Run it end to end with the ilab CLI

The open source path is a command line tool called ilab. You can run the whole loop on a laptop at low fidelity, which is the right way to learn it before you spend real compute. Here is the sequence, start to finish.

# 1. create the config and pick a student (base) model
ilab config init

# 2. add your leaf under the taxonomy, then validate it
#    a knowledge leaf needs a qna.yaml with 5 or more Q and A pairs
ilab taxonomy diff

# 3. generate synthetic data from your seed examples
#    a teacher model expands 5 seeds into thousands of samples
ilab data generate

# 4. train in phases on the generated data
ilab model train

# 5. serve the tuned model and sanity check it
ilab model serve
ilab model chat

Expected output: ilab taxonomy diff lists the qna.yaml files you changed and prints that the taxonomy is valid. ilab data generate writes a synthetic dataset to your local output directory. Failure mode to plan for: a qna.yaml with fewer than five pairs, or a stray space in a folder name, or broken indentation, makes ilab taxonomy diff mark the leaf invalid and ilab data generate refuse to run. Fix the leaf before you spend compute, because a generate run against a broken taxonomy fails late and wastes the cycle.

InstructLab on watsonx.ai versus your laptop

The laptop run teaches you the mechanics, but it does not give you a production model. Full scale SDG and phased training need real GPUs, a data pipeline, and a place to evaluate and track what you built. That is the case IBM makes for running InstructLab inside watsonx.ai rather than on your own hardware: the same method, with the compute attached and the governance the rest of the platform already gives you, data lineage, evaluation, and the factsheets that a regulated shop needs anyway. The trade is the usual one. The laptop is free and local and small. The platform costs capacity and gives you scale and a paper trail.

When InstructLab beats prompt tuning and RAG

Three tools solve three different problems, and mixing them up wastes money. Retrieval, which I covered in Part 9 on RAG, is for facts that change or are too many to train in. Prompt tuning, from Part 11, steers behavior the model can already almost do. InstructLab is for a skill the model genuinely lacks and cannot be steered into, or a body of knowledge you want baked into the weights rather than retrieved every call. If a better prompt or a retrieval step closes your gap, stop there. Both are cheaper to build and far cheaper to keep running.

Reach for InstructLab when the model keeps failing a task in a way no prompt fixes, and you can write a clean handful of examples of the task done right. The teacher and critic loop is the same teacher and student idea other platforms use for distillation, which my Google Cloud Series part on Vertex AI distillation walks through in Gemini terms. And the underlying lesson, that the data you feed a model matters more than the size of the model, is the one I argued in the vendor neutral Generative AI Series part on data versus model size.

My call on InstructLab in a governed shop

InstructLab is the right tool for a narrow, real problem: teaching a base model a skill or a body of knowledge it does not have, from examples your own experts can write in an afternoon. In a governed environment its taxonomy gives you something most tuning approaches do not, a readable record of exactly what you taught the model and why, which is the kind of artifact an auditor can follow. That alone is worth a lot in a regulated shop. What I would not do is treat it as the default answer to every gap. Most gaps are prompt or retrieval gaps wearing a training costume.

For your first run this week, pick one skill the model keeps getting wrong, write five clean examples in a single leaf, and run the open source ilab loop end to end on a laptop just to feel the shape of it. Do not aim for a production model yet. Aim to understand the flow before you spend platform compute on it. Once the seed data is real and the taxonomy is clean, the next question is where that data comes from and how you prepare it at scale, which is exactly where Part 13 on data prep and synthetic data with watsonx.data picks up.

References

• About the InstructLab Taxonomy, InstructLab documentation
• Democratizing LLM development with InstructLab support in watsonx.ai, IBM
• Aligning models for enterprise use cases with InstructLab in watsonx.ai, IBM
• LAB: Large-Scale Alignment for ChatBots, arXiv 2403.01081

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts