Five hand written question and answer pairs went into a file. A few hours later the data generator had turned them into more than a thousand training examples, and not one of them came from a proprietary model like GPT-4. That ratio, roughly five inputs to a thousand outputs, is the whole reason InstructLab exists, and it is the number that reframes how you think about tuning a model on your own documents.

Synthetic alignment instead of hand labelling

Aligning a model to new material used to mean paying people to write thousands of training examples by hand, or renting a frontier model to write them for you and hoping the licence let you keep the output. Both are slow and expensive, and the second one quietly ties your private model to somebody else's terms of service. LAB, the method described in the 2024 paper Large-scale Alignment for chatBots, replaces that grind with a taxonomy driven generator. You supply a small number of grounded examples, a teacher model expands them into a large, varied dataset, and a quality filter throws out the weak ones before any tuning happens.

Two words there deserve a plain definition, because the rest of the part leans on them. A teacher model is a larger, capable model that writes the synthetic training data. A student model is the smaller model you are actually tuning to ship, in our case a Granite 3.1 8B model, meaning eight billion parameters. The trick of LAB is that the teacher does not need to be a proprietary giant. A well aligned open model does the job, which is what lets the whole pipeline stay inside your own network, the exact property Part 4 spent its length protecting.

A word on what is upstream here and what Red Hat adds, because the honesty matters for a series about one vendor. InstructLab is an open source project, and the document ingestion that turns your PDFs and Markdown into clean chunks now runs on Docling, another open source library. Red Hat AI 3 packages these as modular Python libraries, one each for ingestion, synthetic data generation, tuning and evaluation. What you pay Red Hat for is the hardening, the support, and a chain tested against a supported Granite release rather than assembled by hand. You can run the upstream tools yourself for free; the RHEL AI bundle is the version somebody answers the phone about at two in the morning.

This is a different lever from retrieval, and it is worth being precise about which one you are pulling. Retrieval, covered in fine tuning vs RAG vs prompting, hands the model documents at question time. Alignment with LAB bakes behaviour and facts into the weights before the model ever serves a request. Later parts of this series use both together. For now, hold the distinction: taxonomy tuning changes what the model is, retrieval changes what it can see.

Taxonomy as a map of model knowledge

The taxonomy is a directory tree, nothing more exotic than folders on disk. Each path down the tree names a topic, and the folder at the end of a branch, the leaf node, holds one qna.yaml file that teaches one narrow thing. InstructLab reads the whole tree, works out which leaves are new or changed since the base model, and generates data only for those. That is why the tree shape matters: it is both how you organise contributions and how the generator decides what work to do.

Two branches carry most of the weight. A knowledge leaf teaches facts drawn from a source document, a product manual or a changelog, and it must point at that document so the generator can ground its examples. A skill leaf, which lives under compositional_skills, teaches a way of doing something, such as formatting a reply or extracting fields from a ticket, and it may not need a source document at all. Keep them straight, because they validate against different rules and tune in different phases.

Anatomy of a qna.yaml knowledge file

A knowledge qna.yaml is a small, strict file. It carries a schema version, a domain label, a creator, and an array of seed examples. Each seed example pairs a context, a short chunk copied straight from the source document, with at least three question and answer pairs grounded in that chunk. At the bottom sits a document block that pins the exact source by git repository and commit. Get any of the counts wrong and the validator refuses the file, which is a mercy, because the alternative is discovering the mistake three hours into a generation run.

Skill leaves differ in two ways worth knowing now. A skill qna.yaml drops the document block, because a skill teaches a behaviour rather than a fact, and a grounded skill instead carries its context inline in each example. Knowledge leaves also want an attribution.txt beside the qna.yaml naming the source, its licence and a link, which is mandatory for an upstream contribution and good hygiene even when the taxonomy stays private. Sorting knowledge from skills before you write a line saves a rewrite later, since the two validate and tune along separate paths.

Here is a real knowledge leaf for our support assistant, grounded in a public document so you can follow along. I am using the InstructLab sample taxonomy layout and pointing the document block at a public GitHub repository of Markdown docs.

# knowledge/support/product_docs/qna.yaml
# schema version 3, InstructLab bundled with RHEL AI 3.5
version: 3
domain: product_support
created_by: pranay
seed_examples:
  - context: |
      Invoices export as CSV or PDF from the Billing tab. CSV export
      caps at 10,000 rows per file; larger accounts are split into
      multiple files named invoices_part_01.csv and upward.
    questions_and_answers:
      - question: |
          What formats can invoices be exported in?
        answer: |
          Invoices export as either CSV or PDF from the Billing tab.
      - question: |
          Is there a row limit on CSV export?
        answer: |
          Yes. CSV export caps at 10,000 rows per file, and larger
          accounts are split into multiple numbered files.
      - question: |
          How are split invoice files named?
        answer: |
          They are named invoices_part_01.csv and upward in sequence.
  # ... four more seed_examples, each a different context chunk ...
document_outline: |
  How billing export works, formats, limits and file naming.
document:
  repo: https://github.com/instructlab/taxonomy.git
  commit: 0a1f2672b9b90582e6115333e3ed62fd628f1c0f
  patterns:
    - billing_export.md

Knowledge tuning and skill tuning

LAB runs alignment in two phases, and the order is deliberate. First comes knowledge tuning, where the model absorbs the facts generated from your knowledge leaves. Then comes skill tuning, where it learns the behaviours from your skill leaves on top of that knowledge. Doing knowledge first means the skills have facts to stand on; a model that has learned to format a billing answer but does not know the row limit is a confident liar. This staged shape is why InstructLab is a tuning method and not just a data generator.

Neither phase is quick or free. On a single accelerator, generating data for a handful of leaves and running both tuning phases is an overnight job measured in hours, not minutes, and it holds the GPU the whole time. That cost is why the cheap validation step earns its keep: every schema error you catch with a two second diff is hours of GPU time you do not waste discovering the hard way.

You do not run these phases by hand in this part. Generation lives in a later Part of this series and tuning in the one after, both on the single RHEL AI box we placed last time. What matters now is that the shape of your taxonomy is what feeds both phases, so time spent on the tree is time saved on the far more expensive tuning step. Evaluating whether the tuning actually worked is its own discipline, and the leakage traps in model evaluation and data leakage apply here exactly as they do to any supervised model.

Teaching the support assistant its own docs

Last part we placed the running project, an internal support assistant, on a single on premises RHEL AI server so its customer data never leaves the building. This part gives that server its first taxonomy. The workflow to author and validate one is three commands, and only the last of them touches a GPU, which is why validation is cheap and worth doing constantly.

# tested with the ilab CLI bundled in RHEL AI 3.5
# 1. create a working config and the sample taxonomy tree
$ ilab config init
Generating config file and taxonomy at:
  /home/pranay/.config/instructlab/config.yaml
  /home/pranay/.local/share/instructlab/taxonomy
Cloning taxonomy from github.com/instructlab/taxonomy ... done

# 2. drop your leaf in, then check what changed and whether it is valid
$ ilab taxonomy diff
knowledge/support/product_docs/qna.yaml
Taxonomy in /home/pranay/.local/share/instructlab/taxonomy is valid :)

That ilab taxonomy diff step is the one to lean on. It behaves like git diff for your tree, listing only the leaves you have added or changed, and it runs the schema validation that the generator would otherwise run three hours later. On a clean tree it prints the valid banner. On a broken one it tells you exactly which file and which rule failed, which brings us to the part everyone hits.

Where taxonomy authoring goes wrong

The single most common failure is writing fewer than five seed examples, usually because the source chunk felt small and three seemed enough. The schema requires at least five, and the validator is blunt about it. Here is the actual error, not a paraphrase of it.

$ ilab taxonomy diff
knowledge/support/product_docs/qna.yaml
Taxonomy in /home/pranay/.local/share/instructlab/taxonomy is not valid :(

knowledge/support/product_docs/qna.yaml:
  Value error, seed_examples: List should have at least 5 items
  after validation, not 3 [type=too_short]

Reading taxonomy failed with 1 error, please fix and try again

The fix is not to pad. If you copy your three examples and tweak a word to reach five, the file validates and the model still suffers, because the teacher generates variations of whatever you seed. Five near identical contexts produce a thousand near identical training rows, and the tuned model overfits one narrow fact while learning nothing around it. That is the trap the documentation does not warn you about loudly enough: the schema checks the count, never the diversity, and diversity is the thing that actually matters.

A quieter failure is a context that runs past the five hundred token ceiling, usually from pasting a whole section instead of a tight chunk. It gets rejected the same way the count error does, and the fix is to split the passage into two seed examples with two shorter contexts, which improves diversity as a happy side effect. Both failures share one root cause: treating the qna.yaml as a place to dump text rather than a place to show the teacher a pattern worth imitating.

How I would structure your first taxonomy

Start narrow and deep, not wide and shallow. For the support assistant I would write one knowledge leaf per real document, billing export, refund policy, one changelog, each with five to eight seed examples whose contexts are pulled from genuinely different sections of that document. I would avoid the beginner move of one giant leaf that tries to teach an entire product, because the generator cannot spread attention it was never given, and a sprawling context set becomes impossible to review. Better to ship three tight, diverse leaves than one bloated one that validates and disappoints.

One thing to do on Monday: take a single real document, pick five genuinely different passages from it, and write one knowledge leaf with a context and three grounded question answer pairs for each passage. Run ilab taxonomy diff until it prints the valid banner, then read your own contexts back and ask whether they cover the document or just repeat one corner of it. Next in the series we choose the actual model and accelerator this taxonomy will tune, sizing the first hardware the assistant runs on.

References

• LAB: Large-Scale Alignment for ChatBots, MIT-IBM Watson AI Lab and IBM Research, arXiv 2403.01081
• InstructLab documentation, knowledge qna.yaml file structure and schema version 3
• Red Hat blog, how InstructLab synthetic data generation enhances LLMs
• Red Hat documentation, Red Hat Enterprise Linux AI 3.5

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts