Five seed examples, each a short context and three question and answer pairs, became roughly 1,200 training rows the first time I ran generation against the assistant docs. That ratio is the point of synthetic data generation, and it is also the trap. A teacher model amplifies whatever you feed it, so five careless examples give you 1,200 careless rows and a tuned model that is fluent and confidently wrong about your own product.

Last part the assistant served the stock granite-3.1-8b-instruct and handled generic questions well enough. This part hands it something to learn: a taxonomy leaf for the company product docs, a knowledge qna.yaml that grounds it, a validated tree, and a synthetic dataset generated from those seeds. Training on that set is Part 10; here we earn the data. Synthetic data generation is a fine tuning method, and where fine tuning sits against retrieval and prompting the GenAI series maps out under fine tuning vs RAG vs prompting.

Building a taxonomy leaf for the company docs

A taxonomy in InstructLab is a directory tree, and each leaf holds one qna.yaml that teaches the model a single narrow thing. Three top level branches exist: knowledge for facts grounded in a document, and compositional_skills and foundational_skills for performative abilities. Our support assistant needs facts about the product, so its content goes under knowledge. That tree lives at ~/.local/share/instructlab/taxonomy after ilab init, and adding to it is just making directories and dropping in a file.

# ilab bundled in RHEL AI 1.5, taxonomy schema version 3
$ cd ~/.local/share/instructlab/taxonomy
$ mkdir -p knowledge/company/support/product_docs
$ ls knowledge/company/support/product_docs
attribution.txt   qna.yaml

Keep the leaf path descriptive and shallow. A path like knowledge/company/support/product_docs reads as a fact domain a human can find later, and the directory name has no bearing on training beyond organisation. One leaf per coherent topic beats one giant leaf, because smaller focused seeds generate cleaner data than a grab bag of loosely related questions.

Writing a knowledge qna.yaml that grounds SDG

Version 3 is the current knowledge schema, and the fields that carry the weight are the seed examples. Each seed example holds a context, a snippet under 500 tokens pasted straight from your document, plus at least three question and answer pairs grounded in that context. You need at least five seed examples, and the document block pins the source with a git repo URL, a full commit hash, and the file patterns to read.

Here is a trimmed knowledge leaf for the assistant, built from an Acme Relay changelog and admin docs. A real leaf would carry five distinct contexts; one is shown for length.

version: 3
domain: acme_support
created_by: pranay
seed_examples:
  - context: |
      Acme Relay 4.2 lowered the default session timeout from 30 minutes
      to 15 minutes. Administrators can raise it under Settings, Security,
      Session, up to a maximum of 8 hours. Values above the maximum are
      clamped to 8 hours and logged as a warning.
    questions_and_answers:
      - question: |
          What is the default session timeout in Acme Relay 4.2?
        answer: |
          In Acme Relay 4.2 the default session timeout is 15 minutes,
          down from 30 minutes in earlier releases.
      - question: |
          Can an administrator set a 12 hour session timeout?
        answer: |
          No, the maximum is 8 hours. Higher values are clamped to 8 hours
          and a warning is written to the audit log.
      - question: |
          Where is the session timeout configured?
        answer: |
          Under Settings, Security, Session in the admin console.
  # four more seed examples, each a distinct context
document_outline: |
  Acme Relay 4.2 administration, covering session security, changelog
  entries and common support ticket resolutions.
document:
  repo: https://github.com/acme/relay-docs.git
  commit: 3f9a1c2b7d4e5f6a8b9c0d1e2f3a4b5c6d7e8f90
  patterns:
    - 'changelog/*.md'
    - 'admin/session.md'

The commit hash is not decoration. It pins the exact document version SDG grounds on, so an edit next week cannot silently change what your seeds mean. Pin it to a real commit, not a branch name, and bump it deliberately when the docs change. Seed examples are a curated set much like the eval set the AI Engineering series builds under building an eval set, and the same discipline applies: write the cases you actually care about getting right. Keep the table below next to your editor as the field reference for a knowledge leaf.

Validating the taxonomy with ilab taxonomy diff

Before generating anything, validate. ilab taxonomy diff lists the leaves you changed and checks each against the schema, and it runs in seconds. Catching a broken file here is free; catching it after a two hour generation is not.

$ ilab taxonomy diff
knowledge/company/support/product_docs/qna.yaml
Taxonomy in ~/.local/share/instructlab/taxonomy is valid :)

# copy an old file as a template and you inherit version 2:
$ ilab taxonomy diff
knowledge/company/support/product_docs/qna.yaml:
Version 2 is not supported for knowledge. Minimum supported version is 3.
Reading taxonomy failed with the following error: 1 taxonomy with errors!

A version mismatch is the failure I hit most often on a fresh leaf, because copying an old qna.yaml as a template drags version 2 along, and the knowledge schema no longer accepts it. Set version to 3 and the same command turns green. Other failures land the same way: a context over 500 tokens, fewer than five seed examples, or a commit hash that does not resolve. Use the lookup below to go from the message to the fix.

Running synthetic data generation

With a valid tree, one command produces the dataset. ilab data generate reads your changed leaves, starts a temporary vLLM server for the teacher model, and runs the pipeline that synthesises, scores and filters question and answer pairs against your seeds.

flowchart LR
  A[Product docs in git] --> B[Knowledge qna.yaml leaf]
  B --> C[ilab taxonomy diff]
  C -->|invalid| B
  C -->|valid| D[ilab data generate]
  D --> E[Temporary vLLM teacher mixtral]
  E --> F[Synthesise score filter]
  F --> G[knowledge and skills train msgs jsonl]
  G --> H[Part 10 multi phase training]

# RHEL AI 1.5, mixtral-8x7b-instruct-v0-1 teacher, agentic pipeline
$ ilab data generate --num-cpus 4
Starting a temporary vLLM server at http://127.0.0.1:49311/v1
INFO instructlab.model.backends: Waiting for the vLLM server to start ... Attempt: 74/120
INFO instructlab.model.backends: vLLM engine successfully started
Generating synthetic data using the agentic pipeline,
  mixtral-8x7b-instruct-v0-1 model, against http://127.0.0.1:49311/v1 server
INFO instructlab.sdg.datamixing: Mixed Dataset saved to
  ~/.local/share/instructlab/datasets/knowledge_train_msgs_2026-07-28T20_54_21.jsonl
INFO instructlab.sdg: Generation took 1355.74s

One flag matters on the four card node: a box with 4xL40S needs –num-cpus 4, or generation stalls partway, and the docs bury this in a callout that is easy to miss. Add -dt to run detached when you want your terminal back, then watch it with ilab process list. Generation time scales with your seed count and hardware; the run above took about 22 minutes for a handful of leaves. The JSONL it writes is just another data format to load and inspect, which the Data Science series covers under getting data into Python.

Why the teacher model will not fit the serving card

My first instinct was to generate on the same L40S box that serves the assistant. It failed immediately. SDG does not use the granite model you serve; it loads mixtral-8x7b-instruct-v0-1 as the teacher, and that model carries 46.7B parameters. At bf16 the weights alone are about 50 GB, which does not fit a single 48 GB L40S, so vLLM aborted with a CUDA out of memory before generation even began.

$ ilab data generate                 # attempted on the single L40S serving box
Starting a temporary vLLM server at http://127.0.0.1:47825/v1
ERROR   vLLM failed to start
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 224.00 MiB.
GPU 0 has a total capacity of 44.35 GiB of which 18.12 MiB is free.
# mixtral-8x7b weights are ~50 GB at bf16; a single L40S has 48 GB

What fixed it was not a smaller batch or a lower utilisation; it was the wrong machine. Generation moved to the four card tuning node from Part 6, where 192 GB across four L40S holds the teacher with room for its KV cache, and vLLM shards it with tensor parallelism. Serving and generating are different workloads with different homes, and I wasted about fifteen minutes treating a placement problem as a tuning problem.

Read the bars plainly: teacher weights overflow one card, so there is no utilisation tweak that rescues it. Generation is a batch job that wants the big node; serving is a latency job that lives on the small one. Where the teacher runs is a sizing decision, not a flag.

Reading the generated dataset before you train

Before handing anything to training, open the output. Generation writes into ~/.local/share/instructlab/datasets, dated, with the two files that matter: knowledge_train_msgs and skills_train_msgs. Note the newest knowledge_train_msgs filename, because Part 10 needs it by name.

$ ls ~/.local/share/instructlab/datasets/
knowledge_recipe_2026-07-28T20_54_21.yaml
knowledge_train_msgs_2026-07-28T20_54_21.jsonl
skills_recipe_2026-07-28T20_54_21.yaml
skills_train_msgs_2026-07-28T20_54_21.jsonl
$ wc -l knowledge_train_msgs_2026-07-28T20_54_21.jsonl
1214 knowledge_train_msgs_2026-07-28T20_54_21.jsonl

Read a few rows by hand. Each is a chat style record with a system, user and assistant turn, and if the answers are wrong, off topic, or repeat your seed almost verbatim, stop now and fix the seeds, because training will only entrench the noise. A ten minute read of the JSONL is the cheapest quality gate you have before hours of tuning, and it is the step most people skip on their way to a disappointing model.

Recommended taxonomy and SDG setup for the support assistant

For the assistant I would keep each knowledge leaf small and specific, write five or more seed examples with genuinely distinct contexts under 500 tokens, pin the document block to a real commit hash, and run ilab taxonomy diff before every single generate. Generation runs on the four card node with –num-cpus 4, never on the serving box, because the mixtral teacher does not fit one L40S. Then read the JSONL before you train. Two habits to avoid: copying an old qna.yaml and inheriting version 2, and treating the document repo as a substitute for good seeds.

Next in the series we take these JSONL files and actually tune Granite on them, running the multi phase alignment that turns generated data into a model that knows the product.

References

• Red Hat Enterprise Linux AI 1.5, generating a new dataset with SDG
• InstructLab, knowledge file structure and the qna.yaml schema
• InstructLab, creating new knowledge or skills
• Red Hat Customer Portal, taxonomy diff version 2 not supported for knowledge
• InstructLab taxonomy repository

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts