Prompt Flow has a retirement date. Microsoft set it at April 20, 2027, and flagged the tool as no longer recommended for new development, with a migration path to the Microsoft Agent Framework. I still open it most weeks, because the flows I shipped last year run in production and are not going anywhere before that date. So this part is two things at once: how the tool works, and an honest read on where it still belongs now that the clock is running.

What Prompt Flow is

Prompt Flow is a development tool for building applications on top of large language models. It gives you a way to chain a prompt, a model call, and your own code into one pipeline, run that pipeline on a single input or a whole test set, and score the results. It shows up in three surfaces: an open-source Python package called promptflow with a command line named pf, a Visual Studio Code extension with a visual graph editor, and a hosted authoring experience inside Microsoft Foundry and Azure Machine Learning. Same flow definition underneath all three.

The unit you work with is a flow. A flow is a directed acyclic graph, which means nodes connected by arrows that never loop back. Each node does one job and passes its output to the next. The whole graph, its inputs, its outputs, its nodes, and the connections they use, lives in one YAML file named flow.dag.yaml. That file is the source of truth. The visual editor and the portal both read and write it, so you can hand-edit the YAML or drag boxes and get the same result. If you called an Azure model from code back in Part 11, a flow is that same call wrapped in a graph that you can test and score without writing the harness yourself.

How a flow is put together

Four ideas carry the whole tool: tools, connections, inputs and outputs, and variants. A tool is the kind of work a node does. The three you use constantly are the LLM tool, which calls a chat or completion model with a templated prompt; the Prompt tool, which builds prompt text to hand to another node; and the Python tool, which runs a function you wrote for anything the model cannot do, such as parsing JSON, calling an internal API, or shaping the final answer. LLM and Prompt nodes are written as Jinja2 templates, so the prompt text sits in its own file with placeholders. Python nodes are plain Python files with a decorated entry function.

A connection is where secrets go. Rather than pasting an endpoint and key into the graph, you create a named connection once, an Azure OpenAI connection for example, and each LLM node references it by name and names the model in a deployment_name field. This is the part I appreciate most, because it keeps keys out of the YAML and out of source control. The same pattern wires a Content Safety node to the service I covered in Part 14, so a moderation check becomes one more node in the graph instead of a separate service call you bolt on by hand.

Inputs and outputs define the contract. Flow inputs are the fields you feed in, a question string say, and flow outputs are the fields you read back, referenced from a node output with the dollar-brace syntax. Variants are the last idea and the reason people stick with the tool longer than they planned. A variant is an alternate version of a Prompt or LLM node, a different prompt or a different temperature, stored side by side in the same flow. You run all variants over the same data and compare. That turns prompt tuning from guesswork into a table you can read.

Should you still build on Prompt Flow?

My answer is no for anything green-field, and a firm yes for maintenance. If you already run flows, keep running them. They will work until April 2027 and the migration guidance is not a same-week emergency. But for a project starting today, building on a tool with a published sunset means you inherit a rewrite. The place Microsoft points new agent work is the Foundry Agent Service and the Microsoft Agent Framework, which is exactly the runtime I walked through in Part 13.

There is one real exception. The evaluation half of Prompt Flow, batch runs and scored metrics, is genuinely useful and has a longer life outside the graph editor through the Azure AI Evaluation SDK, which I cover properly in a later part. So the honest split is this: skip Prompt Flow as an orchestration layer for new builds, but do not skip the discipline it teaches, which is scoring a change on a fixed test set before you ship it. That habit outlives the tool.

Author a two-node flow

Here is the smallest flow worth running: one LLM node that answers a support question, with the prompt in a Jinja2 file. First the graph. This is the full flow.dag.yaml.

inputs:
  question:
    type: string
outputs:
  answer:
    type: string
    reference: ${answer_node.output}
nodes:
- name: answer_node
  type: llm
  source:
    type: code
    path: answer.jinja2
  inputs:
    deployment_name: gpt-4o
    temperature: 0.2
    question: ${inputs.question}
  connection: aoai_connection
  api: chat

The prompt itself sits in answer.jinja2, with system and user roles marked as plain text sections:

system:
You answer product support questions in one short paragraph.
If you are unsure, say so plainly.

user:
{{question}}

Create the connection once, then test the flow on a single input from the command line:

pip install promptflow promptflow-tools

pf connection create --file aoai.yaml --name aoai_connection

pf flow test --flow . --inputs question='How do I reset my password?'

Expected output is a small JSON object keyed by your flow output name:

{
    'answer': 'Open Settings, choose Security, then select Reset password and follow the emailed link.'
}

Failure mode: if the connection name in the YAML does not match a connection you created, pf raises a connection not found error before any token is spent. Create the connection first, confirm it with pf connection list, then rerun. A second common trip is a deployment_name that does not exist in the linked Azure OpenAI resource, which surfaces as a 404 from the model call, not a flow error. The pf sub-commands have shifted slightly across promptflow releases, so pin the version. [VERIFY: pf command names against your installed promptflow version]

Variants and how I test prompts

The reason I reach for a flow rather than a raw script is the variant loop. On that single answer_node I can define variant_0 as the terse prompt at temperature 0.7, variant_1 as the same prompt with a formatting rule added at temperature 0.2, and variant_2 with two worked examples baked into the system message. All three live in the same flow. I run each over the same test set and read the scores side by side instead of eyeballing a handful of replies and guessing which prompt is better.

The sample below is one run of exactly that, three variants scored on a groundedness metric from 1 to 5. The lift from variant_0 to variant_2 is the kind of gap you cannot see by reading ten answers, but it jumps out once you plot it. This is illustrative, not a benchmark of the models themselves. [AUTHOR: add anecdote about a variant that scored well but felt worse in production]

Evaluation flows and metrics

An evaluation flow is a second flow whose job is to score the first one. It takes the outputs of a batch run and, row by row, produces a metric: groundedness against a context, relevance to the question, similarity to a reference answer, or a pass or fail from a rule you write in a Python node. You run your main flow over a data file to get answers, then run the evaluation flow over those answers to get numbers. The batch run and the eval run are separate commands, which feels like one step too many at first and turns out to be the right seam once you have more than one metric.

Run the batch and evaluation from the command line against a JSONL data file with a column mapping that tells the flow which column feeds which input:

pf run create --flow . --data ./questions.jsonl 
  --column-mapping question='${data.question}' 
  --name base_run --stream

pf run create --flow ./eval_groundedness --data ./questions.jsonl 
  --run base_run 
  --column-mapping answer='${run.outputs.answer}' context='${data.context}' 
  --name eval_run --stream

The eval run prints an aggregate metric per variant, which is what feeds the comparison table below. The number that matters is not the top score, it is the gap between variants on the same data. A prompt that reads better to you is worth nothing if it does not move the metric on your own examples.

Tracing a flow run

When a flow returns a wrong answer, the useful question is which node went wrong, not which prompt looks off. Prompt Flow records a trace for every run. Each node shows its inputs, its output, the tokens it spent, and how long it took. Local pf runs open that trace in a browser view, and each batch run in the portal keeps the same per-node detail. I lean on this more than any other feature, because it turns a vague bad output into one specific node I can fix.

Two habits pay off. Read the LLM node output before you read the Python node that parses it, because most failures I chase are a model that returned prose where the next node expected JSON, not a bug in the parser. And watch the token count per node across a batch, since one wordy variant can double your spend without moving the score at all. The trace puts cost and quality on the same screen, so tuning one stops hiding what it does to the other.

From local flow to an endpoint

Once a flow behaves and scores well, you deploy it and it becomes an HTTP endpoint. In Azure that is a managed online endpoint: the flow, its connections, and a scoring script get packaged and hosted behind a URL your app calls with the flow inputs as JSON. You pick the compute size, the endpoint scales within the limits you set, and you pay for the compute while it runs plus the model tokens each call spends. The connections you used locally must exist in the workspace you deploy to, which is the single most common reason a flow that ran clean on a laptop returns a 500 in the cloud.

Maintain what you have, build new on Agent Framework

Here is where I land after a year of running flows in production. Keep the ones you have; they are stable and the April 2027 date is far enough out that a rushed migration would cost you more than it saves. Use the variant-and-evaluate loop on every prompt change you make, in Prompt Flow or out of it, because scoring on a fixed set is the habit that actually moves quality. But do not start a new orchestration project here. Put new agent work on the Foundry Agent Service from Part 13, and carry the evaluation discipline over with you.

If you have an existing flow, do one thing this week: run it over a 200-row set and record the score. That single number is your baseline for the migration, and you will want it the day you rebuild the flow on the newer runtime. Next in the series is Part 16, fine-tuning Azure OpenAI, where the question shifts from shaping the prompt to changing the model itself.

References

• Prompt flow in Microsoft Foundry (classic), Microsoft Learn
• How to build with prompt flow (classic), Microsoft Learn
• Prompt flow open-source documentation

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts