TL;DR: In a workflow, your code decides the sequence and models do specific jobs inside it. In an agent, the model decides what to do next. Agents earn their place when the path varies in ways rules cannot capture; where the path is known, begin with a workflow and compare added autonomy against its quality, cost and recovery behavior.
The distinction
The useful line is about control of the path. Chatbot describes an interface, so a chat product can contain either architecture. Tool use alone does not establish model-directed control.
Workflow. Engineers define the steps and the transitions between them. A model might classify an input, extract fields, draft a message or summarise a document, but it does so at a step the designers chose, and the next step is decided by code. Even a workflow with branches ("if the document is an invoice, go here") is a workflow, because the branches were designed.
Agent. The model is given a goal and a set of tools, and decides which tool to call, in what order, how to react to the result, and when it is finished. The path emerges at run time.
Both use the same models. The difference is how much decision-making you hand over, and that decision has consequences for reliability, cost, latency, testing and accountability.
What autonomy buys and costs
| Workflow | Agent | |
|---|---|---|
| Handles cases nobody designed for | Only by escalating | Can adapt |
| Predictability | High | Lower: paths vary run to run |
| Reliability | Each step testable | Errors compound over self-chosen steps |
| Cost and latency | Known number of calls | Variable, with long tails |
| Testing | Step by step | Needs outcome and trajectory evaluation |
| Explaining a decision | The path is in the code | The path is in a trace |
The variability is the point of an agent and also its main risk. An agent that can try a different approach when the first fails is valuable. The same freedom lets it loop, call tools in an unexpected order or take an action nobody intended.
A test for each step
Rather than asking "should this product be an agent?", ask of each step in the process: does the right next action vary in ways rules cannot capture?
- No. The step always follows from the previous one, or follows a rule. Make it code, possibly with a model doing a bounded job inside it. Examples: verify a tax ID against a registry, create a record, send a confirmation.
- The input varies, the action does not. A model handles the messy input inside a fixed step. Examples: extract fields from a scanned document in any format, classify a support message, translate a reply.
- The action itself varies. The best next move depends on what has been found so far, in ways too numerous to enumerate. Here agent-like autonomy can pay. Examples: investigating why a customer's integration is failing, researching a question across many sources, handling unusual exception cases.
Applied with discipline, most business processes are mostly the first two kinds, with a smaller exception path of the third.
The spectrum in between
It is not a binary. Useful designs sit at different points:
- Fixed workflow with model steps. The default for structured business processes.
- Workflow with a model-chosen branch. A model picks among a few designed paths (route this ticket to billing, technical or account). Still bounded.
- Agent inside a box. An agent handles one sub-task (diagnose the integration error) with a small set of tools and a budget, then hands back to the workflow.
- General agent with guardrails. The model drives the whole task, with tool permissions, budgets, approvals and stop conditions enforced in code.
Read diagram description
Four design points from workflow to agent. A fixed workflow with model steps, where code picks every step, such as invoice field extraction. A model choosing among a few designed branches, such as routing to billing or technical support. An agent inside one sub-task with a small toolset and a budget, such as diagnosing an integration. A general agent with guardrails enforced in code, driving an open-ended research task. Moving right adds flexibility; moving left adds predictability, testability and an audit trail in code. Under an illustrative independent model, ten required steps at 95% each finish about 60% of the time.
Moving right adds flexibility and removes predictability. The default is to stay as far left as the problem allows, and move right only for the parts where you can show that the variety is real.
Product consequences
Reliability. Every model-chosen step is another chance to fail, and failures compound: under an illustrative independent, all-steps-required model, ten steps at 95% each complete a task about 60% of the time. Correlation, retries and recovery change that calculation. Workflows contain the number of model-driven decisions. See reliability compounding.
Cost. A workflow makes a known number of calls per item; an agent's cost per task varies, sometimes by an order of magnitude. Pricing and budgeting are easier with workflows.
Accountability. In regulated processes (lending, insurance, payments, hiring), an auditor asks why a decision was made. A workflow's path is in the code; an agent's is in a trace that differs every run. Decisions with compliance consequences should come from deterministic steps.
Speed to build. Agents can look faster to build because you skip designing the path. That saving often comes back as time spent debugging unpredictable behaviour in production.
When to choose an agent
Agents are the right choice when the task is open-ended, the path depends heavily on intermediate findings, the cost of an occasional wrong path is acceptable or recoverable, and the value of handling the variety is high. Research, debugging, exploratory data analysis and complex customer investigations often qualify. Even then, the agent's tools, permissions and budgets are set by code, and irreversible actions pass through approval.
In the interview
Agent design questions often begin with "should this be an agent?" The answer that scores maps the process, applies the step-by-step test, puts code on deterministic and compliance steps, models on messy input, and agent autonomy only where the path really varies. Saying "I'd use the least autonomy that does the job, and here is where in this process autonomy earns its keep" signals that you have seen agents fail in production.
Next: reliability compounding · deterministic plus probabilistic