AI Agents for Business: Use Cases, Costs and Build Versus Buy
A plain playbook for operations leaders. What an AI agent is, where it replaces manual work, how to pick a first use case, what drives cost, and whether to build, buy or partner.
By Raza Memon and Muhammad Usman AliEditor responsible: Muhammad Usman Siddiqui
Playbook · Introductory · 32 min

On this page · 17 sections
- 01What an AI agent is, in plain words
- 02Where agents replace manual work today
- 03When one agent is enough, and when it is not
- 04Human in the loop, in practice
- 05Memory and decision making
- 06How to choose a first use case
- 07How to measure ROI
- 08What drives the cost
- 09Build, buy or partner
- 10Risks to plan for
- 11Governance and the operating model
- 12How to test and configure an agent for accuracy
- 13A 30, 60 and 90 day plan
- 14When an agent is the wrong tool
- 15Common mistakes
- 16Glossary
- 17Next step
Key takeaways
- An agent chooses its own next step toward a goal. Automation follows fixed rules and a chatbot answers questions. Pick the simplest one that does the job.
- Pick the first use case with the five-question First-Agent Score. Favor a frequent task with a clear right answer where a wrong result is cheap to catch.
- One agent is enough for most first projects. In Anthropic's research system, a multi-agent setup used about 15 times the tokens of a chat, so add agents only when a test shows the gain. That is one system's figure, not a general rule.
- Put a human approval point before anything that cannot be undone, then review a sample of the rest. Name an owner for the agent after launch.
- Gartner predicts over 40% of agentic AI projects will be canceled by the end of 2027, mostly on cost, unclear value or weak risk controls. Set the success measure before you build.
What we tested
Author process, 2026-10-27: the Python scoring script in this guide was run and its printed output matches the text. The payback math (src/lib/guides/calc.ts) was run with the starting inputs and four changed inputs, and every payback figure in the worked example is its output. The four diagrams were drawn by hand and rendered to check for overflow. The Microsoft, Anthropic (research system, effective agents, thinking and effort documentation), OWASP (2026 edition page and the 2026 LLM01 and LLM03 entries from the project's GitHub repository), NIST, Mem0 (2026 memory round-up and Zep comparison) and TypeSafe pages were opened and checked against the text on 2026-10-27. We searched for newer sources for each older item: Gartner's newer 2026 agent releases could not be opened, and Anthropic's research-system post has no newer figure. The Gartner press release and the McKinsey report were saved from the owner's browser on 2026-10-27, and every figure we quote from them was checked against those originals. The EdgeFirm posts, service pages and the two case study pages linked here exist and the case study descriptions follow what those pages state. The OWASP 2026 LLM03 Excessive Agency entry was also opened on 2026-10-27 for the three security callouts; the injection test idea is our own suggestion and was not run. The blog posts linked in the added links had their titles and excerpts read. No agent was run in production, no memory product or the Jev model was tested, and no client data was used.
This guide is for the person who has to decide: a COO, a head of operations or an engineering lead who wants to know where an agent fits, what it costs to run, and how to avoid the projects that stall.
This is an engineer's report, written in the first person plural. Where a fact comes from an outside page, we say which page we read, on what date, and what we make of it. Where a number is an illustration, we label it. We give no savings figures, because we found no source we would trust for one.
What an AI agent is, in plain words
An AI agent is software that pursues a goal. You give it a task, a set of tools and some rules. It decides which step to take next, uses a tool, looks at the result and keeps going until it is done or asks for help.
Anthropic draws the line this way. In a workflow, models and tools follow predefined code paths. In an agent, the model directs its own process and tool use. We use that as the working test on a new project. If you can draw the full path in advance, you have a workflow. If the path depends on what the software finds, you have an agent.
Automation, chatbot and agent
That sounds close to two things you already know. The picture and table show the typical differences.
Three columns compare automation, a chatbot and an agent on what each does, who picks the next step, an example and the main risk.
| Automation | Chatbot | Agent | |
|---|---|---|---|
| Who picks the next step | You, in advance | The user, one question at a time | The software, within limits you set |
| Handles surprises | Usually not, it stops or fails | Answers, but does not act | Tries another route |
| Can change things in your systems | Yes, in fixed ways | Usually not | Yes, through approved tools |
| Typical example | Copy a form field into the ERP every night | Answer a question about a leave policy | Read an invoice, find the order, flag a mismatch |
| Main risk | Breaks when the input changes | States a wrong fact | Takes a wrong action |
| Best when | The steps never change | The user needs information | The path depends on what is found |
Use the simplest tool that works. A fixed rule is cheaper, faster and easier to audit than an agent. Our post on agent versus chatbot covers that choice. For the wider market view, see the reality check on AI agents and agentic AI for enterprise.
What an agent is made of
Every business agent has the same five parts.
- A goal. A written task with a clear finish, such as "match this invoice to its order and receipt and report any difference."
- Tools. The systems the agent may read or change, such as the ERP, the ticket queue or a document store. Each tool is a permission you grant. The Model Context Protocol is one common way to connect tools to a model.
- Rules. What the agent must never do, what it must check, and when it must stop and ask.
- A loop. The agent picks a step, runs a tool, reads the result and decides again. A limit on the number of loops keeps cost and errors bounded.
- A human point. A named person who approves risky steps and receives hard cases.
The software that wraps the model and runs this loop is called an agent harness. Our guide to agent harnesses covers its parts in detail. This guide stays on the business decisions around it.
Where agents replace manual work today
Agents earn their place on work that is repetitive, reads messy input and ends in a decision a person can check. They are a poor fit for work that is rare, has no clear right answer or cannot be undone.
A use-case catalog
The table lists the back-office areas we would look at first. It carries no savings numbers, because we found no fair figure for each. The task, the human check, the main risk and the data needed are our own engineering summary, drawn from the posts we have written, and not a measured result.
| Area | Typical task | What the agent does | Human check | Main risk | Data needed |
|---|---|---|---|---|---|
| Back office | Invoice and purchase order (PO) matching | Reads the invoice, finds the order and receipt, compares them and flags differences | Approve each mismatch fix before it posts | Paying or rejecting the wrong amount | Invoices, orders, receipts, supplier records |
| Support | Ticket triage | Tags topic and urgency, drafts a reply, routes hard cases | Review drafts for sensitive topics, sample the rest | A wrong or tone-deaf reply sent to a customer | Ticket history, help articles, product data |
| Finance operations | Reconciliation and month-end evidence | Gathers supporting records for each open item and lists gaps | Preparer signs off every item | A missed item or a wrong match | Ledger, bank feeds, supporting documents |
| IT | Access and request tickets | Sorts requests, resets access where policy allows, escalates the rest | Approve any access change above a set level | Granting access that should not be granted | Ticket queue, access policy, directory |
| Sales operations | Lead research and record clean-up | Collects company facts, fills missing fields, flags duplicates | Sample review of changed records | Overwriting good CRM data with wrong data | CRM records, approved public sources |
| Legal review | First pass on contracts | Pulls out clauses you care about and marks anything unusual | A lawyer reads every flagged clause and a sample of the rest | Missing a clause that matters | Contract text, your clause playbook |
| Knowledge work | Research and summary packs | Gathers material from many sources, with links to each | Reader checks the cited passages | A claim with no real source | Document stores, approved web sources |
What the linked EdgeFirm case studies show
Two EdgeFirm case studies bear on this guide. We report only what the case study pages state. Neither page gives a savings figure.
The AI code security analysis platform is an agent-driven platform that, according to its page, uses the Claude Agent SDK. It models a client's repository as a graph, runs agent workflows to find likely vulnerabilities, and then tests each finding at runtime with Playwright and OWASP ZAP. The page says this runtime validation means only exploitable findings reach engineers, and that the client's own tests showed fewer edge-case failures. It gives no percentage. Our read, not the page's: the lesson for a buyer is the checking step. In that design the agent proposes, and a separate test confirms before a person is asked to act.
The competitive intelligence platform, called Harbinger AI, is an AI monitoring and analysis pipeline with a conversational assistant. The page calls it an autonomous engine. It is a monitoring and analysis pipeline with a chat assistant, not an agent in the sense used in this guide, so it is not an example of an agent replacing manual work. According to the page it monitors six data dimensions, classifies each signal from critical to low, caps the number of alerts per day so analysts are not flooded, and delivers weekly briefings. Users give thumbs up or down with a reason, which feeds refinement. The page states 14 weeks from kickoff to production. What we take from it for a buyer: the daily alert quota and the feedback loop are both controls that keep a model's output useful and reviewable.
When one agent is enough, and when it is not
Most first projects need one agent. A multi-agent system splits a job across several agents that work together. It can help, and it costs more.
We read Microsoft's architecture guidance on 2026-10-27, and it matches how we plan. It says to use the lowest level of complexity that reliably meets your requirements, and it treats a single agent with tools as often the right default for enterprise use cases. It adds that multiple agents bring coordination overhead, latency and extra failure modes, and that they are justified when one agent cannot handle a task reliably because of prompt complexity, too many tools or security needs.
Single agent, router or supervisor
Three layouts cover most business cases. The picture below shows each one and where a person approves.
Three agent layouts side by side: one agent with tools marked start here, a router that sends each request to one specialist, and a supervisor that splits a job across specialists, each followed by a human approval step before any action.
- Single agent. One agent with a small set of tools handles the whole job. It is the cheapest to run and easiest to test. Start here.
- Router. A first step reads each request and sends it to one specialist agent, such as billing, returns or IT. Only one agent works on each request. Use it when requests differ by type. We found this layout under Microsoft's handoff pattern, which lists routing and triage among its alternate names. In it, full control moves from one agent to another.
- Supervisor with specialists. One supervisor breaks the job into parts, hands each part to a specialist and combines the results. Use it when parts of one job can run apart from each other, such as research across several sources. It is the most expensive layout.
An approval point is a gate you can add to any of the three. Put one before any payment, deletion, customer message or other action you cannot undo.
How the patterns map to business processes
We counted five orchestration patterns in Microsoft's guidance: sequential, concurrent, group chat, handoff and magentic. Sequential chains agents in a fixed order. Concurrent runs several agents on the same task at once. Group chat lets agents contribute to one shared thread. Handoff moves full control from one agent to another. Magentic uses a manager agent that builds and adjusts a task list for open-ended problems. The table pairs each with a business process. The process examples are ours.
| Pattern | What happens | Business process it could fit (our example) | Where a person sits |
|---|---|---|---|
| Sequential | Each agent works on the previous agent's output, in a set order | Invoice intake, then matching, then exception write-up | Approves the exception write-up |
| Concurrent | Several agents analyze the same input at once, then results are combined | Vendor risk review, with separate checks on finance, security and legal records | Reads the combined findings |
| Group chat | Agents and people add to one shared thread, often as a maker and a checker | Drafting a customer reply, with a second agent checking it against policy | Joins the thread or approves the final draft |
| Handoff | One agent at a time, and it passes control to a better-suited one | Support: general triage, then billing or technical agent, then a human for hard cases | Receives escalations |
| Magentic | A manager builds and revises a plan, other agents act on systems | Open-ended incident response across several tools | Reviews the plan and any risky step |
For a first project we would not go beyond a single agent or a simple sequence.
What multi-agent costs
We read Anthropic's write-up of its research system, which has a lead agent and several helper agents. Anthropic reports that the multi-agent setup beat a single agent by 90.2% on its own internal research test (an "eval", a fixed set of test tasks), scored by a language model against a rubric. The lead agent was Claude Opus 4 and the helpers were Sonnet 4, models several generations behind today's, and we found no newer Anthropic figure. We treat it as direction, not a forecast for your system. The same note says agents use about four times the tokens of a chat, and multi-agent systems about 15 times. For us, the cost side of that pair is the number to plan around. A token is a small chunk of text that the model reads or writes, and you pay per token. Anthropic also says domains where all agents must share the same context, or have many dependencies between agents, are a poor fit today, and that most coding tasks have fewer parts that can run in parallel than research does. We check for exactly those two traits before we propose a second agent.
Our rule: start with one agent. Add a second only when one job has parts that can run apart from each other and a test shows the gain justifies the extra cost.
Human in the loop, in practice
"Human in the loop" means a person takes part at set points. It is the main way you keep an agent safe in the first months. It is also easy to do badly.
In Microsoft's guidance we found three forms a person can take: observer, reviewer and escalation target. It asks the designer to decide which points need a person, whether that input is required, and whether the person approves or sends feedback back. It also says approval can attach to specific tool calls (each time the agent uses a tool), so low-risk actions go ahead alone and only sensitive ones wait. Anthropic's page says agents can pause for human feedback at checkpoints or when they hit a blocker, and teams often add stopping conditions such as a maximum number of steps.
Approval thresholds
Decide in advance which actions need approval. We group them into three bands. The bands are our recommendation.
- Act alone. Read-only steps and reversible changes with low impact, such as tagging a ticket.
- Approve first. Anything that moves money, changes a customer record, sends a message outside the company or grants access. A named person says yes before it happens.
- Never. Actions the agent must not take at all, such as deleting records or changing its own permissions. Block them in the tool settings, not just in the prompt.
Set the dollar or record-count line for each band with your finance and risk leads, and write it down.
Sampling review
For actions that run alone, a person still checks a sample. Pick a share of cases at random each week, and add every case where the agent reported low confidence or the checking step disagreed. Start with a larger share and lower it as the error rate stays low. We give no standard figure, because the right share depends on the cost of a wrong answer. Record each reviewed case as right or wrong, and add it to your test set.
Escalation
An escalation hands a case to a person when the agent cannot or should not continue. Define the triggers: the agent is unsure, a rule says stop, a tool fails twice, or the request falls outside its scope. Define who receives the case, how fast they must respond, and what the agent passes along: the goal, what it tried and what it found. Microsoft's guidance notes that a maker and checker pair needs an iteration cap, and that the fallback when the cap is reached can be a human reviewer. We cap every loop.
Memory and decision making
An agent forgets everything between runs unless you give it memory. A memory layer is a service that stores facts from past work so the agent can look them up later. Mem0, Letta and Zep are products in this space.
They work in different ways. In the Mem0 comparison we read, Mem0 describes its own design as storing extracted facts instead of raw conversation turns, and describes Zep as storing the raw history and building a temporal knowledge graph on top. A temporal knowledge graph tracks how facts and relationships change over time.
What they do, as we read the vendor pages: keep preferences, past decisions and facts across sessions so you do not paste them in each time. What they do not do: make the agent right. A stored wrong fact stays wrong. They also add cost, and you still need to decide what is allowed to be remembered, for how long and who can see it.
We would not rank memory products by their published scores. The vendors dispute each other's numbers, and Mem0 itself says the LoCoMo benchmark has an active methodology dispute between its team and Zep. In Mem0's own 2026 round-up, read on 2026-10-27, we found 92.5 on LoCoMo for its April 2026 algorithm, 80.32% to 83% for Zep, and 74.0 for a Letta test that stored the conversation in a plain file with basic file tools and a small model (GPT-4o mini). The same page says scores shift with the dataset revision, the judge model, how many results are retrieved and the prompt format, and it does not name the answer model behind Mem0's 92.5. Every vendor here sells agent memory, so we treat each number as a vendor result. Our recommendation: choose a memory layer by how it stores data, how you can inspect it and how you can delete it, not by a leaderboard.
A small decision model for routing and scoring
Not every decision needs a large language model. We read the documentation for TypeSafe AI's Jev, a different kind of model. It says Jev returns typed answers and probabilities instead of generated text, and that it does not write replies, produce code or explain its reasoning. It answers three kinds of question: pick one option from a set, give a score on a scale you define, or give a yes or no answer with a confidence value. The docs say it takes text and JSON input only, and that calibration is measured across groups of predictions and does not guarantee that any single answer is correct.
From the documentation alone, we would shortlist it for cheap steps inside an agent, such as sending a ticket to the right team or scoring how urgent a message is. The concepts page gives no speed or price figures, so we quote none. Any team that adopts it should measure it on its own data.
How to choose a first use case
We use a short scoring method so we do not pick a task because it sounds impressive. We call it the First-Agent Score. It is our own method, not an industry standard.
The First-Agent Score
Score each candidate on five questions from 1 to 5, where 5 is best. Count three of the questions twice, because they matter more: how often the task happens, whether you can say what a correct result looks like, and how cheap a wrong answer is to catch and undo. The other two count once: whether the agent can reach the data it needs, and whether you can count time or errors before and after. A 5 on the cost question means a mistake is cheap and easy to undo.
"""Score candidate agent use cases on a 1 to 5 scale per question. Higher is better on every question. For "cost of a wrong answer" a 5means a mistake is cheap and easy to undo.""" WEIGHTS = { "volume": 2, # how often the task happens "clear_outcome": 2, # can you say what a correct result looks like "data_access": 1, # can the agent reach the data it needs "cheap_mistakes": 2, # how cheap a wrong answer is to catch and undo "measurable": 1, # can you count time or errors before and after} CANDIDATES = { "Invoice and PO matching": dict(volume=5, clear_outcome=4, data_access=3, cheap_mistakes=3, measurable=5), "Support ticket triage": dict(volume=5, clear_outcome=3, data_access=3, cheap_mistakes=4, measurable=4), "Contract review": dict(volume=2, clear_outcome=2, data_access=3, cheap_mistakes=1, measurable=2),} def score(answers: dict) -> float: total = sum(WEIGHTS[k] * answers[k] for k in WEIGHTS) best = sum(5 * w for w in WEIGHTS.values()) return round(100 * total / best, 1) if __name__ == "__main__": ranked = sorted(CANDIDATES.items(), key=lambda kv: score(kv[1]), reverse=True) for name, answers in ranked: print(f"{score(answers):5.1f} {name}")In our run on 2026-10-27 it printed the ranking below. The scores are examples for illustration, not data from a client.
80.0 Invoice and PO matching 77.5 Support ticket triage 37.5 Contract reviewA weighted scoring grid with five criteria scored 1 to 5 for three candidates. Invoice and PO matching scores 80.0, support ticket triage 77.5 and contract review 37.5.
Our read of the output: the top two are close, so the score is a way to start a discussion, not a verdict. Contract review ranks low in this example because volume is low and a mistake is costly. We would not make it a first project, though it is not a bad idea. Replace the example scores with your own, then pick the top task. Also check the reality check for the failure patterns to avoid.
An agent-readiness checklist
A high score on a use case still fails if the basics are missing. This checklist is our engineering judgment, not a standard. We check it before a build. Each should be a yes.
| Check | What a yes looks like | If the answer is no |
|---|---|---|
| Owner | A named business owner for the process and the agent | Pause. Nobody can accept the results |
| Baseline | Current hours, error rate and cost per case are recorded | Measure for a few weeks first |
| Test cases | 50 to 100 real examples with right answers (a rule of thumb from us) | Collect them before building |
| Data access | The agent can reach the records through an approved route | Fix access, or choose another task |
| Clear rules | Written policy for the decisions the agent will make | Write it down with the process owner |
| Approval points | You know which actions need a person and who that is | Define them before the first test |
| Undo path | You can reverse a wrong action | Add approval before that action |
| Security review | Access is limited to what the task needs, and logged | Do it before connecting any tool |
How to measure ROI
Return on investment (ROI) compares what the agent saves or earns with what it costs to run. Set the baseline before you build: current hours per case, error rate and time to resolve. Then track the same numbers after launch. Our post on enterprise AI ROI covers building the business case before you commit budget.
The payback calculator
The calculator below turns six inputs into a payback time. You give it tasks per month, minutes per task today, the loaded cost per hour, the share of tasks the agent handles alone (from 0 to 1), the one-off build cost and the running cost per month. It works out hours saved a month as tasks times minutes times the share, divided by 60. Benefit a month is those hours times the hourly cost. Net is the benefit minus the running cost. Payback in months is the build cost divided by the net. If the net is zero or less, it reports "never at these numbers."
Agent payback calculator
The starting numbers are an illustration, not a benchmark or a client result. Use your own measured baseline.
Hours saved a month
480
Benefit a month
$19,200
Net after running cost
$17,700
Payback
3.4 months
Saved hours are only money if the time moves to other work or headcount changes. See the post on enterprise AI ROI for how to test that.
The starting numbers are an illustration, not a benchmark or a client result. Replace every one with your own measured baseline. It has limits we would flag in any review. It counts saved hours as money, which is only true if the time moves to other work or headcount changes. It does not count the time a person spends reviewing the agent's work, so add that to the running cost. The full method for building the case is in our post on enterprise AI ROI.
A worked example
This section is an illustration. It is not a client, and no EdgeFirm project produced these numbers. Every figure is the output of the scoring script and the calculator above, which we ran on 2026-10-27.
Imagine an accounts payable team choosing a first agent. They score three candidates with the First-Agent Score. Invoice and PO matching scores 80.0, support ticket triage 77.5 and contract review 37.5. The team picks invoice matching, with the caveat that triage is close.
They then size it with the calculator's starting inputs: 4,000 tasks a month, 12 minutes each, $40 an hour, 60% handled alone, a $60,000 build cost and $1,500 a month to run. The $60,000 is a placeholder in the calculator. It is not an EdgeFirm price. The output is 480 hours saved a month, a benefit of $19,200, a net of $17,700 after running cost, and a payback of about 3.4 months.
Now the team tests the weak points by changing one input at a time. These are the outputs we recorded:
- If the agent handles only 30% alone, hours saved drop to 240 a month, net falls to $8,100 and payback stretches to about 7.4 months.
- If running cost doubles to $3,000, net falls to $16,200 and payback is about 3.7 months.
- If both happen, net is $6,600 and payback is about 9.1 months.
- If the agent handles only 10% alone, hours saved are 80, net is $1,700 and payback is about 35.3 months.
What we see in these runs: the share the agent handles alone moves the answer far more than the running cost does. We would measure that share first, on real cases, in the shadow run.
What the surveys say about returns
We keep expectations grounded. In McKinsey's State of AI 2026 report, which we checked against a saved copy, 37% of respondents attributed at least some EBIT (earnings before interest and taxes) impact to AI, about the same share as the year before, and about 20% said AI operating costs constrained their AI use. These are shares of survey respondents, not of companies, and we would not read them as a savings rate.
What drives the cost
We give no price for a generic agent because the range is too wide to be useful. These are the drivers we price against.
- Token cost. Every step the agent takes reads and writes tokens, and multi-agent setups multiply them.
- Thinking depth. Anthropic's thinking documentation states that thinking tokens are billed as output tokens even when you do not see the thinking text.
- Evals. An eval is a set of test cases you run on every change. Building and running them takes real effort, and in our judgment skipping them costs more later.
- Integration. Each system the agent must reach adds build and security work.
- Human review. Time spent approving and sampling is a running cost.
- Maintenance. Models, tools and your own processes change. Someone must own the agent after launch.
For ranges by agent type, read how much an AI agent costs to build.
Build, buy or partner
Buy when a vendor product already fits your process and your data can stay where it is. Build when the process is unique to you and gives you an edge. Partner when you want a custom build and lack the in-house team to run it. Our build, buy or partner decision matrix scores these options against your situation in more depth.
A scoring table for the choice
The table below is our engineering judgment, not a measured result. For each row, pick the column that fits your situation. Then count which option fits the most rows.
| Question | Favors buying when | Favors building when | Favors partnering when |
|---|---|---|---|
| Does a product already fit the process? | Yes, with little change | No, the process is unusual | Partly, and it needs custom work |
| Is the process a source of edge? | No, it is routine | Yes, it is core to how you win | Yes, but you lack the team |
| Can your data stay where it is? | The vendor works inside your setup | You keep full control | A partner builds inside your setup |
| Do you have people to run it? | Vendor runs it | You have engineers and an owner | You want a build and a handover |
| How fast do you need it? | Fastest, if it fits | Slowest | In between |
| Can you leave later? | Check export of logs and data | You own the code | Check that you own the code |
A vendor evaluation scorecard
Whether you buy a product or hire a partner, we ask the same questions. This is our checklist. Score each answer as clear, vague or missing.
- What does the agent do when it is unsure? A good answer names a stop and a hand-off.
- Show it choosing its own steps on a task you give it, not a fixed script.
- Which actions can it take, and can I limit each one?
- Who can see my data, where is it stored and is it used to train a model?
- What logs do I get, and can I export them and the code if I leave?
- How do you test it, and can I run my own test set?
- How do you handle prompt injection, where text in a document tries to give the agent new orders?
- What does it cost per run, and what do model and tool changes do to that cost?
- Who supports it after launch, and what happens when the model it uses changes?
The red flags we screen for: a demo that only runs a scripted path, no way to see logs, no answer on data use, no stop rule, and a claim that the agent is fully autonomous with no human point. The Risks section explains agent washing. Our guide to agent harnesses has a longer list of questions for technical teams, and it covers the runtime parts a vendor should be able to explain.
Risks to plan for
Every agent carries risks that a chatbot does not, because it can act. Plan for them before launch and give each an owner.
A risk register
The register uses plain words for likelihood and impact because we have no sourced numbers. The ratings are our judgment. Adjust them for your process.
| Risk | What it looks like | Likelihood | Impact | Owner | Control |
|---|---|---|---|---|---|
| Data leakage | The agent exposes private records to the wrong person or service | Medium | High | Security lead | Least-privilege access, no sensitive data in prompts without approval, logging |
| Wrong action | The agent posts, sends or changes the wrong thing | Medium | High | Process owner | Approval point before irreversible steps, undo path, checking step |
| Prompt injection | Text in a document or web page gives the agent new orders | Medium | High | Security lead | Treat outside text as untrusted, limit tools, approval for risky actions |
| Runaway cost | A loop or a large job burns tokens | Medium | Medium | Engineering lead | Step limits, budget alerts, effort settings |
| Drift | Quality falls as models, data or processes change | High | Medium | Agent owner | Test set on every change, sampling review |
| Vendor lock-in | You cannot move your logs, prompts or code | Medium | Medium | Procurement and engineering | Exit terms, exportable logs, your own test set |
| Compliance | The agent breaks a rule on records, privacy or audit | Low to medium | High | Compliance lead | Written policy, audit trail, review before launch |
In the OWASP LLM01 entry we read, prompt injection is input that changes a model's behavior in ways the developer did not intend. The entry notes that models make no architectural distinction between instructions and data, so there is no clean equivalent to the parameterized queries that stop SQL injection. Its advice includes least privilege per operation (give only the access the task needs) and explicit human confirmation before any privileged or irreversible action. Our conclusion: these controls limit damage, they do not remove the risk. See securing AI agents for the controls in more depth.
Why projects get canceled
In Gartner's press release of 2025-06-25, which we checked against a saved copy of the original, the prediction is that over 40% of agentic AI projects will be canceled by the end of 2027, with rising costs, unclear business value and weak risk controls named as the reasons. The release also warns of agent washing, where vendors relabel chatbots and robotic process automation (software that repeats fixed clicks and keystrokes) as agents. Gartner estimates only about 130 of the thousands of agentic AI vendors are genuine. What we do with that: we ask a vendor to show the agent choosing its own steps, not a fixed script. Our post on why enterprise AI projects fail lists the common causes. These are 2025 figures, so check Gartner's newsroom for newer ones.
Governance and the operating model
Governance is the set of rules that decide who may build, approve and change an agent. It does not need to be heavy. For a first agent, a one-page policy and a named owner are enough.
Governance basics
We read the NIST AI Risk Management Framework, a voluntary framework released in January 2023. Its core has four functions: Govern, which sets a risk culture across the others, Map, which frames the context and risks, Measure, which assesses and monitors them, and Manage, which assigns resources and plans responses. NIST says AI RMF 1.0 is being revised, so the details may change. In our view you do not need to adopt it in full to borrow the shape. Write down who is accountable, what could go wrong, how you will test and track the agent, and how you will respond to problems. Keep a list of every agent with its allowed actions, data, owner and last review date.
Who owns the agent after launch
Most agents fail slowly: quality drifts, nobody checks, and trust fades. Name the roles before launch. This split is our recommendation, so adjust it to your team.
| Role | Owns | Does this each week or month |
|---|---|---|
| Business owner | Results and the list of allowed actions | Reviews the numbers and approves any change in scope |
| Agent engineer | The build, tools, prompts and test set | Runs tests on every change, handles incidents, updates for new models |
| Reviewer | Quality of output | Samples cases and records them as right or wrong |
| Security and compliance lead | Access and audit trail | Checks permissions and logs, signs off on new tools |
| Finance partner | The baseline and the payback case | Compares actual cost and time saved with the plan |
Hold a short monthly review of the sample error rate, cost per run, escalations and share handled alone. Decide to keep, change or stop.
How to test and configure an agent for accuracy
An agent that works in a demo can fail on real cases. Anthropic's page recommends extensive testing in sandboxed environments (isolated setups that cannot touch live systems) with appropriate guardrails, because autonomous agents can compound errors and raise costs.
Configure for accuracy
- Grounding. Tell the agent to answer only from your documents and data, and to say when it cannot find the answer.
- Citations. Make it show which document or record each claim came from, so a reviewer can check in seconds.
- A verification step. Add a second pass, by a second model or a rule, that checks the result against the source before anything is sent. The code security case study above is built this way, as its page describes, with runtime validation before a finding is reported.
- Human review. Route low-confidence or high-impact cases to a person.
- Evals. Keep a test set and run it on every change to the prompt, model or tools.
An evaluation plan
The cadences below are our recommendation. Adjust them to the risk in your process.
| What to measure | How | When |
|---|---|---|
| Correct result rate | Run the test set and compare with the known right answers | Before launch, and on every change to prompt, model or tools |
| Cost per run | Add model and tool costs from the run logs | Daily in the pilot, then weekly |
| Time per case | Compare the agent's time and the human review time with the baseline | In the shadow run and the pilot |
| Escalation rate | Count cases handed to a person and why | Weekly |
| Sample error rate | A person marks a random sample as right or wrong | Weekly at first, then monthly |
| Safety checks | Try inputs that attempt to give the agent new orders, and requests outside its scope | Before launch, and after any new tool |
| Drift | Re-run the test set against new model versions | When the vendor changes the model, and on a schedule |
Model and thinking settings
Two settings affect cost and quality. We read both in Anthropic's documentation on 2026-10-27, and other vendors differ.
Thinking is when the model works through a problem before it answers. On most current Claude models (Opus 4.7 and later, Sonnet 5 and later, Haiku 5.5), the old fixed thinking budget returns an error, and you use adaptive thinking with an effort level instead. Effort controls how many tokens the model spends, from low to max. Lower levels save cost and time, and Anthropic suggests low for simple tasks such as subagents. We would test a lower level on the eval set before accepting it.
Temperature controls how random the output is. On the newest Claude models, including Opus 5.5, Sonnet 5.5 and Haiku 5.5, a non-default temperature, top_p or top_k value returns an error, whether or not thinking is on. From API version 2026-10-21, any of these settings returns an error on every model. For accuracy, we rely on the evals and the checking step, not a temperature setting.
A 30, 60 and 90 day plan
The strip shows what to decide, build, test and measure by week 4, week 8 and week 13. For the step from a pilot to a live system, see proof of concept versus production AI.
A three-column strip showing what is decided, built, tested and measured by week 4, week 8 and week 13 of a first agent project. It is our recommendation, not a standard.
Days 1 to 30: choose and baseline
Score your candidate tasks with the First-Agent Score. Pick one. Record the current hours, error rate and cost per case. Name an owner and a reviewer, and write down the success measure. As a rule of thumb from us, not a standard, collect 50 to 100 real examples with the right answers for your first test set.
Days 31 to 60: build and test
Decide which tools and data the agent may touch and where a person approves. Build one agent with the fewest tools it needs. Add a checking step and a human approval point. Run the test set on every change and fix what fails. Run it beside the current process without acting on its output. This is called a shadow run.
Days 61 to 90: pilot and decide
Let the agent handle a small share of live cases with a person approving risky actions and a sample of the rest reviewed. Compare against the baseline you recorded. Decide to expand, change or stop.
These are planning steps, not a promised timeline.
When an agent is the wrong tool
An agent is not the answer to every process problem. Choose something simpler in these cases.
- The steps never change. A fixed rule or a script is cheaper, faster and easier to audit.
- The task needs a person's judgment or duty. Legal, medical or safety decisions belong to a qualified person. An agent can prepare the file, not make the call.
- A wrong answer cannot be undone. If you cannot add an approval point or an undo path, do not automate the step.
- There is no clear right answer. You cannot test what experts cannot define.
- The data is not reachable or clean. Fix the data first.
Common mistakes
- Starting with the hardest task. Teams pick a prestige process with low volume and high risk. Pick a frequent, low-risk one first.
- No baseline. Without hours and error rates from before, you cannot show a result. You will argue about it instead.
- Approval on everything, or on nothing. The first trains people to click through. The second invites a costly error. Use bands.
- No owner after launch. Quality drifts when nobody is responsible. Name the owner before go-live.
Glossary
| Term | Meaning |
|---|---|
| Agent | Software that works toward a goal by choosing its own steps and using tools. |
| Token | A small chunk of text a model reads or writes. Usage is billed per token. |
| Eval | A fixed set of test cases with known right answers, run to score an agent after each change. |
| RPA | Robotic process automation: software that repeats fixed clicks and keystrokes, with little or no judgment. |
| ERP | Enterprise resource planning: the system that holds orders, invoices, stock and finance records. |
| PO | Purchase order: a buyer's written request to a supplier to supply goods or services. |
| Grounding | Making the agent answer only from supplied documents or data. |
| Human in the loop | A person who approves, reviews or receives hand-offs at set points. |
| Orchestration | How several agents or steps are coordinated to finish one job. |
| Subagent | A helper agent that a lead agent starts to do one part of a job. |
| Prompt injection | Text in an input that tries to give the agent new orders. |
| Shadow run | Running the agent beside the current process without acting on its output. |
| LoCoMo | A public test built on long conversations, used to score how well a memory system recalls facts. |
| Adaptive thinking | A model mode where the model decides how long to think before answering, steered by an effort level. |
| top_p and top_k | Settings that limit which next words a model may pick: top_k keeps the k most likely, and top_p keeps the smallest set whose combined chance reaches p. |
Next step
If you want help scoping a first agent, see our AI agent development service, our AI automation agency page, or our AI implementation services. You can also start with the First-Agent Score above and bring us the top candidate.
Frequently asked questions
Automation follows steps you wrote in advance and breaks when the input changes. An agent is given a goal and tools, and it chooses the next step itself. Use automation when the steps never change. Use an agent when the path depends on what it finds.
Usually not at first. A single agent with the right tools handles most back-office tasks. Add a second agent only when one job has clearly separate parts, such as one agent that gathers records and another that checks them. Each extra agent adds cost and failure points.
It depends on the number of systems the agent must reach, the cost of a wrong answer, the volume of work, and how much checking it needs. Running costs matter as much as the build, including model usage, testing and maintenance. To see how pricing works at EdgeFirm, visit the pricing page.
Buy when a vendor product already fits your process. Build when the process is unique and gives you an edge. Partner when you want a custom build but lack the team to run it. Our build, buy or partner matrix post walks through the decision.
Yes. Agents can state wrong facts or take a wrong action. You reduce this with grounding in your own documents, a separate checking step, human review on risky actions, and a test set you run on every change.
A task that is frequent, has a clear right answer, uses data the agent can reach, and costs little when it errs. Invoice matching and support ticket triage often fit. Rank your options with the First-Agent Score in this guide and pick the top one.
It depends on your volume, the time each task takes today, the share the agent handles alone and the running cost. Our calculator shows how the answer moves when you change those inputs. Its starting numbers are an illustration, not a benchmark or a client result.
One named business owner, who is accountable for results and for the list of actions the agent may take. An engineer keeps it running, and a reviewer samples its output. Without an owner, nobody notices when quality drifts.
Last updated Oct 27, 2026. Spotted something out of date? Tell us at hello@edgefirm.io.
Where we can help
Read next
- What Is an Agent Harness? How to Build One for Production
- The Reality of AI Agents in 2026: What Nobody Tells You Before You Build
- Agentic AI for Enterprise: What Your AI Vendors Aren't Telling the C-Suite
- AI Agent vs Chatbot: Which One Does Your Business Actually Need?
- You Shipped an AI Agent. Did Anyone Think About Security?
- How Much Does It Cost to Build an AI Agent?
- Enterprise AI ROI: How to Calculate the Business Case Before You Build
- Build vs. Buy vs. Partner: The AI Decision Matrix
- What is MCP? The Model Context Protocol Explained for Business Leaders
- AI Proof of Concept vs Production AI: What Should Enterprises Build First?
- Why Enterprise AI Projects Fail and How to Avoid the Common Mistakes
Written by
CEO & Co-Founder
Raza brings 9+ years in finance, audit and business management to EdgeFirm. He began as a Senior Auditor at Hashmi & Co. Chartered Accountants and runs Work Hall as General Manager and Company Secretary, working with over 100 startups and companies.
Written by
Co-Founder & Director of Engineering
Usman brings 8+ years of experience building enterprise systems. He specializes in system architecture, DevOps, and data pipelines that power production AI.