LLM Evaluation: Evals, Tracing and Fact-Checking for Production Agents

Most teams ship an AI agent after a handful of good demos. Then a prompt changes, a model updates or a new kind of question arrives, and quality moves without anyone noticing. Evaluation is how you notice. This guide is our engineers' report on how to test, trace and fact-check an agent for production. It sets out what we read, ran and computed, and what we recommend as a result. It is written for two readers: leaders who need to know what to ask for, and engineers who need to build it.

You will get a five-rung model for maturity, which we call the Eval Ladder. You will also get tested code that runs offline, setup steps for Langfuse and LangSmith checked against their own documentation, hallucination figures from primary sources with dates, and a production checklist. Where we give our own engineering judgment instead of a sourced fact, we label it.

What LLM evaluation is, and what it is not

An evaluation, or eval, is a test for an AI system. You give the system an input, you know what a good result looks like, and you score what comes back. That sounds like ordinary software testing, and it shares the idea. The difference is that the output varies from run to run and is often free text, so a score is rarely a plain pass or fail on exact equality.

In the Anthropic engineering article we read on 2026-10-27, the authors split evals into two kinds that answer different questions, and we use the same split. Capability evals ask what the agent can do well, and they should start at a low pass rate so there is room to improve. Regression evals ask whether the agent still handles everything it used to, and they should sit near 100%. A capability eval that reaches a high pass rate can graduate into the regression set.

Evals, observability and fact-checking: three jobs

We split the topic into three jobs that share one data trail. This split is our engineering judgment, not a standard.

• Evals answer the question "did this change help?" They run on a fixed set of cases, usually before release, and produce scores you can compare.
• Observability answers "what did the agent do?" It records every model call, tool call, cost and delay, so you can replay a bad run.
• Fact-checking answers "is this answer supported?" It checks claims against sources, in code where possible, and decides whether to answer, refuse or escalate to a person.

The three share traces. A trace is the record of one run. Scores attach to traces. Failed runs become test cases. Fact-checks write their results back as scores. If you build the three on separate data, you will end up reconciling them by hand.

The Eval Ladder

Teams do not need every practice on day one. We describe five rungs, each adding one capability. The rungs are our own model, built from practices we read in the sources below. They are not a standard, and we make no claim about how many teams sit on each rung.

Rung 1 alone shows what happened. It cannot say whether a change helped. Rung 2 is where most of the value starts, and rung 3 is where it stops depending on memory.

Eval types and when to use each

Three kinds of grader score an output. They differ in cost, flexibility and trust. Most good systems use all three, each where it fits.

We took the strengths and weaknesses in the table from Anthropic's description of the three grader types, and they match how we would use each one.

Code graders

A code grader is a function. It checks whether the answer contains a required fact, whether the output matches a schema, whether a tool was called, whether a quote appears in the source, or whether cost stayed under a cap. Anthropic lists string matching, binary tests, static analysis and tool-call checks as examples and calls these graders fast, cheap, objective and reproducible, but brittle to valid variations that do not match the expected pattern.

Our rule is to use code graders first. If you can write the check as code, do. The offline harness later in this guide has three: answer match, citation match and refusal check.

Regression and capability evals, and the pass^k measure

An agent can give a different result each time it sees the same input. A single run per case hides that. We checked the definitions in Anthropic's article, which describes two measures for repeated trials. The measure pass@k is the likelihood of at least one correct answer in k attempts, and it rises as k grows. The measure pass^k is the probability that all k trials succeed, and it falls as k grows. The same article gives an example: a 75% success rate per trial over three trials gives roughly 42% for passing all three. We checked the arithmetic. 0.75 cubed is 0.42, which matches.

The choice depends on the product. Use pass@k when one success is enough, such as a coding agent where you can try several times and pick one. Use pass^k when users need it right every time, such as a support agent. The article recommends pass^k when consistency matters. Our harness prints both so you can see the gap.

LLM-as-judge and its biases

A model judge reads an output and scores it against a rubric. It is the practical way to grade tone, completeness and whether a claim is supported, none of which a string match can do. The research basis we start from is the MT-Bench paper. In the version we read, GPT-4 as a judge reached over 80% agreement with human preferences, the same level as agreement between humans.

The same paper names the weaknesses we plan around. It lists position bias, verbosity bias and self-enhancement bias, plus limited reasoning ability. In our words, a judge may prefer the answer shown first, prefer the longer answer, and prefer text written by its own model family. Those are tendencies, not guarantees, and the paper proposes mitigations for some of them.

We also read a June 2026 study of 21 judge models from nine providers, about 541,000 judgments in all. What we take from it is that the calibration step below matters. Raw agreement overstates quality: switching from exact-match agreement to Cohen's kappa, which corrects for chance, lowered MT-Bench agreement by 33 to 41 percentage points. Two production judges had test-retest reliability above 0.95 and still showed position bias above 0.10, so a judge can be consistent and still biased. Verbosity bias was small, below 0.011, under one pairwise rubric. We recommend reporting kappa, not only percent agreement, when you check your own judge.

In the multi-agent post, Anthropic's research team reports that it tried several judge designs. It found that a single model call with a single prompt, returning scores from 0.0 to 1.0 and a pass or fail grade, was the most consistent and closest to human judgement. The rubric covered factual accuracy, citation accuracy, completeness, source quality and tool efficiency.

Our rules for judges, stated as engineering judgment and not as a standard:

• Give the judge the source text and the claim, not just the answer. A judge that cannot see the source can only guess.
• Ask for a short reason and a score. Read the reasons when scores look odd.
• Calibrate before you trust. Label 30 to 50 outputs by hand, run the judge, and compare.

The Langfuse judge page, which we read on 2026-10-27, gives the same advice: a judge is a prompt, so check its scores against human labels before relying on it, and use its Score Analytics view to compare judge and human scores.

Human review

People remain the reference, and two passages we read agree. Anthropic's research team wrote that human evaluation catches what automation misses, and that manual testing stays essential even with automated evals. Anthropic's evals article gives a related instruction that we follow: read the transcripts. It says you will not know whether your graders work unless you read the transcripts and grades from many trials, because transcripts show whether the agent made a real mistake or the grader rejected a valid answer.

Build the test set from production traces

A test set built from imagination tests what you imagined. A test set built from real traces tests what users do. In the documentation we read, Langfuse and LangSmith describe the same practice. Langfuse says datasets can be built from interesting examples you find in production. LangSmith says datasets can come from curated cases, historical production traces or synthetic data.

Where cases come from

Four sources cover most needs.

1. Failures you find. A user complaint, a wrong answer someone flagged, a run that hit a cap. Each becomes a case with the right answer written down. Anthropic recommends drawing early tasks from real failures and manual checks.
2. A random sample of normal traffic. Failures alone make the set pessimistic. Add a plain sample so the set also stands for ordinary use.
3. Hard cases you design. Questions the sources cannot answer, questions that tempt the agent to guess, and hostile inputs. These test refusal and safety.
4. Synthetic cases. A model can draft variations of real cases. Treat them as a supplement and read them. They inherit the habits of the model that wrote them.

How many cases

We would start small. Anthropic's article says 20 to 50 simple tasks drawn from real failures is a great start, because early changes usually have large effects that a small sample can detect. Its multi-agent team started with about 20 queries that matched real usage. A small set does not give you fine-grained percentages. It gives you a fast signal. Grow it as you find failures.

Each case needs four fields: an input, the expected answer or the facts it must contain, the source that supports it, and a label for the kind of case, such as normal, hard or must-refuse. The harness below uses exactly these fields.

Keep the set trustworthy

Three habits keep a test set useful.

• Version it. Store it in the repository next to the code, with a changelog. A score only means something against a known set.
• Do not tune on it blindly. If you edit prompts until every case passes, the set stops predicting new traffic. Hold back a share of cases that you never look at while you tune. This is a common practice in machine learning, and we recommend it here.
• Remove private data. Production traces hold personal data. Strip or replace it before a case goes into a shared dataset, and apply the same access rules as the source system.

An offline eval harness you can run

This is a small eval harness with no network and no API key. The agent and the judge are both stubs, so the output repeats exactly. The structure is what matters: a dataset, code graders, a judge, repeated trials, and a gate. To use a real agent, replace agent_v1 and agent_v2 with functions that call it. To use a real judge, replace stub_judge with a model call.

"""An offline eval harness. No network, no API key. The agent and the judge are both stubs, so the output is repeatable. Swap in a realagent function and a real judge call and the rest of the file stays the same. What it shows:  1. A small dataset with known good answers and the source text each answer must come from.  2. Code graders (cheap, exact): answer match, citation check, refusal check.  3. A stub judge, and a check of the judge against human labels (agreement and Cohen's kappa).  4. pass@k and pass^k over repeated trials.  5. A release gate that fails (exit code 1) when a candidate is worse than the baseline.  6. Latency and cost per task from the run records."""import statisticsimport sys SOURCES = {    "refund-policy": "Refunds are issued within 14 days of purchase for unused licenses.",    "support-hours": "Support is open Monday to Friday, 09:00 to 17:00 UTC.",    "data-region": "Customer data is stored in the EU region by default.",} # Each case: a question, the expected answer fragment, the source that must be cited.# `should_refuse` marks a question the sources cannot answer.DATASET = [    {"id": "q1", "question": "How long do I have to ask for a refund?", "expect": "14 days", "cite": "refund-policy"},    {"id": "q2", "question": "When is support open?", "expect": "09:00 to 17:00 UTC", "cite": "support-hours"},    {"id": "q3", "question": "Where is my data stored?", "expect": "EU", "cite": "data-region"},    {"id": "q4", "question": "Do you offer a 24/7 phone line?", "should_refuse": True},    {"id": "q5", "question": "Can I get a refund after 30 days?", "expect": "14 days", "cite": "refund-policy"},] PRICE_PER_1K_TOKENS = 0.002  # made-up unit price so cost has something to count  def agent_v1(question: str, trial: int) -> dict:    """Baseline stub. Answers from sources, cites them, refuses when it has nothing."""    q = question.lower()    if "refund" in q:        key = "refund-policy"    elif "support" in q:        key = "support-hours"    elif "data" in q:        key = "data-region"    else:        return {"text": "I do not have that information in the documents.", "cites": [], "latency_s": 0.4, "tokens": 220}    return {"text": SOURCES[key], "cites": [key], "latency_s": 0.8 + 0.05 * trial, "tokens": 640}  def agent_v2(question: str, trial: int) -> dict:    """Candidate stub with two planted regressions: it invents an answer for q4,    and it cites the wrong source for q3 on every other trial."""    out = agent_v1(question, trial)    if "phone" in question.lower():        return {"text": "Yes, our phone line is open 24/7.", "cites": [], "latency_s": 0.9, "tokens": 700}    if "data" in question.lower() and trial % 2 == 1:        out["cites"] = ["support-hours"]    out["latency_s"] += 0.6    out["tokens"] += 300    return out  # ---- code graders: deterministic, cheap, run on everything ---- def grade_answer(case, out):    if case.get("should_refuse"):        return "do not have" in out["text"].lower()    return case["expect"].lower() in out["text"].lower()  def grade_citation(case, out):    if case.get("should_refuse"):        return out["cites"] == []    # The cited source must be the right one, and it must support the answer text.    return out["cites"] == [case["cite"]] and case["expect"].lower() in SOURCES[case["cite"]].lower()  def code_pass(case, out):    return grade_answer(case, out) and grade_citation(case, out)  # ---- stub judge: stands in for an LLM-as-judge ---- def stub_judge(case, out):    """A fake judge. It is lenient on fluent answers: it says pass when the answer is    long and confident, even if it is wrong. That plants a verbosity-style bias so    the calibration step below has something to find."""    if case.get("should_refuse"):        return len(out["text"]) < 60 and "do not have" in out["text"].lower()    return len(out["text"]) > 30  HUMAN_LABELS = {  # what a person decided for agent_v2, trial 1 (1 = pass)    "q1": 1, "q2": 1, "q3": 0, "q4": 0, "q5": 1,}  def cohen_kappa(a, b):    n = len(a)    po = sum(x == y for x, y in zip(a, b)) / n    pa1, pb1 = sum(a) / n, sum(b) / n    pe = pa1 * pb1 + (1 - pa1) * (1 - pb1)    return 1.0 if pe == 1 else (po - pe) / (1 - pe)  def run_suite(agent, trials=4):    rows = []    for case in DATASET:        for t in range(trials):            out = agent(case["question"], t)            rows.append({"id": case["id"], "trial": t, "ok": code_pass(case, out), "out": out})    return rows  def pass_metrics(rows, trials=4):    per_case = {}    for r in rows:        per_case.setdefault(r["id"], []).append(r["ok"])    at_k = sum(any(v) for v in per_case.values()) / len(per_case)       # pass@k    hat_k = sum(all(v) for v in per_case.values()) / len(per_case)      # pass^k    rate = sum(r["ok"] for r in rows) / len(rows)    return rate, at_k, hat_k  def ops_metrics(rows):    lat = sorted(r["out"]["latency_s"] for r in rows)    p95 = lat[min(len(lat) - 1, int(round(0.95 * (len(lat) - 1))))]    cost = sum(r["out"]["tokens"] for r in rows) / 1000 * PRICE_PER_1K_TOKENS / len(rows)    return statistics.median(lat), p95, cost  def main():    trials = 4    base, cand = run_suite(agent_v1, trials), run_suite(agent_v2, trials)     print("== code graders")    for name, rows in (("v1 baseline", base), ("v2 candidate", cand)):        rate, at_k, hat_k = pass_metrics(rows, trials)        p50, p95, cost = ops_metrics(rows)        print(f"{name}: trial pass {rate:.0%}  pass@{trials} {at_k:.0%}  pass^{trials} {hat_k:.0%}  "              f"p50 {p50:.2f}s  p95 {p95:.2f}s  cost/task ${cost:.5f}")     print("== judge calibration (agent v2, trial 1)")    ids = [c["id"] for c in DATASET]    by_case = {c["id"]: c for c in DATASET}    outs = {r["id"]: r["out"] for r in cand if r["trial"] == 1}    judge = [int(stub_judge(by_case[i], outs[i])) for i in ids]    human = [HUMAN_LABELS[i] for i in ids]    agree = sum(j == h for j, h in zip(judge, human)) / len(ids)    print(f"judge  : {judge}")    print(f"human  : {human}")    print(f"agreement {agree:.0%}  kappa {cohen_kappa(judge, human):.2f}")    missed = [i for i, j, h in zip(ids, judge, human) if j == 1 and h == 0]    print(f"judge passed what a human failed: {missed}")     print("== release gate")    b_rate, _, b_hat = pass_metrics(base, trials)    c_rate, _, c_hat = pass_metrics(cand, trials)    problems = []    if c_rate < b_rate - 0.02:        problems.append(f"trial pass fell from {b_rate:.0%} to {c_rate:.0%}")    if c_hat < b_hat:        problems.append(f"pass^{trials} fell from {b_hat:.0%} to {c_hat:.0%}")    refusal_fail = [r["id"] for r in cand if by_case[r["id"]].get("should_refuse") and not r["ok"]]    if refusal_fail:        problems.append(f"invented an answer when it should refuse: {sorted(set(refusal_fail))}")    if problems:        print("BLOCK release")        for p in problems:            print(" -", p)        sys.exit(1)    print("PASS release")  if __name__ == "__main__":    main()

We ran it with Python 3.13.3 on 2026-10-27 and got this output:

== code gradersv1 baseline: trial pass 100%  pass@4 100%  pass^4 100%  p50 0.85s  p95 0.95s  cost/task $0.00111v2 candidate: trial pass 70%  pass@4 80%  pass^4 60%  p50 1.45s  p95 1.55s  cost/task $0.00178== judge calibration (agent v2, trial 1)judge  : [1, 1, 1, 0, 1]human  : [1, 1, 0, 0, 1]agreement 80%  kappa 0.55judge passed what a human failed: ['q3']== release gateBLOCK release - trial pass fell from 100% to 70% - pass^4 fell from 100% to 60% - invented an answer when it should refuse: ['q4']

In our test run the data is invented, the two stub agents have planted faults and the unit price is made up. The percentages describe the harness, not any real model. What we read in the output is how the pieces behave.

The harness also prints median and 95th percentile latency and cost per task. We come back to those in the section on cost and latency. The worked example near the end reads this output as a story.

Observability: traces, spans and scores

Observability is the record of what the agent did. A trace is one run. It is made of spans, one per step: a retrieval, a model call, a tool call. Spans nest, so a root span for the whole request holds the steps inside it.

Our agent harness guide calls this record the sixth part of its anatomy, trace and eval hooks. In the multi-agent post, Anthropic's team says full production tracing let them diagnose why agents failed and fix issues systematically. They also say they monitored agent decision patterns and interaction structures but not the contents of individual conversations, to protect user privacy. For the loop, limits and permissions that produce these traces, read our agent harness guide.

What to record

Record enough to replay a bad run and to compute cost, and no more.

• A run ID, a session ID and a user ID or hashed ID.
• Every model call with the model name, version, token counts and latency.
• Every tool call with its name, arguments, result and time.
• Retrieved documents, with their ids and scores, for any answer that uses retrieval.
• Scores from code checks, judges and users, attached to the run or the span.
• The version of the prompt, the test set and the code that produced the run.

Message content is the sensitive part. The OpenTelemetry conventions treat it that way: the attributes for input messages, output messages, system instructions and tool definitions are marked opt-in. We follow the same default. Record content only where you need it for debugging, protect it like the source data, and set a retention period.

OpenTelemetry for LLM traces

OpenTelemetry is the open standard for traces, and it has conventions for AI systems. The GenAI semantic conventions define spans for inference, embeddings, retrievals, tool execution and memory, and attributes such as gen_ai.operation.name, gen_ai.request.model, gen_ai.usage.input_tokens and gen_ai.usage.output_tokens. On 2026-10-27 the spans page carried the status Development, which means the names can still change. The conventions now live in their own repository, and the old page on the main OpenTelemetry site says it is no longer maintained there.

Our recommendation is to use OpenTelemetry when you already run it for the rest of your system or when you want to avoid tying your code to one vendor, and to expect to update attribute names as the conventions mature. Both tools in the next sections accept OpenTelemetry data, according to their documentation. Langfuse takes OTLP over HTTP, with both HTTP/JSON and HTTP/protobuf, at the path /api/public/otel, with Basic Auth built from the public and secret keys, and it does not support gRPC yet. LangSmith takes OpenTelemetry at https://api.smith.langchain.com/otel with an x-api-key header, and has a LANGSMITH_OTEL_ENABLED=true setting for LangChain and LangGraph.

Langfuse: setup and how-tos

Langfuse is an LLM engineering platform with tracing, datasets, evaluation and prompt management. As of our check on 2026-10-27, the latest Python package was langfuse 5.0.4, uploaded on 2026-10-21. Langfuse announced on 2026-01-16 that it joined ClickHouse, and said it stays open source and self-hostable with no planned licensing changes. The repository says it is MIT licensed except for its ee folders.

Set up tracing in Python

1. Install and set keys

   Run pip install langfuse. Set LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY and LANGFUSE_BASE_URL in your environment. The documented cloud hosts are https://cloud.langfuse.com for the EU, https://us.cloud.langfuse.com for the US, https://jp.cloud.langfuse.com for Japan and https://hipaa.cloud.langfuse.com for the HIPAA region. A self-hosted install uses its own URL.

2. Mark the steps

   Put the @observe decorator on functions, or open a span with start_as_current_observation. The decorator takes a name and a type, such as as_type="generation" for a model call. To reduce overhead from large payloads, pass capture_input=False and capture_output=False.

3. Record the model and the tokens

   Open a generation observation with the model name, then call update with usage_details, for example input and output token counts. Langfuse uses these to show cost.

4. Tag the run

   Wrap the request in propagate_attributes to set a user ID and session ID on every observation inside it.

5. Flush in short-lived processes

   Call langfuse.flush() before a script or serverless function exits, so queued events are sent.

The sample below follows those steps. Names come from the documentation we read on 2026-10-27. The instrumentation page does not state a Python SDK version, so match the names to the version you install.

"""Trace an agent run with the Langfuse Python SDK (v5 style) and score it. NOT RUN for the guide: it needs a Langfuse project and keys. We only checked that it parses.Names follow the Langfuse docs we read on 2026-10-27. Check them against the SDK version you install. Setup:    pip install langfuse    export LANGFUSE_PUBLIC_KEY=pk-lf-...    export LANGFUSE_SECRET_KEY=sk-lf-...    export LANGFUSE_BASE_URL=https://cloud.langfuse.com   # EU. US: https://us.cloud.langfuse.com"""from langfuse import get_client, observe, propagate_attributes langfuse = get_client()  @observe(name="search-docs")def search_docs(query: str) -> list[str]:    # A retrieval step. Its input and output are captured as an observation.    return ["Refunds are issued within 14 days of purchase for unused licenses."]  def answer(question: str, passages: list[str]) -> str:    # Replace the body of the with-block with your real model call. Record the model and the    # token counts on the generation so Langfuse can show cost per run.    with langfuse.start_as_current_observation(as_type="generation", name="llm-call", model="your-model-name") as gen:        text = passages[0]        gen.update(output=text, usage_details={"input_tokens": 640, "output_tokens": 40})    return text  @observe(name="support-agent")def handle(question: str) -> str:    passages = search_docs(question)    return answer(question, passages)  def run(question: str, user_id: str, session_id: str) -> str:    # propagate_attributes tags every observation created inside the block.    with propagate_attributes(user_id=user_id, session_id=session_id):        with langfuse.start_as_current_observation(as_type="span", name="request", input={"question": question}) as root:            result = handle(question)            root.update(output={"answer": result})            # A score from a code check. Online judges and human review write scores the same way.            root.score_trace(name="cites_source", value=1.0, comment="answer matches the retrieved passage")    return result  if __name__ == "__main__":    print(run("How long do I have to ask for a refund?", user_id="u-123", session_id="s-1"))    langfuse.flush()  # short-lived scripts must flush before exit

Trace a LangChain or LangGraph app

Langfuse traces LangChain through a callback handler. Install with pip install langfuse langchain langchain_openai langgraph, set the same environment variables, import CallbackHandler from langfuse.langchain, and pass the handler in the config argument as {"callbacks": [langfuse_handler]}. For agents, pass the same config to agent.invoke. The docs note that Python SDK v5 requires langchain-core>=1.

To set a user, session and tags, add them to metadata under the keys langfuse_user_id, langfuse_session_id and langfuse_tags. In short-lived scripts, call get_client().flush() to send queued events, or get_client().shutdown() before exit so nothing is lost. Our second sample shows this, together with the LangSmith equivalent.

"""Trace a LangChain or LangGraph call with Langfuse, and with LangSmith. NOT RUN for the guide: it needs keys and a model provider. We only checked that it parses.Names follow the Langfuse and LangSmith docs we read on 2026-10-27. Langfuse:  pip install langfuse langchain langchain_openai langgraphLangSmith: pip install langsmith langchain_openai langgraph, then           export LANGSMITH_TRACING=true           export LANGSMITH_API_KEY=...           (LangSmith needs no code change for LangChain and LangGraph. The env vars are enough.)"""from langfuse import get_clientfrom langfuse.langchain import CallbackHandlerfrom langsmith import traceable langfuse = get_client()langfuse_handler = CallbackHandler()  def ask_with_langfuse(chain, topic: str):    return chain.invoke(        {"topic": topic},        config={            "callbacks": [langfuse_handler],            "metadata": {                "langfuse_user_id": "u-123",                "langfuse_session_id": "s-1",                "langfuse_tags": ["support-agent", "release-2026-10"],            },        },    )  # For a plain function (a tool, or a non-LangChain SDK call), LangSmith uses a decorator.@traceable(run_type="tool", name="Search Tool")def search(query: str) -> str:    return "Refunds are issued within 14 days of purchase for unused licenses."  if __name__ == "__main__":    print(search("refund window"))    get_client().flush()

Scores, datasets and experiments

Langfuse describes evaluation as online, on live production traces to track quality, and offline, on a dataset before you ship a change. The pieces are datasets, experiments, scores and annotation queues.

Scores come from code, judges, people or users. In Python you can attach one from inside a span with span.score_trace(name=..., value=..., data_type=...), or use langfuse.create_score with a trace ID. For a boolean score, pass data_type="BOOLEAN" because a bare number is read as numeric.

For offline runs, langfuse.run_experiment takes a name, data, a task function and a list of evaluators. Each evaluator returns an Evaluation with a name, a value and an optional comment, and run-level evaluators can compute aggregates such as an average. The page mentions a migration from v4 to v5 that removed the dataset run helpers, so we would match the examples to the SDK version in use.

For release gates, Langfuse documents CI/CD experiments: a RegressionError raised in your experiment script makes the GitHub Action fail the job. That is the same idea as the exit code in our harness.

LLM-as-a-judge in Langfuse

To run a judge on live traffic, connect a model through an LLM connection, create an evaluator from a template, write the judge prompt with variables, and choose a score type of numeric, categorical or boolean. Then map variables to the data you want, test on sample observations, and attach the evaluator to a rule.

Two details from the page matter most for planning, as we read it on 2026-10-27:

• Observation-level evaluators are the recommended target for live production data, and they see only the observation they matched, not its children. To judge a whole request, target the root observation and write the context onto it.
• Trace-level evaluators are deprecated. Existing ones keep running until the v4 cutover on 2026-11-16, then stop producing results. Plan to move them to observation-level evaluators before that date.

LangSmith: setup and how-tos

LangSmith is the observability and evaluation product from the LangChain team. As of 2026-10-27 the latest Python package, langsmith, was version 0.14.13, uploaded the same day.

Set up tracing

For a LangChain or LangGraph app, tracing needs environment variables and no code change. The docs show LANGSMITH_TRACING=true and LANGSMITH_API_KEY, with LANGSMITH_AGENT_ID and LANGSMITH_AGENT_ENVIRONMENT for agent-based workspaces, and say LangChain modules are traced automatically. Set LANGSMITH_WORKSPACE_ID if your key belongs to more than one workspace, and set LANGSMITH_ENDPOINT if your account is outside the US region. The page gives https://eu.api.smith.langchain.com for the EU.

For a function that is not a LangChain module, such as a tool or a call to another SDK, import traceable from langsmith and decorate it, for example @traceable(run_type="tool", name="Search Tool"). LangSmith nests the traces in the right context. Our second code sample includes this decorator.

Datasets, offline and online evaluation

LangSmith splits evaluation into offline and online. Offline runs on curated datasets during development, to compare versions and catch regressions. Online evaluates real user interactions in real time on live traffic. It lists four evaluator types: human, code, LLM-as-judge and pairwise, where outputs are compared head to head.

Online evaluators are set up in the Evaluators tab of a tracing project. You can filter which runs trigger the evaluator, by user feedback, by a specific tool call or by metadata, and set a sampling rate. A rate of 0.1 applies the evaluator to 10% of matching runs, which controls cost. A spend limit caps the weekly LLM cost per evaluator, and evaluators pause when it is reached. Backfilling past runs is possible only when the rule is created.

Langfuse and LangSmith compared

This table compares what each product's documentation states, as we read it on 2026-10-27.

Our read of the documentation, not a benchmark: if your agents are built on LangChain and LangGraph and you want the least setup, LangSmith is the shorter path. If you need self-hosting, an MIT core or a vendor-neutral OpenTelemetry route, look at Langfuse first. We recommend a one-week trial of each on a real agent before you commit, because the hard parts are your data rules and your team's habits, not the feature list.

Why models hallucinate, and the measured rates

A hallucination is output that sounds right and is not supported. OWASP's 2026 edition defines the risk as an LLM or LLM-enabled application producing incorrect, incomplete, unsupported or misleading information that appears credible enough to influence a decision or an agent action, and notes that overreliance on fluent, confident output makes the impact worse.

Why it happens

We read the paper by Kalai, Nachum, Vempala and Zhang. Its argument is that language models hallucinate because training and evaluation reward guessing over admitting uncertainty. The authors say errors arise statistically, and persist because most benchmarks grade an uncertain answer as badly as a wrong one. They propose changing how existing benchmarks score responses, and write that models are optimized to be good test-takers.

OWASP released its 2026 LLM Top 10 on 2026-08-03, where Misinformation is LLM07. It lists causes that include hallucination, incomplete or stale context, weak grounding, ambiguous prompts, biased or corrupted data, misleading summaries and unvalidated tool outputs. For your agent, we turn these into four practical causes, which is our own list:

1. The answer is not in the sources you gave it, and it guesses.
2. The right source was in the data but retrieval did not find it, so the model answers from memory.
3. The source was found, but the model wrote something the source does not say.
4. The task rewards a confident answer, so uncertainty is never an option.

Each cause has a different fix. Causes 1 and 4 call for a refusal path. Cause 2 is a retrieval problem, covered in our RAG guide and in our post on enterprise RAG architecture. Cause 3 calls for citations and a check in code.

Measured rates

Published rates depend on the task, the data and the judge, so no single number describes your system. The table lists what we read on the primary sources on 2026-10-27.

What we see in these numbers, in two points. First, the Vectara figure counts summaries of supplied text, a narrower setting than open questions, and it still finds more than 10% for several frontier models. The leaderboard defines the hallucination rate as 100 minus the factual consistency rate, as scored by Vectara's HHEM-2.3 model. Second, the Stanford authors state that vendors' claims of being hallucination-free are overstated, and the tools they tested were retrieval-based. Our read: this is a warning for any buyer. A tool that retrieves is not exempt from the problem.

We would not use these numbers as a forecast for your system, because task, data and judge all differ. Run your own eval on your own questions and report that rate.

Fact-checking with citations and verifier agents

Fact-checking turns "the model said so" into "a source says so." The pattern has four parts: ground the answer in sources, require a quote for each claim, check the quote in code, and have a way out when the check fails.

Citations

The Anthropic reduce-hallucinations page, which we read on 2026-10-27, recommends three basic strategies: let the model say it does not know, extract word-for-word quotes before answering on long documents, and make answers auditable by citing quotes for each claim. It adds that you can ask the model to find a supporting quote for each claim after drafting, and to retract any claim without one.

The Anthropic API has a Citations feature. It returns the exact passages that support each claim. The docs say the API parses citations into the response and extracts the cited text itself, so citations are guaranteed to contain valid pointers to the provided documents. The cited_text field does not count toward output tokens. Our read: a valid pointer means the passage exists in the document you gave. It does not mean the passage supports the claim, so you still need a support check.

One clash to plan for, from the Citations page: the docs say citations cannot be used together with structured outputs. Enabling citations on a document and also setting output_config.format returns an error. If you need both a cited answer and a strict JSON shape, do it in two steps: a cited draft first, then a second call or a code step that builds the structure.

Verifier agents

A verifier is a second check on the first answer. It can be code, a model, or both, in that order.

• A code verifier checks that each quote appears in the cited source, that the source id exists, and that numbers in the answer appear in the source. It is exact and costs nothing.
• A model verifier checks that the quote supports the claim as worded. This is what code cannot do. Treat it as a judge, and calibrate it the same way.
• A refusal path handles the failures: drop the claim, retry once with a rewritten query, or escalate to a person. Cap the retries.

This sample is the code verifier. It checks four claims against two sources and drops the ones it cannot support. The model pass is a marked stub, so the file runs offline.

"""A citation verifier. No model needed for the first check. Each claim in a draft answer carries a quote and a source id. The verifier checks, in code,that the quote really appears in that source. Claims that fail are removed and reported.A second, model-based pass (does the quote actually support the claim?) would go wherethe marked line is. We leave it as a stub here so the file runs offline."""SOURCES = {    "refund-policy": "Refunds are issued within 14 days of purchase for unused licenses.",    "data-region": "Customer data is stored in the EU region by default.",} DRAFT = [    {"claim": "Refunds are available for 14 days.", "quote": "within 14 days of purchase", "source": "refund-policy"},    {"claim": "Data is stored in the EU by default.", "quote": "stored in the EU region", "source": "data-region"},    {"claim": "Refunds also cover used licenses.", "quote": "including used licenses", "source": "refund-policy"},    {"claim": "Data is encrypted with AES-256.", "quote": "encrypted with AES-256", "source": "security-whitepaper"},]  def verify(draft):    kept, dropped = [], []    for item in draft:        text = SOURCES.get(item["source"])        if text is None:            dropped.append((item, "unknown source"))        elif item["quote"] not in text:            dropped.append((item, "quote not found in source"))        else:            # Model-based pass goes here: ask a second model whether the quote supports the claim.            kept.append(item)    return kept, dropped  if __name__ == "__main__":    kept, dropped = verify(DRAFT)    print(f"kept {len(kept)} of {len(DRAFT)} claims")    for item in dropped:        print(f"dropped: {item[0]['claim']!r} -> {item[1]}")

We ran it on 2026-10-27 and got this output:

kept 2 of 4 claimsdropped: 'Refunds also cover used licenses.' -> quote not found in sourcedropped: 'Data is encrypted with AES-256.' -> unknown source

The output cannot show the failure that worries us most. A claim with a real quote that does not support it passes the code check. That gap is why the model pass exists and why a person samples the results. A verifier lowers the rate of unsupported claims. It does not remove it, and the Anthropic page says the same of its own techniques: they reduce hallucinations but do not eliminate them, and critical information should still be validated.

An EdgeFirm project

EdgeFirm built an AI research assistant for Sokrateque.ai, a tool for graduate students that searches 50,000+ academic papers. Its case study says the platform gives citation-verified answers, and lists a citation extraction and verification system among its technical parts. We cite it only as the case study states it and add no figures.

Configure agents for accuracy

Accuracy settings sit in four places: sampling settings, the prompt and sources, the output format, and the thinking effort. Reading the migration guides on 2026-10-27, we found that some common advice is out of date, so check the model's own documentation.

Temperature and sampling

Temperature controls how random the next-word choice is. Older advice says to set it near 0 for factual tasks. That advice has two problems today.

First, it never delivered fixed output. Anthropic's Opus 5.5 migration guide says that if you used temperature = 0 for determinism, it never guaranteed identical outputs on prior models. Second, on the newest Claude models the parameter is gone. The Opus 5.5 guide says to omit temperature, top_p and top_k and use prompting to guide behavior. The Sonnet 5.5 guide says sampling parameters return an error. If your code sends a temperature value to Sonnet 5.5, the request fails with a 400 error. Omit it for Opus 5.5 too.

Grounding and refusal

Grounding means the answer must come from the sources you supply. In practice it is three instructions and one design choice. Tell the model to use only the supplied documents and not its general knowledge, which Anthropic lists as external knowledge restriction. Give the model explicit permission to say it does not know. Ask for a quote before the claim on long documents. The design choice is a refusal path in the application, so that "I do not have that information" is a first-class result and not an error.

Test refusal as hard as you test answers. The harness above has a must-refuse case, and the release gate blocks a version that invents an answer for it.

Structured output

A schema narrows what the model can return, and makes checks in code easy. Anthropic's structured outputs take a JSON schema in output_config.format. The docs say they constrain responses to follow the schema, with valid, parseable output, and also qualify that there are scenarios where output might not match your schema. Two cases matter for safety. If the model refuses for safety reasons, the response has stop_reason: "refusal" and the output might not match your schema. If it hits the token limit, the output might be incomplete. Our rule is to check the stop reason before parsing.

Structured output makes the shape reliable, not the content. A valid JSON object can still hold a wrong claim, so keep the content checks.

Effort and thinking

Modern models let you trade cost and delay for more reasoning. On Claude Sonnet 5.5, the migration guide says manual thinking budgets return an error, and that an effort setting controls how much the model reasons. It advises running an effort sweep on your own evals instead of translating an old budget value. That is the use of an eval set we recommend. Run the same cases at each effort level, and record pass rate, cost and latency side by side. Pick the lowest level that clears your gate.

A settings table

Guardrails

A guardrail is a check on an input, an output or an action that can stop or change a run. Guardrails and evals are close relatives. A guardrail runs on every request in production and acts. An eval runs on a set and reports. A good guardrail started as a failing eval.

The OWASP 2026 prompt injection entry, read on 2026-10-27, lists eleven mitigations, including constraining the model's role, validating output against a strict schema in application code, filtering every input type, holding credentials in application code with least privilege, separating and labeling untrusted content, requiring human confirmation for privileged actions, and testing against adaptive attackers. It states that no reliable prevention mechanism exists today and that defense is architectural, so these measures reduce impact and do not remove the risk.

We group guardrails by where they sit.

• Input. Reject or label untrusted content, strip secrets, and limit length. Mark anything fetched from outside as data, not instructions.
• Output. Validate the schema, run the code verifier, block disallowed content, and refuse when the check fails.
• Action. Default deny for tools, human approval above a threshold, and hard caps on turns and spend. The agent harness guide covers this layer with code.

For each guardrail, write a test that tries to break it, and put that test in the regression set. A guardrail with no test is a hope. Our post on securing AI agents goes further on the security side.

Release gates

A release gate is a rule that decides whether a change ships. It turns eval results into a yes or no that does not depend on who is in a hurry.

Set the gate from a few kinds of number, drawn from your baseline run.

We set tolerances and ceilings from a baseline run, and so should you. The thresholds in our harness are placeholders for made-up data. Also write the override rule in advance: who can ship past a failed gate, and what they record.

Langfuse documents the CI/CD form: raise a RegressionError in the experiment script and the GitHub Action fails. Our harness does the same with an exit code. Either way, the gate runs on every change to the prompt, the tools, the model or the code, because any of the four can move quality.

Cost and latency monitoring

Quality is one axis. An agent that is right and costs ten times the budget, or takes a minute, is still a failed release. Track cost and latency from the same traces.

• Cost per finished task, not per call. An agent makes many calls, so the unit that matters is the task. Our harness prints cost per task from run records, using a made-up unit price.
• Median and slow-tail latency. Report the median and a high percentile, such as the 95th. Users feel the tail. The harness prints both.
• Tokens by type. Cached, reasoning and plain input tokens are priced differently. Langfuse stores each usage type as its own bucket, and its cost docs say a token should be counted in only one bucket. OpenTelemetry has separate attributes for cache read, cache write and reasoning tokens for the same reason.
• Tokenizer changes. A model update can change the token count for the same text. The Sonnet 5.5 migration guide states that the same text produces about 30% more tokens than on Sonnet 4.6, Sonnet 4.5 and Haiku 4.5, depending on content. We recommend re-running your cost baseline after any model change. Our post on reducing LLM costs without losing quality covers the levers once the numbers are visible.

Worked example: catching a regression before release

This is the harness run from earlier, read as a story. It uses invented data and stub agents, so it shows the method, not a real result.

A team has a support agent that answers from three policy documents. The baseline, version 1, passes all five test cases on all four trials. Version 2 is a candidate with a new prompt that made answers longer and added a step. Nobody has changed the sources.

1. Run the set. In our run, version 2 passes 70% of trials, and only 60% of cases pass on every trial. The p50 latency rose from 0.85 s to 1.45 s, and cost per task rose from about $0.0011 to $0.0018 on the made-up price.
2. Look at which cases failed. The must-refuse case, a question about a phone line the documents do not mention, fails on every trial. The agent now says the line is open 24/7. The data-region case fails on alternate trials because it cites the wrong source.
3. Check the judge. The stub judge passed the wrong-source answer, because it was long and fluent, and the human labels in the sample failed it. Agreement was 80% and kappa 0.55, on five cases, so treat both as a demonstration of the method. The planted bias imitates the verbosity bias in the MT-Bench paper. The code citation check would also have caught the answer.
4. Apply the gate. It blocks the release for three reasons and exits with code 1.
5. Fix and add cases. The team fixes the prompt and adds the wrong-source pattern to the set as a permanent case.

Without repeated trials, the flaky case could have passed by luck. Without the must-refuse case, nobody would have seen the invented answer, because it looked helpful.

Common mistakes

These are the mistakes we would guard against first, based on the sources above. They are our engineering judgment, not survey results.

• Starting with a judge. Teams reach for a model grader first because it feels general. Start with code checks, add a judge where code cannot reach, and calibrate it.
• Trusting a judge without a human check. A judge can be systematically wrong in the same direction on every run, and its scores will still look consistent.
• Treating citations as proof. A valid pointer shows the passage exists, not that it supports the claim.
• Skipping evals until late. Projects that cannot show quality tend to stall, which is one of the patterns in our post on why enterprise AI projects fail.
• Setting temperature to 0 and calling it solved. It never guaranteed identical output, and newer models reject or advise against the parameter.

When not to build all of this

Many projects need less than the full ladder.

• A low-risk internal drafting tool. If a person reviews every output before use, rung 2 is enough: a small set, run by hand before each change.
• A single model call with no tools. A prompt, a schema check and a handful of cases are enough. Tracing a loop that does not exist adds noise.
• No way to define a good answer. If the team cannot agree what correct looks like, evaluation will not settle it. Write ten good and ten bad examples first, and settle the argument there. If you are still deciding whether to build at all, our post on proof of concept versus production AI explains what each one proves.

Production checklist

Use this before an agent answers real users. Each row names the Eval Ladder rung it belongs to.

Glossary

What to do next

If you have no traces yet, start at rung 1 this week: pick one tool, trace one agent, and look at ten runs. If you have traces, write the first 20 cases from real failures and run them before the next prompt change. If you have both, add the gate.

EdgeFirm builds and tests agents for clients. If you want help scoping evaluation for a real system, see our AI agent development service or our AI implementation services. Two companion guides cover the parts around this one: the agent harness guide for loops, limits and permissions, and the RAG guide for the retrieval that fact-checking depends on.