Applied AI · Practical guide

How can you tell whether an AI model works well?

An accuracy percentage does not tell the whole story. Look at the mistakes a model makes, where they occur and their consequences in use.

Explore the guide
Conceptual schematic. No client data or metrics.

The central idea

Mistakes matter as much as correct results.

Evaluation means comparing an estimate with a reference and deciding whether it serves a specific task. This guide uses document classification; predicting quantities or generating text requires different criteria.

At T2G we start with the task, its data and operating conditions. We agree on a scope that can be checked and explain technical decisions throughout the work.

Step by step

Evaluate against a reference and in context.

Four points to organise the work. Each diagram supports the explanation; the process adapts to its context.

  1. 01 / 04

    Separate learning from evaluation

    Reserve cases not used for training or choosing adjustments. If files from the same document appear in both sets, results may look better than they will on new information. Splits should respect data origin and, where relevant, time.

    Training · Validation · Test
  2. 02 / 04

    Inspect the types of error

    An alert task has four outcomes: a correct alert, a false alarm, a missed case and a correct dismissal. A confusion matrix shows them separately. Their operational costs may differ.

    Reference · Estimate · Errors
  3. 03 / 04

    Choose metrics for the task

    Precision measures the share of alerts that were correct; recall measures the share of actual cases detected. Accuracy counts all correct classifications. When alerts are rarely needed, a model that never alerts may have high accuracy while being unhelpful.

    Precision · Recall · Accuracy
  4. 04 / 04

    Review use and exceptions

    A threshold changes which cases receive alerts. Assess it on validation data, keep the final test separate and review different document types, periods and sources. Define human review and how errors will be tracked after integration.

    Threshold · Review · Monitoring

An explained example

An illustrative matrix of 100 documents

01 / Starting point

Twenty documents need review and eighty do not. The model flags twenty-five: fifteen actual cases and ten false alarms.

02 / An approach

Five cases are missed and seventy correctly dismissed. Precision: 15/25 = 60%. Recall: 15/20 = 75%. Accuracy: 85/100 = 85%. These are teaching figures, not T2G results or performance targets.

For your decision

What needs to be clear.

01

Reference

Who defines the correct category and how disagreements are resolved.

02

Consequences

The impact of false alarms compared with missed cases.

03

Coverage

The sources, periods and conditions represented in evaluation.

Common questions

Questions before getting started.

Does high accuracy guarantee a useful solution?

No. It may hide imbalanced classes, data leakage or expensive errors. Review it alongside the context and types of failure.

How many examples are needed?

It depends on variation, categories and the certainty required around the error estimate. Small samples make conclusions more uncertain.

Further technical reading: Google · Accuracy, precision and recall ↗

Keep exploring

Connect this guide with your challenge.

Let’s discuss your context

Let’s define a useful first step.

We review the problem, available information and constraints with you to assess where it makes sense to start.

Book a diagnostic session directly with the partners