Validating a model against real data

Anyone can build a model that produces a number. Getting that number to agree with reality — and being able to state, with evidence, how closely it agrees — is the hard part, and it's the part that separates a digital twin from a decorated guess. A prediction nobody has checked against measurement is just a hypothesis with good production values.

Validation is the least glamorous and most important stage of building a digital twin. It's where a model earns the right to be trusted. Skip it, or do it loosely, and you have an animation that happens to output numbers. Do it properly and you have an instrument. Here are the notes on doing it properly — grounded in validating heat exchanger twins, but the principles carry across.

What validation actually means

Validation is not "the model runs" and it's not "the output looks plausible." It's a specific claim, backed by evidence: the model's predictions agree with real measurements to within a stated, quantified band, across the range of conditions the twin will be used in. Every part of that sentence matters — stated, quantified, and across the range, not at a single convenient point.

The process

1. Run the real experiment

You need real data from the actual instrumented rig — not just the model talking to itself. Run the physical experiment across a range of conditions and record the measurements you'll compare against. This is the ground truth, and everything downstream depends on it being trustworthy, which means the instrumentation itself has to be sound first.

2. Run the model under the same conditions

Feed the model the same inputs the real experiment ran under, and record its predictions. The comparison is only fair if both are looking at the same scenario — same flow rates, same inlet conditions, same everything you can match.

3. Compare, and quantify the gap

Now put prediction against measurement and quantify the disagreement — not "they're close" but an actual figure: the model tracks the measured outlet temperature to within a stated percentage, say. This number is the whole deliverable of validation. Without a number, you don't have a validated model; you have an impression.

4. Do this across the range, not at one point

A model can agree beautifully at one operating point and drift badly at another. If the twin will be used across a range of flow rates, you validate across that range — because an error band that only holds at one convenient point is not an error band, it's a coincidence. This is the step most often shortcut, and it's the one that catches the models that would have embarrassed you later.

Understanding the disagreement

When prediction and measurement diverge — and they always do, somewhat — the divergence has causes worth understanding rather than papering over:

• Idealisation. The model assumes clean conditions; reality has fouling, losses, and imperfections the equations don't capture. This is usually the biggest contributor and, notably, is exactly the physics you want students to learn from.
• Measurement uncertainty. The sensors themselves have accuracy limits and can be slightly misplaced. Some of the "error" is in the measurement, not the model.
• Model scope. Every model simplifies. The question is whether the simplifications are acceptable for the twin's purpose — and being explicit about that is part of honest validation.

Understanding why the gap exists is what lets you state it defensibly. "The model runs about two percent high at high flow because it doesn't capture fouling" is a defensible, honest statement. "The error is two percent" with no account of why is weaker — and won't survive a sharp question from a reviewer.

What "holds up to review" means

A validated model has to survive someone competent looking hard at it — an examiner, a reviewer, an industry buyer. That means the error band is documented, the range it holds over is stated, the reason for the discrepancy is understood, and the whole thing is reproducible. If any of those is missing, the right response to scrutiny is a wince, and you can feel it coming.

This is also why validation is genuinely hard and why it's tempting to skip. It takes real experimental work, honest comparison, and the discipline to state limitations rather than hide them. But it's precisely this that makes a twin trustworthy — and trustworthy is the entire value proposition.

Why we build the model first, then validate hard

The reason this standard matters so much to us specifically: a teaching twin that hasn't been validated teaches students the wrong thing — that the idealised prediction is reality. A validated one teaches the truer, more valuable lesson: that models and reality differ in understandable ways, and that quantifying and explaining that difference is what engineering actually is. The validation isn't overhead on top of the teaching. It is the teaching.