Home
Our Approach Advanced Services Expertise Why EUCALEX
AI‑LEX Self-Assessment Confluence Contact us Blog & News Careers All articles
AI

The answer is only as good as the data behind it

Written Dec 2020published 21 Apr 20253 min read

How good an AI answer is depends partly on the question you ask, and mostly on the data the model draws on. In cybersecurity and industrial digital transformation, that second part decides whether the assistant is useful or dangerous.

Retrieval-augmented generation (RAG) gives the model a controlled body of documents to work from before it answers: technical books, equipment manuals, training material, lessons learned from the field, regulatory texts and good practice guides on industrial technologies, cybersecurity, standards and compliance. The model still writes the answer, but it writes it from passages it has just retrieved, not from memory.

Done properly, that gives you an assistant that is more consistent, more accurate, and closer to what people on site actually need.

Why a general model is not enough here

A general-purpose model has read a great deal about industrial systems and remembers it approximately. Approximately is fine for drafting an email. It is not fine when the question is which clause of a standard applies to a given zone, or what the safe parameter range is for a particular controller.

The problem is rarely ignorance. It is fluency. Ask a model about a standard it half-remembers and it will give you a clause number, a title and a summary, all plausible, some of it made up. When the reader cannot check on the spot, a confident wrong answer does more harm than no answer at all.

What grounding changes

The answer can point to something. A response built from a retrieved passage can show that passage; a response built from memory has nothing to show.

The corpus becomes a decision. You choose which manual, which revision and which internal procedure count as authoritative, and anyone can audit that choice later.

Keeping it current means editing documents, not retraining a model. When a procedure is revised, the new version replaces the old one in the corpus and the assistant uses it the same day.

And wrong answers become traceable. When the assistant gets something wrong, you can look at what it retrieved and usually see why: an obsolete revision, a missing document, a passage taken out of context. That turns a mystery into something you can fix.

Where the effort really goes

Grounding is not a switch you flip. Most of the work is unglamorous: deciding which documents are authoritative, hunting down the three obsolete revisions still sitting on a shared drive, agreeing on what the assistant must refuse to answer. Those are governance decisions, and anyone who has run a document control system will recognise them.

It is also where the measurement belongs. Four numbers tell you whether grounding is working, and none of them is a model score: the share of answers whose source passage can be found and read, the share that cite a current revision rather than a superseded one, the share a subject-matter expert accepts without correction, and the number of questions the system declines rather than guessing. A system that improves on the first three while the fourth stays above zero is working.

Source

EUCALEXNotes from field experiencesTalk to us about this

EUCALEX

Have a project or a technical challenge?

Contact us: we will be pleased to discuss it with you, with no commitment.

Discuss your project with us

Stay informed

Receive new EUCALEX articles and technical insights by email.