A plausible menu is not enough: VALID-Diet proposes how to test AI advice

A menu can be beautifully written and still be wrong for the person expected to follow it. An orderly response can make the distance between a plausible suggestion and reliable advice less visible. VALID-Diet, published on 13 October in Frontiers in Nutrition, attempts to organise the work needed to measure that distance.

The Perspective proposes four stages: define intended use and risk; test analytical validity; examine safety and context; assess human factors, utility and outcomes. Five domains run through the pathway, from nutritional checking to documentation. The system still needs prospective validation: it is an assessment structure proposed by the authors, not a conformity mark.

Risk depends on intended use

The initial shift in attention is decisive. Assessing a system without specifying who will use it and for which decision leaves even the meaning of “correct” unresolved. Suggesting meal ideas and intervening in a clinically constrained diet raise different questions. A satisfactory average result could conceal a rare error that matters greatly to a particular person.

Numerical accuracy is only part of the work. Assessment must ask whether the user’s information was taken into account, whether the advice is feasible and whether the system can handle difficult cases. Even nutritionally coherent advice can fail in use, for example when it assumes resources or habits someone does not have. The authors’ original diagram, reproduced in full, makes the proposed pathway available for inspection.

The final step concerns people

For platform investors, the implication is that evaluation cannot stop at the quality of the prose. The choice of tests must follow the claimed benefit: saving time, increasing understanding and improving a health outcome are not interchangeable results. Each requires relevant observations beyond a convincing interface demonstration.

The proposal’s value lies in organising the questions, while its effectiveness as a method remains to be tested. Provisional thresholds are not guarantees, and the retrospective illustrative application does not generate new clinical results. The next test is whether the structure actually helps identify errors, guide revisions and improve system use in defined populations.

Editorial note. This content is intended for industry professionals. It does not constitute medical advice, therapeutic guidance or regulatory advice. Read our editorial method.