Decisions a board can defend

The test of a progression decision is not whether it was correct. Reasonable committees disagree, and a decision that later proves wrong may still have been the right one on the evidence available. The test is whether, eighteen months later and under formal challenge, you can still show how it was reached.

That is a documentary question, and most committees discover it is a documentary question at the worst possible moment.

What a challenged decision actually requires

When a progression decision is contested — internally, through a university appeals process, or externally — the committee is asked to produce four things:

1. The evidence that was before the committee at the time. Not the evidence that exists now. The evidence as it stood on the day, including anything that was subsequently amended.
2. The criteria applied. Written down before the decision, not reconstructed afterwards, and demonstrably the same criteria applied to this student as to the others reviewed in the same sitting.
3. The reasoning. How the committee got from that evidence to that conclusion, in enough detail that a reader who was not in the room can follow it.
4. The process. Who was present, what they had access to, who declared an interest, and what the student was told and when.

A committee working from a folder of spreadsheets, a shared drive and the collective memory of the people who happened to attend can usually produce the first of these, partially. The other three are reconstructed, and reconstruction is exactly what an appeals process is designed to detect.

The uncomfortable question is not "was the decision right?" It is "can you show what the record looked like on the day, before anyone knew it would be challenged?"

The three failure modes we see most often

The evidence has moved

A supervisor amends an assessment two months after the committee met, in perfectly good faith, having remembered something. Now the record no longer matches the decision. Without a change log, there is no way to demonstrate which version the committee saw, and the discrepancy looks far worse than the underlying facts ever were.

This is why an immutable change log matters more than it sounds. Not as a cryptographic guarantee, and it should not be sold as one, but as a plain record of who changed what, when, and from what value. It is the difference between "the record changed and here is exactly how" and "the record does not say what you were told it said".

The reasoning was never written down

Committees discuss at length and minute briefly. The discussion is where the reasoning lives; the minute records the outcome. Six months later the outcome is all that survives, and a bare outcome is indistinguishable from an arbitrary one.

The remedy is not longer minutes. It is capturing the justification attached to the decision itself, at the moment it is made, in the same place as the evidence it refers to.

Consistency cannot be demonstrated

This is the one that does the most damage. A student challenges a decision by pointing at a peer with an apparently similar record who progressed. Unless the committee can show the two records side by side, against the same thresholds, the challenge is very hard to answer, even when the decision was entirely sound and the records were in fact materially different.

Hauer and colleagues, studying clinical competency committees, found that the committees which functioned best were those that reviewed every learner systematically rather than only the ones already flagged. That practice is defensible because it is consistent, and it is consistent because it is systematic. It is also extremely difficult to sustain when each review means assembling a folder by hand.

What the model asks of a board, and what it gives back

Programmatic assessment makes a specific and demanding bargain with a committee. It asks you to stop treating any individual assessment as decisive — no single failed mini-CEX ends a progression — and in exchange it asks you to make the high-stakes judgement on the accumulated body of evidence.

The Ottawa 2020 consensus put the principle plainly: high-stakes decisions will and can be made on rich information. The measurement literature supports it. Moonen-van Loon and colleagues examined the composite reliability of a workplace-based assessment toolbox and showed that acceptable reliability comes from combining different instruments rather than from accumulating more of any single one — which is the technical argument behind triangulation, stated as a measurement result rather than as a principle.

The bargain only works if the accumulated body of evidence is actually in front of the committee in a form it can use. In practice that means:

• Every data point for a student, in sequence, with the narrative text intact rather than reduced to scores.
• Evidence organised by competency, so the committee can see which parts of the framework the concern actually touches.
• The same view for every student under review, which is what makes consistency demonstrable rather than merely asserted.
• Thresholds set by the programme, visible in the interface, applied identically.
• The decision and its justification recorded against the evidence, and time-stamped.

Where we would push back on our own product

An audit trail is change traceability, not proof. eLEA.to records who changed what, when, and from what value. It is not a cryptographic notarisation and we do not describe it as one. It is sufficient for an internal appeal and for an accreditation review. If your legal office needs something stronger, tell us early, because that is a different engineering conversation.

Aggregated evidence can make a weak decision look strong. This is the sharpest risk in the whole model and it deserves stating from the vendor's side. A screen full of data points has an authority that the underlying observations may not deserve. If twelve assessments were completed hastily by two supervisors who were also the student's clinical line managers, the aggregate is not twelve independent observations, and no visualisation will tell you that. The committee's job is still to interrogate the quality of the evidence, and a better interface makes that job easier to skip. Schut and colleagues, reviewing 27 empirical studies, found that programmatic assessment generated sufficient information to enable robust decision-making — and identified an overload of assessment information, with the workload that comes with it, as one of its recurring challenges. A platform is as capable of causing the second as of delivering the first.

A progress indicator is a summary, not a verdict. Any traffic-light or threshold display compresses a rich record into a colour. That is useful for triage across two hundred students and dangerous if it becomes the decision. Our position is that indicators should route committee attention and never substitute for reading the evidence, and that a programme should say so explicitly in its own regulations rather than leaving it to habit.

We are not your legal advisers. What makes a decision defensible depends on your institution's regulations, your national framework, and in Spain on the specific appeal routes available to a student. We can give you the record. Your academic secretary and legal office have to tell you what the record needs to contain.

A short checklist

Whatever you use, a progression decision should be able to answer these without anyone having to remember:

• What evidence was before the committee, as of that date?
• Has any of it changed since, and how?
• What thresholds applied, and were they in force before the review?
• What was the stated reasoning?
• Who was present and who had access?
• What was the student told, when, and what were they invited to respond to?
• Can the same account be produced for every other student reviewed that day?

If the answer to the last one is no, the decision is exposed regardless of how carefully the individual case was handled.

References

• Heeneman, S. et al. (2021) "Ottawa 2020 consensus statement for programmatic assessment — 1. Agreement on the principles", Medical Teacher 43(10), 1139–1148.
• Moonen-van Loon, J.M.W., Overeem, K., Donkers, H.H.L.M. et al. (2013) "Composite reliability of a workplace-based assessment toolbox for postgraduate medical education", Advances in Health Sciences Education 18(5), 1087–1102.
• Hauer, K.E., Chesluk, B., Iobst, W. et al. (2015) "Reviewing residents' competence: a qualitative study of the role of clinical competency committees in performance assessment", Academic Medicine 90(8), 1084–1092.
• Schut, S., Maggio, L.A., Heeneman, S., van Tartwijk, J., van der Vleuten, C. and Driessen, E. (2021) "Where the rubber meets the road — an integrative review of programmatic assessment in health care professions education", Perspectives on Medical Education 10(1), 6–13.