The first time a programme builds an honest curricular coverage matrix, the reaction is rarely satisfaction. It is closer to discomfort. Competencies that everyone assumed were covered turn out to be assessed once, in one subject, by one instrument, in one year. Others are assessed nine times, all of them in the same two-week block, all by the same three people. A couple have no evidence against them at all.
The matrix did not do this. The matrix found it. It has been true for years.
A coverage matrix does not create gaps. It ends the state of not knowing where they are, which is why building one is uncomfortable and why it is worth doing anyway.
Why volume is not coverage
Most health sciences programmes assess a great deal. When the total count is put in front of a faculty board it usually sounds reassuring: several thousand assessment events a year across a cohort, hundreds of hours of supervisor time, an evident and genuine commitment.
Volume tells you almost nothing about coverage. Four questions have to be answered before a number means anything:
- Which competency? Not which subject. Assessment events are usually recorded against a course; the accreditation question is asked about a competency, and the mapping between the two often lives in one person's head.
- By how many different assessors? Twelve observations by one supervisor is one supervisor's opinion, recorded twelve times.
- With how many different instruments? Triangulation is a pillar of programmatic assessment for a reason. Six mini-CEX records tell you about consultations. They tell you nothing about procedures or about how the student works in a team.
- Spread across what period? Evidence clustered into a single fortnight describes a fortnight. Competence is developmental, and an assessment programme that samples only one point in the development cannot see the development.
A programme can be assessing constantly and still have a blueprint with holes in it. The holes are usually not random, either. They cluster in exactly the places that are hardest to observe and easiest to assume: professionalism, communication under pressure, collaboration, the parts of practice that everyone agrees matter most and nobody has an obvious instrument for.
What the matrix is actually for
Two uses, and they pull in different directions.
The accountability use
Accreditation bodies increasingly ask for evidence of coverage rather than for a statement of intent. In the Spanish system, the AQU and ANECA seguiment and acreditació cycle asks what the programme actually did, and RD 822/2021 changed the vocabulary underneath it, so programmes now have to demonstrate coverage of official resultados de aprendizaje using records that were captured against an older competency structure.
The remapping work this creates is substantial, thankless and easy to underestimate. It is also the single most common reason a programme's coverage evidence is assembled by hand in a panic in the six weeks before a submission.
The improvement use
This is the one that is worth more and gets less attention. A live matrix, consulted during the year rather than assembled after it, changes what a programme does:
- A competency with thin evidence in November is a competency you can still schedule assessment against in January.
- An instrument that is contributing nothing anywhere, because nobody ever completes it, is an instrument you can retire, which reduces load without reducing coverage.
- A competency assessed only by a single retiring supervisor is a succession risk you can see before it becomes an emergency.
The difference between the two uses is timing. A matrix produced once a cycle is a compliance artefact. A matrix that is always current is a management instrument. The second is much harder to maintain by hand, which is most of why programmes end up with the first.
What a platform can and cannot do here
It can keep the matrix current without anyone maintaining it. If every assessment is recorded against the competencies it addresses at the point of capture, coverage is a query rather than a project. This is the whole argument, and it is a strong one: the reason most coverage matrices are out of date is that keeping them current is an unpaid manual job that competes with teaching.
It can hold the mapping and its versions. Legacy competency to official learning outcome, with a mapping table, a version label and an audit trail, so that when the memoria is modified you can still explain what the evidence meant under the previous structure. This is unglamorous and it is the part that saves the most time in practice.
It can show you the shape of the gap, not only its existence. Coverage by competency crossed with subject, year and instrument, so you can tell the difference between "not assessed" and "assessed nine times by the same person in the same fortnight". These are very different problems with very different remedies.
Now the limits.
It cannot decide what good coverage looks like. How many data points, from how many assessors, across what spread, is sufficient for a given competency in your discipline. That is an academic judgement belonging to your programme. A platform can apply the thresholds you set and show you where they are not met. It cannot set them, and any system that claims to has quietly replaced your academic governance with a vendor's default configuration.
It cannot fix a blueprint that was never designed. If competencies were mapped to subjects in a spreadsheet in 2019 and nobody has revisited the mapping since, the matrix will faithfully display a picture of that. It will show you the problem clearly and early, which is the useful part, but the redesign is academic work and it will take a term.
It will make the programme look worse before it looks better. This is worth saying to a faculty board in advance, in writing. The first honest matrix is the worst one you will ever produce, because it is the first one that is true. Programmes that are not prepared for this sometimes respond by quietly loosening the thresholds, which converts a management instrument back into a compliance artefact.
A practical sequence
For a programme starting from scratch, the order that tends to work:
- Settle the framework and the mapping first. Before any assessment data moves, agree which competency set is authoritative and how legacy records map onto it. Everything downstream inherits the mistakes made here.
- Start with one discipline and one year group. A full-programme rollout produces a matrix nobody trusts, because too many mappings were guessed at once.
- Publish the first matrix internally, with its gaps. Circulating an honest one builds the political cover you will need. Circulating a tidied one destroys it permanently the first time someone checks.
- Set thresholds after you have seen real distributions, not before. Any threshold set in advance is a guess, and a guess that later has to be relaxed looks worse than a threshold set from evidence.
The claim, stated narrowly
We are not claiming that a coverage matrix improves your programme. It does not. It tells you, continuously rather than retrospectively, where your assessment programme is thin, which competencies rest on a single assessor, and which parts of your blueprint exist only as an intention.
What you do about that is the actual work, and it is yours. The value of the tool is that you get to start it in October rather than discovering the problem in a report from your accrediting agency, or from a student who has an entirely fair complaint.
References
- Van der Vleuten, C.P.M. and Schuwirth, L.W.T. (2005) "Assessing professional competence: from methods to programmes", Medical Education 39(3), 309–317.
- Heeneman, S. et al. (2021) "Ottawa 2020 consensus statement for programmatic assessment — 1. Agreement on the principles", Medical Teacher 43(10), 1139–1148.
- Real Decreto 822/2021, de 28 de septiembre, por el que se establece la organización de las enseñanzas universitarias y del procedimiento de aseguramiento de su calidad.
- Association for Dental Education in Europe, The Graduating European Dentist curriculum framework, published in the European Journal of Dental Education.