Your questions, your bank, your statistics
The first time a candidate disputes a result, somebody at your body has to answer a question the scoring system cannot: is this a good question? Scoring a paper and knowing whether the paper measures anything are different capabilities.
Authored and reviewed where it lives
Questions are written, reviewed and held in your own bank, under your body’s control. Subject-matter experts work in a secure space rather than on a document emailed around, and a review workflow records who looked at an item and what they concluded. Items carry their content area, so a form can be assembled to a blueprint rather than by hand.
Assembled deliberately
A form is built from sections, each drawing on the bank. Pretest items can be carried on a live form and scored for nobody — the only honest way to gather statistics on a new question is to put it in front of real candidates under real conditions, and the only honest way to do that is to make sure it cannot affect their result. How many pretest items a form may carry is capped per program, and whether they are visible as such is your decision.
Including the statistic that finds a miskeyed question
After a sitting, every item is analyzed:
- Difficulty — the proportion who answered correctly. Cheap, and most useful at the extremes: an item everybody passes and an item everybody fails both consume a candidate’s time and separate nobody.
- Discrimination, computed two ways — the point-biserial correlates getting this item right with doing well on the rest of the paper; the upper–lower 27% index says how much more often the strongest quarter got it right than the weakest. The first is the better statistic and the second is the one you can explain to a board in a sentence. A body defending a cut score needs both.
- Distractor analysis — which wrong answers were chosen, and by whom. This is the one that pays for the whole feature. A distractor that the strongest candidates pick more often than the key is the signature of a miskeyed item, and a miskeyed item does not merely fail to measure: it marks the people who know the material wrong. Nothing else in a certification system can find one.
A route that ends in a real decision
A candidate may challenge a specific question within a window you set. The challenge goes to your people with the item and its statistics in front of them. You decide, and the decision propagates to every score that item touched — rescoring the cohort if that is what the decision means, rather than leaving one candidate corrected and the rest wrong.
Ask about bringing your bank across
Tell us how many candidates you certify and what you run today. You will get a written quote against the published rates — not a discovery call before a number.