The Statistics That Mislead

Driver scoring produces numbers with a lot of digits and very little stability. Four effects account for most of that.

Regression to the mean

A driver who scores badly one month will usually score better next month, whatever you do.

Because part of any month's score is chance: which routes, which traffic, which weather.

Which means a coaching intervention applied to the worst performers will always appear to work.

And it will appear to work whether or not it does anything.

To know: compare against a control group of similar drivers who were not coached, which almost nobody does.

Small numbers

A driver with three events this month and one last month has not improved by sixty-seven percent.

They have had a different month.

Event counts at individual level are small enough that ordinary variation swamps any signal, which is why monthly individual scores move dramatically and mean little.

Quarterly, and with the count shown alongside the rate, is the minimum for a figure anyone should act on.

Exposure not accounted for

Two drivers with the same event count and different mileage are not comparable, which everyone knows.

Two drivers with the same events per mile and different route types are also not comparable, which is less obvious and matters more.

Any ranking across route types is measuring the routes.

The denominator changing

A driver whose mileage drops — through illness, a different assignment, a quiet month — sees their rate move without any change in behaviour.

A small denominator makes a rate unstable, which is the same small-numbers problem wearing different clothes.

Set a minimum exposure below which no score is produced. Most products do not, and produce confident scores from eighty miles.

What to do instead

Report counts and rates together, with the exposure visible.

Compare a driver against their own trend over quarters.

Investigate outliers as events with context, not as positions in a table.

And if you run a coaching programme, run it properly — with a comparison group — because otherwise you will never know whether it worked and will keep paying for it.

Show the count beside the rate

A formatting decision that prevents most misreadings.

Three events in four hundred miles is a rate and it is also three events.

A reader who sees only the rate treats it as stable; one who sees the count knows it is not.

And set a minimum exposure below which no score is produced, because most products will happily rank someone on eighty miles.

Follow the difficult record

Use visit the official website to frame one representative case. The useful evidence is the record created when a value is challenged, corrected, approved and exported.

Independent reference

For a thematic point of reference, see the Insurance Institute for Highway Safety. It provides useful context beyond supplier documentation and should be checked in its current form.