Are strict judges always strict?

To study the impact of the legal system on convicts and offenders, researchers have long relied on the random assignment of cases to judges. By comparing outcomes from "strict" judges to those from "lenient" judges, experts can measure the causal effect of decisions like pretrial detention or prison sentences.

For decades, the legal system and the researchers who study it have relied on a "golden rule" called monotonicity. This principle assumes that if Judge A is generally stricter than Judge B, Judge A will be stricter in every single case they both encounter. It’s the idea that a "tough-on-crime" judge doesn't suddenly become a "bleeding heart" for specific defendants.

A new study of courts in Norway, Brazil and the U.S. reveals that this assumption is not true. By analyzing judicial panels—where multiple judges decide the exact same case—I found that the "strictness" rule is violated in up to 50 percent of cases where judges disagree. In many instances, the judge with the reputation for being the most lenient was actually the one pushing for a tougher sentence, while their "strict" colleague argued for leniency.

For example, a judge who is tough on drug offenses might be lenient toward white-collar crime, while a generally "soft" judge might have a personal zero-tolerance policy for specific infractions

Why our research tools still work

If judges are so inconsistent, does that mean years of research on the "justice gap" are wrong? The answer is thankfully no. Despite frequent violations of the strictness rule, these "errors" tend to cancel each other out.

The study’s innovation is to measure not just if the rule was broken, but how much it skewed the final results. While individual judges might behave unexpectedly, they still follow a "weaker average monotonicity". In other words, the group of judges who vote for a tough sentence is still, on average, stricter than the group that does not.

The research demonstrates that the bias introduced by these judicial "glitches" is remarkably small. This confirms that the statistical tools used to evaluate bail reform, sentencing guidelines, and the long-term effects of incarceration are still highly accurate. For leaders and decision-makers, this provides a "green light" to continue using random-judge assignments as a gold standard for evaluating legal and social policy.