Andrew Blair-Stanek, Steven Boggs, Donald G. Gifford, Seema M. Kakade, Guha Krishnamurthi, Russell McClain & Donald B. Tobin (all University of Maryland Francis King Carey School of Law) have posted Is AI’s Law School Exam Performance Plateauing? on SSRN. Here is the abstract:
Last spring, we had OpenAI’s reasoning model o3 take our final exams, with the reasoning effort parameter set to “high,” and graded its answers on the same curve as our students. o3 got grades ranging from A+ to B. This spring, we repeated the experiment, using OpenAI’s latest reasoning model, GPT-5.5, with the reasoning effort at the new “xhigh” setting. GPT-5.5 got two A+s, three As, two As, and a B+, a good performance but far short of superhuman. Depending on the metric, GPT-5.5 may have actually performed worse than o3 did last year, despite the new “xhigh” setting. These results may fit the broader pattern of frontier AI models’ performance plateauing on other legal benchmarks.
Recommended!
To receive new posts from Legal Theory Blog by email, get a free subscription to Legal Theory Stack.
Lawrence Solum