Tl;dr
The invention scores free-response questions given the text of answers AND NOTHING MORE. That’s right—it can “grade” essays based on the text of answers without a model answer or any external context, and it does NOT use an LLM. You can find the patent here, and an explanation of how it works below (text or video) after a preamble discussing how we got here.
On this page
Why a patent?
When I founded the LIT Lab one of my first decisions was that the Lab’s IP should default to open because defaults matter. This is why the overwhelming majority of our work is released under an MIT License, and the fact that multiple funders continue to support our work on open source software is a point of professional pride. Defaults, however, are just that. Sometimes open source doesn’t align with a project’s goals. This realization is what drove the Lab to use trust law to help us build and share free pro-social non-extractive AI.
When I started teaching law students, I knew I wanted to lean into frequent formative assessments. I also knew that grading free response questions was hard, and I felt multiple choice questions (which are easy to grade) didn’t meet my needs. And as the saying goes, “necessity is the mother of invention.” The result was a method for the unsupervised machine scoring of free response answers. It allowed me to feed in a batch of student answers and get an ordering of those answers that more closely resembled a sort based on their grades than a random shuffle.
I could then build on this signal to more closely engage with those texts flagged negatively as outliers. I hoped the tool I created could help foster more formative assessment by making it easier for instructors, but I worried about automation bias and the possibility that not everyone would use it responsibly. As I discussed in the corresponding paper, I worried about instructors abdicating their responsibilities as graders.
This was back before my position at Suffolk was hard funded in full, before we regularly received outside grants, and it occurred to me that I might be able to kill two birds with one stone. If the university held a patent on the method we would be in a position to block inappropriate use and to license its use only to parties aligned with our mission as an educational institution. It might also generate some revenue for the Lab. I saw the potential for Lab patents, but honestly, I was anxious about whether or not my interests and those of the university would align. So, I decided I would take this potential patent to the university, approaching the endeavor as a sort of trial run to see how things worked out.
There was no requirement for me to assign the invention to Suffolk as the efforts involved did not trigger that part of our IP policy. I’m pleased to say, however, the university has been very supportive, and I’m happy with where things stand. I assigned the invention to Suffolk in accordance with our IP policy at the time which does provide for revenue sharing with the inventor (that’s me). Suffolk was recently awarded the patent, and we are currently exploring the possibility of licensing the tech for responsible use.
Recently there has been a flurry of activity in “automated grading” fueled by large language models/AI, but this is not that. To understand why, let’s consider a simple question.
How does it work?
This description was adapted from a Mastodon thread.
A paper describing how the method works was included in a collection at the MIT Computational Law Report. It describes a novel machine learning method for scoring free-response questions. Given the text of student answers, AND NOTHING MORE, it can produce a ranked list of answers that more closely match the order given by a human grader than a random shuffle.
“Wait, you can ‘grade’ essays based on just the text of answers without a model answer or any external context, and there’s no LLM either? Impossible,” you say. By the end of this thread, if I’ve done my job right, you’ll be saying, “Well, of course that works.”
Is it magic? No, it’s math. Yes, we’re talking about unsupervised machine grading. For more nuance than social media can provide, check out the paper with links to code and data here.
To understand how this is possible, consider: if you’re running a competition to guess the number of candies in a jar—or weight of a cow—the average of the guesses is probably close to the right answer. This is often attributed to the wisdom of crowds.
Human judgment can go wrong in a lot of ways, and there are more ways to be wrong than right. This is sometimes called the Anna Karenina principle, named for the 1st line of the Tolstoy novel. “Happy families are all alike; every unhappy family is unhappy in its own way.”
It is easy to see how this helps us with numeric predictions, like the count of candies or guessing a cow’s weight. Some folks guess too high, others too low, and if there is a diversity of wrong answers, the different wrong answers “cancel” each other out when averaged.
We can see this at work with 2D targets like a bullseye. Bias pulls some folks to the side, noise scatters things … but if there are enough people with different biases, different ways of being wrong “cancel” each other out such that the middle/center/average of hits is close to the bullseye.
But how the heck do we do this with text answers? First, remember that when people think of the average answer to a test they often mean an answer with the average grade. That’s not the average of the answers, which is what we are looking for.
We need some way to actually average text answers. Luckily we can turn texts into numbers using something like word2vec. How exactly this works is more than I can fit here, but I do provide an overview in the full paper talk around the 10 min mark in this video.
The point is we can map words and collections of words (like essay answers) to points in some many-dimensional space (e.g., 300 dimensions for word2vec). Mapped into such spaces, words/answers with similar content are close to each other. For an example, see TensorFlow.
For a collection of answers in such a space, we can think of their average as their center, just as we did with the bullseye, and we can infer that this average is close to the “right” answer.
The closer an answer is to this centroid, the “better” it is in some sense. So we can rank answers by seeing how far they are from the centroid. In this way, we can score a set of exam answers without ever having to define a correct answer, based only on the text of all answers…
… assuming, of course, that a few things are true:
- We can map the texts to the right space;
- Everyone is answering the same question. That is, right answers look similar; and
- There are enough diverse answers for different wrong answers to “cancel” each other out
Now, when considering this model for scoring, we should recall the sage advice of George Box: “All models are wrong but some are useful.”
Since all models are wrong, their output should start—not end—discussion; and to know if a model is useful, we need to ask, “compared to what?”
Keeping these suggestions in mind, the methods we use to turn words into numbers have a lot of limitations, and these limitations suggest common ways in which this method might go awry. For example, they don’t do well with idioms.
This means we should look carefully at outliers, like those answers placed at the end of our rankings, since they might really belong somewhere else. For more discussion, see the paper.
But the question is, “even with this need for review, does our machine ranking help us out somehow?” And I think the answer is, “yes.” We just need to ask “compared to what?”
Law schools are notorious for basing grades on a single high-stakes final exam. Despite the proven benefits of frequent formative assessments, they are rarely deployed. Why is that? Because law school classes are large, and grading questions is hard.
The method described here could be embodied in a tool that takes a first pass at scoring to remove some of the friction, and who knows, maybe this would be enough to promote more formative assessment.
Of course, sometimes you can make things too easy. There’s a chance graders could rely too heavily on this first pass, abdicating their responsibilities. So there’s work to do figuring out how best to frame the tool’s usage to improve instruction. But, the status is not quo.
So, how much better than random is this machine-scored ranking? We looked at 1,000+ answers across 6 law school exams (13 questions) to see how many neighbor swaps were needed to transform the machine rankings into that of the human grader. We did the same for random shuffles.
The p-value for a paired t-test of the two populations’ swaps, with the machine-grader acting as treatment, came to <0.000001, allowing us to reject the null hypothesis that the machine’s ordering is just a random shuffle. So, yay! It’s significant, and the effect size is large.
Paper talk
I presented this work at the Computational Legal Studies Conference in 2022 (CLS2022). Here’s the session’s slide deck and a video:
_________________________
* As far as the GC’s office is aware. I have a friend in our library system who likes to point out that the school’s early record keeping wasn’t the best. So, given that we’re more than 100 years old, it’s hard to say for sure.