What the College Board Could Learn from the Netflix Hit, NARCOS

Updated on April 19, 2023

If you haven't been following the fallout from the College Board's massive data breach, then you may want to begin by reading this article from Reuters.  

Here's a quick recap: About 400 unpublished test questions were exposed, though it's unclear just what exactly "exposed" means. The exposed test items include the questions and answers to 21 reading passages and more than 150 math questions. As a side note, there's a whole subplot involving an FBI raid of the home of a former College Board employee, who seems to consider himself a whistle blower. And to complicate matters, some Chinese test-prep schools claim to be in possession of the materials. 

In deciding how to deal with this problem, the College Board may want to check out NARCOS, the hit Netflix show, which follows two DEA agents as they attempt to capture Pablo Escobar. The parallels aren't immediately obvious, but the security challenges faced by the College Board are in many ways similar to those faced by the men and women tasked with thwarting international drug traffickers. 

I think it's pretty clear that the College Board's number one priority should be to make sure that no students or tutors are able to exploit the security breach and gain an unfair advantage on future SAT® tests. In order to prevent this from happening, the College Board needs to do 3 things, and quickly.

1.) The folks at the College Board must compile a comprehensive list of all the test items that were compromised.

It seems that they've already done this. Nice work. 

2.) The folks at the College Board must provide assurances that no compromised test items will ever show up on future SAT® tests.

They've done this, sort of, depending on your reading of the word "upcoming." If upcoming means "occurring in the future" then we're good. If, on the other hand, it means something like, "between now and the end of the year" or "already assembled," then we've got a major problem. Fortunately, the College Board can rectify the situation by declaring unambiguously that no stolen test question will ever appear on any future SAT® test. Problem solved! It's a simple fix, but it's 100% necessary, considering the alternative. 

In order to be justified in using the stolen test questions, the College Board's investigations would have to conclude—with 100% certainty—that the stolen test questions were never available to unscrupulous test-prep tutors or students. Even then, the College Board would still have to convince the public that this conclusion could be believed with certainty.

Including any stolen test questions on any test prior to ruling out the possibility that they were viewed by bad actors would be detrimental to the College Board's credibility at a time when it has faced significant criticism for recycling compromised test questions. Using these stolen test items without the certainty that they were never compromised would play into an existing narrative.    

3.) A decision will have to be made as to what should be done with the compromised problems.

Supposing for a moment that it can somehow be shown conclusively that the stolen test questions were never leaked publicly, but rather transmitted securely from the person who stole them directly to team at Reuters, then perhaps a case could be made for using those stolen problems on future SAT® tests. 

But consider what this evidence would necessarily entail. And who could credibly provide such evidence? Certainly no one at the College Board, Reuters News Agency, or even the FBI is in any position to guarantee that the stolen test questions were never photo copied, reproduced, posted online, or otherwise shared with anyone else. 

Only the thief—supposing they’re discovered—could claim to know this for certain. Of course, there's a problem with simply trusting the word of the person who stole the test questions. Besides the obvious lack of credibility, there's an issue with incentives. If the thief were to admit having shared the questions with anyone else, that admission would warrant additional charges.

As a result, the thief would be strongly incentivized to deny having shared the stolen SAT® questions whether or not that is actually the case. Consequently, we cannot ever know for sure the extent to which the stolen test items were available to bad actors. In light of this fact, we must acknowledge the possibility that the stolen test questions were made available to bad actors.

Thus, the inescapable conclusion: the stolen test items must not be used on any future test.

The only options are to destroy the test items, use them internally as question archetypes, or—and this sounds crazy—make them available to the general public. 

There are pros and cons to each choice, but I cannot imagine that destroying the test items is the best course of action. At the very least, the test items could provide some use as an internal resource for test developers. Even so, if the stolen questions (particularly math problems) were ever to be used as archetypes for future test questions, some advantage could be gleaned by a bad actor who acquired the stolen test questions. Of course, it's worth noting that some bad actors already claim to be in possession of those test questions.

I do not know whether or not to believe the owners of test-prep schools in China who claim to possess the questions. Nevertheless, I do know that some of these international test-prep schools operate as sophisticated cartels, dealing in illegally acquired intellectual property and enriching themselves in the process.

The cartel comparison is actually quite apt. And it shows us the best method for dealing with the compromised test items. We should release the stolen test items to the public. All of them. As soon as possible. 

The College Board is up against an international cartel. Just as a narcotics cartel succeeds or fails based on the scarcity and potency of its product, so too do international test-prep schools.

If we assume that at least some of these schools do in fact have access to these stolen test items, then they are in possession a product that is incredibly rare and incredibly potent. Unlike narcotics traffickers, the bad actors in test prep had a major distribution advantage: their product can be digitally replicated and instantly transmitted around the globe.

The question is not how the College Board can stop these cartels from stealing and trafficking intellectual property. It cannot be done. The real question is how the College Board can effectively prevent the cartels from gaining an unfair advantage as a result of their illegal activity. It turns out, this second question is a very easy one to answer.   

Consider Milton Friedman's analysis of a similar situation:

Today, the College Board finds itself in a position that is analogous to that of the United States government attempting to carry out the war on drugs. Every success in the war on drugs, a major drug bust for example, is necessarily accompanied by a rise in the price of the drugs that still exist in the market.

The same phenomenon occurs when the College Board attempts to prevent the spread of the stolen test items. Any success, preventing the general public from seeing the tests for example, only serves to enhance the unfair advantage enjoyed by the bad actors who already possess the tests. The College Board's prohibition policy will only succeed in protecting the profits and prestige of the test-prep industry's bad actors.

Just as with narcotics, the alternative to prohibition is "legalization." But while legalization of narcotics is understandably controversial, there is not likely to be any opposition to 400 SAT® questions hitting the streets. There's literally no downside for the test takers.    

Suppose the College Board were to tweet a link to all of the stolen test items tomorrow morning. What would happen?

By the end of the first week, a few thousand tutors and perhaps fifty thousand students would have downloaded the test questions and spent a few hours studying them. But before lunch on the first day, any unfair advantage enjoyed by the bad actors in the test prep industry would have been completely eliminated. Problem solved. This would be a major blow to all bad actors who make their livings by tutoring students with unreleased tests and these stolen tests. 

In writing this, I should acknowledge that I'm self interested—but not in the way you're probably thinking. It's much better for my company if these stolen test questions are never released because we've put in the time to meticulously develop and test more than 1600 representative SAT®-style practice problems, each with it's own video solution.

Tutoring companies use system because our custom branded software and workbooks enable them to deliver a test-prep experience that cannot be matched by any old tutor with a book he got on Amazon. The more problems the College Board releases, the more that we'll need to keep developing in order to maintain our competitive advantage. Good thing we enjoy creating new curriculum!

Even if it's not in my direct self-interest for the College Board to release the stolen test questions, I still believe it's the right thing to do. The College Board's primary objective must be to prevent unscrupulous tutors and students from gaining an unfair advantage on the test. And with that in mind, the solution is clear: release the stolen test items as soon as possible.     

If you enjoyed this post, please take a moment to subscribe. You can be sure that we will never send you anything that we don't think you'll find useful or enjoy reading. 

If you'd like to learn more about how you can grow your tutoring company, then contact us to schedule a time to speak with us!