It Can't Fail to Fail

What are these Constitutional Rights you speak of?

...the Florida State Department of Juvenile Justice will use analysis software to predict crime by young delinquents, putting potential offenders under specific prevention and education programs. Goodbye, human rights!

They will use this software on juvenile delinquents, using a series of variables to determine the potential for these people to commit another crime. Depending on this probability, they will put them under specific re-education programs. Deepak Advani—vice president of predictive analytics at IBM—says the system gives "reliable projections" so governments can take "action in real time" to "prevent criminal activities?"

Really? "Reliable projections"? "Action in real time"? "Preventing criminal activities"? I don't know about how reliable your system is, IBM, but have you ever heard of the 5th, the 6th, and the 14th Amendments to the United States Constitution? What about article 11 of the Universal Declaration of Human Rights? No? Let's make this easy then: Didn't you watch that scientology nutcase in Minority Report?

Sure. Some will argue that these juvenile delinquents were already convicted for other crimes, so hey, there's no harm. This software will help prevent further crimes. It will make all of us safer? But would it? Where's the guarantee of that? Why does the state have to assume that criminal behavior is a given? And why should the government decide who goes to an specific prevention program or who doesn't based on what a computer says? The fact is that, even if the software was 99.99% accurate, there will be always an innocent person... that is exactly why we have something called due process and the presumption of innocence. That's why those things are not only in the United States Constitution, but in the Universal Declaration of Human Rights too.

Other people will say that government officials already makes these decisions based on reports and their own judgement. True. It seems that a computer program may be fairer than a human, right? Maybe. But at the end the interpretation of the data is always in the hands of humans (and the program itself is written by humans).

But what really worries me is that this is a first big step towards something larger and darker. Actually, it's the second: IBM says that the Ministry of Justice in the United Kingdom—which has an impeccable record on not pre-judging its citizens—already uses this system to prevent criminal activities. Actually, it may be the third big step, because there's already software in place to blacklist people as potential terrorist, although most probably not as sophisticated as this.

IBM clearly wants this to go big. They have spent a whooping $12 billion beefing up its analytics division. Again, here's the full quote from Deepak Advani:

Predictive analytics gives government organizations worldwide a highly-sophisticated and intelligent source to create safer communities by identifying, predicting, responding to and preventing criminal activities. It gives the criminal justice system the ability to draw upon the wealth of data available to detect patterns, make reliable projections and then take the appropriate action in real time to combat crime and protect citizens.

If that sounds scary to you, that's because it is. First it's the convicted-but-potentially-recidivistic criminals. Then it's the potential terrorists. Then it's everyone of us, in a big database, getting flagged because some combination of factors—travel patterns, credit card activity, relationships, messaging, social activity and everything else—indicate that we may be thinking about doing something against the law...

While everything may seem driven by the desire to achieve better security, one single false positive would make the whole system unfair. And that's not even getting into the potential abuse of such a system. Like the last time IBM got into a vaguely similar business for a good cause, during the 1930s. They shipped a lot of cataloguing machines to certain government in Europe, to put together an advanced census. That was good. Census can improve societies by identifying needs and problems that the government can solve. At the end, however, that didn't end well for more than 11 million people.

And yes, this comparison is an extreme exaggeration. But one thing is clear: No matter how you look at it, cataloguing people—any kind of people—based on statistical predictive software, and then taking pre-empetive actions against them based on the results, is the wrong way to improve our society. Agreeing with this course of action will inevitably take us into a potentially fatal path.

Agreed, but it's just what our Company handlers have wanted. These critiques of course don't touch on another flaw: all software delivers according to the GIGO principle. Garbage In, Garbage Out. Somebody like J. Edgar doesn't like you and you're bound for the pokey, no matter who you are.

Ah yes, J. Edgar, the head of the FBI who insisted there was no organized crime left in 'Merika, 'cause they owned him.