If you thought western democracies like the Netherlands could never become surveillance states, guess again. Enter the Dutch SyRI system. A system that would make George Orwell cry.
What is SyRI?
SyRI is an acronym for ‘Systeem Risico Indicatie’ which is Dutch for ‘System of Risk Indication’. The government feeds data about citizens on welfare into a computer system and out pops a risk analysis that tells the government who might be committing fraud.
Sounds legit, right. Each year, millions of euros are — probably — lost to fraud. Using AI to find fraud seems nice.
What is the problem?
The idea is sound, however when you learn what the government feeds into SyRI, you might get uncomfortable. There are seventeen categories of data, including your known possessions, financial data, taxes, water and power usage, as well as your insurance data and a slew of other data. Basically, everything the government can get their hands on.
That alone is pretty scary, but add to that, that these people are not under suspicion for any crime. The fact that they receive welfare is enough to make the government feed their lives into a computer and see what shakes loose.
A second problem is that it is unclear what SyRI does exactly. The people building and using the system know — at least, I hope; neural networks in AI don’t divulge their secrets by default. The government doesn’t want to share the information, because that would make it easier to circumvent. Apparently, this outweighs due process.
Trusting the government
The government has a lot of data on us. In the Netherlands, we’ve come to believe that — since we live in a democracy with checks and balances — we can trust the government. In recent years, though, there has been an increasing number of breaches of that trust.
The government is creating more and more systems like SyRI. It demands you submit your privacy in exchange for receiving welfare or other benefits. Then they turn around and assume you’re a committing fraud by default. Each year new laws are passed to allow data dragnets and invasive systems to counter this supposed fraud.
Populist sentiments have sky-rocketed across the world. The Netherlands is no exception. We have our own versions of Trump and Bolsonaro, although they haven’t accrued as much power. Still, generally, parliament has been increasingly pushing for repressive measures and politicians increasingly thrive on fear-mongering. They disregard warnings from other branches of government and simply push through scary things like SyRI. You don’t want these kind of systems with people at the helm that will — for example — lock children in prison without a second thought.
Third, the Dutch government is terrible at IT. Projects to create unimplementable IT systems can fester for years, pissing away hundreds of millions of euros. Worse, because of the money sunk into them, substandard buggy systems go live all the time. Data leaks occur. So, imagine all your private data fed into a system that erroneously marks you as a criminal, taking away your privileges without due process. And then that data might end up on the dark web.
Lawsuits
Luckily, not everybody is asleep. Recently, several privacy organizations and individuals fired the first salvos in a lawsuit against SyRI and the government. Populists like to hate on the EU, but in this case the EU guidelines for basic human rights can be used to challenge the government.
Interesting side note: the Netherlands doesn’t allow judges to look at the constitution for guidance. Parliament supposedly looks at that when passing laws. Unfortunately, there is no mechanism to enforce that, allowing parliament to simply ignore the constitution, making it a more or less dead document in practice.
Anyway, law suits are happening, and they might stop this system. Might. This time. More is probably coming.
Conclusion
Modern technology allows us to do really cool things, but it also allows bad or stupid people to do really terrible things. We can oppress people in a way and on a scale that has never been possible before in history. And the only thing standing between us and oppression… is us.
