I have been working in analytics for over twenty years, and have witnessed first hand how these technologies have made the world a better place. I’ve seen thousands of examples, from every type of corporate efficiency imaginable, to improving customer satisfaction at theme parks and making better use of limited blood supplies.
Yet we’ve only seen the tip of the iceberg when it comes to “big data analytics.” There are many different definitions of this term, but my favorite is “using data that was previously ignored because of technology limitations” — and those limitations are falling away fast, leading to cascades of new opportunities.
We can now gather, correlate, and analyze information in ways that were unthinkable in the past. The book “The Human Face of Big Data” does a particularly good job of rounding up some of the most interesting stories of how these technologies will touch our lives.
However, with great power comes great responsibility. Analytics is a very powerful weapon, and weapons can be abused.
The past clearly shows that without proper controls, there can be irresistible temptations for companies and governments to combine data in ways that threaten personal liberties. Misuse of every previous data gathering technology has eventually come to light, sometimes only decades after the facts, leading to new laws re-establishing privacy limits.
Modern technology makes the potential threat much greater than in the past. Combining “metadata” from online activities, mobile devices, payment systems, surveillance cameras, medical histories, and social networks can reveal every nuance of our online and offline lives.
While nobody is arguing that the Thought Police exists today, for the first time since I read George Orwell’s “1984” thirty years ago, we have their technology capabilities:
“There was of course no way of knowing whether you were being watched at any given moment. How often, or on what system, the Thought Police plugged in on any individual wire was guesswork. It was even conceivable that they watched everybody all the time. But at any rate they could plug in your wire whenever they wanted to.”
Analytics is, at best, a wonderful opportunity to shine light into the dark, to reveal what was previously concealed, and make it better. People and governments must be in the forefront of establishing clear, transparent guidelines that make the right tradeoffs between the public good and citizen’s rights.
We should not wait for abuses to come to light before acting. One common refrain that defends data gathering is “if you’re not doing anything wrong, what are you worried about?” – if this were true, it should surely also apply to the organizations currently storing and using our data in “secret” ways today.
I believe the vendors of analytics software should play a part in this debate, help encourage data safety and transparency, and provide the technology features that makes it easy for organizations to support these initiatives.
Comments
8 responses to “The Ethics of Big Data: Vendors Should Take A Stand”
[…] Big data is an incredibly powerful weapon. We must use it wisely and industry vendors should be in the forefront of addressing the ethical concerns. […]
I asked a question about ethical big data practices for corporations at a big data road show put on by a major vendor a few months ago. They didn’t have a well thought through response. Since much big data technology is open source, providing leadership on processes and ethics would be a way for vendors to differentiate themselves.
[…] The Ethics of Big Data: Vendors Should Take A Stand | Business Analytics […]
A proposal of a “Code of professional conduct” for data scientists: http://www.kdnuggets.com/2013/03/data-science-code-professional-conduct-proposal.html
Will Titanic Data Hit an Iceberg?…
<p>I had a fun conversation on Twitter with Sam Perrin (@SamPerrinND) who is with <a href="http://www.nakeddata.com/" target="_blank">Naked Data</a> in Melbourne, ……
Dave,
Good point — and I’m also wondering at what point people will remember that there are many other companies beyond Telcos that store interesting metadata in their systems (including, to take a random example, cloud EPM vendors?).
There will indeed be areas that can’t be talked about when it comes to Government, but the commercial opportunity for abuse is just as important an issue…
Hi Timo,
While I agree with you in spirit, the closer your company is to actually working with the intelligence community on these topics, the more you cannot say anything about what you are doing or engage in the debate because doing so starts to draw on lines that you only know about because you are doing classified work. That’s why you can tell who is doing classified work by their PR strategy. Some vendors like Attivio are out doing interviews about mining unstructured data — which to me suggests they aren’t doing such real work while those who I know are doing such work are silent, as they should be.
It reminds me a bit of the dilemma Sen Wyden faced — as an intelligence committee member he knows a lot about what’s going on but he can’t talk about it because it’s classified. Ergo, he put together pretty carefully worded communications to try and highlight issue which we definitionally dangerous for him to talk about http://www.wyden.senate.gov/news/press-releases/wyden-places-hold-on-fisa-amendments-act-extension
So yes, they can join a general debate about privacy. But no, they can’t join a debate about how the IC collects and monitors information on different classes of folks.
Here’s a nice related post with more details of the type of functionality that should come as part of any customer data system: http://blogs.hbr.org/cs/2013/06/beware_trading_privacy_for_con.html