They’re in everything from mobile phones to cars, buildings and even parking meters.
The IT industry phrase “big data” was coined to describe the situation. It is about the need to handle very large, fast-moving volumes and different varieties of data at high speed.
Volume, variety, and velocity
Experts realised big data could be used to learn new things about businesses, industries, or even people.
The idea is that by collecting more information and analysing it the right way, the results can be applied much more quickly – perhaps even as changes actually happen, in “real time”.
This could potentially improve a vast range of systems, products and services.
Business leaders and politicians have often relied on intuition and emotion to make decisions.
Yet we know from advances in behavioural economics and psychology that this can be a bad idea.
We all suffer from unintentional biases and thinking traps that can skew our beliefs and actions.
Brave new world …
The promise of big data is that it can make real evidence available quickly so both public and private sector organisations can make better decisions.
Many industries – including banking and financial services – use big data. One main way of doing this is “mining” and analysing data to work out how to serve customers better.
Special big-data technology – including predictive, advanced or real-time analytics software – is often used to help organisations work with big data.
Better deals and discounts?
Many of us have felt annoyed by misguided sales and marketing approaches at times.
An eZonomics poll in 2012 suggested that only 52% of people believe store loyalty programmes help them save, and some believe they can actually tempt people to spend.
But if companies monitor customer preferences and circumstances, they can develop targeted deals and discounts for individual customers.
In many countries, inaccurate credit scores, perhaps based on out-of-date information, can cause real problems when applying for bank loans, mortgages or credit cards.
Some companies are adopting big-data technology to more quickly check, combine and update the information held – potentially saving individuals time, money and frustration.
Big IT companies like IBM provide big-data analytics offerings to many of the organisations, including finance firms, that now collect and store vast amounts of information about us.
But there is a downside
How any data is sliced and diced to make sense of it is critically important. Simply collecting large amounts of data won’t give a clear picture of what’s really going on.
Correlation and causation can be easily confused.
Privacy issues have been raised as well: big data must be handled carefully and correctly.
Sometimes data collected can be anonymised before use – this means that personal identifying details have been removed.
Do keep an eye on how your personal information is being used by organisations – whether private companies or government bodies. In Europe, the EU Directive 95/46 EC on data protection says organisations should handle data carefully, as well as give people access to their data when requested.
And don’t forget: you can also often opt out of data collection.
Weighing in the balance
As always, it’s a matter of governments and organisations needing to strike the right balance between the advantage of having more data and the potential risk to individuals.
This guide from the Open Data Center Alliance explains big data in more detail, giving industry examples.