But when data gets big, big problems can arise. That’s the message from Nate Silver, who works with data a lot. Silver is the founder of data-driven journalism site FiveThirtyEight (now owned by ESPN); he spoke at the HP Big Data Conference in Boston recently, outlining some of the problems that can come along with big data.
Where to put it
Silver says that even small and medium amounts of data can be difficult to manage, both technically in terms of how to store it and in terms of analyzing it. So, the more data companies have the even more complex the problems of managing it can become. Do you buy hardware? Do you store it in the cloud? How often will you need to access it? Can you deal with latency? These are the types of questions that will help you decide how to manage your data.
One issue with a lot of data is that it can create bias. Let’s say you have two polls, it can be pretty easy to decipher what those polls are saying. Now if you’re analyzing 100 surveys, there can be much more nuanced issues within that data. Ever heard of that saying that you can make statistics say whatever you want? Well the more data you have, the more wiggle room there can be to sway the stats.
Silver referenced a book by Daniel Kahneman titled “Thinking, Fast and Slow,” the point of which is that sometimes people rush decisions based on a subset of data (thinking fast). A better practice is to “think slow” and really rationalize data. With big data, thinking fast (not analyzing the data fully) can lead to false positives.
Silver calls this extracting the “signal from the noise.” Said another way, it’s the problem of finding the needle in the haystack. The more data you have, sometimes the harder it can be to find true value from the data.
That’s not what I was looking for…
Imagine Google Maps giving you directions and suggesting an alternate, “faster” route. You take it only to find that it’s a dirt road under construction. Sometimes big data systems think they have found a shortcut, but in reality, it’s not exactly what the user was looking for.
ARN Connect - Transforming the data centre one block at a time
As businesses realise the importance of removing the IT shackles, in the quest for increased innovation, future-proofing the data centre has become a leading priority. Driven by a relentless need for more agility, control, simplicity and scale, partners are now tasked with creating full solution sets to meet the new demands of the customer. But in a data centre driven world there is no silver bullet, rather an assortment of solutions that when combined, align with the digital transformation priorities of Australian businesses. This ARN Connect event, in association with Exclusive Networks and BigTec, sidestepped the hype to look at the bigger industry picture, peeling back the infrastructure layers to outline how partners can Transform the Data Centre One Block at a Time. Photos by Maria Stefina.
Arrow exclusively introduces Tenable Network Security to A/NZ channel
Arrow Electronics introduced Tenable Network Security to local resellers in Sydney last week, officially launching the distributor's latest security partnership across Australia and New Zealand. Representing the first direct distribution agreement locally for Tenable specifically, the deal sees Arrow deliver security solutions directly to mid-market and enterprise channel partners on both sides of the Tasman.
ARN Roundtable - Finding and selling a niche in a digital world
As digitalisation strategies begin to take shape across Australia, organisations no longer accept the notion of a generalist technology provider. In light of the rise of emerging technologies, partners that can add unique layers of value are gaining greater prominence. Whether vertical or technology specific, forward-thinking partners now realise that staying vanilla seldom brings success, as customers place a premium on niche experts. Yet in parallel to the changing role of the partner, new influencers of IT are emerging across the business, with 72 per cent of IT buying decisions now made outside of the IT department. This exclusive ARN roundtable, in association with Telstra, discussed the changing dynamics of the channel in 2017, outlining how partners can specialise in a digital world, while attracting a new breed of buyer. Photos by Maria Stefina.
iasset.com is a channel management ecosystem that automates all major aspects of the entire sales, marketing and service process, including data tracking, integrated learning, knowledge management and product lifecycle management.