Big Data is not a myth, but it is extremely overhyped. That’s the resounding sentiment among channel players.
Some executives are even feeling the need to put their hands over their ears to avoid the cacophony and the accompanying investments in the latest “must-have” corporate toys.
But if the Federal Government’s recent National Commission of Audit is anything to go by there are a lot of people taking what’s been dubbed Data 2.0 (akin to Web 2.0) very seriously.
The much-vaunted Commission has devoted an entire section to the role of Big Data. It has even recommended the government prioritise several Big Data projects in major service delivery agencies and increase the amount of anonymised data to be made available through Dat.gov.au. This follows the example set by the US and the UK governments.
“The commission considers the government should do more to open access to its administrative data holdings, including medical data [the Pharmaceutical Benefits Scheme, Medicare Benefits Schedule and Medicare] and welfare and social data [social security payment data],” the report said.
“At the state level, Western Australia has demonstrated that such administrative data can deliver substantial benefits with low risks, manageable costs and in ways that protect people’s privacy, for example, by anonymising data before it is released.”
While this would create opportunities for the private sector, most companies are looking for more granular insights and the big end of town in Australia’s finance, telecommunications, retail and health sectors are bringing this to bear as we speak.
However, the common theme among Big Data vendors is that the channel has a pivotal role to play, with system integrators, “data stewards” and “data scientists” critical to its proliferation over the next decade.
InfoReady managing director, Tristan Sternson, said data was the new oil the world could not live without. It encapsulates new technology and new ways to store and interrogate unstructured data sources,” he said. “It is Data 2.0, like we have Web 2.0.
“2014 is experiencing the largest uptake of Big Data to date, driven by large and complex data sets requiring immediate results.”
He said social analytics of customer behaviours, for the simple solutions, and network traffic and reliability, for the more complex, were the most common projects in play at the moment. But Australia is still lagging. “Generally, we are the most advanced in Asia-Pacific, but it seems we are trailing world leaders by a good couple of years,” he said.
“This is because we have a mature but small market in Australia where the reliance on data is not as critical as it is in a more highly competitive market with a larger population, so the urgency is not there as much.”
According to Gartner, the business intelligence and analytics market in the US grew about eight per cent to $US1.4 billion in 2013. But despite Big Data hype reaching a fever pitch last year this did little to move the dial for analytics.
Only eight per cent of organisations surveyed actually deployed a Big Data project, with 57 per cent still in the research and planning stages.
However, more recent research from Dell found 41 per cent of mid-market executives said they had one or more Big Data projects already in place, with another 55 per cent planning to start one in the foreseeable future.
Cloudera vice-president, APJ, Chris Poulos, said it was the big companies that were trialling and implementing projects, as opposed to the mid-market.
“We’re at a tipping point in Australia in terms of interest in, and adoption of, Big Data,” he said. “Some organisations have dived in deeply – banks in particular will see immediate benefits from this more advanced collection and interrogation of Big Data.
“We have been using expensive and limited 1980s technology to hold and interpret data, but the shape of data today is very different; it is unstructured, it is social, it is verbose. That type of data can’t be easily interrogated unless you use new technology.”
Steep rise in interest
He said he was seeing a steep rise in interest and had customers in the finance, telco, retail and government markets.
“These are the organisations that want to not only collect but to also analyse, and be able to respond to, data in real-time. But Australia’s barely scratched the surface with Big Data.”
Poulos said his company sold only through the channel and that it was crucial to its success.
“We are already working with a number of large systems integrators but we are also looking for specialist companies who have a different way of thinking when it comes to extracting the value of IT, who understand how to install Big Data systems, to help their customers use that data,” he said.
“But given this sector is still relatively new the issue is educating channel partners quickly – and thoroughly – enough so they can help us leverage the demand for our systems.”
Fear and resources are the only things holding Big Data back in Australia, Poulos said. “The fear of something that requires a new way of thinking, and the resources to correctly deploy Big Data projects.“
Elasticsearch vice-president of sales, Justin Hoffman, said the term Big Data was overused and overhyped, but that the channel had an important role to play in a skyrocketing market.
“Over the past year, growth and interest in Big Data in Australia has skyrocketed and channel players have a massive opportunity,” he said. “Systems integrators often play a critical role in the deployment of a Big Data solution as there are typically many moving pieces that need to fit together.”
He said the channel also served as a critical point of entry for cashed up Silicon Valley businesses looking to expand to new markets. “It will be very important as we continue to expand our footprint in Australia. It has the experience with a wide variety of complementary technologies and strong localised relationships.”
Dimension Data Learning Services chief executive (DDLS), Mal Shaw, said his company was seeing a spike in demand for Microsoft SQL server training. “Australian corporate and government clients are investing significantly in training to support their implementations of SQL Server,” he said. “Big Data is not a myth.”
DDLS NSW instructor manager, Richard Boxall, said companies were contemplating new roles such as data steward and data scientist in the search for gold in the data mines.
“The data steward is someone who understands what the data are and where to look,” he said. “The data scientist has the analytical and statistical skill set to analyse the data and thereby create value from it for the business.”
Gartner analyst, Ian Bertram, said Big Data was a bit of a misnomer, but that growth was strong in the Australian market.
He said it is causing conflict in organisations between the “suits”, who take a structured approach, and the “hoodies”, who like to experiment. “Integration is going to come back with a vengeance and that’s where the channel is very good because they get down and dirty and do those integration projects.”
Fujitsu solution director, Chris Gaskin, said, typically, the opportunities were on a smaller scale in Australia than Fujitsu had seen in other countries.
“In the future, the implementation of Big Data will rely on deployment of infrastructure and that is where we see the channel will play a role,” he said. “The channel model allows for a more rapid discovery and implementation of Big Data as players bring a wide set of toolkits and functions.”
He said there was a great opportunity for partners with a definite value add, such as Splunk, which solves a measurable set of use cases, by easing the way to get massive current data from machine logs.
Where do YOU start?
Cloudera chief architect, Doug Cutting, said most organisations start with either cost savings on an existing application or a brand new application, or a blend of both.
“To get cost savings, they move some of their expensive data warehouse workloads onto a Hadoop-based solution.
“They identify a type of analysis that wasn’t previously possible but that would have significant impact on their business.“
He said a Proof of Concept system to validate the idea is created, then deployed into production. Cutting said Big Data was no more a myth than the Internet was in the 1990s, and this time he did not see the “irrational exuberance“.