Machine data is the data that has grown exponentially over the last three to five years, according to Splunk Asia-Pacific Cloud sales director, Daniel Miller. And this data needs to be secure and visible.
Miller was speaking at a session at the AWS Summit 2016 in Sydney.
“As organisations increasingly now relate to customers on mobile applications, leveraging Cloud technology, etc. there’s a lot of data created by these machines. It’s out of control.
“So when it comes to harnessing and extracting value, people are starting to realise the value of having a Big Data platform,” he said.
“Just as 20 or 30 years ago, we built an enterprise data house so we could put all these little repositories of structured data sitting around the organisation that contained records of transaction, people now realise that having a big Data platform gives them better visibility into what their business is doing.”
Miller said as businesses change, evolve and run different data sources, they’re starting to realise the value of building a data platform for all of this machine data. But building a platform isn’t quite as simple as it seems.
“You need to think about how to collect the data, where to store it, how to manage it, exploit, search and correlate it, then visualise and export it, generate it, automate it and enable predictive analytics around it.
“Then you are faced with the task of enriching it with other things and develop more on top of it and do all of it at scale and in a secure manner,” he said.
According to Miller, it is important when people are embarking on a Big Data journey to think about where they’re going to go.
“You don’t necessarily need to know all the questions you’re going to ask. We just need to build this big environment and we need flexibility and agility of a Big Data platform.
“As the data sources that we’re now interested in are unstructured, temporal and event based, you can’t structure that data up as you collect it. Applying a schema at write will send you down a narrow path that is very hard to get out of,” he explained.
But just talking about getting into the Bid Data space isn’t enough. People want results and value out of it for IT and business.
“The reality is there isn’t an IT and business division anymore. If you were to think about the business, none of them can survive now without IT doing what IT does. The lines are blurred and the data is important now for everybody.
“And as the lines of businesses blur, people are beginning to think if IT as services. As you move things to the Cloud, make sure you have the appropriate controls and visibility in place,” he said.
Miller also mentioned that as we move into a world of IoT and industrial data, the plethora of smart devices and sensors will result in an interesting future for businesses.
“It creates scary and weird possibilities but as machines start to talk to each other and make decisions, we will have a lift into that conversation and it will be a journey,” he added.