Microsoft partner Versor has planted the seeds of a new data analytics system for the Queensland Department of Environment and Science.
The Melbourne-headquartered consultancy was tasked with developing a new cloud-based data and analytics system to help track flora and fauna for East Trinity Reserve, a natural wetland in Tropical North Queensland.
The department had identified the reserve as a good candidate for its data and analytics project designed to demonstrate the value of cracking open data silos.
Since the remediation of East Trinity Reserve began in 2001, the department has collected data every ten minutes from sensors located in 15 stations. Although it is in possession of all the data, the department said it is often siloed and hard to analyse.
To make it easier for scientists to use the data, Versor began transferring scientific data into Azure Data Lake where it is cleansed using Azure Databricks and turned into an SQL database, which is then available for analysis using Power BI.
The data project, which was rolled out in around two months, was said to have saved the efforts of the equivalent of half of a full-time employee.
Currently, just two out of 20 years of East Trinity Reserve data has been loaded onto the platform, with another 18 to follow.
The workload indicates that once the system is in full production with all data loaded, authorised scientists will have unfettered access to data and tools that dramatically simplify analysis.
Michelle Martens, Queensland land resource officer, said the project will make the data accessible for a lot of people and make a big difference with “wrangling” with the material.
“There were some existing environments in place that the guys are using. But a lot of it was manual work in respect to that,” added Nigel Rablin, principal consultant at Versor. “Pretty much, Michelle and the groups around her for the past 20 years have been collecting and analysing data with whatever tools they could get their hands on. So obviously they can only analyse so much data at a time.”
Following the East Trinity pilot, there are plans to broaden the strategy in order to support the entire science division.
Last year, Versor also helped BlueScope Steel’s "data-driven" digital transformation efforts by revamping its data warehouse and analytics platform.
The company leveraged its experience with Databricks, as well as expertise in Azure Synapse Analytics, to create a data warehouse and analytics platform for the steel vendor.