The Australian taxation Office (ATO) is gearing up to switch out its faulty Hewlett Packard Enterprise (HPE) storage hardware over the Easter period, with online services again set to be taken offline.
The move to replace the storage hardware comes months after an “unprecedented” failure in December last year of storage hardware that had been upgraded in November 2015 by HPE, which resulted in widespread outages among many of the ATO’s systems.
In a statement delivered at the Tax Institute National Convention 2017, Australian Taxation Commissioner, Chris Jordan, flagged the impending hardware replacement, once again apologising to Australians for the fallout of the hardware failure.
“As you probably already know, when we replace the faulty SAN [storage area network] equipment and migrate data to the new one, we will be deliberately taking our systems offline to minimise any risks to future performance and stability,” Jordan said. “We hear loudly and clearly how important that is to you.
“Our schedule for these planned outages is intended to occur at Easter, in addition to our usual scheduled outages on the first weekend of April and May. From consultation, we understand these are the most sensible and least inconvenient times for the tax profession and the broader community,” he said.
At the time of writing, the ATO’s system maintenance schedule indicated that most online systems would be unavailable from 31 March to 3 April, and again from 13 April to 18 April.
Jordan reiterated the cause of the initial outage and subsequent intermittent outages of systems affected by the hardware issues, citing hardware faults in the ATO’s primary SAN, provided and maintained by HPE on the agency’s behalf.
“This SAN hosts ATO applications such as the Tax Agent and Business Portals and the Australian Business Register (ABR), and its failure caused the initial disruption to a range of ATO services on 12 December,” he said.
A subsequent system outage in February was due to critical hardware becoming dislodged when a faulty cable was being replaced as part of the work to stabilise the faulty SAN following the December incident, according to Jordan.
This, in turn, caused the SAN to enter into shutdown mode, which disrupted a range of critical ATO services, similar to the scope of systems and services impacted by the initial outage.
“The SAN is not old equipment, and in fact had only just been installed in November 2015 and was seen to be ‘state-of-the-art’ at the time,” he said.
“The 3PAR SAN hardware provided to us as a data storage facility by HPE is similarly installed in many large organisations here and around the world. In fact, there are 67,000 of these SANs around the globe, of which we have two; supporting major business operations including mission-critical ones like trading floors and banking platforms.
“Nothing like the failure we experienced has been experienced by any HPE client before. So this was an extremely unusual and unfortunate event,” he said.