Lord of the Rings, The Day The Earth Stood Still and Avatar share a couple of common threads – they are major, special effects-heavy studio productions, and the digital effects were created by New Zealand-based Weta Digital.
Formed in part by Academy Award-winning director, Peter Jackson, Weta Digital is now a major global effects studio, with 14 major features under its belt, and another two coming. It has operated in collaboration with A-list filmmakers including Alex Proyas, Stephen Spielberg and James Cameron.
These movies operate to very fine production windows and missing a lucrative launch date or holiday season is inexcusable for a multi-million dollar production.
As a result, Weta Digital cannot afford downtime in its own systems.
Since the organisation is dealing with huge media files, an efficient storage infrastructure is critical to Weta delivering the movie on time. “The demands are getting more and more as time goes on,” NetApp account executive, James Kennedy-Moffat, said.
Kennedy-Moffat operates out of Wellington in New Zealand and, as Weta’s storage partner, NetApp is well-placed to observe the changing demands media and entertainment organisations have for storage.
“Under peak load, when things are really happening and the director is standing over the artist’s shoulder, the artists are all accessing the same data – it gets hot and things slow down.”
It’s only going to get worse. The advent of 3D technology has essentially tripled the workload for a typical effects project. Now an object needs to be rendered three times (right eye, left eye, centre) to create the 3D effect.
While the audience gets to enjoy a very smooth and elegant effect in movies such as Avatar, the spindles in the drives at the likes of Weta are finding themselves working harder than ever.
Further compounding the problem is the fact that – despite needing hundreds of TBs of storage for larger projects – media organisations don’t have nearly as much budget to play with as you might expect.
“If you look at organisations such as Weta, they are very mature in the way they go about their technology – and are now planning far more than they have in the past,” Kennedy-Moffat said.
“No one has an open checkbook these days.”
Backups are an obvious first challenge – it’s not good enough to simply archive an entire render farm to tape – the time involved to do that would be astronomical. Rendering is a 24/7 activity, so the best solution is to use snapshot based data protection.
The second thing NetApp does is make use of 64-bit file systems as well as global namespaces to create large content repositories. This reduces the number required of those 100TB containers, and in turn reducing the administration of the environment.
“Not only are they able to scale, they also need to be able to scale this upwards so any given node in the system is able to form really quickly; this is where we offer the ability to put in some very fast storage controllers, and then accelerate them through the use of flash-based performance acceleration models,” NetApp principal technologies, John Martin, said.
“Media and entertainment organisations are probably the most storage savvy people I’ve come across. They push the envelope a lot harder than others, and they are people that need extreme levels of performance and availability. I think they’ve hit the main pain points of things that people have promised to them in the past and the difference between what’s promised and what’s deliverable.
“Every now and again they do have extreme storage needs and they’ll try and go with something that sounds good and is cheap, but quite often they find out quickly – within a project or two – what really works and what doesn’t.”
The cloud and entertainment
Like much of the industry, cloud computing has presented an intriguing opportunity for media and entertainment organisations. Although Animal Logic, of The Matrix and Happy Feet fame, is still uncertain about the value of the cloud to its business (see breakout below), other vendors have a very positive view on the space.
Hitachi Data Systems, for instance, is actively working with partners building Infrastructure-as-a-Service (IaaS) projects, specifically to be targeted to the media and entertainment industry. According to the vendor’s director of presales and solutions, Australia and New Zealand, Adrian De Luca, it’s the massive raw quantity of storage required that makes cloud attractive.
Media and entertainment organisations also tend to be project-based, which means a pay-per-use model would potentially benefit them more than having in-house technology sitting around unused for periods of time.
“In Australia we have positioned ourselves as a leader of the world in terms of digital media,” De Luca said.
“We have some pretty cool companies in our part of the world like Animal Logic and Foxtel Studios which has become quite a hub. Typically that’s been because of cost factors, but also because of our know-how.
“There’s a lot of money potentially floating around in the media industry, but that doesn’t necessarily extend to the afterthoughts of data protection and archiving.”
Which is where a properly secured IaaS, backed with a trusted consultative approach might find itself a place in the market.
However, CEO of 3 Networks, John Eaves, is quick to point out that while internal clouds are definitely something media organisations are keen to adopt, external or public clouds are simply not supported by a bandwidth fast enough for the more grunt-heavy applications that a media company requires.
“These customers will have five or 10GB fibre links that lead out to different points so they can have instant point-to-point access. No Internet link enables you do to that over the Web,” he said.
“There is a place for public clouds in these organisations, but not at the level of rendering information. You can have many clouds offering many applications, but for the animation side of things, they require a connection that is not a general ‘on Net’ connection.”