Largest Digital Animation Studio

A Study In Opimizing Render Farm

THE BIG PICTURE

The Company is one of the biggest names in the media and entertainment space, in the world today. The Company develops, produces and exploits animated films and their associated characters across the world theatrical, home entertainment, digital, television, merchandising, licensing and other markets. To support these activities, the company has deployed a huge IT infrastructure that consists of a render farm spanning over 4000-5000 nodes. Each render node accesses data from over 2000 NAS mounts. So this makes it a highly distributed NAS setup. Typically, in such a massively distributed environment, even with the most sophisticated computing animators end up waiting on storage systems to deliver their data and performance is nearly always a concern in such render farm settings. For such a big league company, this usually translates into big losses.

THE COMPANY'S PROBLEM

The main problem with company was that it was difficult for administrators to know exactly what was happening in the 5000 node grid environment. For instance, which applications used which files frequently? Too often animators were repeatedly writing and retrieving the same files from the network storage. With hundreds of jobs scattered over multiple move projects operating in this fashion, it was small wonder that applications were slowing down and the NAS’s were overloaded with I/O demands. I/O latency was high and network bottlenecks were also contributing to the slowdown.

THE DATAGRES PERFACCEL SOLUTION

PerfAccel gave administrators at the company, visibility of the data in the grid and its dynamics. Administrators could see what applications were doing, they could see what files typically were being frequently used, where the network bottlenecks were. For the first time, the data of a 500 node render farm came into full view. Through an analytic mode, PerfAccel allowed administrators the capability to understand these dynamics, at a file level. They could now adopt appropriate data movement and data management strategies, that resulted in improved speeds, lower latency and higher productivity.

BENEFITS

Understanding & View of the Active Data

Administrators could now:

  • Get in depth, granular understanding of how each render node behaved
  • Detailed information about the use of each NAS mount point
  • Get job level details of what data is used and how
  • Track Macro and Micro level reporting of all I/O
  • Identify Hot files and hot areas within each file, Over time, in a particular movie project, the same scene files were now used again and again, improving performance

Control of the Active Data

Administrators could now:

  • Cache only hot areas of the file or frequently accessed files
  • Control which data to cache and which to not
  • Sticky flag indicator for performance critical data
  • Pre-fetch files in cases where a large job was going to be deployed.

Accelerate Data

Administrators could now:

  • Offloading of I/O from NAS to local cache
  • Perform file level caching on the server itself
  • Save network load and improving NAS response time

Value to the Company

As data began to be serviced to applications, from the cache, this translated into not only performance gains for the company; it also did away with the requirement to constantly upgrade expensive NAS storage. So, besides performance and productivity improvements, the company also realized IOPS cost savings.

To sum up, the overall value for the company was:

  • Increased IOPS, Users see an IOPS increase of at least 2X
  • Reduced latency, Users see a latency reduction of 10X-100X within days
  • Faster applications
  • Improved ROI

USABILITY

DATAGRES’ PerfAccel provided administrators with a single pane that combined analytics and insight through performance dashboards, as well as a simple command line, for running commands i.e. creation of cache/source, deletion of cache/source, adjusting sizes and so on.

Its flexible interface let users configure their own policies of persistent cache, pre-fetching, predictive cache, real-time cache size configuration and auto-caching hundreds of NFS mount points.

PerfAccel Commands were easy to use and an administrator could learn them in a few minutes. The company's system administrators were able to learn the commands easily.

Try PerfAccel Now

Download PerfAccel right away for Free and see how data intelligence can dramatically improve the visibility, control and acceleration of your applications as well as data storage network


Get hands-on with many exciting features like PerfAccel File Analytics, IOPS Analytics, Process Analytics, Context Analytics, Intelligent & Dynamic Data Management etc.