How To Get a Flock of SATA Hard Drives to Soar in the Clouds

First Half of the Story
What is a perfect use for a bunch of servers with 7200 RPM SATA hard drives? Cluster them into an object-based storage system that becomes the storage pool for tens or hundreds of client hosts and KVMs. Why an object-based storage system? Because object-based storage such as Ceph stores data more efficiently while significantly improving the I/O performance of the drives by parallelizing them across different storage nodes.

Now, it would be even nicer to build this all with SSD instead of SATA hard drives but all SSD for the full deployment would not only significantly raise the cost, it is also unnecessary. The performance can be dramatically increased (5-8X) simply by adding PerfAccel along with a slice of SSD storage per server. This is because PerfAccel, acting as a Data Service in the Software Defined Storage (SDS) Layer, can dynamically place hot data on storage of your choosing. Therefore, when the most active data is now residing on, say, Intel® SSD, the overall I/O performance of the object-based storage is dramatically increased. And PerfAccel will perform this all while also providing a rich framework for analyzing workloads! So to show this, DATAGRES set up its own Ceph cluster using 4 Intel®-based storage nodes each with plenty of inexpensive storage (9 x Seagate 3.5” 3TB 7200 RPM SATA HDD) and a sprinkle of high performance, low latency caching space using Intel® SSD (2 x Intel® 2.5” 480G DC3500 SSD). The Ceph cluster now becomes the base storage for two OpenStack servers hosting multiple VMs each.

Ceph cluster by DATAGRES

With PerfAccel using only 400 GB per SSD, (the remaining 80 GB used by Ceph’s Object Storage Devices), the results were obvious.
Sequential I/O with PerfAccel    Random I/O with PerfAccel

The left graph shows sequential read and write tests while the right graph shows random read and write tests. The first group of columns in each graph shows the results for read tests and the second groups of columns are the write tests. The tall light blue column in each of the first group of columns shows the 5 to 8 X increase in performance of I/O throughput of reads once PerfAccel has dynamically placed hot data on Intel SSD storage.

Second Half of the Story

Yet, this is only half the story. PerfAccel also reveals a rich amount of information about the I/O moving between clusters and hosts while also increasing its throughput. While tests were run, PerfAccel analytics where also run on the storage nodes and reports were generated on the Ceph Cluster and VM disks. This means that in-depth I/O information is known at the Object Storage Device (OSD), disk device, node, and even VM disk levels.

A wealth of metric are available at all levels yet, just for a sampling, here is an example of information available at each level:
OSD Level
PerfAccel shows the amount of I/O transferred in terms of both data size and number of I/O operations, I/O throughput, and I/O latency for each OSD.
Disk Device Level
PerfAccel gives a top view of all of the VM disk images that are in use providing a good idea of what’s going on with the disks in the Ceph cluster as a whole.
Node Level
PerfAccel gives a top view of all of the Ceph storage nodes that are serving the VM disk images providing a good idea of what’s going on with each node of the Ceph cluster.
VM Disk Level
PerfAccel gives information about each VM disk such as the VM name to which each disk is attached, the nature of I/O that is happening on the disk in terms of reads or writes, and nodes from which this disk is being served.

The Whole Story
Give PerfAccel some high performance, low latency storage for dynamic data placement and it will both greatly increase read throughputs of your Ceph cluster while giving you in-depth detail about the I/O movement between disk device to VM disks at the same time. And that is how you get a flock of SATA hard drives to soar in the clouds!

To get the full story, please find the technical paper written by DATAGRES in collaboration with Intel.

Leave a reply