Last week developers of enterprise applications got some new toys to play with in the storage memory realm. The newly released ioMemory SDK will grant developers the ability to better utilize the potential of Fusion-io’s line of enterprise flash memory storage. Fusion-io expects the SDK will simplify code bases while providing a sizable performance boost.
We begin our discussion with Brent Compton, Senior Director of Product Management with Fusion-io. The first entry in the series will focus on the need for the new SDK and begin to discuss some the functionality developers should expect.
Ben: Brent, I’m looking forward to this interview because I get to wear both my administrator hat and my developer hat! To start off with, can you please tell me a little about what you folks have been doing up to this latest announcement?
Brent: In short, Fusion-io has pioneered a next generation flash memory storage platform for enterprise data centers. Our flagship product, ioMemory, is a hardware-software combination which provides enterprise grade high performance flash memory for enterprise apps.
You may have read academic white papers over the last couple of years talking about providing native access to this flash memory tier. As far as we’re aware, we’ll be the first ones to offer that through this software development kit. Before we get into what will be provided to developers, let me give some brief history for context.
As you are aware, around 1956 IBM defined a half century of I/O with a single word, seek, when they invented the disk drive. To exploit this new hardware innovation, they also gave developers a primitive to access data randomly.
Random access to data transformed software development, which before had been working with sequential access devices. In the half century that followed, virtually all operating systems, databases, and applications have been playing to that tune. In other words, they have been built to work with. and more recently, sometimes to work around, that rotational latency.
When flash memory first came on to the scene as an enterprise grade device in 2007 or so, it was disguised to look like a disk.
This was an important first step. As enterprises learned to trust the medium and learned where to use it, it was important that it be offered transparently and easy to integrate. So the industry disguised flash memory to look like a disk for ease of adoption.
Now that’s where Fusion-io’s path diverges. Unlike the other vendors who are classically described as “SSD vendors,” we’ve always described our media as ioMemory. From the beginning we chose a fundamentally different architectural path, looking forward to the day when this would be a new tier, a hybrid of memory and storage.
And yes, while the important first stage of adoption was to disguise it to look like a disk, it’s now mainstream and obviously a great deal of market enthusiasm surrounds the adoption of flash memory and flash solutions in general. With flash memory deployments now commonplace, it’s time for us to crack open and exploit some of the native underlying properties of flash which have been effectively hidden to date.
Returning to the comparison with the disk drive where we began, the parallel we like to use with the new ioMemory SDK is that when application I/O converted from sequential access tape to random access disk, a couple of new programming primitives were exposed to allow developers to exploit that hardware innovation. Likewise, to tap into the native properties of flash memory hardware/software innovation, a new set of primitives, or APIs, need to be exposed to developers for them to harness and exploit those native characteristics.
Ben: Let’s start talking about that. What functionality should developers expect from the new SDK?
Brent: There are four key capabilities that developers will be provided. The first capability will be direct-access I/O. Direct-access I/O enables developers to bypass file systems the kernel block layer, and other I/O layers tuned over decades for disk drives. Application I/O is plumbed directly through to the ioMemory device.
The directKey-Value Store API is an example of APIs found within the direct-access I/O family. One of the native characteristics of the ioMemory flash translation layer is that it is natively a key value store. Everything that it stores inside of its log append structure are a key value pair. It stores a logical block address together with the associated data with every single write.
As you’re aware, the world is abuzz with activity right now in the whole unstructured data, NoSQL, key value store solution space. All kinds of vendors new and old are building solutions for that market. We are already working with a number of them.
Minimally, we enable them to shorten the code path when performing key-value store I/O operations by eliminating duplicate logic in their code and in the ioMemory software layer. Maximally, we actually reduce their code because we’re doing some of the low-level heavy lifting for key value store implemented natively on flash, exporting to them a key value store API interface.
Using this directKey-Value Store API, they can effectively pass in the key and the value, and we take over from there providing the hashing, collision management, and native storage and retrieval of key-value pairs. So it offloads a lot of complexity from their code so they can focus on their value add without doing some of the native stuff which we can do better, faster, cheaper, being closest to the hardware.
In Part 2 of this interview series Brent will continue to discuss the primitives that developers will have access to including atomic multi-block writes. We also discuss how familiar the API will be to developers.
In Part 3 in this interview series we will discuss the “DirectFS” API, a native POSIX compliant direct file system layer and discuss the more technical aspects of how the SDK works.
In Part 4 in this series Brent and I discuss the semantics of using the API in the C language and how Fusion-io is leveraging its early access partnerships.