A couple of weeks ago I attended the Flash Memory Summit in Santa Clara, CA, where I had the opportunity to talk to a number of providers, fellow analysts and developers in attendance about the topic of flash memory. The focus of many of these conversations was less about what flash means right now as its performance ramifications are already pretty well understood by the enterprise. Rather many are already looking ahead to take further advantage of flash’s particular idiosyncrasies and, in so doing, give us some good insight into what will be hot in flash in the years to come.
Everyone in the storage industry knows that flash is not spinning media. If anything, the fact that everyone is largely forced to treat flash as storage is arguably one of flash’s big drawbacks to date. Exacerbating the issue, storage vendors are still going through a learning curve of their own. When they think of storage, many are still predisposed to approach it and think of it in an “HDD First” mentality. While that is only to be expected after years of designing storage systems for HDDs, a new “Flash First” mentality is now required and different vendors are at various stages internally at making this shift in thinking.
This is what made the Flash Memory Summit particularly intellectually stimulating to attend. A fair number of the vendors at the conference approach and think of flash in this context of “Flash First.” As such, they are pushing the boundaries of flash and while their ideas and products are, in most cases, probably not ready for enterprise adoption or use today, they do provide some good insight into what flash is poised to deliver in just a few years.
1. No Read Penalty. Everyone always talks about the write penalty associated with flash and how their systems manage that. But everyone has been so obsessed with overcoming and dealing with this issue is that they have overlooked a big opportunity with flash: there is no read penalty.
Once data is written to flash, whether you read a piece of data once, a hundred times, a million times or more, there is no penalty (as in there is no penalty to be paid in the wearing out of the flash.) This is a significant divergence from HDDs which are mechanical devices and have fairly defined wear out periods. Whether you are reading from or writing to HDDs, if you get 5 years out of an HDD, that is about it.
Acknowledging and taking advantage of this “No Read Penalty” property of flash is going to force a major rethinking of flash’s role for use as a preferred media in archive. Unlike HDDs, flash has the potential to last for many more years (potentially 10 or more,) provide 5-10x improvements in read performance over HDDs, use less power, produce less heat and consume far less rack space due to its density.
Already I saw a 64 TB (yes, TB) flash memory card being demo’d at the Flash Memory Summit by NxGn Data that fits into a PCI slot. This card is strictly intended for Read intensive environments where one write or maybe at most a few writes occur to the card. After that, the rest of the I/Os are read traffic with the first version of its card able to achieve read I/Os in the 250,000 – 300,000 IOPS range. Using this type of technology. I can already think of a number of Big Data applications that can benefit from it. Using it, organizations can put data that never changes (videos, photos, images, etc.) on these cards to get unparalleled read performance at a fraction of the capital cost of today’s flash solutions.
2. More Application Integration is Coming. Another major challenge with flash is that applications and operating systems have yet to adapt to take advantage of the performance that flash currently offers. Until now, applications and OSes had to be tuned to account for IO queue depths and IO wait times due to slow storage response times.
Those days are over. Now it is the storage that is waiting on the applications and OSes to send it more IOs and more data to process. This sudden availability of storage performance is prompting the redesigns applications and OSes so that they can begin to take advantage of storage performance in ways they never could before.
To do so, integration has to occur between the applications and the OSes and the underlying storage to facilitate faster data transfers and increased throughput. Already we see early examples of this from Oracle between Oracle Database and its ZFS storage solutions and IBM DB2 and its storage systems. Look for much more of this type of integration in the months and years to come and that by 2016, most leading storage arrays will offer some type of integration with leading applications and operating systems.
3. The Demise (or End?) of Storage as We Know It. One major theme that came out of the Flash Memory Summit was the current inefficiency of needing to translate data into a format that storage understands and then back into a format that flash understands. The thought was: why not just create a bus from flash memory directly into the CPU to avoid all of these theoretically unneeded translations of data back and forth between these different formats?
As the situation stands now, the translation of data between the CPU and underlying flash-based storage occurs very quickly today – so quickly that most applications and OSes cannot detect the delay. But as applications and OSes catch up and again get out in front of today’s flash-based storage solutions, look for a new generation of storage to emerge that may not even appear as storage at all but as ever-present memory cache. Forerunners of solutions like this already ship from Fusion-io (now a part of SanDisk) and I would expect more to emerge in the years to come as the benefits of storing data in this format become more apparent.