The introduction of first generation software-defined storage solutions (often implemented as appliance and storage controller-based virtualization) went terribly awry when they were originally introduced years ago for reasons that the industry probably only now fully understands and can articulate well. While the value of software-defined storage has never been disputed, best practices associated with its implementation, management and support short and long term took time to develop. We are now seeing the fruits of these efforts as evidenced by some of the successful ways in which software-defined storage solutions are packaged and shipped.
The impact that software-defined storage solutions are poised to have on the traditional storage market is significant. Recent IDC research suggests that traditional stand-alone hybrid systems (mix of disk and flash) are expected to decline at a 13 percent compound annual rate while new system (all-flash, hyperconverged and software-defined) adoption will grow at 22 percent clip from 2014 to 2018.
The exact percentage that software-defined storage solutions will contribute to this overall 22% growth rate is unclear. However it is clear that doubts about their short and long term viability have largely evaporated.
Contributing to this increased confidence in using software-defined storage is the growing number of successful implementations of this technology on appliances and storage controllers. While software-defined storage has had a presence on these devices for well over a decade, the increased availability of software-defined storage solutions from vendors and growing adoption by end-users stems from the ability to better mitigate the issues associated with the use of software-defined storage and improved best practices for its initial implementation and ongoing management that optimize its inherent strengths.
Specific use cases where DCIG is aware of software-defined storage (SDS) solutions being successfully implemented and used on appliance and storage controller-based devices include:
- Non-disruptive (or near non-disruptive) data migrations. This is historically where appliance and storage controller-based SDS solutions have been used successfully for years. By inserting the appliance or storage controller SDS solution into an existing storage network between the server and back end storage, the SDS solution is then used to virtualize the storage volumes on both existing and new storage arrays and then migrate the data from the existing array to a new storage array.
The appeal of using this approach was that the appliance or storage controller could be inserted non-disruptively or nearly non-disruptively (application downtime of only seconds or minutes) into the environment. Data may then be migrated from one storage array to another while the application continues to operate unaware that a data migration is occurring.
The HP 3PAR StoreServ storage arrays with their SDS solution now provide such an option. When migrating from an existing HP 3PAR, EMC VNX or EMC VMAX array to a new HP 3PAR StoreServ array, organizations may deploy the new HP 3PAR StoreServ, virtualize the volumes on the existing storage arrays, non-disruptively migrate the data to the storage on the new HP 3PAR StoreServ array and then cut the application(s) over to the new HP 3PAR StoreServ array with minimal to no application downtime.
- Better managing deployments of utility storage. Many if not most organizations have a growing need for deployments of large amounts of utility storage in their environments. Organizations increasingly have vast amounts of data for which they cannot quantify its value but know that it is sufficiently valuable that they cannot easily or justifiably delete it. In these cases they often want to use storage arrays that are reliable, stable, economical (e. – provide storage capacity at well under $1/GB,) perform moderately well and remain easy to manage and scale.
The storage upon which this data resides needs relatively few bells and whistles. In other words, it typically does not need integration with any VMware APIs, will not host any Oracle databases, does not need any flash nor will it need any special automated storage tiering features. In short, the storage array deployed needs to be cheap and deep.
SDS solutions play nicely in these environments. Whether the SDS software resides on a storage controller (such as on a Dell EqualLogic, EMC Isilon, ExaBlox OneBlox or HP P4000 array) or on an appliance (DataCore SANSymphony, FalconStor FreeStor or IBM SVC), more storage capacity can be quickly and easily added to these environments and then just as easily managed and scaled since many of the interoperability and performance issues that have hindered SDS deployments in the past do not really come into play in these situations.
- Heterogeneous vendor multi-tiered storage environments. One of the big issues with appliance and storage controller-based SDS solutions is that they attempted to do it all by virtualizing every vendors’ storage arrays. But by attempting to do it all, they often failed to deliver on one of the biggest benefit that SDS has to offer – creating a single pane of glass to manage all of the storage capacity and provide a common, standardized set of storage management features. Virtualizing all storage from all vendors made it too complicated to implement all of the features associated with each of the underlying arrays that were virtualized.
IBM with its SAN Volume Controller (SVC) has smartly avoided this pitfall. Rather than trying to virtualize every vendor’s storage arrays and deliver all of their respective capabilities, its primary focus is to virtualize the various IBM storage arrays and deliver their respective capabilities. While organizations arguably sacrifice some choice and flexibility to buy from any storage vendor, many would rather have less choice with a more predictable environment than more choices with more risk. Further, IBM provides organizations with a sufficient number of storage array options (flash, hybrid, disk, etc.) that they get most if not all of the tiers of disk that they will need, the flexibility to manage all of this storage capacity centrally and the ability to present a common set of storage array features to all attached applications.
Software-defined storage may not yet be fully mature but neither is it a half-baked or poorly thought out solution anymore. Vendors have largely figured out how to best implement it so they can take advantage of its strengths while mitigating its risks and have developed best practices to do so. Ultimately, this developing and maturing set of best practices will probably contribute more to SDS’s long term success than any other new features that SDS solutions may offer now or in the future.