Entries categorized under “Deduplication”

When you really look into the data that your organization is archiving, there is usually a good reason for the business to retain that data, otherwise why would it be archived at all? Placing that data on a system designed and built from the ground up to be an archival repository is probably a really good idea. (read more)

My visit to this fall's Storage Decisions conference in New York City on Wednesday, September 24, was an abbreviated stay. I only had the afternoon to spend at the conference before leaving in the evening for another set of meetings the next day. So while my time was short, I did catch a couple of briefings as well as a little industry chatter. Some of the talk on the exhibit hall floor had to do with the current crisis facing the banking industry and what that may mean for technology as a whole. One of the sentiments expressed which I generally agree with is that the financial crisis is probably not good news for the larger storage vendors at the show but likely bodes well for emerging storage technologies in the market as it will force some companies to look beyond traditional solutions.. (read more)

For most end-users the promise of a truly open archival system has been mostly out of reach. The proprietary approaches taken by storage vendors in the deployment of archival systems inside the hardware and application stacks has left users without a viable, standards-based archiving option. (read more)

A self-evolving platform is one of the promises behind products like the NEC HYDRAstor that are based on grid storage architectures. Grid storage architectures automatically take over data migrations during technology refreshes which eliminates the need for application downtime or for companies to do forklift upgrades. Yet up to this point it was difficult to establish the validity of that promise for the NEC HYDRAstor since its HS8-1000 series was still in its first release. (read more)

Replication and deduplication are features that are fast becoming necessities when disk libraries are introduced into enterprise IT backup environments. But as I brought out in a previous blog entry, introducing multiple functions into disk libraries intended for enterprise caliber backup environments typically has some unpleasant trade-offs. A primary concern in enterprise IT shops is how large (or small) to initially configure the solution so companies neither overspend on oversized hardware nor purchase undersized hardware that cannot scale to meet their future requirements, so they need some way to forecast how their IT environment is going to look going forward. (read more)

As a former end-user, I know companies internally spend a lot of time talking about the importance of cutting costs. But at the end of the day, often their bigger concern is to mitigate risk in their environment. So no matter how much money a new technology saves, companies do not want to adversely impact their existing production environment. However, standing still is not an option either. The question for companies then becomes which technologies are worth taking a chance on and innovating versus preserving the status quo. While everyone's criteria for making that decision and tolerance for risk are different, a major question they always ask is, "What happens if it does not work in my environment?" (read more)

The enterprise data center continues to evolve, driven by ever-growing amounts of data and new demands for data availability - local and remote. These demands are driving companies to identify alternatives to existing data protection methods with deduplicating disk-based storage systems, such as Quantum's DXi Series, becoming a preferred backup target. However deduplicating data is only half the equation. To fully deliver on enterprise data protection, companies need efficient, cost-effective options so they can move this deduplicated data off-site for long term compliance and disaster recovery, or centralize and consolidate data from remote offices. (read more)

However my intent is not to leave readers hanging or fretting as to what storage systems they can select that take this problem into account. The NEC HYDRAstor is one such product that has taken steps to address this issue. HYDRAstor includes a feature called Distributed Resilient Data™ (DRD) that is able to offer more protection than RAID 5 or RAID 6 without their rebuild performance drawbacks. Because HYDRAstor is based on a grid storage architecture, it can by default survive the failure of not only multiple disk drives but also multiple Storage Nodes. The default setting is 3 disk drives or 3 Storage Nodes if multiple nodes are present (based on the video on the HYDRAstor web site, it looks like a company needs at least 12 nodes for a company to have assurance it can recover from the failure of 3 different nodes). (read more)

Almost any disk-based solution - deduplicating or otherwise - is going to expedite backups and recoveries. Sure, some solutions may deduplicate better or do it faster but at the end of the day most companies are at the point that putting in place any disk-based system that supports replication and deduplication is better than dealing with the current backup pain. However what companies often fail to account for is how fast their backup data stores grow when they start backing up data to disk. More than once I've talked to system administrators in companies where "undisclosed" or "hidden" departmental application servers start to come out of the woodwork once department managers hear that corporate IT backup processes actually work. (read more)

The juxtaposition of deduplication and replication in disk-based backup appliances is a powerful combination that companies can use to protect backed up data across data centers as well as data backed up at remote and branch offices (ROBOs). Yet where deduplication ends and replication starts can get a little confusing in grid storage architectures such as is supported by the NEC HYDRAstor that features global deduplication capabilities. (read more)

Deduplication is currently one of the hottest topics in data protection but it takes more than one form. The CommVault® Simpana® software suite implements deduplication as a Single Instance Store (SIS). In this iteration, SIS deduplicates archived and backed up files at the file level and then only stores one occurrence of the file. In part 2 of this interview series with CommVault Systems' Senior Director of Information Access and Management, Simon Taylor, elaborates on how Simpana leverages SIS for information search and data mobility as well as how this approach complements block-based deduplication approaches found on certain disk-based storage solutions. (read more)

One would think that at some point organizations would reach the tipping point for storage consumption and that year-over-year storage capacity growth rates of 30%, 50%, 100% or more would come to an end, or at least slow down. If so, it hasn't occurred yet and, if anything, it shows every sign of continuing for the foreseeable future. Nowhere is this more evident than with the amount of data that companies need to archive and retain. (read more)

Grid computing is starting to appear in some unlikely places. It is easy to assume that grid computing appears primarily in the world of academia or high tech corporate IT engineering labs. In these environments, computer scientists typically have the time and expertise to engineer complicated, high performance, low cost computing solutions that can perform tasks like mapping out the human DNA or identifying possible new sites to drill for oil. But applying grid computing to address a low-tech problem like backup and recovery? That almost seems like a misnomer. (read more)

At recent storage conferences (Storage Decisions, Storage Networking World, etc.) replication has emerged as a hot topic of discussion among end-users. In talking with these different users and listening in on a number of end-user panel discussions, there are a number of factors that they attribute to their increased interest in using replication as part of their company's overall disk-based data protection strategy. (read more)

The Computerworld column I wrote a few weeks ago on the topic of "A Bit of a Flaw with SATA disk drives" sparked quite a bit of debate around just how safe is data on today's RAID-based storage systems that use SATA disk drives? A series of comments appeared on Computerworld's site where the column appeared as well as on a forum at Nabble's web site. Also, at least one storage system vendor felt obligated to send me their white paper that explains how its RAID-based storage system accounts for this bit error rate problem on SATA disk drives. (read more)

As the deduplication debates rage on, it is easy to forget that the debate occurs on many fronts. The debate that tends to receive the most attention is on which method of deduplication is better, "inline" or "post-processing", as it pertains to its use by disk-based backup appliances. However in the "Which is the best deduplication?" debate, an angle that tends to get overlooked is the one between file-based, such as is used by the CommVault® Simpana® software suite, and block-based deduplication. (read more)

Backup to disk is now seen as "The" solution for any company looking to solve its backup problems. Factor in deduplication as part of the disk-based backup solution and it is easy for companies to believe that they are well on their way to solving their backup problems. To a certain degree, that's true. Introducing disk almost always solves the immediate corporate pain of failed backups while shortening their backup windows. In fact, I am only aware of a few, isolated instances where that is not the case. (read more)

Have you ever looked at a technology solution that left you wondering why a specific technology matters? I often look at a technology solution and take note of its many features but then may walk away thinking it was a slick presentation but wonder why I should care about it? So when I evaluate a technology solution, I first look to see if it solves a real problem. This can include making me more productive, addressing a key operational pain point or in some way reducing operational costs. (read more)

It's 2008 and one would think that disk-based storage systems are beyond the point of catastrophic outages and/or data loss as a result of disk drive failures. The prevalent use of RAID in storage systems for disk drive protection in its many forms would seem like ample insurance against the loss of data. However a careful examination of the facts exposes the flaws in assuming that RAID alone is sufficient as a means of data protection; especially when used in conjunction with today's high capacity SATA disk drives. (read more)

Now that this agreement between EMC and Quantum is out in the open, the real question becomes what does Quantum hope to gain from this relationship? On the surface, it appears this agreement puts EMC and Quantum at loggerheads in the rapidly growing space of deduplication. While EMC and Quantum will use different hardware, the software that drives their respective disk systems will be based on Quantum's technology. In this respect, much of the functionality found in the software will be the same, including the policy-based deduplication I detailed in an earlier blog, though Quantum is putting more emphasis on features such as direct tape creation given its continued focus on integration of disk and tape resources within the enterprise. (read more)

However as companies move towards archiving data on disk-based storage systems, you can't just always build bigger buildings or knock down walls. If anything, companies want to store more data in a smaller footprint. Making it more complicated, companies are creating exponentially more data than they were 10, 5 and even 2 years ago and keeping it for longer periods of time. Factor in mobile devices that manipulate existing data and create new data and the increasing use of video in corporations and the result is millions, billions and even trillions of file-based data elements that create thousands of terabytes of data. (read more)

One of the more agonizing choices that some companies face when looking to implement the same deduplication scheme across the enterprise is quantifying which version of deduplication to use: inline or post-processing. From a purist's viewpoint, inline (deduplicating data as it is ingested) is sometimes viewed as the best approach since data is deduplicated immediately as it is ingested. (read more)

Quantum is aiming for the enterprise with its deduplication technology and looks to make a serious run at the enterprise datacenter with its DXi7500. Designed to anchor Quantum's deduplication strategy, companies can use the scalable DXi7500 when it is receiving replicated data from Quantum's DXi3500 or DXi5500 appliances in remote offices; replicating to disaster recovery site(s); or deduplicating terabytes of data during nightly backup jobs in the datacenter. To accomplish this, Quantum designed the DXi7500 to become the focal point for its DXi portfolio. (read more)

In the face of these fundamental shifts among corporate data centers in server data protection and virtualization, data protection software needs to do more than just adapt. It needs to embrace backup-to-disk and server virtualization in order to transform data protection software into an information recovery platform. That is exactly what today's 8.0 release of Asigra Televaulting brings to the table in the following ways: (read more)

One can hardly have a conversation about storage management these days without the topic of archiving surfacing. Part of the reason that archiving is commanding more attention is because as companies create and keep ever greater amounts of referential data on their production storage systems, it is creating a host of new problems (read more)