What is most valuable?
Federated Catalyst Replication allows us to centrally control and store all backup data with minimal effort and administration, and also removes relying on onsite staff at remote locations to change tapes.
How has it helped my organization?
Not every office has skilled IT staff, so changing backup tapes was always a challenge. A person was typically “burdened” with the task of changing tapes daily or weekly and would often be neglected for other priorities or be completely forgotten if the staff member was away or on leave. Now, there is no requirement for any staff member at remote sites to be involved in any task related to data backup and recovery.
We averaged at least three missed backups a month due to issues with the right backup tapes not being in the drive at the time of a backup. This has been completely eliminated since moving to StoreOnce and using Catalyst Stores and Catalyst Replications since all data moves are automated and bandwidth friendly, so they complete well within our nightly backup windows, and tapes are only used at the Datacentre which is controlled by trained IT professionals. We now have no one to blame but ourselves
What needs improvement?
- Other products are able to read and write directly to/from StoreOnce Catalyst Stores (Symantec NetBackup and Backup Exec, Microsoft SQL Server, SAP, Oracle and soon Veeam Backup & Replication). However, they cannot integrate with StoreOnce Catalyst Replication. The replication must be triggered by software such as HP Data Protector, which must first know about the data that exists on the Catalyst Store. Unfortunately, data written to a Catalyst Store by one of the other products is not visible to Data Protector, so automating actions to occur based on new data being written to a Catalyst Store is not available. This really limits the ability to integrate all backups in to the Catalyst Store using their own native tools (MSSQL Maintenance Plans for example) and leverage the Federated Catalyst Replication capabilities. All backups must be written by Data Protector use Federated Catalyst.
- The performance of NAS/CIFS Shares on a StoreOnce is acceptable for a standard write operation, but performance during restores (such as Virtual Machine Restores from Veeam Backup and Replication) is unacceptable. This is being improved as more products natively support StoreOnce Catalyst but is not widely adopted as yet.
For how long have I used the solution?
We have been using StoreOnce for approximately two years, and couldn't be happier. It's used across two main datacentres and over 15 remote sites that use a combination of the 4430, 4220, and 2620 systems. All StoreOnce systems run the same OS Version, currently 3.12.1. We do not use the VSA, as we strive to maintain physical hardware separation between our production data and disk backups. We backup data at all remote sites to a local StoreOnce device, then replicate it to Datacentre A, then again to Datacentre B, and then finally copy to tape for secure storage.
Also in-use, is HP Data Protector 9.03 which controls backup of data into StoreOnce Catalyst Stores and automated object copy operations which utilise the HP StoreOnce Federated Catalyst to replicate backed up objects between StoreOnce Backup Systems. All data backups at remote sites follows the same routine:
- Single Data Protector Backup Specification for all clients in the site.
- Data Protector Disk Agents back up data to a single HP StoreOnce Gateway (we use one of the same disk agents being backed up to also serve as the gateway).
- Data is written to a StoreOnce Catalyst Store at the local site.
- Data Protector Post-Backup Object Copy Job with the “Use Replication” checkbox enabled is triggered automatically upon completion of the backup to replicate data to Datacentre A.
- The StoreOnce Gateways at both sites trigger a Catalyst Replication where the StoreOnce's replicate the Data Protector Backup Objects from one StoreOnce to the other utilising source-side deduplication has dramatically reduce WAN usage (only blocks that do not exist are transferred).
- Once completed, another Post-Backup Object Copy Job copies the data from the StoreOnce at Datacentre A to a similar StoreOnce at Datacentre B.
- A final Post-Backup Object Copy Job then copies the data from Datacentre B on to LTO6 Tape.
Using this process results in many restore options, and the entire process is automated. We can restore from:
- StoreOnce at the remote site
- StoreOnce at Datacentre A
- StoreOnce at Datacentre B
- LTO6 Tape at Datacentre B
What was my experience with deployment of the solution?
No real issues other than the change in mindset for our backup administrators and the increase in the number of jobs that are now configured. For a typical remote site the following seven jobs exist:
- Daily Differential Backup (Monday to Thursday) to local StoreOnce
- Daily Post-Backup Object Copy to Datacentre A
- Daily Post-Backup Object Copy from Datacentre A to Datacentre B
- Weekly Full Backup (Friday) to local StoreOnce
- Weekly Post-Backup Object Copy to Datacentre A
- Weekly Post-Backup Object Copy from Datacentre A to Datacentre B
- Weekly Post-Backup Object Copy from Datacentre B to LTO Tape
There is also now the requirement to upgrade HP Data Protector to provide certified support for newer StoreOnce OS releases. HP Data Protector upgrades typically bring with them enough problems on their own.
What do I think about the stability of the solution?
We did experience an issue for many months where Post-Backup Object Copy jobs would remain stuck “In Progress” and never report any data movements even though the two StoreOnce devices successfully replicated data. The jobs had to be aborted and manually re-run. We experienced approximately five of these a week out of over 200 jobs that would run. This was eventually fixed in Data Protector 9.03, and a post-hotfix.
How are customer service and technical support?
The HP StoreOnce support team are very responsive and quick to react to problems and offer ongoing support and guidance. However, StoreOnce really shines when integrated with a backup product that utilises the StoreOnce capabilities to better protect your data. This is where the HP Technical Support breaks down. It is often difficult to have a team take ownership of a problem or admit there is a fault, the blame is often directed to someone else. Typically, we have found this to be more of an issue with the Data Protector Support than with StoreOnce, but being the same company it is quite jarring at times how little cross-communication and teamwork there is despite Data Protector and StoreOnce being so heavily dependent on each other.
Which solution did I use previously and why did I switch?
We previously used Symantec Backup Exec at remote sites to back up data locally to tape, and HP Data Protector at the Datacentres. Managing two separate products was difficult, particularly with staff trying to remain current on both products that have very different lifecycles. Since we already used Data Protector at the datacentres and HP StoreOnce is the most tightly integrated with Data Protector the choice to use StoreOnce was very easy. We did not evaluate any other disk backup products.
How was the initial setup?
The initial setup was complex, at least compared our previous backup solution of each site being their own installation of Symantec Backup Exec, and only a couple of small jobs to maintain. We have a much larger number of backup jobs to maintain now, and must be very careful when running jobs during business hours as most jobs now have other jobs that chain off them, so you run one job and it could trigger many others.
However, all configuration is done from a central location, all replications are triggered automatically, and we have full tracking of where every piece of backed up data is and where all of its copies also reside. A sore head upfront to have automated backups of every location and centrally replicated and multiple restore locations is well worth it.
What about the implementation team?
We leverage the HP Level Two StoreOnce Startup Service which gave us access to the HP StoreOnce engineers for a few days. We were able to workshop our requirements and vision and come up with a solution that would technically achieve what we wanted. Unfortunately, HP Federated Catalyst depends heavily on HP Data Protector and there doesn’t seem to be much cross-skill of engineers between Data Protector and StoreOnce.
Thankfully we have over eight years of in-house technical experience with Data Protector so we were able to leverage what we learnt during the StoreOnce Startup Service to expand upon the solution and deliver above and beyond what we thought was possible. To make use of Federated Catalyst I strongly recommend a deep understanding of HP Data Protector (it’s not as scary as it first appears to be).
What's my experience with pricing, setup cost, and licensing?
It is not abundantly clear, but the HP Catalyst License also includes the Catalyst Replication capability, so no other license is required if Catalyst Stores are the only backup target being used. StoreOnce devices cannot replicate Catalyst Stores automatically for you, the replication must be triggered by other software such as HP Data Protector.
Depending on the backup product being used with HP StoreOnce, there may be disk backup capacity licenses also required. HP Data Protector requires Advanced Disk Backup Licenses for the usable capacity of the StoreOnce Systems (this is the native usable capacity, regardless of how much data you can actually store thanks to deduplication benefits).
You can use a StoreOnce as a NAS/CIFS Target and also as a Virtual Tape Library. These backup targets can be replicated to other StoreOnce Backup Systems, however these require additional licenses.
What other advice do I have?
We do not have much experience with other disk backup solutions. The Federated Catalyst Replication and the deduplication ratios obtained are amazing. As an example, we only keep two weeks of daily backups and six weeks of weekly backups on disk, and have over 470 backups from around the country stored at Datacentre A which equates to over 140TB of data that consumes less than 13TB on disk. The dedupe ratio is 10.7:1. This ratio grows even larger as retention periods are increased. We are still fine-tuning our retention periods and will likely be doubling them within the next six months.
The StoreOnce VSA is a great place to start as it is 100% functionally the same as a StoreOnce physical device, and you can use a free trial. Definitely the best place to start. The same goes for HP Data Protector.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
HP StoreOnce has very limited paralel streams (upto 128). This situation prevents enterprise customers with lots of database servers can't backup on StoreOnce at the same time due to the low number of paralel streams. For example EMC DD9800 has 1024 paralel streams..
Really needs to be worked on...