Fusion-io [EOL] Reviews

Originally posted at http://www.v-strange.de/index....

Recently I got the chance to gather some experience with Fusion-IO cards. One of our customers decided to add AutoTiering support to their DataCore environment and as he has HP hardware in place, he decided to buy some I/O Accelerator cards from HP. These cards are nothing else than Fusion-IO cards labeld with a HP sign.
So he bought four 1.2TB ioDriveDuo Gen1, two for each server. Mirroring one card to the other via DataCOre's MirrorDisk feature adds some pretty level of hardware redundancy.

So we installed the two cards in each server and booted them up. The cards were detected as "mass storage controller" with a yellow warning sign as we haven't installed the appropriate drivers from Fusion-IO. This is where the real fun starts. With Fusion-IO cards it is essential to have the proper firmware/driver combination in place. If you don't you won't get them online. Searching on the HP support site you will get two downloads, one is called VSL and one ioSphere (the names of both packages change quite often so in older versions VSL was called 

HP_IO_Accelerator_SNMPDefault_xxxx, ioSphere was called IO_Manager, so it's a bit tricky to find your way). VSL basically is the driver you load within the OS whereas ioSphere is a WebGUI that gives you the ability to configure the ioDrives. So we grabbed the latest version, installed it, rebooted the server and the cards were recognized as HP Accelerators in Windows device manager. The problem is, the drives were not seen by the logical disk manager. So we started the WebGUI and saw that the drives were in "minimal mode" which basically means you can't do anything with them until you match the firmware of the drives to the driver you use. We found that the firmware on the drives was really old so we flashed them to a later version. Not to the latest, this is not supported. Fusion-IO gives you an upgrade guide from which version to which version you can go. So in our case we had to flash twice until we got the latest firmware on our drives. By the way, upgrading from a old v2 to one of the first v3 firmware took nearly 3h for two ioDrive duo.....

Next funny thing is to find the appropriate firmware. In older releases, the firmware was bundled with the ioSphere installer. Then Fusion-IO decided to move it from the ioSphere bundle to the VSL bundle. The problem now is, the ioSphere doesn't find the firmware and doesn't give you an option to define the firmware location. You can try putting the firmware in the ioSphere installation directory under \Firmware but that didn't work all the time. Easiest way was to use the CLI tools to update the firmware as you can provide the full path to the firmware file within the command line.
After a very long time we finally matched the firmware to the latest VSL and we could configure them in ioSphere. A low level format is mandatory after firmware upgrades so we decided to check DataCore's TBs which sector size to chose. DataCore recommends 512bytes so we used it. Formatting was quite fast, 1.2TB in less than 2minutes...

After the low level format we were able to "attach" the ioDrive to Windows which finally makes them visible to the Windows LVM.

Initializing the drives within DataCore was quite fast, ~500MB/s.

One really important thing to remember when using Fusion-IO cards: Fusion-IO needs to grab memory from the OS depending on the sector size you chose during low level format. So if you use 512bytes as recommended by DataCore you have to reserve ~20MB of RAM for each GB of storage on your ioDrive. Now take your calculator and type in: 20MB of RAM + 1200GB formatted capacity = 24GB of RAM. The problem is, we have two of these drives so just to get the two IO Accelerators to run you have to reserve ~48GB of RAM. This is a HUGE amount of RAM you simply can't use as DataCore cache just to be sure your Fusion-IO cards won't deplete all your ressources.... I'm not sure if this is related to all flash drives from all vendors (sTec for example doesn't need any additional RAM) but this is really a problem. Okay, the 48GB is worst case scenario but I have to be sure my DataCore node won't crash if this scenario happens so I simply have to reserve this amount of memory.

Another thing we learned is that some of the ioDrives (and especially the big ones) will draw a whole lot of power from your PCIe subsystem. Normally this is rated at 25 watts per slot but these cards need up to 55 watts. If the card doesn't get the power it will still work but with less performance. You won't spent thousands of dollars for those drives to accept a lower performance just becasue these drives don't get enough power. So remember to buy these special power cords that draw additional power from the power supply instead of the PCIe slots. This way your system will be much more stable.

Another option is to allow the drive to override the 25 watt limit on the PCIe bus but your hardware manufacturer has to support that. In the worst case you will grill your mainboard.

Here is the bottom line for all Fusion-IO cards in DataCore servers:

• Fusion-IO is DataCore ready certified so you can be pretty sure these cards are fast as hell and work perfectly with SANsymphony-V
• It's a bit tricky to get your cards and drivers up-to-date especially if you have old cards like we had
• Don't rely on the names for the drivers/firmware/tools as they change randomly
• Size your RAM according to the requirements by the Fusion-IO cards and your SSY-V cache !!!!!!
• Do not use the ioDrive Duos without additional power
• Sometimes the WebGUI has problems (e.g. low-level-format), then use the CLI tools