MSA1500cs periodically grinding to a halt with VMWare ESX 3.5

Last night for the third time since going live with VMWare in Sept. 2007 we had an issue where the MSA1500cs SAN that acts as the datastore for our 3 host ESX cluster ground to a halt. As the MSA1500 has no ability to log what it is doing this is a tricky one to figure out.

The first time it happened, I though it was because I had set the multipathing on the ESX hosts to fixed. Apparently this is a not the suggested configuration for the MSA1500 as even though it has the v7.00 Active/Active firmware on it, ESX doesn't see it as a true Active/Active setup. So I changed the path setup to Most Recently Used (MRU) and left it to run.

The second time it happened was shortly after upgrading to v3.5 which caused High Availability (HA) on one of the hosts to go nuts and fill up the log partition.

Last night it happened again but nothing had been changed on the setup for weeks. Basically, the VMs grind to a halt and then become unavailable and then some or all of the LUNs presented to the hosts disappear. The only way I've found to fix it is to shut the ESX hosts down (which takes about 30 minutes each as it is trying to gracefully flush the cache to the LUNs but they aren't there) and then power down the MSA1500 controller shelf and bring everything backup again in the correct order.

I opened a ticket with HP about it today but they couldn't see anything from the CLI show tech_info dump I sent them (which I guess they wouldn't as the thing had been power cycled!) and suggested that it could be the drivers or firmware in the BL480c servers that we are using or a firmware upgrade may be needed in the Brocade fiber channel switches.

As I had the time last night, I also applied the last batch of v3.5 patches.

So far it has happened after about 50-60 days of uptime so I think for now we will be proactive and do a manual shut down of everything on a weekend every 6 weeks or so.

Bow equipment for me

Here is the setup that I am looking at for my bow. The limbs and the sight are subject to change as I'm not sure what is going to be better to spend the money on. I want to get something that Adan can move up to once he outgrows the T-Rex. After that I may move to a compound bow.

KAP Winstar II 25" Black Riser
KAP 70" 30# Winstorm Craft Limbs
Sure-Loc Contender-X Target Sight or Shibuya Dual Click Standard
2 Dynaflight '97 70" 14 Strand bowstrings (one black, one white)
Spigarelli Bow Stringer
Aurora Techno Jazz Quiver
Aurora Dynamic Take-Down Recurve case
Win & Win 360 Perfect Finger Tab
Arco Sport Spigarelli Z/T (Zero Tolerance) Rest for W&W - Now that I look into things a bit more I'm seeing that there have been some people having problems with this rest on UK Archery Interchange
Cartel Midas Cushion Pressure Button

KAP T-Rex has arrived

Adan's bow and equipment arrived last week from Alternative Sports in the UK and we had George from the Victoria Bowmen club set it up for us. George had to make a few last minute substitutions for us as the arrow rest and cushion plunger I ordered weren't suitable for the T-Rex. This was my fault for not running it by George first but they should work fine with the KAP Winstar II riser that I want to get for me.

Adan used the bow for the first time on Saturday and seemed pleased with it. The weight of the bow and pull weight of the limbs is a bit heavier than the wooden recurves we had been using during the lessons and this is causing him to be a bit shakier. I'm sure after a few weeks of use he will be fine.