Supermicro IPMI BIOS upgrade fail [Solved!]
Sunday, August 24. 2014
I tried to upgrade my Supermicro SuperServer 5015A-EHF-D525 IPMI BIOS to have the Heartbleed fixed in it. It failed on me. Badly. When I run:
lUpdate -f SMT_316.bin -i kcs -r y
The not-so-friendly response is:
If the FW update fails,PLEASE TRY AGAIN
update part 0, the size is 0x800000 bytes
Transfer data ................
40K bytes 1%ERROR !! BMC did not in correct state
ERROR:SEND "ReceiveFWData" COMMAND TO BMC FAILED
It looks like Supermicro's Linux upgrade tool is the culprit. It enters BMC upgrade mode, starts pushing bits to FlashROM, and then segfaults. I tried couple of BIOS-versions, but to make things worse, I was going from version 2.x to 3.x and there was no downgrade possibility anymore. The BMC was semi-concious, but it really couldn't do much. For example, it didn't have a proper MAC-address, and its networking was effectively out of play.
The worst part of this failing upgrade is, that to get the BMC upgrade mode disabled, you need to pull the plug. If there is electricity connected to the machine, the BMC will stay on.
Update:
Also the ipmitool bmc reset cold helps.
Luckily, there are couple of options for accessing the BMC directly from OS-side. One of them is IPMItool, but it didn't yield any results. The BMC was stuck somehow. Same story with manufacturer's IPMICFG.
I was almost going to give on on this and was planning to RMA it to Supermicro support in Netherlands. Then a newer version of IPMI BIOS was released and I attempted to upgrade into it. Same story, Linux utility crashes badly causing havoc. As the last move, I USB-booted the hardware into DOS-mode. There is a flash-utility in their ZIP-file for DOS. IT WORKED!! How lucky was that!
The new BIOS-version was sane, it knew its own MAC-address and started operating properly. I was so happy!
Who would have thought, that DOS was abandoned almost 20 years ago, and it once more saves the day.