My tests are with HP Proliant DL380 servers generations G6 and G7. I'll bet this will also happen with the DL385, DL580, DL180, DL185, and who knows - maybe all of them. Maybe your server is a different version and if so and you found data to add, please send me an email and I'll update the tables..

If you are from the Ukraine, click here for a translation of this page .. though I suspect if you are from the Ukraine, you found that link first!

якщо ви хочете прочитати цю статтю в Україні, будь ласка, натисніть тут
 

Server Symptoms

In the HP Proliant DL380 servers of various generations (G5, G6, G7 I've tested, it could apply to others) with the built-in P410i or an external P400 RAID card, some hard drives will make the system operate strangely. HP branded server drives, no problem. If you plug in a non-HP drive, like your typical notebook SATA drive, it will likely work for the most part just fine - it will recognize, read, format, write, ... and if you aren't near the system you might think nothing is wrong.

If you are sitting next to the system in a reasonably quiet room, some drives will cause the following to happen:

• All the fans will ramp themselves up to a very fast speed - as in "Jet engine ready for take-off" speed.
• The system will think the drives are operating at a very high temperature
• The system might shut down and reboot to protect itself from the high temperature

Some visual snapshots of the symptoms are shown here.

ILO showing the fan speeds (smaller pictures are thumbnails, click to see a full size version):

+

If you look at the temperature tab of the ILO, temperature sensor 29 might appear near overheating (55-58C) or as shown here that sensor could show over the caution level and claim it is failed:

+

The drives themselves show they are overheated:

The top 3 are real HP drives in the left bay, the bottom 4 are non-HP drives that appear to be overheating. When you pull the drive and feel it, the drive is a little warm but nowhere near 61C and failing.

The mystery of Temperature Sensor 29

HP has a lot of documentation on the temperature sensors in various flavors of the DL380 servers. None of the documents show a physical temperature 29 sensor and its location. From various bits of data and a quantum leap of a guestimate, I believe the value shown in the slot for Temperature Sensor 29 represents the maximum value reported by the drives themselves via SMART or other parameter. If the drive does not report the parameter in a standard enough way, then the HP doesn't really know what temperature the drive is at, the rule of  'Garbage In, Garbage Out' applies, and unknown things will happen

The lowest levels of the HP system seem to read the parameter and make the fans spin abnormally fast in an attempt to cool the system. I belive this is true - the evidence is listening while BIOS is doing its tests before the OS loads. If this is going to be fixed (not likely) it means someone will have to reverse engineer the firmware and find where the mystery parameters are read. Getting HP to do this on systems that are either end-of-life or near it isn't going to happen.

So the best we can do is document what drives cause this behavior, which don't, and choose drives that are HP DL380 friendly.

For lack of a better term, I'm going to call these symptoms "Thermal Runaway" even though the temperature might be perfectly fine.

Drives that operate normally:

Drives that cause Thermal Runaway:

Real-world power impact

I took a reading on a server before swapping out 4 drives that caused the fans to run fast VS. after - I was trying to answer "How much extra power was being drawn by the system due to the fans attempting to keep the system cool?"

Before (screen snaps are thumbnails - click for a larger version):

All 6 fans were running at 90%, shown in the screen shot earlier in this article. Single CPU, single power supply.

After:

And the fans? See for yourself:

So with a little math, it turns out that due to thermal runaway 75 watts of power were expended trying to keep a cool system cool.

Not cool.

<sorry - couldn't resist that one.>

Final Words...

If you have an update - a third party drive you know works or you know causes thermal runaway, please send me information on it and I'll add it to the above table.

If you found this helpful or not, please send me a brief email -- one line will more than do. Or more! I love hearing tidbits from users I've helped. Maybe share a line of what you searched for or how you found this article.

I can be reached at:

das (at-sign) dascomputerconsultants (dot) com

Enjoy!
David Soussan

Copyright (C) 2016, 2017, 2018, 2019 DAS Computer Consultants, LTD.  All rights reserved.