Temperature above threshold, cpu clock throttled

[golden3]

Hi All,

We're having this weird issue that I'm hoping someone from the CentOS team can answer:

We have a couple Dell PowerEdge R720 , which have theIntel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz in them. Works great. Only thing is, whenever the server comes under load, we get these messages spewed to the console (and every ssh based terminal to all users logged in):

Message from syslogd@ at Tue Mar 15 16:25:27 2011 ...
server kernel: CPU26: Temperature/speed normal

Message from syslogd@ at Tue Mar 15 16:25:27 2011 ...
server kernel: CPU20: Temperature/speed normal

Message from syslogd@ at Tue Mar 15 16:25:27 2011 ...
server kernel: CPU24: Temperature / Speed normal

Message from syslogd@ at Tue Mar 15 16:25:27 2011 ...
server kernel: CPU30: Temperature/speed normal

Message from syslogd@ at Tue Mar 15 16:25:27 2011 ...
server kernel: CPU14: Temperature/speed normal

Message from syslogd@ at Tue Mar 15 16:25:27 2011 ...
server kernel: CPU0: Temperature above threshold, cpu clock throttled

Message from syslogd@ at Tue Mar 15 16:25:27 2011 ...
server kernel: CPU24: Temperature above threshold, cpu clock throttled

Message from syslogd@ at Tue Mar 15 16:25:27 2011 ...
server kernel: CPU22: Temperature/speed normal

Message from syslogd@ at Tue Mar 15 16:25:27 2011 ...
server kernel: CPU6: Temperature/speed normal

Message from syslogd@ at Tue Mar 15 16:25:27 2011 ...
server kernel: CPU12: Temperature above threshold, cpu clock throttled

after i was checked the --> /var/log/mcelog

it showing the below error .

HARDWARE ERROR. This is *NOT* a software problem !
Please contact your hardware vendor
CPU 24 THERMAL EVENT TSC 835d6e58f1b5 [at 2900 Mhz 0 days 13:50:3 uptime (unreliable)]
processor 39 below trip temperature. Throttling disabled
STATUS 880770a82 MCGSTATUS 0
MCE 29

I suspect only these servers have this particular chipset.

kindly confirm with this error , we need to proceed accordingly..

Thanks & Regards
Golden John S

[TrevorH]

You have a hardware problem.

[golden3]

Could any one please suggest a way to disable these prints in user terminal console , while in parallel we will find a solution for this ,,
if there any update need to be taken care in /etc/sysct.conf file , So that the messages can only appear on a root console and not on user console .

[TrevorH]

Probably need to edit /etc/rsyslog.conf adn redirect all emergency messages to a different location. On my el7 system I have a commented out line for

Code: Select all

#kern.*                                                 /dev/console

so it's probably not that one unless it defaults to that. More likely to be as a result of

Code: Select all

# Everybody gets emergency messages
*.emerg                                                 :omusrmsg:*

[avij]

I'm also of the opinion that you're having hardware problems and you should contact Dell to get this fixed.

As a workaround, you can use the cpupower command to set the max CPU frequency to something that the system can handle. See the man pages for details.

What kind of CPU temperatures are you seeing? The omreport command from Dell OpenManage used to be useful for this, but sadly it isn't yet available for RHEL7 / CentOS7. If you have an iDRAC card, you can see the temperatures from there. On my R320 the System Board Inlet Temp is currently 24 °C and CPU1 Temp is 54 °C (75.2 °F and 129.2 °F respectively for those using odd units of measure). I'd imagine these temperatures can also be seen by launching the integrated Dell diagnostics at server startup, but I can't check that now.

[golden3]

This Problem has been fixed , Due to Dell hardware issue only , we just did the firmware update in the servers.