Tuesday, November 15, 2016

Some Tips about running a Dell PowerEdge Tower Server as your workstation

Some use workstations as servers.
I'm using servers as workstations.

Over the years, I've changed computing gear on quite a few occasions. I've been using Tower Servers for the past 5 years and would like to share some tips to help others.




  • But why would anyone want to do that??


- Servers are well integrated systems and are usually seriously designed and tested.

- They offer greater expandability (6x3.5" hotswap bays in my previous T410, now 8x3.5" in my T430).

- They usually include some kind of Remote Access Card (RAC) which is great for remote'ing in when all else has failed.

- I can get tons of server equipment on ebay that will be compatible with that system.

- Where else can I get 192Gb of ECC DDR4 RDIMM, dual 6-core Xeons and 8 hotswap bays?

  • Tip #1 : Choose your chassis with care.
Not all servers are created equal:

- Rack servers are usually thin and noisy (those 8k rpm fans have the job of cool that 2U enclosure). It is not uncommon for them to be in the 60-70dBA range.

- Tower servers are much bigger and less noisy. The are also more expensive -but- you get an electricity bill that's lower than a comparable Rack server so the price difference will shrink after a few months. And having a server that makes less noise and draws less power is more environment-friendly!

- Most pre-2011 tower servers from Dell and HP (before Dell 11th Gen and before HP's Gen8) are less quiet than their modern counter-parts.

In 2016, I'd recommend getting a 12th or 13th Gen from Dell.. If you are into HP Gear, get a Gen8 or a Gen9. I've never done Lenovo or Cisco gear, so I can't help here.

- Most modern towers from Dell feature a single 120mm PWM fan to cool the entire chassis. That's the T410, T420 or T430. I assume the T310, T320 and T320 are similar since they feature the same chassis.

- The environmental ratings for current and past servers can usually be obtained from the manufacturer. Check the specs carefully. I found the spec for most recent Dell Tower servers here:

Dell-13G PowerEdge Acoustical Performance and Dependencies

  • Tip #2 : Choose your components carefully.

Now that you've selected the system, let's pick the components.

- CPUs

- Most recent tower servers feature PWM (4-pin) fans that are controlled by the iDrac/iLo controller. The sensors on these systems feed the former with information which they use to drive the speed of the fans.

- Consequently, even if you want enough Xeon cores, you probably don't want one of their 145W 12-core monsters. Such a chip (or a pair of them) will increase thermal response under load in your system which will result in increased fan speed. On the other hand, lower Wattage Xeons usually have a low core frequency that might make the user experience in interactive sessions oh-not-so-great.

I usually pick Xeons in the 65W-85W range. These typically feature decent punch while keeping heat (and noise) tolerable.

Wikipedia has a great list of all Xeon processors with Wattage, Cores, etc.. here:
List of Intel Xeon Microprocessors

- Graphics!

The bundled graphics adapter in your server will not let you run much else than a 2D environment. This can be solved by adding a PCI-E GPU which will give you decent 3D performance.

Forget about the latest Radeon or NVidia monster, it's not going to work at all.
When I tried my NVidia Quadro K2000 (a 65W card) in my Dell PowerEdge T130, the system simply refused to boot and told me that the card was drawing too much power to power on all components.

GPUs can usually work fine if they are in the 45W or below range. I've used with great success NVidia Quadro K620 and K600 cards in my Poweredge. The Passive Geforce cards from the previous gens (GT730, etc...) can also be used successfully.

Here's my Poweredge T130:
# lspci |grep VGA
01:00.0 VGA compatible controller: NVIDIA Corporation GF108 [GeForce GT 620] (rev a1)
That card was replaced by a GT 730:
lspci |grep VGA
01:00.0 VGA compatible controller: NVIDIA Corporation GK208 [GeForce GT 730] (rev a1)


And here's the Poweredge T430:
# lspci |grep VGA
03:00.0 VGA compatible controller: NVIDIA Corporation GF108GL [Quadro 600] (rev a1)

- Sound

Servers don't have sound cards.. but I've used with much success USB audio adapters to get sound from Videos and Games on my Linux Servers/Workstations.
These can usually be obtain for about USD10 on amazon or e-bay:


  • Tip #3 : Use the right settings
Dell servers need some parameters passed to the iDrac in order to keep noise to a minimum even when using 3rd Party PCI-E cards.

Disable PCI-E 3rd Party thermal reponse (can also be done from the iDrac submenu of the BIOS GUI):

Here's a 13th Gen server. I hightlighted the most important fields.

/admin1-> racadm get System.ThermalSettings[Key=System.Embedded.1#ThermalSettings.1]
#FanSpeedHighOffsetVal=75
#FanSpeedLowOffsetVal=15
#FanSpeedMaxOffsetVal=100
#FanSpeedMediumOffsetVal=45
FanSpeedOffset=Off

#MFSMaximumLimit=100
#MFSMinimumLimit=5
MinimumFanSpeed=255
ThermalProfile=Minimum Power
ThirdPartyPCIFanResponse=Disabled 


Some of these can be modified by using the iDrac CLI:
/admin1-> racadm set  System.ThermalSettings.FanSpeedOffset Off      
/admin1-> racadm set  System.ThermalSettings.ThirdPartyPCIFanResponse 0
[Key=System.Embedded.1#ThermalSettings.1]
Object value modified successfully

To be continued...

8 comments:

  1. Hi,
    I'm trying to add a graphics card to my T130. I bought the same model as you indicated: MSI NVidia GeForce GT 730. Unfortunately when I try to disable the built-in graphics card, the system does not boot and says "No boot device available".

    What BIOS options have you configured to boot with the onboard graphics card disabled and the new one activated?

    Thank you,
    Miguel.

    ReplyDelete
    Replies
    1. Hi Miguel,
      I don't think I configured anything specifically. Here are my BIOS settings:
      racadm>>get BIOS.IntegratedDevices

      racadm get BIOS.IntegratedDevices
      [Key=BIOS.Setup.1-1#IntegratedDevices]
      #CurrentEmbVideoState=Disabled
      EmbNic1Nic2=DisabledOs
      EmbVideo=Disabled
      InternalUsb=On
      MmioAbove4Gb=Enabled
      OsWatchdogTimer=Enabled
      UsbPorts=AllOn

      I cannot remember if I disabled EmbVideo manually but perhaps that's something you could try.
      Regards,
      Vincent

      Delete
  2. Fixed: I updated the BIOS to version 2.0.8

    ReplyDelete
  3. Life saver! I was pulling out my hair trying to get the fan to return to normal after installing a third party add-on card.

    "racadm set System.ThermalSettings.ThirdPartyPCIFanResponse 0" did the trick!

    ReplyDelete
  4. This blog resolved all my queries I had in my mind. Really helpful and supportive subject matter written in all the points. Hard to find such kind of blogs as descriptive and accountable to your doubts.
    HPE StoreVirtual VSA

    ReplyDelete
  5. Installed a Mellanox ConnectX-3 card and the system fan got so loud I couldn't stand it.

    "racadm set System.ThermalSettings.ThirdPartyPCIFanResponse 0" worked for me too!

    Thank you for restoring my sanity!

    ReplyDelete
    Replies
    1. Hi,
      Glad to hear it helped but please be aware of the following: Mellanox CX-3 are known to run very hot and overheat. Please make sure you used the sensors on the card to monitor its temperature. I recall having CX-3 cards run too hot because they were in a PCI-E slot that was getting a little less airflow.
      When this happens, the card will shut itself down, drop off the PCI-E. The server will then reset and an event will be logged to the iDrac's log.
      Good luck,
      Vincent

      Delete
  6. This comment has been removed by a blog administrator.

    ReplyDelete

LVM2 bootdisk encapsulation on RHEL7/Centos7

Introduction Hi everyone, Life on overcloud nodes was simple back then and everybody loved that single 'root' partition on th...