Rack Build Specification

This activity puts into practice the concepts from the On-Premises Infrastructure and Data Center Architecture lecture. Where the Hardware Build Specification activity stopped at one machine, this one scales the same thinking up to an entire rack. By the end you will have a concise rack specification document that names real vendor stock keeping units (SKUs), meaning concrete sellable configurations rather than generic server categories, totals worst-case watts, identifies the highest Uptime Tier the rack can participate in, and would survive a basic sanity check from a data center operator.

What You Will Need

• A blank document in your preferred editor, note-taking app, or word processor
• Access to vendor configurators in a browser: Dell PowerEdge, HPE, Supermicro
• A calculator or spreadsheet for capacity, bandwidth, and power math

Before you begin, create a document with these headings: Scenario, Servers, Storage, Networking, Management, Power, Cooling, and Final Sanity Check. Leave room under each heading for short calculations, vendor model numbers, and one-sentence justifications.

Pick Your Scenario

Read all three scenarios before choosing. Each one stresses a different design constraint: VM density, storage durability, or AI bandwidth and power density. Picking deliberately matters because every later section depends on that choice.

A. Virtualization Rack. A mid-sized company is consolidating 200 modest internal-app virtual machines (typical VM: 4 vCPU, 8 GB RAM, 100 GB disk) into one rack. The workload is general-purpose: web apps, internal tools, a couple of dev environments. Optimize for VM density per U and per watt. The rack will live in a regional colocation facility with a 5 kW per rack cooling allowance.

B. Data Platform Rack. A growing analytics team needs a small, durable data platform in one rack: one PostgreSQL primary plus one read replica (each backing a 4 TB working set), and a Ceph object storage cluster sized for 200 TB usable at three-way replication. Optimize for IOPS on the database tier and durability on the object tier. The rack will live in your own data hall with a 7 kW per rack cooling allowance.

C. AI Inference Rack. A product team needs to serve large-language-model inference in production. Eight GPU servers (each holding eight accelerators), plus two head nodes, meaning smaller control-plane servers that run orchestration and cluster coordination rather than serving inference traffic directly, and one storage server holding model weights. Optimize for power density, east-west bandwidth between GPU servers, and resilience to a single power supply unit (PSU) loss. The rack will live in a purpose-built AI hall with direct-to-chip liquid cooling rated at 40 kW per rack.

Stay with your scenario

Every later section refers back to the scenario you pick here. Switching scenarios partway through means redoing earlier sections. Pick once and commit.

In the Scenario section of your document, write the scenario letter, a one-sentence summary of the workload, and the facility envelope you must stay inside: 5 kW for A, 7 kW for B, or 40 kW for C.

---

Server Selection

Server selection is the backbone of the rack. Chassis height, socket count, memory population, and local boot storage determine how much work fits in the rack and how much power and cooling the rack will need.

In a two-socket server, each CPU has its own local memory banks. Populate DIMMs evenly across both sockets so the server exposes balanced memory channels instead of leaving memory bandwidth idle.

1. Open a vendor configurator. The Dell PowerEdge rack-server catalog, HPE rack-server catalog, and Supermicro system selector are the quickest starting points. Pick one vendor and stay with it for this activity.

2. Pick the chassis form factor.

   • Scenario A: A dense 1U or 2U virtualization host. Single-socket systems can win on core density per watt, while two-socket systems can win on per-host memory capacity and expansion (Dell R660 / R760, HPE DL360 / DL380, or a single-socket EPYC platform are all reasonable starting points).
   • Scenario B: A 1U or 2U for the database servers, and a 2U-with-many-bays chassis for Ceph nodes (Dell R760xd, HPE DL380, or a Supermicro storage-dense 4U).
   • Scenario C: A dense GPU server sized to the accelerator class. Lower-power inference GPUs can fit in 2U or 4U PCIe servers, while hotter training-class boxes are often 4U or 5U (Dell XE9680, HPE Cray XD, or Supermicro SYS-821GE are examples of the larger class).

3. Pick the CPU. AMD EPYC and Intel Xeon are the current mainstream server CPU families. Pick one socket count and one SKU per server role.

   • Note the per-socket core count, base and boost frequencies, and supported memory channels per socket. Server CPUs typically support eight or twelve memory channels per socket.
   • Two sockets means two NUMA nodes; record this.

4. Populate memory. Pick a single DIMM capacity and quantity per server. The goal is to fill memory channels evenly across both sockets so the BIOS reports balanced NUMA nodes.

   • For a two-socket server with eight channels per socket, one DIMM per channel means 16 DIMMs total. Most chassis support two DIMMs per channel (2 DPC), at which point speed often drops a tier and LRDIMMs become the right choice for the densest configurations.
   • Record total RAM per server.

5. Pick drive bays. Decide which drive bays each server needs for boot only versus boot plus local data. Most bulk data lives on the storage architecture chosen in the next section; for now, ensure each server has two small SSDs in a boot mirror.

6. Note the redundancy items. Confirm dual power supply units (PSUs) are configured (almost always the default for rack servers). Confirm the Baseboard Management Controller (BMC) is the standard one for the vendor (iDRAC on Dell, iLO on HPE, IPMI/Redfish on Supermicro).

   • If the configurator or data sheet lists a maximum system power or maximum input wattage, record it now. You will use that number in the Power section instead of inferring load only from PSU sizes.

7. Calculate quantity. How many of this server SKU do you need to meet the scenario’s demand? Show your math.

   • Scenario A: Enough servers to host 200 VMs with the stated memory requirement. You may start with a conservative 2:1 vCPU-to-physical-core consolidation ratio if you state it explicitly. Here, 2:1 means planning that two virtual CPUs can share one physical CPU core on average because not every VM is busy at the same moment. In a very tight power envelope, you may justify a somewhat higher ratio for bursty internal workloads, but explain why that tradeoff is still acceptable. Do not overcommit memory. Leave at least one whole host worth of headroom so a single failure does not break the scenario.
   • Scenario B: One primary plus one replica for PostgreSQL (two database servers). For Ceph, enough storage nodes to hold 200 TB usable at three-way replication. These are often called OSD nodes because they run Ceph’s Object Storage Daemon (OSD) processes, which are the software components that actually store the data. Three-way replication means roughly 600 TB raw plus headroom. Three Ceph nodes is the smallest practical starting point; use more if losing one node would drop you below target usable capacity.
   • Scenario C: Eight GPU servers, two head nodes, and one storage server, all given.

Think About It

Your scenario specifies a workload, but you chose a specific CPU SKU and memory configuration. Trace the chain of decisions backward: which property of your workload drove the choice of socket count? Which one drove the choice of per-socket memory bandwidth? If the workload doubled in working-set size, would you keep the same SKU and add servers, or change the SKU?

In the Servers section of your document, record the vendor and model, CPU SKU, RAM total per server, drive-bay configuration, BMC type, quantity, and rack units (U) consumed per server.

---

Storage Architecture

Now decide where the data lives. Some workloads want storage inside the server, some want a shared appliance on the network, and some spread data across several storage nodes so the cluster survives a drive or node failure.

The key choice is where the failure boundary sits: inside one chassis for DAS, at a shared filer for NAS or SAN, or across several hosts for distributed storage.

1. Pick the storage model for the scenario.

   • Scenario A: Each virtualization host likely uses some local NVMe for VM data plus a shared NFS or iSCSI back-end so VMs can migrate between hosts. Decide whether you are running fully shared storage (NAS or SAN with a separate storage server in the rack) or local storage per host with replication at the hypervisor layer.
   • Scenario B: PostgreSQL almost always lives on local NVMe for latency. Ceph is a distributed system that places data across OSDs on several storage nodes and presents it as object, block, or filesystem to clients on the network. There is no separate “storage server” in Ceph; the OSD nodes are the storage.
   • Scenario C: Model weights live on a fast shared filesystem (NFS over Remote Direct Memory Access (RDMA), or a parallel filesystem like Lustre or IBM Storage Scale). GPU servers stream weights at job start. Decide whether the storage server in the rack runs NFS, or whether a parallel filesystem is justified.

2. Pick drive types. Decide HDD, SATA SSD, or NVMe SSD per role.

   • PostgreSQL data: NVMe, with the highest sustained-write endurance class your budget allows.
   • Ceph object data: NVMe SSDs are the preferred choice for new all-flash deployments; SATA SSDs are still acceptable when budget or an older platform constrains you. Large HDDs (16 TB or 20 TB) are still common for colder capacity tiers. Mixed-tier Ceph clusters are common.
   • Virtualization shared back-end: NVMe is the default in new deployments.
   • Boot drives: small SATA SSDs in a mirror; reliability matters more than performance.

3. Pick the RAID level for each role. For database and shared-storage roles, RAID 10 is the default when you want mirrored performance and fast rebuilds, while RAID 6 trades some write speed for more usable capacity and tolerance of two drive failures. For Ceph, RAID is replaced by replication; record the replication factor instead. A replication factor of 3 means the cluster stores three copies of each object on different OSDs or nodes.

   A note on RAID 5 and large drives

   RAID 5 has become risky for large drives because rebuilds on multi-terabyte arrays take many hours, and a second drive failure during rebuild loses everything. Default to RAID 6 or RAID 10 for any array with drives larger than 4 TB.

4. Calculate usable capacity. For each storage role, compute usable capacity from drive count, drive size, and RAID level (or replication factor for Ceph). Show your work in the document.

Think About It

Compare your storage choice for this scenario against the storage choice that would be wrong for it. For example, if you picked NVMe with RAID 10 for PostgreSQL, what specifically would happen if you instead used 8 TB HDDs in RAID 5? Name the operational failure mode that choice would surface.

In the Storage section of your document, record the storage model (DAS / NAS / SAN / distributed), drive type per role, RAID level or replication factor, drive count, drive size, and usable capacity per role.

---

Top-of-Rack Networking and Uplinks

Every server needs a path into the rack network, and some workloads need much more than simple connectivity. Focus on two numbers: access speed, which is the bandwidth of the server-facing ports, and oversubscription, which is the ratio between all server-facing bandwidth and the rack’s uplinks toward the rest of the data center.

1. Count access ports needed. Each server typically needs two redundant data network ports, often presented as one dual-port Network Interface Card (NIC), plus one BMC NIC. Total the data ports first.

   • For Scenario C, GPU servers often have two or four data NICs at higher speed; count carefully.

2. Pick a top-of-rack switch. Look at a real vendor data sheet: Arista 7050X, Juniper QFX5120, or Cisco Nexus 9300. Pick one with enough access ports at the speed your scenario needs.

   • Scenario A: 10 or 25 GbE access ports are fine for general-purpose VMs.
   • Scenario B: 25 GbE access ports; Ceph rebuilds saturate links.
   • Scenario C: 100 GbE or higher access ports for the GPU servers, with RoCE (RDMA over Converged Ethernet) enabled. Lossless behavior matters here. In this context, lossless means the RoCE traffic class is configured to avoid ordinary packet drops across the path, not that the whole network literally never drops traffic.

3. Plan uplinks to the spine. Count the uplink ports your chosen ToR has at the higher speed tier (usually 100 GbE or 400 GbE). Decide how many of those uplinks to use.

   • The oversubscription ratio is (sum of access port bandwidth) divided by (sum of uplink bandwidth). 3:1 is a reasonable target for general workloads; 1:1 (non-blocking) is what AI fabrics aim for; storage fabrics typically run 1.5:1 or better. These are workload-dependent rules of thumb, not universal laws: all-to-all traffic patterns need lower oversubscription than bursty request-response workloads.
   • If you choose two active ToRs, compute the steady-state ratio using the combined access and combined uplinks across both switches. Here, steady-state means normal operation with both ToRs up and forwarding traffic. Then add one sentence describing how the ratio changes if one ToR fails.
   • Record your computed oversubscription ratio for the scenario and explain whether it is appropriate.

4. Decide on a second ToR for redundancy. A single ToR is a single point of failure for the whole rack. Most production designs use two ToRs and cable each server to both. The exact host-side design depends on the platform: some environments use MLAG or vPC-style active-active uplinks, while others use active-standby bonding. Note this decision; it doubles your switch count.

Think About It

Your oversubscription ratio is one number, but it implies an assumption about how much of the access-port bandwidth is used at once. Under what workload pattern would a 3:1 ratio still feel non-blocking, and under what pattern would the same 3:1 design cause visible problems? Tie your answer to the scenario you picked.

In the Networking section of your document, record the ToR vendor and model, access port count and speed, uplink port count and speed, computed oversubscription ratio, and whether the rack uses one or two ToRs.

---

Out-of-Band Management

Every server you picked has a BMC, which is a small management computer that can power-cycle the host and open a remote console even when the operating system is down. Those ports belong on a separate management network so a production outage does not also take away your recovery path.

1. Count BMC ports needed. One BMC NIC per server, plus one per ToR (most switches have a dedicated management port).

2. Pick a 1U management switch. A simple 1 GbE switch is enough. List a real model from the same vendor as your ToR or from a separate small vendor.

3. Note the management VLAN. All BMC ports terminate in one VLAN, separate from any production VLAN, reachable only through a jump host or firewall with MFA. Record the VLAN ID you would use. Pick any unused ID and keep it separate from every production VLAN.

4. Decide how operators reach the rack. Do operators reach the BMC over the corporate LAN through a firewall, over a dedicated out-of-band management WAN, or both? One sentence is enough.

In the Management section of your document, record the management switch model, BMC NIC count, management VLAN ID, and operator access path.

---

Power Budget

A rack design is not finished until the electrical side closes. Use conservative numbers here: the goal is to prove the rack can survive the loss of one power path without overloading the remaining side.

For the tier call at the end of this section, use this shorthand: Tier I means basic capacity, Tier II adds redundant components, Tier III supports concurrent maintenance, and Tier IV means the surrounding facility is fault tolerant.

1. Find the worst-case power per server. Prefer a vendor maximum-system-power or maximum-input-watts figure if the configurator or data sheet provides one. If not, use the load one power side must be able to carry by itself as a conservative upper bound. For a dual-PSU server, that usually means one PSU’s rating; for a 3+3 GPU chassis, it is the combined rating of one side. A typical 1U or 2U server has redundant 800 W to 1400 W PSUs. A GPU server may have six 3000 W PSUs.

2. Multiply by server count. Sum the worst-case watts across all servers in the rack, then add the network gear. If the switch data sheet lists maximum system power, use that number. If it only lists PSU sizes, use a stated planning estimate instead: about 200 W to 300 W per ToR and 25 W to 50 W for a small management switch is reasonable for this activity.

3. Apply 80% derating. Rack Power Distribution Units (PDUs) and circuits are typically rated to be loaded to no more than 80% of their nameplate capacity, to leave headroom for inrush and to comply with electrical code. Divide your total by 0.8 to get the PDU capacity you actually need to specify.

4. Pick a PDU model and amperage. A real example: after 80% derating, a 30A 208V three-phase PDU supports roughly 8.6 kW of usable continuous load, while a 60A 415V three-phase PDU supports roughly 34.5 kW. Confirm the phase count before you do the math: a 30A 208V single-phase circuit is only about 5.0 kW usable after derating. Pick a real Eaton, APC, or ServerTech model from a vendor data sheet that meets your derated total.

5. Confirm A/B feeds. State explicitly that the rack has two PDUs fed from independent power paths (A and B), each sized to carry the full rack load alone, and that every dual-PSU server is wired one PSU per PDU. This is what lets you survive the loss of one feed.

6. Name an Uptime Tier. Based on that shorthand, which tier can this rack participate in, and which higher tier would still require facility features outside the rack? Write one sentence for each. For the official terminology and certification criteria, see the Uptime Institute Tier classification.

Think About It

Your A/B feed design is what makes the rack tolerant of a PDU failure. But the rack alone is not Tier IV; the data center facility around it is what determines the achievable tier. Name the part of the design that lives outside this rack and must also be present for Tier IV to apply.

In the Power section of your document, record the worst-case watts per server, total rack watts, derated total, PDU model and rating, A/B feed confirmation, and supported Uptime Tier.

---

Cooling Envelope

Electrical load becomes heat. Once you know the rack’s kW, you can match it to a cooling method and check whether the facility named in the scenario can actually support the design.

1. Read the kW per rack from your actual worst-case rack load, not from the derated PDU capacity you sized to support it. (Use kW, not W.)

2. Match it to a cooling technology:

   • Up to about 8 kW per rack: standard CRAC or CRAH room cooling is often fine.
   • 8 to 20 kW per rack: in-row cooling or rear-door heat exchangers are common choices.
   • 20 kW per rack and above: direct-to-chip liquid cooling, or immersion.
   • Treat these as conservative planning guidelines. Actual limits depend on the facility’s airflow design, rack spacing, containment, and cooling plant.

3. Cross-check the scenario. The scenario named a cooling envelope at the start (5 kW for A, 7 kW for B, 40 kW for C). Does your rack design fit inside that envelope?

   • If your design exceeds the envelope, you have two options: redesign the rack to fit (fewer servers, lower-TDP CPUs, smaller GPUs), or choose a different facility class. State which you would do and why.

Think About It

The same servers in the same rack require dramatically different cooling depending on the facility they live in. Why is the cooling decision a facility decision rather than a rack decision? Name one upstream consequence of needing direct-to-chip liquid cooling that would not exist with air cooling.

In the Cooling section of your document, record the kW per rack, chosen cooling technology, and whether the design fits the scenario’s stated envelope.

---

Your Rack Specification

You now have every section of the document filled in. The last pass is coherence: does the rack fit physically, electrically, and operationally, and does your write-up show enough reasoning that another engineer could challenge it?

1. Sanity-check the U count. Sum the rack units consumed by all servers, ToRs, and the management switch. Most rack PDUs mount vertically and consume zero front-facing U; only count PDUs against 42U if you intentionally chose a horizontal model. Confirm the total fits in a 42U rack. If it does not, note what you would do (a second rack, denser servers, fewer storage nodes).

2. Sanity-check the network. Confirm that every server has the right number of NICs at the right speed to reach your chosen ToR. Confirm the uplink count and the resulting oversubscription ratio.

3. Sanity-check the power. Confirm worst-case watts fit inside the cooling envelope and inside the derated PDU capacity.

You should now see a concise rack specification document that names a real vendor, real models, real numbers, and one Uptime Tier the design supports. That is what a junior infrastructure engineer hands their senior on day one of a real rack build.

Going Further

You speced one rack. A real data center has hundreds. The natural next step is to think about how those racks connect to each other, and what the facility around them looks like.

If you want to see a real published rack design, the Open Compute Project publishes the hardware specifications used by Meta, Microsoft, and others. Read one OCP server spec and note three ways it differs from the Dell or HPE SKU you picked.

If you want to push on the facility side, read the Uptime Institute Tier classification overview and look up the closest commercial colocation provider to where you live. Read their facility brief and identify which Tier they claim and what features back the claim: dual utility feeds, generator runtime, N+1 cooling, concurrent maintainability.