Professional Documents
Culture Documents
Vaibhav K. Arghode
Yogendra Joshi
Air Flow
Management in
Raised Floor Data
Centers
123
SpringerBriefs in Applied Sciences
and Technology
v
Acknowledgments
This book is based on the work undertaken at the Data Center Laboratory at Georgia
Tech. This research was supported by the National Science Foundation Industry/
University Cooperative Research Center on Energy Smart Electronic Systems
(ES2). Additional support from Degree Controls, Inc., and Triad Tiles, Inc., is
acknowledged. The authors also acknowledge help from Yunji Gu and Ian Mok in
the preparation of Chap. 5.
vii
Contents
1 Introduction ................................................................................................. 1
1.1 Data Centers ......................................................................................... 1
1.2 Energy Usage Analysis of a Raised Floor Data Center ....................... 3
1.3 Thermodynamics of Data Center Cooling ........................................... 6
2 Metrology Tools ........................................................................................... 11
2.1 Tile Air Flow Rate Measurement ......................................................... 11
2.1.1 Commercial Flow Hood ........................................................... 12
2.1.2 Operating Principle of Flow Hood for Passive Tiles ............... 12
2.1.3 Operating Principle of Flow Hood for Active Tiles ................. 14
2.2 Rack Air Flow Rate Measurement ....................................................... 16
2.2.1 In-house Developed Tool ......................................................... 17
2.2.2 Tool Resistance Compensation ................................................ 17
2.3 Flow Field Measurement ..................................................................... 21
2.3.1 Particle Image Velocimetry (PIV) Technique .......................... 21
2.3.2 Principle of PIV Technique ...................................................... 21
2.3.3 PIV System at the Data Center Laboratory, Georgia Tech ...... 22
2.4 Thermal Field Measurement ................................................................ 23
2.5 Pressure Measurement ......................................................................... 24
2.5.1 Mass Flow Based Pressure Sensor ........................................... 25
2.5.2 Piezoresistive Pressure Sensor ................................................. 26
3 Cooling Air Delivery Through Perforated Tiles....................................... 27
3.1 Experimental Investigation .................................................................. 28
3.1.1 Experimental Setup .................................................................. 28
3.1.2 Cases Investigated .................................................................... 30
3.1.3 Effect of Tile Porosity .............................................................. 31
3.1.4 Effect of Tile Pore Size ............................................................ 31
3.1.5 Effect of Blocked Edges .......................................................... 32
3.1.6 Effect of Tile Width ................................................................. 32
ix
x Contents
References .......................................................................................................... 67
Index ................................................................................................................... 71
List of Tables
xi
List of Figures
xiii
xiv List of Figures
Fig. 2.5 Rack air flow rate measurement tool. (a) Photographs of the tool.
(b) Calibration of the tool .................................................................... 17
Fig. 2.6 Typical rack air flow characteristics with posterior cover.
(a) Typical server rack air flow characteristics.
(b) Schematic of a typical server rack. (c) Rack air flow
characteristics with posterior cover. (d) Without posterior
cover. (e) With posterior cover ......................................................... 18
Fig. 2.7 Details of the server simulator rack under investigation.
(a) Server simulator rack. (b) Measured fan speed.
Non-uniformity within ±10 % ......................................................... 19
Fig. 2.8 Measured slope (S) for a range of fan speeds. (a) Rack air
flow characteristics, (ΔPr = S × Q + T). (b) Reduction in air
flow rate due to tool resistance......................................................... 20
Fig. 2.9 Velocity field measurement using particle image
velocimetry (PIV) set-up. (a) Principle of PIV.
(b) PIV set-up at Georgia Tech ........................................................ 22
Fig. 2.10 Thermal field measurement tool. (a) Mobile 3D temperature
field measurement tool. (b) Mobile planar temperature
field measurement tool ..................................................................... 24
Fig. 2.11 Two types of differential pressure sensors for air cooled
data centers. (a) Mass flow rate based differential pressure
sensor (source: www.sensiron.com). (b) Piezoresistive
differential pressure sensor (source: www.allsensors.com)............. 25
Fig. 3.1 Experimental setup for investigating airflow supply
through perforated tiles. (a) Experimental set up.
(b) Plenum. (c) Perforated sheet ...................................................... 29
Fig. 3.2 Effect of tile geometry on air delivery from a perforated tile.
(a) Base case. (b) Lower porosity. (c) Smaller pore size.
(d) Larger edge blockage. (e) Reduced tile width ........................... 30
Fig. 3.3 Geometrically resolution of perforated tile ...................................... 33
Fig. 3.4 Representation of rapid models for air flow through
perforated tiles. (a) Porous jump (PJ) model. (b) Body force (BF)
model. (c) Modified body force (MBF) model. (d) MBF model
with blocked edges. (e) MBF region. (f) MBF region
height (H) estimation......................................................................... 34
Fig. 3.5 Comparison of modified body force (MBF)
with the geometrical resolution (GR) model. (a) Computational
domain. (b) Base case. (c) Smaller pore size. (d) Higher porosity.
(e) Blocked edge .................................................................................. 38
Fig. 3.6 Comparison of contours of geometrical resolution (GR)
and modified body force (MBF) models for small pore sizes.
(a) Computational domain. (b) D = 0.125″, GR model.
(c) D = 0.125″, MBF model. (d) D = 0.25″, GR model.
(e) D = 0.25″, MBF model ................................................................ 42
List of Figures xv
Symbols
xvii
xviii Nomenclature
Sub-/Super Scripts
*
To be specified across the tile
100 100 % fan speed for dial setting 10
air Property of air
air-con Property of air flowing through the condenser
air-CRAC Property of air flowing through CRAC units
Nomenclature xix
In this chapter we introduce the thermal management aspect of air cooled data
centers based on thermodynamics. The energy consumption by various cooling
components and the influencing factors are discussed with a view towards minimizing
the overall energy expenditure.
In USA, data centers are estimated to consume about 2 % of the total electricity gener-
ated, see Fig. 1.1. Energy consumed for cooling purposes accounts for about 31 % of
the total energy usage by a typical data center (Koomey 2011). Air cooling is most
commonly used in current data centers because of low installation and maintenance
costs, and minimal hardware intrusion to the servers (Joshi and Kumar 2012).
Two commonly used air delivery schemes are overhead and underflow supply as
depicted in Fig. 1.2. In the overhead supply scheme, the cold air from the computer
room air conditioning (CRAC) units is pumped into overhead plenums and supplied
through perforated roof tiles into the cold aisle. The cold aisle consists of rows of
server rack inlets on either sides. The hot air exiting the racks (from hot aisle) is
recirculated from the room space near the bottom of the CRAC units to complete the
loop, see Fig. 1.2a. In the underfloor supply scheme a raised floor is required to
form a subfloor plenum which is pressurized from the cold air pumped from the
CRAC units. The cold air is supplied to the cold aisle using perforated floor tiles,
and hot air exiting the racks is recirculated to the top end of the CRAC units either
through the room space or through over-head plenum, see Fig. 1.2b.
Fig. 1.1 Electricity consumption by data centers (source: Koomey 2011). Reprinted with permis-
sion from Analytics Press
Both schemes have their own advantages and limitations. Here we will discuss
only the raised floor data centers employing underfloor air delivery.
Servers in a data center are generally arranged in standard sized racks and have
built in fans to entrain cold air from the front, and exhaust hot air from the back. Air
flow management is required to achieve the desired server inlet temperature profile
by minimizing mixing between the supplied cold air, and the hot air exiting from the
back of the rack.
Generally, racks are arranged in rows such that the inlets and exits of adjacent
rows of racks face each other, thus providing an alternating hot aisle-cold aisle
(HACA) arrangement, as seen in Fig. 1.3. This arrangement partially separates hot
and cold air paths, however, hot air entrainment in the cold air stream or direct
by-pass of cold air may still be present (Arghode et al. 2013b).
1.2 Energy Usage Analysis of a Raised Floor Data Center 3
a b
(Down flow)
(Up flow)
Data Racks
Data Racks
Data Racks
Data Racks
CRAC
CRAC
Fig. 1.2 Air delivery schemes at the Data Center Laboratory (DCL), Georgia Tech. (a) Overhead
supply of cooling air. (b) Underfloor supply of cooling air (raised floor data center)
Fig. 1.3 Raised floor Data Center Laboratory (DCL) at Georgia Tech
The components used in a typical raised floor data center layout are shown in
Fig. 1.4. Cooling air is pumped in an underfloor supply plenum using powerful
blowers in the CRAC units. Air from the plenum reaches the cold aisle through
perforated floor tiles and is entrained through the servers due to inbuilt fans. The hot
air exiting the servers returns to the CRAC units either via an overhead return
4 1 Introduction
plenum as shown in Fig. 1.4, or directly from the room space. The heat from the
return air is rejected to the cooling coils in the CRAC unit to complete the loop. The
heat rejected to the water loop in the cooling coils can be eventually rejected to
the atmosphere using a refrigeration based chiller plant. Figure 1.4 also shows impor-
tant thermal/flow measurements at various locations, that can aid in effective
thermal management of an air cooled data center.
Typical energy consumption for cooling purposes from various data center
components, based on benchmarking studies, is shown in Fig. 1.5. The chiller and
the computer room air conditioning (CRAC) fans account for majority of the energy
consumption, with typical contributions of 46 % and 28 % respectively (Schmidt
and Iyengar 2009). The chiller energy consumption is mostly for the compressor of
the vapor compression refrigeration plant. For the CRAC, this is primarily for air
movement. Thus to reduce the cooling energy consumption it is imperative to focus
on these two components.
One way to reduce energy consumption in the chiller plant is by setting higher
cooling air temperature, supplied through the CRAC units (Moore et al. 2005). This
is because the coefficient of performance (COP) of the chiller increases with reduc-
tion in the temperature difference between the evaporator and the condenser. For the
case with uniform server inlet air temperature and small difference between the
supplied air and server inlet air temperature, the supply air temperature from the
CRAC units can be set to higher value, while meeting the server inlet air tempera-
ture recommendations. This, in turn, can lead to higher evaporator temperature and
hence higher chiller COP.
To further minimize the hot and cold air mixing, physical separation of hot and
cold aisles has been suggested by containing the cold aisle (Joshi and Kumar 2012).
1.2 Energy Usage Analysis of a Raised Floor Data Center 5
It was observed that cold aisle containment resulted in more uniform thermal field,
as compared to the case with open aisle (Arghode et al. 2013b). In this case, near
ideal server inlet temperature profile was obtained for the over-provisioned case,
with higher total supplied air flow rate than the total rack air requirement. In this
case, higher energy is required for the movement of excess air by CRAC fans. Thus
energy savings in the chiller plant can be offset by extra energy requirement for
CRAC fans for an over provisioned case and an optimized setting needs to be
employed for energy efficient operation. Note that, for the over provisioned case
with open aisle, hot air recirculation near the end racks was present, resulting in
higher server inlet temperatures for the end racks (Arghode et al. 2013b). Partial
cold aisle containment, where only the top curtain or only the doors at the aisle
entrance, was investigated in (Sundaralingam et al. 2013). It was observed that
deploying only the doors at aisle entrance was more effective than deploying only
the top curtain. This also suggests the importance of curtailing the hot air entrainment
from the aisle entrance. In another experimental investigation, cold aisle contain-
ment was observed to result in lower fan power and about 15 % of energy savings
(Takahashi et al. 2008).
Data center air delivery is a multi-scale problem where floor tile pore size is of
the order of mm, while the room size can range upto tens of m, as shown in
Fig. 1.6. Note that air flow emerges from the sub-floor plenum as jets (~cm) from
the pores. These jets interact and merge to form large length scale air flows, which
enter the adjacent rack (~m). The air flow exiting a tile can interact with the air
flow from surrounding tiles, as well as entrain the adjacent hotter room air to
cause large scale (~m) mixing of hot and cold air, which governs the thermal field
in the room (~10 m). Hence, overall understanding of flow and thermal field
dynamics in a data center requires investigation from length scale of tile pores to
the full room space.
The focus here is on air flow management to minimize energy consumption for
cooling. The next section describes basic thermodynamics for data center cooling.
6 1 Introduction
A heat flow model for a typical air cooled data center and the corresponding
formulae are shown in Fig. 1.7. Also refer to Fig. 1.4 for various components of a
typical raised floor data center. Cold air at temperature Tsup is supplied to the data
center space using computer room air conditioning units (CRAC). CRAC units have
powerful fans to move large quantities of air in a loop through the air-water heat
exchangers and the room space. The supplied air picks up the heat generated by the
information technology (IT) equipment (QIT) and returns to the CRAC units having
chilled water heat exchanger at Tret. The power consumed by the CRAC fans is
denoted as PCRAC, which scales linearly at design condition with the amount of sup-
plied air (mair-CRAC) for different systems. Note that according to the fan laws, the fan
power varies as cubic power of the volumetric flow rate of air for a given system
(Bleier 1997), consisting primarily of CRAC heat exchanger and the perforated tiles
for data centers. However, for higher air flow rates, more CRAC units and more tiles
or larger CRACs can be used to maintain the linear scaling of air flow rate with
CRAC power. For active tiles, additional power will be spent for the air movement,
denoted here as Ptiles.
The chilled water loop from the CRAC units transfers heat to the evaporator of
the chiller plant using water-refrigerant heat exchanger. The leaving chilled water
temperature from the evaporator, same as entering the CRAC heat exchanger, is
denoted as Teva-wat-out. The water enters the evaporator, same as leaving the CRAC
1.3 Thermodynamics of Data Center Cooling 7
Fig. 1.7 Data center energy modeling (source: Arghode et al. 2015b). Applied for permission
from Taylor and Francis
heat exchanger, at Teva-wat-in. The refrigerant temperature in the evaporator is Teva. The
chiller plant in turn transfers the heat to the condenser side of the plant, while con-
suming input mechanical power (Pchiller). Using water-refrigerant heat exchanger on
the condenser side the heat is transferred via warm water loop to air-water heat
exchanger having fans to move the ambient air for cooling. Cooling towers, employ-
ing water spray and evaporative cooling can also be used, however in the present
model only the air-cooled condenser is considered. The refrigerant temperature in
the condenser is denoted as Tcon. The warm water enters the condenser at Tcon-wat-in
and leaves it at Tcon-wat-out. Note that the warm water leaving and entering the con-
denser is considered at the same temperature as water entering and leaving the air-
water heat exchanger on the condenser side, respectively.
Major input power for rejecting the heat from the data center space (QIT) is for
the compressor (Pchiller) of a generally used vapor compressor refrigeration based
chiller plant, CRAC fans (PCRAC), and condenser fans (Pcon). The energy consump-
tion from the liquid pumps is relatively small and is not considered here. The coef-
ficient of performance (COP) of the chiller is modeled based on Carnot efficiency,
which depends on the evaporator (Teva) and condenser (Tcon) temperatures (Carnot
COP = Teva/(Tcon − Teva)). Note that with increase in evaporator temperature, linked to
supply air temperature, and decrease in condenser temperature, linked to ambient
air temperature, the chiller COP will increase, resulting in lower power consumption
8 1 Introduction
at the chiller plant. The chiller COP also varies based on part load operation (Gordon
and Ng 1994), however, for simplification only the design load condition is
considered.
The temperature of heat exchanging fluids is calculated based on the respective
heat exchanger’s effectiveness. The effectiveness of air-water CRAC heat
exchanger is denoted as εeva-wat-air and εcon-wat-air, for air-water heat exchanger at the
CRAC unit, and at the condenser side respectively. Also, εeva and εcon denote water-
refrigerant heat exchanger effectiveness at the evaporator and at the condenser
respectively. Note that the heat exchanger effectiveness is the ratio of actual heat
transfer rate to the maximum possible heat transfer rate for counter-flow heat exchanger.
The low thermal capacitance fluid is air for air-water heat exchanger, and water for
water-refrigerant heat exchanger. The constants used for the present model are
summarized in Table 1.1. The expression for effectiveness for water-air heat
exchangers corresponds to that for cross-flow heat exchanger, where both the fluids
are unmixed (Incropera and Dewitt 1996).
The power usage effectiveness (PUE) can be obtained from the ratio of total
power consumed in the data center (QIT, PCRAC, Ptiles, Pchiller, Pcon) and the IT power
(QIT). IT power (QIT) can be measured using a calorimetric method, refer to (Arghode
et al. 2013b) for more details, or directly obtained from supply power measurement.
The power consumption from the CRAC units (PCRAC) and the active tiles (Ptiles) can
also be measured. Hence, chiller compressor power (Pchiller) and condenser fan
power (Pcon) can be estimated using the model shown in Fig. 1.7 and Table 1.1.
It has been experimentally observed that even if the amount of air supplied from
a perforated tile matches the adjacent rack air requirement, significant entrainment
of hot room air is present (Arghode and Joshi 2015a; Kumar and Joshi 2010). This
suggests that the server air inlet temperature can be higher as compared to the sup-
plied air temperature even in a fully provisioned case, and would depend on the cold
and hot air mixing characteristics. The same is illustrated here in Fig. 1.8, using an
example of a fully provisioned case, where measured thermal field and server air
inlet temperatures are shown. From the figure it may be noted that due to hot air
entrainment from aisle ends (see Fig. 1.8b), the thermal field non-uniformity at the
server inlets is as high as 5 °C (see Fig. 1.8d).
Note that there is an upper limit for acceptable server air inlet temperature
(ASHRAE TC 9.9 2011), hence, due to thermal field non-uniformity the supply air
temperature needs to be reduced so that all the servers meet the specified limit.
Lowering of supply air temperature will lead to reduced COP of the chiller, see
Fig. 1.8c (model based on Fig. 1.7), and higher cooling energy consumption. Thus
an efficient air delivery scheme would intend to minimize the hot and cold air mix-
ing, so as to achieve uniform server inlet air temperatures, as close as possible to the
supply air temperature.
It is possible to flood the cold aisle by supplying much larger quantities of cooling
air, as compared to the rack air requirement, i.e. over provisioning. This is more likely
to result in uniform server inlet air temperatures closer to the supplied air temperature,
and would allow the chiller plant to work more efficiently (Arghode et al. 2013b).
1.3 Thermodynamics of Data Center Cooling 9
Table 1.1 Constants used for the energy modeling (source: Arghode et al. 2015b). Applied for
permission from Taylor and Francis
Authors Others
Chiller performance
γchiller 0.5 0.48 (Breen et al. 2012), 0.55
(Khalifa and Demetriou 2010)
Heat exchanger effectiveness (ε)
εwat-air-eva 0.75 0.40 (Breen et al. 2012), 0.77 (Iyengar and Schmidt
2009), 0.76 (Khalifa and Demetriou 2010)
εwat-air-con 0.75
εeva 0.9 0.88 (Breen et al. 2012)
εcon 0.9 0.90 (Breen et al. 2012)
Fan power coefficient
δCRAC (kW/(m3/s)) 1.22 1.82 (Iyengar and Schmidt 2009), 2.03 (Khalifa and
Demetriou 2010), 0.982 (Samadiani et al. 2010)
δcon (kW/(m3/s)) 0.2 0.23 (Yu and Chan 2006), 0.19 (Johnsons Controls 2015)
Capacity rate ratio (Cr = Cair/Cwater)
Crwat-air-eva 0.625 0.60 (Breen et al. 2012), 0.45 (Iyengar and Schmidt 2009)
Crwat-air-con 0.6
Number of Transfer Units (NTU)
NTUwat-air-eva 2.4 2.78 (Iyengar and Schmidt 2009)
NTUwat-air-con 2.1
NTUeva 2.3 2.12 (Breen et al. 2012)
NTUcon 2.3 2.30 (Breen et al. 2012)
Temperature rise
(Tret − Tsup) (°C) 8 9.04 (Breen et al. 2012), 10.23 (Khalifa and Demetriou 2010)
(Tout − Tamb) (°C) 10 10.17 (Johnsons Controls 2015)
(Teva-wat-in − 5 5.93, 5.5 (Yu and Chan 2006), 5.58
Teva-wat-out) (°C) (Johnsons Controls 2015)
(Tcon-wat-out − 5 6.32 (Breen et al. 2012)
Tcon-wat-in) (°C)
However, as observed from the model above, movement of larger quantities of air
would increase the energy consumption and may even offset the energy savings at
the chiller plant, thus an optimal solution needs to be sought for energy efficient
operation. This example illustrates the importance of effective air flow manage-
ment, which will directly impact the energy consumption in a data center.
Closure
Data centers are building scale facilities and require careful thermal management to
potentially achieve significant energy savings associated with cooling. As air cool-
ing is extensively used in current data centers, efficient air flow management is very
important. One of the major objectives of air flow management is to reduce hot and
cold air mixing, so that uniform and low temperature can be maintained at the server
10 1 Introduction
Fig. 1.8 Hot air entrainment in fully provisioned case. (a) Temperature measurement in middle
plane. (b) Thermal field (°C) in middle plane. (c) Chiller COP vs. supply air temperature. (d)
Server inlet air temperature (source: Arghode et al. 2015b). Applied for permission from Taylor
and Francis
inlets, thus allowing the chiller to operate at higher efficiency level. Air movement
through the data center room space also consumes a significant portion of the cool-
ing energy. This depends on the amount of air supplied, and pressure losses due to
CRAC cooling coils and perforated tiles present in its flow path. Often, reduction in
amount of supplied energy can result in lower energy expenditure on air movement.
However, it can adversely affect the thermal field uniformity at the server inlets, and
thus may increase energy consumption at the chiller plant. A careful balance is
sought, so as to minimize the overall cooling energy consumption by designing an
effective air flow management system.
Chapter 2
Metrology Tools
To understand the air flow and thermal field characteristics in a data center it is
imperative to experimentally measure the relevant parameters. Here we discuss
some metrology tools for thermal and flow measurements relevant to air cooled data
centers. The tools discussed here include measurements of tile and rack air flow
rates, and flow, thermal, and pressure fields. Both commercially available, and in-
house developed tools are discussed.
In raised floor data centers, cooling air is supplied from a sub-floor pressurized ple-
num through perforated floor tiles. These tiles can be either passive, or active, with
integrated fans, depending on the mode of operation (Arghode et al. 2015b). Active
tiles are mostly used to satisfy localized higher cooling air requirement of an adja-
cent higher heat load rack, placed among lower heat load racks. For effective cool-
ing, the desired flow rate of air, satisfying the adjacent rack air requirement, must be
supplied through the perforated tiles. Hence, to design and understand effective
cooling air delivery schemes, it is important to measure the tile air flow rates.
Commonly, a commercial measurement tool “Flow Hood”, also known as
“Balometer”, is used to measure the tile air flow rate (Shortridge Instruments Inc. 2015;
TSI Inc. 2015). This tool is used in many applications requiring air flow rate measure-
ment in buildings. The tool air flow resistance is compensated by measuring the tile air
flow rate at two different resistance levels, from which the actual tile air flow rate is
deduced. Note that the tool manufacturers have not shared the operating formulation to
obtain the actual tile air flow rate. Here, we present the possible air flow rate measure-
ment principle. The tool is designed to measure the air flow rate through passive tiles.
However, we also discuss its applicability for active tiles, with integrated fans.
a b Room
Flap open (A0) Am, Vm, Pm
Flowhood Flap close (Ac)
Outer
Hood boundary A
Perforated
Tile
Plenum
Ap, Vp, Pp Open area At
Fig. 2.1 Commercial tile air flow rate measurement tool. (a) Photograph of the tile air flow rate
measurement tool. (b) Schematic of the tile air flow rate measurement tool, photograph is of a pas-
sive tile
Photograph and operation of a Flow Hood are presented in Fig. 2.1. The tile volumetric
air flow rate measurement tool has a probe grid, which calculates the average flow
velocity, based on the difference between the total and static pressures. In the plane
of measurement, it has 16 probes to measure the total pressure in the direction fac-
ing the flow, and an additional 16 in the posterior. Air temperature is measured using
a thermistor, from which the corresponding air density is calculated. A cloth skirt
surrounds the velocity grid, and directs the air flow across it. It may be noted that the
inlet of the cloth skirt has the same dimensions as a standard perforated tile
(0.61 m × 0.61 m, 2 ft × 2 ft), to ensure negligible leakage of air during the measure-
ments. To compensate the flow resistance due to the tool, two air flow measure-
ments are obtained in “Flap Open” and “Flap Close” conditions, using which the
“Actual” air flow rate is obtained, see Fig. 2.1.
The operating principle and the possible formulation to obtain the “Actual” air flow
rate through a passive tile are shown in Fig. 2.2. Here we present the operating prin-
ciple based on the resistance analysis of air flow through restrictions. We have
assumed that the air flow is fully turbulent, which results in pressure loss character-
istics independent of the Reynolds number (Re), and hence the tile air flow rates.
In this case, the pressure loss through a restriction is proportional to the specific
kinetic energy of the air flow. The constant of proportionality is known as the
2.1 Tile Air Flow Rate Measurement 13
pressure loss factor (ΔP = K × 0.5ρV2) (Freid and Idelchik 1989). Miller (1990)
suggested that for Re > 103, the effect of Re is absent on K, which is applicable for
flow through perforated tile in data centers (Re > 104). Note that the Re is based on
the pore diameter and the average flow approach velocity.
In the system for tile air flow rate measurement using Flow Hood, there are
three different resistance levels involved, which result in three different air flow
rates. In “Actual” condition, the resistance is only posed by the perforated tile (Kt).
The corresponding air flow rate is Qt, which is the desired outcome and is finally
displayed, see Equation 1, Fig. 2.2b. In the “Flap Open” condition, the resistance is
from the tile (Kt) and the tool with open flaps (Ko). The flow rate in this condition is
denoted by Qt &o, which can be measured and is displayed first in the Flowhood, see
Equation 2, Fig. 2.2b. For the “Flap Close” condition the resistance of the tool is
further increased (Kc) by closing the flaps, see Fig. 2.2a. This results in further
reduction in the air flow rate to Qt &c, which can also be measured but is not dis-
played in the Flowhood, see Equation 3, Fig. 2.2b.
As the resistances of the tile and the tool are in series, they can be added to get
the total resistance of the flow network. The driving potential for all three cases is
the difference between the plenum (Pp) and the room pressure (Pm). We assume that
Fig. 2.2 Air flow rate measurement analysis for passive tiles using Flow Hood. (a) Representation
of air flow system in three different flow resistances. (b) Formulation of the air flow system in three
different flow resistances (source: Arghode et al. 2015a). Applied for permission from IEEE
14 2 Metrology Tools
this differential pressure does not change with the use of the tool. This assumption
was found to be reasonably valid for the present system (Arghode and Joshi 2014b).
From the flow resistance analysis we have three equations and three unknowns
as listed in Table 2.1. Note that Qt &o and Qt &c are measured directly and Ko and Kc
are properties of the tool, and can be obtained separately. The actual air flow rate
(Qt) can be obtained from the expression given in the inset of Fig. 2.2a. The repre-
sentation of the three systems is shown graphically in Fig. 2.2a, where the driving
potential is constant (Pp − Pm), and the air flow rates for the three cases can be deter-
mined from the intersection with the respective system resistance curves. Based on
the analysis we notice that apart from the actual tile flow rate (Qt), other parameters
such as differential plenum pressure (Pp − Pm) and the tile pressure loss factor (Kt)
can also be determined (Arghode and Joshi 2014b). While these values are not dis-
played in the presently available Flow Hood, they can provide useful information
without any change in the tool hardware. Uncertainty in tile air flow measurement
using Flow Hood is estimated to be ±5 % of the measured value.
Active tiles have integrated fans to regulate and supply higher air flow rate to rela-
tively higher heat load adjacent rack (see Fig. 2.3d). Comparison of typical air flow
characteristics through passive and active tiles is shown in Fig. 2.3a. For illustration
purposes the passive tile corresponds to the active tile without the fans, see Fig. 2.3b,
c. We assume that the pressure profile generated by the fans with respect to the air
flow rate, or the fan curve, is represented by ΔPn. The system characteristic for the
active tile is given by ΔPt,a = ΔPt,s − ΔPn, where ΔPt,s is the system curve or pressure
loss characteristic for the active tile without the fans, essentially a passive tile, see
Fig. 2.3a. Note that, for the passive tile the air flow rate is zero for zero differential
pressure (Pp − Pm). However, this is not the case for an active tile because of the pres-
ence of the fans. In fact, the air flow rate through the active tile for zero differential
pressure corresponds to the intersection of system curve for passive tile (ΔPt,s) and
the fan curve (ΔPn), see Fig. 2.3a.
For a given differential plenum pressure (Pp − Pm), the flow rate for the passive
tile will be given by the intersection of constant differential plenum pressure curve
with the system curve of the passive tile (ΔPt,s), denoted here as Qt,s. With the use of
active tiles, the air flow rate is higher (Qt,a), and corresponds to the intersection of
2.1 Tile Air Flow Rate Measurement 15
Fig. 2.3 Typical air flow characteristics through active tiles. (a) Air flow characteristics
through passive and active tiles. (b) Schematic of a passive tile. (c) Schematic of an active tile.
(d) Photograph of an active tile
constant differential plenum pressure curve and the active tile system curve (ΔPt,a).
However, note that for the differential plenum pressure above a critical value, active
tile may result in lower air flow rate, as compared to the passive tile, shown as the
intersection of system curves for the active (ΔPt,a) and passive tiles (ΔPt,s). Note that
this critical air flow rate corresponds to the air flow rate from the fan curve (ΔPn) for
zero differential pressure across the fan; see Fig. 2.3a. Above this flow rate, the fans
are not able to provide a positive driving pressure and they actually act as resistance
to the air flow through them. This suggests that the active tiles are most useful for
low plenum pressures, where passive tiles do not result in sufficient air flow rates.
The air flow analysis for the three conditions “Actual”, “Flap Open” and “Flap
Close” for active tiles is presented in Fig. 2.4. Similar to passive tiles, due to
increased resistance from the tool, the air flow rate will decrease. Note that for
active tiles, the air flow driving pressure is augmented by the fans. Hence the fan
curve (ΔPn) can be added to the differential plenum pressure (Pp − Pm) in the flow
resistance network. In other words, the fan curve (ΔPn) can be subtracted from the
pressure loss through the active tiles (see Equations 1–3, Fig. 2.4b). As the fan pres-
sure drop (ΔPn) is a function of the flow rate (Q), the formulation used for the pas-
sive tiles to obtain the “Actual” flow rate (Qt) may not be generally valid, as seen in
the expression in the inset of Fig. 2.4a.
However, the formulation can be valid for some specific forms of the fan curve
expression. For example, if the fan curve can be expressed as ΔPn = XQ2 + Z, the
coefficient of the quadratic term (X) can be combined with the coefficient having K
and the constant (Z) can be combined with (Pp − Pm) in Equations 1–3, Fig. 2.4b.
After rearranging the equations, we can arrive at the same expression of Qt derived
previously for passive tiles, see inset of Fig. 2.2a. Note that if the fan curve has non-
zero coefficients for other terms (Qn, n ≠ 0, 2), the formulation used for passive tiles
will lead to errors in the measurement of “Actual” air flow rate (Qt) using active
tiles. With this assumption, the differential plenum pressure (Pp − Pm) and active tile
characteristics constant (Kt − X/(ρ/(2A2))) can also be estimated from the Flow
16 2 Metrology Tools
Fig. 2.4 Air flow measurement analysis for active tiles. (a) Representation of air flow system in
three different flow resistances for active tiles. (b) Formulation of the air flow system in three dif-
ferent flow resistances for active tiles
Hood (Arghode and Joshi 2014b). We assume that this differential pressure does not
change with the use of the tool. Note that for active tiles also, (Pp − Pm) was found to
be nearly unchanged with the use of the tool (Arghode and Joshi 2014b).
The measured air flow rate from an active tile using Flow Hood was compared
with an Anemometric Tool developed at the Data Center laboratory (DCL) at
Georgia Tech. It was found that even with the assumptions discussed earlier, the
Flow Hood worked reasonably well (Arghode et al. 2015a). It may be noted that the
developed Anemometric Tool measures the tile air flow rate based on the air veloc-
ity measurement at multiple points using a grid of thermal anemometers. It does not
require tool resistance compensation, as the back pressure created by the tool was
found negligible.
In data centers, computing servers and other information technology and telecom-
munication equipment housed in racks incorporates built in fans to circulate
cooling air, typically front to back (Joshi and Kumar 2012). It is important to
measure the rack air flow rate for effective design of air management systems for
the data center facility.
2.2 Rack Air Flow Rate Measurement 17
Fig. 2.5 Rack air flow rate measurement tool. (a) Photographs of the tool. (b) Calibration of the tool
A tool was developed at the DCL for measurement of rack air flow rate using an
array of 45 (3 (along width) × 15 (along height)) integrated thermal anemometer and
thermistor units, AccuSense F900 (Degree Controls Inc. 2015). The tool is attached
to the back of the rack for measurement of air flow rate. Cloth skirt around the tool
directs the rack air flow through the sensor array and prevents air leakage, see
Fig. 2.5a. The flow rate was previously calibrated using a calorimetric method, i.e.
using a known heat load and the measured temperature difference across the server
simulator rack to obtain the air flow rate. The uncertainty was estimated to be ±10 %
of the measured value (Arghode and Joshi 2015b), see Fig. 2.5b. This tool was used
to measure the rack air flow rate for both open and contained aisle conditions
(Arghode et al. 2013b, 2015b).
In the presence of an air flow rate measurement tool, the pressure at the back of the
rack is expected to be higher, resulting in lower measured air flow rate, compared to
the actual. The associated error in the measured air flow rate needs to be quantified
and, if required, compensated for. This could be done by measuring the sensitivity
(S) of air flow rate to the differential pressure across a server rack. The tool resis-
tance quantification is presented in this section.
18 2 Metrology Tools
Fig. 2.6 Typical rack air flow characteristics with posterior cover. (a) Typical server rack air flow
characteristics. (b) Schematic of a typical server rack. (c) Rack air flow characteristics with poste-
rior cover. (d) Without posterior cover. (e) With posterior cover (source: Arghode and Joshi 2015b).
Reprinted with permission from ASME
Figure 2.6a shows typical air flow characteristics for a server rack having
integrated fans. A schematic of a server rack is shown in Fig. 2.6b. The purpose of
fan(s) in a rack is to force the cooling air across the rack. Note that a typical rack has
multiple servers and fans. However, if we consider the fans to be operating in paral-
lel, we can represent them as a single fan with a higher air flow rate, corresponding
to the sum of individual fan flow rates (ASHRAE 2012; Bleier 1997). The pressure
differential created by the fan, as a function of air flow rate, is represented by the fan
curve (ΔPn vs. Q) in Fig. 2.6a. Due to restricted area, the rack will pose resistance
to the air flow through it, represented by a pressure loss factor, Kr (Freid and Idelchik
1989). The air flow rate through the rack (Qr) can be obtained from the intersection
of the fan and the system curves, see Fig. 2.6a. Note that the differential pressure
across the server rack for this case is zero (ΔPr = 0, refer to Fig. 2.6b). To obtain the
air flow rate through the rack with fans for different values of ΔPr, we can subtract
the system curve from the fan curve; see Fig. 2.6a (ΔPr vs. Q). Here, our objective
is to measure the slope (S) of the ΔPr curve at zero differential pressure across the
2.2 Rack Air Flow Rate Measurement 19
rack (ΔPr = 0), see Fig. 2.6a. This slope will represent the sensitivity of the air flow
rate to the differential pressure across the rack, and can be used to estimate the tool
resistance error.
To measure the slope (S), we assume that the ΔPr vs. Q curve is locally linear at
ΔPr = 0. Then, we can measure the rack air flow rate (Qr) at ΔPr = 0 condition (see
Fig. 2.6c, d), and rack air flow rate (Qr &v) by using a perforated cover at the back
of the rack (cover pressure loss factor = Kv), see Fig. 2.6c, e. The details of perfo-
rated covers used and experimental conditions are included elsewhere (Arghode
and Joshi 2015b). We can measure the differential pressure at a location between
the rack and the cover (Pv) with respect to the room pressure (Pm), see Fig. 2.6e.
Note that (Pv − Pm) is nothing but the differential pressure across the rack (ΔPr) for
the case with posterior cover. Now, the slope (S) can be calculated from the ratio of
the differences in the rack differential pressures and the air flow rates for the two
cases (see Fig. 2.6c).
The rack under investigation is a standard 42 U (1 U = 4.45 cm, 1.75″) unit popu-
lated with four server simulators, as shown in Fig. 2.7a. Each server simulator has
four fans whose speed can be individually controlled using a common dial setting.
For the present study, no heat load is supplied to the server simulators. For details of
the server simulators used here, refer to (Arghode et al. 2013a; Nelson 2007).
The average fan speed for different dial settings was measured using an optical
tachometer and the results are shown in Fig. 2.7b. Here we have only considered
dial settings from 5 to 10, which correspond to average fan speed of 49–100 %. The
uncertainty in the fan speed measurement is ±2.5 %. The non-uniformity among the
16 fans for the same dial setting for all four server simulators is ±10 %.
The measured differential pressures across the server simulator rack and the
corresponding air flow rates are presented in Fig. 2.8a. The axes are scaled accord-
ing to the fan laws for different fan speeds, or dial settings (Bleier 1997). From the
figure it can be deduced that the air flow rate (Q) and the rack differential pressure
Fig. 2.7 Details of the server simulator rack under investigation. (a) Server simulator rack.
(b) Measured fan speed. Non-uniformity within ±10 % (source: Arghode and Joshi 2015b).
Reprinted with permission from ASME
20 2 Metrology Tools
b 3%
Q/ (Nx/N) (CFM)
a 0 750 1500 2250
30 0.12
(Qr–Qr&I) / Qr
2%
Fv=20%
25 0.1
Dial Setting 1%
DPr/ (Nx/N) (Pa)
20 0.08
5 ΔPr=–26.7Q+14.5
6 ΔPr=–31.0Q+18.5 Fv=40%
15 0.06
0%
7 ΔPr=–35.7Q+24.4
5 6 7 8 9 10
10 8 ΔPr=–37.5Q+30.6 0.04
Fv=100% Dial Setting
9 ΔPr=–44.3Q+43.1
5 0.02
10 ΔPr=–52.9Q+57.9 Qr Qr Qr&I Qr&I
Rack
Tool
Rack
(Pa, m /s)
0 0
0.0 0.4 0.8 1.2
Q/ (Nx/N) (m/s) Actual With Tool
(Qr = –T / S) (Qr&I = Measured)
Fig. 2.8 Measured slope (S) for a range of fan speeds. (a) Rack air flow characteristics,
(ΔPr = S × Q + T). (b) Reduction in air flow rate due to tool resistance (source: Arghode and Joshi
2015b). Reprinted with permission from ASME
(ΔPr) follow the fan laws quite well for different fan speeds. Note that for each case,
the dial settings are same for all four server simulators, resulting in nearly uniform
air flow through all the fans. Note that for the case without the posterior cover
(Fv = 100 %, see Fig. 2.6d), the differential rack pressure (ΔPr) is slightly greater
than zero. This is because of the back pressure created by the tool, which would
result in slightly lower air flow rate, as compared to the actual flow rate without the
tool (ΔPr = 0). As the porosity of the posterior cover decreases (Fv = 40 %, 20 %), the
rack differential pressure (ΔPr) increases, and the corresponding rack air flow rate
decreases. Also note that the data are represented as a linear fit, and the assumption
of locally linear profile for (ΔPr vs. Q) is reasonably valid. From the data, linear
regression fits of the form ΔPr = S × Q + T can be obtained for different fan speeds,
or dial settings, which are included in Fig. 2.8a. From the linear fits the slope can be
calculated (S = dΔPr/dQ).
The effect of tool resistance on air flow rate is shown in Fig. 2.8b. Due to the
presence of the tool, the differential pressure across the rack (ΔPr) is slightly higher
than zero, see data corresponding to Fv = 100 % in Fig. 2.8a. However, for the case
without the tool, ΔPr = 0, and the actual rack air flow rate can be obtained from the
intersection of linear fits for ΔPr vs. Q curves with the X-axis. The difference
between the actual rack air flow rate (Qr), and the rack air flow rate with the tool
(Qr &l) is shown in Fig. 2.8b. See Fig. 2.6d, e for representation of these two cases.
It can be noted that the effect of tool on the rack air flow rate reduction is minimal
(<3 %). For the case where the reduction in air flow rate due to back pressure from
the tool cannot be neglected, the methodology used here, measuring the slope (S),
can also be used to compensate for the effect of tool resistance to obtain the actual
air flow rate.
2.3 Flow Field Measurement 21
PIV measures instantaneous velocity vectors in a selected plane of flow. This tech-
nique has been successfully demonstrated for range of velocities from natural con-
vection to supersonic speeds, and for micro to macro length scales. It may be noted
that the air velocities encountered in a data center are generally less than 10 m/s,
however, the overall length scale of the flow of interest is large (few meters). The
advantage of using PIV technique, as compared to point by point velocity measure-
ment techniques such as laser Doppler velocimetry (LDV) or hot wire anemometry,
is measurement of a spatially resolved flow field at an instant of time. This can save
considerable effort in obtaining large scale flow fields encountered in data centers.
Moreover, gradients such as strain rates and vorticity can also be computed using the
instantaneous flow field. The PIV technique is also non-intrusive, as compared to
probes which can disturb the flow and introduce errors in the measurements. It may
be noted that the PIV technique actually measures the velocities of micron-sized
particles suspended in the flow, and the accuracy of results depends on how closely
the particles follow the flow, as well as how accurately the particle motion is calcu-
lated using image processing. On one hand, the measurements obtained from PIV
can provide deeper insights on the flow-field and heat transfer characteristics. This
can help in devising more effective ways to supply cold air to the servers, as well as
help in detecting the causes of anomalies and failures. On the other hand, the use of
PIV technique for large scale flows such as in data centers requires intricate control
of variables and considerable expertise to obtain good quality measurements.
In PIV the flow is seeded with small particles and illuminated twice with a pulsed
laser light sheet (thickness ~mm), at a preset interval of time (~μs) in the region of
interest. The illumination duration of each pulse is of ~ns duration, so that the
particle motion is practically frozen during the illumination period. The velocity
vectors are derived from sub-sections of the target area of the particle-seeded flow,
by measuring the movement of particles between two laser light illuminations. A
charge coupled device (CCD) camera captures the position of particles at each light
22 2 Metrology Tools
a b
Cylindrical lens
Pulsed Laser Δt
Light Sheet
CCD camera
Flow seeded
with particles
Interrogation
area
Δx, Δy (Particle Displacement)
Velocity vectors
Cross correlation function Dx Dy
u= V=
Dt Dt
Vector map
Noise filtering and random
vector elimination
x y u v
Data Analysis 5 5 2.3 0.8
Velocity Vector
5.2 5.4 0.8 0.32
Stream lines Statistics
6 6.8 0.3 0.1
Vorticity
Turbulence
Fig. 2.9 Velocity field measurement using particle image velocimetry (PIV) set-up. (a) Principle
of PIV. (b) PIV set-up at Georgia Tech
One of the first PIV system for data centers was customized at the Data Center
Laboratory, Georgia Tech and is shown in Fig. 2.9b. The system is able to measure
flow field in a plane of the size of a standard tile width (0.61 m, 2 ft) and a standard
rack height (1.91 m, 6 ft 3 in.). Two cameras are positioned side by side as shown in
the Fig. 2.9b, and the width of combined field of view of both the cameras corre-
sponds to a standard tile size. The cameras and the laser are mounted on a traverse
system that could be moved vertically to cover the height of a standard rack by acquir-
ing velocity field at multiple locations, and finally stitching them together. As the field
2.4 Thermal Field Measurement 23
Fig. 2.10 Thermal field measurement tool. (a) Mobile 3D temperature field measurement tool.
(b) Mobile planar temperature field measurement tool
This type of sensors measure the mass flow rate by measuring the temperature
differential across a heated surface with known power input. The measured mass
flow rate is used to determine the differential pressure across the flow path having
a known flow resistance. The pressure drop in the tube, connecting the sensor and
the measurement location, needs to be compensated and this can be done by using
standard pressure drop correlations for a circular tube for laminar flow. One such
sensor is SDP 1000–L025 (Sensirion 2015) which has measurement range of −62
to +62 Pa (−0.25 to +0.25 in H2O) and size of about 30 mm (1.2″), refer to
Fig. 2.11a. This sensor internally compensates for temperature variations. The
manufacturer specified accuracy is 0.5 % of full scale or 2 % of measured value,
whichever is larger which is well within ±1 Pa for full measurement range.
Measurement of negative differential pressure is applicable in under provisioned
contained cold aisle where the pressure in the cold aisle will be lower than the
room pressure (Arghode et al. 2013b). Sensors such as this has been used in devices
Fig. 2.11 Two types of differential pressure sensors for air cooled data centers. (a) Mass flow rate
based differential pressure sensor (source: www.sensiron.com). (b) Piezoresistive differential
pressure sensor (source: www.allsensors.com)
26 2 Metrology Tools
which can send the differential pressure data in real time to a personal computer
and can be used to control the supplied tile air flow rates to regulate a desired pres-
sure level in a contained cold aisle (AdaptivCool 2015).
Closure
Metrology tools are indispensable aspect air flow management in air cooled data
centers. Some of most important measurements required are temperature, velocity,
pressure and air flow rates (tile and rack). As data centers are relatively large length
scale facilities, specially customized tools are required. In this chapter we discussed
specially designed, movable grid based thermal field measurement tools. Velocity
field measurement is more involved and requires advanced laser based particle
image velocimetry systems. On such unique system was described, which had the
capability to measure finely resolved velocity field covering a large area. The oper-
ating principle of a commercial tile air flow measurement tool was included, and its
applicability for both passive and active tiles was discussed. An in-house developed
rack air flow rate measurement tool was also presented, along with an estimation of
tool resistance related errors in the measurement. Commercial pressure measure-
ment sensors were presented briefly with their salient features. Note that metrology
tools equip a data center designer with deeper insights of the physics governing the
data center air flows, and can help in designing more energy efficient systems.
Chapter 3
Cooling Air Delivery Through Perforated Tiles
For energy efficient operation, uniform air temperature within the guidelines
specified by the American Society of Heating Refrigeration and Air-Conditioning
(ASHRAE TC 9.9 2011) at the rack inlet is desired. However, due to mixing with
the hot room air, higher temperatures as compared to the supplied air temperature
from the floor tiles can be present at the rack inlet, thus lowering the cooling effec-
tiveness (Arghode et al. 2015b). In a raised floor data center, cooling air is supplied
from an underfloor pressurized plenum through perforated tiles. Perforated floor
tiles can have different geometrical features such as size, porosity, pore size and
shape, blocked region at edges where perforations are absent, and anterior structures
such as flow guiding fins or dampers for air flow control that can affect the air flow
delivery to the adjacent rack, and hence the thermal field uniformity.
Until recently, it was believed that the air delivery can be largely addressed if
proper consideration is given to the underfloor region, and the desired amount of air
is supplied through the tile (Patankar 2010). However, in recent experimental inves-
tigations it was observed that even if sufficient amount of air, matching the rack
requirement, is delivered through the tile, a significant amount of cold air by-passes
from the top (Arghode and Joshi 2015a; Kumar and Joshi 2010). This was attributed
to higher momentum of air above the tile, which prevents it from turning and enter-
ing the rack, and thus escaping from the top of the aisle. Hence, to make up the rack
air requirement, room air is entrained, leading to higher inlet air temperature and
thus higher cooling energy consumption. Thus it is important to investigate the flow
field development above a perforated tile and factors influencing the hot air entrain-
ment characteristics. Here we present both experimental and numerical investiga-
tion of this aspect.
Figure 3.1 shows the experimental setup for investigating air flow delivery from a
perforated tile to the adjacent rack. The plenum was 0.91 m (3 ft) deep and the tile
opening was ducted from all four sides using acrylic sheets to allow the air flow to
enter the perforated tile normally. The height of the acrylic sheet was 0.61 m (2 ft),
thus allowing open height of 0.3 m (1 ft) for air to enter the ducted region. An
Aluminum honeycomb structure with opening size and thickness of 2.54 cm (1″),
and width and length of 0.61 m (2 ft) covering the size of the perforated tile was
placed 46 cm (18″) below the tile surface. Honeycomb structure was used to
straighten the flow, improve the uniformity, and remove large scale eddies from the
flow approaching the perforated tile. The opening of the tube connected to a pres-
sure sensor was placed 23 cm (9″) below the tile surface, and along one of the edges
as shown in Fig. 3.2b. The tube opening was oriented to face upwards so as to avoid
the effect of dynamic pressure, and hence static pressure measurements can be
obtained. Further details on the experimental setup could be found in (Arghode and
Joshi 2015a).
It may be noted that in data centers, ducted air flow is generally not implemented
and transverse air flow may also be present. This is even more pronounced for shal-
low plena, where the transverse velocity cannot be neglected, as compared to nor-
mal velocity. However, here we focus on the case where the air flow approaching the
perforated tile is normal, which is characteristic of fairly deep under floor plena.
The plenum pressure was measured using a mass flow rate based differential
pressure sensor, refer to Sect. 2.5.1 for more details. Data were acquired every 5 s
and averaged over a 40 min period for reporting.
Pressure loss factor (Keff) was calculated based on the measured differential
plenum pressure with respect to the room pressure (ΔP), and the average inlet
velocity (Vin) based on the tile air flow rate and total tile area (0.61 m × 0.61 m,
2 ft × 2 ft). The expression is given in (3.1). The uncertainty in calculation of Keff
was estimated to be ±11 %.
3.1 Experimental Investigation 29
DP
K eff = (3.1)
æ1 2ö
ç rVin ÷
è 2 ø
For the present study, air flow emerging from a single tile and entering an adjacent
rack is investigated. The rack adjacent to the perforated tile is a standard 42 U
(height) (1 U = 4.45 cm, 1.75″) unit, populated with four server simulators (APC 2015).
Each has a fan speed dial setting to control the air flow rate passing through it. For
the present study, no heat load is supplied to the server simulators. For details of the
server simulator used here, refer to (Nelson 2007). For the present study, all four
server simulators were set to the same air flow rate. The rack air flow rate was mea-
sured using the tool described in Sect. 2.2.
Perforated steel sheets with circular pores arranged in staggered pattern of holes
on corners of an equilateral triangle are used as tiles, and are shown in Fig. 3.1c.
Volumetric air flow rate through the perforated tile was measured using Flow Hood,
see Sect. 2.1 for details. CRAC settings were varied to achieve desired flow rate
through the perforated tile, refer to (Arghode and Joshi 2015a) for further details.
Note that the region between the bottom of the rack and the floor is blocked to disal-
low air from entering through this gap.
Fig. 3.1 Experimental setup for investigating airflow supply through perforated tiles. (a)
Experimental set up. (b) Plenum. (c) Perforated sheet
30 3 Cooling Air Delivery Through Perforated Tiles
a Top (67.7%) b Top (100.1%) c Top (64.6%) d Top (109.3%) e Top (39.1%)
Height of the rack measured from the performed floor tilt (mm)
1800
1600
1400
1200
Aisle
1000
Rack
800
600
400
200
0
0 200 400 600 0 200 400 600 0 200 400 600 0 200 400 600 0 200 400 600
Tile (K=9.7) Tile (K=38.3) Tile (K=9.2) Tile (K=15.0) Tile (K=16.1)
Distance measured from the centre of cold aisle (mm)
Fig. 3.2 Effect of tile geometry on air delivery from a perforated tile. (a) Base case. (b) Lower
porosity. (c) Smaller pore size. (d) Larger edge blockage. (e) Reduced tile width (source: Arghode
and Joshi 2015a). Reprinted with permission from ASME
Five cases were investigated to highlight the effect of tile porosity (Feff), pore size,
edge blockage, tile width. The base case had air flow rate of 0.56 m3/s (1177 CFM)
corresponding to rack heat load of 10 kW with air temperature rise of 15 °C, even
though no heat load is imposed on the racks in the present study. For the base case,
the tile sheet porosity is 40 %. Due to edge blockage of 12.7 mm (0.5″), because of
the support structure to place the tile, the effective porosity is reduced to 36.7 %. For
this case, the pore size is 6.35 mm (1/4″), see Fig. 3.2a. From the base case, a single
parameter is varied to investigate its effect. To investigate the effect of tile porosity,
a case with lower porosity of 23 % (effectively 21.1 %) in included, see Fig. 3.2b.
Another case with smaller perforated sheet pore size of 3.18 mm (1/8″) is investi-
gated, see Fig. 3.2c. The effect of edge blockage is examined by blocking a region
of 38.1 mm (1.5″) along the edges, see Fig. 3.2d. Lastly, the tile width is reduced to
0.46 m (1.5 ft), see Fig. 3.2e. The ducted plenum width, not shown, is also reduced
accordingly. Reynolds number (ReD) is calculated based on the pore diameter (D)
and average pore velocity (Vin/Feff) as shown in (3.2).
æV ö
r ç in ÷÷ D
çF
ReD = è ø
eff
(3.2)
m
3.1 Experimental Investigation 31
Stream traces corresponding to Feff = 36.7 % and Feff = 21.1 % shows that even though
the tile air flow rate is same for both the cases, the flow field above the tile surface
is significantly different, see Fig. 3.2a, b. For case with lower tile porosity, higher
velocity of air above the tile surface is observed, as compared to the base case. For
both cases, significant amount of air by-pass is observed from the top, even though
the flow rate of air emerging from the tile is matched with the air entering the rack.
This trend was also observed earlier (Kumar and Joshi 2010). With decrease in
porosity, significantly higher air flow rate is observed from the top (100.1 % vs.
67.7 %). This may be attributed to higher momentum of air above the tile surface for
lower porosity case due to higher acceleration of air as it passes through a more
restricted area. Also for the lower porosity case, higher air entrainment from the
aisle side is observed, which is absent for the base case, see the stream traces near
aisle side. From the figure, we also note that the pressure loss factor (Keff) is signifi-
cantly higher for lower effective tile porosity. With higher Keff, higher excess
momentum of air is expected above the tile surface, which can lead to higher cold
air by-pass from the top (Abdelmaksoud et al. 2010; Arghode et al. 2013a), as also
observed here.
The results suggest that due to higher momentum of air above the tile surface for
lower porosity, higher cold air by-pass from the top, and reduced efficacy of air to
bend and enter the adjacent server rack may be encountered. On the other hand, for
an under-provisioned case, where the supplied air flow rate is lower than the rack air
requirement, tile with lower porosity can allow air to reach near the top of the rack
and possibly improve the cold air delivery effectiveness.
The characteristics of two pore sizes of 6.35 mm (1/4″) and 3.18 mm (1/8″) are
shown in Fig. 3.2a, c. For the smaller pore size 3.18 mm (1/8″), air is not observed
leaving the measurement plane from the aisle side, as seen with larger pore size
6.35 mm (1/4″). This resulted in air entrainment from the aisle, as compared to air
exiting from the aisle side, as observed from stream traces. With smaller pore size,
the number of pores is larger, and pore to pore center spacing is smaller, resulting in
a greater number of closely spaced jets. Smaller pore size will also result in faster
decay of the jets, as the jet decay length varies linearly with pore diameter (Arghode
and Joshi 2013), thus reducing the region of influence of entrainment and flow
development above the tile surface. This may be the reason for differences in flow
profiles near the aisle for the two cases. Minimal influence on pressure loss factor
(Keff) for the two pore sizes was observed.
It is possible that the influence of pore size on flow at the top may not be evident
from these measurements, as the two pore sizes are very small as compared to tile
size. CFD investigation presented in the Sect. 3.2 provides more insight on this aspect.
32 3 Cooling Air Delivery Through Perforated Tiles
Generally, data centers have 1.22 m (4 ft) wide cold aisles, which accommodate two
standard 0.61 m (2 ft) tiles placed adjacently. However, the aisle width can be
reduced to 0.91 m (3 ft) by using only one 0.91 m (3 ft) wide and 0.61 m, 2 ft long
perforated tile to save floor space. Here a 1.22 m (3 ft) cold aisle is simulated by
reducing the width of the tile from 0.61 m (2 ft) to 0.46 m (1.5 ft), and the stream
traces for this case are shown in Fig. 3.2e. It is seen that most of the air exiting from
the perforated tile enters the adjacent rack, which results in considerably lower
(39.1 % vs. 67.7 %) air by-passing from the top. Note that in this case, the effective
tile porosity is lower than the porosity corresponding to the base case (28.0 % vs.
36.7 %). Even then, there is a reduction in cold air by-pass from the top. This sug-
gests significantly improved air delivery, as compared to the base case with standard
0.61 m (2 ft) wide tile, further emphasizing the importance of appropriate tile geom-
etry to achieve desired air flow above its surface.
Note that reduction in aisle width may not be practically feasible in some cases,
and reducing the tile width by blocking a portion of the standard tile, as done here,
can be used to improve air delivery.
3.2 Computational Fluid Dynamics (CFD) Modeling 33
Air flow modeling is extensively used in designing and thermally managing data
centers. From the previous section, it can be recognized that the tile’s geometrical
features have significant effect on the downstream flow characteristics. Hence for
improving predictive capability of the numerical models to simulate air flow in data
centers, it is important to incorporate these effects. Including all the geometrical
features of the tile can be computationally prohibitive, see Fig. 3.3, where to resolve
key flow features for a single tile about two million cells are needed (Arghode et al.
2013a). Hence, rapid turnaround models are desired to simulate air flow through tiles
in data centers. While tile porosity has been taken into consideration for estimating
the air flow rate supplied through the tile, only a few studies consider the effect of
other geometrical parameters on the air flow development downstream of the tile.
The rapid tile air flow models reported in literature, which obviate the resolution
of tile geometrical structure, are discussed below and summarized in Fig. 3.4.
The PJ model specifies a step pressure loss across the tile surface. The pressure loss
can be computed from (3.3), where, the pressure loss factor (K) can be calculated
based on the tile porosity (F) [see (3.4)] (Freid and Idelchik 1989).
1
DP = K rU in2 (3.3)
2
Fig. 3.3 Geometrical resolution of perforated tile (source: Arghode et al. 2013a). Reprinted with
permission from ASME
34 3 Cooling Air Delivery Through Perforated Tiles
a b e
Velocity Profile
H W
Low Pressure Low Pressure
Uniform Velocity W
Sx Pressure profile 4”
D P
Porosity (F) ∆P, K ∆P, K
X X
L B L-2B
K Keff D
P
c d Pr f
Pr ∆P • Aligned square pores
• Turbulent free jets [ref D]
∆P Pb Ub • Constant jet spreading (S)
• Origin at jet entry
• S = R / X ~ 0.11
Pin Uin
• D / P = F0.5
Pin Uin • H ~ J ~ P for constant S
K=(Pin-Pr)/(0.5ρUin2)
• H ~ D / F0.5
K*=∆P/(0.5ρUin2) K=(Pb-Pr)/(0.5ρUb2)
K*=∆P/(0.5ρUb2)
Keff=(Pin-Pr)/(0.5ρUin2)
Fig. 3.4 Representation of rapid models for air flow through perforated tiles. (a) Porous jump (PJ)
model. (b) Body force (BF) model. (c) Modified body force (MBF) model. (d) MBF model with
blocked edges. (e) MBF region. (f) MBF region height (H) estimation (source: Arghode et al.
2014a). Applied for permission from IEEE
2
æ 1
ö
1
K= 2 ç æ 1 - F ö 2 + (1 - F ) ÷ (3.4)
F ç çè 2 ÷ø ÷
è ø
It may be noted that the use of this model results in no alteration in the velocity
field across the tile surface and only a step pressure loss is specified across the tile,
see Fig. 3.4a. However, in reality, as air flows through a perforated tile, there is
acceleration of air through the pores, resulting in increase in the air momentum flow
rate. This higher momentum flow rate of air above the tile surface was attempted to
be captured by the body force (BF) model, described next.
3.2 Computational Fluid Dynamics (CFD) Modeling 35
U in
U pore = (3.6)
F
DM 1
= -1 (3.7)
M in F
M
Sx = (3.9)
"
" = W ´W ´ H (3.10)
In this model the height of the momentum source region (H) was taken as 10.2 cm
(4″) and the width and length correspond to the tile dimensions (0.61 m × 0.61 m
(2 ft × 2 ft)). Due to specification of momentum source above the tile, the down-
stream velocity field is affected. Step pressure loss across the tile can be specified
similar to the PJ model. Refer to Fig. 3.4b for the representation of the model.
The BF model, described before, uses average velocity at the tile pores, while
calculating the excess X-momentum flow rate of air above the tile surface.
However, even higher X-momentum flow rate of the air is expected downstream,
due to further necking of the air jet (vena contracta) downstream of the pores. The
authors suggested a modification to the body force model to capture this effect
(Arghode and Joshi 2014a). In the MBF model the X-momentum rise across the
36 3 Cooling Air Delivery Through Perforated Tiles
tile surface is calculated based on the velocity at the vena contracta, Uneck, instead
of the average pore velocity, Upore, used in (3.6). The velocity at the vena con-
tracta, Uneck, is estimated using Bernoulli’s equation and the pressure loss across
the pores, see (3.11) and (3.12).
1 1 1
DP = K rU in2 = rU neck
2
- rU in2 (3.11)
2 2 2
1
U neck = U in ( K + 1) 2 (3.12)
Note that the MBF model will have higher magnitude of excess X-momentum
flow rate (ΔM) as compared to the BF model, because the velocity at the vena
contracta is expected to be higher than the average velocity at the pores. The expres-
sion for excess X-momentum scaled with the inlet X-momentum for this model is
given in (3.13). Note that the scaled excess X-momentum is only a function of the
pressure loss factor.
DM 1
= ( K + 1) 2 - 1 (3.13)
M in
In the BF model the height of the momentum source region (H) was chosen to be
4″ (Abdelmaksoud et al. 2010) without providing any physical basis for this selec-
tion. The width (W) corresponded to the tile size (L). In the MBF model, the height
of the momentum source region (H) was calculated based on the geometrical param-
eters of the tile and turbulent jet characteristics (Arghode and Joshi 2014a). It was
proposed that the H varies with the distance of meeting of adjacent jets (J), while
considering fully developed, turbulent free jets, with origin at the jet entrance and
constant jet spreading rate (S) (Lipari and Stansby 1990). Hence, it was assumed
that the adjacent jet meeting distance (J) will vary linearly with the pore pitch (P).
For aligned square pores, the pitch increases linearly with the pore size (D) and var-
ies inversely with the square root of the porosity (F−0.5). Hence, the formulation for
MBF region height (H) can be expressed as given in (3.14). The constant Cref was
evaluated in (Arghode and Joshi 2014a).
0.5
æ Fref ö
H = Cref D ç ÷ (3.14)
è F ø
Fref = 25%
Cref = 4
The width of the momentum source region (W) was taken as the width of the
porous region, see (3.15). This excludes the blocked region at the tile edges. Note
that the tile is considered to be square and blocked edge strips are present all around
the tile and are of constant thickness (B).
3.2 Computational Fluid Dynamics (CFD) Modeling 37
W = L - 2B (3.15)
In the MBF model, a step pressure loss across the tile is specified. However, due
to presence of momentum source above the tile, the pressure immediately above the
tile will be lower than the room pressure (Pr). Hence, pressure loss specification
across the tile, based on pressure loss factor K = ( Pin - Pr ) / ( 0.5 r Vin ) , will result in
2
lower pressure immediately upstream of the tile which, in turn, will result in lower
inlet pressure (Pin). Hence, higher value of the pressure loss factor (K*) across the
tile is required, also refer to Fig. 3.4c, d. The pressure loss factor ratio, γ = K*/K, is
assumed to be dependent on the MBF region height to width ratio (H/W), the tile
porosity (F) and the edge blockage (B), and is modeled as shown in (3.16). The
constants a, b, n and m were evaluated in (Arghode and Joshi 2014a)
n m
K* æ æ H ö ö æ Fref ö æ L - 2 B ö
g = = çaç ÷ + b÷ç ÷ ç ÷ (3.16)
K è èW ø øè F ø è L ø
a = -0.351, b = 1.214
H
1 / 12 < < 1/ 3
W
n = -0.138
Fref
0.563 < < 2.25
F
m = 0.287
( L - 2B )
> 0.75
L
From the formulation of MBF model we can note that it incorporates the effect
of porosity (via K, H and γ), pore size (via. H), and edge blockage via resolution of
blocked edges and γ. These parameters were found to influence the downstream
flow field, as observed from experimental investigation discussed in Sect. 3.1. Note
that the PJ and BF models will only account for effect of porosity. While the former
will influence the pressure field but not the flow field (via. K), the latter will also
affect the flow field from excess momentum imposition above the tile.
The empirical constants Cref and a, b, n and m are evaluated based on comparison
(calibration) with the geometrical resolution (GR) model. Square pores with aligned
arrangement (pore centers at the corners of a square) are considered for the GR
model. a, b and Cref are evaluated by varying the height H of the MBF region and
comparing the results with the GR model for a particular case (referred here as base
case), porosity (F = 25 %) pore size (D = 2.54 cm, 1″) and edge blockage (B = 0).
38 3 Cooling Air Delivery Through Perforated Tiles
Three porosities (F = 11 %, 25 % and 44 %), and three edge blockages (B = 0,
3.81 cm, 1.5″ and 7.62 cm, 3″) are investigated to evaluate the constants n and m
respectively. For all the cases, square tiles with size (L = 0.61 m, 2 ft) are considered,
and the Reynolds number (ReD = (ρUporeD)/μ) is above 5000, suggesting fully turbu-
lent flow (ReD critical = 1000 (Miller 1990)).
The computational domain is shown in Fig. 3.5a. Flow through the tile with aligned
square pores is simulated using the geometrical resolution (GR) model. The geo-
metrically resolved porous structure is absent for the MBF model. The velocity
magnitude contours in the diagonal plane are presented. The domain extends from
X = −30.5 cm (−1 ft) to +198.1 cm (+6 ft 6 in.), where the tile is positioned at X = 0,
and X is along the normal to the tile. Thus, we consider air flow from plenum, reach-
ing upto the height of a typical data center rack. Pressure outlet boundary condition
is used at the top. The plenum sides are considered as wall, and its inlet is prescribed
with mass flow inlet boundary condition. The computational domain is extended by
15.2 cm (6″) laterally on all four sides of the tile to minimize the influence of the
finite domain size on the flow field. Pressure inlet boundary condition is used at the
side boundaries, see Fig. 3.5a. The base mesh size was 0.64 cm × 0.64 cm × 0.64 cm
(0.25″ × 0.25″ × 0.25″), and for the GR model, the mesh was refined in the region of
X = −3.8 cm (−1.5″) to +15.2 cm (+6″) to achieve eight cells per pore (mesh size =
0.32 cm × 0.32 cm × 0.32 cm (0.125″ × 0.125″ × 0.125″), pore size (D) = 2.54 cm
(1″)), following the guidelines from (Arghode and Joshi 2013). The total cell count
Fig. 3.5 Comparison of modified body force (MBF) with the geometrical resolution (GR) model. (a)
Computational domain. (b) Base case. (c) Smaller pore size. (d) Higher porosity. (e) Blocked edge
(source: Arghode et al. 2014a). Applied for permission from IEEE
3.2 Computational Fluid Dynamics (CFD) Modeling 39
for the GR model was 2.3 million. For the MBF model, no refinement was required
and cell count was 1.8 million. Note that as the pore size decreases, the cell count
will significantly increase for the GR model due to further requirement of mesh
refinement, discussed later. Here, only a quarter geometry is simulated due to sym-
metry. Standard k − ε model is used for turbulence modeling.
Figure 3.5b shows the velocity magnitude contours for both GR and MBF models
for base case (D = 1″, F = 25 %, B = 0″). From the figure it can be observed that for
the GR model the jets emerging from the pores decay and merge with the adjacent
jets. Downstream, the individual jets are not distinct, and the flow develops as a
single large jet. The air flow rate at the top plane at height of 1.98 m, 6 ft 6″, and
same dimension as the tile, (Qtop), scaled with the inlet air flow rate (Q) is also
included in the figure. For the GR model, Qtop is 148.9 %, suggesting 48.9 % of air
entrainment from the sides. The flow field predicted by the MBF model matches
well with that from the GR model, with very close estimation of Qtop (148.8 %).
Note that individual jets are not present in the MBF model, as the pore structure is
not resolved in order to save computational effort.
Pore sizes, D, of 1.27 cm (0.5″), and 2.54 cm (1″) are considered, where the latter
corresponds to the base case discussed before. All other inputs are the same for both
cases. Note that increase in pore size will result in reduction in the number of pores,
which will affect the overall behavior of the jet array. For D = 1.27 cm (0.5″), another
level of mesh refinement as compared to base case, was required, which resulted in
total cell count of 4.2 million. In the second level, the mesh was refined in the region
of −1.91 cm (−0.75″) and +7.62 cm (+3″). This shows that as the pore size decreases,
the total cell count increases significantly.
The results for the effect of D are presented in Fig. 3.5b, c. From GR model it can
be noted that for larger D the pressure loss factor (K) is slightly higher, and the air
flow from the top (Qtop) increases significantly. This may be because for larger D,
the number of pores (N) is lower, and hence, a larger fraction of jets are at the
periphery, which behave closer to a free jet and entrain higher amount of air. This
suggests that for almost similar pressure loss across the tile, different Qtop, as well as
entrainment rates, can be achieved with variation in D. This trend was successfully
captured by the MBF model, and very close prediction of Qtop for smaller pore size
was obtained (126.8 % vs. 126.9 %).
Furthermore, for very small pore sizes, the downstream excess X-momentum
flow rate will be negligible and the flow field may be similar to that obtained from
40 3 Cooling Air Delivery Through Perforated Tiles
the PJ model. This may be the reason for the minimal difference observed in the
flow field for tile pore size of 0.64 cm (0.25″) and 0.32 cm (0.125″) in the experi-
mental investigation discussed in Sect. 3.1. Smaller pore sizes and difference in
computational effort between GR and MBF model will be discussed later.
Lower tile porosity (F) will result in greater acceleration of air flow through the
pores and hence higher pressure loss (higher K) across the tile. Greater acceleration
of air flow is expected to result in higher ΔMx above the tile surface and higher Qtop.
This was also discussed in Sect. 3.1, based on experimental investigation. Lower F
will also result in larger pore pitch (P), and longer adjacent jet meeting distance (J).
The results for the GR and MBF models for the effect of tile porosity are presented
in Fig. 3.5b, d. Note that for the GR model, only F is varied, keeping D and B same
as the base case. From the GR model, we observe that K decreases significantly
(33.7–7.6) for higher F (25–44 %), and Qtop decreases to 108.8 % for the case with
higher porosity (F = 44 %), as compared to the base case (F = 25 %). The flow field
obtained by the MBF model reasonably closely resembled that obtained from the
GR model. The value of air flow at the top was also closely predicted (Qtop = 111.2 %).
Tiles used in data centers do not have perforations extending till the edges, and a
solid strip is present all around the periphery, blocking air flow near the edges. In
Fig. 3.5e the effect of edge blockage (B) is investigated for B = 3.81 cm (1.5″). The
case without edge blockage (B = 0) has porosity F = 44 % (Fig. 3.5d). Note that with
edge blockage the effective porosity (Feff) will decrease and for the present investi-
gation Feff = 34 % for B = 3.81 cm (1.5″). The pore size (D) is same as the base case.
From the GR model, it can be noted that Qtop increases significantly with B. For
B = 3.81 cm (1.5″) (Feff = 34 %), Qtop is 137.1 %, see Fig. 3.5e. This is significantly
higher than the case without edge blockage (Qtop = 108.8 %). Higher Qtop could be
partly attributed to higher effective pressure loss factors (Keff) for the case with edge
blockage (15.3 vs. 7.6). However, it was noted previously that the edge blockage
results in higher Qtop even for the cases with similar effective pressure loss factors
(Keff). This was attributed to the array of jets behaving closer to free jet for the case
with edge blockage (Arghode and Joshi 2014a).
The MBF model is able to capture the effect of edge blockage quite well and is
able predict Qtop close to that obtained from GR model (135.6 % vs. 137.1 %). Note
that for the MBF model, the excess X-momentum flow rate ΔM, used to calculate
the momentum source in the MBF model, see (3.13), is calculated based on the
pressure loss factor (K) for the porous region only. Hence for the present case, K is
3.2 Computational Fluid Dynamics (CFD) Modeling 41
7.6 for both with and without edge blockage. This is because for the MBF model,
the edge blockage is geometrically resolved, whereas the perforations are modeled.
The width of the MBF region (W) corresponds to the porous region of the tile, see
(3.15). Here, F = 44 % (porosity of the porous region only) and Fref = 25 %, and
hence H is same for both the cases.
3.2.9 S
mall Pore Sizes and a Note on Computational Effort
Using MBF Model
Small pore sizes (D) = 0.32 cm (0.125″) and 0.64 cm (0.25″) are investigated here,
with all other parameters being same as that for the base case. Note that for the
base case, the pore size (D) = 2.54 cm (1″). For these small pore sizes, the compu-
tational domain used previously (see Fig. 3.5a) cannot be used due to the prohibi-
tively high cell count. Hence, a smaller domain is used for these simulations, as
shown in Fig. 3.6a. The base mesh size for this domain is 0.16 cm × 0.16 cm × (along
height) 0.32 cm (0.0625″ × 0.0625″ × (along height) 0.125″). For D = 0.64 cm
(0.25″) the region from height of −0.32 cm (−0.125″) to +0.95 cm (+0.375″) (base
area corresponding to tile dimensions) is refined and the total cell count was 2.3
million. For D = 0.32 cm (0.125″), another level of refinement from −0.16 cm
(−0.0625″) to + 0.48 cm (+0.1875″) was performed, and the total cell count for
this case was 6.4 million. For both cases, there were eight cells per pore (Arghode
and Joshi 2013). This suggests that with decrease in pore size, the cell count
increases significantly.
For the MBF model, it was found that at least four cells along the height (H)
were required to reach nearly grid independent results. Here H = 4D [see (3.14)],
and for D = 0.64 cm (0.25″) and 0.32 cm (0.125″) the number of cells along H were
eight and four respectively. The total cell count was 1.2 million, which is signifi-
cantly lower as compared to the GR model. As per the requirement of four cells
along H, and as H ~ 4D, the smallest mesh size for the MBF model will be ~D,
whereas, as per the requirement of eight cells per pore for the GR model (Arghode
and Joshi 2013) the smallest mesh size will be ~D/8. Considering same mesh size
along all the three directions, the cell count ratio for the GR and MBF model will
be ~83 (=512). Note that this pertains to a particular region in the vicinity of the tile
where the mesh refinement is imposed and not in the full computational domain.
Hence, this suggests that with the MBF model significant reduction in computa-
tional effort can be achieved.
The results are presented in Fig. 3.6. From the GR model it can be noted that with
increase in D from 0.32 cm (0.125″) to 0.64 cm (0.25″), Qtop increases only slightly,
see Fig. 3.6b, d. This suggests that due to large number of jets for small pore sizes
examined here, the air flow profile is closer to a symmetric array of jets, with mini-
mal downstream excess X-momentum. Hence, for very small pore sizes, negligible
air entrainment is expected even though the pressure loss across the tile does not
reduce noticeably. Moreover, the results also suggest that the influence of pore size
42 3 Cooling Air Delivery Through Perforated Tiles
a
Pressure
b 107.6% c 109.4%
Symmetry
Inlet
Tile Top
8×8 K=30.3 γ=1.254
Diagonal Plane
cells/pore
D=0.125”, GR model D=0.125”, MBF model
Wall d 111.6% e 114.1%
Tile Size = 2 ft × 2 ft
Domain length = −0.75” to +3” K=30.9 γ=1.220
Domain width = 2 ft 3”
D=0.25”, GR model D=0.25”, MBF model
Computational domain
Fig. 3.6 Comparison of contours of geometrical resolution (GR) and modified body force (MBF)
models for small pore sizes. (a) Computational domain. (b) D = 0.125″, GR model. (c) D = 0.125″,
MBF model. (d) D = 0.25″, GR model. (e) D = 0.25″, MBF model (source: Arghode et al. 2015a).
Applied for permission from IEEE
will be insensitive to very small pore sizes (D ≤ 0.64 cm (0.25″)), whereas for larger
pore sizes the effect is noticeable (D ≥ 1.27 cm (0.5″)), see Sect. 3.2.6. This may be
the reason for minimal effect on the air by-pass from the top in previous experimen-
tal investigations for small pore sizes (D ≤ 0.64 cm (0.25″)), see Sect. 3.1.
The results from the MBF model are also presented in Fig. 3.6c, e. It may be
noted that the MBF model agrees well with the GR model for small pore sizes also,
and shows that it works well for a wide range of pore sizes.
Closure
Air flow development above a perforated tile and its entry into an adjacent rack
were investigated experimentally. With decrease in effective tile porosity, signifi-
cantly higher air by-pass from the top is observed, possibly due to higher momen-
tum of air above the tile surface for the lower porosity case. Reduction in pore size
from 6.35 mm (1/4″) to 3.18 mm (1/8″) had a weak effect near the tile surface at
both rack and aisle ends. This is possibly due to smaller flow development region
for smaller pore size. The results also suggest that influence of pore size cannot be
neglected and should be included in numerical models for improved predictive
capability. Larger edge blockage resulted in significantly higher air by-pass from
the top. The air by-pass was even higher than the case with lower effective tile
porosity, even though the effective porosity for the case with edge blockage was
higher. Reduction in tile width resulted in significantly improved air delivery with
considerably lower air by-pass from the top, even with lower effective porosity.
These cases show the importance of appropriately considering the tile geometry to
achieve desirable result.
Geometrical resolution (GR) model is used to investigate the influence of pore
size (D), porosity (F), and edge blockage (B) on the flow field development down-
stream of a perforated plate. It was observed that for larger D, as the number of
pores is lower, the jet array behaves closer to a free jet, and the air flow rate at the
3.2 Computational Fluid Dynamics (CFD) Modeling 43
top (Qtop) is higher. For smaller F, due to higher acceleration of air through the pores,
smaller number of pores and larger pore spacing, higher Qtop is observed. With
larger B, significantly higher Qtop was observed, possibly because the flow field
behaves as a combination of a large free jet (from the open region) and jet array
(from the porous region).
The MBF model was able to successfully capture the flow field trends and agreed
well with the GR model. MBF model shows promise to significantly reduce the
computational effort, while maintaining a reasonably good prediction capability.
Chapter 4
Cold Aisle Containment
One of the objectives of an effective air flow management scheme is to minimize hot
and cold air mixing. Appropriate arrangement of racks, such as hot-aisle-cold-aisle
(HACA) partially meets this requirement. However, due to the tendency of entrain-
ment of surrounding room air by emerging cold air stream from perforated tiles, hot
and cold air mixing is very difficult to eliminate in open aisle condition. Use of
physical barriers, separating the hot and cold regions shows promise to minimize
this problem. In this chapter we will discuss the thermal characteristics of contained
cold aisles, and their influence on the energy savings of a data center.
Physical separation of hot and cold regions in a data center can be achieved by either
containing the cold aisles or containing the hot aisles. Overall, the major benefit of aisle
containment is maintenance of server air inlet temperatures, due to minimum mixing
with the hot air. In the absence of high air temperature peaks at the server inlet, the supply
air temperature from the computer room air conditioning (CRAC) units can be set to a
higher value, while meeting the server inlet temperature recommendations (ASHRAE
TC 9.9 2011). This will in turn allow higher supply chilled water temperature, leading
to lower power consumption at the chiller plant. The other way to suppress high air
temperature peaks at server inlets is to over supply the cold air through the tiles, flood-
ing the cold aisle. However, this will result in higher energy spending on the movement
of air by CRAC fans, as well as, losses due to direct cold air bypass. Hence a balance
is sought to minimize cooling energy consumption. This was also discussed briefly
based on a thermodynamic model in Chap. 1.
Here we focus on the thermal characteristics of contained cold aisle in a raised
floor data center. Containing, or physically separating, the cold aisle poses
restrictions on the air flow, as in this case the CRAC fans and the server/rack fans
are arranged in series. Under ideal conditions, the total air supplied from the CRAC
fans would match the total air drawn from the server fans, and there will be no air
leakage either in, or from the cold aisle. However, depending on the containment
leak-tightness, the air flow rate supplied from the CRAC fans can be higher than the
air flow rate through the rack (over-provisioned case), or the rack air flow rate can
exceed the supplied air flow rate (under-provisioned case). The balance would then
be made up from the leakage from, or to the cold aisle. Hence, deployment and
operation of cold aisle containment requires consideration of these important fac-
tors, and investigation of thermal characteristics can aid in effective implementation
of such air delivery schemes. Note that active fan tiles could be used to supply
higher amount of air in an under-provisioned scenario to meet the rack air require-
ment, though at an expense of power supplied to the tile fans.
In previous computational investigations it was observed that the deployment of
cold aisle containment can result in significant energy savings (Schmidt et al. 2011;
Shrivastava et al. 2012). In an experimental investigation, cold aisle containment
was observed to result in lower fan power (15 % energy savings) (Takahashi et al.
2008). Moreover, in case of cooling failure, cold aisle containment was found to
result in reduced hot spot generation with time, as compared to the case without
containment (Takahashi et al. 2008). Partial containment has also been investigated
computationally (Gondipalli et al. 2008, 2009). In this case, undersupply of air from
perforated tiles, as compared to the demand of air from racks was employed (under-
provisioned case). Various designs with openings in the containment system and
meshed partitions were investigated to balance the flow rates and achieve improved
performance. A solid door with slit on the roof was suggested as one of the optimal
choices (Gondipalli et al. 2009).
Here, the thermal characteristics of open and contained aisles are examined
using thermal field measurements for both passive and active tiles. Open aisle con-
dition is examined to understand the hot air entrainment and mixing behaviour in
the cold aisle.
The layout of the DCL is shown in Fig. 4.1. The length of data center room space is
8.84 m (29 ft), width 6.25 m (20.5 ft) and height 2.64 m (8 ft 8 in.). The floor area
of the data center room is 56 m2 (600 ft2.). The data center has three CRAC units,
two down-flow and one up-flow, and two power distribution units (PDUs). In the
present investigation only the down flow units 1 and 2 are used, and CRAC 3 is
turned off and blocked to disallow air flow through it. Identification of tiles, racks
and outlet vents can also be seen in Fig. 4.1. The data center houses different types
of servers/equipment in standard 42 U (1 U = 4.45 cm) racks, with overall dimen-
sions 61 cm (width) × 122 cm (depth) × 198 cm (height) (24 in. × 48 in. × 78 in.). The
hot air return vents (size is 0.61 m × 0.61 m (2 ft × 2 ft)) are placed above the hot
aisles. The data center has a fairly deep plenum (0.9 m, 3 ft), which results in
4.2 Experimental Setup 47
uniformly pressurized underfloor region, and helps in achieving nearly uniform tile
air flow rates. For the present investigation, the data center was operated with rela-
tively low heat load, as well as server fan speed variation (<2 %), resulting in nearly
constant air flow rates through the racks.
The data center has a single cold aisle comprised of 14 generic 0.61 m × 0.61 m
(2 ft × 2 ft) perforated tiles arranged in two rows. Figure 4.2a, b shows two contain-
ment conditions: open aisle, and closed (contained) cold aisle. A total of 14 racks
are arranged on either side of the cold aisle. In fully contained, or closed aisle, con-
dition both the curtain on the top of the cold aisle, and doors at the cold aisle entrance
are closed. The internal view of the containment system is shown in Fig. 4.2c.
For representation, all the rack inlets and outlets are divided into six equal seg-
ments and numbered such that inlet/outlet 1 is at the bottom and inlet/outlet 6 is at the
top. For the networking rack (Rack 1), the inlet is blocked from the top region (seg-
ments 3–6), and cold air enters only from the bottom region (segments 1–2). The
opening for the hot air outlet for Rack 1 is near the top of the rack exit (segments 4 and
5). Racks 2–7 and Racks 10–12 house six IBM blade center servers equipped with
chilled water rear door exchangers to partially extract the heat from the rack exit air.
The rear door heat exchangers have a separate pure chilled water loop which exchanges
heat with the chilled water loop connected to the building chiller plant. Rack 8 is
empty and is completely blocked. Rack 9 is populated with standard 1 U (4.45 cm)
servers, and does not have a rear door heat exchanger. Rack 13 is similar to Racks 2–7
and Racks 10–12, except that only four IBM blade centers are operational (segments
3–6 are active). The two blade centers near the b ottom are turned off (segments 1–2
are in active). Rack 14 is a storage rack with a rear door heat exchanger. For Rack 14,
only segments 1–5 are active, and segment 6 is blocked using blanking panels.
Fig. 4.2 Cold aisle containment at Data Center Laboratory, Georgia Tech. (a) Open cold aisle. (b)
Contained cold aisle. (c) Inside view of a contained cold aisle
Both active and passive tiles are used in this investigation. Refer to Fig. 4.3 for
the photographs of the tiles. Passive tiles have anterior dampers, which for the pres-
ent case were fully open. Active tiles had integrated fans running at 100 % speed.
4.3.1 T
ile/Rack Air Flow Rate, Plenum Pressure
for Passive Tiles
Tile air flow rates were measured using the Flow Hood, see Sect. 2.1, and the rack
air flow rates were measured using the tool described in Sect. 2.2. For an ideal con-
tainment, without air leakage, the total tile flow rate and total rack flow rate will be
same. In the open aisle condition, if the total tile flow rate is less than the total rack
4.3 Experimental Results 49
Fig. 4.3 Photographs of (a, b) passive and (c, d) active tile. (a) Top view of passive tile. (b)
Bottom view of passive tile. (c) Top view of active tile. (d) Bottom view of active tile
air flow rate (under-provisioned case), the containment would tend to increase the
tile flow rate, and decrease the rack air flow rate in order to equalize the tile and rack
flow rates. The vice-versa will also hold for the case where the total tile flow rates
are higher than the total rack flow rates (over-provisioned case).
The overall effect of the containment on tile/rack air flow rates is shown in
Fig. 4.4a. The figure shows the ratio of total tile to total rack flow rates. We note that
for open aisle condition where only CRAC 1 is turned ON, the supplied air flow rate
from the tiles is 79 % of the rack flow rate. This shows that only CRAC 1 is insuf-
ficient to supply the requisite amount of air required by the racks, which represents
an under-provisioned case. With containment of the cold aisle, the total tile to total
rack air flow rate increases to 89 %, as containment would tend to equalize the tile
and rack flow rates. For the other case, where both CRAC 1 and 2 are turned ON,
for the open aisle condition the total tile air flow rate is 121 % of the total rack air
flow rate. Hence, this case represents an over-provisioned condition. In this case,
also due to containment, the tile and rack air flow rate would tend to equalize. Hence
50 4 Cold Aisle Containment
Fig. 4.4 Tile, rack air flow rate ratio and aisle, plenum pressures. (a) Total tile/rack air flow rate
ratio. (b) Aisle pressure. (c) Plenum pressure (source: Arghode et al. 2013b and Arghode et al.
2015b). Reprinted with permission from ASME, applied for permission from Taylor and Francis
in the contained condition the total tile to total rack air flow rate ratio was decreased
to 110 %. With ideal containment with no air leakage the total tile to total rack air
flow rate ratio would be 100 %.
For the case where active tiles are used, for the open aisle condition the total tile
air flow rate is 98 % of the total rack air flow rate. Hence, this case represents a
fully-provisioned condition. In this case, for contained condition also the tile to rack
air flow rate suggests fully provisioned case (~103 %).
Figure 4.4b, c shows the measured aisle and plenum pressures, where the refer-
ence pressure was of the data center room space. Aisle pressure was measured at a
location in the middle plane of the cold aisle, 0.3 m (1 ft) above the rack height and
0.3 m (1 ft) away from the aisle entrance near Tile 1, 14 (see data center layout shown
in Fig. 4.1 for reference). The middle plane is the vertical plane along the height of
the racks, passing along the length of the cold aisle. The plenum pressure was mea-
sured at a location in the middle plane, 0.3 m (1 ft) below the floor and 1.83 m (6 ft)
away from the aisle entrance near Tile 1, 14. For measurement of plenum and aisle
pressures, the tube opening was oriented to face vertically upwards. This orientation
is close to the direction of the surrounding flow for the plenum pressure measure-
ment so as to have minimal effect on the static pressure measurements.
From Fig. 4.4b, we can observe that for open aisle conditions the aisle pressure
is close to the room pressure, as there is minimal restriction for air movement from
the cold aisle to the room space. However, as the cold aisle was contained for the
under-provisioned case, where only CRAC 1 was used, negative aisle pressure was
4.3 Experimental Results 51
observed. This is because in this case the racks are trying to entrain more air as
compared to the air supplied through the tiles, resulting in negative static pressure
in the aisle. Lower pressure in the aisle will also result in air leaking into the cold
aisle from the room space to make up the difference between the tile and rack air
flow rates. The opposite trend is observed for the over-provisioned case where both
CRAC 1 and 2 are turned ON. In this case, the aisle pressure is higher than the room
pressure. The balance air would thus leak out of the contained cold aisle due to
higher pressure in the aisle. For the fully-provisioned case where active tiles were
used, the aisle pressure is close to the room pressure. In this case, minimal air leak-
age in or out of the contained cold aisle is expected.
Figure 4.4c shows the measured plenum pressure, which as expected was higher
than the room pressure for all the cases, as this will drive the air flow across the
perforated tiles. The plenum pressure is higher for the case where both the CRACs
were turned ON, due to higher air flow rate (note that DP ~ Vin2 ). For the
under-provisioned case where only CRAC 1 is turned ON, the plenum pressure
reduces for contained aisle condition, as compared to open aisle case. This is
because in this case the racks are trying to entrain more air, as compared to the air
supplied through CRAC 1, thus reducing the pressure compared to the open aisle
condition. For the over-provisioned case, where both CRACs 1 and 2 are turned
ON, we observe that the plenum pressure increases for contained aisle condition, as
compared to open aisle case. This is because the CRACs tend to supply more air as
compared to rack air requirement, resulting in increase in plenum pressure due to
flow restriction caused by the containment.
For the fully-provisioned case, where active tiles are used, we observe that the
plenum pressure is close to the room pressure for both open and contained aisle condi-
tions. This is because the fans on the active tiles are in series with the CRAC fans and
the tile fans would tend to reduce the plenum pressure as it is on the suction side of the
fans. This also suggests that for this case minimal air leakage from the plenum to room
space will be present, and almost all of the supplied air will pass through the perfo-
rated tiles. Note that with passive tiles, air leakage from the plenum to room space can
be about 20 % of the total air flow rate through the CRAC units (Khalifa and Demetriou
2010). Hence, use of two fan systems (tile and CRAC) in series, promises to be more
energy efficient, as compared to using only one fan in the CRAC unit.
Fig. 4.5 Thermal field for open and contained aisle. (a) Open, Passive, 1 only. (b) Closed, Passive,
1 only. (c) Open, Passive, 1 and 2. (d) Closed, Passive, 1 and 2. (e) Open, Active, 1 only. (f) Closed,
Active, 1 only (source: Arghode et al. 2013b and Arghode et al. 2015b). Reprinted with permission
from ASME, applied for permission from Taylor and Francis
tiles, the central and lower regions of the middle plane show lower temperatures.
However, due to under supply of cold air, the rack air requirement is made up by the
hot air recirculated from the room space. From the figure it can be observed that the
hot air is recirculated from the aisle entrance, with most of the air recirculated from
the entrance near Tiles 1, 14. The hot temperature region is also present near the top
region (for region near Tiles 4, 11), suggesting that there is a possibility of hot air
entering the aisle from the top.
Measured temperature contours for contained aisle condition for the under-
provisioned case are shown in Fig. 4.5b. As compared to open aisle condition (see
Fig. 4.5a), it may be noted that the temperature profiles are more uniform for con-
tained aisle condition. This may be mostly due to the restriction to the hot air recir-
culation from the aisle entrance. Note that for this under-provisioned case, even
though the doors and the top curtain are deployed in the contained aisle case, there
is still hot air leakage into the cold aisle and hence we see some hot regions in the
middle plane, see Fig. 4.5b. Overall, the data suggest that containing the cold aisle
4.3 Experimental Results 53
results in more uniform and lower aisle temperatures, as compared to the open aisle
for this under-provisioned case.
The temperature contours at the middle plane for the over-provisioned case
where both CRAC 1 and 2 are turned ON are shown in Fig. 4.5c, d. For the open
aisle condition, see Fig. 4.5c, much larger cold regions as compared to the under-
provisioned case are seen (compared with Fig. 4.5a). This is expected because of
higher supply of cold air, which tends to flood the cold aisle. From the figure we also
note that, even though the cold air supply is greater than the rack air demand, hot
regions are present near the aisle entrances, suggesting hot air is entrainment into
the cold aisle, see Fig. 4.5c. The temperature contours for contained aisle for the
over-provisioned case are shown in Fig. 4.5d. The figure shows nearly uniform and
low temperature region in the middle plane. This suggests that the hot air entrain-
ment from the aisle entrance has been curtailed due to the closed doors, and that the
deployment of cold aisle containment can significantly reduce the recirculated hot
air mixing with the supplied cold air.
The temperature contours at the middle plane for the fully-provisioned case with
active tiles are shown in Fig. 4.5e, f. The measured temperature contours for the
open aisle condition, see Fig. 4.5e, show cold regions in the center, and hot regions
near the door entrances similar to that observed for under-provisioned case with
passive tiles, compare with Fig. 4.5a. Note that even though the supplied air flow
rate nearly matches the rack air requirement, considerable hot air entrainment is
present in the aisle. Most of the entrainment is expected from the aisle doors. The
temperature contours for the contained aisle for the fully-provisioned case are
shown in Fig. 4.5f. The figure shows nearly uniform temperature profile in the mid-
dle plane. This suggests that the hot air entrainment from the aisle entrance has been
curtailed due to the closed doors, and that the deployment of cold aisle containment
can significantly reduce the recirculated hot air mixing with the supplied cold air.
Figure 4.6a shows server inlet air temperatures for the under-provisioned case
where only CRAC 1 is turned ON. Measured temperatures in Fig. 4.6a show a large
variation for this case. In this case, the hot room air will be recirculated into the cold
aisle to make up the rack air requirement. Higher temperatures are encountered for
the end Racks 7 and 14 due to entrainment of hot air through the aisle entrance; also
see Fig. 4.5a for the thermal field for this case. Other racks such as Racks 2, 3 and
4, have higher server inlet temperatures for the higher placed segments, suggesting
that the high temperature room air may also be entrained from the aisle top.
Measured server inlet temperatures for the under-provisioned case with con-
tained aisle are shown in Fig. 4.6b. Except for Rack 1, the server inlet air tempera-
tures are fairly uniform, and lower as compared to the open aisle case, refer to
Fig. 4.6a. Rack 1 is a network rack and it is possible that part of the exit hot air may
enter the cold aisle through the rack inlet because of low aisle pressure for contained
aisle case. This may be the reason for high server inlet temperature for this rack.
Overall, it was observed that containing the cold aisle led to significant improve-
ment in the cold air delivery to the servers.
Figure 4.6c shows the measured server inlet temperatures for the over-provisioned
case. With open aisle, possibly due to hot air entrainment, the server inlet tempera-
tures are higher for the end Rack 7. Hot air entrainment can also be observed from
thermal field measurements shown. With contained aisle condition for the over-
provisioned case, the server inlet temperatures are significantly reduced with mini-
mal variation across the racks, representing a near-ideal air delivery scheme, see
Fig. 4.6d. This further reinforces the utility of using cold aisle containment to
achieve effective cold air delivery to the servers.
Figure 4.6e, f shows the measured server inlet temperatures for the fully-
provisioned (active tiles) case. With open aisle, possibly due to hot air entrainment,
the server inlet air temperatures are higher for the end Rack 7 and 14; see Fig. 4.6e.
Hot air entrainment can also be observed from thermal field measurements shown
in Fig. 4.5e. With contained aisle condition for the fully-provisioned case, the server
inlet air temperatures are significantly reduced with minimal variation across the
racks representing a near-ideal air delivery scheme, see Fig. 4.6f. This further rein-
forces the utility of full-provisioning and using cold aisle containment to achieve
effective cold air delivery to the servers.
From the analysis of the server inlet air temperatures for different cases, it appears
that the standard deviation and difference between maximum and minimum server
inlet air temperatures decreases with better provisioning and using cold aisle contain-
ment; see Table 4.1. Note that Rack 1 is excluded in this analysis, as it is a network
rack and there is a possibility of hot air entering from the back to the cold aisle. For
the under-provisioned case (passive tiles, only CRAC 1) with open aisle the differ-
ence between the maximum and minimum server inlet temperature is 7.4 °C, while
for the over-provisioned case (passive tiles, both CRACs 1 and 2) with contained
4.3 Experimental Results 55
Fig. 4.6 Server air inlet temperatures. (a) Open, Passive, 1 only. (b) Closed, Passive, 1 only. (c)
Open, Passive, 1 and 2. (d) Closed, Passive, 1 and 2. (e) Open, Active, 1 only. (f) Closed, Active,
1 only (source: Arghode et al. 2013b and Arghode et al. 2015b). Reprinted with permission from
ASME, applied for permission from Taylor and Francis
aisle the temperature difference is 1.3 °C. The CRAC supply air temperature can be
appropriately set for different cases so as to meet the ASHRAE maximum server
inlet temperature recommendation. For the present case the CRAC supply air tem-
perature is calculated as:
Tsupply = Tserver-max − Tallowance − (Tmax − Tmin),
Tserver-max = 27 °C, (ASHRAE TC 9.9 2011),
Tallowance = 1 °C = Tmin − Tsupply, (assumed)
56 4 Cold Aisle Containment
Table 4.1 Comparison of server inlet temperature uniformity (excluding Rack 1) (source:
Arghode et al. 2015b). Applied for permission from Taylor and Francis
Tmax − Tmin Std. dev. Proposed CRAC supply
Aisle Tile Provisioning (°C) (°C) air temperature (°C)
Open Passive, 1 Under 7.4 1.8 18.6
only
Passive, 1 Over 3.5 0.8 22.5
and 2
Active, 1 Full 5.0 1.3 21.0
only
Closed Passive, 1 Under 5.2 1.0 20.8
only
Passive, 1 Over 1.3 0.3 24.7
and 2
Active, 1 Full 2.6 0.5 23.4
only
Hence, for the present case, the supply air temperature through CRACs can be
increased from 18.6 °C for the under-provisioned case (open aisle) to 24.7 °C for
over-provisioned case (closed aisle). On one hand, increase in supply air flow rate
will result in more energy consumption, while on the other hand, increase in supply
air temperature can result in energy savings in the chiller plant and this will be dis-
cussed next.
Table 4.2 shows the overall power usage effectiveness (PUE) and chiller coeffi-
cient of performance (COP) for different cases having supply air temperature listed
in Table 4.1. Note that with increase in supply air temperature, the chiller COP
increases, which will result in reduced chiller power. Both passive and active tiles,
using containment resulted in higher chiller COP (~0.4 increase) and lower PUE
(~0.02 decrease). This suggests that the use of containment slightly improves the
energy efficiency by supplying more uniform temperature air to the servers. Active
tiles require extra energy to power the fans (1.8 kW in this case), while resulting in
higher air flow rate. This resulted in full-provisioning and improved thermal field
uniformity, as well as higher chiller COP (~0.45 increase as compared with passive
tiles, only CRAC 1) for both open and contained aisle conditions. However, it seems
that the extra energy spent on air movement is not compensated by the reduced power
consumption in the chiller compressor, and active tiles resulted in slightly higher
PUE (0.01 increase) as compared to passive tiles (only CRAC 1) for both open and
contained conditions. Similarly, in the over-provisioned case (passive tiles, both
CRACs 1 and 2), though the chiller COP is higher (~0.7) as compared with the
under-provisioned case (passive tiles, only CRAC 1), the overall cooling efficiency is
not favorable (PUE higher by ~0.04). This is because of higher energy expenditure
on extra air supplied for the over-provisioned case offsets the benefit of higher COP
due to better thermal field uniformity. The above analysis suggests that improving
thermal field uniformity not necessarily improves the overall cooling efficiency and
extra energy is spent on the air flow movement needs to be considered.
4.3 Experimental Results 57
Table 4.2 Energy utilization, QIT = 64 kW, PCRAC = 9.3 kW (1 only), 13.6 kW (1 and 2), Ptiles =
1.8 kW (active tiles), Pcon ≅ 1.6 kW, Tamb = 35 °C (source: Arghode et al. 2015b). Applied for
permission from Taylor and Francis
Aisle Tile Provisioning Chiller COP PUE
Open Passive, 1 only Under 4.30 1.44
Passive, 1 and 2 Over 4.93 1.48
Active, 1 only Full 4.68 1.45
Closed Passive, 1 only Under 4.65 1.42
Passive, 1 and 2 Over 5.37 1.46
Active, 1 only Full 5.11 1.43
Closure
Cold aisle containment was investigated for under-provisioned, fully-provisioned
and over-provisioned conditions. Containment of the cold aisle tended to equalize
the tile and rack air flow rates, however, due to imperfect containment, some leak-
age was present, either from or to the cold aisle depending on provisioning of the
supplied air. Thermal field measurements suggested significant improvement in
temperature uniformity in the contained cold aisle, as well as at the server inlets for
all the cases as compared to open aisle. For the under-provisioned case with con-
tainment, due to hot air leakage in to the cold aisle, the cold aisle and server inlet
temperatures were higher, as compared to the better-provisioned cases with contain-
ment. The fully and over-provisioned cases with containment resulted in near-ideal
temperature distribution in the cold aisle as well as at the server inlets. However,
these cases required higher energy expenditure to move larger quantity of air as
compared to the under-provisioned case and resulted in lower overall cooling effi-
ciency. Investigation of thermal characteristics for open aisle shows that significant
air entrainment is present through the aisle entrance due to higher momentum of air
jets emanating from perforated tiles even for the fully and over-provisioned cases.
Active tiles were used in to increase the tile air flow rates as compared to passive
tiles, and for the present investigation, resulted in full-provisioning of the air supply.
Active tiles also resulted in insignificant difference in plenum and room pressures,
and this suggests possibility to achieve minimal air leakage from the plenum to the
room. This promises more effective air delivery from CRAC units to the cold aisle
with minimal losses through leakage as compared to passive tiles.
Chapter 5
Other Air Delivery Schemes
While the raised floor HACA layout with peripheral CRAC units discussed in the
previous chapters is the most commonly employed one in most legacy, as well as
many recently constructed data centers, a growing number of new facilities are
employing alternate air delivery approaches. In this chapter we discuss some of
these. Figure 5.1 provides a summary of several possible approaches used in data
center thermal management. All of these assume local heat rejection from the IT
equipment to the data center room air (Evans 2012). The hot room air then transmits
heat via multiple possible coolants or interfaces outside the IT facility envelope. An
array of options also exists for the ultimate heat rejection to the environment from
the intermediate heat transmission devices that transport the heat to outside the data
center envelope.
Raised floor placement of IT equipment has been utilized for the past several
decades. In addition to providing a convenient way to deliver the cooling air, the
plenum is also utilized for routing of electrical cabling and chilled water piping for
localized liquid cooling of high power density equipment racks. With average rack
power dissipations increasing above 20 kW in many facilities, adequate air delivery
to individual racks using raised floors becomes a major challenge, often requiring
plenum depths of 1 m or more. Even for a very deep plenum adequate cold air deliv-
ery to individual racks can be challenging. Hard floor placement of equipment has
been proposed to address this challenge (Rasmussen 2014). Such placement has
been used traditionally for telecommunications facilities, located in office build-
ings, not requiring specially constructed spaces.
Data center
envelope
Fig. 5.1 Several possible internal and external thermal management approaches for data centers
Figure 5.2 shows a non-raised floor configuration, which utilizes upflow CRAC
units to supply cold air from ceiling diffusers. The hot air returns via return vents on
the CRAC sidewalls near the floor. The diffusers are symmetrically placed below
the ceiling above the cold aisles. Overhead ducts feed chilled air across the room,
and blowers deliver cold air along the length of the cold aisle.
Shrivastava et al. (2005) performed computational fluid dynamics studies of sev-
eral air delivery schemes with peripheral CRAC units. They compared different air
distribution configurations by monitoring the maximum and mean rack inlet tem-
perature. The cold air was supplied at a temperature of 20 °C at 0.685 m3/s (1452 ft3/
min) per rack, and the computational domain consisted of 20 racks, each dissipating
12 kW. Figure 5.3 compares the mean rack inlet air temperature, and the maximum
rack inlet temperature for seven data center configurations.
The highest temperature at rack inlet (42.7 °C) was observed for the ceiling sup-
ply and room return configuration. The poor performance of the non-raised floor
layout was attributed to the short-circuiting of cold air. Some amount of the cold
supply air leaves the cold aisle, and returns directly to the vents without passing
through the racks. This short-circuiting of cold air decreases the static pressure in the
cold aisle, and results in the entrainment of hot air from the other portions of the
room, which further increases the undesirable recirculations. To minimize
short-circuiting of supply cold air, short partitions were recommended around the
supply openings by Schmidt et al. (2005).
5.2 Ceiling Delivery of Cold Air with Perimeter CRAC Layout 61
. .
.
. .
RAISED FLOOR RAISED FLOOR CEILING SUPPLY RAISED FLOOR CEILING SUPPLY RASIED FLOOR RAISED FLOOR
SUPPLY / SUPPLY / / ROOM AND CEILING / CEILING AND CEILING AND CEILING
ROOM RETURN CEILING RETURN SUPPLY / RETURN SUPPLY / SUPPLY /
RETURN ROOM RETURN CEILING AND CEILING
ROOM RETURN RETURN
Average Rack Inlet Temp (degree) Maximum Rack Inlet Temp (degree)
Fig. 5.3 Average and maximum rack inlet temperature for different data center configurations
(Shrivastava et al. (2005))
There are other possible solutions to reduce the mixing of hot and cold air, while
continuing to employ peripheral CRAC unit layout. The concept of cold aisle con-
tainment has been explored in Chap. 4. An alternative approach is to use hot aisle
containment, seen in Fig. 5.4. It has been argued that this may be preferable com-
pared to cold aisle containment in new data center facilities due to its higher flexibil-
ity in applying to individual racks, or an entire aisle (Sasser 2015). It may also not
require a raised floor, unless chilled water is being delivered to specific racks. Since
the ceiling plenum is typically not used for placement of hardware for supporting
the IT equipment, it may be easier to design the air return path.
62 5 Other Air Delivery Schemes
Fig. 5.4 No raised floor air delivery with hot aisle containment
Moving the source of cooling delivery closer to the equipment racks can reduce the
complexity of coolant distribution, and reduce the unwanted mixing of hot and cold
air, as the size and power density of data centers increase (Dunlap and Rasmussen
2006). Such close coupling may move the cooling source close to a row of racks,
known as in-row cooling. An even closer placement is to incorporate the cooling
within the individual racks, termed in-rack cooling.
In the in-row cooling approach, the CRAC units are associated with rows of racks.
These units can placed among the racks, or mounted above the racks. Both configu-
rations are shown in Fig. 5.5.
Compared with the traditional room-based raised floor architecture, the in-row
cooling design does not require a raised floor, which reduces installation costs. The
airflow paths are also shorter and less complex, which may reduce the required
power of CRAC fans. In-row cooling also allows CRAC capacity to be varied
between various rows, based on actual needs of the specific rows. This helps one row
of rack to accommodate high density equipment, such as blade servers, and another
row to incorporate lower power density equipment such as communication units.
Several in-row architectures have been employed, with increasing rack powers.
The in-row pumped refrigerant units use refrigerant to chill water, which is circulated
5.4 Free Air Cooling 63
Fig. 5.5 Close-coupled cooling approaches: (a) Plan view of in-row cooling, (b) Elevation view
of overhead cooling
through overhead air cooling units, which provide cold air from the top. In-row
direct expansion units cool the hot air from the IT equipment directly in the evapo-
rator of the refrigeration unit. For even higher heat removal rates from individual
racks, chilled water can be directly delivered to the individual rows.
In-row cooling approaches allow locating higher power equipment in certain
areas of an otherwise lower power density data center. The electrical power costs of
in-row cooling may be lower compared with the room cooling approach, when
higher rack power density is needed. This is because the CRAC units are well cou-
pled to the loads and their capacity can be increased in a modular fashion to match
the load requirements.
As seen in Fig. 5.1, there are many different types of chillers used for data center
cooling, including air-cooled, water-cooled, and evaporatively cooled. When a
chiller system is used, a water-side economizer can be implemented to reduce com-
pressor run hours and energy usage. Water-side economizer and chiller both provide
cold water to cool the supply air. The difference between the two installations is that
in the water-side economizer, the cooling water rejects heat to the environment by
heat and mass transfer, without the use of mechanical cooling. This scheme can
work in both raised floor and non raised floor configurations.
A growing trend in low and medium power density data centers is to eliminate
the use of external chillers to reduce energy consumption. The use of air-side econo-
mizers allows the ambient air to be brought directly into the data center space for
cooling. Prior to being delivered into the data center, the external air is filtered to
remove particulates and harmful gases. During dry and hot seasons, it may be
further cooled through the use of a wet evaporative medium. It is possible to
incorporate this scheme in a non raised-floor arrangement, with hot aisle contain-
ment (Sasser 2015), as seen in Fig. 5.6.
64 5 Other Air Delivery Schemes
Fig. 5.6 Non raised floor air delivery with direct evaporative cooling and contained hot aisle
Cooling with a regenerative heat exchanger is somewhat similar to free air cooling in
that both systems use ambient air, and each can work in raised floor, as well as non
raised floor configurations. The regenerative system can also achieve refrigeration-
free cooling when the ambient temperature is below a certain maximum, which
depends upon the supply and return air temperature set points. A supplemental sys-
tem must be installed and used when the ambient temperature is too high for the
system to efficiently dissipate the heat. Typically, the system uses a regenerative
5.5 Cooling with a Regenerative Heat Exchanger 65
Fig. 5.7 Rotary regenerative heat exchanger based data center thermal management
thermal storage wheel to move the heat generated by the IT hardware to the environ-
ment. Figure 5.7 shows an overview of a regenerative heat exchanger installed in data
center and overall airflow patterns.
The region enclosed by the dashed line occupying the right side of the wheel is
the cold side. The cold ambient air drawn into the lower chamber by a set of fans
cools the wheel, and the discharge air is exhausted back to the ambient. The heated
internal air from the computer room is drawn into the higher chamber by another set
of fans. This air passes through the wheel, transferring heat to the external air. The
cooled air is then recirculated into the data center.
Figure 5.8 shows a schematic of a typical thermal energy storage wheel.
66 5 Other Air Delivery Schemes
The wheel is typically made of a porous metal honeycomb material that enables
air to pass through, while transferring heat. Increasing the rotational speed of the
wheel increases the overall system efficiency. However, it is necessary for the wheel
to rotate slowly enough in order to prevent the air from mixing between the cold and
hot sides.
The main difference between the direct free air cooling and cooling with a regen-
erative heat exchanger is the necessity of air quality control in the former. Due to
only a small exchange of air between the computer room and the ambient, the
regenerative wheel system is potentially applicable even if the ambient air is too dry,
moist, or polluted.
Closure
The conventional raised-floor HACA air delivery is widely used in legacy facilities.
As facility power densities increase, adequate cold air delivery to the IT equipment
is becoming an increasing challenge, requiring very deep plenums, and careful air
flow management. In the previous chapters we have focused on the use of metrology
and simulation tools for achieving this. An emerging trend in new facilities is the
use of alternate air delivery techniques. In many instances, this involves going away
from the raised floor plenum and placement of IT racks on the hard floor. The room
level cold air delivery can be implemented via the ceiling, or the room space.
Close coupled air delivery can be used to bring the cooling air to the aisle or rack
level. Also, hot air can be more effectively returned while preventing mixing with
cold air using hot-aisle containment. In low and medium power density facilities,
cooling energy costs can be reduced through the use of air-side economizers. These
allow transfer of heat to the ambient air, without the use of mechanical chillers.
Under the right conditions, the outside air can be filtered and brought into the data
center facility. Alternately, through the use of regenerative heat exchange, this heat
transfer can be accomplished without direct introduction of outside air.
References
Abdelmaksoud, W. A., Khalifa, H. E., Dang, T. Q., Elhadidi, B., Schmidt, R. R., & Iyengar, M.
(2010). Experimental and computational study of perforated floor tile in data centers. In
(ITherm) Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic
Systems, June 2–5, Las Vegas, NV.
AdaptivCool, Milford, NH. (2015). Retrieved from www.adaptivcool.com
All Sensors Corp., Morgan Hill, CA. (2015). Retrieved from www.allsensors.com
APC, Kingston, RI. (2015). Retrieved from www.apc.com
Arghode, V. K., & Joshi, Y. (2013). Modeling strategies for air flow through perforated tiles in a
data center. IEEE Transactions on Components, Packaging and Manufacturing Technology, 3,
800–810.
Arghode, V. K., & Joshi, Y. (2014a). Rapid modeling of air flow through perforated tiles in a raised
floor data center. In (ITherm) The Intersociety Conference on Thermal and Thermomechanical
Phenomena in Electronic Systems, May 27–30, Orlando, FL.
Arghode, V. K., & Joshi, Y. (2014b). Measurement of air flow characteristics through perforated
tiles in a raised floor data center. In (ISTP) International Symposium on Transport Phenomena,
November 5–7, Krabi, Thailand.
Arghode, V. K., & Joshi, Y. (2015a). Experimental investigation of air flow through a perforated
tile in a raised floor data center. Journal of Electronic Packaging, 137(1), 011011.
Arghode, V. K., & Joshi, Y. (2015b). Measurement of air flow rate sensitivity to differential pres-
sure across a server rack in a data center. ASME Journal of Electronic Packaging, 137(4),
041002.
Arghode, V. K., Kang, T., Joshi, Y., Phelps, W., & Michaels, M. (2015a). Anemometric tool for air
flow rate measurement through perforated tiles in a raised floor data center. In (SEMI-THERM)
IEEE Semiconductor Thermal Measurement and Management Symposium, March 15–19, San
Jose, CA.
Arghode, V. K., Kumar, P., Joshi, Y., Weiss, T., & Meyer, G. (2013a). Rack level modeling of air
flow through perforated tile in a data center. Journal of Electronic Packaging, 135(3), 030902.
Arghode, V. K., Sundaralingam, V., & Joshi, Y. (2015b). Air flow management in a contained cold
aisle using active fan tiles for energy efficient data center operation. Heat Transfer Engineering.
doi: 10.1080/01457632.2015.1051386.
Arghode, V. K., Sundaralingam, V., Joshi, Y., & Phelps, W. (2013b). Thermal characteristics of
open and contained data center cold aisle. Journal of Heat Transfer, 135(6), 061901.
ASHRAE. (2012). 2012 ASHRAE handbook: Heating, ventilating, and air-conditioning systems
and equipment. Atlanta, GA: ASHRAE.
ASHRAE TC 9.9. (2011). Thermal guidelines for data processing environments: Expanded data
center classes and usage guidance.
Bleier, F. P. (1997). Fan handbook: Selection, application, and design. New York: McGraw-Hill.
Breen, T. J., Walsh, E. J., Punch, J., Shah, A. J., Kumari, N., Bash, C. E., et al. (2012). From chip
to cooling tower data center modeling: Validation of a multi-scale energy management model.
In (ITherm) Intersociety Conference on Thermal and Thermomechanical Phenomena in
Electronic Systems, May 30–June 1, San Diego, CA.
Degree Controls Inc., Milford, NH. (2015). Retrieved from www.degreec.com
Dunlap, K., & Rasmussen, N. (2006). The advantages of row and rack-oriented cooling architec-
tures for data centers (APC White Paper #130).
Evans, T. (2012). The different technologies for cooling air (White Paper 59, Rev. 2). Schneider
Electric.
Freid, E., & Idelchik, I. E. (1989). Flow resistance, a design guide for engineers. New York:
Hemisphere.
Gondipalli, S., Bhopte, S., Sammakia, B., Iyengar, M. K., & Schmidt, R. (2008). Effect of isolating
cold aisles on rack inlet temperature. In (ITherm) Intersociety Conference on Thermal and
Thermomechanical Phenomena in Electronic Systems, May 28–31, Orlando, FL.
Gondipalli, S., Sammakia, B., Bhopte, S., Schmidt, R., Iyengar, M. K., & Murray, B. (2009).
Optimization of cold aisle isolation designs for a data center with roofs and doors using slits.
In (InterPACK) Pacific Rim Technical Conference and Exposition on Packaging and Integration
of Electronic and Photonic Systems, July 19–23, San Francisco.
Gordon, J. M., & Ng, K. C. (1994). Thermodynamic modeling of reciprocating chillers. Journal of
Applied Physics, 75, 2769–2774.
Incropera, F. P., & Dewitt, D. P. (1996). Fundamentals of heat and mass transfer. New York:
Wiley.
Iyengar, M., & Schmidt, R. (2009). Analytical modeling for thermodynamic characterization of
data center cooling systems. Journal of Electronic Packaging, 131(2), 021009.
Johnsons Controls. (2015). Model YCAL0028EE air-cooled scroll compressor liquid chillers,
R-410A. Retrieved from http://www.johnsoncontrols.com
Joshi, Y., & Kumar, P. (2012). Energy efficient thermal management of data centers. New York:
Springer.
Khalifa, H. E., & Demetriou, D. W. (2010). Energy optimization of air-cooled data centers. Journal
of Thermal Science and Engineering Applications, 2(4), 041005.
Koomey, J. (2011). Growth in data center electricity use 2005 to 2010. Oakland, CA: Analytics
Press. Retrieved from http://www.analyticspress.com/datacenters.html
Kumar, P., & Joshi, Y. (2010). Experimental investigations on the effect of perforated tile air jet
velocity on server air distribution in a high density data center. In (ITherm) Intersociety
Conference on Thermal and Thermomechanical Phenomena in Electronic Systems, June 2–5,
Las Vegas, NV.
Lee, T. W. (2008). Thermal and flow measurements. Boca Raton, FL: CRC.
Lipari, G., & Stansby, P. K. (1990). Review of experimental data on incompressible turbulent
round jets. Flow, Turbulence and Combustion, 87, 79–114.
Miller, D. S. (1990). Internal flow systems. Houston, TX: Gulf.
Moore, J., Chase, J., Ranganathan, P., & Sharma, R. (2005). Making scheduling ‘cool’:
Temperature-aware workload placement in data centers. In USENIX Annual Technical
Conference Anaheim, April 10–15, Anaheim, CA.
Nelson, G. (2007). Development of an experimentally-validated compact model of a server rack.
MS thesis, Georgia Institute of Technology, Atlanta.
Patankar, S. V. (2010). Airflow and cooling in a data center. Journal of Heat Transfer, 132(7),
073001.
Rasmussen, N. (2014). Raised floor vs hard floor for data center applications (White Paper 19,
Rev. 3).
Samadiani, E., Amur, H., Krishnan, B., Joshi, Y., & Schwan, K. (2010). Coordinated optimization
of cooling and IT power in data centers. Journal of Electronic Packaging, 132(3), 031006.
References 69
Sasser, J. (2015). A look at data center cooling technologies in design. Uptime Institute. Retrieved
August 30, 2015, from https://journal.uptimeinstitute.com/a-look-at-data-center-cooling-
technologies/
Schmidt, R., Cruz, E., & Iyengar, M. (2005). Challenge of data center thermal management. IBM
Journal of Research and Development, 49(4/5), 709–723.
Schmidt, R., & Iyengar, I. (2009). Thermodynamics of information technology data center. IBM
Journal of Research and Development, 53, 9/1–9/15.
Schmidt, R., Vallury, A., & Iyengar, M. (2011). Energy savings through hot and cold aisle contain-
ment configurations for air cooled servers in data centers. In (InterPACK) Pacific Rim Technical
Conference and Exposition on Packaging and Integration of Electronic and Photonic Systems,
July 6–8, Portland, OR.
Sensirion AG, Staefa, ZH, Switzerland. (2015). Retrieved from www.sensirion.com
Shortridge Instruments Inc., Scottsdale, AZ. (2015). Retrieved from www.shortridge.com
Shrivastava, S. K., Calder, A. R., & Ibrahim, M. (2012). Quantitative comparison of air contain-
ment systems. In (ITherm) Intersociety Conference on Thermal and Thermomechanical
Phenomena in Electronic Systems, May 30–June 1, San Diego, CA.
Shrivastava, S., Sammakia, B., Schmidt, R., & Iyengar, M. (2005). Comparative analysis of differ-
ent data center airflow management configuration. In Paper IPACK2005-73234, San Francisco,
July 17–22, 2005.
Sundaralingam, V., Arghode, V. K., Joshi, Y., & Phelps, W. (2013). Experimental characterization
of cold aisle containment for data centers. In (SEMI-THERM) Semiconductor Thermal
Measurement and Management Symposium, March 17–21, San Jose, CA.
Takahashi, M., Uekusa, T., Kishita, M., & Kaneko, H. (2008). Aisle-capping method for airflow
design in data centers. In International Telecommunications Energy Conference, September
14–18, San Diego, CA.
TSI Inc., Shortview, MN. (2015). Retrieved from www.tsi.com
Yu, F. W., & Chan, K. T. (2006). Modelling of the coefficient of performance of an air-cooled
screw chiller with variable speed condenser fans. Building and Environment, 41, 407–417.
Index
E P
Edge blockage, 40–41 Particle image velocimetry (PIV) technique, 21
Electricity consumption, 1 DCL, 22–23
Energy principle of, 21
consumption, 5, 10, 54–57 Passive tiles, 12–14, 46–48
modeling, 7–9 Perforated tile, 12, 13
usage analysis, 3–6 Piezoresistive pressure sensor, 26
PIV technique See Particle image velocimetry
(PIV) technique
F Plenum pressure, 48–51
Flow field measurement, PIV technique, Porous jump model (PJ), 33
21–23 Power usage effectiveness (PUE), 8
DCL, 22–23 Pressure loss factor (Keff), 28
principle, 21 Pressure measurement, 24–25
Flow Hood, 11–16 mass flow based pressure sensor, 25–26
active tiles, 14–16 piezoresistive pressure sensor, 26
commercial, 12
passive tiles, 12–14
Free air cooling, 63–64 R
Rack air flow rate measurement, 16
Regenerative heat exchanger, 64–66
G
Geometrical resolution (GR) model, 38 S
Server inlet temperature, 53–54
Server rack, 18
I
In-house developed tool, 17
In-row cooling, 62–63 T
Interrogation areas, 22 3D mapping tool, 23
Thermal field, 23–24, 51–53
Thermocouples, 23
M Thermodynamics, 6–10
Mass flow based pressure sensor, 25–26 Tiles
Modified body force (MBF) model, active, 14–16
35–38 passive, 12–14
edge blockage, 40–41 perforation, 12, 13
small pore sizes and computational effort, pore size, 39–40
41–42 porosity, 31, 40
tile pore size, 39–40 width, 32
tile porosity, 40 Tool resistance compensation, 17–20
N V
Non raised floor layout, 59 Velocity vectors, 21