You are on page 1of 10

Room Level Modeling

of Air Flow in a Contained


Data Center Aisle
Vaibhav K. Arghode1 Cold aisle containment is used in air cooled data centers to minimize direct mixing
Post Doctoral Fellow between cold and hot air. Here, we present room level air flow field investigation for
George W. Woodruff School open, partially and fully contained cold aisles. Our previous investigation for rack level
of Mechanical Engineering, modeling has shown that consideration of momentum rise above the tile surface, due to
Georgia Institute of Technology, acceleration of air through the pores, significantly improves the predictive capability as
Atlanta, GA 30332 compared to the generally used porous jump model. The porous jump model only speci-
e-mail: vaibhav.arghode@me.gatech.edu fies a step pressure loss at the tile surface without any influence on the flow field. The
momentum rise could be included by either directly resolving the tiles pore structure or
Yogendra Joshi by artificially specifying a momentum source above the tile surface. In the present work,
George W. Woodruff School a modified body force model is used to artificially specify the momentum rise above the
of Mechanical Engineering, tile surface. The modified body force model was validated against the experimental data
Georgia Institute of Technology, as well as with the model resolving the tile pore geometry at the rack level and then
Atlanta, GA 30332 implemented at the room level. With the modified body force model, much higher hot air
entrainment and higher server inlet temperatures were predicted as compared to the
porous jump model. Even when the rack air flow requirement is matched with the tile air
flow supply, considerable hot air recirculation is predicted. With partial containment,
where only a curtain at the top of the cold aisle is deployed and side doors are opened,
improved cold air delivery is suggested. [DOI: 10.1115/1.4026540]

Keywords: aisle containment, hot air recirculation, air flow distribution, modified body
force model, porous jump model, server inlet temperature

1 Introduction In previous computational investigations with cold aisle con-


tainment, mass flow rate with uniform velocity has been specified
Physical separation of hot and cold aisles can help to reduce
at the tile surface. Specification of uniform velocity at the tile inlet
cold air by-pass from the cold aisle or hot air recirculation in the
results in lower momentum of air emanating from the tile, as it
cold aisle. The physical separation can be deployed by either con-
does not considers the acceleration of air through the tile pores
taining the cold aisles or containing the hot aisles. Chimney has
[7,8]. Consideration of lower momentum, affects the entrainment
also been used to separate hot air exiting from each rack and
of surrounding air as well as the flow field development in the
directly transferring it to the return duct [1]. Each of these techni-
downstream of the tile. This can affect the hot air recirculation
ques have their respective benefits and short comings and this has
in the aisle and supply of cold air to the adjacent racks. This
been discussed in Refs. [1,2]. Overall, the major benefit of aisle
phenomenon was illustrated by comparing the results with
containment is mitigation of high temperatures at the server inlets.
experimental (PIV) data and with the model considering tile geo-
With containment, the supply air temperature from the computer
metrical details as compared to specifying a uniform velocity at
room air conditioning (CRAC) units can be set to higher value,
the tile [7].
leading to higher supply chilled water temperature and hence
In the present work, room level air flow is investigated for both
lower power consumption at the chiller plant. Moreover, supply
open aisle and contained cold aisle with consideration of momen-
of excess cold air can be obviated, thus resulting in lower energy
tum rise above the tile surface. First, the results for a rack level
expenditure on the movement of air by the CRAC fans.
computational fluid dynamics (CFD) model is validated with the
In previous computational investigations, it was observed that
experiments and then the model was extended for full room scale.
the deployment of cold aisle containment can result in significant
Partial containment, where only the top of the cold aisle is capped
energy savings [2,3]. In an experimental investigation, cold aisle
to allow easy access to the cold aisle, is also considered. The
containment was observed to result in lower fan power (15%
effect of including the momentum rise above the tile surface on
energy savings). Moreover, in case of cooling failure, cold aisle
the flow field and the thermal characteristics is discussed.
containment was suggested to result in reduced hot spot genera-
tion with time, as compared to the case without containment [4].
Partial containment has also been investigated computationally 2 Models for Air Flow Through a Perforated Tile
[5,6]. In this case, undersupply of air from the perforated tiles as Resolving the tile geometrical structure for air flow modeling
compared to the demand of air from the racks was employed. Var- through a perforated tile, results in significantly large computa-
ious designs, with different openings in the containment system tional effort [7]. Hence, rapid models are desired which do not
and meshed partitions were investigated, to balance the flow rates require resolution of the tile geometry, and are still able to capture
and achieve improved performance. Solid door with slit on the the flow features downstream of the tile. Some of these models
roof was suggested to be one of the optimal choices [5]. are discussed below.
1
Corresponding author. 2.1 Porous Jump Model. Porous jump model specifies a step
Contributed by the Electronic and Photonic Packaging Division of ASME for
publication in the JOURNAL OF ELECTRONIC PACKAGING. Manuscript received August
pressure loss across the tile surface. The pressure loss can be com-
14, 2013; final manuscript received January 6, 2014; published online February 14, puted as is given in Eq. (1), where the pressure loss factor (K) can
2014. Assoc. Editor: Amy Fleischer. be calculated based on the tile porosity (F) (see Eq. (2)) [9].

Journal of Electronic Packaging Copyright V


C 2014 by ASME MARCH 2014, Vol. 136 / 011011-1

Downloaded From: http://electronicpackaging.asmedigitalcollection.asme.org/pdfaccess.ashx?url=/data/journals/jepae4/929392/ on 05/03/2017 Terms of Use: http://www.asme.or


1 of 56 cm (22 in.) is used for the momentum source specification in
DP K qVin2 (1) the present case. Height is chosen based on the turbulent jet decay
2
!2 characteristics. For a free jet (diameter D) significant jet decay
 1 is observed at an axial distance of about 18D, whereas for a per-
1 1F 2
K 2 1  F (2) fectly symmetric array of jets significant jet decay is observed at
F 2
an axial distance of about 8D [10]. In reality, the scenario of jet
decay will be in between the perfectly symmetric condition and
Vin inlet velocity; q density; F porosity Open Area=Total the free jet condition, and hence, an intermediate distance of 12D
Area is chosen here. Increasing the height of modified body force
It may be noted that the use of this model results in no altera- (MBF) region results in higher cold air by-pass, and in a sense, we
tion in the velocity field across the tile surface and only a step have calibrated the height of the MBF region using the geometri-
pressure loss is specified across the tile. However, in reality, as air cal resolution model. For the present case D 1.27 cm (0.5 in.)
flows through a perforated tile, there is acceleration of air though for the generic tile (refer Ref. [7] for further details), hence, the
the pores resulting in increase in the air momentum. This higher height of the momentum source region is chosen as 15 cm (6 in.).
momentum of air above the tile surface was attempted to be cap- Note that in Ref. [7] the height of the momentum source region
tured by the body force model which is described later. For the for the modified body force model was 10.16 cm (4 in.) and the
present case the pressure loss factor (K) across the tile surfaces is width was 61 cm (24 in.) following the suggestions given in
16.5 and the tile porosity (F) is 31%. Refer to Ref. [7] for further Ref. [8], and further improvement in the model was desired to
details of tile geometry. achieve closer comparison with the experiments. Note that the
perforated tiles used in data centers can have different pore sizes,
2.2 Body Force Model. In this model, a momentum source is shapes, arrangements, and porosities and it will be interesting to
specified above the tile surface to simulate the acceleration of investigate the influence of these parameters on the air flow
air passing through the tile pores (see Eqs. (3)(5)) [8]. The distribution.
magnitude of the momentum source is calculated based on the
difference of momentum of air flow through the tile pores as com- 3 Cases Investigated
pared to the momentum of air flow approaching the tile (see
Eq. (3)). The momentum of air flow through the pores is calcu- Table 1 summarizes the two cases considered here for the
lated based on the tile porosity and assuming a top-hat velocity investigation. In the under provisioned case, air flow rate
profile through the pores (see Eq. (4)). through the perforated tile is 60% of the rack air requirement. In
this case the rack will draw the remaining air from the room, lead-
 
DM qAtile Vin Vpore  Vin (3) ing to a minimum of 40% of hot air recirculation. In the fully
provisioned case, tile air flow rate matches with the rack air
Vin requirement. Ideally, this case can lead to uniform server air inlet
Vpore (4)
F temperatures and absence of hot air recirculation, where neither
DM the hot air entrainment nor the cold air by pass is present.
Sx (5) Fully turbulent flow was considered and here the pore Reynolds
8
number (Repore) (based on pore velocity (Vpore), and pore size
where Sx is the source term for x momentum equation, x is the (D)) is in the range of 55009200. To validate the modified
disance along height of the rack, is the volume of momentum body force model, first, rack level modeling was undertaken. Geo-
source region, and Atile is the total area of tile. metrical details of the perforated tiles, such as pore size and pore
distribution were included in the geometrical resolution model
(refer to Ref. [7] for more details). Inclusion of length scales of
2.3 Modified Body Force Model. The body force model sug- pores resulted in significantly large mesh size (4  106 cells/
gested previously considers a top hat velocity profile at the tile rack, [7]), requiring prohibitively large computational effort for
pores while calculating the excess momentum of air above the tile the full room level modeling (14 tiles in this case). The rack level
surface. However, even higher momentum of the air flow is modeling results were compared with the experimental data to
expected in the downstream, as in reality, there will be a further identify salient features of the modified body force model. Later
necking of the air jet (vena contracta) in the downstream. Hence, the modified body force model was incorporated in the full room
we suggested a modification to the body force model to capture scale modeling.
this effect [7]. In the modified body force model the momentum
rise across the tile surface is calculated based on the velocity at
vena contracta, Vneck, instead of the average pore velocity, Vpore, 4 Rack Level Modeling
in the Eq. (3). The velocity at the vena contracta, Vneck, is esti- In Figs. 1 and 2, the flow field obtained from the modified body
mated using Bernoullis equation and the pressure loss across the force model is compared with the data obtained from the PIV
tile (see below equation). experiments [11], as well as with the porous jump model and the
model resolving the tile geometry. Both under provisioned (see
1 1 2 1 1
DP K qVin2 qVneck  qVin2 Vneck Vin K 12 (6) Fig. 1) and fully provisioned (see Fig. 2) cases are investigated.
2 2 2 The model simulates an isolated rack, where the air can be
entrained from the aisle sides and the aisle center (see Figs. 1(a)
where Vneck is the jet velocity at the vena contracta
Note that the modified body force model will have higher mag-
nitude of momentum source as compared to the body force model, Table 1 Different cases investigated
because the velocity at the vena contracta is expected to be higher
Rack flow rate Tile flow rate Tile flow/
than the average velocity at the pores.
Case (CFM, m3/s) (CFM, m3/s) rack flow
For the present investigation, the region of momentum source
specification was chosen to be 56 cm  56 cm  15 cm (22 in.  22 Under 2594, 1.224 1598, 0.754 60%
in.  6 in.) above the tile surface. Note that a generic tile has provisioned
dimensions of 61 cm  61 cm (24 in.  24 in.). However, the per- Fully 2594, 1.224 2594, 1.224 100%
forations starts about 2.54 cm (1 in.) away from the tile edges provisioned
(refer Ref. [7] for geometrical details of the tile), hence the width

011011-2 / Vol. 136, MARCH 2014 Transactions of the ASME

Downloaded From: http://electronicpackaging.asmedigitalcollection.asme.org/pdfaccess.ashx?url=/data/journals/jepae4/929392/ on 05/03/2017 Terms of Use: http://www.asme.or


Fig. 1 Comparison of PIV and CFD results for the under provisioned case

Fig. 2 Comparison of PIV and CFD results for the fully provisioned case

and 2(a)). The aisle top is also open which will allow for air Fig. 1(c)). The porous jump model is also not able to capture the
by-pass or entrainment. For further details of the computational air entrainment from aisle center, near the tile surface (bottom left
set up, geometry, and operating conditions, refer to Ref. [7]. corner) and the delayed entry of air in the lower portion of the
From the experimental data shown in Fig. 1(b), we note that rack inlet (bottom right corner). The porous jump model also
even though the air flow rate through the tile is lower than the suggests entrainment from the aisle top which is contrary to the
rack air requirement (under provisioned, case 1), a portion of the experimental observation.
supplied air is observed to escape from the aisle top. We can also Flow field with the geometrical resolution model that included
observe the delayed entry of the tile air at the lower portion of the the geometrical details of the tile and with the modified body force
rack inlet (see bottom right corner of Fig. 1(b)). This was attrib- model is shown in Figs. 1(d) and 1(e), respectively. From the fig-
uted to the higher momentum of air emerging from the tile [7]. ure we observe that the flow field features for both of these models
Air entrainment from the aisle center can also be noticed for this are closer to the experimental results. A portion of air flow is
case (see bottom left corner of Fig. 1(b)). The porous jump model observed to escape from the aisle top as observed from the experi-
is not able to capture these features and the air flow from the ments. These models also capture the delayed entry of air to the
tile is predicted to reach only up to a certain height of rack (see lower portion of the rack inlet (bottom right corner). The air

Journal of Electronic Packaging MARCH 2014, Vol. 136 / 011011-3

Downloaded From: http://electronicpackaging.asmedigitalcollection.asme.org/pdfaccess.ashx?url=/data/journals/jepae4/929392/ on 05/03/2017 Terms of Use: http://www.asme.or


Table 2 Comparison of net air entrainment for the geometrical (13%) entering from the aisle top which is contrary to that
resolution, modified body force and the porous jump models observed with the other two models. The entrainment predicted
(negative values refer to flow exiting the domain) from the gap is also very less for the porous jump model as com-
pared to the other two models.
Air flow/ Geometrical Modified Porous
For the fully provisioned case, considerable entrainment is pre-
rack flow (%) resolution body force jump
dicted from aisle sides with geometrical resolution and modified
Under provisioned case body force models predicting 35% air entrainment. For this
Aisle top 5.32 8.56 12.95 case, the porous jump model predicted much lower air entrain-
Aisle sides 33.62 35.62 28.99 ment (12%). The porous jump model predicted negligible air
Aisle center 9.33 10.64 2.00 by-pass from aisle top whereas the other models predicted signifi-
Gap 2.39 2.31 0.08 cant amount of air by-pass (43%). These results suggest that the
Fully provisioned case modified body force model predicts flow field closer to the geo-
Aisle top 46.18 43.48 0.77 metrical resolution model and the use of porous jump model can
Aisle sides 35.77 35.31 12.26 lead to large errors in the prediction of the air flow field.
Aisle center 7.02 4.78 11.41 By choosing appropriate boundary conditions on the computa-
Gap 3.39 3.32 0.15 tional domain contained aisle conditions can be simulated. The
middle racks of a contained aisle are simulated using the computa-
tional domain as shown in Fig. 3(a) for the fully provisioned case.
entrainment from aisle center could also be observed from Symmetry is used for the aisle sides and aisle center (to simulate
Figs. 1(d) and 1(e) (bottom right corner), concurring with the middle racks) and the aisle top is treated as wall (to simulate the
experimental data. aisle containment). No air flow is allowed through the gap as the
For the fully provisioned case, the model with tile geometrical entrance is treated as blocked; see Fig. 3(a) for details.
details as well as the modified body force model showed closer From the velocity magnitude contours shown in Figs. 3(c) and
comparison with the measured flow field, see Fig. 2. For this case 3(d), we observe that due to higher momentum of air above the
also the porous jump model was not able to capture the prominent tile surface for the geometrical resolution and the modified body
flow features such as significant cold air by-pass and entrainment force model, higher air velocity above the tile surface is present.
from aisle center, which were observed in the experimental In the downstream region the flow turns and enters the racks.
results. There is no entrainment or air bypass in this case and all the flow
Air entrainment from the aisle top, aisle sides and aisle center exiting the tile enters the adjacent racks, based on the imposed
is compared for different models in Table 2. Open space between boundary conditions. A low velocity V-shaped region is
the rack and the floor was also included in the model (referred observed near the aisle center for both modified body force model
here as Gap, see Figs. 1(a) and 2(a)) and entrainment through it and the geometrical resolution model. The flow field predicted
is also compared. Both geometrical resolution and the modified with the porous jump model is very different as compared to the
body force model predict similar air flow rates through these other two models; see Fig. 3(b). For the porous jump model,
boundaries. However, the porous jump model shows noticeable the flow uniformly decelerates along the height, as it enters the
difference between the flow rates. adjacent rack.
For the under provisioned case, the porous jump model predicts Note that the computational effort required with the geometri-
air entrainment from the aisle sides (29%) and the aisle top cal resolution model is prohibitively large (4.2 M cells/rack) [7].
(13%) to satisfy the rack air requirement (see Fig. 1(c) for the Whereas the modified body force model can give good prediction
flow field). No entrainment is noted from the aisle center (note even with fairly small computational effort, as it does not requires
negative flow rate, suggesting flow going out of the domain). For the resolution of the tile pores. As such, the model considering
both geometrical resolution and modified body force model, modified body force above the tile surface was extended to room
significant air entrainment is predicted from the aisle center level as it showed better comparison with the geometrical resolu-
(10%). Porous jump model suggest significant amount of flow tion models and experiments.

Fig. 3 Comparison of geometrical resolution, modified body force and porous jump model for middle racks in a contained aisle
for the fully provisioned case in contained aisle condition

011011-4 / Vol. 136, MARCH 2014 Transactions of the ASME

Downloaded From: http://electronicpackaging.asmedigitalcollection.asme.org/pdfaccess.ashx?url=/data/journals/jepae4/929392/ on 05/03/2017 Terms of Use: http://www.asme.or


Fig. 4 Computational domain representative of the Data Center Laboratory (DCL)
at the Georgia Institute of Technology

5 Room Level Modeling I 0:16Retile 0:125 (7)


Figure 4 shows the computational domain, illustrative of the
DCL at the Georgia Institute of Technology. The modeled data For the convective term discretization, first order upwind
center room space with open aisle and with the contained scheme was used. Energy equation was also solved, along with
cold aisle is also shown. The 56 m2 (600 sq. ft) data center has the fluid flow equations, to obtain the heat transfer and mixing
a single cold aisle with three CRAC units and two power distribu- characteristics. Convergence was assumed when the normalized
tion units (PDUs). The cold aisle has 14 generic, 0.61 m  0.61 m residuals for all the variables were less than 104 and the selected
(2 ft  2 ft), perforated tiles (tile size L 0.61 m, 2 ft) arranged velocity monitors have stabilized within a variation of 1%. Com-
in two rows. Total 14 racks are arranged on either sides of mercial CFD software FLUENT was used for the simulations.
the cold aisle. The hot air return vents are placed above the hot The significance of temperature gradient induced buoyant
aisles as shown in Fig. 4. The vent size is 0.61 m  0.61 m forces can be quantified using Richardson number (Ri) [1], see
(2 ft  2 ft). Eq. (8). For the cases investigated here, Ri is about 0.1, suggesting
The computational domain was meshed with full rectilinear that the effect of buoyancy cannot be neglected. Here the effect of
grid. The mesh in the cold aisle was refined such that the cell size buoyancy is considered using the Boussinesq approximation, such
is 2.54 cm  2.54 cm  2.54 cm (1 in.  1 in.  1 in.), which that for solving all equations constant density is considered and a
resulted in 45 K cells/rack. The region outside the cold aisle has buoyancy term is included in the momentum equation based on
cell size of 5.08 cm  5.08 cm  5.08 cm (2 in.  2 in.  2 in.). the local temperature, see Eq. (9). The validity of the Boussinesq
The grid refinement was performed using hanging-node approach. approximation is confirmed using Eq. (10).
As the air entrainment and the temperature profiles in the cold
aisle are of interest here, hence we have chosen refined mesh in bgDTrack Hrack
Ri (8)
the cold aisle (1 in.  1 in.  1 in.) and a coarser mesh (2 in.  2 2
Vtile
in.  2 in.) in the room space. We did the grid sensitivity analysis
by comparing (2 in.  2 in.  2 in.) mesh and (1 in.  1 in.  1 in.) q  q0 g  q0 bT  T0 g (9)
mesh for the full domain and found negligible difference (<2%) bT  T0  1 (10)
in the entrainment rates at the aisle doors. Using (1 in.  1 in.  1
in.) mesh in the full domain led to very high mesh count The layout of the data center is shown in Fig. 5(a). The length
(8  106), and this required significantly longer computational of data center is 8.84 m (29 ft), width is 6.25 m (20.5 ft), and
time (>10). This is the reason why we only used the refined height is 2.64 m (8 ft 8 in.). Mass flow rate boundary condition
mesh in the cold aisle only. For the present case the total grid size is specified for the 14 perforated tiles. All the standard 42 U
was about 1.8  106 cells where the cold aisle has about half of (1 U 4.45 cm) racks are modeled to be housing six IBM
the total cells. Hence, with the use of modified body force model, blade center servers with rack overall dimensions of
full room level simulation was possible with acceptable grid size. 61 cm  122 cm  198 cm (24 in.  48 in.  78 in.). The air inlet
Note that the geometrical resolution model, discussed previously, of each blade center is considered as rectangular opening of
required 4 M cells/rack. Hence, this model could not be used for 41 cm  20 cm (16 in.  8 in.) as shown in Fig. 5(b). Target mass
the full room level flow simulations. flow rate boundary condition is specified at the blade center inlets.
The flow and thermal field was solved using a steady state, fi- In this case the pressure at the blade center inlets is adjusted itera-
nite volume based method and SIMPLE algorithm was used for tively, based on Bernoullis equation; so as to meet the specified
pressure velocity coupling [12]. In SIMPLE algorithm, a relation- flow rate through the boundary, see below equation:
ship between velocity and pressure corrections is used to obtain
the pressure field as well as to enforce mass conservation. Stand- 0:5qave m_ 2  m_ 2req
ard k-e turbulence model was used with inlet turbulence intensity dP (11)
of 5%. In the absence of measured data, we have chosen a value qave A2
of the turbulence intensity in between low turbulence (1%) and
high turbulence (10%). Also for fully developed turbulent flow at Six rectangular openings of 30 cm  30 cm (12 in.  12 in.) are
the core of ducts turbulence intensity can be estimated by the cor- considered at each rack outlet, as shown in Fig. 5(c), to simulate
relation given in Eq. (7) below [13]. For the present case tile the rear door heat exchanger exit. Mass flow rate is specified at
Reynolds number (Retile) (based on tile size (L) and the tile inlet the rack outlet, and it is same as that of the mass flow entering
velocity (Vin)) is 82,400134,000. For the present case the calcu- through the blade center boundaries. A temperature rise with
lated turbulence intensity is about 4%. Note that the correlation is respect to the corresponding blade center inlet temperature is
used as guideline to get an estimate of inlet turbulence intensity. imposed to simulate the heat generated by the racks. For the

Journal of Electronic Packaging MARCH 2014, Vol. 136 / 011011-5

Downloaded From: http://electronicpackaging.asmedigitalcollection.asme.org/pdfaccess.ashx?url=/data/journals/jepae4/929392/ on 05/03/2017 Terms of Use: http://www.asme.or


Fig. 5 (a) Data center layout, ((b)(c)) rack modeling, and (d) modified body force
term specification region

Fig. 6 Comparison of open and contained aisle for middle racks (4,11) with modi-
fied body force and porous jump models

Fig. 7 Comparison of open and contained aisle for end tiles (1,14) with modified
body force and porous jump models

present case temperature rise of 10  C is specified corresponding The velocity contours along the center plane crossing the end
to a rack heat load of 15 kW with rack air flow rate of 1.224 m3/s tiles 1 and 14 is shown in Fig. 7. For the MBF model with open
(2594 CFM). Region inside the racks, CRAC units and PDUs aisle, the flow field is very different as compared to that corre-
is not considered and no mesh is present in it. The modified sponding to the middle tiles (compare Figs. 6(a) and 7(a)). This
body force source term is specified in the region of may be due to air entering the cold aisle near the bottom portion of
56 cm  56 cm  15 cm (22 in.  22 in.  6 in.) above the tile sur- the aisle. This can be seen as high velocity region near the bottom
face for all the tiles, see Fig. 5(d). of the aisle, see Fig. 7(a). This can be further understood from the
Figure 6 shows the velocity contour plots for fully provisioned temperature contours shown in Fig. 10 later. For the contained
case along the center plane crossing the middle tiles 4 and 11. aisle, the velocity field at the middle and the end of the aisle is simi-
From the figure we observe that the flow field is almost similar for lar due to absence of air entrainment; see Fig. 7(b). With PJ model
both open and contained aisles. The modified body force model the velocity field is similar for both open and contained aisle, sug-
predicts a low velocity (V-shaped) region near the aisle center gesting much lower entrainment as compared to MBF model.
which is absent for the porous jump model (PJ) results. The flow Figure 8 shows the temperature contours at the center plane
field is similar to that calculated for the case simulating middle along the length of the cold aisle for the under provisioned case.
racks in a contained aisle as shown in Fig. 3. The balance rack air requirement is met by hot air recirculation

011011-6 / Vol. 136, MARCH 2014 Transactions of the ASME

Downloaded From: http://electronicpackaging.asmedigitalcollection.asme.org/pdfaccess.ashx?url=/data/journals/jepae4/929392/ on 05/03/2017 Terms of Use: http://www.asme.or


Fig. 8 Comparison temperature contours of (a) modified body force and (b)
porous jump model along the center plane of open cold aisle for the under
provisioned case

Fig. 9 Comparison of (a) modified body force and (b) porous jump model along
the center plane of partially contained cold aisle (only the top curtain deployed)
and under provisioned case

Fig. 10 Comparison temperature contours of modified body force and porous


jump model along the center plane of open cold aisle for the fully provisioned case

from the room. From the figure we can observe that MBF model may be because the top curtain can curtail the hot air rising and
suggests significantly higher hot regions as compared to the PJ escaping from the aisle top and the entrainment effects are
model, possibly due to higher air entrainment. This result suggests suppressed.
that the PJ model may underestimate the server inlet temperatures The temperature contours for the fully provisioned case is
and can lead to flawed air flow pattern design. shown in Fig. 10. From the figure we note that, even though the
With partial containment, where only the top curtain is cold air supply and demand are equal, hot regions are present sug-
deployed while the aisle doors are opened to allow free access to gesting hot air is entrainment into the cold aisle (see Fig. 10). The
the cold aisle (refer to Fig. 4), the temperature field suggests MBF model suggests much higher air entrainment as compared to
improvement in cold air supply scheme. There is minor difference the PJ model similar to that observed for the under provisioned
between the MBF and PJ models for this case (see Fig. 9). This case (also, see Fig. 8).

Journal of Electronic Packaging MARCH 2014, Vol. 136 / 011011-7

Downloaded From: http://electronicpackaging.asmedigitalcollection.asme.org/pdfaccess.ashx?url=/data/journals/jepae4/929392/ on 05/03/2017 Terms of Use: http://www.asme.or


Fig. 11 Comparison of (a) modified body force and (b) porous jump model along
the center plane of cold aisle for partially contained aisle for the fully provisioned
case

Figure 11 shows the temperature contours at center plane along


Table 3 Comparison of gross air entrainment through open
the length of the partially contained cold aisle for the fully provi-
doors for modified body force and porous jump models
sioned case. In this case only, the top curtain is deployed while
Air flow/rack flow (%) Modified body force Porous jump the doors are opened (refer Fig. 4). From the figure we observe
that with partial containment, the hot air recirculation is signifi-
Under provisioned case cantly reduced as compared to the open aisle case. In this case,
Open aisle 35.53 24.78 the hot air from the room enters the cold aisle from bottom portion
Top curtain closed 40.21 40.48 of door openings and escapes from the top portion of the door
Fully provisioned case openings. Hot air entrainment can be inferred from the higher
Open aisle 27.43 13.40 temperature region near the doors as show in Fig. 11. The air flow
Top curtain closed 16.04 8.83 emerging from the tiles will entrain the room air, and hence will
lead to entrance of room air from the bottom portion of the door.

Fig. 12 Server inlet temperature for the under provisioned case

011011-8 / Vol. 136, MARCH 2014 Transactions of the ASME

Downloaded From: http://electronicpackaging.asmedigitalcollection.asme.org/pdfaccess.ashx?url=/data/journals/jepae4/929392/ on 05/03/2017 Terms of Use: http://www.asme.or


Fig. 13 Server inlet temperature for the fully provisioned case

Note that in this case no net air entrainment can be present inlet temperatures corresponding to the six blade center for each
because the supplied air is matched with the rack air requirement. rack (see Fig. 5(b) for more details). Each bar in the figure
For this case also the MBF model suggests higher gross entrain- shows the mass weighted average temperature of a modeled
ment as compared to the PJ model. This investigation shows that blade center inlet. The bars from left to right represent servers
for the fully provisioned case, partial containment can be an effec- from bottom to top.
tive means to supply cold air to servers and curtail hot regions in From Fig. 12 we observe that there is a large variation in server
the cold aisle. inlet temperature (25  C) for the under provisioned case. The
The entrainment rates through the open doors for various cases MBF model suggests much higher server inlet temperatures as
investigated here are given in Table 3. For open aisle, we observe compared to the PJ model; possibly due to higher air entrainment
that the hot air entrainment rates are significantly higher with the rates (see Table 2). The results also suggest that the use of PJ
MBF model as compared to the PJ model. For the under provi- model may not provide appropriate prediction while designing air
sioned case, with the MBF model, the entrainment through the flow schemes. Use of partial containment suggests reduced server
doors is 36%, whereas, with the PJ model it is 24%. With par- inlet temperatures with MBF model. This shows that the use of
tial containment the balance air has to enter through the open partial containment can be an effective for cold air management
doors, hence the entrainment rates for the PJ and MBF models are to reduce high server inlet temperatures. This difference is not no-
similar in this case (40%). ticeable with the use of PJ model.
For the fully provisioned case, with both open and partially For the fully provisioned case, much lower server inlet temper-
contained aisles significant gross entrainment rates are present. atures are observed (see Fig. 13). With open aisle, due to hot air
However, note that with partially contained aisles the air mostly entrainment, the server inlet temperatures are higher for the end
enters through the bottom portion of the open door and escapes racks (rack# 1, 14, 7, and 8). Refer to Fig. 5(a) for the layout of
through the top portion. Hence the influence of the hot air on sever the data center. In this case also the MBF model predicts higher
inlet temperatures is expected to be minimal. server inlet temperatures as compared to the PJ model. With use
ASHRAE has recommended server air inlet temperature range of partial containment the server inlet temperatures are signifi-
from 18 to 27  C [14] for effective operation of the servers. To cantly reduced and the variation is within 5  C suggesting effec-
meet this recommendation, lower variation in the server inlet tive air delivery scheme for this case.
temperatures as well as appropriate supplied air temperature is
required. However, with higher hot air recirculation the tempera-
ture range may be higher, posing challenge to meet the ASH- 6 Conclusions
RAE recommendation. Figures 12 and 13 shows the computed Effective air delivery schemes require appropriate choice of
server inlet temperatures for the cases with under provisioned computational models to accurately predict the flow and thermal
and fully provisioned cases, respectively. Note that each rack is fields in a data center. For a single rack, it was observed that the
modeled to have six blade centers and hence there are six server modified body force model closely predicted the major flow

Journal of Electronic Packaging MARCH 2014, Vol. 136 / 011011-9

Downloaded From: http://electronicpackaging.asmedigitalcollection.asme.org/pdfaccess.ashx?url=/data/journals/jepae4/929392/ on 05/03/2017 Terms of Use: http://www.asme.or


features of the flow field. The modified body force model also Acronyms
quantitatively compared well with the model which includes the
tile pore geometrical details. The generally used porous jump BC blade center
model failed to capture the prominent flow features and suggested CFD computational fluid dynamics
much lower air entrainment rates. For full room level modeling, CRAC computer room of air conditioning unit
with open aisle the modified body force model predicted much DCL data center laboratory
higher air entrainment rates and higher server inlet temperatures EXP experiments
as compared to the porous jump model. This suggests that the po- MBF modified body force model
rous jump model can lead to underestimation of the server inlet PIV particle image velocimetry
temperatures which are critical for effective data center design. PJ porous jump model
Partial containment promises to be an effective air delivery solu- S server
tion as it prevents the hot air to rise and escape from the aisle top.
In this case, only curtain at the aisle top is deployed leaving the References
aisle doors open for easy access. [1] Joshi, Y., and Kumar, P., 2012, Energy Efficient Thermal Management of Data
Centers, Springer, New York.
[2] Schmidt, R., Vallury, A., and Iyengar, M., 2011, Energy Savings Through Hot
Acknowledgment and Cold Aisle Containment Configurations for Air Cooled Servers in Data
This research is supported by the National Science Foundation Centers, Pacific Rim Technical Conference and Exposition on Packaging
and Integration of Electronic and Photonic Systems (InterPACK), Portland,
Industry/University Cooperative Research Center on Energy OR, July 68, ASME Paper No. IPACK2011-52206.
Smart Electronic Systems (ES2). Additional support from Degree [3] Shrivastava, S. K., Calder, A. R., and Ibrahim, M., 2012, Quantitative
Controls, Inc., and Triad Tiles, Inc., is acknowledged. Comparison of Air Containment Systems, 13th IEEE Intersociety Conference
on Thermal and Thermomechanical Phenomena in Electronic Systems
(ITherm), San Diego, CA, May 30June 1, pp. 6877.
Nomenclature [4] Takahashi, M., Uekusa, T., Kishita, M., and Kaneko, H., 2008, Aisle-
Capping Method for Airflow Design in Data Centers, IEEE 30th International
A area of the surface Telecommunications Energy Conference (INTELEC 2008), San Diego, CA,
Atile total tile area September 1418.
D pore size [5] Gondipalli, S., Sammakia, B., Bhopte, S., Schmidt, R., Iyengar, M. K., and Murray,
B., 2009, Optimization of Cold Aisle Isolation Designs for a Data Center With
dP pressure adjustment to achieve target mass flow rate Roofs and Doors Using Slits, Pacific Rim Technical Conference and Exposition on
F fractional tile open area or tile porosity Packaging and Integration of Electronic and Photonic Systems (InterPACK), San
g gravitational acceleration Francisco, CA, July 1923, ASME Paper No. InterPACK2009-89203.
I inlet turbulence intensity [6] Gondipalli, S., Bhopte, S., Sammakia, B., Iyengar, M. K., and Schmidt, R.,
2008, Effect of Isolating Cold Aisles on Rack Inlet Temperature, 11th Interso-
K pressure loss factor ciety Conference on Thermal and Thermomechanical Phenomena in Electronic
L perforated tile size Systems (ITHERM 2008), Orlando, FL, May 2831, pp. 12471254.
m_ current computed mass flow rate [7] Arghode, V. K., Kumar, P., Joshi, Y., Weiss, T., and Meyer, G., 2013, Rack
m_ req required mass flow rate Level Modeling of Air Flow Through Perforated Tile in a Data Center, ASME
J. Electron. Packag., 135(3), p. 030902.
Ri Richardson number [8] Abdelmaksoud, W. A., Khalifa, H. E., Dang, T. Q., Elhadidi, B., Schmidt, R.
Sx momentum source term for x momentum equation R., and Iyengar, M., 2010, Experimental and Computational Study of
T air temperature Perforated Floor Tile in Data Centers, 12th IEEE Intersociety Conference on
T0 operating temperature Thermal and Thermomechanical Phenomena in Electronic Systems (ITherm),
Las Vegas, NV, June 25.
Vin tile approach velocity [9] Freid, E., and Idelchik, I. E., 1989, Flow Resistance, a Design Guide for Engi-
Vneck velocity at the vena contracta neers, Hemisphere, New York.
Vpore average velocity across the pores [10] Arghode, V. K., and Joshi, Y., 2013, Modeling Strategies for Air Flow
x direction along the height of the rack Through Perforated Tiles in a Data Center, IEEE Trans. Compon., Packag.
Manuf. Technol., 3(5), pp. 800810.
b volumetric coefficient of thermal expansion [11] Kumar, P., and Joshi, Y., 2010, Experimental Investigations on the Effect
DM excess momentum above the tile of Perforated Tile Air Jet Velocity on Server Air Distribution in a High Density Data
DP pressure drop Center, 12th IEEE Intersociety Conference on Thermal and Thermomechanical
DTrack temperature rise across rack Phenomena in Electronic Systems (ITherm), Las Vegas, NV, June 25.
[12] Patankar, S. V., 1980, Numerical Heat Transfer and Fluid Flow, Hemisphere,
DHrack characteristic length scale (height of rack) New York.
q air density [13] FLUENT 6.3 Users Guide, 2006, Fluent Inc., Lebanon, NH.
qave average density at the surface [14] ASHRAE Technical Committee 9.9, 2011, Thermal Guidelines for Data Proc-
q0 operating (constant) density essing EnvironmentsExpanded Data Center Classes and Usage Guidance,
American Society of Heating, Refrigerating and Air-Conditioning Engineers,
8 volume of momentum source specification region Inc., Atlanta, GA.

011011-10 / Vol. 136, MARCH 2014 Transactions of the ASME

Downloaded From: http://electronicpackaging.asmedigitalcollection.asme.org/pdfaccess.ashx?url=/data/journals/jepae4/929392/ on 05/03/2017 Terms of Use: http://www.asme.or