You are on page 1of 42

HVAC System Design

Mark Hydeman, P.E., FASHRAE


Taylor Engineering, LLC
mhydeman@taylor-engineering.com

Taylor Engineering, LLC

How do you effectively fight a fire?


it takes 2,000 to 3,000 times the
volume of air to cool what you can
with water!
With air, or

Taylor Engineering, LLC

with water?

State of the present: with air

Taylor Engineering, LLC

Air system design overview


Data center layout
Airflow configurations

Distribution:

overhead or underfloor
Control: constant or variable volume

Airflow issues
Economizers
Humidity control issues

Taylor Engineering, LLC

Data center layout


Server airflow front to
back or front to back and
top are recommended

Cold Aisle
Hot Aisle

2004, American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc. (www.ashrae.org). Reprinted by permission from
ASHRAE Thermal Guidelines for Data Processing Environments. This material may not be copied nor distributed in either paper or digital form
without ASHRAEs permission.

Taylor Engineering, LLC

Data center layout

Underfloor Supply

Cold Aisle
Hot Aisle
Only 1 pressure
zone for UF!
2004, American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc. (www.ashrae.org). Reprinted by permission from
ASHRAE Thermal Guidelines for Data Processing Environments. This material may not be copied nor distributed in either paper or digital form
without ASHRAEs permission.

Taylor Engineering, LLC

Data center layout


You can
incorporate VAV
on each branch
Overhead Supply

Cold Aisle
Hot Aisle

2004, American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc. (www.ashrae.org). Reprinted by permission from
ASHRAE Thermal Guidelines for Data Processing Environments. This material may not be copied nor distributed in either paper or digital form
without ASHRAEs permission.

Taylor Engineering, LLC

Typical temperature profile with UF supply


Too hot

Too hot

Just right

Too cold
Elevation at a cold aisle looking at racks
There are numerous references in ASHRAE. See for example V. Sorell et al; Comparison of
Overhead and Underfloor Air Delivery Systems in a Data Center Environment Using CFD
Modeling; ASHRAE Symposium Paper DE-05-11-5; 2005
Taylor Engineering, LLC

Typical temperature profile with OH supply


Too warm

Too warm

Just right

Elevation at a cold aisle looking at racks

Taylor Engineering, LLC

Hot aisle lid

Aisle capping
End cap
Cold Aisle Caps
APC reprinted with permission

Cold Aisle
Hot Aisle

2004, American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc. (www.ashrae.org). Reprinted by permission from
ASHRAE Thermal Guidelines for Data Processing Environments. This material may not be copied nor distributed in either paper or digital form
without ASHRAEs permission.

Taylor Engineering, LLC

10

Aisle capping
LBNL has recently performed
research on aisle capping
Cold Aisle Caps

Cold Aisle
Hot Aisle

2004, American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc. (www.ashrae.org). Reprinted by permission from
ASHRAE Thermal Guidelines for Data Processing Environments. This material may not be copied nor distributed in either paper or digital form
without ASHRAEs permission.

Taylor Engineering, LLC

11

Overhead (OH) vs. Underfloor (UF)


Issue
Capacity
Balancing

Overhead (OH) Supply


Limited by space and aisle velocity.
Continuous on both outlet and branch.

Control

Up to one pressure zone by branch.

Temperature
Control
First Cost
Energy Cost
Aisle Capping

Most uniform.

Underfloor (UF) Supply


Limited by free area of floor tiles.
Usually limited to incremental changes by
diffuser type. Some tiles have balancing
dampers. Also underfloor velocities can
starve floor grilles!
Only one pressure zone per floor, can
provide multiple temperature zones.
Commonly cold at bottom and hot at top.

Best (if you eliminate the floor).


Best.
Hot or cold aisle possible.

Generally worse.
Worst.
Hot or cold aisle possible.

Taylor Engineering, LLC

12

Airflow design disjoint


IT departments select servers and racks
Engineers size the fans and cooling
capacity
Whats missing
in this picture?

Taylor Engineering, LLC

13

Airflow with constant volume systems

Taylor Engineering, LLC

Hot spots
Higher hot aisle
temperature
Possible equipment
failure or degradation

V HVAC _ Supply V Servers

14

Airflow with constant volume systems

Taylor Engineering, LLC

Least hot spots


Higher air velocities
Higher fan energy
Reduced economizer
effectiveness (due to
lower return
temperatures)

V HVAC _ Supply V Servers

15

Airflow with constant volume systems


Note most of these observations apply to
overhead and underfloor distribution
With constant volume fans on the servers
you can only be right at one condition of
server loading!
The solution is to employ variable speed
server and distribution fans

Taylor Engineering, LLC

16

Airflow with variable volume systems

Partial flow condition

Taylor Engineering, LLC

Best energy performance


but difficult to control

V HVAC _ Supply V Servers

17

How Do You Balance Airflow?

Spreadsheet
CFD
Monitoring/Site
Measurements

Image from TileFlow


http://www.inres.com/Products/TileFlow/tileflow.html,
Used with permission from Innovative Research, Inc.

Taylor Engineering, LLC

18

Thermal report

From ASHRAEs Thermal Guidelines for Data Processing Environments


Taylor Engineering, LLC

19

Whats the server airflow?


SUN
V490

SUN
V240

DELL

DELL
2850

6850

num fans
9
3 n/a
n/a
total CFM (max)
150
55.65
42
185
total CFM (min)
27
126
fan speed
single speed variable
2 speed
2 speed
fan control
n/a
inlet temp. 77F inlet 77F inlet
Form Factor (in U's)
5
2
2
4
heat min config (btuh)
798
454
heat max config (btuh)
5,459
1,639
2,222
4,236
heat max (watts)
1,599
480
651
1,241
dT min config
13
3
dT max config
33
27
48
21
servers per rack
8
21
21
10
CFM/rack (hi inlet temp)
1,200
1,169
882
1,850
CFM/rack (low inlet temp)
1,200
567
1,260
max load / rack (kW)
13
10
14
12

Taylor Engineering, LLC

20

Best air delivery practices

Arrange racks in hot aisle/cold aisle configuration


Try to match or exceed server airflow by aisle

Get thermal report data from IT if possible


Plan for worst case

Get variable speed or two speed fans on servers if possible


Provide variable airflow fans for AC unit supply

Also consider using air handlers rather than CRACs for improved
performance (to be elaborated on later)

Use overhead supply where possible


Provide aisle capping (preferably cold aisles, refer to LBNL
presentation for more details)
Plug floor leaks and provide blank off plates in racks
Draw return from as high as possible
Use CFD to inform design and operation

Taylor Engineering, LLC

21

Air-side economizer

Taylor Engineering, LLC

22

Air-Side Economizer issues

Hygroscopic dust
LBNL

is doing some research on this

Design humidity conditions


See

following slides

Taylor Engineering, LLC

23

Design conditions at the zone

2005, American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc. (www.ashrae.org). Reprinted by permission from
ASHRAE Design Considerations for Data and Communications Equipment Centers. This material may not be copied nor distributed in either
paper or digital form without ASHRAEs permission.

Taylor Engineering, LLC

24

San Francisco
40

.020
80

.019

75

.018

.017

35
75

70

WE
TB

Normal Temperature

.016
UL
BT
EM
PE
RA

.015
TU
R

I-P Units
30

16 FEET

65

E-

70

Upper Allowed Humidity Limit

BAROMETRIC PRESSURE: 29.904 in. HG

.014

.013

.012

65

25

Class1;Recommend

45

35
13.0

40

.008

.007

.006

50

%
60

40

25

IR

55

A
RY

%
70

D
LB.

45

15

%
90

%
80

.009

PER

60

50

.010

Design Target

55

20

.011

.FT.
- CU

Negligible time of possible


concern for humidification

60

UME
VOL

Weather Hours
360 to 321
320 to 281
280 to 241
240 to 201
200 to 161
160 to 121
120 to 81
80 to 41
40 to 1

14.0

San Francisco Climate Data Bins


with Data Center Guideline Zones

Class1;Allow

.005
15%

50%

.004

NEBS;Recommend

40%

35

10

.003
30%

30

.002

20%

10% RELATIV

30

35

HUMIDITY RATIO - POUNDS MOISTURE PER POUND DRY AIR

PSYCHROMETRIC
CHART

40

45

50

Chart by: HANDS DOWN SOFTWARE, www.handsdownsoftware.com

Taylor Engineering, LLC

55

Lower Allowed Humidity Limit (20%RH)

E HUMIDITY

60

.001

65

70

75

80

85

90

95

100

DRY BULB TEMPERATURE - F

25

Los Angeles
40

.020

80
75

.019

.018

.017

35

75

70

WE

TB

.016

UL
BT

EM
PE
RA
TU

Normal Temperature

.015

RE

I-P Units

30

105 FEET

65

.014

70

Upper Allowed Humidity Limit

BAROMETRIC PRESSURE: 29.808 in. HG

.013

.012

60

40

13.0

35

.006

50

Class1;Allow

.004

NEBS;Recommend

40%

.003

.002

20%
10% RELATIV

35

.005
15%

50%

30%

30

30

.007

IR

45
35

10

Class1;Recommend

%
60

40

.008

A
RY

%
70

25

D
LB.

45

.009

ER

55

%
80

15

%
90

50

.010

Design Target

55

20

.011

P
.FT.
- CU

Only a few hours of possible


concern for humidification

65

25

UME
VOL

Weather Hours
396 to 353
352 to 309
308 to 265
264 to 221
220 to 177
176 to 133
132 to 89
88 to 45
44 to 1

60

14.0

Los Angeles Climate Data Bins


with Data Center Guideline Zones

HUMIDITY RATIO - POUNDS MOISTURE PER POUND DRY AIR

PSYCHROMETRIC
CHART

40

45

50

Chart by: HANDS DOWN SOFTWARE, www.handsdownsoftware.com

Taylor Engineering, LLC

55

Lower Allowed Humidity Limit (20%RH)

E HUMIDITY

60

.001

65

70

75

80

85

90

95

100

DRY BULB TEMPERATURE - F

26

Sacramento
40

.020

80
.019

75

.018

.017

35

75

70

WE
T

Normal Temperature

.016

BU
LB
TE
MP
ER
AT

.015

UR

I-P Units
30

26 FEET

65

E-

.014

70

Upper Allowed Humidity Limit

BAROMETRIC PRESSURE: 29.893 in. HG

.013

.012

65
25

45
35

13.0

40

.008

IR

Class1;Recommend

.007

.006

50

%
60

40

25

A
RY

%
70

D
LB.

45

15

55

%
80

.009

PER

60

%
90

50

.010

Design Target

55

20

.011

.FT.
- CU

Negligible time of possible


concern for humidification

60

UME
VOL

Weather Hours
270 to 241
240 to 211
210 to 181
180 to 151
150 to 121
120 to 91
90 to 61
60 to 31
30 to 1

14.0

Sacramento Climate Data Bins


with Data Center Guideline Zones

Class1;Allow

.005
15%

50%

.004

NEBS;Recommend

40%

35

10

.003

30%

30

.002

20%
10% RELATIV

30

35

HUMIDITY RATIO - POUNDS MOISTURE PER POUND DRY AIR

PSYCHROMETRIC
CHART

40

45

50

Chart by: HANDS DOWN SOFTWARE, www.handsdownsoftware.com

Taylor Engineering, LLC

55

Lower Allowed Humidity Limit (20%RH)


E HUMIDITY

60

.001

65

70

75

80

85

90

95

100

DRY BULB TEMPERATURE - F

27

Lower humidity limit

Mitigate electrostatic discharge (ESD)

Recommended procedures

Recommended equipment

Grounding wrist straps on racks


Grounded plate for cables
Grounded flooring
Servers rated for ESD resistance

Industry practices

Personnel grounding
Cable grounding

Telecom industry has no lower limit


The Electrostatic Discharge Association has removed humidity control as a primary
ESD control measure in their ESD/ANSI S20.20 standard

Humidity controls are a point of failure and are hard to maintain


Many data centers operate without humidification
This needs more research

And for some physical media (tape storage, printing and bursting)

Old technology not found in most data centers


It is best to segregate these items rather than humidify the entire data center

Taylor Engineering, LLC

28

ESD control: floor grounding

Image from Panduit, reprinted with permission

Taylor Engineering, LLC

29

Water-Side Economizer
Integrated

Heat
Exchanger in
series with
chillers on
CHW side

Economizer Summary
Air-Side Economizers

Provides free cooling when


dry-bulb temperatures are
below 78F-80F.
May increase particulates
(LBNL research indicates this
is of little concern).
Should be integrated to be
most effective.
Improves plant redundancy!
Can work in conjunction with
water-side economizers on
data centers!
Need to incorporate relief.

Water-Side Economizers

Provides low energy cooling


when wet-bulb temperatures
are below 55F-60F.
Avoids increased particulates
(and low humidity if that
concerns you).
Should be integrated to be
most effective (see previous
slide).
Improves plant redundancy!
Can work in conjunction with
air-side economizers on data
centers!

Both are proven technologies on data centers!


Taylor Engineering, LLC

31

A case study of two designs

Collocation facility in the


Bay Area
Side by side designs in
same facility over two
phases
Motivation for the second
design was to reduce
cost

Taylor Engineering, LLC

Case study was


developed by Lawrence
Berkeley National
Laboratory (LBNL)

Data Centers 8.1 and 8.2

Both sections at ~30%


build-out during
monitoring

32

A tale of two designs: overview

Phase 1 Data Center (8.1)

26,200 ft2
27 W/ft2 design
Traditional under-floor design
with CRAC units
Air-cooled DX
Humidity controls (45%-55%)

Phase 2 Data Center (8.2)

Taylor Engineering, LLC

73,000 ft2
50 W/ft2 design
Under-floor supply from
central AHUs with CHW coils
Water-cooled plant
Air-side economizers
No humidity controls

33

A tale of two designs: a closer look


Normalized efficiency metric: cooling
1.60

Phase 1 Data Center (8.1)

1.40

Phase 2 Data Center (8.2)

kWcooling _ systems
kWservers
~1/4 of the
normalized energy

Normalized energy

1.20
1.00
0.80
0.60
0.40
0.20
Computer Loads

UPS Losses

HVAC

Lighting

Data normalized to computer loads


Taylor Engineering, LLC

34

A tale of two designs: results

Phase 1 Data Center (8.1)


Around 2x the HVAC
installed cost ($/ft2)
Around 4x the energy bills
(when normalized to server
load)
Acoustical problems
Higher maintenance costs
Lost floor space in data
center due to CRACs

Taylor Engineering, LLC

Phase 2 Data Center (8.2)


Preferred by the facility
operators and data center
personnel

35

Two data centers: summary

What made the difference?


Airside

economizers
No humidity controls
Water-cooled chilled water system
AHUs instead of CRAC units

Taylor Engineering, LLC

36

Custom CRAH Unit (Large)

Taylor Engineering, LLC

37

Example CRAH Unit Comparison


Model
Budget Cost
Number of units
net total cooling (btuh)
net sensible (btuh)
sensible (tons)
CFM
SAT
airside dT
Internal SP
no. fans
fan type
no. motors
HP/motor
total HP
BHP/motor
Unit BHP
unit width
depth
height
filter type
Water PD (ft)
CHW dT
GPM
Total GPM
Total BHP

Taylor Engineering, LLC

Option 1
Option 2
Std CRAC
Custom Model 1
Custom Model 2
$
16,235 $
23,000 $
41,000
21
13
4
434,900
410,000
841,000
397,400
399,000
818,000
33.1
33.3
68.2
16,500
25,000
50,000
49.90
59.30
59.00
25.10
15.70
16.00
2
0.8
0.8
1.8
1.8
3
3
2
Centrifugal
Plenum
Plenum
1
3
2
15
5
15
15
15
30
15
4.7
11.5
15
14.1
23
122
122
122
35
36
72
76
156
168
ASHRAE 20%
MERV 13
MERV 13
13.5 ft
11.1
11.1
14F
20
20
66.80
44.00
88.00
1,403
924
66%
315
275
87%

38

Example CRAH Unit Comparison

34% less water flow


13% less fan energy

Excess fan capacity on new units


36% higher cost for units, but

More if you consider the supply air temperature and airflow issues

Fewer piping connections


Fewer electrical connections
Fewer control panels
No need for control gateway
Can use the existing distribution piping and pumps (case study)
Can use high quality sensors and place them where they make sense

Possibly less turbulence at discharge?

Taylor Engineering, LLC

39

Air cooling issues

Limitations on the data densities served (~200w/sf)

Working conditions

Air delivery limitations


Real estate
Hot aisles are approaching OSHA limits

Costly infrastructure
High energy costs
Management over time
Reliability

Loss of power recovery


Particulates

Taylor Engineering, LLC

40

Take Aways

Use air- or water-side economizers where possible


Consider personal grounding in lieu of humidification
Consider AHUs as an alternative to CRACs
Consider VSDs on fans, pumps, chillers and towers
Refer to ASHRAE, LBNL and Uptime Institute for more
recommendations

Taylor Engineering, LLC

41

State of the future: cooling with liquid

Taylor Engineering, LLC

42

You might also like