Professional Documents
Culture Documents
Best:
Version
Student: Cuide
• •
DC110
Data Center
Design
Best Practices
o Copyrlgh l 201 0 by BICSI
All righlS re\!'"rwd. No pan of this workbook may be used, reproduced, or uan'>min ed in any form or
by any means, electronic or mechanical. including photocopymg, recording. or by any information
storage and retrieval system, without prior agreement and "'fltten per mission from t he pu blisher,
except for the inclusion of brief quotat ions i n a review.
Thecoments of th is wo rkbook are subject to revision without nol ice due to continued progress in
met hodology, design, and manufacturing.
THE GUIDE IS PART OF TH E BleSI DESIGN COURSE CURRICULUM AND IS
PRESENTED H 1\ WITHOUT WARRANTY OF ANY KIND, RESPECTING THE
CONTENTS OF THIS GUIDE INCLUDING BUT NOT UMITED TO IMPUED
WARRANTIES FOR THE GUIDE'S QUALITY, PERFORMANCE, MERCHANTABILITY,
OR FITNESS FOR ANY PARnCULAR PURPOSE. BI CSI SHAll NOT BE LIABLE TO THE
PURCHASER OR ANY OTHER ENTITY WITH RESPECT TO ANY LIABILITY, lOSS, OR
DAMAGE CAUSED DIRECTlY OR INDIRECTLY BY TH IS GUIDE.
All brand names, trademarks, and registered trademarkS are th e property of their respective holders.
BleSI lX'odd He.ld<juarlcr ~
8610 Hidden River Parkway
Tampa, Fl33637· 1000 USA
800.242:.7405;+ 1813.979. 1 991 ; ~x "' 1813 .97 1.4311
\Neb si ll': www.blcsi.org
S1"dom Guido o..t.t Centw& & Tho Datolgn Pmco..
This Is;, c:ourse em Oat.l Center Design and Best Practices, the plimery
source for the matenal presented here Is:
...
llIat Pl""' .... y Pro"ft)'-OG NootCop,
Stud ..... GukIo Data C ........ & Tn. Doslgn Proe ...
<4OI.aiC$/'
Slude", G.. de
VOK:~&
Power Data
C.bling
Autom.~on &.
Cooling Control
~ Syslenu
............ ,
I
............
D
Et'u.J1K1 roo!'I()l DltAQtIll:efd~ .l1li
_tI\Tif'rC.Jl ~ m .
,
"\ / "
0
~room,allll
Tw"""m~~1
;
I
can,llwbl. Raon
The Compuler Room 2.11
The heart of a data center is a computer mom housing IT equipment
racks, a main cabling distribution "",,,, and often a special area for
gigital information storage (or Storage Area Network (SAN»
,- - - ;i~c;.,;;,,;-~ - -- I
I I
- '-
"~-·a
I M ""'. ...-
~""r=Q '''''A. t--~"'l{It'U
::lo" .... """M I
-
OfIIbot . c.-~
c - t .• ~
.i
,.1-hI<\.I~'
: e:. .."i!
I
- --
.......
( AI .. ,..
MaiI1 Oi$(ribullon
- ___ I
,---- .... ---,
I
I -, """-
""'" ~i~ I
I
Te l..ooro RMm
n.r..:.tII~'" A,... ~~ . I
I1r.ot11Ont.1ll Cro!;1li'COOI'1llU
,oa,:, e eq.rp!T'el1 ~
""'"'" /l,t.:lln elU!!l!f-Cllm-..'1,i
~1f1. 1 Hdril'QI'IUoI
C:jIm4If £qu~ 0I-
o.JJWn:CI~DI'l
I
I
QI~,-r.-nt'\I ""..t I
. I ~~Ir' l~"'~R~_1
,,/
"
/ ~laI
.,..DIroIll'ilMltivn
ijMi:....,1,fI 1 Ct~1WI9d
~
.....
OI .... lmutioII
HlilrIllll!Ml c~·c.:..~
." ""A():,... f._"".
.,~' ! "
,,1
,,':
,J
student Guide Data Coni". & Tho Doo;go P"' .....
"1.1-0", I
~ l' ''' EI ",,~
~~ ....'''. lon:~" olI,. NC1'.I.oro I t ".crf~~ ~ ·
~~~r~L=I.l.;lln=
""',
MQ~ ~.",
ep,i l ~ r
!
';'OE;<r
I. l;<=Im
~ I .r .....,.,., ....' mt~' ""~ ~
Off-sfte 2.11
Off'slte components of the d. ", centerolten Include:
• Power and Telecom Access Providers and Feeds
• External 'Generators
• Emmal Thertnil l St oroge I Water Tanks
• Offslte - Data Storagel Dis. ster RecOIIery F"dlities
Q - H.I.......,.
11ft ~_
...
IICSI Prapri.t;;uy ~ frlot tOM!
Student ~Ide
The data ~ef1ter design process needs to refiect the different n~ture of
data ce'nter requirements
• Data center design should be integrated rlIther than linear to
ensure effldent use of IT, power, and cooling assets
• Maximum coordination is required during 111e earliest plMnlng
stages for efficient data center power usage. cooling. and cabling
arcn~res
AvaHablllty 2.20
oDl'O'IO Bidil'
:>-10
BICSI p ...... taty pr,~ ~otCo~'1
StudonlGuldo 0 ... c.nhM'& & Tho Dooign PrOCH.
5<:alability 2.21
The potent i~1 for growth must also be consi~ere~ in ttle data center's
design
• Large businesses typically experience 50% growth In data per year
• Power reqUirements for a data center can dou ble over the course of
2 years
• IT equipment will often completely turnover over the course of 3
yea... IL . ,f
. ,.. r\rk .... ~:~~,.o.'\ ~ -:. ,J(t H.~~ S(....1.:... ''() (,.tr"'h_~ · )
Security
Efficiency/Cosl 2.23
Data center design must consider not ",,'y initial costs, but also the
total cost of a fadllty over Its IIfetlme. Considerations r<!garoing
effiCiency that designers should address Inelude:
• How much redundilncy is it worth p"ying for to ensure availabil~y?
Risk analysis i. the process of balancing fulllre costs <>f downtime with
til e present costs <>1 p~ntion
Wh~e customers and clients may seek "6 nines" In terms of reliability.
risk analysis I~ the process of balancing that desire against the high
costs assodated with Insuring limited downtime
O2IItO .1CSi'
student Guide Data ~ & The D.,.lgn Proceso
."
a!csIP~~NotcQFIY
StucMnt Guld. 0$ C."tar& & Tho DesIgn Pro:esl
H.
811:11 f'rupri.w.y p~ Mot Copy
Student <kIld. ON Centers &Thll o.sign P'tocMI
Alter ~ clesi9" solution has been chosen, the final stage of data center
design takes place which Includes;
• Construction
, Commjssioning (· . . . 'c(Ii---"~~'d' +""" ;" O'~» [).'l~J"i)
Conflnlction 2.35
The construction phase of design typically lndudes the coordination of
• Structural proje<ts
• Mechanical projects
• Electrical prOjects
• cabling projects
Commissioning Includes;
• In~lI.tion of vendor equipment
• Fi<!ld testirlll equipment
• Oevekll>ment and lmplementatlon of oper-..tio",,1 and maintenance
procedures
Dot> C."".r& & Tho Dool911 Proan
Review Questions
1. What are the 3 types of Oata Centers?
,
- j
'::f~~tr r ,
I , .,~
_ {I",,\ h !e" .l .. ~
- '7(,,\,,~ I,~
• S<~rl~
- (\\';L'C •• ;,{\ f[c'J\
- Ri4<- f\""\~ ~ ..
- ,(robi"",- \)i.\' I" \.'\)1'.
_ .:,,! ""~\'tl" \)<I..,;t\ ~1Y\~\-
r. . . ~\t IV\V\.\<I-\\~"
Risk. RMllability. .. Class Ranklngl
'0'
. . . , p,..~PI""~oc, Not Copy
Reliability & tile Risk of Downllm& 3.2
,. target reliab ility class results from balancing the f uture cost of
downtime vs. the present cost of prevention
• Reduced downtime ~ increased redundancy
• Greater redu ndancy = increased cost
\""'e~~
,h \7-J .... • OilSS F1 is the lowest In terms of reliability (avalla~ lity - 99%).
and Class F4 is the hl9hest (availability g 99.999%)
"'f I-i""~· ( • As dasses correspond {in part) to fulu", availability forecasts, their
I /"'n ;, "'>" classlfk:!tkm tends to be both art and science
-h-V cft... .... ,1+IY?i ~d.Jj~' E~en SOl it i$ Oil usefuJ met1ic lor balandng the cost of downtime
with the cost of prevention
"~"
· .......... 1 .
...
' .!"":':::;.,.''"
--
~~~
" . .. -. , ,.
' _ft4iInd~
I r ' ,
I _"
O2IItO aics,"
Student Guide R.... Rollobilly. & C..... Ranklngs
.....
!hall 24 ,",ou'! ~ d.y, hiss tha~ 7 dt"'~"
Annual Allowable
Maintenance Operational
DeIIcrlpilon
HOUfS level
H
I leSi P'rOpti""'Y~ Na1 c ....Y'
Student Guide RJ~. Reliability, a CIUG RAnking,
Opel'Btiol1ii1
Lo ..1
__
Alow.bie Mauntwn An"ual OtJlMltme ( .. in ....)
Description Classlflcatl""
lmp,(t of downtime Is toca l ln scope r and iIIf'Yects on ly a sft'lgle site or
QperBticn, or results in a minor disruption or delay In 8chle\l1ng keo; La ...1
orqanizatiooill Qbjective.s
0'
Impact downtime i$ regional In s cope, affeaing a portion of tt,le
enteo'pn •• (although not in Its entirety) or ,esvlling In a mod...,.te Regional
dlsruptjon Of'" delay in achieving key crganlutfonal Objed:tve5
Multi-Rag., .....
,." .~. ( CIM_ F3) CI... F3 CIaO$ F4
Class F4 3.16
'i201o Illcar'
_".Guldo Rllk. RlU.biltty... CIas& Rankings
- 5Ysc-C-';--
.1\1
) R~SO%
~ystE1fl.n ~Y<"'"
Bl
A~ 50'> hlO,.
I
Syst~m 1\2
R~SO%
I
Paral"'l Sys .. m
I
I
Serial System
Stllllont Guide Rllk. R41loblitty. & Ci8s. Renklng"
The foregoing calculations show that it isn't soleiV the reliabil ity of
individual comporients that is crucial t o el1Suring a reliab le S'/stem
• Rather, the reliability of a system depend. upon how its
compo!1ents are put together
A d"ta center may have different class ratings for different portions of
tts In frastructure
• For example, a data center mav be rated Class F4 for UPS power,
but C"'ss Fi. for the mechanical system's power train
- He", tile eJectr~ I S'/stem class rating is equal to tile lowest
rating of its components, yield ing a aass f2 electrical rating
l .n
BlCSI Propri.tlry ProtMlrCY-Oo Moe CQ,y
Btua.ntGuldl
Review Questions
Module 4: Location
The ta,.get reliabilitY' class will determine the type of utility feed
reoommended for a data center
• 5i "!lIe FMd from 51 ngle utility $11 .... Statlon - the least
desirable method for power delivery to a site, suitable only for
Class F1
I
student Guide
4.9
IWfIolmurn 06 f l / 20 rn )
...
ateSI fI....". .....,. P....Cy-OO NcIt COpy
cr_ F2 Acceu R&comrnendatlons 4.12
Centra! Offlc" ~ 1
Data Cenr~r
long Dirtance
C~tral Office'2
~d."Gc.oldo
The data center should be at wttl1ln at least 10 miles (16 km) of;
• A hospftal ,,'£ ~
\) ... tt~\·, h-\.. ;:'" \ L IN'~
• A major metropolitan area <'" , 1"' Lt..<- u\... /.,....,..,
" Q ~ .
C0l1111 aal'
_ .. Ould. LoeoIIon
Wind 4,26
-_... ...
,- '.,.
Sludont Guido Lotallon
Review Questions
1. What type of power utllity feed Js preferred for a Class' F4 data eenter7
3. How f~r away should a data center be from the nearest Jake~
021)11) BleSi"
•. t6
afest ~pritlwfY ,,~~ !'tot i;QPJ
Buildings
:1 I
...
IICIf ,",p"'" p,,,,,,Jtro-Oo .. "" COPJ
student Quk:le llulldlnll"
The ground noor is often the most advant.l9eou~ for data center
placement due to equipment access and Hoot loading considerations
• Frequent changes In IT and Oata Center equipment are common
• The average IT sel'ller has a life span of approximately 3 years
• The average data (:(!nter will eIlange out its entire equipment
inventory over the cou~ of 3·5 years
Although less convenient for equipment acress, upper floor data center
placement can provide added security against unauthorized Intrusion
and wMer damage
, In areas subject to major lateral force issues (.uell ..s hurricane,
Wind, earthquakes, etc.) M upper floor will contribute to structural
illliecw:itv • , .I j
1 ... ) \ l liJ ;' ' ; 6'
Dete Canter Neighbors 5.7
Walls 5.8
The data center'S perimeter walls s~all be full height (slab to deck
above), l·~our rated construction, fire sealed, and sealed ro prevent
chemical fire suppnesslon leaks
• Interior walls sholl be constructed of a minimum 22 gauge (0.65
mm) metal studs lor walls up ro 11 feet (3.5 meters), 8 gauge (1
mm) lor walls exceeding 11 feet (3.5 meters)
• Studs 5h,,11 be " minimum 01 5.5 In (140 mm) depth to
oo:ommodate boxes and piping requited to be Installed In the wall
• Walls shall be sheathed 10 ftre rated wall board such as .6 in<h (16
mm) type ")to 9)IPSum board
• Non·,ated walls should be braced no more than every ten feet (3
meters)
• Gypsum boa rd for non-rated wall Is to be .6 inch (16 mm) type "X"
5.11
The data center should be motsture/v" POT sealed around the
perimeter, floors, and ceiling.
• If data center Is a 'ground up' building, slab and all below grade
components should be continuously sealed with a rubberi.ed
mojsture barrier
- All exterior opening/penetrations are to be sealed prior to wori<
on intenor walls In the data center
Floora 5.12
A data cent..,- Hoor shal l be a minimum slab of 5,5 inches (14 em) thick
• The Hoor slab shall be designed for" minimum of 150 Ibs/It' (7.2
kPA)
- for data centers with hiqh-denslty racks, floor slab is to be a
minimum of 8 inches (20 em)
- Centers with high-density racks shall have" slab designed for
2.50 IbS/ft' (12 kPA)
...
BlCS. PftI~J)' Jtrro,.lty-C~ HQt c~
Studlnt Guldo Bulldln!!"
atl10SICSi"
54
I IClI Prvpr'iIo'*"Y Pi~HDI. Cop,.
Bulldl"ll'
Ceiling_ 5.16
Doors 5.17
The primary entrance door to the data center shall minimally be it P<lir
of 3 ft (0.9 m) wide by 7 It (2.1 m) hig h doors
• The primary entrance doors sh""ld have no center post a nd no
doorsills
• Data ~nter doors shall be solid core, minimum 1 3/4 in (4.5 em)
th ick, either wood or steel, mounted in steel frames
• The rest of the data center doors should be a minimum of 3.67 It
(1.1 m) wide by 7 it (2 .1 m) high for it single door, or 6 It (1.8 m)
wide by 7 ft .(2. 1 m) high for it pair
The in itia I layout of a data center site, the computer room + support
places, shou ld ideally allow for future computer room expan;lon.
Avoid placing the computer room next to :
• Outside wa lis
• Elevator shal\s
• Other Int~maHlxed buildin9 wa II!;
8Ndent Guide elllidings
""""' RoI~n
R""m
I
I
- - - - - - - -I ...
" " , _'
~,
,- - - - - - - - .- - -
I
1\>,,"
~d ...ltlt. 1 I 'elecorn
'5oi*-l, Uri-! thl.dIAg
""', (M1t'1' R.oom RiSOItI I If ~:IOfIone l Sp.ll~
W"'" I'
Student O4.Ikte Bolldl"1I"
&611) BICst'
S1udonl Ouill.
A battery room should be adjacent to the electrical room. The size will
depend on the type and number of batteries required
• The batte ry reom may also need to Include the Installation of
disconnect switches
• The battery room will require two exits
• Battery rooms sha ll include halardous containment, either a
contain ment system or hazardous mats
Check with AHJ regartling local cocle and dedicated exh8ust systems
BICSI
....
""""1' fI'r<IP4'I1Y-Oo'" Q)py
Student Gulet. Bulldl",,"
• The print mom should have its own dedicated air handling sy<tem
• A separate puper storage area near the print mom should be
provided ]4.'''f Fr ... .t-.-, -\;- ,.1i" .... /.• r.,.''"'' /<> ... ,~;M ,-~-{
Fire Control Room (,·~ ~~,... .. ,..,-h"" . ~4 LIZ . 5.30
For the highest tier data centers, a separate room should be provided
for the pre-action control valve system
"11
.lCSl ~""t8IV""rft-OO Hoi Con
lkudontGulclo Buildings
A secure loading dock should be provided for the delivery of high value
equipment
• Tne secure loading dod< should protect equ ipment from .evere
weather
• The secure loading dock should be monitored by CCTV
"11
tRCII ,.,...~~-OO NotCopt
Review Questions
2. What 3 things shOllld one avoid placing the computer room ne~t to?
~IIIIIJICSI"
_ntGuldo
Once the locatio", building, and lloor layout are In place, one can tum
to the desi9n ·c onsiderallons for the computer room, the heart 01 the
data center
, ,
J
-.
....1
- -
I I
Or
~
-..
oj ',
. -.-... .-.
dO
. ~'
-
...I~.
...
r .
--- -
~
i "
i i ...;;;;..
.....
I
~ '-
'. I
.-I
............
M
h , IICSt'
."
atcSl P~tIt1t'~"'" Copy
studlnt Goldo The CGmpt.lter Room
Tl1e star topology of the data center indude> a Main Distri bution Area
(MDA) as a central "backbone" hub, and horizontal distribCltion o..,os
(HDAs) as the end points 01 the networ1c 'bad<bone'
;;s;
...,
,,00.....
.' ... ,. ~
d='." •""""'" .""""" m
~
•.
._ _ J _ ",. , .
IiIiiil;U:;'
•
. U \ •I
" ',.1 .
,.
Student Guide The Compu,*r ROOf't'I
. ,,
~G'AJ! TEHJ- TR
~ f.tWA1-' [kp ---
\ "-
~" j~ I II
,
fWA ~
BO . ~ n.
I TR_I _ c. ,
BockOOM
EF co, .. ER
CI2O'I'OBICSI"
...
lIeSi ~y~p~N~Copy
Siudoni Ouido The ecmputer Room
-,
.... 'U:vf'!t.1
1IIJn:r1~"Io.,. ......
(1.At4( 6#t.N / KV1"
S"'~'~l
....... I_n._Ic.."'"
ltqWlllftHlnt
......... tlen .. _
(,'acI(/CIlItl lnet)
---------------
.10.CSi"
Stud~nt Guide T.... CGmpulor Room
• The rnaiO distribution area Is the primary interface between the data
re"terand the outside voice and data feed • • It supports the main
cress-connect for the data center 8 nd houses core routers, LAN
switches. and SAN switdJes, PBX., and IP-PBXs.
T_
I
el'1tlilnC;:1!I
Cclrnputer Room
H~2 0 f'\UMI
Olwlbvtlon
"""'" Main Distribution
A...
Ar ••
=m
SuikllKlg
f-"
StQl"0911
Ar.iI
NdwOII!
t Platti!)twIIl Di1illjbu(Joo A,eOl
Offic!!'
Support
Space
. ."'" ... •
ElJCSf
C2Gl1
InCsa ,~
.... DC '10;
HDA 6.10
Horizontal distribution areas support horizo nta I ca bllng to the server
cabinets, the LAN, SAN, consoles, and KVM switches
6.11
PaTe" Co-t"d
into Server
Pe1chCord
into Swik.h
IlBA J~DA
..,
SIC&f ',¥PI""",,yPJof*il-Ol) Hoi tIPPY
Student GWde lh. Camp0t6r Room
.....
-
-;
'
·-1
,-,
- ,.
I.--.--~
The Computw Room
Air Is brought into the cool aisles from undem .... th and exhausted from
the hot aisles overhead
• This produces constant air circulalion through the rack5, pro~lded
there are no "holes' through which cool "ir can leak
. .
...F"'-·-.- . - - - -......
,-
.....
·
·
.
...
- :
.. .
• •
HGtAJsle 6.17
The Hot aisle should allow for 11 minimum of.3 It (.9 m) of eiearnnce
·• · ·•
;--:-
·
'------'--
·· ·
~
· ·
·
"----'
• •
;--;
·
:..--'-
·
'---'
• •
:-;-
·
· ·
• ·•
• I~'V ... .
•
M .."".... •• H •• ' 4 ·..4 .. "'f' .... "':;
Ensure !hat racks and cablnelS are positioned and aligned so that
adjacent cleat Hoor tile can be removed without Interference from the
rack/cablnet
Studmt G~jd. The Computer Roo",
....
"'C"'.~=
The preferred location for remote power panels supplying power to the
racks is at one or both ends of an equipment row
CI2ItD BldiI"
...,
B rCS! PraprlIItII)' "'o~ Mgt copt
The Compuw Room
The preferred location for the Power Oistribu~on Units feeding the
Remote Power Panels is outside of the computer room, In adjac""'t
space
• locating the POUs in a<!iacent space removes a ma int enanee item
and heat Source from the computer room
a u, d G PO ......... F~ .,.CII
--'-
I u"'" . PP
Pou
I \:" rl.~ ..,...,
'--
$e"~ ~6o(f\1.
SAN frilmes are network In terface racks d1at typically conSist of large
quantities of high-density fi ber termination panels
• Where SAN frame. consist of core and access switches, they snould
be Installed In separote racks
• SAN equipment often does not fit standard se/Ve.- rock dimensions
and requl"" special placement
Cable trays shall be no more than 6 Inches deep (150 mm) and shall
be at least 2 inches (SO mm) above the slab and below the access
floor.
0-1.
IIlIte.t PI'Opr'ital1')' PW~ Ho1 CoJ2'1
Stud.nt Guido
Lighting 6.30
When occupied, the computer room shQuld h.we a minimum of 500 lux
(SO It .:and les) maintained in the horizontal plane and a minimum of
200 lux (20 It cand les) maintained In the v~rtH:a1 plane, both
measured at 1 m (3 tt) above the fini!hed Hoor
• PortatNe, battery-powered lanterns are suggeste(i in all computer
rooms
Ramps 6.32
e201a BIleif'
... ,
. tcsrp.,.,letary ..... rty-tJro .01 capy
TM Computer Room
Review Questions
2. What ~ the hot/cold aisle met hod of laying out rack mounted equipment?
Ciiii. BiCii"
Studa.-c Guilde
!:ID10 BIles"
Equipment ~yout & Inte!Jrated Deelgn 1.2
The prjm"fY purpose of a data center is the storage and rna intena nee
of IT equipment ; here we will loolc at some of the mai n types of IT
equipment to be stored and best practices fer storing it
• It is important to understand that proper eq ulpment layout also
jJlvolves consideration of:
- Power Distribution Planning
- Cooling Planning
- Data Cobling Planning
- Future Growth and Equipment Planning
• Power, Cooling , and oat. Cabling planning wJII be examined In later
modules.
,OB
1984 ~
I.~
student Guido
The general trend for data centers over the past 5 Ye<!rs has been
lewa rd shrinking space requirements wil7\ IT eqUipment growing
denser
• Server equipment has been getting smaller, but its power
requirements have remained constant
- Higher densities of equlpMel1t require greater cooling and noor
loading capadties
IT Equipment 7.5
The IT equipment typle.lty Iloused In a data center includes:
• ,.,,,twork Servers
• Digital storage Equipment (Disk Anavs I Tape storage)
• Network Route.-. and Swill:hes
N
elC .. _ _ .... c ...
lIIudont Guide JT Equipment & RacMs
Network routers and swltthes connect the data center hardwd re and
external client users; they c.;n come In either a rac k- mounta~e form,
or a .til nd-alo ne form
g .
g
• IIwningIinsui:Ili
• l'il:hesflll~ mml
~
i
Student Guidlt IT Eqillpmont & Roots
7.12
• 1 RU ~ 1.75 In ~ 44.5 mm
,
.•"'
I _..W
1 t· -t.. ."
c..~:l~W
•
_
r
lllu .n ")
I
Jn.II'-';~
. .j ... , .
g.... --
_
I I
, JQ' ~ . ~"
-, iI};;:-~- . -
J·U- 1·
-:--t_- '.
j
O,IllS' i -
.j ~,
v. s--
r . -~
, J -i.--
I O.6"1 ~"
..
1..25' -,
I au;a-
_L-., _-4_- ;
u _!"
! .. ..,
- ' . -;
-
1 J,.:--'. 't:"':
T-;;:;',..
. f •
l .,-t><l~·
f ,
-' -
-.
"'
UPSI~r
l5U rot;l!y .U
""""'"
N_ 4U
.....
Switct'l
$0< . . ,
.u
1.15
Data Centers shall emplov 2-post racks that utilize itandard .i>lng
cr;mventions where channel dimensions, spadng, thread system, and
usable aperture conform to CEA'31()-E
• Maximum height should be eight reet
• Each rack should have vertical ca ble managers attached on both
sides
rack
• A l'aci!>grouncl bar should be installed in the Dr a ground wire
should be mou nted on the bold< of one of the uprights
IT Equipment & Racks
7.16
Cabinets 7.17
•
... posed threads under the access fioor should be covered
. ~ -
When installed on an acress Roor, racks should be located so that
the rods securing the rael< to the slab lOliIl opt .... AelMte It>&lIQor
tile stringer
-
• Racks shou Id be Ieyeled throughout a row
-
• ShlmmlOg of anchor points shOUld not exceed '1.0 inch (12.5 mm)
~ - \J ........J -..
-~:'
• Adjacent racks should be ganged together belon! anchors are
'1 ~ ~~
-
<X ~ t v.....
Best practices for cabinets 7.234
EaOia liCit'
,.
IICSI ~plietary PlopMfj 00 MIrl c~
studem Guide IT Equipmenl & Racks
Equipment Labeling
-
systems
7.26
..
---
h
--.
~
•
~
"
"t
'~ · ~I_'"'
Q20H' IICiSi"'
StudOlIt GulcM IT Equ/pm... & Rooko
, " A .. .. - -
7·10
B I (09 1 "~~Not~
student Guide
"
omilicsr
51udent Guide IT Equipment & Rad<s
Revi_ Questions
i!iOOi; lli¢sl"
student Guide Security. DlNlto< PIo""ing . .. Fii& 8(oPp4ng
Wall~ 8.10
Al l concrete walls should indude steel reinforcing bIlrS (or'rebar"), as
concrete offers little security by Itself
• Additional rebar will Increase wall durability and reinforcement
Any building opening less than 18 It (5.5 m) above gl'Qund and larger
than ~ in' (620 em') should be"protected by a barrier, alarm and/or
t!)onitcr
~
• B.. rr"'" are not impenetrable, but are used to delay entry
-
entrance
• Barriers may be m.de out of:
-
So rrlers sheu Id foro! the intruder to leave evidence of their
Doors 8.13
Windows mOU nted on the exterior of a dati> center building shell are
not advisable for high security c.ata centers; best practices for data
center windows Include :
• Only pictun! windows shouKl be installed in any ""tenor wall of the
data center shell
• If a oomputer room wall is an ex teriOr wal l, then that wa II should
not ha lie • window
• Exterior windows should never use putty or molding to secure the
panes of glass or plastiC In the frame. OnlV rr"me mounted
(grooved) wi ndow mountings should be used.
• Window frames should be constructed with rigid sash material that
Is anchored on the inside and Is resistant to being pried open
• If windows are used in the computer room, the 5111 should be at
least VI m (l! feet) "bove fini.hed floor
...
BlCSI froprfoaa.y Pf~ Ncn CsJP1
Stullon' GullIo S-tilY. O.....r Planning. & fire stopping
Keypads: When keypads are used, they should utilize high security
approach when: ea<;h numeral Is randomly scrambled to a new
position each time the start button Is pl1!S5ecr= -
• It Is rec:ommended that keypads be used in conjunction with a card
reader or other EAC method
Accns Cards: Access card technology should be integrated into the
1D badge when possible. Access card systems should Incorporate an
active anti'passbad< feature, permitting only one entry without an
acrom >'inylr19 ex it
Biometrics: If Fingerprinl/Eye/Face recognition hardware Is used, It
snould be used in conjunction with card access to decrease databese
seard> times
All pedestrian doo~ I~ted within the data center an d computer room
should possess:
._-
• A full-sare lock
• PaniC/fire-exit hardware that can unlod< the door
-~
atriIIUCSi"
.~
aeel p"''''''''y~ NiI!It Copy
Studoott Goklo s..ct."Y. 01 •• _ Plann'"". & Fire Stopping
Ala(t1llnlegrallon 8.22
mtUIBIC3"
Studont Guido Saeurity, 01_ Pllnnlng, & FI,. SlopjIlng
..
~
.
~. t!"
I I
Vi",", rT'I!Ineltttrletl ~
Sl/9te h"l l)luS ~ Of
J
lstr~r
"~ ..-I(I I!O ""'1lI'W
......
IICSJ Pr4lpriealyPrIIP8rtY Dg NotCop'l
Student GUilM
lighting 8.27
The data center security plan shO<Jld have separate component polie~
regarding tt1e protection of:
• Personnel
• Physical assets
• Data
@»H'BLe'"
Slueont Guldl s.curtty, Dbl.ne.r PI.nnlnG. & Rre stoppJng
The· Data Center Security Pion needs to detennine the following access
control pollc~ '
• Who should h~ve access to the data center?
• During what hours should the dal<l center be «cessible?
• What are the entry points of the data center?
• How Is access typically granted?
• How is access granted to visitors and contractors?
• Is sole occupancy prohibited or allowed?
• How are \llsltors to be monitored?
• Are access control proced ures different during non-business hours?
- What types of doors/b<lmers and locks are to be used for each
entry-point?
• How are keys mIlnaged?
• How are access breQches to be handled?
Access control should Include one or more of the follow ing methods of
authorization ~uthentication ,
• What"a perSOn has, e.g., keys and cards
What a perSOn knows, e.g ., passwords and codes
• What a person Is, e.g., guard recognition, biometric data
Stude'" Guido Soout1ty.III...tar Planning, & Flro Stopping
Slgllage 8.38
The sewrity plan should identify the wording and location of .ignage
and displays intended 1:0 control pedestrian traffic from straying into
unaL1l:horized areaS
The Data Center security plan should have written guidelines detailing
the appropriete responses to all bui lding alanns
• The security plan should specify what types of build ing alamis are
in place, as well a. the appropriate personnel response to each
alarm
• The security plan should specify special ~ttess control proced .. res
In the event an a larm is activated
The security pli!n should derail any high-risk aneas in need of special
sLlrveillance .
• In the eYfllt, that closed-circuit television surveillance is to be used
for such aneas, the security plan should detail the appropriate
placement for surveillance cameras
o The sewrity plan should derail:
- types of event s t hat should trigger any surveillance alarms
- appropriate responses to those events and alarms
The .data center should have a detailed asset log or database In which
all IT and telecommun icotions equipment i$ traCked
• Unique Identification numbers should be permanently affixed,
etched or stamped on all computer, network and
telecom mun Icatlons eq uipment
• In high-riSk areas, computer and network equipment should be
pennanently fastened to some part of the building or pi<!ce of
furniture
• The us<! of an electronic asset program (EAP) is encouraged. EAI'
tags should be affixed to all IT and telecommun ications equipment
O2t1liCSf
Personnel PoIiciM 8.43
The security plan shoUl" detail "II hiring and termination procedures
and should detail mandatory information from applicants, including:
• Name and vanatlons (venf..,d)
• Address
• Otlzenshlp
• Military record
• References
• Security dearances
• Education
• Employment history
• Spousal/Family infonnatlon
• Criminal convictions
The security plan should require that "II data center employees have
an exit interview to;
• Reduce the loss of wnfidential information through disrussion of
non-disdcsur-e agreeme:nts
• Discover problems that existing em ployee. won't revea I
• Reduce lOSS of dat.. center property though the use of checklists
• Give the employee a chance to express grievances, reducln-g the
possibility of sabotage
mn,'JiSi'
Security, O.....r Planning. & Fn StoppinG
8.46
• Landslides
• Loss of utility services
• Severe weather
The disaster recovery plan should take into account the following typeS
of crimin,,; actMty:
• Arson • Kidnapping
• Blackmail • Looting
• Burglary • Rape
• Bribery • Riot
• Conspiracy/Racketeering • Terrorisn1
• Disorderly conduct • Theft
• Emt>ez.zlement • Trespassing
• Extortion • VandaliSm
• Fraud • White-collar crime
~.BICSt'
Disaster Planning: Tellti"g 8.49
Data centers should regularly and routinely cooouct disaster recovery
tests
• Dlsaster .plannlng should Indude the creation of mutual aid
Bgreem!!l1ts with other businesses and erga nizatlons
• Disaster planning should indud. coordinatiM with federal, state,
and k>l:al agencies
Disaster recovery plans should indude detailed plans for all aspects of
emergency oper..tion of the data center InCluding:
... Access control plans for emergency cmd non-emerg~ncy personnel
• Operatlonal checklists, with detailed documentation and
InstrucIJons on operation of equipment
• Facility shut-down procedures
• Emergency security procedures
• The k>l:ation of all emergency eqUipment
• The use of fire and eme'1lency equipment
@JCI108U:::Si"
S<ocurlly, DI ...... ' PIII""i"!/. a ron> 81Oj1p1ng
amI
L
T"'~ ~I.ooPIlDrt "[]I~
hl r d~~thllfl
O.'!H m (1 tI) ""I~~
_01'11 Guide _ t y, D I _ Planning, & A .. SlOPlNng
lonization-type Detector
PIloto.,ectrie·Type Detector
$Iud.", ","do Security. 01....' Plonnlng. & Fire 51""ping
~UI.ICSi"
....
• Ieta "",~p,~ D-o N«"Copy
Stud_'" Oula. 8.curtty, Di"'lter PhuuUrtg, & fIN Stopping
-
different valve than t hose used by the ,est of the building
'PlQIHCI Blcsr
....
IlIIt8rPreJN..,"~M04copy
Studani Guida _"rtiy. 0 ....1or P......ing. & Fin> ~ping
Every Data Center shall have a fill! protection system which should
include :
• Dual- intE 'lock pre-action sprinklers
• An early smoke dEtection system
• A water leak detection system
• An under floor fire p,otectJon system
• H and-held fire extinguishe,,;
• The system shall provide manual pull stalion. at all e.it doors
• Firefighters' control ponel, graphic smoke control panel, pnnter and
annunciator shan be IOC<Ited at the main security office
• The system shall provide connections (rom flow/t<lmper swi\d1es to
the fire alann system
• The system shall interface with security system; upon activation,
the fire alann system shall release all se<:u lily doors
• The system shall provide monitoring of fire pump and generators, jf
provided
.
• Handheld, dean agent extinguishers shall be provided as required
by local codes and jurisdictions
• ElCtinguishers that use dry·chemlcal agents shall not be used
• The system shall provide smoke detectors In all unoccupjed spaces
• Ionization detectors shall be provided In conjunction with magnet"
door holders, where applicable
• Fixed temperature heot detectors Or Hame detectors shall be
provided in generator rooms and the temperatu re set point
coordinated with the sprinkler system
Review Questions
3. What are the five policy a rea. a data center's secu rity pl.n sI10uld
addNlss7
Power &Eloecrick~
mcI10 BiCSi"
..,
sal ,~.rtICBry Pr~ I'Iol co~
Studont Gul. ""wa' a. EIoCll1cil:y
'""10, .. :.= J
i
L
~ I:,h~ '_'~IL4~------1L__~__"_U_P____~
_____ __ ~ (jene~a[OI'~
1
\ JPS
I_S. .
dk hbo.olnk
1
_IIIOulde P..... , & El.ClriCily
UPS#'
I PDU *1
-
oper~tjQn~ ' .
~~,_U_P_S*_1-,~~_____1
PDU.1
__
4 UPS *2 ~ PDU i2
W
BICS. ",pn.wy Pr~~ ,.,r Cop~
st._Guide
UPSNI PDU'l
PDU t2
PDU.I
UPS" rou .2
r Uf'S')
-molIIUCS"
. ...
8/CSI ~, Pt'l:lp.rty-DQ Mot ~
Student Gukte P....r & Electrlolly
t Utility
l~ ·.,· .
Electrical Distribution
"
.DU
I
~
CRITICAL
-TTl Loads
SllldonI Gul ....
,
- - -- .
,
'.
'DU .,
•
CIII1ICAL
TIl Load.
_ & EI.<1riclly
~~
_..,......_" .
. c . ." " - . . . . , .. _ - - - '
I ...... .
-1>11.......
. . " - ; .,
-. .... --
.... -~
-.~
•
-:-.;. I
-.-_
_
~-
..
..... _~. ; •. ,;.....,...:.:.......-~\ll ~
III!:
" _4 '
. ..
.com<Il
11« •
'< .
I :
•
JI*l- ~ tl..Pc....1
CRflt~
. PIlU
. . ~ "- I'~
- "~
. . ,' ... .............
.-'
", ..... ...
CU1C1 BiCSi"
Shuiettl Guide Powo< & Electricity
,-- .-
,J ,.;",;,;-
, • ·1
....
'-
-- - - • ..
....,.... -..
~Dirln~~.. t·---..····-······-······--..········· .CIUIj:IJt~~.J
PW ~ .. - .. :~ J .' i:~J p~
L. CIIITI~ I ...
FIIB / """ j
. ~
iii
It..., '. -, . ,~......
.. _ ,. ,
_ . ...... ;""'<3
t - - , '
__ - ...- .', .1
-. .- UP5
, ..:
r· -
t
,
,. .
,
.' .~
.:
- .....--- ....
CtIIk '
UPS
P. ,'
......
--~"tr""'''''t
, ...
-----....> ~
.;1 --~ , ~,","' ...,. ' . ,
L___
1·10
lICit ,~ .. ,<lp,""""'OO H,ot COH
Studonl Guido Power & Eloc1riclly
• MechanlcaVCooling 50%
• Li<}hting 3%
• UPS I Batteries n%
• Critl~1 Load 36%
Power Consumption
Lighting
stu_tGuldo
Power eonS4.Jmptiorl isn't the only ttlirig that must be considered when
designing a data center's power system as gtowth and safety factors
must be considered also
• Capadty is the target power requirement for which the data center
is designed. To determine the capacity for a given data center one
rn L!St estimate!
Critoe" I Load
Growth Factor
Safety Factor
UPS Allowa nee
Ughllog Allow"nce
- Cooling Allowan""
-
. run ning loa cis
".2
BICSI Pra,....ry Pre!,...,. Do MIll: Copy
student Guide Po ...... & Elednl);ty
"
' .~Max·: '} ,. Jd!e'- • Ayerag~,
~'F~~l 9.22
-
. OS by the init,a I load ~timate + the growth factor
Both UPS o!>"ration and power $torage (i.e. batteries) wnsume power
that need. to be accounted for In estimating the dat<o center's capacity
requirements
• Batteries: fully charged new batteries will consume neQliglble
pewer, but batt"", charging can consume 20% of UPS Ioed
• UPS Ineffldency: The amount of power the UPS consumes while
operating 15 lIccounted for in its 'iDeffideo<l'. ",,(tag:"
- For example, a UPS system w~h a 90% effiCiency rating will
consume 10% of the critical load it 'serves
Ughtlng costs for the data center are" fUnction of the area of the data
center. This is known as a Ughtlng Power Density (LPD) calculation.
LPD avera9es lighting usage as .0021<W per It' (,021SkW ~r m').
'-
• For example, if the data center was 1000 It' thenI w/ff"-
lightIng costs can
be estimated as
Lighting Allowance •• 002*1000 = ;;tkW
Cooling & Initial Thermal Load Calculations 9.27
To get a rough estimate of the power for t he cooling load, fi rst sum
the ",quirements for the Initial Crltkalload , Growth Factor, Peak
Power Allowi,"ce, UPS and Battery requirements
• UPS - 15.6 kW
• Ughtioq ~
2 IsW
.
I
The cool ing load is typically the biggest power draw for the data center
because \lirj]lally aU pf tbi' pgwer suoplied is..sonwertsc1 into heat t~
must be removed
• Cooling systems will vary in efficiency; the most efficient will
typically, require about.?5% o~ the po!;:er of the thermal load they
are .servlfl9
- Thus to get a rough estimate of the power required for the
cooling system, We can simply multiply OUr previously calculated
power load, 72.3 kw, by .75
' -18
SICS. Pnlprr.c.ry Property-DG Holt c.,..y
Student Gukkl
w~ can now sum the previous requirements- for the total power
reqUirement for our data ~nter:
/
( •
•
Inillal Critical Load;
GrowU1 Factor =
43.4
8.7
kW
kW
, • Safety Factor = 2.6 kW
• UPS ~ 15.6 kW
11 • Lighting ~ 2 kW
I • QJoli[lg 54.2 I5W
• Total Power Requirement - 126.5 kW
-
Power Requirement and Sizing Electlical SeMe.
The electrical service required for the data center can be calculoted
'rom the previously detennined power req u I~ent as follows,
9.30
With a Ulnjet class rating and service requirement in hand, one shou Id
then tum to power. provk:ter planning. Key Questions to address when
consider power service providers Include,
• Will you be on a shared service? If ~, who else Will be on the
service?
• Are there any high Impulse loads (foundries, paper, plants.
smelters) that will impact on power quality?
• 15 the 5el"\llce to the site primarily overh""d or underground 7
• What are the required capacities of the service initially and
u IIJ mate IY? How can services be expanded?
• Are diverse services av.llable? How are they routed to the site?
• What are ttle setvice voltages in the .re.7
• What are' the reqUirements for dosed transition operation, if
employed?
• What a re the drcuit proteCl!on requirements that the site must
provide?
.. What are the service construction requiremef\ts~
• What has been t he reliability and availability of the service to the
site?
• What Is the available fa ult duty at the main s\V~c h?
.,.,.~
- """"'"'"'-. r-
- ..
~ - '--
;-v
e:teUI- BlesI'
.
,.,
• rc. I P~ry~"ot~
Pow... & Elo<Iriolly
..
1" lflii u1~~1
~
I "'"
it
!>It,t_wNI
u. 1:
h '" It.w,,l trrm ! I!Mt1 IUl ftf
UPS
GGncrrt'lOr ~ P- p'
_~Ro om
SCUdent Guide
Swlldlgear 9.38
Service entrance switchgear provides several functions :
• The interface !>etween the Utility and the site
• The distribution of Uti lity power to downstream systems
• Transfer cootrol between the Util ity t<.ed and the generator
- The switchgear should be designed for growth and the IIltlmate
loai:I expected
for the facility ~
] II: ....
+
Input Souna Trane~' 9.39
BlC$'~~NGtCopy
...,
Powor & Elootrt.1IjI
Opet'l trans/lJon OCOJrs when there is gap between t~e main power
feed's failure and the second~ry feedCOtning online
• Upside:
- Requires the least amount II. ~1...tri<;~1 "'''tr!l.ls and relaying
- Typically d(;es not require the Utility's approval to deplOy
• Downside:
- Transfer results in a brief disronnect that 01te<1 causes a
mechanical system restart and reduces the U~ 's battery life
t ·.,
Bal ~P~IYP~ Not cap,
S.udonl Gu.de
-_.
Switchgear and CIa8MS
e..,. F4
DodI_,.
""'*".-
.. 81,....,.,.& 1 tor .....~C'1IIcaI
IooId
DeiiIGOIod 10 UPS Oed.".t....'!
UPS poI!t UIIS_
lMechani'cal
TIWl5"r& Control
Po_
CorltIota
1 For atnett tOld
5Ingilatslaild"'GM"
DlClIelc.d 10 anti
,"d~ MechBnical
po",
81.« _ _
-.....
DHlcaIad to
TflKhltlic.1
a.d.-del,.
.....
Dedicated tD
II'IItCh1rIlc:1I
""'"
~dlim
Generators 9.44
,,,,
SICS! p . . . reuryfl~ NotCo,'1
Generatonl - Gen_' Considerations 9.46
The following are Important general"considerlltion. For data center
generi31tors :
• How will pollution abatement be handled?
• Wf1at Is the ru n time for the system?
• liow is coord inatlon of hl9 h reslsta nee grounding with the eJectri",1
system to be handled?
• liow are UPS battery ~chargin9 loads to be handled?
111e generator Cllntrol module should possess all controls and metering
for the individ ual machine, and not require any outside connections for
its o~ratiOn
• Unless multiple generators an: being operated In panillel, generator
controls for 'individuaJ machines should be completely contained on
the genenltor skid or within the generator control module
.--
--- ~ -- -. -
." General,,", & Cia __ ) 9.52
I
"
-...... .
"'"" ItUn 11mB 6 hours mirlNnum ~ I"IOl.Irs mirtfnla.jm "12h~
"" h.....
minrmwn rri~lmuni
~.numpt;on ,m 2m 1m
.Imo ""
FU.eI FlItw . ,.... ~illrd SbDndard
...""" p,tarimt
"""
•
UPB: Responses 10 Failure 9.54-15
UPS failures should automatically heal without riskin9 the load
• failu,;, response should allow Ihe UPS $Ystem to qul<;kly settle into
• steady state
• The response should accomplish one of the following with a slnr!e
.....,.,feT _ation:
- fall the UPS system to ~other UPS system, or
- fall to .n unconditioned bypass source
• failures should !Ie limited to the portion of the power chain that
experienced the failure
- In Ihe case of a .d lstrlbuted power system, the I nlerconnection of.
the systems should not allow failure in one system to C<luse a
failure in the supporting systems
- A U~lIty provider power failure should result in the generator
starting • nd the facility being transferred to the generator
- The ability to carry vltalloads during the power tr""sfer"
necessary for any data center design
---
~,'I\'\oool·.'"
....'-=.....=---'-----1~-
0-,
• ~!!der
_.... and -,...
Heavier
...
-..".-
III.... '
~r ~,.
A~~ ~=~~'-L-----i~r-~=f--r-~~:J
-.
N'
lIleSI Pr9.p~f~ HCltCoP1
student Gukle
......,"" _ .".......
..zr
~ PrcprilWy ProP."lP.t:t'lll"l"·-<00
.. "",I Copr
Student Gulae Po...,.. & Electrioity
~tx.htd
".'• •A•
.."
"•
-..,
,•0
,
•
•0 0
M
0
0
•• -I . " ••• • 1
I-I POU , .
"
"••
,\
•"
N
•
,
0
-,
,•
0
•
•
N
•
"•
M
O.
•
••
0,
N - I ,, -~ ~
J-l .. Olin
<aUI BlCSr
Studont Gukl.
Automatic Bypasses:
• Dedican.d automatic swItch Inputs are mandatory for all Class F3
aod F4 systems and for paralleled ~ystem contml cabinets;
automalJc Inputs may be combined with rectifier Inputs for Class Fl
and Class F2
Maintenance Bypasses:
• Maintenance bypas.ses are opbanal for dass F1 and Class F2 and
.mandatPl'Y for all.. f3 and Class F4; when used they shall have ..
Q:edlgl'fd keEl-
Mainlenance Bypass
u
T
1
R
S
W
- - - - - + PDU
I A I
iu~
L N 1
I
T
Y
s
F
.."•
C
Ii
"•
A
It ~
l R
~ Siotit 8yl>O"
- - - - J
CllI10 Bleil"
student Guide
-
90% of capacity is reccmmended
- for example, if a 200kW load is being served, the UPS should be
sized to serve at least a 210kW load, and preferably a 220kW
load
Poww & EI..mcily
- ......... ...,;
MiIIlllUnlnc. eypu. Byptcs pOIll'et takeR ByPSSS"powe:r E5y~ power- T.... frorn 0
fr1Im IJtiIIt\' r.ecs te~.. fhJ';' utI/!V
. _ ""'" ""'LtV
"teet! pOwen.l by
alternate SD~
VOIt.te~ 12Q120BV ~ to 141O/108V ~p to t20/ 208V up to 12aJ208\1 ~ m
ar 1440 WA
....
lOads 0( H'IO WA • IOIld5 I~ Of 1440 tvA loads of 1440
POu.t...r .. U
.....
oISOV
No
lor ~rgB" , .OOVfor_
""do;
He
, 4S<J'WI rot .....-ger
,
y••
kvA • ..aOV for
larger leadS"
Yes
ceJIIIpIb!r' eqIIIprna ..
",ad _
'10 flO y.. y"
,...__
S","cru"onlmtlan
..........-
Il..tu. .nt UPS .Al./tomiltlc UPS /'dom~or
_ryUf>S,.
~e ..
R.DtzI'1J', AutoI11.IC
...-..-.
R(l1;Bry, i!!" .
""
UPS I)q ~M'.~
tI~1041 .,.~
tram t;ompd\W
~o
A.otaltDg MoG set
...
(:onwrt.1tr1i Hvtwld IJIIS.
V.,
-.uipn.n\:
'os
....
Mantt9,. .
~rt.ilC8
OrMI~
with
No No
~d ves
Yo'
Yes
. " , . pM'
MIftiImumlull
rttt:MNM ""
5 mlJM!:!i ~O mlftVte. 10 minutes, 10f1IIin~e:~, (:OOI'(!lnated ..nth
.....
Joed~~bV roordlnat1:!d with
UPS topoqv
UPS wlXllogy
II:IDUntJ,.
---tv
W~p~d"'"
"frOm .... ,
..w.or
ubi,*,
No
No
"'-
"""" '"
'es
y~
~p.n~
Yet
'IS
a- ..<b
Ye,
,.
SWIitCl~r room
.- - -
,- Distribution Switchboards & C188Be8 9.68
-
_100,
cant ....
..iUl~m"~c: or
0",..,.,
SI"'Ule
"t>ft)mari~ ~r
~UI
51ngteor
redumlant
or.rwoot only
Rtd\I,..C~
D r;nr,l~ "(Jrfy
Redunda nt
"DU POU
m:I10 BICSi"
....
IIlesI Ptopliltlltyf',..." hMot Co.py
_.tGutde P _ & Eloclriclly
• FacUitJes with dual P<lWer feeds shou ld employ equipment wit" dual
power cords: -----
•
--
Single-corded equipment, and equipment wi\l1 multJple CCrdS that
are not fault tolerant, should be plugged into a cabinet power
distribution unit or fed by an automatic transfer switch (STS)
• l·corded eq.uipment should have each cord fed by a different-----
source, with no more than one of the cords fed by an ~utomatic
transfer switch - -
• All recepta~s and gower stnes shall be labeled with the PDU/RPP
and'Circuit breaker number -
• All POU/RPP circuit breakers shall be I~beled with the name of the
cabinet/rack ~r grtd CO<!rdinates of the equipment that it supports
.. Every compute.- room, entrance room, access provider room and j
...
e lCl1 PWlprietaty flropetty--Oo Not CoP'J'
8t~GLlld. Pow... & eleelri.iIy
..
R.squh~lI:j
'"
... ,,,
UPS_
......
"'IIS/Powef No too
--
Yes Yes
""'~""
-
. -
system shall be pl"ted on lJPS...power
OJ iIIe" w"ter will often stay cool for long enough to bridge the
moling system restart time
As such, the condenser water pump and ventilation systems are
more vital to keep on bac kup/con~nuous I!0wer.
SIudont Guido
_In
NDne ManualOf'" Maooalor
-- --,
AUl1)m~ti( Auromat:ic
"'.pped ro MiIIPPllc·1],
Medlanici!ll Syl!itam
51""'" s,.tEm Red_""",
Iterl\l~!]ty Redundi!lrg
Grounding 9.61
Grounding and bonding are essential for the .. fety and perform"",e of
electrical power systems, IT equipment, and surge protection devices
• Data center grounding add.",;.",,:
- Building Grounding
- Power System Grounding
- The Mesh Bonding Networ1</Computer Room Grou nding System
- personal grounding and automatic discharge
. ....
I ICIA P'rvpriltw-y PI1:lPf"'tY-gq "DI C(lW
Stuunt GukN. " .....," Electricity
A building ground electrllde ring shall be installed for all facilities. The
ground ring shoulo-tre a m ltiomum 'of '4 A~ ~:ZS-m-m~ifie Copper
--
wire buried 1m (3 ft) deep' and !.!!1Jn£)frolll tf1e ~I.
- .....
• Grllund rods, typically 19mm x 3 III ('!Io In x 10 It) cop;>erdad steel,
should be connected to the ring and spaced every 6 to 12 III (20 to
40 It) along-the ""ri_e< gmu",Hoop
• The ground ring should be bonded to building steel at least at every
-
other..building colu Illn -- " - - -- -
TIle key grounding connection for the EqUipment ground bu. is the
exothermic bond on the PDU directly upstream of the loads in the data
center
• All lo.<ls in the critical environment will be provided with dedicated
and e.press ground wires from the load to the derived ground- point
at the PDU
b:::;
~
::::
tsolilted Ground 8~
€l20,. Blear
student G'-do
,- - - ,'- -_. - , - ,- - - .- - - .- -- -- .- ._ ._ -- -_ .-
- ,
ezo-tIBtCSf'
.~)
ete_ "l"OfMlII*'1P~.....o& McotClJpf
Studont Oukte P..... & EI ••irfc!ty
-
• All non·current·canylng metallic endosures shall be grounded
• Ea", equ ipment cabinet/rack reQuires Its own ground ing connecUon
to the Me." Bonding Network
• A minimum of a #6 AWG insulated ~tranded copper conductor
mechanically tenmlnated to the Mesh EIondlng Network and cabilu!t
via a machine .screw, bolt connection or spot weld, Is recommendtld
• Grounds.t opposite ends of a series of racks shou ld be attached
with the framework grounding conductor gOing directly to.cnest>
-
bonding network
• Bare metal·to·meta l contact i. mandatory for all bonded
connectkms1 with antj-oxidaot applied at the connection pOint
-
• Grounding Via tna mounting system
• Grounding via the power cord_
allldontGUd. _ & Eloc:frIcWy
c._. . .
"_ ....
CiMl6ftdlna
RQiSWr.oe
S oiVll lohm lohm loflm
~tO 11tS,.
.... DC ,10: ~ c ...... DMlGII , BMI PI"BI:Ikle&
1I1eSI A'Op."tttlty p~ Hot e..,
Stodotrt Guide __ & Eloctric~y
Review Questions
;< , What are the four components of power usage that can be uoed to
detennine electric,at service sizing for the computer room?
-'v'rs
_ ~" 1- , '-"' \
I
\t"v\
_ \t·LI--.'\
~ . I)
_ \-";,, Il()'
3. What are the 3 main types of UPS systems and d1eir relative
merit.?
e:20tO DiCit"
...,
. ltSIP, .....y,../'tIIMI~ . . C""
stucMnt Gukie HVAC & Coolil1g
Though some clata center support spaces (such as the battery room)
have special air condltJoning requirements, the milin forus of cooling
planning in data center design Is the computer room
, Modern computer rooms are typically not designed to provide
human comfort
- High densityrr equipmef.1t produces extreme heat loads that put
the equipment at risk tOt'" fi>ilure
- Tempet"atures, air velodties, and noise levels often exceed
human c:omfort levels
tOo,
BICSI PropMIary p~ _COpy
HYAC & Cooling
Note that for data centers, both equipment exit air, as well as the cold
.ir supply are often outside the therma l recommenda~on5
- Equipment exit air conditions are very often dose to 100°F (38'C)
and 20'1'0 RH. and the CRAC supplied air Is often between SSc6SOF
(13·l8°C) and close to 90'1'0 RH
• To achieve 6S • 77°F (ZO-2S°C) at the Inlet, altstream mixing must
OCCur in the proper proportion. and the resultant ~ir distributed
uniformly along the server ''lei<
GatO BlCSI"
mAC & C<>allng
, ".
; """
'"
"' "
,","-
I
---1 n
J&6 l"
, , , ,,
~
1?-....
p j.
"';:.-
:-
- ....
_Guido HVAC & Cooling
QnCfl the ~rget l11ermal conditions for the computer room are
understood, the ne><t ste~ in designing the cooling system Is the
calculabon of the thermal load for the computer room
• The heat output of the computer room, assuming there are no
'outsicle' faCing walls or windows, depends on the thermal output of
its components, including:
- The IT equipment
Ughting Equipment
- Power Distribution Equipment
• While AC units will produce excess heat, thiS is typically accounted for
In their sizing and does not create an extra thermal load that needs to
be conslclered In calculating data cent'lr cooling needs
1'1<1
etesl 'MpfJatMy Property-Do Nol CDpy
Stullfent Guide HVAC & COC>Ilng
• IT Load 100 kW
PDU 4kW
t Ug~tlDg
Tota l
10.16
Once the COOling targets and requirements for the computer room are
determined, decisions then need to be made "boot how the cooling will
be delivered to the equipment
• The baSiC types of computer room ·cooling anchi\ecrures:
Qyeme.,,;!, air distribution
UnderHoor air distribution
Rack-oriented liquid cooling systems
- ;;0"
,..,
8ICSI PnI~1". Pt(:l~ rt« CoJl'
Studlnt Guidi ItVAC & CooJlri9
----- Lt
·"S..i~ l"dAJI
,
, I
\1 •• J
,
•
•·•.
-- -
- I
· j
1'"
BICSI P'I1IIprlMatyP~O Mot Cow
tll/AC & Cooling
-
performance.
• Central air handlers also allo... the use of air- side economizers for
improved effiCiency when the phYSical conflguratlon of the building
and location are ravorable
II ceiling height of ~t least 10 It pm) above the raised acceS5 Hoar will
typica lly allow for an effective 110t air area abOve racks and cabinet<
and optimize the return air path
1$.0
BQI Ptofrilta" ~ Not Cc:I~
Underfloor Air DIstribution 10.23
The mo:;t commo!l..ilir cooling design for data centers involves cool air
delivery under a raised floor through perforate< tile with self conta Ined
Computer Room Air ConditiOning (CRAQ units
-
• This sy.tem allows for maximum flexibility of cool air delivery
th rough modification of perforated tile placement
The retum air p~th is typically 'through the room' i" tflis type of
system with many CRACs h avin g retum air duct extensiOns built over
tflem
,.
, ~ ~ ~ .
, f-l- fL ' ':' - ~ ..J... .'
iJ;,
I'; ~ :. •
I'
rr
~
c- . r :..; c-
H- ,
~
,,
- C-
I ..
I: '
. I-
r- T_
c-'- I- I- ' - ,
eaGtllal"
1"1.
~C:l 1 PropMta'J PrOlNl~Do tfo4; COPt
Studa., Gu,d<> HVAC & COOling
,.
10·n
. IC$I ............. i PrtJp.1ty-Do Hot COp,'
_Guld. HYAC & Cooling
- -
• Cable penetrations into the botttlm of racks should be n lled
-- --
com puter room - do not mi~ perforated til ... with al r supply gri lies
• Chi lied air should always be delivered into the cold aisle in front of
the cabinets. rather than from beneath the racle since:
- Some of the air will pass out throug h t he back of the rack
- Air supplied directly Into the bottom of a cabin·e.t may be cooler
tho n manufactu...... recommendations
CRAC units shou ld be located in the hot aisle path when the return air
path is the free .pace in the room
• If CAACs/CRAHs are not fu lIy ducted for both air iota ke and
dgharge, they shOUld be arranged p~ndiOJta·i Itl equipment
rows
• Return air Intake ducts fur CRACs/CRAHs should be placed as high
up in the ceiling as possible and be ,\!Igned with hot aisles
10.12
ltell PRllft'liNr'/ P,.ftY-{)o NDIt Copy
Studont Guido HVAC & Cooling
~1~1I(k.
~t!rw'I"~d;
C<tn1""'~1 ~ AC unit
The maximum rack thermal load that can be air cooled is likely in the
• -
vicinity 0115· 25 k~ per
rack
A 42U rack Stilt"'' ' WIth
. 42 1U servers would oper.te at
approximately 16.8 kW (.4*42) average power and 24 kW at peak
power
• A 42U rack can currently hold 60 blade servers and operate 24 kW
average power and 32 kW peak power
• GiVEn that cunrent technologies are already pushi og the I imits of ai r
<coUng, rack oriented liquid cooling systems may be conSidered for
data centers geared toward especially high and dense thermal
loads
"-1'
BICSI ~Pn:lP'WtY'-1)Q ~",C4py
Sludalll Golde HVAC & CoOling
....,"'
F_
EqulprTlI!In!
GRAC
••
R.etvmAif
---.:.: ""'"
,...
HVAC '& Cooling
_W,W".... '
... , ..... t ... .,.
... - ...
... ...
... ...
---
.. 'Given that rack ortent ed solutions involve more coolin9 equipment
maintenance and repairs tend to be coSlller and more freQuent
• Individual rack cooling .",its can constitute a single point of failure,
and supplemental row oriented CRAC units may be required fo'f-
-
j
reliability/redundancy
1,,"U
sal f"rc~ P..,...,t)'-Oo Nt.! Ccpy
HVAC & Cooling
Ventilation 10.38
Ventilation is air su pplied to Or removed from a S9<'CE! for the purpose
of CO/'ltrolling air contaminant levels, humidity, and temperature
• Ventilated air is typically interpreted as t he portion of the SlJpply air
that Is "fresh" outdoor air lt1at has not been re-drl:ulated Or
translerred
• Human 0= pa ncy In data centers i. typically low; however,
remova l of internally generated pollutants needs to be oonsidered
• GR-3029-CORE recommends 0.25 air manges per hour for removal
of Internally generated contaiiiTlllrnts
- ThiS equals standard code reqU irements when tile total ceiling
height + raised floor height ~ 12 feet (3.7 m)
- It exceeds code requirements when this dimension is greater
than 12 f1: (3.7m)
1~tI
BICst ,~Pf'hbn't P~"01 CtI .. ,.
Student Gold& HV AC & Cooling
-
exhaust system for 2 cfm/sf (0.61 m' per mlnutelm')
• Redundallt exhaust fans should be providoq, along with an
iiftennate soorq of mal<eup air in the event of venti lotion system
r.ulure
• Hydrogen detectl<>n systems should be provided, and monitored by
the bulldln§"""aGIOffldtion/man.gement system
--
• D..ta cables under the raised access floor should run in the bot aislE
• Cab'l es shall not be lett lIbandoned.<mder raised noor, In overhead
cable trays, or 'aboye suspended ceilings. Inactive cables should be
removed or termloated on at least QFIo6..e.o1J and marked "'for future
use" ,
10'"
BICS! Pl'Clprillbl')'PYOf*'tY-lla ~~ Cqpy
S1vcIent Gukle HIIAC & CoolifIg
The rndundancy of a dat<l center's cool ing system needs to match the
target dass requirements
• Electl'lCal power for critical HVACequipmenl should be on generator
backup
• Data <lOnters with high density loads should Include provisions tD
keep equipment fmm overheating during generator start-up, sudh
as "ilving the fans powered by the UPS
• The lil/AC.centrlll system should have backup powerfeeds
• There shou ld be two Independent sourceS of w"ter for HVAC
systems that employ liquid coo ling
• Every facility should have a I!2rtab!eHVAC unit and an adequate
number of.)!!me fans..in storage to provide temporary cooling to
ec
critical uopmen!
Aoo....iId<
SUpploe:met1ti;ll
COOling Fan
ilmIU'I u:St'
10-11
SICSllI'tep,,",pY '«:IP'I~Ilj) Hot eo",
tfVAC 8. Cooing
Review Questions
2. What ere some of the main concerns wilh underffoor cool ing
architectures?
@2lnOSlcsr
$N<»m Guld.
U-1
IJCIf PnlPJWaryP~Do ~ot CaPf
Energy SfflclQncy & Green Data Cantllr DesIgn 11.2
D21110 1BiCSi"
Sludo .... Gulclo Green Olta c.t'Qr D.s~n
Green tI~ta center design attempts to denne the ways th~t in ~ial
design decisions affect power consumption over the life of the data
center-
While electrical power costs will vary by region, rurrently $.12 per kW
hr provides a reasonable estimate in the U.S.
• This trcmsl~tes to approximately $1000 per vear for each I<W of
load
• $400 a year to power a server
• $400 a year to cool a server
• Cun-ently, energy prices ~re t rend ing upwand
There are many steps one can lake to reduce the lifetime ~wer costs
associiltl:Cl with data centers. These Include:
• Equipment retirement planning
• Implementing server vlrtualization
• Designing ~dent lighting systems
• Right-sizing UPS systems
• Employing server power management
• Employing outside air and water economizers
• Improving under-flOOr air delivery efficiency
• USing supplemental enollng
" ..
I4C:M ,.. . . . rJ~~Copy
Student Qulde
"..
IUCSI PtofWletarypr~ Hol¢QPY
_010.01 .. G.- DaIa Conlet Design
In many data cenrers ventec1 tUes are orren misplaced ~nd too many
a re Installed
• If there are too many venred tiles In me data center, then cool air
pressore will be too low for cool air to reach the tops of racks
ClO10 BICSf
....
Brest ~ry'f'CIPIIIC)'--DO tb: Copy
S"dontGulde
.Right-sizing the UPS power system to the load will often have one of
the. largest Impacts on increasing the electncal system's elfrdency
• There are fixed 10S5eS In UPS power systems that are present
whether the rr load Is being served or not
• These losses are proportional to the over.11 power "'ling of the
system
• A UPS system runs at highest eff"'''ncy when it Is near Its
maximum rated capacity-as load level drops, so does etr",iency
11·7
lal Prv~1')' ProJ*tY Do- Hili Co","
1IIud.", BlOIdo
In most U.S. 'd ata centers, another opportunity for power efficiency
lies downstream from the uPS-!f1e UPS system typically provides
power at 4S0V, 'which is then 'stepped' down to 208V In the power
distribution system
• These step downs incur power losses that can be el lminated by
converting UPS output pawer directly to 41SV
- The 41SV three-phase output can proVide 240V sinllie-phase
input direc~y to a server eliminating step·down losses
- Most servers and IT equipment can handle 240V AC Input
witMout problem
......'
lAIWP'\,HH6
One Wi> Y to Increase cooling system efficiency ,md often create la rge
energy savings Is to raise the target temperature of!f1e computer
room air conditioning «(RAC) units
mtl10BlCS"
st:ucIent Guida
Another opport on~y for coo ling system savings comes from
coordinating ""mputer room air rond itioners
- Air conditioners runfling corronuOlJ sly with low output temperatures
can over-dry air which other AC units will then try to re- humidify by
hea~ng it
- This creates a condit ion for a room with multiple air conditioners
to have those air con ditioners simultaneously (and inefficiently )
heating and cooling a room at the same time
• Computer room air conditioners are rurrently available w ith
intelligent communication systems that allow for multiple units 10
coordinate room Illoling
An add itional measure that heips te> improve cool ing system effidency
is the employment of CRAC with variable speed drives
• Using va riable speed blower systems to m"lch cooling requirement.
can save signiftcant enel'1ly--a 10% reduction In air now
can result
in a 25% n:ductlon in the energy consumption
11 ...
INCSI proptttt.) ProJ*fY-UO Not CODY
Air Condltlonlnll Economizers 11.26
Mlsskln critical facilities are sized for peak usage condition; ihat rarely
exist; server ma nagement software c.on help power down IT
equipment during periods when usage is low
• Pata center IT equipment usage typlClllly falls off from ~ pm to S
am, but its power consumpllon remains rela~velv high eVeil after
computation load decreases
• In Idle mode, most servers still consume around 75 percent of full
capacity power
• Most modem servers have built-In power management feature. to
reduce this idle power consumption; how~ver, these f~ab..Hes a~
often di.abled because of concerns over response time during pea k
usage hours
• Server management software can oversee and regulate a ~ervers
built-In power management features so that these feature. are only
employed during off-peale hours
1'·10
lUeSI P'rap~P1'o~~ Hot Co",
Energy Efficiency SUmmary 11.28
1h1t
l a l Ptut-.... , '"'~ 00 ttol CtIpJ"
StuMm Guide ~ ~ Can.... O•• lgn
Review Questions
1. Why ti~s,,'t
energy efllclen.:y tr;Idit"",~lIy been part of the Initial
~esjgn criteria?
3. Wh~t do you thlnl< Is likely the best w~y to adl leve energy
effICiency In the data center?
12.1
BICSt PrcfIrIaiMy hofN'rfY--Oo "'CIt ~
_on! Guido Telooom""mlorion. '" CaillIng
--_. -
(rIIl"roU
fdlfT
iMllloo!~
";roI~mr
1,
J
"""
no,..""......
.....
..... . ""... I
f)I.~., ..I...'1
I.<p.llpmljllt
1lI\1'~TllOtI
N", '- llllO:n
~~2"1
Coordinating with Aece5s Provide .... 12.3
One of th~ earlie·s t ancl most important step. in the design ~nd
impJementatlon of a data center's telecommunications system i.
coordination with the provld~«5) . Information the design,,, will
typically need to give ta service providers includes;
• Add reS/; ot the building
• PI~ns of the entrance conduits from the property II~ to the
entrance area
• Assignment of conduits and inner ducts to the provider
• Routing ot cables in the entrance area (acceS/; floor vS. overhead)
• Quantity and type of circuits to be provided
• Locat"'" and interface for demarcation of each type of circuit
provided
• Camer office diversity required for target r~lIablllty class
,W
OIell .. -.yP~ J.loI COPt
" ............
S1udent Guidlr TCtltoommunlcatlons & CI~lng
Once co-ordination with the access provider has begun, deslg ning the
pathway from the provider. Into the data ce nter.",," occur
• Underground cabling is recommended
• COstomer owned satellite dish Fanns or communication towers
should be located as close as possible to the perimeter of the
fatility in a secure a rea
,»
SICS' • . . . . , . , . . , _ .... Co"
_Guide
Muttip!e "n("moe pathways from the property line to the entra nce
mom(.) eliminate a Single point of failure and are required for Class F2
fac1J itie:s and above
• The ""'Intenance holes and entrance pathways should be on
oppos;te sides of the building and at le"st 20 m (66 It) apart
- Data centers with two I!ntrnnce rooms should have at least two
entrance abl,," through diverse maintenance holes
6i610 SiCsI'
The Entra IKe Room 12.10
B ,
J
=
12.11
The entronce room/area should be outside the computer room to
improve security
• Addition ill entr. nce moms may be ""luired to support distance
limited Circuits such as T ·lIE- l and T·3/E-3
caDl0 iiiCif
SI\Idooi Guide rolecGm>unlcotk>n& & Cabling
The entrance room will have up to tour separate areas for aCC<!SS
provider demarcation Including:
• Low-speed copper-pair circuits, (05-0, ISDN BRI, and telephone
line')
• H/gh-speed.copper-pair circuits (DS-i, T-l, ISDN PR!, CEPT-l/E-
1)
• Coaxial.;,,_ circuits (DS-3/T-3 and CEPT-3fE-3)
• Optk:alllber circuits (SONET OC-x, SOH S'rn-" FDDI, F"st
Ethemet)
Access providers should provide demarcation toelheir circuits in
one of these area s (delineated by circuit type1
l.ntr.lllrt! ArC'~
t:r20tll IIIclI,.
, ~..,
• The 10C connecting hardware can be pl.ced on the same ,""ck for
low-speed drw its
. tH
'JC"~~Pr,,~IfOtC~
studonl Guido T_unlcatlon. & Cabling
Cabling from the fiber demarcation area to the main distribution area
snould be sing Ie-mode optical fiber cable
• If the .entrance services are terminated in multi mod" optical fiber
cabJe, 111.. data center cabling can also include multi mod" optical
Hber cabie
_.
The heart of the backbone cabling system is the Main Dlstr1butjo n Area
(MDA), the centrol acteSS distribution point for the data center
• Backbone cabilng also connects tile Main DistTibution Ar"'a to the
rest of Ihe Oata Center
- The Ma in Distribution Area is typically in the computer room,
though It may be placed in Its own room for i mpreved se<:urit;'
H.>rizornhll
Dln,l"",nolfl. ~
H c> r Ib:I J)'IoJo
lntr.m=fbam
...
Dn.! IlootlDflo
~,
~"
....
C~m.IIIlClIItlon ~In
/\"J4
Dbmhutlon
I
."""~
1,,,,,,,,,,
RboM
rnl:t'UllIIC3r
12'·10
lreSI ~'Yp,~ Not Copy
T".commUftJeat~n. & elbing
Ring
I Il n.o.
"DO.
'1-1,
.C~I~ryP'""ty OG"'~
student Guida T._muNe .. lons & Cabllnog
I;;;~;-'l 2-;i' u I
Mr..'
DistJibuled \IS.
02Q10 BiCSf
1MZ
1110 ............ "'.....,.....0. wore ...
studern Gukle TelKomrnunlatians & Cab ....
M,,. .DO
"".
€l2110 IiCit'
I IC81p ..
___ '2·"
, P/'6f*1)-Cc ~ot: Copy
.....
S...-.cGuldo T.~mmunfc~n. & CalMIng
I·.., .. ,"~ I P
F.tll'~i'I. 41
E""~""'''
"""" , "-,
Mo.o, OlD>
H l)l loWnl.'
C~Soi '~ Cln~CI
i -"""
t.J=-~-CCJn lM'!:t
ibii1' Ilear
11-14
81CS. "'q~ Pt9pe..,....oo Hen COpit
Te4ec:ommUnjcoti.,... II. Cablk>g
1M8
BlC31 'io,rre'I:WI"~Hot eo",.
Student Guide ToI._nlo..vono & Cabling
D2f1tO IiIICSr
SWdontOukio Teleeommunlcattons &cabling
The IiDA typically includes a horizontal cross connect, LA~, SA~, and
KVM switd1es
1~·1.
IIM:SI ,,~w ~I*"t-DD Nel Cop,.
Student Guidi
oNtll Bar
12:~19
oc "110: ow. e.,.,. ,*-.." &m PnIoJ:IIo_
ElICa. """'p,.....,~ ,..Copy
HDf1zontal Cabling System ~dundan"y 12.<19
Horizontal cabling redundancy should also be employed to preV<!nt
outages d~e to cable failure
• ~edundant horizontal cabling can be achieved either by di""~e
noutes within a single HOA, or by dual noutes from different HOAs
- The dE!llree of redundancy from diverse HOAs typically will not
offer much mo~e In terms of increased reliability if the HDAs are
in the same fire zone
- Ensure that redundant cabling does not exceed its length limits
--
HDA l HI>A
, /
- -'
--
En>
'''''
"'"
.nj......"'f PAJPfit7-00 Not ¢op,.
SlCSI f'T'......
student Guide T,"communlu'lions & Cabling
"Ft<d &uancdoom
-;
Mllol
.., • ~,..
/ "
fc.AE ""~,; "-'/
F.:' ''+1',':1::' .~
--
b~ll'UI::I' ~
£OAt Itadr!.
Class F3 12.55
.
Primary Au.f?5
'ZFeed SC'I;Qn~r)'
(nh'~I'c. ROOfl"l t:nrrMce ROcIm
,
-
MilA
Fu II y Redundil nt
HOA Networi< oevl <es
EOA J Racks:
SludontO.-
Class F4 12.56
"n~
frim2lry A
En!..,nc. Room Entnn~ Room
/
~
..,
........
- .--
MOA MOA -
~........
./
HDA HDA •
- I
EDA I I\4ciu
i
~tll.ICSi"
Studonl Guide reteeommunle.ttons & Cabling
Review Questions
&015 Blei"
"'"
Bleil PrvtIIfW;IJY "~"Y-Do MDt CcP1
Module 13: Building Automation, Monitoring, & Control Systems
u ·,
.ncII Prop~ PrOfMl'tY-Do WcIot CoPW'
Student Gujd. II<JIldlng Automation & Control 811._.
Building Automation Systems 13.2
om~tOBICiii"
,.. OC1tCl'; om e.m.ro.lIIn .... Pr,IIW,","
UlO11 BlCSF
studont Guide
The BAS cabling system will be deployed throughout the data center
and should :
• Integ rate common services
• Accommodate d iverse. BAS appl ications
• Provide the capacity for rapid deployment of new equipment and
services
• PrOvide redundancy to satisfy safety and reliability d ess
requirements
• POE midspan devices shan be instal~ where til. link end (such as
the MDA or HDA) ~ides
C IF" l nA~rm
TD Indi",;dLNI e .....
S~:Iodor.l. __ =- "-. •
""A
r;c.t'I"~1 ROlOn'!
Q2IItO .iCii" DC ttl: 0-. c.tI .... 0.-'0" & Beat Pnll:ticH
Student: GuIde 8ulldlrlllwloma."" & ConlrOI S)<$tam.
The BAS ca bling system may use sej:>arate pathways lor improved
safety and security
• BAS and communications cabling should always use separate
pathways whenever there is likely to be electromagnetic
Interference
• Pathways and spaces for BAS cabling should be clearly markee
• Data center BAS cabling should terminate In separate dedicated
enclosures and patch panels
• BAS endosu",", and p..tch Pilnel. shall be cI""rly marked
e2:C110 Dlesl'
Studont GuI<W
021'1 Blei"
...-
.."................
_ ...",. rrw.~ NatcbpV
Student Guide Building Au'e .... '1on & Contlol Systemo
-
Ulility 1kI. N. No ~"" Yes
Genl!lfBtor 8.,./
... loG V"" ". Yeo
Generilt4;tn Veo Yeo V.. Yeo
Mechlinlc.r Power No · Na Yeo .y.,;.
&r-
UPS Bus No Ve. Yo. Yes
_ .. ,_ed
UPSJ4Odut_ Ye,
Ot!$i5"8ble
Ye~
Yes
Yes
V••
Ye,
Y••
h~Svstem
FlreAI.",
No
No
NO
vos
Yes
No
~es
Yes
YO!;
·V.,
Ves
Yes
Yes
y.,
Yos
Yo<
sy .... m
........rQlNoIity No UPS OLltpoJt only utillt'f, -rhroughDUt
Genem:Qf5 alld
·UPS· output
blitab ... tor No No Yes y""
"..
alannl!ii
_..,.·Q...itty No ·Yes yO<.
-.1",,1"11
........-
Utility
''"'.
IIIICB. PftIptieUty ~ 00...,. Copy
Student Guide Building Automation & c""""" Syatomo
Revi_ Questions
130-11
bICSf PNprllOf)'h..."".....Oo WrJt CI;IP'f
student Guide Commuionlng & Malmelllnc.
- Construction
- Acce ptance
- Post accepta nee
~UI8iCSi'
CommISlk)nlng & Malm:enant::11!1
Dunng the "acceptance" phase, performance test ing takes place and
the following occurs:
• Systl!ms are calibrated
• Performance gUIdelines established during the design phase are
checked
• Nonperforming systems are fixed
Objectives in the acceptance phase Inelude :
FUnctional performance beSting
ComplelJon of site audit
ComplelJon of warranty audit
Final dorumentatlcn submittals and all test reports
The five phases of the commiSSion Ing process can be broken down
further Into II1ree cruda l components :
t Documentation
• Testing
• Creation of maintenance procedures
,....
I81CSI PruP"'rfPt~ Mol. COp';
Commissioning & Maintenance
Documentation 14.10
Documentation is a CEntral fec>ture of the
commissioning prlXess .md
involves having foor types of records on hand:
• D~i9n DOOJment<
• PrO<;ess Documents
• VerilJcation Documents
• Ope",tion '" Maintenance Documents
• Building Drawings
• General equipment spedficatlons
• System subm~tals
• An:hl~ural requirements
• Change order(s)
• Seq uence of operation
....
II(" ProprlltrfPropwt)' 00 N;otCopy
51_Guido Comml..klnfng &Mafntenlnce
• Control drawing.
S.udtnt Gukla CommiMl."ng & Molin_nee
Once the data center has been commissioned the following document
sheu Id be stored ensite :
• Contract Documents
• Specffic;:ations
- Floor l'1a n Drawings
- CltJtside Plant Drawings
- Connectivity Riser Oiilgrams
- EqUipment Layout Drawing & Details
• Rack &. Cabinet Elevations
• Cilble Pathway Details
I .... As- BuiltH Drawings
• Con.tructlon Administration Reports
• Test Reports
• Punch -list Report
• Opera~en5 and Maintenance (0&",) MilnUals
• Close-out, Sign-<>ff and Atteptance CemHcates
- Certificate of Oc<:upancy
• Ch~ nge Milnilgement documentation Ind "ding system inventories
• Relocation planning documentiltioo such as eqUipment inventories
Fln,,1 Commissioning Roport
'''.17
The following should be included in the Anal Commissioning Report:
o Project name
• Name, address, firm t and phone number of Commiss4oning
Authority
o Description of the building (5i<e. Location, Cllnstructlon)
• HVAC and other il'lStalied systems
• Ust and description of commissioning !aSks
• Commissioning Plan
• Completed design intent document
• Completed pre-functiona l test cheddists
• Completed functional pruionn. ne<! tests
o Any non-rompli"nce forms
o Summary of f,nd lngs
o Recommendations for monitoring the ongoing performance
o Ree<>mmendations for system improvements
.U
DK:IlfI~Pro~MGt Copt
Funcllonal Testing Components 14.19
Functional testing of individual data c..nt er components should Involve
the following ,
• Equipment descMptlon
• Description of purpose of t he test
• Detail of requ ined personnel and equ ipment
• Design information pertlllent to the system under test
• Detailed sequence of operatiOns
• Special Instruc:tions or warnings
• Dennltlon of ex~ed results
• Sampling strategies
The proced ure for carrying out fu nctional tests should be as follows,
• Inspect equipment for manufacturing and installation defects
• Document test <DndltiOns
• Perform Integrated sy.terns test
• Verify test ~on$e
• Compare actual response to acceptance criteria
. ~
1UCS1.....,....iaryPrl)peIt)'-Do MIJt CIIpY
Studont Guido
,. UI"PC> s.
1'0 est.!lblbh '9U!p$ nlPCe!SSiilry ~(I fini&h QOmml:n~o'f'linO tnt J:lOIJ" N'I<I periotm tI'I~ 11'1tegrl!ltN Systerrr\S
Tat in cu~ 1'001'10 "1 ..... hite .,...,lrnll:lI\g p~u. ifJIpe,ct to a1llcel cu9tOmer ~u"m~1
Ol'ervitllll
• COI'ltli!d temporal)' polll'l!lr jw.p." from M188 tit~"" 10 1MClPSOP2
• Shift crilitlilil bad <0 UPS 28 jl tM ASlS POIJ f.e'¥ei .
I Triill'lSftlt UP'S 11$"""A. to Jntemtll bws" flom UPS modI!
, Eniffijl~ A side POV!!; .. CU5CDmBr room #,
• TllIn~ UPS ~tam 1A from intftnaJ b)'Das:.ti to UPS mOCl:t
,,,ottO
elC$l ""opo~ryPI~ NolCop't
Student Guida Comfr'M ..lonlng & M.ime,.fllillOe
Each da~ center sub-syst<!m will have unique maintenance needs and
reqUire special maintenance planning
/ """)
...,,~
, /. •
~
I f:Oalll'lJ.
! "
~ule
Commlulonlnu & Mointonano.
. 1.1)
IiICSI Ptoplk.ibd, PNopIIrty-Oo NatCop'f
_Goide
Monthly:
...,.
BICSI ... opiCIry PmpIII'ty-Oo HlIt Copy
5tudont Guklo Commtsslon~1lI & Yaintenanctt
Quarteriy:
• Check watEr lines for leaks
, Vefify tightness of blower and its motor hardwa re
• Record amperages for critical components (such as cooling coli fan
motors, reheater, humidifier, etc)
Semi-Allnually:
IT Maintenance 14.32
.4-1'
BleSI "'p........_
......1J!~ Ntli Cop,
student Oukle Comml..Jonlng & Mllnt«nan.-
Review QuestIons
'<-1'
IICSlflnoJiMbtyA'Operty-Oo Nat Copy