Tri-Generation for Data Centres Lessons Learnt from a recent installation

Michael McPhee, Dip Mech Eng, MIE Aust, Chartered Professional Engineer, MAIRAH MASHRAE Associate Director, Umow Lai Pty Ltd Michael has specialist expertise in Data Centre Upgrades having completed a number of major projects for financial institutions and telecommunication service providers in recent years. Brian Lacey, Dip Mech Eng, MIE Aust, Chartered Engineer Senior Mechanical Engineer, A.G. Coombs Pty Ltd Brian has been involved in a number of data centre upgrades as well as the installation of a number of cogeneration systems including both micro-turbine and lean gas engines for commercial, data centre and industrial projects.

Abstract
With data centre sector carbon emissions expected to exceed those of the airline industry by 2020 and rapidly increasing energy costs, the use of Tri-Generation systems is seen as a key strategy available to Australias Data Centre owners and managers to mitigate these risks Requiring significant capital investment, large Tri-Generation systems can provide high payoffs in terms of carbon emission and energy cost reductions. Whilst tri-Generation systems are being considered for new data centre facilities, retrofitting and commissioning of large Tri-Generation systems is considered to be particularly challenging within a live data centre environment. NAB are making a significant commitment to the environment through their Carbon Neutral 2010 Program, with carbon emissions associated with their Data Centres targeted for particular attention. In 2010 they completed the installation of a 2000kW Tri-Generation system at their primary data centre facility with the joint aims of reducing the Data Centres carbon emissions by some 20,000 tonnes of CO2 per annum as well as providing a positive financial return. It should be noted that the installation of the Tri-Generation plant in the Data Centre was an NAB initiative. The initiative was supported by Banks Facility Managers, United Group Services, who provided significant input and cooperation during the design and construction phases of the project. The project presented many engineering, installation and commissioning challenges whilst assuring the overriding requirement that the data centres service availability and system reliability were not compromised through the project delivery. Now that the plant is up and running, Brian and Michael present some of the key lessons learnt through the project implementation from the design phase through to final client handover, as well as review the performance of the system to date.

Achieving the Green Dream Tomorrows technology today, Melbourne, 2010

1.1. Green Data Centre

Data centres operate 24 hours per day and are large users of energy. IT research company Gartner has released information noting that 2% of the world wide carbon emissions produced from Data Centres According to Gartner, this places the energy usage in Data Centres on a par with the Aviation Industry Consequently due to this large energy consumption, there is a push in the Data Centre industry to build Green Data Centres Data Centre Electrical Load demand has increased significantly in recent times driven by Technology developments. Blade servers have become more prevalent. The performance capabilities of these servers has increased exponentially, so while the Data Centre white space or Data Centre hall has not grown much in size, power and cooling demands have also increased significantly. The Data Centre industry has addressed this by Virtualisation and Cloud Computing in an effort to ensure that servers are fully operational at all times so that the power consumed is being converted to maximise computer output. Umow Lai have monitored this design power loads for Data Centre projects in recent years. This growth can be shown in the following graph
3500 3000 2500
3000

W/m2

2000 1500
1200 1500

1000
800

500
300

600

0 1990

1995

2000

2005

2010

Fig 1 Umow Lai Experience in Data Centre Design Loads

Achieving the Green Dream Tomorrows technology today, Melbourne, 2010

In recent years there has been a number of initiatives developed to reduce the power and cooling demands. These initiatives include Hot/Cold Aisle Configuration Hot Aisle Containment Air / Water Side Economy Cycles Kyoto Cooling Chilled water supply temperature Raised Internal Design Conditions (Refer 2008 ASHRAE Environmental Guidelines for Datacom Equipment Expanding the Recommended Environmental Envelope) Supplementary cooling, in-rack / in-row cooling Tri-Generation

1.2. What is Tri-generation?

Traditionally Data Centre utilised Electricity to provide power for General Light and Power Chillers and Fans IT equipment TriGeneration is a method of using one fuel supply (Natural Gas) to produce 3 sources of energy, namely: Electricity Heating Cooling For a Data Centre there is no need for the heating so waste heat is converted to Cooling energy in an absorption chiller. A schematic representation of the principal elements of a Tri-Generation system are shown below:-

Fig 2 Tri-Generation system Fundamental Schematic

Achieving the Green Dream Tomorrows technology today, Melbourne, 2010

Fig 3 - Exhaust gas 2 Stage Absorption Chiller fitted with Gas Burner (Photo courtesy of Broad)

Fig 4 Natural Gas fired Spark Ignition Reciprocating Gas Engine and Alternator (Photo courtesy of Cummins Engines)

1.3. Why is Tri Generation Suitable for Data Centre

Anyone who has worked on the design of a Data Centre would be aware of the criticality of the Data Centre operation. Any downtime can result in significant cost to the Data Centre Operator. The criticality and performance of the IT equipment as well as the relatively low cost of electricity has meant that in the past energy consumption has been of less concern. With the exponential growth in the Data Centre loads and the reduction in Server costs, the power and cooling plant and the associated energy costs to run this plant are now becoming more important in the eyes of the Data Centre Managers. Tri-Generation is now being proposed as an option for supplying power and cooling to a Data Centre Tri-Generation plant has a number of advantages which can be summarised as follows: Local generation reduces transmission and distribution system losses. Gas has higher calorific value. CO2 emissions for gas fired plant are significantly less that for electricity produced from coal fired plant. CO2 emissions of electricity and gas supplies in Victoria are: Electricity 1.34kg/kWh Gas 0.21kg/kWh Gas is cleaner. Gas is usually reticulated to the site even though an upgrade may be required to suit the new load. Heat recovery increases efficiency of energy sources. Lower carbon footprint. In addition to the above, the Commercial Property industry is embracing Tri-Generation technology as a means or reducing Greenhouse Gas emissions in order to achieve higher NABERS and Green Star ratings. The effect of this is that Engine and Absorption Chiller manufacturers are providing more commercially available equipment, and the level of expertise in the industry is improving.

Achieving the Green Dream Tomorrows technology today, Melbourne, 2010

Data Centres usually have a constant power demand for the IT equipment. The power to IT equipment is converted to heat in the Data room, so unless there is an economy cycle in operation, the constant power demand results in a constant cooling load. This is an ideal application for Tri-Generation, as a plant selection can be matched to meet the power load and cooling load. Below are examples of the load profiles for a Data Centre project.
6.0 5.0 4.0

MVA

3.0 2.0 1.0 0.0

1/ 01 /2 00 8 20 /0 2/ 20 08 10 /0 4/ 20 08 30 /0 5/ 20 08 19 /0 7/ 20 08 7/ 09 /2 00 8 27 /1 0/ 20 08 16 /1 2/ 20 08 4/ 02 /2 00 9 26 /0 3/ 20 09 15 /0 5/ 20 09 4/ 07 /2 00 9

Fig 4 Data Centre Maximum Demand Over 1 year

4.9 4.8 4.7

1/02/2009 2/02/2009

M VA

4.6 4.5 4.4 4.3 4.2

0:00:00 2:24:00 4:48:00 7:12:00 9:36:00 12:00:00 14:24:00
Time

3/02/2009 4/02/2009 5/02/2009 6/02/2009 7/02/2009

16:48:00

19:12:00

21:36:00

0:00:00

Fig 5 Maximum Demand over 1 day

Achieving the Green Dream Tomorrows technology today, Melbourne, 2010

Also below is a graph showing the break up of the maximum demand load
4,500

4,000

3,500

3,000
UPS Loads Max Demand Light & Power Loads

KVA

2,500

2,000

Mechanical Loads

1,500

1,000

500

0
A ug -0 8 M ay -0 8 Ja n08 A pr -0 8 N ov -0 8 Ju n08 D ec -0 8 Se p08 M ar -0 8 Ja n09 Ju l-0 8 O ct -0 8 M ar -0 9 Fe b09 Fe b08

Fig 6 Break up of the Maximum Demand Load

From the above graphs it can be seen that the load for a Data Centre is reasonable constant over one years operation. This is a quite different load profile from other types of commercial buildings, where there is a high cooling load in summer and low load in winter. This load profile allows a Tri-Generation plant to be selected, which can match the power and cooling load and result in a fully loaded gas engine and absorption chiller.

2. Design Phase Lessons Learnt

During the design phase of the project a number of challenges have arisen that would not normally be encountered on a traditional Building Services Dsign Project.

2.1. Reliability Redundancy and Risks in the Data Centre Environment

As mentioned above the continuity of operation of the Data Centre is a major concern to Data Centre operators. While energy saving features can be considered, these will not be introduced if there is any risk to the Data Centre continuity of operation. During the design of our project, four (4) options for the TriGeneration plant connection to the Data Centre were considered. Option 1 - Tri-Generation connected in island mode and supplying the whole Data Centre (2 x 1500kW gas engines with matching 2 stage Absorption chillers) Option 2 - Tri-Generation operating in island mode and supplying the Data Centre UPS load (1 x 200kkW gas engine with matching 2 stage absorption chiller) Option 3 - Tri-Generation operating in island mode and supplying the less critical mechanical and building light and power loads (1 x 1500kW gas engine with matching 2 stage absorption chiller) Option 4 - Tri-Generation embedded with the Electrical power grid and supplying the data centre on the HV side parallel with the Grid connection (1 x 2000kW gas engine with matching 2 stage absorption chiller)

Achieving the Green Dream Tomorrows technology today, Melbourne, 2010

Option 4 was selected for the project for a number of reasons including the shortest overall pay back period. However, one of the main advantages of Option 4 was that the on gas engine failure, the grid seamlessly took the site load with no interruption to power to the site. The Tri-Generation system adopted for the project therefore stands alone on the side providing power to the Data Centre. The Grid supply and standby generator supply are still fully operational so the Tri-Generation does not reduce the electrical supply reliability. If one of the other options had been adopted, as the TriGeneration was operating in island mode from the grid supply, there would have been a break in the power supply to the data centre on gas engine failure, which would have required the UPS to maintain the site load until the grid supply could be restored or alternatively a type of bumpless transfer would be required. So the lesson learnt here is that, for a Data Centre application, the Tri-Generation plant should be connected embedded with the grid to ensure there is no unreliability added to the power supply system. There are some other risks that need to be considered that may arise from the installation of the new TriGeneration plant. These include fire and explosion risk of introducing a gas engine into the Data Centre. These risks need to be addressed and risk mitigation strategies developed.

2.2. Engine Selection Considerations

The experience gained on this project is based on the installation of a Natural Gas Driven Reciprocating Spark Ignition Engine connected to a High Voltage alternator. Other types of gas engine are available on the market such as gas turbines, but reciprocating engines are becoming the norm in Tri-Generation installations in commercial projects. Some of the interesting points to be considered in selection of Gas engines are as follows: Generally gas engine suppliers provide 2 types of engines. One type is a high efficiency machine. This machine is a light construction suitable for base load operation but not well suited to step loads. These machines are best suited to run embedded with the grid as the grid can absorb the large step loads. The other machine type is normally labelled as a standby duty machine. This is a more robust machine capable of taking larger step loads and is more suitable for standby duty. However a gas driven spark ignition engine will never have the step load capacity as might be expected from a diesel engine and generally gas engines are not good at absorbing large step loads. So careful programming of controls in order to limit step loads is essential in the design of Tri-Generation equipment. Gas engines consume lubricating oil. A separate lubricating oil storage tank and fill arrangement and pumping system needs to be installed to ensure that there is adequate engine lubrication. On our project, we needed to install a storage tank of 3000litre capacity to provide approximately 1 month supply of lubricating oil Flue design from the engine is important. Flues need to be designed to minimise back pressure due to the additional pressure drop through the absorption chiller. Also as gas is used as a fuel for the engine, and there is the possibility of unburnt gas entering the exhaust, the flue must be designed to meet the requirements of Australian Standard AS3814 Industrial and Commercial Gas Fired Appliances and flue must be able to withstand 700kPA pressure.

Achieving the Green Dream Tomorrows technology today, Melbourne, 2010

2.3. Electrical Connection Considerations

As mentioned above the connection of the generator to the site electrical installation is an important aspect of the Tri-Generation design. Some points to be considered include: The installation of the gas engine in parallel with the grid increases the fault levels in the Electricity supply network. Many proposed Tri-Generation installations in Melbourne have been put on hold due to the low fault level capabilities of the electrical supply network. There are methods available such as fault level limiters available however the cost of this equipment is high and this cost can render the project not viable. As mentioned above, the plant can be connected in Island mode, however, in Island mode the total generator must have the total capacity to handle the load including any step loading scenarios. Therefore the gas engine capacity must be larger than the predicted load to ensure that the plant is not overloaded at any time. Overloading of the generator will stall the set. This oversizing means that the plant capacity is not fully utilised. Also it should be kept in mind that if the generator is not fully loaded, then there will be less waste heat available, and the absorption chiller will not produce full capacity. With the gas engine connected in embedded mode, the generator can be sized at lower than the site load and so run at 100% as a base load. The additional site capacity required can then be made up using the grid supply. Generators can be connected on the HV side or the LV side depending on the arrangement of the power supply at the Data Centre. There is significant electrical design required to ensure that there adequate protection devices to prevent the back flow of power into the grid under fault conditions. These details are spelt out in the Supply authority specific requirements for embedded operation. These requirements need to be obtained from the Supply authority prior to commencement of the design

2.4. Absorption Chiller Considerations

The selection of Absorption chillers is important for a successful Tri-Generation project. Some aspects to be considered include: There are many different absorption chillers available on the market with equipment from China and India becoming available in recent times.. The technical expertise on servicing absorpton chillers is improving as in the past this servicing was often seen as a black art with only a few people knowing how to fine tune these machines. Chillers are available in 2 stage high efficiency (COP of Approximately 1.3) and single stage less efficient (COP of approximately 0.7). Chillers are also available as exhaust gas machines where the engine exhaust is taken into a heat exchanger integral with the chiller. In addition hot water chillers are available which can use waste heat from engine jacket water and an exhaust to water heat exchanger to produce chilled water. There are also chillers available which can take the engine exhaust into the first stage of the chiller and the jacket water waste heat in to the second stage. In a Data Centre there is unlikely to be a requirement for waste heat to heat the building, so it is preferred to utilise all the waste heat to produce chilled water. Careful consideration of all the options needs to be carried out to ensure the best overall chiller selection is made for the project. For our project we selected a 2 stage exhaust gas chiller and utilised the jacket water waste heat in the second stage to maximise the chilled water production. Absorption chillers reject large volumes of heat to the cooling towers. Cooling towers need to be sized to handle the heat rejected from the Data Centre, the engine heat and any jacket water heat not

Achieving the Green Dream Tomorrows technology today, Melbourne, 2010

utilised in the absorption chiller. Cooling towers for absorption chillers are 2 to 2.5 times larger than cooling towers required for conventional electric chillers of similar capacity. For our project the absorption chillers did not provide sufficient cooling capacity on their own to handle the site load and needed to be supplemented by the site electric chillers. Chiller controls need to be arranged so the absorption chiller acts as a base load chiller and the electric chiller tops up the load. If the controls are arranged so the electric and absorption chillers share the load, then the absorption chiller may not operate in a fully loaded condition. This would result in the waste heat not being fully utilised.

2.5. Technical And Building

Some of the technical and building issues which have arisen during the design our project have been detailed below.: Data Centre power and cooling loads need to be fully understood over the year and over a day to ensure plant correct sizing. Large items of Plant The engine on our project weighs approximately 17 tonnes and the absorption chiller weighs approximately 32 tonnes. The building structure must be able to accommodate these weights. Also the large footprint of the equipment means significant plant space is required. These are more detailed in the Installation section of this paper. Noise issues As the engine runs 24 hours the gas engine should ideally be located in a separate room to ensure the noise issues do not compromise maintenance on other data centre plant and equipment. Exhaust noise levels need to be considered. Noise levels from this type of plant will need to be lower than a typical standby generator to meet EPA requirements. OH&S issues need to be considered for maintenance engineers operating the plant. During our project, we had a specific OH&S meeting with Facility Manager and prepared a risk assessment at the completion of the meeting. Engine room ventilation needs to be considered. The engine consume air for combustion. Significant heat is rejected from the engine to the engine room. Large volumes of air need to be introduced and exhausted from the room. This ventilation system needs to be designed to ensure that room temperatures do not derate the engine performance or that room temperatures are too high for OH&S maintenance. Evaporative cooling was provided on our project to reduce the air volumes however the water consumption of the evaporative cooling unit needs to be considered.

2.6. Authority and Utility Considerations

A number of Authority and Utility considerations need to be considered in the design including the following. EPA Approval The EPA in Victoria determined during our project that a licence to discharge was not required for a plant having a capacity of less than 5MW. However this needs to be considered on a case by case basis, as the requirements in each state may vary. It is understood that NSW have more stringent requirements for discharge. It would be a good idea to allow space in the exhaust piping to install a catalytic converter should authority requirements change in the future in this regard Town Planning Approval from the local council may be required if a new plant is installed. They could have concerns about the physical appearance of the plant on the exterior of the building and the possible increase in the background noise level. Electrical Grid Connectivity Approval from the local electrical supply utility must be obtained particularly if the plant is to run embedded with the grid. On our project the utility required us to carry out a Network Fault Analysis of the grid at our expense. A separate sub consultant with

Achieving the Green Dream Tomorrows technology today, Melbourne, 2010

specialist expertise in this area was engaged to carry out this analysis. The analysis was needed to be done and submitted to the Utility prior to approval being received. In addition the electricity authority costs, which are significant need to be included in the project budget. Consideration to completing the contract documentation between the electric utility and the client early in the design phase may minimise the possibility of delays in the delivery of the project. Gas Supply Utility Approval must be obtained for the gas supply company to connect a gas engine to the site gas supply. As gas engines can operate 24 hours a day, the normal reserves in the gas supply street reticulation are not available to supply the gas engines. On our project, reinforcement was required to the gas supply mains in order to supply the new plant with significant associated costs. It should be noted that the gas companies are encouraging the installation of Tri-Generation plant as there is an increase in the gas usage.

2.7. Financial Analysis

The detailed financial analysis of a Tri-Generation project is critical to the successful project. The following points need to be considered in the analysis: The cost of gas and electricity. These costs vary from client to client depending on their purchasing power. Whether of not the cost of carbon is to be included in the analysis An accurate assessment of the ongoing operating costs (Opex) needs to be included. This means obtaining information from the engine manufacturers. It should be noted that spark plugs need to be replaced at 1000 hours, oil changed at 2000 hours, engine overhauls at 30,000 hours, and major engine overhaul at 90,000 hours (or complete replacement). Therefore it could be said that a gas engine which is operated 24 hours per day has a life of approximately 12 years A 2000kWe gas engine does not provide 2000kWe of power to the site. The parasitic losses such as power for cooling towers ventilation fans and pumps needs to be deducted for the engine output to ensure an accurate financial analysis. Capital costs need to include Gas supply costs, electricity utility costs, Network analysis consultant costs, Design consultants cost etc, as well as the plant supply and installation.

2.8. Project Procurement Methods

There are a number of methods to deliver a Tri-Generation package for a client. Consideration can be given to purchasing a turn key package from a specialist supplier. This supplier can build own and operate the TriGeneration package in the clients premises. The supplier will then sell electricity and chilled water to the client. This has a number of advantages for the client in that the responsibility of running the plant falls on the supplier. However, with this arrangement the client may not see the full savings. On our project the client elected to build their own plant and operate the plant themselves and so realising all the savings available from Tri-Generation plant During the design process we needed to decide on the project procurement method. Due to the long delivery of the major plant of approximately 6 months it was decided to pre purchase the gas engine /generator and absorption chiller. Tenders were called from the major engine companies for the supply only of this equipment. This method allowed us to select the most cost effective equipment for the project. Once this was selected, we prepared the installation package documents and called tenders for the installation. This method allowed the installation documents to be developed around an actual engine and absorption chiller and so ensure that all equipment was covered in the installation. The other advantage of this method was that it allowed the best engine and chiller to be selected. The lowest capital cost engine does not necessarily mean the best return over the equipment life.

Achieving the Green Dream Tomorrows technology today, Melbourne, 2010

2.9. Client Ownership of the Concept

A Tri-Generation project adds significant complexity to the already onerous maintenance regimes in place for a Data Centre. The engines have many moving parts and controls interfaces are complex. The Client and Facility Manager need to be aware of this complexity and what they have taken on. It would be a shame if it all got too hard for the Facility Manager and he was to switch off the plant because he did not have the resources to adequately maintain the plant. Clients and Facility Managers need to have a long term commitment to the ongoing system operation. On our project the Client and Facility Managers have this commitment and have shown great interest in ensuring the plant remains operational and that the Greenhouse gas savings are realised.

3. Construction Phase Lessons Learnt

During the construction phase of the project a number of challenges have become apparent which are not normally encountered in the construction of a traditional Building Services project. This paper addresses these issues from the point of view of highlighting issues which might not be expected in a traditional project.

3.1. Controls Systems Complexity

In a typical TriGeneration plant the various major components and sub-systems each have their own standard or typical controls system. All of these systems need to coordinated and integrated to operate seamlessly to provide safe and efficient operation. The typical sub-systems involved are: Generator Set on board controls Electrical System interlock and synchronising system Absorption chiller on board controls Chilled, condenser and heating water systems controls (usually the building BAS or DDC) Generator Set gas train controls In addition to these, some buildings may also have a separate Chiller management system The buildings fire alarm system may also be involved with alarm interfaces. The fundamental task facing the construction engineer is to coordinate these controls systems to achieve the seamless operation by addressing the following key issues: Critical functions for safety and system protection must be provided by independent and robust systems Each control system should provide only the core functions to which it is specifically designed to provide Transmission of signal between controls systems for critical functions should be by low level interfaces A selection of typical issues arising from these fundamentals are: The Electrical System interlock and synchronising system is typically PLC based. Its basic functions are to control circuit breakers involved in the synchronisation of the generator to the grid, to provide reverse power protection and external trip functions to protect the electrical grid distributors interests. Being a PLC based system, this system would be capable of controlling any other elements of the system. However in the interests of ensuring that this system is as robust as possible and not compromised by the addition of less critical functions, the other elements of the

Achieving the Green Dream Tomorrows technology today, Melbourne, 2010

systems should be provided with independent controls leaving the PLC system to carry out the critical functions alone. The Generator Set on board controls may have the ability to carry synchronising functions and perhaps reverse power protection. This control system would typically be a programmable electronic type, often a PLC type and will typically control many internal generator functions such as engine starting, speed/output frequency, power management, and engine safeties & fault alarms etc. Due to the complexity of these issues and the likelihood that they will need adjustment and even upgrading of their software during commissioning and throughout the life of the installation, the critical functions described above should be independently managed (by a PLC system as described above). The Absorption Chiller on board controls will probably have the ability to control the stop/start and speed control functions of the chilled and condenser water pumps. Whilst the chiller does need control of certain pump functions, or at least input of pump status, there are some external issues involved: The chillers control system will not inherently accommodate the engine cooling system need for the condenser water system pump(s) to run, and the chilled water pump operation needs to be controlled in concert with the main chilled water pump system to ensure that no undesirable effects are imposed onto the main chilled water system. These pumps should be controlled by the facility BMS/DDC system, with pump status signals output to the chiller control. The buildings BMS/DDC system will control all of the systems external to the TriGeneration plant. Given the generalist nature of these controls, the broad extent of the system and the likelihood that technicians who are not familiar with the TriGeneration plant will need to access the BMS/DDC system our general opinion is that BMS/DDC ought to be only used for less critical functions where direct interface needs exist. The generator sets gas train controls are a critical sub-system. Due to the variation in gas types and gas safety regulations around the world (noting that the gas engine generator sets are a global product range for all genset suppliers) the gas train controls are usually not fully integrated into the genset and an external control system is provided. If the building is provided with an independent chiller control system for the optimisation of the chiller system functions, this system should sequence the chillers according to the availability of the absorption chiller. The absorption chiller would normally be used to its maximum, to gain the best utilisation from the TriGeneration system overall. The BMS/DDC would monitor this system and control the associated systems (such as cooling towers, pumps etc).

3.2. Integration and Coordination

The integration and coordination of the controls systems described above is a key element of the successful completion and commissioning a TriGeneration system. Due to the complexity of the systems and the diversity of skills amongst the specialist suppliers of the respective systems an extensive amount of coordination is required. This coordination starts during the tender period, recommences in the procurement phase and continues well into the detail design / shop drawing preparation. The fundamental issue to be addressed is which system is to carry out each function. The principals discussed in 3.1 Control Systems Complexity are considered and applied to each function and every interface between systems. The creation of a concept diagram as shown in Figure 1 provides a basis for record of discussions, and documentation of decisions taken. This diagram is not a wiring schematic nor a logic diagram, but is simply a representation of the relationships between systems. The diagram shown covers the electrical systems, a second similar diagram is used to cover the mechanical systems For each of the systems involved, the interfaces with each other system need to be identified as the resolved in detail and the following issues addressed: signal format (dry contacts, pulsed signal, high level protocol Modbus, BACnet etc), interface location and interface responsibility (who carries out the wiring and who makes the connections).

Achieving the Green Dream Tomorrows technology today, Melbourne, 2010

Figure 1

Typical Controls Concept Overview (electrical)

3.3. Commissioning in a Live Facility

In an existing facility, and particularly in a live data centre, many of the various commissioning activities require starting, stopping or changing the operating conditions of essential systems. This carries inherent risks of adversely effecting the operation of those systems to the detriment of the data centre. One example of the many manifestations of this issue is the starting of the absorption chiller: To start the chiller, the chilled water pump must run. However until the chiller does start, the water delivered from the chiller will be at return water temperature and when mixed with chilled water supply from other operating chillers will raise the chilled water temperature supplied to the field, potentially compromising the ability of the air handling systems to provide adequate cooling to the facility. These risks require management by attention to detail, close liaison with the facility managers and rigorous compliance with work permit and change management procedures. The single biggest implication to the Contractor from this issue is the potential for delays. Work involving risks usually requires planning and pre-preparation by the Facility Manager and the Data Centre operators, with advance notice being a perennial interruption. The proper preparation for these activities involves a thorough consideration of the activities, a detailed program of moratorium periods (when risky works are not appropriate) and a realistic assessment of the amount of work which can be carried out in each approved works window. Ensuring that suppliers and sub-contractors are prepared for working in such an environment is a head contractors odious responsibility. Dry runs for many activities are appropriate: There is much to be discovered by sitting with the sub-contractors commissioning technicians and saying Show me what youre going to do. And how and with what!

Achieving the Green Dream Tomorrows technology today, Melbourne, 2010

Whilst it is perhaps a statement of the obvious, as much pre-commissioning as possible should be carried out. Off-site demonstrations of soft-ware should be carried out. These demonstrations should be presented by the technicians who will eventually carry out the work to avoid miscommunications interrupting the process when it reaches site and time becomes critical.

3.4. Electrical Load Testing and Commissioning

The control and safety interlock features of the electrical system need to be tested, commissioned and proven operational. This process is particularly rigorous in a high voltage (HV) system which is typical of a Data Centre application. At each stage of the process, commencing with proving of interlock communications signals and individual circuit breaker settings and progressing through to final synchronising checks, there is potential (in varying degrees) to have an impact on the Data Centre operations. It is therefore essential that the system be designed and installed with sufficient duplicated pathways and bypasses to allow thorough testing and commissioning with reasonable isolation from the operating systems. An important issue in this category is that of load testing: The generator set obviously needs to be tested under load during commissioning. What is perhaps not given sufficient consideration is the need to adjust and tune the generator sets controls during this testing. This will result in the need to start and stop the unit quite frequently, and to vary its load from maximum to minimum. This sort of activity is quite at odds with the risk management issues discussed in 3.3 above, and can only be carried out successfully with an independent load bank. The temporary installation of a load bank is in itself a task of some significance, particularly for a high voltage generator set: The temporary installation will need to include a HV/LV transformer to suit a typical LV load bank. The temporary power connections to the transformer and the load bank requires all of the protection considerations due to an HV installation. The location of the temporary load bank also requires some careful planning: This unit needs to be in an area where the full rated power of the generator set (in the case of the project, 2 MW) can be dissipated. The noise level generated by this equipment is also very high, so the location needs to consider the duration and time-of-day of the tests to avoid creating unacceptable noise conditions to site users and neighbours.

3.5. Vibration Isolation and Noise Abatement

A fundamental issue requiring recognition and appropriate attention in design and installation is that a TriGeneration system will operate continuously, 24/7/365! So whilst there might be a natural tendency for the building services engineer to subconsciously liken this installation to an emergency generator system, there are installation issues which might be tolerable in a generator system which operates two or four hours a month but which can become a problem in a system operating 24/7/365. Airborne noise is one such issue. A generator set of 2MW capacity will create sufficient noise to break out through ventilation ducts and the like and become a problem outside the building, and inside the building occupied areas. Structure borne noise, or vibration issues are also an issue to be addressed. The typical engine in a large TriGeneration system has a displacement capacity of about 90 litres, and at 18 cylinder configuration each cylinder displaces 5 litres. At these sort of piston sizes, in spite of the engines being carefully balanced in design and manufacture, the unavoidable truth is that large forces are at work. With even a small residual of these forces driving the external vibration of the generator set, there is a considerable amount of energy to be dissipated. The generator set manufacturers recommendations on anti-vibration mounts should be followed, and if the installation is in a sensitive area (perhaps in an occupied building) then specialist advisors should be consulted.

Achieving the Green Dream Tomorrows technology today, Melbourne, 2010

One specific issue of noteworthiness: All connections to the engine, whether they are piped services or electrical cables, should have ample accommodation for vibration. Pipework connections in particular should have anti-vibration connections of generous length a length of more than our times the diameter would a be sensible minimum to adopt as a rule of thumb. Pipework connections must be aligned with the crankshaft orientation (ie horizontal and lengthwise along the engine) to accommodate the vertical vibration in operation and the rotational torque reaction movement under starting/stopping and sudden load changes.

3.6. Size of Plant Installation Issues

To most building services engineers there is probably an inherent understanding that a generator set of 2 MW (or even only 1 MW) capacity is a large piece of equipment. And similarly a 1500 or 1900 kW chiller all engineers would naturally appreciate that this is a machine which takes some forethought to design into a plantroom and to transport and install. The traps for the unwary in a TriGeneration system are items like the engine exhaust diverter valve. This innocent looking 3 way motorised valve shown on the Concept Diagram in Figure 1, is a piece of hardware which is about 2.5 metres tall, 2 metres flange to flange and weighs 500 kg. Installed within the exhaust flue system, it requires its own support from the building structure and needs to be installed with the actuator shaft vertical all of which adds to the complexity of the detail design and installation works. Even the engine muffler of the generator set is unexpectedly bulky. At 4m long and 2m diameter this too requires careful consideration. The photo below illustrates the real size of the exhaust valve:

4. Acknowledgements
The Client and Project Partners are warmly acknowledged for their input into this paper and permission to use their project as a case study: National Australia Bank, Mr Gary OConnor, Manager Facilities & Workplace, Commercial Services United Group Limited, Mr Alan Sloane and Mr David Brooke, Facilities Managers

Achieving the Green Dream Tomorrows technology today, Melbourne, 2010