Generator Maintenance:
The Uptime Institute’s Annual Outages Analysis 2023 has been released and when it comes to backup power, the message is clear: “Generators are reliable, but require regular scheduled maintenance, fuel checks and testing.”
This advice comes after the Institute’s 2023 resilience survey revealed that just under a third of operators experienced a major outage at their site caused by a power problem in the last three years. This figure was similar to that in 2022, meaning little has improved in how data centres maintain their generators.
Paul Brickman, Commercial Director for Crestchic Loadbanks, comments:
“UPS failure is the single biggest cause of power outages for the data centre market, by some distance, and industry bodies like the Uptime Institute have been banging the drum for regular generator maintenance and testing for many years.
“The latest research shows that very little improvement is being made into the number of power failures happening, which means more action is required to reduce these avoidable and financially crippling outages.”
The cost of outages is increasing
Power outages are usually sudden and binary. They can be site-wide and have an immediate impact on service delivery.
According to the Uptime Institute, restoration of power can be quick, but in many instances, it can take hours to restart IT systems and fully synchronise databases. This loss of equipment can mean data centres operate outside SLAs until full replacements are installed.
A massive 40% of power outages are caused by UPS failure and, according to the 2023 analysis, they are now costing more than ever. Based on the Uptime Institute’s last global survey, a quarter of respondents said their most recent outage cost more than $1 million in direct and indirect financial losses.
A further 45% said their most recent outage cost between $100,000 and $1 million – a figure that is increasing.
Why is the cost of outages increasing?
The Uptime Institute cites a number of contributory factors, including inflation, fines, SLA breaches, the cost of labour, emergency call-out charges, and replacement parts. The biggest single reason, they say, is the growing dependency of corporate economic activity on digital services and on data centres. This is because the loss of critical IT services almost always instantly translates into disrupted business and lost revenue, which can include compensation, fines and lost business with costs running into the tens of millions of dollars.
In fact, the Uptime Institute professes to not calculating an average outage cost, because the business and outage impacts are so varied that each year a handful of outages will be so large, they distort the entire picture.
The price will continue to rise
The economy’s reliance on digital services will continue to increase, and with it a reliance on stronger SLAs could make outages even more financially crippling. Higher regulatory fines and increased compensation for customers that are affected by service disruption will further increase the impact.
Prevention is better than cure
With the cost of outages set to increase further, and evidence proving that not much headway is being made in the prevention of power supply-related outages, data centre operators should heed the Institute’s warning and uphold regular scheduled maintenance, fuel checks and testing.
Paul adds, “As the Standard bearer for Digital Infrastructure performance, the Uptime Institute has made it clear that backup generators must be well maintained and fit for purpose. To achieve this, data centres should use load banks according to some simple best practice guidelines to ensure that should an outage occur, the power remains on.”
Best practice tips for scheduled load bank testing
Test annually when using resistive-reactive load bank testing
All generators should be tested at least annually for real-world emergency conditions using a resistive-reactive 0.8pf loadbank. All gensets (where there are multiple) should be run in a synchronised state, for 8 hours. Where this isn’t possible, they should be run for a minimum of 3 hours.
Test 2-4 times per year when using resistive-only load bank testing
Where a resistive-only load bank is used, testing should be increased to 2-4 times per year at 3 hours per test. In carrying out this testing and maintenance, fuel, exhaust and cooling systems and alternator insulation resistance are effectively tested, and system issues can be uncovered in a safe, controlled manner without the cost of major failure or unplanned downtime.
Why is resistive-reactive the best approach to generator maintenance?
Capable of testing both resistive and reactive loads, this type of load bank provides a much clearer picture of how well an entire system will withstand changes in load pattern while experiencing the level of power that would typically be encountered under real operational conditions.
Furthermore, the inductive loads used in resistive/reactive testing will show how a system will cope with a voltage drop in its regulator. This is particularly important in a university where generators will run in parallel should blackouts occur, where a problem with one generator could prevent other system generators from working properly or even failing to operate entirely. This is something which is simply not achievable with resistive-only testing.
Begin scheduled generator maintenance today
The Uptime Institute advises regular, scheduled maintenance of testing for all generators. For support implementing a schedule, or to find out more about the right load banks for your specific requirements, contact the Crestchic team today.
