First, let me define SLA as it relates to this discussion. SLA is service level agreement and for the facility minded individual is how we guarantee conditions at the server. We guarantee uptime and with properly corded devices we guarantee customers will not lose power. We also have temperature and humidity SLAs. They might be do not exceed 60-89 degrees f and 20-80% relative humidity for more than 4 hrs, for example. Now we have a target band tighter than that but that is our promise.
With that understood, we can start the discussion. Let’s say we have 2 MW busses with 4 primary and 1 redundant bus using static transfer switches to transfer to the reserve (redundant) bus. Let’s say that each bus has 1400 kW of UPS power, requires 100 kW for house loads, 100 kW for UPS losses, 40 KW for lighting, and 760 kW reserved for design day cooling. As we can see we will not be able to use about 400 kW of installed UPS capacity.
This leaves us a challenge: If we’re spending about $15mln per MW of IT load, we see that infrastructure is an expensive component. If we can find a way to utilize the rest of that UPS, across the 4 primary busses, being 1600kW of load is worth about $24mln in found infrastructure.
Let’s walk through a few scenarios to see how we can find that capacity. We could first replace the existing ups with high efficiency ups, bringing down the ups losses per bus to be redistributed as IT load. If we use 97% efficient at full load upss the loss will drop to about 35 kW and allow a redistributing of about 65 kW of ups power. At $15,000 per kW, that would be 260 kW of capacity over 4 primary busses worth $3.9 mln. This project might cost around $2mln, so that’s a good step. It also is saving some energy, call it $500,000 a year if we get the IT load high, and let’s say we got a $500,000 utility rebate. But there are better opportunities.
Let’s focus on the 100 kW house load (plug load). If we automate its breaker so that when the bus exceeds 1900 kW, house loads go dark, we can add 100 kW. Easy peasy right? And when are we ever really gonna get there? Because honestly we can only get there near the design day with an entire bus failure and IT load right at its peak. As a colo, it’s hard to be near the max, and a paperwork change to raise capacity helps us out a lot. Considering the real scenarios that can lead us to bus maximization are important as well. So really, those plug loads will probably never drop out, but we need a plan in place to use more of that bus.
I promise I’ll get to SLAs soon, but so far we’ve found 165 kW on each bus for a modest spend with energy savings. Not too shabby.
Let’s look at that HVAC load since it’s pretty large. We know those things struggle on the design day when we ask all those compressors to start working overtime. If the design day is 95 degrees f, we can evaluate an evaporative assist system on the condensing units. Let’s say design wet bulb is 79 degrees f and the evaporative assist can get within 4 degrees of wet bulb. We just made the design day 83 degrees. On that design day, the kW draw of the cooling equipment is 180kW lower per bus in my hypothetical not at all closely based on reality example. Now when we redistribute, we do have to be aware that as we take hvac load and add it to it load, that additional it load creates an additional cooling demand. So for every redistributed HVAC kW, let’s only add .66 kW to IT to include the additional cooling demand. That system will add another 120 kW to each bus 480 kW total, for let’s say $1mln plus energy savings of $200,000 a year if we achieve moderate loads. Not too bad.
Now, let’s talk SLAs. The designers of your building envisioned a 72 degree mixed environment and designed for a 95* dry bulb outside. With containment, return temperatures can approach 90 degrees. Now, it’s a good idea to make sure the equipment’s refrigerant can handle higher temps and pressures. Ask your vendor about the impact of increased return temperatures. Typically, however, you won’t spend many hours operating at this increased return temperature, so if its just a few hours per year the impact on compressors shouldn’t be very significant. If they were running 8700 hours a year with a 95* return, well that might be a problem.
As the return temp increases we see a 2-5% increase in capacity and efficiency for each degree Fahrenheit as a vague thumb rule. With a 72 degree mixed air we were delivering 55 degree supply air. Now we can deliver 72 degree air, get almost 90 back to the cooling unit, reduce the demand on the design day further, and reduce energy use year round. But our SLA, from above, was 89 degrees. We can ramp the fans up, reduce the delta, and scratch some more kW. But how?
This is where we get into the fun and interesting stuff. If you use your BMS to stage cooling, it could reduce compressor stages on units if the bus approached 2MW. Gathering data from your HVAC vendor, you determine that if you limit the RTUs to 4 of 6 stages, your design day kW drops another 60 KW per bus. Gather data from the manufacturer, and see if at the dc load you have the cooling unit will deliver air below your SLA temperature. The temperature will likely even be in the recommended window, below 81 degrees. By allowing the BMS to balance the load to a degree, you will be able to comfortably use more of those busses. Instead of filling 4 busses near 1300 kW, you’re trying to bring one load to 5200 kW (or whatever we get to in the end for total bus capacity). It’s much easier to manage. Anyway, it also gives you so e piece of mind. You can sacrifice the temperature even more if necessary, and still protect the generator alternator from overload and burning up. And again, all the concerns only apply on the design day, with a bus failure, and IT load at the redline. So it’s also important to evaluate the statistical probability of an event like this.
Let’s go even further because why not. If the SLA is 89 degrees and the equipment has economizers with direct evaporative humidifiers, the unit can supply around 89 degree air without even cooling (looking at that design day wet bulb). Add two compressors for trim, and you’re delivering air at 77 degrees and using a lot less energy. I see us using that whole 1400 kW.
Let’s say you don’t have the direct evaporative humidifier. Instead, you’re looking at the dry bulb. If the supply air was 82 degrees , the return would be about 100. The design day being 95, and mcwb (mean coincident wet bulb) being much lower at 72 (telling us the humidity issue isn’t bad) we can get 5 degrees of cooling from the economizer. The compressors can deliver the other 15 with 3 compressors per unit using much less power.
You can argue with the numbers here and there but the principle remains. Using a these techniques you can get a lot more out of your existing infrastructure and likely never see a difference in operation (still be in the recommended band) but in the rare event on the design day with a complete bus failure, things will still be “ok” and you will still keep your customer promise(SLA).
So let’s summarize. In my scenario, we spent 200 k on controls, 1 mln on evaporative assist equipment, 200 k on breakers for the house load, and 2 mln on ups for 3.4 mln spend in total. We’re saving 700 k a year on energy and we found 1600 kW of capacity that would have cost 24 mln to deliver.
The only challenge left after that, is figuring out how to get all that capacity in your space. Most distribution infrastructure has a lot more capacity than it is allocated so that isn’t a huge issue in my experience, but finding the raised floor to deploy it is the challenge. Also, convincing your capacity managers and sales team that your 150 watt per square foot space is now 210 can be a culture shock and hard to make happen. I refer to those as agency costs and they can be a huge drain on value.
Posted from my iPad.
2 thoughts on “Maximizing Utilization by Revisiting SLAs”
Great article on releasing the elephant in our data center. New IT technology refresh also enables more power pack into the same footprint. The next step is to densify in order to utilize all the stranded capacity released. Some incentive scheme must be in place to encourage densification. Stakeholders also need to be educated on the benefits of densifying, especially what are the benefits to them. As long as people don’t see the benefits to them, current practices often drive operators to concentrate on short term profit (benefits to them) at the expense of long term benefits.
LikeLiked by 1 person
Reblogged this on My Blog.