Skip to main content

Advertisement

Log in

Electrical Grid and Supercomputing Centers: An Investigative Analysis of Emerging Opportunities and Challenges

Informatik-Spektrum Aims and scope

Abstract

Some of the largest supercomputing centers (SCs) in the United States are developing new relationships with their electricity service providers (ESPs). These relationships, similar to other commercial and industrial partnerships, are driven by a mutual interest to reduce energy costs and improve electrical grid reliability. While SCs are concerned about the quality, cost, environmental impact, and availability of electricity, ESPs are concerned about electrical grid reliability, particularly in terms of energy consumption, peak power demands, and power fluctuations. The power demand for SCs can be 20 MW or more – the theoretical peak power requirements are greater than 45 MW – and recurring intra-hour variability can exceed 8 MW. As a result of this, ESPs may request large SCs to engage in demand response and grid integration.

This paper evaluates today’s relationships, potential partnerships, and possible integration between SCs and their ESPs. The paper uses feedback from a questionnaire submitted to supercomputing centers on the Top100 List in the United States to describe opportunities for overcoming the challenges of HPC-grid integration.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Top500 Supercomputer Sites, http://www.top500.org/lists/2013/11.

  2. Aikema D, Simmonds R (2011) Electrical Cost Savings and Clean Energy Usage Potential for HPC Workloads. In: 2011 IEEE International Symposium on Sustainable Systems and Technology (ISSST), pp 1–6

  3. Aikema D, Simmonds R, Zareipour H (2012) Data Centres in the Ancillary Services Market. In: 2012 International Green Computing Conference (IGCC), IEEE, pp 1–10

  4. ASHRAE (2012) Thermal Guidelines for Data Processing Environments, Special Publication, third edition. American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc., Atlanta, GA

  5. Cameron KW, Feng X, Ge R (2005) Performance-Constrained Distributed DVS Scheduling for Scientific Applications on Power-Aware Clusters. In: Supercomputing, Seattle, Washington

  6. Chen H, Caramanis MC, Coskun AK (2014) The Data Center as a Grid Load Stabilizer. In: 19th Asia and South Pacific Design Automation Conference (ASP-DAC), IEEE, pp 105–112

  7. Chiu D, Stewart C, McManus B (2012) Electric Grid Balancing Through Lowcost Workload Migration. SIGMETRICS Perform Eval Rev 40(3):48–52

    Article  Google Scholar 

  8. Curtis-Maury M, Jevic FB, Antonopoulos C, Nikolopoulos DS (2008) Prediction-Based Power-Performance Adaptation of Multithreaded Scientific Codes. IEEE Trans Parallel Distrib Syst 19(10):1396–1410

    Article  Google Scholar 

  9. Curtis-Maury M, Dzierwa J, Antonopoulos CD, Nikolopoulos DS (2006) Online Power-Performance Adaptation of Multithreaded Programs Using Hardware Event-Based Prediction. In: International Conference on Supercomputing, New York, NY, USA, ACM

  10. Curtis-Maury M, Shah A, Jevic FB, Nikolopoulos DS, de Supinski BR, Schulz M (2008) Prediction Models for Multi-Dimensional Power-Performance Optimization on Many Cores. In: International Conference on Parallel Architectures and Compilation techniques, New York, NY, USA, ACM

  11. David H, Gorbatov E, Hanebutte UR, Khanna R, Le C (2010) RAPL: Memory Power Estimation and Capping. In: Proceedings of the 16th ACM/IEEE International Symposium on Low Power Electronics and Design, ISLPED ’10, New York, NY, USA, ACM, pp 189–194

  12. Etinski M, Corbalan J, Labarta J, Valero M (2012) Parallel Job Scheduling for Power Constrained HPC Systems. Parallel Comput 38(12):615–630

    Article  MathSciNet  Google Scholar 

  13. Etinski M, Corbalan J, Labarta J, Valero M (2010) Optimizing Job Performance Under a Given Power Constraint in HPC Centers. In: Green Computing Conference, pp 257–267

  14. Etinski M, Corbalan J, Labarta J, Valero M (2010) Utilization Driven Power-Aware Parallel Job Scheduling. Comput Sci R&D 25(3-4):207–216

    Google Scholar 

  15. Etinski M, Corbalan J, Labarta J, Valero M (2011) Linear Programming Based Parallel Job Scheduling for Power Constrained Systems. In: International Conference on High Performance Computing and Simulation, pp 72–80

  16. Fan X, Weber W-D, Barroso LA (2007) Power Provisioning for a Warehouse-Sized Computer. In: The 34th ACM International Symposium on Computer Architecture

  17. Feitelson DG, Schwiegelshohn U, Rudolph L (2004) Parallel Job Scheduling – a Status Report. In: Lecture Notes in Computer Science, Springer-Verlag, pp 1–16

  18. Femal ME, Freeh VW (2005) Safe Overprovisioning: Using Power Limits to Increase Aggregate Throughput. In: International Conference on Power-Aware Computer Systems

  19. Foster I (2001) The Anatomy of the Grid: Enabling Scalable Virtual Organizations. In: Proceedings of the First IEEE/ACM International Symposium on Cluster Computing and the Grid, pp 6–7

  20. Foster I, Kesselman C, Lee C, Lindell B, Nahrstedt K, and Roy A (1999) A Distributed Resource Management Architecture That Supports Advance Reservations and Co-Allocation. In: Seventh International Workshop on Quality of Service, IWQoS ’99, pp 27–36

  21. Ganti V, Ghatikar G (2012) Smart Grid as a Driver for Energy-Intensive Industries: A Data Center Case Study. In: Grid-Interop 2012

  22. Ge R, Feng X, Feng W, Cameron KW (2007) CPU Miser: A Performance-Directed, Run-Time System for Power-Aware Clusters. In: International Conference on Parallel Processing, Xi’An, China

  23. Ghamkhari M, Mohsenian-Rad H (2012) Data Centers to Offer Ancillary Services. In: IEEE Third International Conference on Smart Grid Communications (SmartGridComm), pp 436–441, IEEE

  24. Ghatikar G, Ganti V, Matson N, Piette MA (2012) Demand Response Opportunities and Enabling Technologies for Data Centers: Findings From Field Studies. In: PG&E/SDG&E/CEC/LBNL

  25. Ghatikar G, Riess D, Piette MA (2014) Analyis of Open Automated Demand Response Deployments in California and Guidelines to Transition to Industry Standards. Technical Report LBNL-6560E, Lawrence Berkeley National Laboratory, 1 Cyclotron Rd, Berkeley, CA 94720

  26. Hsu C-H, Feng W-C (2005) A Power-Aware Run-Time System for High-Performance Computing. In: Supercomputing

  27. Intel (2011) Intel-64 and IA-32 Architectures Software Developer’s Manual, Volumes 3A and 3B: System Programming Guide

  28. Kappiah N, Freeh VW, Lowenthal DK, Pan F (2005) Exploiting Slack Time in Power-Aware, High-Performance Programs. In: Supercomputing

  29. Kaushik RT, Nahrstedt K (2012) T*: A Data-Centric Cooling Energy Costs Reduction Approach for Big Data Analytics Cloud, SC’12, Los Alamitos, CA, USA, IEEE Computer Society Press, Article no. 52

  30. Li J, Martìnez JF (2006) Dynamic Power-Performance Adaptation of Parallel Computation on Chip Multiprocessors. In: 12th International Symposium on High-Performance Computer Architecture, Austin, Texas

  31. Lifka DA (1995) The ANL/IBM SP Scheduling System. In: Job Scheduling Strategies for Parallel Processing, Springer-Verlag, pp 295–303

  32. Liu Z, Chen Y, Bash C, Wierman A, Gmach D, Wang Z, Marwah M, Hyser C (2012) Renewable and Cooling Aware Workload Management for Sustainable Data Centers. ACM SIGMETRICS Perform Eval Rev 40:175–186

    Article  Google Scholar 

  33. Mu’alem AW, Feitelson DG (2001) Utilization, Predictability, Workloads, and User Runtime Estimates in Scheduling the IBM SP2 with Backfilling. IEEE Trans Parallel Distrib Syst 12(6):329–543

    Google Scholar 

  34. Niccolai J (2013) New Data Center Survey Shows Mediocre Results for Energy Efficiency. IT World IDG News Service, www.itworld.com/article/2709163/hardware/new-data-center-survey-shows-mediocre-results-for-energy-efficiency.html

  35. Palensky P, Dietrich D (2011) Demand Side Management: Demand Response, Intelligent Energy Systems, and Smart Loads. IEEE T Ind Inform 7(3):381–388

    Article  Google Scholar 

  36. Patki T, Lowenthal DK, Rountree B, Schulz M, de Supinski BR (2013) Exploring Hardware Overprovisioning in Power-Constrained, High Performance Computing. In: International Conference on Supercomputing, pp 173–182

  37. Porterfield AK, Olivier SL, Bhalachandra S, and Prins JF (2013) Power Measurement and Concurrency Throttling for Energy Reduction in OpenMP Programs. In: 27th International Parallel and Distributed Processing Symposium Workshops & PhD Forum (IPDPSW), IEEE, pp 884–891

  38. Rountree B, Lowenthal D, de Supinski BR, Schulz M, Freeh V, Bletch T (2009) Adagio: Making DVS Practical for Complex HPC Applications. In: International Conference on Supercomputing

  39. Rountree B, Ahn DH, de Supinski BR, Lowenthal DK, Schulz M (2012) Beyond DVFS: A First Look at Performance Under a Hardware-Enforced Power Bound. In: IPDPS Workshops, IEEE Computer Society, pp 947–953

  40. Rountree B, Lowenthal DK, Funk S, Freeh VW, de Supinski B, Schulz M (2007) Bounding Energy Consumption in Large-Scale MPI Programs. In: Supercomputing

  41. Sarood O, Kalé LV (2011) A “Cool” Load Balancer for Parallel Applications. In: Proceedings of the 2011 ACM/IEEE Conference on Supercomputing, Seattle, WA

  42. Springer R, Lowenthal DK, Rountree B, Freeh VW (2006) Minimizing Execution Time in MPI Programs on an Energy-Constrained, Power-Scalable Cluster. In: ACM Symposium on Principles and Practice of Parallel Programming

  43. Torriti J, Hassan MG, Leach M (2010) Demand Response Experience in Europe: Policies, Programmes and Implementation. Energy 35(4):1575–1583

    Article  Google Scholar 

  44. Yang X, Zhou Z, Wallace S, Lan Z, Tang W, Coghlan S, Papka ME (2013) Integrating Dynamic Pricing of Electricity Into Energy Aware Scheduling for HPC Systems. In: Proceedings of SC13: International Conference for High Performance Computing, Networking, Storage and Analysis, SC ’13, New York, NY, USA, ACM

  45. Zhou Z, Lan Z, Tang W, Desai N (2013) Reducing Energy Costs for IBM Blue Gene/P via Power-Aware Job Scheduling. IPDPS Workshop, IEEE Computer Society

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Natalie Bates.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bates, N., Ghatikar, G., Abdulla, G. et al. Electrical Grid and Supercomputing Centers: An Investigative Analysis of Emerging Opportunities and Challenges. Informatik Spektrum 38, 111–127 (2015). https://doi.org/10.1007/s00287-014-0850-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00287-014-0850-0

Keywords

Navigation