Challenges in Cooling Design of CPU Packages for High-Performance Servers

Abstract

Jie Wei
Corporate Product Technology Unit, Fujitsu Limited , Kawasaki , Japan

Cooling technologies that address high-density and asymmetric heat dissipation in CPU packages of high-performance servers are discussed. Thermal management schemes and the development of associated technologies are reviewed from a viewpoint of industrial application. Particular attention is directed to heat conduction in the package and heat removal from the package/heat sink module. Power dissipation and package cooling characteristics of high-performance microprocessors are analyzed. The development of a new metallic thermal interface technology is introduced, where thermal and mechanical performance of an indium-silver alloy in the chip/heat spreader assembly was studied. The paper also reports on research on other thermal management materials, such as diamond composite heat-spreading materials. Some actual package designs are described to illustrate the enhanced heat spreading capability of heat pipes and vapor chambers.

Korea

고성능 서버의 CPU 패키지에서 고밀도 및 비대칭 열 방출을 처리하는 냉각 기술이 논의된다. 열 관리 체계와 관련 기술의 개발을 산업 적용 관점에서 검토한다. 패키지의 열 전도 및 패키지/열 싱크 모듈에서 열 제거에 특히 주의하십시오. 고성능 마이크로프로세서의 전력 소산 및 패키지 냉각 특성을 분석한다. 칩/열 확산기 조립체에서 인듐-실버 합금의 열 및 기계적 성능을 연구한 새로운 금속 열 인터페이스 기술의 개발이 도입되었다. 이 논문은 또한 다이아몬드 복합 열 확산 재료와 같은 다른 열 관리 재료에 대한 연구도 보고한다. 일부 실제 패키지 설계는 열 파이프와 증기 챔버의 향상된 열 확산 능력을 설명하기 위해 설명된다.

Figure 1 A typical structure of CPU package and heat sink module.
Figure 1 A typical structure of CPU package and heat sink module.
Figure 3 Liquid-cooled MCM package of Fujitsu GS8900 (Turbo).
Figure 3 Liquid-cooled MCM package of Fujitsu GS8900 (Turbo).
Figure 5 Temperature distribution on a test processor chip.
Figure 5 Temperature distribution on a test processor chip.
Figure 10 Impact of voids in TIM-1 on temperature distribution on the chip.
Figure 10 Impact of voids in TIM-1 on temperature distribution on the chip.
Figure 15 Comparison of various heat sinks in terms of their cooling performance and weights.
Figure 15 Comparison of various heat sinks in terms of their cooling performance and weights.

CONCLUSIONS

Challenges in thermal management of CPU packages for high-performance servers are discussed. Described in this paper are the characteristics of power dissipation in the recent generation of CPU processors, the investigations conducted to incorporate advanced thermal interface and heat-spreading materials in the package design, and the evaluations of enhanced cooling capability of air-cooled heat sinks. With further miniaturization and increasing complexity of devices and components in high-performance processors, challenges will continue to mount in the thermal design of high-performance servers. The industry is facing the critical needs for extending the limits of conventional technologies that are attractive from a cost perspective, while attempting aggressively to make advanced cooling solutions viable options.

Korea

고성능 서버용 CPU 패키지의 열 관리 문제에 대해 논의한다. 이 논문에서 기술된 것은 최근 세대의 CPU 프로세서에서 발생하는 전력 소산의 특성, 패키지 설계에 첨단 열 인터페이스와 열 확산 재료를 통합하기 위해 수행된 조사, 그리고 공랭식 열제거원의 강화된 냉각 기능에 대한 평가 등이다. 고성능 프로세서에서 소자와 구성품의 소형화와 복잡성이 증가함에 따라, 고성능 서버의 열 설계에 대한 도전은 계속 증가할 것이다. 업계는 비용 측면에서 매력적인 기존 기술의 한계를 확장하는 동시에 첨단 냉각 솔루션을 실행 가능한 옵션으로 만들기 위해 적극적인 노력을 기울여야 하는 중요한 요구에 직면해 있다.

REFERENCES

  • Wei, J. Challenges in Package-Level High Power Density Cooling. Proc. International Symposium on Transport Phenomena. Toyama, Japan.  
  • Sauciuc, I., Prasher, R., Chang, J. Y., Erturk, H., Chrysler, G., Chiu, C. P. and Mahajan, R. Thermal Performance and Key Challenges for Future CPU Cooling Technologies. Proc. InterPACK1905. San Francisco, California, USA.  
  • Prasher, R. S., Chang, J. Y., Sauciuc, I., Narasimhan, S., Chau, D., Chrysler, G., Myers, A., Prstic, S. and Hu, C. 2005. Nano Micro Technology-Based Next-Generation Package-Level Cooling Solutions. Intel Technology Journal, vol. 9(4): 285–296.  , 
  • Wei, J. 2007. Thermal Management of Fujitsu High-Performance Servers. Fujitsu Scientific & Technical Journal, vol. 43(1): 122–129.  , 
  • Xu, G., Follmer, L. and Cooley, J. Thermal Solution Development for the SunFireTM E25K server. Proc. SEMI-THERM. San Jose, California, USA.  
  • Tanaka, S., Hamaguchi, H., Tsuzuki, H., Takahashi, I., Natori, M. and Nagata, T. 2005. Packaging Technology for SX-8. NEC Tech.,, vol. 58(4): 23–28.  [Google Scholar]
  • Minichiello, A. and Belady, C. Thermal Design Methodology for Electronic Systems. Proc. ITherm. San Jose, California, USA. pp.696–704.  [Google Scholar]
  • Giraldo, M. D. Mechanical Packaging of HP's Superdome Server. Proc. InterPACK1901. Kauai, Hawaii, USA.  [Google Scholar]
  • Knickerbocker, J. U., Pompeo, F. L., Tai, A. F., Thomas, D. L., Weekly, R. D., Nealon, M. G., Hamel, H. C., Haridass, A., Humenik, J. N., Shelleman, R. A., Reddy, S. N., Prettyman, K. M., Fasano, B. V., Ray, S. K., Lombardi, T. E., Marston, K. C., Coico, P. A., Brofman, P. J., Goldmann, L. S., Edwards, D. L., Zitz, J. A., Iruvanti, S., Shinde, S. L. and Longworth, H. P. 2002. An Advanced Multi-chip Module (MCM) for High-Performance UNIX Servers. IBM J. Res. & Dev.,, vol. 46(6): 779–804.  [Crossref], [Web of Science ®], [Google Scholar]
  • Coico, P. A., Messina, G., Ostrander, S., Zitz, J. and Zou, W. Internal Thermal Management of IBM P-Server Large Format Multi-Chip Modules Utilizing Small Gap Technology. Proc. InterPACK1905. San Francisco, California, USA.  [Google Scholar]
  • Harrer, H., Dreps, D. M., Winkel, T. M., Scholz, W., Truong, B. G., Huber, A., Zhou, T., Christian, K. L. and Goth, G. F. 2007. High-Speed Interconnect and Packaging Design of the IBM System z9 Processor Cage. IBM J. Res. & Dev., vol. 51(1/2): 1–16.  [Web of Science ®], [Google Scholar]
  • Kobayashi, F. 2000. Hardware Technology for the HITACHI MP5800 Series (HDS Skyline Series). IEEE Trans. Advanced Packaging, vol. 23(3): 504–514.  [Crossref], [Google Scholar]
  • Yamada, O., Sawada, Y., Harada, M., Yokozuka, T., Yasukawa, A., Moriya, H., Saito, N., Kasai, K., Uda, T., Netsu, T. and Koyano, K. Improvement of the Reliability of the C4 for Ultra-High Thermal Conduction Module with the Direct Solder-Attached Cooling System (DiSAC). Proc. ECTC. Orlando, Florida, USA. pp.1144–1148.  [Google Scholar]
  • Fujisaki, A., Suzuki, M. and Yamamoto, H. 2001. Packaging Technology for High Performance CMOS Server Fujitsu GS8900. IEEE Trans. Advanced Packaging,, vol. 24(4): 464–468.  [Crossref], [Google Scholar]
  • Wei, J., Suzuki, M., Udagawa, Y. and Yamamoto, H. Thermal Management of Multiple MCMs with Low-Temperature Liquid Cooling. Proc. InterPACK1901. Kauai, Hawaii, USA.  [Google Scholar]
  • Pautsch, G. W. An Overview on the System Packaging of the Cray SV2 Supercomputer. Proc. InterPACK1901. Kauai, Hawaii, USA.  [Google Scholar]
  • Pautsch, A. G. and Shedd, T. A. 2005. Spray Impingement Cooling Using Single- and Multiple-Nozzle Arrays, Part I: Heat Transfer Using FC-72. Int. J. Heat and Mass Transfer, vol. 48: 3167–3175.  [Crossref], [Web of Science ®], [Google Scholar]
  • International Technology Roadmap for Semiconductors http://www.itrs.net/Links/2005ITRS/AP2005.pdfAssembly and Packaging. 2005 ed. Available at [Google Scholar]
  • Sery, G., Borkar, S. and De, V. Life Is CMOS: Why Chase the Life After?. Proc. Design Automation Conf. New Orleans, Louisiana, USA. pp.78–83.  [Google Scholar]
  • Deeney, J. Thermal Modeling and Measurement of Large High Power Silicon Devices with Asymmetric Power Distribution. Proc. IMAPS. Denver, Colorado, USA.  [Google Scholar]
  • Warnock, J. D., Keaty, J. M., Petrovick, J., Clabes, J. G., Kircher, C. J., Krauter, B. L., Restle, P. J., Zoric, B. A. and Anderson, C. J. 2002. The Circuit and Physical Design of the Power4 Microprocessor. IBM J. Res. & Dev., vol. 46(1): 27–51.  [Crossref], [Web of Science ®], [Google Scholar]
  • Xu, G. Thermal Modeling of Multi-Core Processors. Proc. ITherm. San Diego, California, USA. pp.96–100.  [Google Scholar]
  • Iyengar, M. and Schmidt, R. Analytical Modeling for Prediction of Hot spot Chip Junction Temperature for Electronics Cooling Applications. Proc. ITherm. San Diego, California, USA. pp.87–95.  [Google Scholar]
  • June, M. S. and Sikka, K. K. Using Cap-Integral Standoffs to Reduce Chip Hot spot Temperatures in Electronic Packages. Proc. ITherm. San Diego, California. pp.173–178.  [Google Scholar]
  • Wei, J., Nori, H., Ishimine, J. and Fujisaki, A. Effects of Asymmetric Power Distributions on Thermal Management of High Performance LSI Processors. Proc. IFHT. Kyoto, Japan. pp.37–39.  [Google Scholar]
  • Mukhopadhyay, S., Raychowdhury, A. and Roy, K. Accurate Estimation of Total Leakage Current in Sealed CMOS Logic Circuits Based on Compact Current Modeling. Proc. Design Automation Conf. Anaheim, California, USA. pp.169–174.  [Google Scholar]
  • Krishnan, S., Garimella, S. V., Chrysler, G. M. and Mahajan, R. V. Towards a Thermal Moore's Law. Proc. InterPACK1905. San Francisco, California, USA.  [Google Scholar]
  • Kim, N. S., Austin, T., Blaauw, D., Mudge, T., Flaunter, K., Hu, J. S., Irwin, M. J., Kandemir, M. and Narayanan, V. 2003. Leakage Current: Moore's Law Meets Static Power. IEEE Computer,, vol. 36(12): 68–75.  [Crossref], [Google Scholar]
  • Inoue, A. High Performance and High Reliability Technologies of the SPARC64 V/VI. Scientific System Research Symposium 2006. Tokyo, Japan. (in Japanese) [Google Scholar]
  • Dani, A., James, C., Matayabas, J. R. and Koning, P. Thermal Interface Material Technology Advancements and Challenges—An Overview. Proc. InterPACK1905. San Francisco, California, USA.  [Google Scholar]
  • Stern, M. B., Gektin, V., Pecavar, S., Kearns, D. and Chen, T. Evaluation of High Performance Thermal Greases for CPU Package Cooling Applications. Proc. SEMI-THERM. San Jose, California, USA.  [Google Scholar]
  • Wilson, J. 2006. Thermal Conductivity of Solders. Electronics Cooling, vol. 12(3): 4–5.  [Google Scholar]
  • Koide, M., Fukuzono, K., Yoshimura, H. and Sato, T. High-Performance Flip-Chip BGA Technology Based on Thin-Core and Coreless Package Substrates. Proc. ECTC. San Diego, California, USA. pp.1869–1873.  [Google Scholar]
  • Standard Test Method for Thermal Transmission Properties of Thin Thermally Conductive Solid Electrical Insulation Materials http://www.astm.org/ASTM D5470-93. Available at [Google Scholar]
  • Gektin, V. Thermal Management of Voids and Delamination in TIMs. Proc. InterPACK1905. San Francisco, California, USA.  [Google Scholar]
  • Wilson, J. 2007. Thermal Conductivity of Common Alloys in Electronics Packaging. Electronics Cooling, vol. 13(1): 6–7.  [Google Scholar]
  • Zweben, C. 2005. Revolutionary New Thermal Management Materials. Electronics Cooling, vol. 11(2): 36–37.  [Google Scholar]
  • Bollina, R., Landgraf, J., Wagner, H., Wilhelm, R., Knippscheer, S. and Tabernig, B. Performance, Production, and Applications of Advanced Metal Diamond Composite Heat Spreader. Proc. IMAPS. San Diego, California, USA.  [Google Scholar]
  • Rowcliffe, D. Cemented Diamond Composites for Thermal Management Applications. Proc. IMAPS ATW. Denver, Colorado, USA.  [Google Scholar]
  • Wei, J., Chan, A. and Copeland, D. Measurement of Vapor Chamber Performance. Proc. SEMI-THERM. San Jose, California, USA.  [Google Scholar]
  • Wei, X. and Sikka, K. Modeling of Vapor Chamber as Heat Spreading Devices. Proc. ITherm. San Diego, California, USA. pp.578–585.  [Google Scholar]
  • Nakayama, W. 2006. Exploring the Limits of Air Cooling. Electronics Cooling,, vol. 12(3): 10–17.  [Google Scholar]
  • Ortega, A. The Evolution of Air Cooling in Electronic Systems and Observations about Its Limits. Proc. International Symposium on Transport Phenomena. Toyama, Japan.  [Google Scholar]
  • Sauciuc, I., Chrysler, G., Mahajan, R. and Szleper, M. Air-Cooling Extension: Performance Limits for Processor Cooling Applications. Proc. SEMI-THERM. San Jose, California, USA.  [Google Scholar]
  • Xu, G., Guenin, B. and Vogel, M. Extension of Air Cooling for High Power Processors. Proc. ITherm. Las Vegas, Nevada, USA. pp.186–193.  [Google Scholar]
  • Sauciuc, I., Chrysler, G., Mahajan, R. and Prasher, R. 2002. “Spreading in the Heatsink Base: Phase Change Systems or Solid Metals”. In Thermal Challenges in Next Generation Electronic Systems, Edited by: Joshi, Y. K. and Garimella, S. V. Rotterdam: Millpress.  [Google Scholar]
Full Article View