Demonstration Testing of Liquid Cooling System Achieves 94% Reduction in Energy Consumption to Cool Servers in Data Centers

KDDI Corporation
Mitsubishi Heavy Industries, Ltd.
NEC Networks & System Integration Corporation

Tokyo, March 6, 2023 - On February 28, KDDI Corporation (KDDI), Mitsubishi Heavy Industries, Ltd. (MHI), and NEC Networks & System Integration Corporation (NESIC) conducted demonstration testing of an immersion cooling system for potential applications of liquid cooling of IT components in a hyper-scale data center. The test protocol, targeting sustainable data centers that will contribute to decarbonization, successfully achieved Tier 4 [1] stable operation in a liquid-cooled unit. Compared to conventionally cooled data centers, immersion cooling reduced energy usage for cooling of the servers by 94% [2] and achieved a PUE, a metric indicating the energy efficiency of a data center, of 1.05 [3]. The demonstration testing was conducted at KDDI's Oyama Network Center (NC).

KDDI, MHI and NESIC will commence provision of immersion cooling data centers within FY2023, anticipating a wide variety of applications from hyper-scale data centers to containerized data centers [4].

• 

  <Immersion Cooling Data Center>

• 

  <Indoors: Immersion Cooling Unit>

• 

  <Outdoors: Free Cooling Unit>

■Overview of the demonstration test

1. Background and challenges

• Today, demand for cloud services and data centers is expanding rapidly as a result of global advances in digital transformation (DX), changes in workstyles amid the Covid-19 pandemic, and business continuity planning (BCP) in preparation for natural disasters.
• As IT components of data centers have become increasingly advanced in performance and density, they generate unprecedented volumes of heat, and dealing with this waste heat presents a challenge. Two factors are necessary in order to achieve sustainable data centers: reduced energy consumption and excellent waste heat efficiency. Calls are therefore being heard for data centers that consume less energy, to be achieved using high-efficiency cooling systems to cool the heat generated by their servers, and that have minimized impact on the environment.
• The demonstration test took place at KDDI's Oyama NC. By integrating the technologies and expertise of the three collaborating companies each specialized in their respective areas of social infrastructure, the testing confirmed the immersion cooling data center's stability and viability.

2. Demonstration outline

On April 1, 2022, demonstration testing got underway at KDDI's Oyama NC of a data center incorporating 100kVA class servers and other IT components and an immersion cooling system. Using a newly developed free cooling unit that performs optimized air cooling, the test demonstrated reduced energy consumption and improved waste heat treatment. In addition, high availability was obtained in the immersion cooling and free cooling units, confirming stable operation and viability of use of such equipment to achieve a Tier 4 level immersion cooling data center.

3. Results

• (1)

  Substantiation of cooling efficiency

  • Using the newly developed immersion cooling system, incorporating a free cooling unit [5] to perform optimized cooling by outside air, and assuming its installation in a data center, the testing resulted in higher waste heat treatment capacity and less energy consumption. Energy consumed to cool the servers was cut by 94%, with a PUE of 1.05.

• (2)

  High availability

  • High availability was achieved in the immersion cooling and free cooling units, and engineering was developed for their installation in a Tier 4 immersion cooling data center. The demonstration substantiated their stable operation and viability. In addition, compared to air-cooled systems, noise generated by the IT components was reduced by approximately 35dB [6].

• (3)

  Preparation of maintenance manual in anticipation of commercialization

  • In anticipation of commercial use in the domestic market, a maintenance system was taken under consideration and a maintenance manual prepared.

4. Roles of the 3 companies

• • KDDI

    : Management to ensure smooth demonstration test operation.
    System engineering for applying immersion cooling to the data center, taking availability into account.
    Field work in preparation for installing IT components, and resolution of issues relating to maintenance and operation.

• • MHI

    : Development and trial production of the free cooling unit.
    Immersion cooling system engineering and fabrication.
    Immersion cooling system control and trial operation.
    Maintenance and operation engineering of the free cooling unit.

• • NESIC

    : Immersion cooling data center component deployment engineering, problem identification and improvement.
    Procurement of immersion cooling units, power supply, etc.; engineering; problem identification and improvement through construction.
    Verification of monitoring, management and control methods through SI engineering and fabrication of an integrated monitoring system.
    Establishment of optimal maintenance engineering, operation, and maintenance scheme.

5. Companies cooperating in the demonstrating test

• AMD Japan, Ltd.
• Arista Networks Japan LLC
• Cisco Systems G.K.
• DC ASIA Ltd.
• Dell Technologies Japan Inc.
• ENEOS Holdings, Inc.
• FXC, Inc.
• Giga Computing Technology Co., Ltd.
• Hewlett Packard Japan, G.K.
• Industrial Technology Research Institute (ITRI)
• Intel Corporation
• MiTAC Computing Technology Corp.
• Nihon Form Service Co., Ltd.
• NVIDIA Corporation
• Panduit Corporation Japan Branch
• Solidigm Japan
• Super Micro Computer, Inc.
• Western Digital Corporation

Going forward, the three companies will apply the results of the demonstration testing as a way of contributing to the advancement of DX in Japan, decarbonization, and protection of the global environment.

<Attachment>

■Progress to date

1. Phase 01 July 2020 to January 2021

KDDI conducted the basic demonstration testing of an immersion cooling data center in Taiwan. The test was carried out on a containerized data center inside a 20-foot container, with the new immersion cooling technology used in combination with outside air-cooling. The test yielded a PUE of 1.09.

2. Phase 02 June 2021 to March 2022

KDDI, MHI and NESIC successfully conducted demonstration testing of an immersion cooling system to liquid-cool the IT components of a data center contained in a small container. Compared to a conventional data center, energy consumption was reduced by 43% and a containerized data center with a PUE of 1.07 was achieved.

June 21, 2021 Press Release
KDDI, MHI and NESIC issued a press release (in Japanese only) announcing the start of demonstration testing targeting use of immersion cooling in a small data center. The testing, aimed toward achieving decarbonization, will demonstrate how immersion cooling of servers reduces energy consumption.

March 29, 2022 Press Release
A press release (in Japanese only) announced successful achievement of a 43% reduction in energy consumption of a containerized data center incorporating liquid-cooled servers. The system underwent trial operation at KDDI's Oyama Technical Center (TC) aimed at commercialization in FY2024, to contribute to decarbonization.

3. Phase 03 April 2022 to March 2023

KDDI, MHI and NESIC successfully conducted demonstration testing of a hyper-scale immersion cooling data center, its IT components liquid-cooled, at KDDI's Oyama NC, achieving stable operation. Compared to a conventional data center, energy consumption was reduced by 94% through liquid cooling of the servers. A PUE of 1.05 was achieved.

Through the efforts made to date, data center scale has steadily increased and PUE gradually lowered. Going forward, further development will continue toward the achievement of fully sustainable immersion cooling data centers.

• [1]

  The tier is a measure of the assessed quality and rating level of a data center. Tier 4 indicates that, based on the redundancy of its auxiliary components, etc., a data center maintains a high level of quality.

• [2]

  Compared to the total power consumed by the typical data centers currently in use (PUE: 1.7).

• [3]

  PUE: power usage effectiveness, a general measure of the energy efficiency of a data center. It is calculated as follows: total energy consumed by data center (kWh) ÷ energy consumption of IT components (kWh). The lower the PUE value, the higher the energy efficiency of the data center.

• [4]

  A containerized data center is a small-scale data center offering mobility to a location closer to the end user. It consists of an immersion cooling system (immersion cooling and free cooling units) integrating 50kVA class servers.

• [5]

  A direct cooling method using water cooled by outside air. By feeding the cooling water without need to operate an air-cooled chiller normally used with air-cooled servers, cooling can be performed only using fan power and the conveyance power of the cooling water, without needing the power of the compressor inside an air-cooling chiller. This enables reductions in running costs, depending on the outside air temperature.

• [6]

  dB (decibel) is the unit used to measure noise level. The noise reduction confirmed in the demonstration test, 35db, represents the difference in noise level between use of air-cooled servers (equivalent to inside a metro station) and use of immersion cooling servers (normal conversation level).