One of the most powerful supercomputers in the world has now been fully installed and tested at its remote, high altitude site in the Andes of northern Chile. This marks one of the major remaining milestones toward completion of the Atacama Large Millimeter/submillimeter Array (ALMA), the most elaborate ground-based telescope in history. The special-purpose ALMA correlator has over 134 million processors and performs up to 17 quadrillion operations per second [1], a speed comparable to the fastest general-purpose supercomputer in operation today [2].

The ALMA correlator’s 134 million processors will continually combine and compare faint celestial “signals” received by as many as 50 dish-shaped antennas in the main ALMA array, enabling the antennas to work together as a single, enormous astronomical telescope. The correlator can additionally accommodate up to 14 of the 16 antennas in the Atacama Compact Array (ACA), a separate part of ALMA provided by the National Astronomical Observatory of Japan (NAOJ), for a total of 64 antennas [3]. In radio telescope arrays, sensitivity and image quality increase with the number of antennas.

Funded by the US National Science Foundation (NSF), and designed, constructed, and installed primarily by the National Radio Astronomy Observatory (NRAO), the ALMA correlator is a critical component in a radio telescope system that astronomers are already using to make new discoveries about how planets, stars, and galaxies form. Unlike optical telescopes, which observe visible light emitted by stars, ALMA explores a region of the spectrum of invisible light, the millimeter and sub-millimeter wavelength realm.

When observing, ALMA’s antennas point at the same celestial object in the sky, gathering faint radio waves. Before astronomers can make detailed images or do other analyses, the information collected by dishes separated by as much as 16 kilometers must be extensively computer processed.

The ALMA correlator performs the first critical steps in this data processing. To make the entire system work as a single telescope, the information collected by each antenna must be combined with that from every other antenna. At the correlator’s maximum capacity of 64 antennas, there are 2,016 antenna pair combinations, and as many as 17 quadrillion calculations every second.

“The completion and installation of the correlator is a huge milestone toward the fulfillment of North America’s share of the international ALMA construction project,”said Mark McKinnon, North American ALMA Project Director at NRAO. “The technical challenges were enormous, and our team pulled it off,” he added.

One of the challenges facing the correlator design team, led until his retirement by correlator expert Ray Escoffier, was cost. “As we confronted this project, we realized that it would have taken, at the time, a billion dollars’ worth of off-the-shelf personal computers to perform the needed calculations. We built our own custom machine for about 11 million dollars,” said John Webber, former head of the NRAO Central Development Laboratory in Charlottesville, Virginia.

Another challenge was the extreme location. The correlator is housed in the ALMA Array Operations Site (AOS) Technical Building, the highest-altitude high-tech building in the world. At 16,500 feet, the air is thin, so twice the normal airflow is necessary to cool the machine, which draws some 140 kilowatts of power. Computer disk drives will not work reliably in thin air, so the correlator and its associated computers must be diskless. Seismic activity is common, so the correlator had to be designed to withstand the vibrations associated with earthquakes.

The thin air at the ALMA site also affects human performance. While the AOS Technical Building is expected to be unmanned during routine ALMA operations, it took about 20 weeks of human effort there to unpack and install the correlator. “There are thousands upon thousands of cable connections that we had to make, and every one of our cables is the same color blue, so I’m just glad we devised a good labeling system while at sea-level,” said NRAO’s Rich Lacasse, leader of the ALMA Correlator Team.

Enrique Garcia, a correlator technician, examines the system while breathing oxygen from a backpack. ALMA’s Array Operations Site, at an elevation of 5000 meters on the Chajnantor Plateau in the Chilean Andes, is so high that only half as much oxygen is available as at sea level.
Credit: ESO/Max Alexander

ALMA’s European partner, led by the European Southern Observatory (ESO), provided an important component of the correlator, 550 “next-generation” circuit boards, built by the University of Bordeaux [4] under the leadership of Alain Baudry, that increase the system’s ability to distinguish fine shades of color, or wavelength. With these filters, the light which ALMA sees can be split up into 32 times more wavelength ranges than in the initial design, and each of these ranges can be finely tuned. “This vastly improved flexibility is fantastic” – said Alain Baudry, from the University of Bordeaux, the European ALMA correlator team leader- “it lets us ‘slice and dice’ the spectrum of light that ALMA sees, so we can concentrate on the precise wavelengths needed for a given observation, whether it’s mapping the gas molecules in a star-forming cloud, or searching for some of the most distant galaxies in the Universe”.

ALMA began science observations in 2011 with a partial array of antennas. A section of the correlator was already being used to combine the signals from the partial array, but now the full system is complete. The correlator is ready for ALMA to begin operating with a larger number of antennas, which will increase the sensitivity and image quality of the observations.

ALMA is nearing completion and will be inaugurated in March 2013.

Notes
[1] 17 quadrillion = 17 000 000 000 000 000.
[2] The current record holder in the TOP500 list of general-purpose supercomputers is the Titan, from Cray Inc., which has been measured at 17.59 quadrillion floating point operations per second. Note that the ALMA correlator is a special-purpose supercomputer and is not eligible for this ranking.
[3] The ALMA correlator is one of two such systems in the ALMA complex. ALMA’s total of 66 antennas comprise a main array of 50 antennas (half provided by ESO, and half by NRAO) and an additional, complementary array of 16 antennas called the Atacama Compact Array (ACA), which is provided by the National Astronomical Observatory of Japan (NAOJ). A second correlator, built by the Fujitsu company and delivered by NAOJ, provides independent correlation of the 16 antennas in the ACA, except for times when select ACA antennas are combined with the 50 more widely dispersed main array antennas.
[4] This work followed work on new concepts for the correlator, done by the University of Bordeaux in a consortium also involving ASTRON in the Netherlands, and the Arcetri Observatory in Italy.

The photo below is tone of four quadrants of the ALMA correlator. The full system has four identical quadrants, with over 134 million processors, performing up to 17 quadrillion operations per second.
Credit: ALMA (ESO/NAOJ/NRAO), S. Argandoña