CASPSR: Phase-coherent dispersion removal using Graphics Processing Units
In collaboration with the Center for Astronomy Signal Processing and Electronics Research (CASPER) at Berkeley, the Swinburne Pulsar Group has designed and installed a new pulsar instrument at Parkes, the CASPER Parkes Swinburne Recorder (CASPSR). This next-generation baseband data recording and processing system consists of:
- 1 Interconnect Break-out Board (IBOB), developed at CASPER;
- 1 Cisco® SFS 7000 InfiniBand Server Switch;
- 2 Dell R610 Server Nodes, each with 48 GB of RAM; and
- 4 Supermicro® NITRO G5 Tesla GPU Compute Nodes, each with dual quad-core Intel® Xeon E5520 (Nehalem-EP) processors, 24 GB of RAM, and 2 NVIDIA Tesla C1060 GPUs with 4GB of memory.
The IBOB performs analog-to-digital conversion of two dual-polarization signals, each with a maximum total bandwidth of 400 MHz. These signals are streamed via 2 × 10 GbE connections to the 2 R610 servers, which buffer the data and remultiplex the UDP packets to deliver contiguous streams to the 4 GPU-based processing nodes.
Once in workstation RAM, the data are:
- routed, managed, and monitored using PSRDADA;
- coherently dedispersed and reduced in real-time using DSPSR; and
- archived for later analysis using PSRCHIVE.
These three Open Source software projects are developed at Swinburne in collaboration with pulsar research groups around the world.
The major technical innovations of CASPSR include:
- streaming data via conventional UDP packets instead of specialized Direct Memory Access (DMA) hardware;
- processing data directly from RAM, thereby eliminating the disk I/O bottle neck;
- exploiting the multiple graphics processing units on each workstation with multi-threaded data reduction software;
- professionally engineering the acquisition and analysis software under an Open Source license as part of our continuing dedication to foster international collaboration.
CASPSR is an integral part of our development path toward pulsar instrumentation for next-generation facilities like the Australian Square Kilometre Array Pathfinder (ASKAP) and the Karoo Array Telescope (MeerKAT).