Full Text Available
Note: Clicking the button above will open the full text document at the original institutional repository in a new window.
Fast online predictive compression of radio astronomy data
This report investigates the fast, lossless compression of 32-bit single precision floating-point values. High speed compression is critical in the context of the MeerKAT radio telescope currently under construction in Southern Africa and Australia, which will produce data at rates up to 1 Petaby...
Saved in:
| Main Author: | |
|---|---|
| Format: | Thesis |
| Language: | English |
| Published: |
Computer Science
2016
|
| Tags: |
No Tags, Be the first to tag this record!
|
| Summary: | This report investigates the fast, lossless compression of 32-bit single precision
floating-point values. High speed compression is critical in the context of
the MeerKAT radio telescope currently under construction in Southern Africa and
Australia, which will produce data at rates up to 1 Petabyte every 20 seconds.
The compression technique being investigated is based on predictive compression,
which has proven successful at achieving high-speed compression in previous research.
Several different predictive techniques (which includes polynomial extrapolation),
along with CPU- and GPU-based parallelization approaches are discussed.
The implementation successfully achieves throughput rates in excess of 6 GiB/s
for compression and much higher rates for decompression using a 64-core AMD
Opteron machine, achieving file-size reductions of, on average 9%. Furthermore
the results of concurrent investigations into block-based parallel Huffman encoding
and Zero-length Encoding are compared to the predictive scheme and it was found
that the predictive scheme obtains approximately 4%-5% better compression ratios
than the Zero-Length Encoder and is 25 times faster than Huffman encoding on
an Intel Xeon E5 processor. The scheme may be well-suited to address the large
network bandwidth requirements of the MeerKAT project. |
|---|