Full Text Available
Note: Clicking the button above will open the full text document at the original institutional repository in a new window.
Fast online predictive compression of radio astronomy data
This report investigates the fast, lossless compression of 32-bit single precision floating-point values. High speed compression is critical in the context of the MeerKAT radio telescope currently under construction in Southern Africa and Australia, which will produce data at rates up to 1 Petaby...
Saved in:
| Main Author: | |
|---|---|
| Format: | Thesis |
| Language: | English |
| Published: |
Computer Science
2016
|
| Tags: |
No Tags, Be the first to tag this record!
|
| _version_ | 1867613219133063168 |
|---|---|
| access_status_str | Open Access |
| author | Hugo, Benjamin |
| author_browse | Hugo, Benjamin |
| author_facet | Hugo, Benjamin |
| author_sort | Hugo, Benjamin |
| collection | Thesis |
| description | This report investigates the fast, lossless compression of 32-bit single precision
floating-point values. High speed compression is critical in the context of
the MeerKAT radio telescope currently under construction in Southern Africa and
Australia, which will produce data at rates up to 1 Petabyte every 20 seconds.
The compression technique being investigated is based on predictive compression,
which has proven successful at achieving high-speed compression in previous research.
Several different predictive techniques (which includes polynomial extrapolation),
along with CPU- and GPU-based parallelization approaches are discussed.
The implementation successfully achieves throughput rates in excess of 6 GiB/s
for compression and much higher rates for decompression using a 64-core AMD
Opteron machine, achieving file-size reductions of, on average 9%. Furthermore
the results of concurrent investigations into block-based parallel Huffman encoding
and Zero-length Encoding are compared to the predictive scheme and it was found
that the predictive scheme obtains approximately 4%-5% better compression ratios
than the Zero-Length Encoder and is 25 times faster than Huffman encoding on
an Intel Xeon E5 processor. The scheme may be well-suited to address the large
network bandwidth requirements of the MeerKAT project. |
| format | Thesis |
| id | oai:open.uct.ac.za:11427/21225 |
| institution | University of Cape Town (South Africa) |
| language | eng |
| last_indexed | 2026-06-10T12:32:39.476Z |
| license_str | Not specified — see source repository |
| provenance_str_mv | Harvested via OAI-PMH from UCTD — University of Cape Town Open Access Repository |
| publishDate | 2016 |
| publishDateRange | 2016 |
| publishDateSort | 2016 |
| publisher | Computer Science |
| publisherStr | Computer Science |
| record_format | dspace |
| source_str | UCTD — University of Cape Town Open Access Repository |
| spelling | oai:open.uct.ac.za:11427/21225 Fast online predictive compression of radio astronomy data Hugo, Benjamin This report investigates the fast, lossless compression of 32-bit single precision floating-point values. High speed compression is critical in the context of the MeerKAT radio telescope currently under construction in Southern Africa and Australia, which will produce data at rates up to 1 Petabyte every 20 seconds. The compression technique being investigated is based on predictive compression, which has proven successful at achieving high-speed compression in previous research. Several different predictive techniques (which includes polynomial extrapolation), along with CPU- and GPU-based parallelization approaches are discussed. The implementation successfully achieves throughput rates in excess of 6 GiB/s for compression and much higher rates for decompression using a 64-core AMD Opteron machine, achieving file-size reductions of, on average 9%. Furthermore the results of concurrent investigations into block-based parallel Huffman encoding and Zero-length Encoding are compared to the predictive scheme and it was found that the predictive scheme obtains approximately 4%-5% better compression ratios than the Zero-Length Encoder and is 25 times faster than Huffman encoding on an Intel Xeon E5 processor. The scheme may be well-suited to address the large network bandwidth requirements of the MeerKAT project. 2016-08-13T18:45:54Z 2016-08-13T18:45:54Z 2013 2016-08-13T18:25:43Z Master Thesis Masters Hons http://hdl.handle.net/11427/21225 eng application/pdf Computer Science Unknown University of Cape Town University of Cape Town |
| spellingShingle | Hugo, Benjamin Fast online predictive compression of radio astronomy data |
| thesis_degree_str | Master's |
| title | Fast online predictive compression of radio astronomy data |
| title_full | Fast online predictive compression of radio astronomy data |
| title_fullStr | Fast online predictive compression of radio astronomy data |
| title_full_unstemmed | Fast online predictive compression of radio astronomy data |
| title_short | Fast online predictive compression of radio astronomy data |
| title_sort | fast online predictive compression of radio astronomy data |
| url | http://hdl.handle.net/11427/21225 |
| work_keys_str_mv | AT hugobenjamin fastonlinepredictivecompressionofradioastronomydata |