Computer Science

If the Compression Type integer of a Channel Block is 0x5A425200020000 (Zebra), a Zebra Compression Stream immediately follows the Compression Type integer, after which the end-of-channel-block marker follows the Zebra Compression Stream.

Zebra is a simple lossless compression scheme for compressing floating point or integer numbers. It consists of splitting up number samples into bytes, and storing the corresponding bytes of the different samples together: the most significant byte of all the samples are stored together, the next most significant byte of the samples are stored together, etc.

Each such group of bytes is called a Byte Channel. If the samples are 32-bit numbers (4 bytes per sample), then there are 4 Byte Channels. If the samples are 64-bit numbers (8 bytes per sample), then there are 8 Byte Channels.

Byte Channels are stored in Big Endian format:  most significant Byte Channel first, next most significant Byte Channel next, etc.;  the least significant byte of all the image channel samples are the last Byte Channel, the second least significant byte is the second-to-last Byte Channel, etc.

The Zebra Compression Stream begins with a start-of-zebra marker, followed with a 64-bit Size integer that stores the Size of the Zebra Compression Stream (including the start-of-zebra marker and this Size integer), followed with a 64-bit Compression Type integer (0x5A425200020000), followed with a 32-bit integer that stores the Sample Stride (must equal the Channel Block Sample Stride), followed with a 32-bit integer that stores the Image Width (must equal the File Header Image Width), followed with a 32-bit integer that stores the Image Height (same as File Header Image Height), followed with a 32-bit integer that stores the Filter Type, followed with Byte Channels (4 Byte Channels for 32-bit samples or 8 Byte Channels for 64-bit samples), followed with the end-of-zebra marker that marks the end of the Zebra stream.

Note: The Zebra Compression Stream is self-contained and may be used in any file format (not just in an XRH file).

The first four bytes of a Zebra Compression Stream store the value 0x535A4200 (ASCII string SZB\0) which marks the start-of-zebra.

Start of Zebra Stream Marker:  SZB\0

The last 4 bytes of a Zebra Compression Stream store the value 0x455A4200 (ASCII string EZB\0) which marks the end-of-zebra (the last 4 bytes of the Zebra Compression Stream).

End of Zebra Stream Marker:  EZB\0

The data bytes of a Byte Channel are compressed using ZST o (a lossless compressor, also called “Zstandard”).

Each Byte Channel consists of a start-of-byte-channel marker, followed with a 64-bit integer that specifies the size of the ZST code stream that stores the data bytes of the Byte Channel, followed with that ZST code stream (the compressed data generated by ZST), followed with an end-of-byte-channel marker.

The first four bytes of a Byte Channel store the value 0x53424300 (ASCII string SBC\0) which marks the start-of-byte-channel.

Start of Byte Channel Marker:  SBC\0

The last 4 bytes of a Byte Channel store the value 0x45424300 (ASCII string EBC\0) which marks the end-of-byte-channel.

End of Byte Channel Marker:  EBC\0

The ZST code stream of each Byte Channel is a compressed byte stream that when uncompressed provides the sample bytes of the Byte Channel in raster order.

For example, to reconstruct the 5th sample of a Channel Block: the 5th data byte of the first Byte Channel becomes the high order byte of the 5th sample of the Channel Block, the 5th data byte of the second Byte Channel becomes the next most significant byte of the 5th sample of the Channel Block, etc.

If all of the bytes of a Byte Channel are the same, then a Default Value (one byte) may be stored that specifies the value of all the bytes of the Byte Channel. In that case, zero is stored in Size of ZST Code Stream (see Figure 3 above) and the single-byte Default Value is stored in place of the ZST Code Stream (immediately after the Size of ZST Code Stream, without storing a ZST Code Stream), after which the end-of-byte-channel marker immediately follows the single-byte Default Value.

If Filter Type equals one (1), the samples are filtered before being split up into Byte Channels, to map floating point numbers into integer sequences.

Otherwise, if Filter Type equals zero (0), there is no mapping of floating point numbers to integer sequences. That is useful if the samples are already integers, for example ancillary data.

For Filter Type equals one (1), each sample must be converted (mapped) from floating point to unsigned integer. To perform this mapping, first copy the floating point bit sequence of each sample directly into integer memory (bitwise bijection), then toggle the leading sign bit if positive, or toggle all bits if negative. The mapped samples (unsigned integers) are then split up into Byte Channels, and each Byte Channel is sent to the ZST encoder.

For data retrieval of Filter Type 1, after uncompressing, the Byte Channels are merged (shuffled) back into unsigned integers which are then inverse mapped back to the original floating point numbers.

For Filter Type equals zero (0), the samples are considered to be unsigned integers already and are split up into Byte Channels, and each Byte Channel is sent to the ZST encoder.

For data retrieval of Filter Type 0, after uncompressing, the Byte Channels are merged (shuffled) back into unsigned integers which are the retrieved samples.

Filter Type 0 is useful for samples that are already integers, or to apply your own custom filter.

ZST compression is used instead of ZLIB, because ZST allows for compression of larger buffers.

Filter Type 1 as described above is used because it creates a monotonic sequence of the floating point values as unsigned integers. This is described in “Mapping to Integer”, Section 3.2 in Peter Lindstrom & Martin Isenburg, “Fast and Efficient Compression of Floating-Point Data”, which uses a different type of compression but uses and explains this filter.