Skip to content

zstd: clean up and fix extraction

q66 requested to merge q66/apk-tools:fix-zstd-extraction into master

The original logic was somewhat wrong. We should not be trying to repeatedly decompress the stream with the same input, as there is no guarantee it will safely advance the input all the way.

Instead, we should call it once and then ask for more input if it has been advanced to the end. Regardless, we should return when the output buffer has been filled, and keep trying to get more input every time; zstd will eventually advance the input buffer so that we either get satisfactory data, an EOF, or an error.

While at it, handle another edge case. The extraction generally works without it but just for robustness and to match what the documentation is saying; if decompressStream fully fills in an output buffer, there may be more data internally, so save that state and next time try getting that data to return it even if we get an EOF from the stream.

If an output buffer is filled only partially, it means zstd has processed everything. In this case, this generally means an end of file, so the input buffer will have been fully advanced and we get an EOF from the wrapped stream on next iteration; since we got some data, we disregard this EOF and return the data.

We've been testing the zstd packaging in Chimera the last few days and noticed some specific packages failing to extract with either unexpected end of file or failing digest comparison. I have been trying to track down the cause of it and this fixes these cases.

Merge request reports