Skip to content

Write BFINAL with the caller's bit pointer, as C's AddBit does - #219

Open
kunpai wants to merge 1 commit into
google:masterfrom
kunpai:bit-pointer-unmasked
Open

kunpai wants to merge 1 commit into
google:masterfrom
kunpai:bit-pointer-unmasked

Conversation

@kunpai

@kunpai kunpai commented Oct 28, 2026

Copy link
Copy Markdown

Write BFINAL with the caller's bit pointer, as C's AddBit does

Problem

ZopfliDeflate and ZopfliDeflatePart (C, Rust) take the caller's bit pointer bp. C's comment says bp is always in 0..7, and so does the port's safety note, but neither side checks it. The first bit a block writes is BFINAL (C, Rust), and the two sides write it differently when bp is 8 or more:

Every bit after BFINAL is the same on both sides, since (bp + 1) & 7 is ((bp & 7) + 1) & 7. For example, ZopfliDeflate with btype = 1, final = 2^31-1, bp = 115, an empty input and one output byte 0xb7:

C Rust
output b7 00 (BFINAL dropped) bf 00 (BFINAL at bit 3)

With final = 0 both sides write a 0 bit, so they agree.

Question

Which behaviour do we want: C's or the Rust port's? Specifically, when bp is 8 or more, should the block's BFINAL bit be shifted by the caller's bp, so that it is dropped and a multiple of 8 starts no new byte (like the C source), or should bp be masked to 0..7 before anything is written (like the Rust port)?

This PR implements C's behaviour.

Fix

  • deflate_helper keeps the caller's bp: let old_bp = *bp in place of *bp & 7. A bp that is a multiple of 8 then continues the caller's last byte, as in C.
  • add_bit shifts as C does: the shift is done on a u32, as C's bit << *bp is done on an int, so a bit shifted past bit 7 is dropped. bp is advanced with wrapping_add, as C's int arithmetic does for bp 255.
-    out[last_idx] |= bit << *bp;
-    *bp = (*bp + 1) & 7;
+    out[last_idx] |= (bit as u32).wrapping_shl(*bp as u32) as u8;
+    *bp = bp.wrapping_add(1) & 7;
-    let old_bp = *bp & 7;
+    let old_bp = *bp;

Replayed before and after (x86-64), on the campaigns' records:

C Rust main Rust patched
ZopfliDeflate, btype = 1, bp 115 (the example) b7 00 bf 00 b7 00
ZopfliDeflate, btype = 1, bp 246, a 2-byte input output different output C's output
ZopfliDeflatePart, btype = 1, bp 244, a 2-byte input output different output C's output

The .outsize records the campaigns found have an out-of-range btype as well as a bp of 8 or more. With this PR and #212, the ZopfliDeflate ones (bp 196, 99 and 64) agree with C too.

Downstream check. The normal path is untouched (bp in 0..7): ZopfliGzipCompress on the repository's README gives the same 831-byte stream from the port before and after this patch, and gzip.decompress restores the file.

Notes for review:

…s C does

C's AddBit shifts the bit by the caller's bp as given, so for bp >= 8 the
BFINAL bit falls outside the byte, and only then masks bp to 0..7.
deflate_helper masked bp before writing anything, so the port wrote
BFINAL at bit bp & 7 and, for a multiple of 8, started a new byte.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant