Skip to content

Reduce size of Brent factor tables - #28

Merged
fingolfin merged 1 commit into
masterfrom
mh/shrink-brent-tables
Aug 7, 2026
Merged

fingolfin merged 1 commit into
masterfrom
mh/shrink-brent-tables

Conversation

@fingolfin

Copy link
Copy Markdown
Member

Fixes #14.

The factor tables of b^k +/- 1 nowadays maintained by Jonathan Crombie have grown to 280 MB uncompressed. That is far too much to distribute with the package, and has blocked a release for several years.

What this does

Two things:

  1. Store the tables more compactly — drop line continuations and whitespace from the data files.
  2. Restrict the shipped data. A factor of b^k +/- 1 is included if
    • b <= 100, or
    • b is a prime below 1000, or
    • the pair (b,k) was already covered by the tables in FactInt 1.6.3.

The third clause is what makes this safe: the result is a strict superset of the data shipped in 1.6.3. No user loses anything they have today.

files raw gzipped
before 9897 280.2 MiB 102.1 MiB
after 2275 22.3 MiB 9.9 MiB
for reference: 1.6.3 2275 10.0 MiB ~5 MiB

Since 23531a13 the .release script compresses these files, so the installed footprint is ~15 MB — the same as 1.6.3 had.

Verification

  • No regression against 1.6.3: 0 bases and 0 factors from 1.6.3 are missing, while 669493 factors are gained.
  • No corruption: every retained factor is present in the full upstream dataset. The filtering was done with a parser/renderer that reproduces all 9897 original files byte-for-byte.
  • Tests pass (0 failures).
  • Removed bases degrade gracefully: e.g. FactInt(1202^60-1) still returns a correct factorization, BRENTFACTORS[1202] simply stays unbound.
  • The .gz files written by the release script are read transparently — Filename, IsReadableFile and Read all fall back to <name>.gz, checked against a live GAP.

Not in scope

The download-on-demand tooling discussed in #14 is left for a follow-up, as is documenting FetchBrentFactors properly (it is currently undocumented and hardcodes a dated URL). The manual now describes which subset is shipped and points at FetchBrentFactors for the full collection.

One observation for that follow-up, from measuring where the bytes actually are: 66% of the full dataset consists of factors with 15 or fewer decimal digits — ones FactInt rediscovers in milliseconds. Dropping those alone would take the complete collection from 280 MiB to 95 MiB, which is a far better lever than restricting bases if we ever want to ship broader coverage.

🤖 Generated with Claude Code

The tables of factors of b^k +/- 1 that are nowadays maintained by
Jonathan Crombie have grown to 280 MB uncompressed. That is far too much
to distribute with the package, and has blocked a release for years, see
#14.

Store the tables more compactly, dropping line continuations and
whitespace, and restrict the shipped data to those (b,k) with b <= 100,
or b a prime below 1000, or (b,k) already covered by FactInt 1.6.3.

The result is a strict superset of the data shipped in 1.6.3: no base and
no factor is lost, and roughly 670000 further factors are gained. At the
same time tables/brent shrinks from 280.2 MiB to 22.3 MiB, or 9.9 MiB
once the .release script compresses it, which matches the on-disk
footprint of 1.6.3.

Users who need factors beyond this range can still install the full
collection using FetchBrentFactors.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@codecov

codecov Bot commented Aug 7, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.49%. Comparing base (5ba8697) to head (f715f5b).

Additional details and impacted files
@@            Coverage Diff             @@
##           master      #28      +/-   ##
==========================================
- Coverage   99.61%   94.49%   -5.12%     
==========================================
  Files          22       21       -1     
  Lines       47006     3323   -43683     
==========================================
- Hits        46823     3140   -43683     
  Misses        183      183              
Files with missing lines Coverage Δ
tables/brent/brfac10 100.00% <ø> (ø)
tables/brent/brfac11 100.00% <ø> (ø)
tables/brent/brfac12 100.00% <ø> (ø)

... and 6 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@fingolfin
fingolfin merged commit b06b7bb into master Aug 7, 2026
5 checks passed
@fingolfin
fingolfin deleted the mh/shrink-brent-tables branch August 7, 2026 12:03
@Stefan-Kohl

Copy link
Copy Markdown
Member

@fingolfin Thank you very much!
Another thing one can do besides not adding additional "small" factors with, say, no more than 15 digits is only adding new factors which yield new complete factorizations, as those are arguably worth much more than just adding more factors to incomplete factorizations where still an unfactored several-thousand-digits composite number remains. I think this will considerably limit the growth of the database, as most newly-computed factors will be relatively small factors (less than, say, 50 digits, often even less than 20 or 30 digits) of quite large b^k+/-1 (3-or 4-digit base b and 4- or 5-digit exponent k).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update the factor databases

2 participants