Repository navigation
GForce fails to do overflow checks for integer64 #7574
Description
Related to #7571.
DT = data.table(i = c(1L, 1L), x = lim.integer64()[2L])
DT[, sum(x), by=i]
# i V1
# <int> <i64>
# 1: 1 -2
DT[, base::sum(x), by=i]
# i V1
# <int> <i64>
# 1: 1 <NA>
# Warning message:
# In sum.integer64(c(NaN, NaN), na.rm = FALSE) :
# NAs produced by integer64 overflow
Activity
ashish-um commentedon Jan 9, 2026
I have reproduced this on the latest dev version.
It seems the overflow happens in
src/gsumm.cbecause theint64path lacks the "fallback to numeric" logic that the standard integer path has.I traced the execution to the
gsum64logic. Specifically, in thecase REALSXPblock (handling int64), the addition happens here without an overflow check:Unlike the 32-bit integer block above it (which checks
if (overflow)and restarts usingREALSXP), this loop blindly adds, leading to the wraparound behavior.Will-78 commentedon Mar 25, 2026
Hello @MichaelChirico,
Our group is interested in working on this particular issue. We would like to know if the issue still has relevancy before we take on this issue.
If there is any other relevant information about this particular issue we would love to hear more about it!
JackOfAllTrades90 commentedon Mar 29, 2026
Hi @MichaelChirico @ben-schwen,
I've been looking into this issue and managed to reproduce the overflow behavior on the current dev version. It's interesting how base::sum catches the overflow while the GForce path in src/gsumm.c just lets it wrap around to a negative value.
I’ve been tracing the logic in gsumm.c and noticed that for the int64 path (around the REALSXP case handling bit64), we're missing the check-and-promote logic that's present in the 32-bit integer block. Essentially, it's doing a 'blind' addition on block_ans.
My plan to fix this:
I’d like to implement a check for the int64 sum. If it hits an overflow, I’ll have it follow the same pattern as the 32-bit logic—stopping the current accumulation and restarting/promoting that group to a double (REALSXP) so we don't lose the result.
I'll also add a few test cases in tests.Rraw specifically using lim.integer64() to make sure sum() and base::sum() stay in sync.
I’m a student looking to contribute to data.table for GSoC 2026, and I’d love to take this on as a first task. Does this approach align with how you'd like the int64 overflow handled, or is there a specific preference for keeping it strictly within the bit64 realm?"
aitap commentedon Mar 29, 2026
Will-78 commentedon Mar 30, 2026
Thank you for researching into the issue,
We are still interested in the issue, like you we are also students looking forward in working on this particular issue. As part of our class group project we aim to resolve the issue for a grade. We will take into account your ideas, however, and explore them as well so that your contributions aren't for naught.
Will-78 commentedon Apr 20, 2026
@MichaelChirico @aitap
Hello, we are close to posting a PR for this issue but we were wondering how the answer should be posted. With the example for int32 overflow the answer is printed correctly through
doubletype. However for int64 there isn't a datatype we can find that can store and return 1.844674e+19 back to the user. Currently we are thinking of leaving in the lost precision of the double while listing it within the error. We wanted to check in to see if there was a preference for the resulting number from the sum.aitap commentedon Apr 21, 2026
Losing precision with a warning is okay. We can lose precision using ordinary integers as well by amassing enough of them: