Repository navigation
Do not interpolate a video's track across a GPX gap in process - #899
Merged
caglarpir merged 3 commits intoOct 24, 2026
Merged
Conversation
A file named on the command line that is neither an image nor a video
that mapillary_tools can process was left out of the description with
only a warning, "Skipping ..., which is not a supported image or video",
and process exited 0. Named on its own, such a file gave an empty
description and a successful run, so a user who named the original of an
Insta360 camera (.insv), which has to be stitched first, saw nothing
uploaded and no error.
Fail such a file with an error entry of its own, of the new type
MapillaryUnsupportedFileError: "Not a supported image or video". For an
Insta360 original, the message goes on to say to stitch it with Insta360
Studio first, and then to geotag the stitched video by the original with
--video_geotag_source '{"source": "exiftool_runtime", "pattern":
"../%g.insv"}', as the README does. Like any error, it makes process
exit 6, or 0 with --skip_process_errors, and upload skips it. The entry
names no file type, as the file is of none; the process summary counts
it as an "other file". A new error type rather than an existing one, as
none says what is wrong: the geotagging errors would let another source
try the file, and the file is not a video.
Unchanged: the files in a named folder that cannot be processed are
still left alone without a word, a named zip file is still uploaded as
it is by process_and_upload, and the process report still lists the
unsupported files, named or found, as unsupported_format.
Checked process on named images, videos, zip files, text files and an
Insta360 original, alone, together, named twice, beside a named folder,
with --filetypes image, and on folders holding such files, and
video_process and sample_video: only the runs that named an unsupported
file change, each by the one error entry. process_and_upload of named
zip files with a named text file uploads the zip files and fails the
text file.
Interval sampling cuts a frame every N seconds of the whole video, whatever GPS track will geotag the frames, so the frames past either end of the track were written and then failed geotagging, which failed the command: the last frame of a BlackVue video, cut after the last GPS point, with "Unable to extract GPS Longitude or GPS Latitude from the image", or the last frame of a GoPro MAX 2 timelapse sampled every 0.25 s against its GPX, 0.386 s past the end of the GPX, with MapillaryOutsideGPXTrackError. At a 1 s interval, 17 of 49 sample videos had such frames: one or two at an end of most, and up to 64 at the start of those whose GPS starts late. Leave such frames out, as distance sampling leaves out the frames outside the track it samples along, and warn once per video, e.g. "2 of the 61 samples of X fall outside its GPS track (A to B) and were left out". The track is the one that will geotag the frames: the video's own track, which sampling now writes into the frames, or a GPX or NMEA track of --geotag_source, which the geotagging stage locates the frames along by their capture times, with its offset, its start time and its gaps of more than an hour, as it does. If the track covers none of the frames, they are all written as before, so that a video that misses its track fails rather than goes without a word. The frames before the start of the track are kept where leaving them out would move the others: the legacy gopro, blackvue and camm sources and exiftool_xml geotag a frame by the time since the first frame of its video, which they place at the start of the video, and so does a GPX with --interpolation_use_gpx_start_time, which also starts at the first frame of all the videos sampled, so that with several videos none is left out. The sample_video command cannot tell which sources will geotag its frames, so it keeps them too. Nothing is left out of a GPX track that another source, which geotags frames by the GPS of their video or by another track file, may geotag instead. On those 49 videos, sampled every second with the default sources, the 17 lose only the frames that failed, and 16 of them now exit 0 (the other fails as before for an unrelated reason); every other description entry is the same. The BlackVue gives 60 frames and the GoPro MAX 2 timelapse 4, both exit 0. With the legacy sources, the frames before a late GPS start are kept and geotagged as before, and with a GPX of --geotag_source only the frames that it rejected are left out. Distance sampling does not change.
process uploads the track it places a GPX on a video by with the video, and the frames are placed along it. Since mapillary#874, a time in a gap of more than MAPILLARY_TOOLS_MAX_GPX_GAP_SECONDS (an hour) between two fixes is outside the track, but the track clipped to the video was still interpolated across such a gap. A video that starts in a gap got a point at video time 0 interpolated between the last fix before the gap and the first one after it. On a GoPro MAX video with a GPX whose fixes resume 99 seconds into the video, 2 hours after the previous one 5 km away, that point was 151 m from where the full track puts the start of the video, and the first 99 seconds of frames were placed between the two. At driving speed it is kilometres off. When video time 0, or the end of the video, falls in such a gap, leave out the fixes on its far side, and interpolate no point across it: the track starts (or ends) at the first fix past the gap, as one that starts after the video does, and process still warns that it covers only part of the video. A gap inside the video fails it with MapillaryOutsideGPXTrackError, which names the gap and suggests sampling the video with video_process and --video_geotag_source gpx (or nmea), which leaves out the frames in the gap. So does a gap that the video falls in but for an instant at one end, such as the gap after a lone first fix that the track is started with the video at. A track assumed to start with a video without a GPS clock keeps its end, which may be that of a timelapse, unless the end of the video at 1x falls in a gap. The limit is in GPS seconds, so it spans less video in a timelapse. Sampling is unchanged. Checked process with GPX and NMEA tracks made to have gaps: - The GoPro MAX video above: its track starts at 99.085 s, with the warning; with --interpolation_offset_time 2, at 97.085 s. With the track ending 159 s into the video and resuming 2 hours later, it ends at 159.085 s. With both, it runs from 99.085 to 159.085 s. - A gap of 70 s inside the video, with the limit at 60 s, fails it, by a GPS clock or with --interpolation_use_gpx_start_time, without a GPS clock, from an NMEA file, and in a timelapse, whose limit of 30 s is in GPS time. With native as the next source, native geotags it. video_process samples the same videos and tracks as before. - Without a GPS clock, a track that resumes 2 hours after the end of the video at 1x ends with the video, and one with a lone first fix 2 hours before the rest fails. With the limit raised above the gap, the tracks are interpolated across it as before. process with the GPX, NMEA and ExifTool tracks of sample videos and images from a dozen cameras (58 runs, among them GPX tracks of several trips with gaps of days, offsets and --interpolation_use_gpx_start_time), plain process, and video_process with GPX tracks write the same descriptions as before.
caglarpir
added a commit
that referenced
this pull request
Oct 24, 2026
A time in a gap of more than MAPILLARY_TOOLS_MAX_GPX_GAP_SECONDS (an hour by default) between two fixes of a GPX or NMEA track is outside the track. process (#899) and interval sampling compared the gaps of a track placed on a video in video seconds, against the limit divided by the rate of the video's clock. The division leaves the two a rounding error apart, so at a rate other than 1 a gap of exactly the limit could count as more than it. On a LabPano video, whose clock runs 1.143 times as fast as its video, a fix exactly an hour before a track that starts 17 seconds into the video was left out, and the track uploaded with the video started at 17.2 s rather than at 0, while interval sampling left out 2 of its 58 samples. With the limit at 60 s, a gap of exactly 60 s inside the video failed it in process, and interval sampling left out 5 of the 58 samples. Judge every gap in GPS seconds: between the GPS times that the points of a GPX or NMEA track keep once placed on the video, or, for points without one, between their times at the rate of the clock; rounded to the millisecond, which no GPS time is recorded more finely than. A gap of more than the limit is one, and a gap of exactly the limit is not, as the geotagging stage judges the gaps of a track for images and as distance sampling judged them already. Checked process, distance sampling and interval sampling of a LabPano, a BlackVue and a GoPro MAX timelapse video, each with GPX tracks made from its own track with a gap of exactly the limit, or half a second more, before the video and inside it, at limits of 3600 and 60 s (54 runs): only the LabPano runs with a gap of exactly the limit change, as above. Gaps half a second over the limit are left out as before, and the BlackVue and timelapse runs do not change. Nor do the 24 runs with real GPX and NMEA tracks of a GoPro MAX, a Sony camera and 70mai dashcams, whose logs have gaps of days.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #898; review the last commit.
process uploads the track it places a GPX on a video by with the video,
and the frames are placed along it. Since #874, a time in a gap of more
than MAPILLARY_TOOLS_MAX_GPX_GAP_SECONDS (an hour) between two fixes is
outside the track, but the track clipped to the video was still
interpolated across such a gap. A video that starts in a gap got a point
at video time 0 interpolated between the last fix before the gap and the
first one after it. On a GoPro MAX video with a GPX whose fixes resume 99
seconds into the video, 2 hours after the previous one 5 km away, that
point was 151 m from where the full track puts the start of the video,
and the first 99 seconds of frames were placed between the two. At
driving speed it is kilometres off.
When video time 0, or the end of the video, falls in such a gap, leave
out the fixes on its far side, and interpolate no point across it: the
track starts (or ends) at the first fix past the gap, as one that starts
after the video does, and process still warns that it covers only part
of the video. A gap inside the video fails it with
MapillaryOutsideGPXTrackError, which names the gap and suggests sampling
the video with video_process and --video_geotag_source gpx (or nmea),
which leaves out the frames in the gap. So does a gap that the video
falls in but for an instant at one end, such as the gap after a lone
first fix that the track is started with the video at. A track assumed
to start with a video without a GPS clock keeps its end, which may be
that of a timelapse, unless the end of the video at 1x falls in a gap.
The limit is in GPS seconds, so it spans less video in a timelapse.
Sampling is unchanged.
Checked process with GPX and NMEA tracks made to have gaps:
warning; with --interpolation_offset_time 2, at 97.085 s. With the
track ending 159 s into the video and resuming 2 hours later, it ends
at 159.085 s. With both, it runs from 99.085 to 159.085 s.
GPS clock or with --interpolation_use_gpx_start_time, without a GPS
clock, from an NMEA file, and in a timelapse, whose limit of 30 s is
in GPS time. With native as the next source, native geotags it.
video_process samples the same videos and tracks as before.
video at 1x ends with the video, and one with a lone first fix 2 hours
before the rest fails. With the limit raised above the gap, the tracks
are interpolated across it as before.
process with the GPX, NMEA and ExifTool tracks of sample videos and
images from a dozen cameras (58 runs, among them GPX tracks of several
trips with gaps of days, offsets and --interpolation_use_gpx_start_time),
plain process, and video_process with GPX tracks write the same
descriptions as before.
Unit and integration tests, doctests,
mypy --python-version 3.9,ruff check,ruff format --checkandusort diffare clean.