Skip to content

Spherical projections: precision near the poles, the equator and the centres; tmerc scale factor; Putnins P2 - #140

Open
SpaceOceanCorp wants to merge 8 commits into
locationtech:masterfrom
SpaceOceanCorp:spherical-precision
Open

SpaceOceanCorp wants to merge 8 commits into
locationtech:masterfrom
SpaceOceanCorp:spherical-precision

Conversation

@SpaceOceanCorp

Copy link
Copy Markdown
Contributor

Precision fixes for spherical projections, the same ones submitted to PROJ in OSGeo/PROJ#4944, plus three plain bugs found while measuring (tmerc, Putnins P2, Hammer). Independent of the separate PR with the PROJ string parameter fixes (+R, polar stereographic, azimuthal defaults); either can be merged first.

Measured on a sphere of radius 1737400 m against 50-digit references on the same double inputs (the inverse as a great-circle distance), master vs this branch:

case master branch
tmerc sphere forward, |lon| <= 4, +k=0.999 121 m 4.5e-10 m
tmerc sphere forward, |lon| > 90 8590 m 2.8e-7 m
tmerc sphere inverse, |lon| <= 4 1.8 cm 5.0e-10 m
ortho inverse, polar / equatorial / oblique 1.7e-4 m / 5450 km / 3640 km 4e-9 to 8e-9 m
stere inverse, equatorial / oblique 1.5e-4 m / 3.7 cm 6e-10 to 9e-10 m
aeqd forward near the centre 240 m 8e-10 m
aeqd inverse, polar / oblique 1.7e-4 m / 3.9 cm 9e-10 m
laea inverse, equatorial / oblique 1.5e-4 m / 2.8 cm 9e-10 m
gnom inverse, polar / oblique 1.7e-4 m / 3.7 cm 9e-10 m
moll forward / inverse near the poles 254 km / 2.5 cm 3.7e-6 m / 4.6e-6 m
eck4 forward near the poles 186 m 2.1e-8 m
putp2 forward near the poles 5190 km 1.1e-8 m
vandg forward, |lon|, |lat| < 1 degree / elsewhere 5460 km (282 points fail) / 8.4 m 8.9e-12 m / 5.0e-9 m
vandg inverse near the origin 14 m 1.7e-4 m
aitoff / wintri forward near the origin 3.8 cm / 1.9 cm 1.2e-11 m / 6.9e-12 m

Changes, in six commits plus a test:

  1. Transverse Mercator on a sphere. ml0 is already 0.5 * k0 for the sphere, and the forward multiplied x by the scale factor again, so with +k=0.999 x was 0.1% short (the 121 m above). The forward also took y from acos() of a value close to 1 (half the digits lost near the equator) and gave y = 0 instead of pi * R * k0 on the equator for |lon| > 90; atan2(|sin(phi)|, cos(phi) * cos(lam)) is the same angle without either problem. The inverse latitude uses atan2(|sin(D)|, hypot(g, h)).
  2. Spherical azimuthal inverses (ortho, gnom, stere, aeqd, laea): the latitude from atan2() instead of asin() of a value close to +/-1, and pi/2 - c for polar stere. The shortcuts within EPS10 of the centre now apply at the centre only: ortho took the longitude from the unscaled x, y there, which put points within 1e-10 radians of the centre at longitude near 180 (the 5450 km above). laea and aeqd no longer return longitude 0 where the longitude is +/-90. The aeqd forward takes sin(c) from hypot() and c from atan2() instead of acos(cos(c)), which sent points with cos(c) within 1e-14 of 1 to the centre.
  3. Mollweide, Eckert IV, Putnins P2 near the poles. Putnins P2's Newton iteration updated out.y instead of the auxiliary angle, so the angle stayed equal to the latitude and the result was not Putnins P2 (the 5190 km above). Near the poles the Newton iterations converge only linearly: Mollweide gave up after 10 iterations and snapped to the pole, Eckert IV stopped 186 m off. Near the poles the equations are now solved for the distance of the angle from its polar value, and the inverses use the same relations instead of asin() near 1. Mollweide gets PROJ's 30 iterations.
  4. ProjectionPrecisionTest: 15 points (22 with the two later commits) on a sphere of radius 6371000 m with 50-digit expected values; it fails on master and passes here.
  5. Van der Grinten near the origin. A, G and P of Snyder's (29-1) and (29-2) are all large there and the forward subtracted them, losing all the digits (points within 1e-7 degrees of the origin failed). The same values now come from a rearrangement in which no step subtracts two large terms, the shortcuts within 1e-10 of the equator and the central meridian apply to exact zeros only, and the inverse longitude avoids the cancellation of sqrt(t) against pi^2 - r. The inverse keeps its shortcut for |y| < 1e-10 (the 1.7e-4 m above), as PROJ does. Three points added to the test.
  6. Aitoff and Winkel Tripel forward near the origin: sin(d) from hypot() and d from atan2() instead of acos() of a value close to 1, as in PROJ. Four forward points added to the test.
  7. Hammer: setW() and setM() had no effect. initialize() replaced w and m by 0.5 and 1 whenever they were positive. They are now kept (new test hammerUsesWAndM).

mvn test gives the same results as on master otherwise (all pass; the same 18 known FAIL lines printed by the existing comparison tests).

Not changed here, noted: Aitoff, Winkel Tripel and Hammer have no inverse in Proj4J (hasInverse() is false), and inverseProject() then returns its input unchanged rather than failing; PROJ has inverses for all three. The PROJ string parser does not read +W / +M for Hammer.

@SpaceOceanCorp

Copy link
Copy Markdown
Contributor Author

Pushed 473f539: the spherical gnom inverse computed cos(z) as sqrt(1 - sin(z)^2), which loses its digits close to 90 degrees from the centre, where the coordinates are very large, and defeated the new atan2() of the equatorial and oblique modes. The proj4rs maintainer found it in the review of the same change (3liz/proj4rs#65). New test gnomonicInverseFarFromTheCentre (6 points, 0.8 mm to 1.6 cm off before, on a sphere of 6371000 m); the full suite passes (74 tests, 2 skipped as before).

The forward took y from acos() of a value close to 1, which loses half the
digits close to the equator, and gave y = 0 instead of pi * R * k0 for
|longitudes| > 90 on the equator; atan2(|sin(phi)|, cos(phi) * cos(lam)) is
the same angle without either problem. The inverse latitude is now
atan2(|sin(D)|, hypot(g, h)) instead of asin(sqrt((1 - h * h) / (1 + g * g))).
(The scale factor applied twice is fixed in master by locationtech#133.)

The precision changes are the ones submitted to PROJ as OSGeo/PROJ#4944.

Signed-off-by: SpaceOceanCorp <info@spaceoceancorp.com>
The spherical inverses of ortho, gnom, stere, aeqd and laea took the latitude
from asin() of a value close to +/-1 near the poles, which loses half the
digits (2 to 4 cm at the Moon's radius); atan2() of rh * sin(phi) and
rh * hypot(cos(phi) * cos(lam), cos(phi) * sin(lam)) keeps them, and the
polar stere uses pi/2 - c. The shortcuts within 1e-10 of the centre now
apply at the centre only (they were up to 1e-10 * R off), and laea and aeqd
no longer return longitude 0 instead of +/-90 where the cos(lam) term is
exactly 0.

aeqd forward: sin(c) from hypot() and c from atan2() instead of
acos(cos(c)); points with cos(c) within 1e-14 of 1 went to the centre
(240 m off at the Moon's radius), as in current PROJ.

The same changes were submitted to PROJ as OSGeo/PROJ#4944.

Signed-off-by: SpaceOceanCorp <info@spaceoceancorp.com>
The Newton iterations converge only linearly at the poles: Mollweide gave up
after 10 iterations and snapped to the pole (254 km off at 89.3 degrees on a
sphere of the Moon's radius), Eckert IV stopped 186 m off, and Putnins P2
(with the iteration on the auxiliary angle, as fixed in master by locationtech#133) also
converges slowly there. Close to the poles the equations are now solved in
terms of the distance of the angle from its polar value, and the inverses
use the same relations instead of asin() of a value close to 1. Mollweide
gets the 30 iterations of PROJ.

The polar handling is the one submitted to PROJ as OSGeo/PROJ#4944.

Signed-off-by: SpaceOceanCorp <info@spaceoceancorp.com>
15 points for tmerc, ortho, aeqd, laea, gnom, moll, eck4 and putp2 on a
sphere of radius 6371000 m, with expected values from a 50-digit
computation; the inverse is checked as a great-circle distance in metres.
The test fails before the previous commits.

Signed-off-by: SpaceOceanCorp <info@spaceoceancorp.com>
Close to the origin A, G and P of Snyder's (29-1) and (29-2) are all large
and the forward subtracted them, which lost all the digits: points within
1e-7 degrees of the origin failed or went up to 5460 km off at the Moon's
radius. The same values are now computed from a rearrangement in which no
step subtracts two large terms. The shortcuts within 1e-10 of the equator
and of the central meridian now apply to exact zeros only (they were off by
up to 1e-10 * R), and the inverse longitude avoids the cancellation of
sqrt(t) against pi^2 - r close to the origin (14 m).

Three van der Grinten points added to ProjectionPrecisionTest.

Signed-off-by: SpaceOceanCorp <info@spaceoceancorp.com>
The forward took d from acos(cos(phi) * cos(lam / 2)), which loses half the
digits close to the origin (3.8 cm off at the Moon's radius). sin(d) from
hypot() and d from atan2() keep them, as in PROJ.

Four Aitoff and Winkel Tripel points added to ProjectionPrecisionTest
(forward only: Proj4J has no inverse for these).

Signed-off-by: SpaceOceanCorp <info@spaceoceancorp.com>
The forward used m after initialize() had divided it by w, so a second call
of initialize() divided it again. m / w now has its own field. (Keeping the
W and M set with setW() and setM() is fixed in master by locationtech#133.)

Test: hammerUsesWAndM, with a 50-digit expected value for W = 0.25, M = 2.

Signed-off-by: SpaceOceanCorp <info@spaceoceancorp.com>
cos(z) was computed as sqrt(1 - sin(z)^2), which loses its digits close to
90 degrees from the centre, where the coordinates are very large, and
defeated the atan2() of the equatorial and oblique modes. The new test
gnomonicInverseFarFromTheCentre checks 6 such points (expected values from
50-digit computations of the inverse of the same double inputs). Found by
the proj4rs maintainer in the review of the same change there
(3liz/proj4rs#65).

Signed-off-by: SpaceOceanCorp <info@spaceoceancorp.com>
@SpaceOceanCorp

Copy link
Copy Markdown
Contributor Author

Rebased on master with #133; the overlapping fixes are kept as they are in #133 (details in #139). The full suite passes: 135 tests, 0 failures, 2 skipped as on master.

@pomadchin

pomadchin commented Nov 14, 2026 •

Copy link
Copy Markdown
Member

Hi there, really appreciate all your work here!

While I could see it as an improvement; we'd have to align with OSGeo/PROJ#4944
i.e. if it is more correct, that does not matter much if users of the library have different expectations; we'll have to be on a little bit of a conservative side unfortunately.

I see some similar sentiments expressed on the PR boostorg/geometry#1513

Let's wait up until it gets merged in some way!

I however I cherry picked and backported some of the fixes from this PR #143

@SpaceOceanCorp

Copy link
Copy Markdown
Contributor Author

Thanks for taking these in #143, and understood on staying aligned with PROJ. One note: OSGeo/PROJ#4944 was closed on 12 Nov rather than merged, so it may not land in that form. Fine to leave this open or close it, whichever suits you.

Paul S. Mamakos, Space Ocean Corp

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants