Skip to content

DM-56783: Update the text about efficient join with a user-uploaded table #240

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
galaxyumi merged 1 commit into from
Oct 29, 2026
Merged

DM-56783: Update the text about efficient join with a user-uploaded table #240

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 14 additions & 3 deletions DP2/100_How_to_use_RSP_tools/102_Catalog_access/102_5_Cross_match_to_uploaded_table.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
"Data Release: [Data Preview 2](https://dp2.lsst.io)\\\n",
"Container Size: Large\\\n",
"LSST Science Pipelines version: r30.0.11\\\n",
"Last verified to run: 2026-08-18\\\n",
"Last verified to run: 2026-10-28\\\n",
"Repository: [github.com/lsst/tutorial-notebooks](https://github.com/lsst/tutorial-notebooks)\\\n",
"DOI: [10.11578/rubin/dc.20250909.20](https://doi.org/10.11578/rubin/dc.20250909.20)"
]
Expand Down Expand Up @@ -199,9 +199,20 @@
"source": [
"### 2.2. Execute the cross-match\n",
"\n",
"To cross-match a user-uploaded table against the LSST catalogs and retrieve the results, Qserv allows users to temporarily upload their catalogs to Qserv and use them in queries. It is recommended to apply a `DISTANCE`-based join in coordinate-based cross-matching as demonstrated in this Section or use unique identifier-based joins (like `ObjectId`) as demonstrated in Section 3, for which Qserv automatically partitions the user-uploaded table using the same algorithm used for the LSST catalogs. This approach makes a query highly efficient by targeting only the relevant chunks of data.\n",
"To cross-match a user-uploaded table against the LSST catalogs and retrieve the results, Qserv allows users to temporarily upload their own catalogs and use them in queries. When a query joins the uploaded table on coordinates, Qserv automatically partitions it with the same algorithm used for the LSST catalogs. Each uploaded source is then compared only against LSST objects in the same or neighboring sky chunks, not the whole catalog, which keeps the query efficient. Two spatial join types are supported: `DISTANCE`-based join and `CONTAINS`-based. For a simple radius match, both approaches give the same result and run with similar efficiency.\n",
"\n",
"Query the `Object` table for objects matching the coordinates from the user-defined table, `ut1`. Use a spatial cross-match radius of 1\\\" (or 0.00027 degrees)."
"Query the `Object` table for objects that lie within 1\\\" (or 0.00027 degrees) of each source in the uploaded table, `ut1`. The `CONTAINS` version below is shown for reference only and is not executed in this notebook:\n",
"\n",
"query = \"\"\" \\\n",
" SELECT ut1.objid AS ut1_objid, ut1.ra AS ut1_ra, ut1.dec AS ut1_dec,\\\n",
" dp2.objectId, dp2.coord_ra, dp2.coord_dec\\\n",
" FROM dp2.Object AS dp2\\\n",
" JOIN TAP_UPLOAD.ut1 AS ut1\\\n",
" WHERE CONTAINS(POINT('ICRS', dp2.coord_ra, dp2.coord_dec),\\\n",
" CIRCLE('ICRS', ut1.ra, ut1.dec, 0.00027)) = 1\\\n",
" \"\"\"\n",
" \n",
"The query run in the next cell uses the `DISTANCE`-based join."
]
},
{
Expand Down
Loading