Remove sc.external #2717

External has outlived its usefulness with the emergence of the scverse ecosystem, and should be removed.

Plan:

1. Decide on which ones to move into our code: https://scverse.zulipchat.com/#narrow/channel/328272-scanpy/topic/Removing.20.60sc.2Eexternal.60/near/616514023
2. Deal with exporting.{spring_project,cell_browser}: Reorg io functions #4296
3. Adopt bbknn: Adopt scanpy.pp.bbknn #4302
4. Move hashsolo: move sc.external.pp.hashsolo into sc.pp #4304
5. Deprecate sc.external: deprecate sc.external #4295
6. remove it

Adopt code

projects without upstream activity

• pp.scrublet
• pp.harmony_integrate (harmonypy has 2.0 depending on Torch so we adopted v1)
• pp.bbknn (implemented in RSC): Adopt scanpy.pp.bbknn #4302

already in-tree: move into main namespace

• hashsolo (upstream has been moved into scvi-tools): move sc.external.pp.hashsolo into sc.pp #4304

  other than scrublet, this operates on hashtags, so both are useful

Maintained: deprecate wrappers and tell the authors

• tl.phate: We’re removing scanpy.external KrishnaswamyLab/PHATE#182
• tl.palantir{,_results}: We’re removing sc.external dpeerlab/Palantir#182
• tl.trimap (has a JAX implementation by Google): We’re removing scanpy.external eamid/trimap#31
• tl.sam (sc-sam): scanpy is getting rid of .external and also has harmony now atarashansky/self-assembling-manifold#51
• exporting.cellbrowser: Reorg io functions #4296

maintained-ish, don’t necessarily wait for reply

• tl.harmony_timeseries (harmonyTS): We’re deprecating scanpy.external dpeerlab/Harmony#27
• pp.magic (magic-impute): see Palantir (the MAGIC algorithm is included in there)

Dead: just deprecate

• pp.dca: chore: remove some long-deprecated functions #4010
• exporting.spring_project: Reorg io functions #4296
• tl.phenograph (PhenoGraph. With feat: add jaccard as an additional method in sc.pp.neighbors #3831 – jaccard connectivities – only label transfer is missing, which exists in https://github.com/quadbio/cellmapper)
• pp.scanorama_integrate (scanorama, but is implemented in SnapATAC2)
• pp.mnn_correct (mnnpy)
• tl.wishbone (wishbone-dev)
• tl.sandbag / tl.cyclone (pypairs)

theislab votes to keep:

• phate: 3 (won’t keep as it’s maintained well)
• phenograph: 2 (because of its performance on cancer data)
• magic, palantir, cyclone, cellbrowser: 1 each