cloud_utils

Small utilities for working with images and logs on S3.

pip install numpy tifffile zarr s3fs opencv-python   # cv2 only for downsampled reads

s3_file_reader.py — read pyramidal TIFFs from S3 or disk

Only the tiles you ask for are downloaded (byte-range requests), so the file should be tiled and ideally pyramidal. Spacings are µm/px, coordinates are level-0 pixels.

from s3_file_reader import S3FileReader

with S3FileReader("s3://bucket/slide.tif") as slide:
    slide.shapes            # [(width, height), ...] per level
    slide.spacings          # [0.25, 0.5, 1.0, ...]

    patch = slide.read_patch(50000, 30000, 512, 512, spacing=0.5)  # x, y = patch centre
    patch = slide.read_patch(50000, 30000, 512, 512, spacing=0.5, center=False)
    thumb = slide.thumbnail(width=1024)     # or area=..., min_size=...
    whole = slide.full_image(spacing=8.0)   # mind the RAM

Handy extras: level_for_spacing, shape_for_spacing, snap_spacing, has_spacing, finer_spacing / coarser_spacing, spacing_for_size, level_for_min_size, and the module-level spacing_from_metadata, mm2_to_px / px_to_mm2, nearest / largest_at_most.

Options: storage_options={...} (passed to s3fs.S3FileSystem), spacing_tolerance=0.3, allow_downsample=True (read a finer level and resize when no level matches the requested spacing).

Aperio, Ventana/BIF, Philips and plain TIFFs are handled; label and macro images are dropped from the pyramid.

logging_utils.py — JSON logs for CloudWatch

from logging_utils import setup_logging, get_logger

setup_logging()                          # once, at startup ($LOG_LEVEL or INFO)
log = get_logger(__name__, run="job-42") # bound to every line from this logger

log.info("reading file", bucket="my-bucket", key="slides/a.tiff")

Every keyword becomes a top-level JSON field, so it is queryable:

fields @timestamp, message, bucket, key | filter bucket = "my-bucket"