Skip to content

FEAT: Share image, audio, and video previews with dataset details - #3251

Open
Julio César Suástegui (juliosuas) wants to merge 10 commits into
microsoft:mainfrom
juliosuas:feat/2749-shared-media-previews
Open

Julio César Suástegui (juliosuas) wants to merge 10 commits into
microsoft:mainfrom
juliosuas:feat/2749-shared-media-previews

Conversation

@juliosuas

@juliosuas Julio César Suástegui (juliosuas) commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Description

Closes #2749

Dataset example details can now preview image, audio, and video members using the same component as chat. I pulled ImageWithSpinner and MediaWithFallback out of MessageList.tsx into a shared MediaPreview, and chat passes the same props/classes as before, so it renders the same way.

In dataset details, nothing is fetched until you click "Show … preview". Then the image uses loading="lazy", audio/video use preload="none", nothing autoplays, and hiding or unmounting the preview pauses playback. If a load fails, you get a message with Retry and Download.

Backend side: the seed detail response now includes value_url on each member. It comes from resolve_client_media_url_async, a thin public wrapper over the existing _resolve_and_sign_media_async, so local files go through /api/media and blob URLs get signed the same way as in chat. value stays the stable stored reference and value_url is just the display URL. Only the detail endpoint resolves URLs. The list endpoint doesn't touch them, so browsing the table never signs or downloads anything.

On the frontend, mapMediaUrl in utils/media.ts decides whether a member is a preview, download-only, missing, or not media. It never turns a bare filesystem path into a browser URL, it uses the same inline allowlist as routes/media.py, and SVG/HTML/other active types are never rendered.

A few things I wasn't sure about, happy to change any of them:

  • Chat keeps retry/download off, so its behavior doesn't change. Easy to turn on there too if you want.
  • binary_path members are download-only, no preview.
  • Cross-origin SVG/HTML gets no download link, only the "cannot be previewed" note. Through /api/media it's still downloadable since the route serves it as an attachment.
  • When the file is missing, the backend currently leaves the raw path in value_url (that's what the existing resolver does) and the UI shows "File is missing." Returning null might be cleaner.
  • Because of preload="none", a broken file or an unsupported codec in an allowed container only fails once you hit play. Expired and inaccessible links share one generic error message for now.

Screenshots

Dataset detail with media members

Dataset detail with media members

Image preview

Image preview

Audio preview

Audio preview

Video preview

Video preview

Tests and Documentation

  • Jest: 130 suites / 3083 tests pass after merging main (new tests for MediaPreview and mapMediaUrl, plus the dataset detail). MessageList.test.tsx is unchanged and passes 63/63, same as on main.
  • pytest: full unit suite 27902 passed on the merged head. New route tests cover local files, a missing file, and blob signing with the signer mocked, so no Azure credentials are needed. (6 signal-timing tests in tests/unit/infra flaked while the machine was loaded; that folder passes 398/398 on its own, same as main, and the PR doesn't touch it.)
  • Playwright full mock suite: 190/190. @seeded against the real backend: 88/88. No retries.
  • datasets.spec.ts repeated 20x with --fail-on-flaky-tests: 200/200, and @seeded 20x: 40/40 (before merging main). It checks there are no /api/media requests before you click, no autoplay, and that broken/SVG members don't break the table.
  • Lint, type-check, and ruff pass.

@romanlutz Roman Lutz (romanlutz) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Three media preview issues to address:

Comment thread frontend/src/components/Media/MediaPreview.tsx Outdated
Comment thread frontend/src/components/Media/MediaPreview.tsx Outdated
Comment thread frontend/src/components/Media/MediaPreview.styles.ts Outdated
Cross-origin recovery links open in a new tab. Retry refetches a signed display URL. Tall images stay inside the preview.
A retry replaces only that member's display URL, so other playback keeps its place. The new URL goes through the same allowlist as the first load, and a failed refresh stays on screen.
A retry can still be in flight when the example is closed or replaced.
Keep that response from writing its display URL, or its failure, onto the example that is open now.
Comment thread frontend/src/hooks/useSeedExamples.ts Outdated
A refresh now follows the same request as the open example, so closing it, switching examples, or changing runtime aborts that request before its URL can be applied. Retrying one member no longer cancels a refresh already running for another.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

FEAT GUI: Share image, audio, and video previews with datasets

2 participants