Multi-image scene editor

The single-image SpatialData viewer streams one image at a time. The multi-image scene editor goes a step further: it loads several SpatialData image elements into one WebGPU scene, lets you click to select an image, and shows a 3D gizmo on the selected object so you can translate, rotate, and scale it — the interaction you need to co-register images by hand.

Each image keeps its own HTJ2K LOD streaming (one tile renderer + spatialdata.js loader per image), so detail re-resolves as you move it. Selection is tracked in the scene by raycasting on click, which is what decides where the gizmo appears and which image the numeric panel edits.

Co-registration. When the store carries a real spatialdata.js global transform per image, the images are placed through one shared global→world mapping, so they land already co-registered (the same tissue overlaps) with an identity editable transform — the gizmo is then for fine-tuning. For example, a Xenium H&E image and its multi-channel morphology image start aligned, one over the other. Stores without a stored transform fall back to a stagger.

Layers & blending. Overlapping co-registered images are near-coplanar, so they composite as layers (viv/MDV-style) rather than z-fighting: the first image is the opaque base, and later images draw over it in add order with a per-image opacity and blend mode (normal = over, additive = sum). This is what lets you fade an H&E overlay onto a fluorescence base to compare modalities.

Running it

The editor is a React frontend built on the playground’s Vite app (it reuses the streaming pipeline and the webgpu / zarrextra build config that already live there). It is published with these docs — open the multi-image scene editor — or run it locally against your own store:

pnpm --filter intraspatial-playground dev

All the prototypes are listed on the Viewer prototypes page.

• Add images: open a store, pick an element, and Add to scene. Add more than one to co-register.
• Select: click an image — a gizmo appears on it and the panel shows its transform.
• Manipulate: drag the gizmo, or press R / T / S for rotate / translate / scale. Edit the numeric fields for precision. Orbit the camera by dragging the empty background.
• Layer: set the selected image’s opacity and blend (normal / additive) to composite it over the layers beneath. Per-channel colour/window controls are below that.

Runtime only

The scene is not persisted yet — transforms live only for the session. The state is modelled as a serialisable object (store + element + transform + channels per image), so a localStorage demo and, later, writing the edited placement back to a SpatialData coordinate system (which needs write support in spatialdata.js plus a backend) are follow-on steps.

Local data

Like the single-image viewer, the demo expects a local CORS SpatialData / OME-Zarr store — by default a server at localhost:8080. Point the store URL at any reachable store; its image elements are listed automatically. A publicly-hosted sample dataset is planned.