Research & engineering

From camera geometry
to spatial playback.

VCS develops the hardware, data workflows and rendering software that connect real scenes to interactive spatial media.

Two monitors displaying depth-map processing in the 2021 GTC presentation. — Depth and reconstruction workflow · VCS at NVIDIA GTC 2021. 2021 (project)

Capture systems

Multiple views.
One moment.

Moving subjects need camera views that agree in time. Our camera development spans synchronized arrays, compact stereoscopic systems, custom optics and calibration.

For Demonic, the team built the multicamera capture system used to record volumetric performances. That hardware work gave us practical experience with camera placement, large image sequences and the demands of reconstructing a moving person.

The capture system

Person inside multi-camera stage during build/testing. — Multicamera capture-stage assembly and testing. 31 Jul 2020 (filename)

Spatial intelligence research · Volusynth

Images with
geometric context.

An image alone does not say where the camera was, how it projected the scene or how far away each surface is. We develop workflows that keep that information together.

One view. Linked spatial information.Data schematic

01RGB

Image & frame identity

02Depth

Distance at each pixel

03Camera

Calibration & pose

intrinsics
rotation
translation

Each image must agree with its depth map and camera coordinates. Schematic of the data relationship, not model output.

Calibrated RGB, depth and camera pose.

Our research includes real-camera reconstruction and synthetic multiview exports with linked colour images, depth maps, camera intrinsics and extrinsics.

We test depth range and direction, image-to-depth pairing, coordinate transforms and transparent surfaces. These checks matter because a plausible-looking image can still carry incorrect geometry.

Volusynth is our research direction for synthetic spatial data and its validation. Dataset development and model evaluation remain ongoing.

Spatial playback

Make depth behave
in a moving view.

Our native-player experiments have tested stereo video, layered-depth imagery and Gaussian-splat rendering. The engineering includes projection, geometry ordering, colour sampling and decoder performance.

Mirage’s current roadmap carries that work into a WebGPU browser player and a dedicated Apple Vision Pro player. The earlier cloud-rendered browser technology was licensed for SuperMuseum: America.

Mirage development

Mirage spatial-player interface shown in the supplied presentation. — Spatial-player prototype interface. By Apr 2025

Accelerated computing

GPU workloads.

Depth processing, reconstruction, AI image operations and interactive rendering each place different demands on compute and memory.

01 / Workstations

Local inference and image processing

VCS’s development hardware has included NVIDIA RTX A6000, GV100 and GeForce RTX 4090 GPUs. HP documents local AI inference, background inpainting, high-resolution masking and multi-GPU work in our image workflows.

02 / Playback

Cloud rendering and browser-native rendering

The demonstrated Mirage browser route renders on a cloud GPU and streams the result. Current WebGPU development instead targets rendering on the viewer’s GPU in a compatible browser.

03 / Evaluation

Depth and reconstruction experiments

We evaluate reconstruction and depth-estimation methods against captured and synthetic imagery, including spatial alignment and temporal stability. That work combines established research models with our data, processing and playback code.

HP’s published account of the VCS GPU workflow

Published technical records