- Video twin = live video + coordinate alignment + engine texture mapping.
- The core is coordinate alignment: map video pixels into 3D world coordinates.
- Four steps: distortion correction → coordinate mapping → texture binding → engine output.
- Works with Unreal, Unity, Cesium and Three.js, reusing existing 3D assets.
1. What a Video Twin Actually Twins
A video twin (also called video fusion or real-scene twin) overlays live camera video onto a 3D scene at exact spatial positions, so operators see the real site inside the model. Unlike a pure digital twin, which synchronises structure and state data, a video twin shows what is happening right now — the live picture, not just numbers.
2. The Four Implementation Steps
| Step | What It Does | Key Point |
|---|---|---|
| 1. Distortion correction | Undistort fisheye / wide-angle footage | Makes geometry reliable for mapping |
| 2. Coordinate alignment | Map video pixels to world coordinates | Camera intrinsics + extrinsics |
| 3. Texture binding | Bind video as texture to scene geometry | Plane or surface in the 3D scene |
| 4. Engine output | Render the fused scene in a 3D engine | Unreal / Unity / Cesium / Three.js |
The hardest part is step 2 — coordinate alignment. You solve each camera's intrinsics and extrinsics (by calibration or photogrammetry), map video pixels into world coordinates, then bind the video as a texture to the matching plane or surface. Point-cloud or BIM input raises accuracy significantly.
Note on terminology: this calibration only maps video pixels to 3D coordinates. The stitching step itself needs no on-site calibration.
3. Supported Engines
- Unreal Engine — high-fidelity rendering, suitable for large-scale command centers.
- Unity — lightweight and flexible, good for rapid deployment.
- Cesium — geospatial scenes; bind video to a rectangle / polygon entity or custom material.
- Three.js — web-based, no plugin, easy to share.
SuperMetaX's engine is SpatioForge, with live-video injection experience across all four.
4. Three Input Conditions
Deployment differs by what 3D data you already have:
- Model available — map video directly onto the existing model, fastest path.
- Point cloud only — rebuild a lightweight scene, then map video.
- Drawings only — model from drawings or photogrammetry, longer schedule, lower precision.
5. FAQ
Can existing 3D models or engines be reused?
Yes. Fusion does not replace existing 3D assets; the SDK is adapted to the engine version while models, scenes and business logic stay unchanged.
How many video channels can one scene take?
There is no fixed ceiling — concurrent channels scale with hardware and scene complexity. Judge by channels shown live on screen, not total channels managed.
Does it need on-site calibration?
Coordinate mapping needs camera calibration, but the stitching step itself needs no on-site calibration. The two are separate concerns.