Copy & Paste the World with Gaussian Splatting - Scott Leaman - NDC Oslo 2026
Summary
This talk explores advanced methods for capturing, editing, and displaying real-world environments in 3D, focusing heavily on Gaussian Splatting. The speaker details traditional 3D capture techniques (Lidar, Photogrammetry) and contrasts them with the novel approach of Gaussian Splatting, which generates explorable 3D scenes from photos or videos using 'Gaussian blobs.' The presentation highlights the utility of these technologies, particularly in infrastructure planning (e.g., railway systems) and real estate, by enabling the creation of highly accurate digital twins for comparing plans against physical reality.
Key takeaways
-
Gaussian Splatting (GS) Overview
20:42
GS is a process for creating explorable 3D scenes from photos or videos, generating the scene using 'Gaussian blobs' (ellipsoids) rather than traditional meshes. Each blob contains position, color, opacity, scale, and rotation data. It is particularly effective at capturing organic or complex environments and retaining fine silhouettes, which can be challenging for 3D meshes.
-
Comparison of 3D Capture Techniques
6:40
Three primary methods for capturing real-world spatial data are discussed: Lidar (Laser Imaging Detection and Ranging), which provides highly accurate point clouds; Depth Cameras (like the original Microsoft Kinect), which are cheaper but less accurate; and Photogrammetry, which uses overlapping photos to calculate the scene's shape and camera positions.
-
Digital Twin Applications in Infrastructure
11:40
In the railway industry, Lidar scans are used to create detailed point clouds of infrastructure. These data are processed on platforms like Nvidia Omniverse, which allows remote streaming and meshing of the data. This enables infrastructure teams to perform critical tasks such as collision detection, testing new equipment placement, and comparing current physical measurements against original 2D schematics.
-
Advanced Editing and AI Generation
26:40
The speaker demonstrated using online tools like Super Splat to copy, paste, and delete parts of a Gaussian Splat. Furthermore, AI tools like Marble can generate Gaussian Splats from text prompts or images, estimating object depth to create surreal or complex virtual scenes (e.g., mixing Oslo and Kyoto).
Technical details
-
Lidar Data Processing
750s
Lidar data is inherently a point cloud. For usability, it can be converted into a mesh for performance and calculations. Platforms like Nvidia Omniverse facilitate the handling of dense, complex 3D data, allowing for remote streaming and meshing of the point cloud data.
-
Gaussian Splat Data Structure
1280s
A Gaussian Splat file (e.g., PLY or SPZ) stores data for each 'Gaussian blob,' including position (3D), color, opacity, scale (3D), rotation (quaternion), and optionally, Spherical Harmonics data to capture view-dependent color changes (reflectivity).
-
Web Viewing and Rendering
1800s
Viewing Gaussian Splats in a web browser is supported by libraries like 3js, which now supports PLY and SBZ files using the WebGPU. For advanced enterprise use, platforms like Nvidia Omniverse support ray-traced Gaussian Splats, allowing virtual objects to cast realistic shadows and reflections.
-
Localization and Positioning
1350s
By co-locating a 3D mesh (derived from the scan) and the Gaussian Splat, users can implement visual positioning systems that offer better than traditional GPS accuracy indoors and outdoors, allowing precise tracking within the modeled space.
Mentioned resources
- Nvidia Omniverse
- Super Splat
- Niantic Scanniverse
- Marble (World Labs)
- 3js
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.