# Copy & Paste the World with Gaussian Splatting - Scott Leaman - NDC Oslo 2026

## Executive summary

This talk explores advanced methods for capturing, editing, and displaying real-world environments in 3D, focusing heavily on Gaussian Splatting. The speaker details traditional 3D capture techniques (Lidar, Photogrammetry) and contrasts them with the novel approach of Gaussian Splatting, which generates explorable 3D scenes from photos or videos using 'Gaussian blobs.' The presentation highlights the utility of these technologies, particularly in infrastructure planning (e.g., railway systems) and real estate, by enabling the creation of highly accurate digital twins for comparing plans against physical reality.

## Key takeaways

- Gaussian Splatting (GS) Overview: GS is a process for creating explorable 3D scenes from photos or videos, generating the scene using 'Gaussian blobs' (ellipsoids) rather than traditional meshes. Each blob contains position, color, opacity, scale, and rotation data. It is particularly effective at capturing organic or complex environments and retaining fine silhouettes, which can be challenging for 3D meshes.
- Comparison of 3D Capture Techniques: Three primary methods for capturing real-world spatial data are discussed: Lidar (Laser Imaging Detection and Ranging), which provides highly accurate point clouds; Depth Cameras (like the original Microsoft Kinect), which are cheaper but less accurate; and Photogrammetry, which uses overlapping photos to calculate the scene's shape and camera positions.
- Digital Twin Applications in Infrastructure: In the railway industry, Lidar scans are used to create detailed point clouds of infrastructure. These data are processed on platforms like Nvidia Omniverse, which allows remote streaming and meshing of the data. This enables infrastructure teams to perform critical tasks such as collision detection, testing new equipment placement, and comparing current physical measurements against original 2D schematics.
- Advanced Editing and AI Generation: The speaker demonstrated using online tools like Super Splat to copy, paste, and delete parts of a Gaussian Splat. Furthermore, AI tools like Marble can generate Gaussian Splats from text prompts or images, estimating object depth to create surreal or complex virtual scenes (e.g., mixing Oslo and Kyoto).

## Technical details

- Lidar Data Processing: Lidar data is inherently a point cloud. For usability, it can be converted into a mesh for performance and calculations. Platforms like Nvidia Omniverse facilitate the handling of dense, complex 3D data, allowing for remote streaming and meshing of the point cloud data.
- Gaussian Splat Data Structure: A Gaussian Splat file (e.g., PLY or SPZ) stores data for each 'Gaussian blob,' including position (3D), color, opacity, scale (3D), rotation (quaternion), and optionally, Spherical Harmonics data to capture view-dependent color changes (reflectivity).
- Web Viewing and Rendering: Viewing Gaussian Splats in a web browser is supported by libraries like 3js, which now supports PLY and SBZ files using the WebGPU. For advanced enterprise use, platforms like Nvidia Omniverse support ray-traced Gaussian Splats, allowing virtual objects to cast realistic shadows and reflections.
- Localization and Positioning: By co-locating a 3D mesh (derived from the scan) and the Gaussian Splat, users can implement visual positioning systems that offer better than traditional GPS accuracy indoors and outdoors, allowing precise tracking within the modeled space.

## Practical implications

- Infrastructure teams can use Lidar data and digital twin platforms to verify physical installations against plans, reducing costly re-planning and travel time.
- Gaussian Splatting enables the creation of highly realistic, explorable virtual environments from minimal input (photos/videos), useful for real estate sales and museum archiving.
- The combination of 3D mesh and Gaussian Splat data allows for highly accurate indoor/outdoor localization, improving on traditional GPS accuracy.
- The technology supports pre-production planning for filmmaking and special effects (e.g., 4D Gaussian Splats).

## Topics

3D Modeling, Virtual Reality (VR), Augmented Reality (AR), Gaussian Splatting, Digital Twins, Lidar Scanning, AI/Generative AI, Nvidia Omniverse, Super Splat, Niantic Scanniverse, Marble (World Labs), 3js

Source: https://www.youtube.com/watch?v=4FlLqv21SxE