3D Gaussian Splatting: A Demo

A "Gaussian Splat" is mathematically just a 3D ellipsoid. To render a photorealistic scene, the computer stores a list of millions of these splats. Each splat has a few simple properties that define it:

1. Position (x, y, z): Where the centre of the splat is located.
2. Covariance (Scale & Rotation): This defines the shape.
   • Scale: How stretched is it? (Is it a sphere? A pancake? A needle?)
   • Rotation (Quaternion): Which way is it pointing?
3. Opacity: How solid is it? (From 0% invisible to 100% solid). The math usually uses a Sigmoid function to keep this value safely between 0 and 1.
4. Color (Spherical Harmonics): Splats use Spherical Harmonics (SH). This allows the splat to change appearance based on the angle you view it from, capturing complex lighting effects like the glint on a wet surface or the way light scatters through leaves.
5. Volume Estimation: We realized that since we know the size of every splat, we can estimate volumes.
   • We use a Convex Hull algorithm to wrap a "shrink wrap" around a cluster of splats.
   • By filtering out the faint "noise" splats (using an opacity threshold) and keeping only the solid ones, we can calculate the volume of a specific organ or growth.
   • This allows for comparative metrics, like calculating the growth percentage of a structure between two scans.
     
   • NOTE: The calculation is in no way accurate right now, just a demo.

The Future

This tech is still in its infancy. Future medical applications could include:

• Direct DICOM Training: Training splats directly from raw medical data for sub-millimeter precision.
• 4D Visualization: Watching an organ function (move and change) in real-time.
• AI Segmentation: Automatically identifying which splats belong to the heart vs. the lungs.

Unfortunately a live demo isn’t available for this project as it runs on my local GPU