Painting with 3D Gaussian Splat Brushes

Karran Pandey1, Anita Hu2, Clement Fuji-Tsang2, Or Perel2, Karan Singh1, Masha Shugrina2
1University of Toronto, 2NVIDIA
SIGGRAPH 2025
PDF Video Code (Coming Soon)
Painting with 3D Gaussian splat brushes.


We propose the first tool for painting in real-time with real-world texture-geometry content, represented as 3D Gaussian splats. It allows artists to source both their brushes and canvases from in-the-wild captures, and swiftly remix highly realistic content through direct and controllable painting.

Abstract

We explore interactive painting on 3D Gaussian splat scenes and other surfaces using 3D Gaussian splat brushes, each containing a chunk of realistic texture-geometry that make capture representations so appealing.

The suite of brush capabilities we propose enables 3D artists to capture and remix real world imagery and geometry with direct interactive control. In particular, we propose a set of algorithms for:

  • selecting subsets of Gaussians as a brush pattern interactively,
  • applying the brush interactively to the same or other 3DGS scenes or other 3D surfaces using stamp-based painting,
  • using an inpainting Diffusion Model to adjust stamp seams for seamless and realistic appearance
  • an ensemble of artistic brush parameters, resulting in a wide range of appearance options for the same brush.

Our contribution is a combination of algorithms, design features and creative affordances, that together enable the first implementation of interactive brush-based painting with 3D Gaussian splats.

Gallery



Result Gallery.

Painting Technique

Method overview.

The steps of our method, described below, compute seamless strokes on surfaces as users paint with oriented 3DGS stamps:
  • A 3D spline is computed from the collection of first-hits on the surface gathered as a user paints.
  • Stamps are then rigidly placed at regular intervals along the spline.
  • Each stamp is then non-rigidly deformed to locally adapt its shape to the underlying spline in 3D.
  • Finally, a background process repairs seams between stamps using diffusion inpainting.

BibTeX

@article{pandey2025painting,
  author    = {Pandey, Karran and Hu, Anita and Fuji-Tsang, Clement and Perel, Or and Singh, Karan and Shugrina, Maria},
  title     = {Painting with 3D Gaussian Splat Brushes},
  booktitle   = {ACM SIGGRAPH 2025 Conference Papers},
  pages = {}
  year      = {2025},
}