Abstract
Physical simulation is paramount for modeling and utilization of 3D scenes in various real-world applications. However, its integration with state-of-the-art 3D scene rendering techniques such as Gaussian Splatting (GS) remains challenging. Existing models use additional meshing mechanisms, including triangle or tetrahedron meshing, marching cubes, or cage meshes. As an alternative, we can modify the physically grounded Newtonian dynamics to align with 3D Gaussian components. Current models take the first-order approximation of a deformation map, which locally approximates the dynamics by linear transformations. In contrast, our Gaussian Splatting for Physics-Based Simulations (GASP) model uses such a map (without any modifications) and flat Gaussian distributions, which are parameterized by three points (mesh faces). Subsequently, each 3D point (mesh face node) is treated as a discrete entity within a 3D space. Consequently, the problem of modeling Gaussian components is reduced to working with 3D points. Additionally, the information on mesh faces can be used to incorporate further properties into the physical model, facilitating the use of triangles. Resulting solution can be integrated into any physical engine that can be treated as a black box. As demonstrated in our studies, the proposed model exhibits superior performance on a diverse range of benchmark datasets designed for 3D object rendering.
GASP produces physical simulations in flat GS. It works with static GS models like GaMeS or dynamic ones like D-MiSo. (In general, GASP can be applied to any GS-based model which uses flat Gaussians.) First, we take the flat Gaussianand convert it to the GaMeS format. Each flat Gaussian component is converted to three points (triangle face). Then, the physical engineis run on a point cloud to obtain trajectories for our 3D model. Finally, we convert points into Gaussian components by reverse GaMeS parametrization
GASP model can work with dynamic scenes. In the example w model dynamic 3D scenes by D-MiSo and physical properties to the selected frame of dynamic scenes.
GASP integrates GS with a physical engine to gener-ate realistic simulations.
Essentially, Our model utilizes set of triangle soup to parameterize Gaussian components. This approach reduces modifying Gaussian components to 3D point clouds (consisting of vertices of triangles) processing, enabling efficient and rapid rendering
Results obtained by GASP on scene bonsai and fox. We can add extra element like falling ducks.
More examples
Model work directly on points extracted from flat Gaussians. In practice, physics moves all particles independently, which can produce artifacts. Therefore, we control the distance of points from each other. If some points are moved extremely far, we automatically correct such behaviors by clipping scaling parameters in GS.