pytorch3d.renderer.splatter_blend

splatter_blend

class pytorch3d.renderer.splatter_blend.SplatterBlender(input_shape: Tuple[int, int, int, int], device)[source]

Bases: Module

__init__(input_shape: Tuple[int, int, int, int], device)[source]

A splatting blender. See forward docs for details of the splatting mechanism.

Parameters:

input_shape – Tuple (N, H, W, K) indicating the batch size, image height, image width, and number of rasterized layers. Used to precompute constant tensors that do not change as long as this tuple is unchanged.

to(device)[source]
forward(colors: Tensor, pixel_coords_cameras: Tensor, cameras: FoVPerspectiveCameras, background_mask: Tensor, blend_params: BlendParams) → Tensor[source]

RGB blending using splatting, as proposed in [0].

Parameters:

  • colors – (N, H, W, K, 3) tensor of RGB colors at each h, w pixel location for K intersection layers.

  • pixel_coords_cameras – (N, H, W, K, 3) tensor of pixel coordinates in the camera frame of reference. It is crucial that these are computed by interpolating triangle vertex positions using barycentric coordinates – this allows gradients to travel through pixel_coords_camera back to the vertex positions.

  • cameras – Cameras object used to project pixel_coords_cameras screen coords.

  • background_mask – (N, H, W, K, 3) boolean tensor, True for bg pixels. A pixel is considered “background” if no mesh triangle projects to it. This is typically computed by the rasterizer.

  • blend_params – BlendParams, from which we use sigma (splatting kernel variance) and background_color.

Returns:

output_colors –

(N, H, W, 4) tensor of RGBA values. The alpha layer is set to

fully transparent in the background.

[0] Cole, F. et al., “Differentiable Surface Rendering via Non-differentiable

Sampling”.