pytorch3d.implicitron.models.renderer.raymarcher

raymarcher

class pytorch3d.implicitron.models.renderer.raymarcher.RaymarcherBase(*args, **kwargs)[source]

Bases: ReplaceableBase

Defines a base class for raymarchers. Specifically, a raymarcher is responsible for taking a set of features and density descriptors along rendering rays and marching along them in order to generate a feature render.

forward(rays_densities: Tensor, rays_features: Tensor, aux: Dict[str, Any]) RendererOutput[source]
Parameters:
  • rays_densities – Per-ray density values represented with a tensor of shape (…, n_points_per_ray, 1).

  • rays_features – Per-ray feature values represented with a tensor of shape (…, n_points_per_ray, feature_dim).

  • aux – a dictionary with extra information.

class pytorch3d.implicitron.models.renderer.raymarcher.AccumulativeRaymarcherBase(*args, **kwargs)[source]

Bases: RaymarcherBase, Module

This generalizes the pytorch3d.renderer.EmissionAbsorptionRaymarcher and NeuralVolumes’ cumsum ray marcher. It additionally returns the rendering weights that can be used in the NVS pipeline to carry out the importance ray-sampling in the refining pass. Different from pytorch3d.renderer.EmissionAbsorptionRaymarcher, it takes raw (non-exponentiated) densities.

Parameters:
  • surface_thickness – The thickness of the raymarched surface.

  • bg_color – The background color. A tuple of either 1 element or of D elements, where D matches the feature dimensionality; it is broadcast when necessary.

  • replicate_last_interval – If True, the ray length assigned to the last interval for the opacity delta calculation is copied from the penultimate interval.

  • background_opacity – The length over which the last raw opacity value (i.e. before exponentiation) is considered to apply, for the delta calculation. Ignored if replicate_last_interval=True.

  • density_relu – If True, passes the input density through ReLU before raymarching.

  • blend_output – If True, alpha-blends the output renders with the background color using the rendered opacity mask.

  • capping_function

    The capping function of the raymarcher. Options:

    • ”exponential” (cap_fn(x) = 1 - exp(-x))

    • ”cap1” (cap_fn(x) = min(x, 1))

    Set to “exponential” for the standard Emission Absorption raymarching.

  • weight_function

    The weighting function of the raymarcher. Options:

    • ”product” (weight_fn(w, x) = w * x)

    • ”minimum” (weight_fn(w, x) = min(w, x))

    Set to “product” for the standard Emission Absorption raymarching.

surface_thickness: int = 1
bg_color: Tuple[float, ...] = (0.0,)
replicate_last_interval: bool = False
background_opacity: float = 0.0
density_relu: bool = True
blend_output: bool = False
property capping_function_type: str
property weight_function_type: str
__post_init__()[source]
Parameters:

surface_thickness – Denotes the overlap between the absorption function and the density function.

forward(rays_densities: Tensor, rays_features: Tensor, aux: Dict[str, Any], ray_lengths: Tensor, ray_deltas: Tensor | None = None, density_noise_std: float = 0.0, **kwargs) RendererOutput[source]
Parameters:
  • rays_densities – Per-ray density values represented with a tensor of shape (…, n_points_per_ray, 1).

  • rays_features – Per-ray feature values represented with a tensor of shape (…, n_points_per_ray, feature_dim).

  • aux – a dictionary with extra information.

  • ray_lengths – Per-ray depth values represented with a tensor of shape (…, n_points_per_ray, feature_dim).

  • ray_deltas – Optional differences between consecutive elements along the ray bundle represented with a tensor of shape (…, n_points_per_ray). If None, these differences are computed from ray_lengths.

  • density_noise_std – the magnitude of the noise added to densities.

Returns:

features

A tensor of shape (…, feature_dim) containing

the rendered features for each ray.

depth: A tensor of shape (…, 1) containing estimated depth. opacities: A tensor of shape (…, 1) containing rendered opacities. weights: A tensor of shape (…, n_points_per_ray) containing

the ray-specific non-negative opacity weights. In general, they don’t sum to 1 but do not overcome it, i.e. (weights.sum(dim=-1) <= 1.0).all() holds.

class pytorch3d.implicitron.models.renderer.raymarcher.EmissionAbsorptionRaymarcher(*args, **kwargs)[source]

Bases: AccumulativeRaymarcherBase

Implements the EmissionAbsorption raymarcher.

background_opacity: float = 10000000000.0
property capping_function_type: str
property weight_function_type: str
class pytorch3d.implicitron.models.renderer.raymarcher.CumsumRaymarcher(*args, **kwargs)[source]

Bases: AccumulativeRaymarcherBase

Implements the NeuralVolumes’ cumulative-sum raymarcher.

property capping_function_type: str
property weight_function_type: str