pytorch3d.implicitron.models.renderer.raymarcher

raymarcher

class pytorch3d.implicitron.models.renderer.raymarcher.RaymarcherBase(*args, **kwargs)

Bases: ReplaceableBase

Defines a base class for raymarchers. Specifically, a raymarcher is responsible for taking a set of features and density descriptors along rendering rays and marching along them in order to generate a feature render.

forward(rays_densities: Tensor, rays_features: Tensor, aux: Dict[str, Any]) → RendererOutput

Parameters:

• rays_densities – Per-ray density values represented with a tensor of shape (…, n_points_per_ray, 1).

• rays_features – Per-ray feature values represented with a tensor of shape (…, n_points_per_ray, feature_dim).

• aux – a dictionary with extra information.

class pytorch3d.implicitron.models.renderer.raymarcher.AccumulativeRaymarcherBase(*args, **kwargs)

Bases: RaymarcherBase, Module

This generalizes the pytorch3d.renderer.EmissionAbsorptionRaymarcher and NeuralVolumes’ cumsum ray marcher. It additionally returns the rendering weights that can be used in the NVS pipeline to carry out the importance ray-sampling in the refining pass. Different from pytorch3d.renderer.EmissionAbsorptionRaymarcher, it takes raw (non-exponentiated) densities.

Parameters:

• surface_thickness – The thickness of the raymarched surface.

• bg_color – The background color. A tuple of either 1 element or of D elements, where D matches the feature dimensionality; it is broadcast when necessary.

• replicate_last_interval – If True, the ray length assigned to the last interval for the opacity delta calculation is copied from the penultimate interval.

• background_opacity – The length over which the last raw opacity value (i.e. before exponentiation) is considered to apply, for the delta calculation. Ignored if replicate_last_interval=True.

• density_relu – If True, passes the input density through ReLU before raymarching.

• blend_output – If True, alpha-blends the output renders with the background color using the rendered opacity mask.

• capping_function –

  The capping function of the raymarcher. Options:

  • ”exponential” (cap_fn(x) = 1 - exp(-x))

  • ”cap1” (cap_fn(x) = min(x, 1))

  Set to “exponential” for the standard Emission Absorption raymarching.

• weight_function –

  The weighting function of the raymarcher. Options:

  • ”product” (weight_fn(w, x) = w * x)

  • ”minimum” (weight_fn(w, x) = min(w, x))

  Set to “product” for the standard Emission Absorption raymarching.

surface_thickness: int = 1

bg_color: Tuple[float, ...] = (0.0,)

replicate_last_interval: bool = False

background_opacity: float = 0.0

density_relu: bool = True

blend_output: bool = False

property capping_function_type: str

property weight_function_type: str

__post_init__()

Parameters:

surface_thickness – Denotes the overlap between the absorption function and the density function.

forward(rays_densities: Tensor, rays_features: Tensor, aux: Dict[str, Any], ray_lengths: Tensor, ray_deltas: Tensor | None = None, density_noise_std: float = 0.0, **kwargs) → RendererOutput

Parameters:

• rays_densities – Per-ray density values represented with a tensor of shape (…, n_points_per_ray, 1).

• rays_features – Per-ray feature values represented with a tensor of shape (…, n_points_per_ray, feature_dim).

• aux – a dictionary with extra information.

• ray_lengths – Per-ray depth values represented with a tensor of shape (…, n_points_per_ray, feature_dim).

• ray_deltas – Optional differences between consecutive elements along the ray bundle represented with a tensor of shape (…, n_points_per_ray). If None, these differences are computed from ray_lengths.

• density_noise_std – the magnitude of the noise added to densities.

Returns:

features –

A tensor of shape (…, feature_dim) containing

the rendered features for each ray.

depth: A tensor of shape (…, 1) containing estimated depth. opacities: A tensor of shape (…, 1) containing rendered opacities. weights: A tensor of shape (…, n_points_per_ray) containing

the ray-specific non-negative opacity weights. In general, they don’t sum to 1 but do not overcome it, i.e. (weights.sum(dim=-1) <= 1.0).all() holds.

class pytorch3d.implicitron.models.renderer.raymarcher.EmissionAbsorptionRaymarcher(*args, **kwargs)

Bases: AccumulativeRaymarcherBase

Implements the EmissionAbsorption raymarcher.

background_opacity: float = 10000000000.0

property capping_function_type: str

property weight_function_type: str

class pytorch3d.implicitron.models.renderer.raymarcher.CumsumRaymarcher(*args, **kwargs)

Bases: AccumulativeRaymarcherBase

Implements the NeuralVolumes’ cumulative-sum raymarcher.

property capping_function_type: str

property weight_function_type: str