pytorch3d.renderer.implicit.raymarching

raymarching

class pytorch3d.renderer.implicit.raymarching.EmissionAbsorptionRaymarcher(surface_thickness: int = 1)

Bases: Module

Raymarch using the Emission-Absorption (EA) algorithm.

The algorithm independently renders each ray by analyzing density and feature values sampled at (typically uniformly) spaced 3D locations along each ray. The density values rays_densities are of shape (…, n_points_per_ray), their values should range between [0, 1], and represent the opaqueness of each point (the higher the less transparent). The feature values rays_features of shape (…, n_points_per_ray, feature_dim) represent the content of the point that is supposed to be rendered in case the given point is opaque (i.e. its density -> 1.0).

EA first utilizes rays_densities to compute the absorption function along each ray as follows:

absorption = cumprod(1 - rays_densities, dim=-1)

The value of absorption at position absorption[…, k] specifies how much light has reached k-th point along a ray since starting its trajectory at k=0-th point.

Each ray is then rendered into a tensor features of shape (…, feature_dim) by taking a weighed combination of per-ray features rays_features as follows:

weights = absorption * rays_densities
features = (rays_features * weights).sum(dim=-2)

Where weights denote a function that has a strong peak around the location of the first surface point that a given ray passes through.

Note that for a perfectly bounded volume (with a strictly binary density), the weights = cumprod(1 - rays_densities, dim=-1) * rays_densities function would yield 0 everywhere. In order to prevent this, the result of the cumulative product is shifted self.surface_thickness elements along the ray direction.

__init__(surface_thickness: int = 1) → None

Parameters:

surface_thickness – Denotes the overlap between the absorption function and the density function.

forward(rays_densities: Tensor, rays_features: Tensor, eps: float = 1e-10, **kwargs) → Tensor

Parameters:

• rays_densities – Per-ray density values represented with a tensor of shape (…, n_points_per_ray, 1) whose values range in [0, 1].

• rays_features – Per-ray feature values represented with a tensor of shape (…, n_points_per_ray, feature_dim).

• eps – A lower bound added to rays_densities before computing the absorption function (cumprod of 1-rays_densities along each ray). This prevents the cumprod to yield exact 0 which would inhibit any gradient-based learning.

Returns:

features_opacities –

A tensor of shape (…, feature_dim+1)

that concatenates two tensors along the last dimension:

1. features: A tensor of per-ray renders

   of shape (…, feature_dim).

2. opacities: A tensor of per-ray opacity values

   of shape (…, 1). Its values range between [0, 1] and denote the total amount of light that has been absorbed for each ray. E.g. a value of 0 corresponds to the ray completely passing through a volume. Please refer to the AbsorptionOnlyRaymarcher documentation for the explanation of the algorithm that computes opacities.

class pytorch3d.renderer.implicit.raymarching.AbsorptionOnlyRaymarcher

Bases: Module

Raymarch using the Absorption-Only (AO) algorithm.

The algorithm independently renders each ray by analyzing density and feature values sampled at (typically uniformly) spaced 3D locations along each ray. The density values rays_densities are of shape (…, n_points_per_ray, 1), their values should range between [0, 1], and represent the opaqueness of each point (the higher the less transparent). The algorithm only measures the total amount of light absorbed along each ray and, besides outputting per-ray opacity values of shape (…,), does not produce any feature renderings.

The algorithm simply computes total_transmission = prod(1 - rays_densities) of shape (…, 1) which, for each ray, measures the total amount of light that passed through the volume. It then returns opacities = 1 - total_transmission.

forward(rays_densities: Tensor, **kwargs) → None | Tensor

Parameters:

rays_densities – Per-ray density values represented with a tensor of shape (…, n_points_per_ray) whose values range in [0, 1].

Returns:

opacities –

A tensor of per-ray opacity values of shape (…, 1).

Its values range between [0, 1] and denote the total amount of light that has been absorbed for each ray. E.g. a value of 0 corresponds to the ray completely passing through a volume.