pytorch3d.renderer.implicit.raymarching
raymarching
- class pytorch3d.renderer.implicit.raymarching.EmissionAbsorptionRaymarcher(surface_thickness: int = 1)[source]
Bases:
Module
Raymarch using the Emission-Absorption (EA) algorithm.
The algorithm independently renders each ray by analyzing density and feature values sampled at (typically uniformly) spaced 3D locations along each ray. The density values rays_densities are of shape (…, n_points_per_ray), their values should range between [0, 1], and represent the opaqueness of each point (the higher the less transparent). The feature values rays_features of shape (…, n_points_per_ray, feature_dim) represent the content of the point that is supposed to be rendered in case the given point is opaque (i.e. its density -> 1.0).
EA first utilizes rays_densities to compute the absorption function along each ray as follows:
absorption = cumprod(1 - rays_densities, dim=-1)
The value of absorption at position absorption[…, k] specifies how much light has reached k-th point along a ray since starting its trajectory at k=0-th point.
Each ray is then rendered into a tensor features of shape (…, feature_dim) by taking a weighed combination of per-ray features rays_features as follows:
weights = absorption * rays_densities features = (rays_features * weights).sum(dim=-2)
Where weights denote a function that has a strong peak around the location of the first surface point that a given ray passes through.
Note that for a perfectly bounded volume (with a strictly binary density), the weights = cumprod(1 - rays_densities, dim=-1) * rays_densities function would yield 0 everywhere. In order to prevent this, the result of the cumulative product is shifted self.surface_thickness elements along the ray direction.
- __init__(surface_thickness: int = 1) None [source]
- Parameters:
surface_thickness – Denotes the overlap between the absorption function and the density function.
- forward(rays_densities: Tensor, rays_features: Tensor, eps: float = 1e-10, **kwargs) Tensor [source]
- Parameters:
rays_densities – Per-ray density values represented with a tensor of shape (…, n_points_per_ray, 1) whose values range in [0, 1].
rays_features – Per-ray feature values represented with a tensor of shape (…, n_points_per_ray, feature_dim).
eps – A lower bound added to rays_densities before computing the absorption function (cumprod of 1-rays_densities along each ray). This prevents the cumprod to yield exact 0 which would inhibit any gradient-based learning.
- Returns:
features_opacities –
- A tensor of shape (…, feature_dim+1)
- that concatenates two tensors along the last dimension:
- features: A tensor of per-ray renders
of shape (…, feature_dim).
- opacities: A tensor of per-ray opacity values
of shape (…, 1). Its values range between [0, 1] and denote the total amount of light that has been absorbed for each ray. E.g. a value of 0 corresponds to the ray completely passing through a volume. Please refer to the AbsorptionOnlyRaymarcher documentation for the explanation of the algorithm that computes opacities.
- class pytorch3d.renderer.implicit.raymarching.AbsorptionOnlyRaymarcher[source]
Bases:
Module
Raymarch using the Absorption-Only (AO) algorithm.
The algorithm independently renders each ray by analyzing density and feature values sampled at (typically uniformly) spaced 3D locations along each ray. The density values rays_densities are of shape (…, n_points_per_ray, 1), their values should range between [0, 1], and represent the opaqueness of each point (the higher the less transparent). The algorithm only measures the total amount of light absorbed along each ray and, besides outputting per-ray opacity values of shape (…,), does not produce any feature renderings.
The algorithm simply computes total_transmission = prod(1 - rays_densities) of shape (…, 1) which, for each ray, measures the total amount of light that passed through the volume. It then returns opacities = 1 - total_transmission.
- forward(rays_densities: Tensor, **kwargs) None | Tensor [source]
- Parameters:
rays_densities – Per-ray density values represented with a tensor of shape (…, n_points_per_ray) whose values range in [0, 1].
- Returns:
opacities –
- A tensor of per-ray opacity values of shape (…, 1).
Its values range between [0, 1] and denote the total amount of light that has been absorbed for each ray. E.g. a value of 0 corresponds to the ray completely passing through a volume.