pytorch3d.implicitron.models.renderer.raymarcher
raymarcher
- class pytorch3d.implicitron.models.renderer.raymarcher.RaymarcherBase(*args, **kwargs)[source]
Bases:
ReplaceableBase
Defines a base class for raymarchers. Specifically, a raymarcher is responsible for taking a set of features and density descriptors along rendering rays and marching along them in order to generate a feature render.
- forward(rays_densities: Tensor, rays_features: Tensor, aux: Dict[str, Any]) RendererOutput [source]
- Parameters:
rays_densities – Per-ray density values represented with a tensor of shape (…, n_points_per_ray, 1).
rays_features – Per-ray feature values represented with a tensor of shape (…, n_points_per_ray, feature_dim).
aux – a dictionary with extra information.
- class pytorch3d.implicitron.models.renderer.raymarcher.AccumulativeRaymarcherBase(*args, **kwargs)[source]
Bases:
RaymarcherBase
,Module
This generalizes the pytorch3d.renderer.EmissionAbsorptionRaymarcher and NeuralVolumes’ cumsum ray marcher. It additionally returns the rendering weights that can be used in the NVS pipeline to carry out the importance ray-sampling in the refining pass. Different from pytorch3d.renderer.EmissionAbsorptionRaymarcher, it takes raw (non-exponentiated) densities.
- Parameters:
surface_thickness – The thickness of the raymarched surface.
bg_color – The background color. A tuple of either 1 element or of D elements, where D matches the feature dimensionality; it is broadcast when necessary.
replicate_last_interval – If True, the ray length assigned to the last interval for the opacity delta calculation is copied from the penultimate interval.
background_opacity – The length over which the last raw opacity value (i.e. before exponentiation) is considered to apply, for the delta calculation. Ignored if replicate_last_interval=True.
density_relu – If True, passes the input density through ReLU before raymarching.
blend_output – If True, alpha-blends the output renders with the background color using the rendered opacity mask.
capping_function –
The capping function of the raymarcher. Options:
”exponential” (cap_fn(x) = 1 - exp(-x))
”cap1” (cap_fn(x) = min(x, 1))
Set to “exponential” for the standard Emission Absorption raymarching.
weight_function –
The weighting function of the raymarcher. Options:
”product” (weight_fn(w, x) = w * x)
”minimum” (weight_fn(w, x) = min(w, x))
Set to “product” for the standard Emission Absorption raymarching.
- surface_thickness: int = 1
- bg_color: Tuple[float, ...] = (0.0,)
- replicate_last_interval: bool = False
- background_opacity: float = 0.0
- density_relu: bool = True
- blend_output: bool = False
- property capping_function_type: str
- property weight_function_type: str
- __post_init__()[source]
- Parameters:
surface_thickness – Denotes the overlap between the absorption function and the density function.
- forward(rays_densities: Tensor, rays_features: Tensor, aux: Dict[str, Any], ray_lengths: Tensor, ray_deltas: Tensor | None = None, density_noise_std: float = 0.0, **kwargs) RendererOutput [source]
- Parameters:
rays_densities – Per-ray density values represented with a tensor of shape (…, n_points_per_ray, 1).
rays_features – Per-ray feature values represented with a tensor of shape (…, n_points_per_ray, feature_dim).
aux – a dictionary with extra information.
ray_lengths – Per-ray depth values represented with a tensor of shape (…, n_points_per_ray, feature_dim).
ray_deltas – Optional differences between consecutive elements along the ray bundle represented with a tensor of shape (…, n_points_per_ray). If None, these differences are computed from ray_lengths.
density_noise_std – the magnitude of the noise added to densities.
- Returns:
features –
- A tensor of shape (…, feature_dim) containing
the rendered features for each ray.
depth: A tensor of shape (…, 1) containing estimated depth. opacities: A tensor of shape (…, 1) containing rendered opacities. weights: A tensor of shape (…, n_points_per_ray) containing
the ray-specific non-negative opacity weights. In general, they don’t sum to 1 but do not overcome it, i.e. (weights.sum(dim=-1) <= 1.0).all() holds.
- class pytorch3d.implicitron.models.renderer.raymarcher.EmissionAbsorptionRaymarcher(*args, **kwargs)[source]
Bases:
AccumulativeRaymarcherBase
Implements the EmissionAbsorption raymarcher.
- background_opacity: float = 10000000000.0
- property capping_function_type: str
- property weight_function_type: str
- class pytorch3d.implicitron.models.renderer.raymarcher.CumsumRaymarcher(*args, **kwargs)[source]
Bases:
AccumulativeRaymarcherBase
Implements the NeuralVolumes’ cumulative-sum raymarcher.
- property capping_function_type: str
- property weight_function_type: str