pytorch3d.implicitron.models.implicit_function.neural_radiance_field

neural_radiance_field

class pytorch3d.implicitron.models.implicit_function.neural_radiance_field.NeuralRadianceFieldBase(*args, **kwargs)

Bases: ImplicitFunctionBase, Module

n_harmonic_functions_xyz: int = 10

n_harmonic_functions_dir: int = 4

n_hidden_neurons_dir: int = 128

latent_dim: int = 0

input_xyz: bool = True

xyz_ray_dir_in_camera_coords: bool = False

color_dim: int = 3

use_integrated_positional_encoding: bool = False

Args: n_harmonic_functions_xyz: The number of harmonic functions

used to form the harmonic embedding of 3D point locations.

n_harmonic_functions_dir: The number of harmonic functions

used to form the harmonic embedding of the ray directions.

n_hidden_neurons_xyz: The number of hidden units in the

fully connected layers of the MLP that accepts the 3D point locations and outputs the occupancy field with the intermediate features.

n_hidden_neurons_dir: The number of hidden units in the

fully connected layers of the MLP that accepts the intermediate features and ray directions and outputs the radiance field (per-point colors).

n_layers_xyz: The number of layers of the MLP that outputs the

occupancy field.

append_xyz: The list of indices of the skip layers of the occupancy MLP. use_integrated_positional_encoding: If True, use integrated positional enoding

as defined in MIP-NeRF. If False, use the classical harmonic embedding defined in NeRF.

get_xyz_embedding_dim()

static allows_multiple_passes() → bool

Returns True as this implicit function allows multiple passes. Overridden from ImplicitFunctionBase.

forward(*, ray_bundle: ImplicitronRayBundle, fun_viewpool=None, camera: CamerasBase | None = None, global_code=None, **kwargs)

The forward function accepts the parametrizations of 3D points sampled along projection rays. The forward pass is responsible for attaching a 3D vector and a 1D scalar representing the point’s RGB color and opacity respectively.

Parameters:

• ray_bundle –

  An ImplicitronRayBundle object containing the following variables: origins: A tensor of shape (minibatch, …, 3) denoting the

  origins of the sampling rays in world coords.

  directions: A tensor of shape (minibatch, …, 3)

  containing the direction vectors of sampling rays in world coords.

  lengths: A tensor of shape (minibatch, …, num_points_per_ray)

  containing the lengths at which the rays are sampled.

  bins: An optional tensor of shape (minibatch,…, num_points_per_ray + 1)

  containing the bins at which the rays are sampled. In this case lengths is equal to the midpoints of bins.

• fun_viewpool –

  an optional callback with the signature

  fun_fiewpool(points) -> pooled_features

  where points is a [N_TGT x N x 3] tensor of world coords, and pooled_features is a [N_TGT x … x N_SRC x latent_dim] tensor of the features pooled from the context images.

Returns:

rays_densities –

A tensor of shape (minibatch, …, num_points_per_ray, 1)

denoting the opacitiy of each ray point.

rays_colors: A tensor of shape (minibatch, …, num_points_per_ray, 3)

denoting the color of each ray point.

Raises:

ValueError – If use_integrated_positional_encoding is True and ray_bundle.bins is None.

class pytorch3d.implicitron.models.implicit_function.neural_radiance_field.NeuralRadianceFieldImplicitFunction(*args, **kwargs)

Bases: NeuralRadianceFieldBase

transformer_dim_down_factor: float = 1.0

n_hidden_neurons_xyz: int = 256

n_layers_xyz: int = 8

append_xyz: Tuple[int, ...] = (5,)

class pytorch3d.implicitron.models.implicit_function.neural_radiance_field.NeRFormerImplicitFunction(*args, **kwargs)

Bases: NeuralRadianceFieldBase

transformer_dim_down_factor: float = 2.0

n_hidden_neurons_xyz: int = 80

n_layers_xyz: int = 2

append_xyz: Tuple[int, ...] = (1,)

static requires_pooling_without_aggregation() → bool

Returns True as this implicit function needs pooling without aggregation. Overridden from ImplicitFunctionBase.