pytorch3d.implicitron.models.implicit_function.scene_representation_networks

scene_representation_networks

class pytorch3d.implicitron.models.implicit_function.scene_representation_networks.SRNRaymarchFunction(*args, **kwargs)

Bases: Configurable, Module

n_harmonic_functions: int = 3

n_hidden_units: int = 256

n_layers: int = 2

in_features: int = 3

out_features: int = 256

latent_dim: int = 0

xyz_in_camera_coords: bool = False

raymarch_function: Any = None

forward(ray_bundle: ImplicitronRayBundle, fun_viewpool=None, camera: CamerasBase | None = None, global_code=None, **kwargs)

Parameters:

• ray_bundle –

  An ImplicitronRayBundle object containing the following variables: origins: A tensor of shape (minibatch, …, 3) denoting the

  origins of the sampling rays in world coords.

  directions: A tensor of shape (minibatch, …, 3)

  containing the direction vectors of sampling rays in world coords.

  lengths: A tensor of shape (minibatch, …, num_points_per_ray)

  containing the lengths at which the rays are sampled.

• fun_viewpool –

  an optional callback with the signature

  fun_fiewpool(points) -> pooled_features

  where points is a [N_TGT x N x 3] tensor of world coords, and pooled_features is a [N_TGT x … x N_SRC x latent_dim] tensor of the features pooled from the context images.

Returns:

rays_densities –

A tensor of shape (minibatch, …, num_points_per_ray, 1)

denoting the opacitiy of each ray point.

rays_colors: Set to None.

class pytorch3d.implicitron.models.implicit_function.scene_representation_networks.SRNPixelGenerator(*args, **kwargs)

Bases: Configurable, Module

n_harmonic_functions: int = 4

n_hidden_units: int = 256

n_hidden_units_color: int = 128

n_layers: int = 2

in_features: int = 256

out_features: int = 3

ray_dir_in_camera_coords: bool = False

forward(raymarch_features: Tensor, ray_bundle: ImplicitronRayBundle, camera: CamerasBase | None = None, **kwargs)

Parameters:

• raymarch_features – Features from the raymarching network of shape (minibatch, …, self.in_features)

• ray_bundle –

  An ImplicitronRayBundle object containing the following variables: origins: A tensor of shape (minibatch, …, 3) denoting the

  origins of the sampling rays in world coords.

  directions: A tensor of shape (minibatch, …, 3)

  containing the direction vectors of sampling rays in world coords.

  lengths: A tensor of shape (minibatch, …, num_points_per_ray)

  containing the lengths at which the rays are sampled.

Returns:

rays_densities –

A tensor of shape (minibatch, …, num_points_per_ray, 1)

denoting the opacitiy of each ray point.

rays_colors: A tensor of shape (minibatch, …, num_points_per_ray, 3)

denoting the color of each ray point.

class pytorch3d.implicitron.models.implicit_function.scene_representation_networks.SRNRaymarchHyperNet(*args, **kwargs)

Bases: Configurable, Module

This is a raymarching function which has a forward like SRNRaymarchFunction but instead of the weights being parameters of the module, they are the output of another network, the hypernet, which takes the global_code as input. All the dataclass members of SRNRaymarchFunction are here with the same meaning. In addition, there are members with names ending _hypernet which affect the hypernet.

Because this class may be called repeatedly for the same global_code, the output of the hypernet is cached in self.cached_srn_raymarch_function. This member must be manually set to None whenever the global_code changes.

n_harmonic_functions: int = 3

n_hidden_units: int = 256

n_layers: int = 2

n_hidden_units_hypernet: int = 256

n_layers_hypernet: int = 1

in_features: int = 3

out_features: int = 256

latent_dim_hypernet: int = 0

latent_dim: int = 0

xyz_in_camera_coords: bool = False

forward(ray_bundle: ImplicitronRayBundle, fun_viewpool=None, camera: CamerasBase | None = None, global_code=None, **kwargs)

class pytorch3d.implicitron.models.implicit_function.scene_representation_networks.SRNImplicitFunction(*args, **kwargs)

Bases: ImplicitFunctionBase, Module

latent_dim: int = 0

raymarch_function: SRNRaymarchFunction

pixel_generator: SRNPixelGenerator

create_raymarch_function() → None

classmethod raymarch_function_tweak_args(type, args: DictConfig) → None

forward(*, ray_bundle: ImplicitronRayBundle, fun_viewpool=None, camera: CamerasBase | None = None, global_code=None, raymarch_features: Tensor | None = None, **kwargs)

class pytorch3d.implicitron.models.implicit_function.scene_representation_networks.SRNHyperNetImplicitFunction(*args, **kwargs)

Bases: ImplicitFunctionBase, Module

This implicit function uses a hypernetwork to generate the SRNRaymarchingFunction, and this is cached. Whenever the global_code changes, on_bind_args must be called to clear the cache.

latent_dim_hypernet: int = 0

latent_dim: int = 0

hypernet: SRNRaymarchHyperNet

pixel_generator: SRNPixelGenerator

create_hypernet() → None

classmethod hypernet_tweak_args(type, args: DictConfig) → None

forward(*, ray_bundle: ImplicitronRayBundle, fun_viewpool=None, camera: CamerasBase | None = None, global_code=None, raymarch_features: Tensor | None = None, **kwargs)

on_bind_args()

The global_code may have changed, so we reset the hypernet.