pytorch3d.implicitron.models.renderer.lstm_renderer

lstm_renderer

class pytorch3d.implicitron.models.renderer.lstm_renderer.LSTMRenderer(*args, **kwargs)[source]

Bases: BaseRenderer, Module

Implements the learnable LSTM raymarching function from SRN [1]. This requires there to be one implicit function, and it is expected to be like SRNImplicitFunction or SRNHyperNetImplicitFunction.

Settings:

num_raymarch_steps: The number of LSTM raymarching steps. init_depth: Initializes the bias of the last raymarching LSTM layer so that

the farthest point from the camera reaches a far z-plane that lies init_depth units from the camera plane.

init_depth_noise_std: The standard deviation of the random normal noise

added to the initial depth of each marched ray.

hidden_size: The dimensionality of the LSTM’s hidden state. n_feature_channels: The number of feature channels returned by the

implicit_function evaluated at each raymarching step.

bg_color: If supplied, used as the background color. Otherwise the pixel

generator is used everywhere. This has to have length either 1 (for a constant value for all output channels) or equal to the number of output channels (which is out_features on the pixel generator, typically 3.)

verbose: If True, logs raymarching debug info.

References

[1] Sitzmann, V. and Zollhöfer, M. and Wetzstein, G..

“Scene representation networks: Continuous 3d-structure-aware neural scene representations.” NeurIPS 2019.

num_raymarch_steps: int = 10
init_depth: float = 17.0
init_depth_noise_std: float = 0.0005
hidden_size: int = 16
n_feature_channels: int = 256
bg_color: List[float] | None = None
verbose: bool = False
forward(ray_bundle: ImplicitronRayBundle, implicit_functions: List[ImplicitFunctionWrapper], evaluation_mode: EvaluationMode = EvaluationMode.EVALUATION, **kwargs) → RendererOutput[source]
Parameters:

  • ray_bundle – A ImplicitronRayBundle object containing the parametrizations of the sampled rendering rays.

  • implicit_functions – A single-element list of ImplicitFunctionWrappers which defines the implicit function to be used.

  • evaluation_mode – one of EvaluationMode.TRAINING or EvaluationMode.EVALUATION which determines the settings used for rendering, specifically the RayPointRefiner and the density_noise_std.

Returns:

instance of RendererOutput