pytorch3d.implicitron.models.base_model

base_model

class pytorch3d.implicitron.models.base_model.ImplicitronRender(depth_render: Tensor | None = None, image_render: Tensor | None = None, mask_render: Tensor | None = None, camera_distance: Tensor | None = None)[source]

Bases: object

Holds the tensors that describe a result of rendering.

depth_render: Tensor | None = None
image_render: Tensor | None = None
mask_render: Tensor | None = None
camera_distance: Tensor | None = None
clone() ImplicitronRender[source]
class pytorch3d.implicitron.models.base_model.ImplicitronModelBase(*args, **kwargs)[source]

Bases: ReplaceableBase, Module

Replaceable abstract base for all image generation / rendering models. forward() method produces a render with a depth map. Derives from Module so we can rely on basic functionality provided to torch for model optimization.

log_vars: List[str] = Field(name=None,type=None,default=<dataclasses._MISSING_TYPE object>,default_factory=<function ImplicitronModelBase.<lambda>>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=<dataclasses._MISSING_TYPE object>,_field_type=None)
forward(*, image_rgb: Tensor | None, camera: CamerasBase, fg_probability: Tensor | None, mask_crop: Tensor | None, depth_map: Tensor | None, sequence_name: List[str] | None, evaluation_mode: EvaluationMode = EvaluationMode.EVALUATION, **kwargs) Dict[str, Any][source]
Parameters:
  • image_rgb – A tensor of shape (B, 3, H, W) containing a batch of rgb images; the first min(B, n_train_target_views) images are considered targets and are used to supervise the renders; the rest corresponding to the source viewpoints from which features will be extracted.

  • camera – An instance of CamerasBase containing a batch of B cameras corresponding to the viewpoints of target images, from which the rays will be sampled, and source images, which will be used for intersecting with target rays.

  • fg_probability – A tensor of shape (B, 1, H, W) containing a batch of foreground masks.

  • mask_crop – A binary tensor of shape (B, 1, H, W) deonting valid regions in the input images (i.e. regions that do not correspond to, e.g., zero-padding). When the RaySampler’s sampling mode is set to “mask_sample”, rays will be sampled in the non zero regions.

  • depth_map – A tensor of shape (B, 1, H, W) containing a batch of depth maps.

  • sequence_name – A list of B strings corresponding to the sequence names from which images image_rgb were extracted. They are used to match target frames with relevant source frames.

  • evaluation_mode – one of EvaluationMode.TRAINING or EvaluationMode.EVALUATION which determines the settings used for rendering.

Returns:

preds

A dictionary containing all outputs of the forward pass. All models should

output an instance of ImplicitronRender in preds[“implicitron_render”].