pytorch3d.implicitron.models.metrics

metrics

class pytorch3d.implicitron.models.metrics.RegularizationMetricsBase(*args, **kwargs)[source]

Bases: ReplaceableBase, Module

Replaceable abstract base for regularization metrics. forward() method produces regularization metrics and (unlike ViewMetrics) can depend on the model’s parameters.

forward(model: Any, keys_prefix: str = 'loss_', **kwargs) Dict[str, Any][source]

Calculates various regularization terms useful for supervising differentiable rendering pipelines.

Parameters:
  • model – A model instance. Useful, for example, to implement weights-based regularization.

  • keys_prefix – A common prefix for all keys in the output dictionary containing all regularization metrics.

Returns:

A dictionary with the resulting regularization metrics. The items

will have form {metric_name_i: metric_value_i} keyed by the names of the output metrics metric_name_i with their corresponding values metric_value_i represented as 0-dimensional float tensors.

class pytorch3d.implicitron.models.metrics.ViewMetricsBase(*args, **kwargs)[source]

Bases: ReplaceableBase, Module

Replaceable abstract base for model metrics. forward() method produces losses and other metrics.

forward(raymarched: RendererOutput, ray_bundle: ImplicitronRayBundle, image_rgb: Tensor | None = None, depth_map: Tensor | None = None, fg_probability: Tensor | None = None, mask_crop: Tensor | None = None, keys_prefix: str = 'loss_', **kwargs) Dict[str, Any][source]

Calculates various metrics and loss functions useful for supervising differentiable rendering pipelines. Any additional parameters can be passed in the raymarched.aux dictionary.

Parameters:
  • results – A dictionary with the resulting view metrics. The items will have form {metric_name_i: metric_value_i} keyed by the names of the output metrics metric_name_i with their corresponding values metric_value_i represented as 0-dimensional float tensors.

  • raymarched – Output of the renderer.

  • ray_bundle – ImplicitronRayBundle object which was used to produce the raymarched object

  • image_rgb – A tensor of shape (B, H, W, 3) containing ground truth rgb values.

  • depth_map – A tensor of shape (B, Hd, Wd, 1) containing ground truth depth values.

  • fg_probability – A tensor of shape (B, Hm, Wm, 1) containing ground truth foreground masks.

  • keys_prefix – A common prefix for all keys in the output dictionary containing all view metrics.

Returns:

A dictionary with the resulting view metrics. The items

will have form {metric_name_i: metric_value_i} keyed by the names of the output metrics metric_name_i with their corresponding values metric_value_i represented as 0-dimensional float tensors.

class pytorch3d.implicitron.models.metrics.RegularizationMetrics(*args, **kwargs)[source]

Bases: RegularizationMetricsBase

forward(model: Any, keys_prefix: str = 'loss_', **kwargs) Dict[str, Any][source]

Calculates the AD penalty, or returns an empty dict if the model’s autoencoder is inactive.

Parameters:
  • model – A model instance.

  • keys_prefix – A common prefix for all keys in the output dictionary containing all regularization metrics.

Returns:

A dictionary with the resulting regularization metrics. The items

will have form {metric_name_i: metric_value_i} keyed by the names of the output metrics metric_name_i with their corresponding values metric_value_i represented as 0-dimensional float tensors.

The calculated metric is:

autoencoder_norm: Autoencoder weight norm regularization term.

class pytorch3d.implicitron.models.metrics.ViewMetrics(*args, **kwargs)[source]

Bases: ViewMetricsBase

forward(raymarched: RendererOutput, ray_bundle: ImplicitronRayBundle, image_rgb: Tensor | None = None, depth_map: Tensor | None = None, fg_probability: Tensor | None = None, mask_crop: Tensor | None = None, keys_prefix: str = 'loss_', **kwargs) Dict[str, Any][source]

Calculates various differentiable metrics useful for supervising differentiable rendering pipelines.

Parameters:
  • results – A dict to store the results in.

  • raymarched.features – Predicted rgb or feature values.

  • raymarched.depths – A tensor of shape (B, …, 1) containing predicted depth values.

  • raymarched.masks – A tensor of shape (B, …, 1) containing predicted foreground masks.

  • raymarched.aux["grad_theta"] – A tensor of shape (B, …, 3) containing an evaluation of a gradient of a signed distance function w.r.t. input 3D coordinates used to compute the eikonal loss.

  • raymarched.aux["density_grid"] – A tensor of shape (B, Hg, Wg, Dg, 1) containing a Hg x Wg x Dg voxel grid of density values.

  • ray_bundle – ImplicitronRayBundle object which was used to produce the raymarched object

  • image_rgb – A tensor of shape (B, H, W, 3) containing ground truth rgb values.

  • depth_map – A tensor of shape (B, Hd, Wd, 1) containing ground truth depth values.

  • fg_probability – A tensor of shape (B, Hm, Wm, 1) containing ground truth foreground masks.

  • keys_prefix – A common prefix for all keys in the output dictionary containing all view metrics.

Returns:

A dictionary `{metric_name_i –

metric_value_i}` keyed by the

names of the output metrics metric_name_i with their corresponding values metric_value_i represented as 0-dimensional float tensors.

The calculated metrics are:

rgb_huber: A robust huber loss between image_pred and image. rgb_mse: Mean squared error between image_pred and image. rgb_psnr: Peak signal-to-noise ratio between image_pred and image. rgb_psnr_fg: Peak signal-to-noise ratio between the foreground

region of image_pred and image as defined by mask.

rgb_mse_fg: Mean squared error between the foreground

region of image_pred and image as defined by mask.

mask_neg_iou: (1 - intersection-over-union) between mask_pred

and mask.

mask_bce: Binary cross entropy between mask_pred and mask. mask_beta_prior: A loss enforcing strictly binary values

of mask_pred: log(mask_pred) + log(1-mask_pred)

depth_abs: Mean per-pixel L1 distance between

depth_pred and depth.

depth_abs_fg: Mean per-pixel L1 distance between the foreground

region of depth_pred and depth as defined by mask.

eikonal: Eikonal regularizer (||grad_theta|| - 1)**2. density_tv: The Total Variation regularizer of density

values in density_grid (sum of L1 distances of values of all 4-neighbouring cells).

depth_neg_penalty: min(depth_pred, 0)**2 penalizing negative

predicted depth values.