pytorch3d.implicitron.models.metrics
metrics
- class pytorch3d.implicitron.models.metrics.RegularizationMetricsBase(*args, **kwargs)[source]
Bases:
ReplaceableBase
,Module
Replaceable abstract base for regularization metrics. forward() method produces regularization metrics and (unlike ViewMetrics) can depend on the model’s parameters.
- forward(model: Any, keys_prefix: str = 'loss_', **kwargs) Dict[str, Any] [source]
Calculates various regularization terms useful for supervising differentiable rendering pipelines.
- Parameters:
model – A model instance. Useful, for example, to implement weights-based regularization.
keys_prefix – A common prefix for all keys in the output dictionary containing all regularization metrics.
- Returns:
- A dictionary with the resulting regularization metrics. The items
will have form {metric_name_i: metric_value_i} keyed by the names of the output metrics metric_name_i with their corresponding values metric_value_i represented as 0-dimensional float tensors.
- class pytorch3d.implicitron.models.metrics.ViewMetricsBase(*args, **kwargs)[source]
Bases:
ReplaceableBase
,Module
Replaceable abstract base for model metrics. forward() method produces losses and other metrics.
- forward(raymarched: RendererOutput, ray_bundle: ImplicitronRayBundle, image_rgb: Tensor | None = None, depth_map: Tensor | None = None, fg_probability: Tensor | None = None, mask_crop: Tensor | None = None, keys_prefix: str = 'loss_', **kwargs) Dict[str, Any] [source]
Calculates various metrics and loss functions useful for supervising differentiable rendering pipelines. Any additional parameters can be passed in the raymarched.aux dictionary.
- Parameters:
results – A dictionary with the resulting view metrics. The items will have form {metric_name_i: metric_value_i} keyed by the names of the output metrics metric_name_i with their corresponding values metric_value_i represented as 0-dimensional float tensors.
raymarched – Output of the renderer.
ray_bundle – ImplicitronRayBundle object which was used to produce the raymarched object
image_rgb – A tensor of shape (B, H, W, 3) containing ground truth rgb values.
depth_map – A tensor of shape (B, Hd, Wd, 1) containing ground truth depth values.
fg_probability – A tensor of shape (B, Hm, Wm, 1) containing ground truth foreground masks.
keys_prefix – A common prefix for all keys in the output dictionary containing all view metrics.
- Returns:
- A dictionary with the resulting view metrics. The items
will have form {metric_name_i: metric_value_i} keyed by the names of the output metrics metric_name_i with their corresponding values metric_value_i represented as 0-dimensional float tensors.
- class pytorch3d.implicitron.models.metrics.RegularizationMetrics(*args, **kwargs)[source]
Bases:
RegularizationMetricsBase
- forward(model: Any, keys_prefix: str = 'loss_', **kwargs) Dict[str, Any] [source]
Calculates the AD penalty, or returns an empty dict if the model’s autoencoder is inactive.
- Parameters:
model – A model instance.
keys_prefix – A common prefix for all keys in the output dictionary containing all regularization metrics.
- Returns:
- A dictionary with the resulting regularization metrics. The items
will have form {metric_name_i: metric_value_i} keyed by the names of the output metrics metric_name_i with their corresponding values metric_value_i represented as 0-dimensional float tensors.
- The calculated metric is:
autoencoder_norm: Autoencoder weight norm regularization term.
- class pytorch3d.implicitron.models.metrics.ViewMetrics(*args, **kwargs)[source]
Bases:
ViewMetricsBase
- forward(raymarched: RendererOutput, ray_bundle: ImplicitronRayBundle, image_rgb: Tensor | None = None, depth_map: Tensor | None = None, fg_probability: Tensor | None = None, mask_crop: Tensor | None = None, keys_prefix: str = 'loss_', **kwargs) Dict[str, Any] [source]
Calculates various differentiable metrics useful for supervising differentiable rendering pipelines.
- Parameters:
results – A dict to store the results in.
raymarched.features – Predicted rgb or feature values.
raymarched.depths – A tensor of shape (B, …, 1) containing predicted depth values.
raymarched.masks – A tensor of shape (B, …, 1) containing predicted foreground masks.
raymarched.aux["grad_theta"] – A tensor of shape (B, …, 3) containing an evaluation of a gradient of a signed distance function w.r.t. input 3D coordinates used to compute the eikonal loss.
raymarched.aux["density_grid"] – A tensor of shape (B, Hg, Wg, Dg, 1) containing a Hg x Wg x Dg voxel grid of density values.
ray_bundle – ImplicitronRayBundle object which was used to produce the raymarched object
image_rgb – A tensor of shape (B, H, W, 3) containing ground truth rgb values.
depth_map – A tensor of shape (B, Hd, Wd, 1) containing ground truth depth values.
fg_probability – A tensor of shape (B, Hm, Wm, 1) containing ground truth foreground masks.
keys_prefix – A common prefix for all keys in the output dictionary containing all view metrics.
- Returns:
A dictionary `{metric_name_i –
- metric_value_i}` keyed by the
names of the output metrics metric_name_i with their corresponding values metric_value_i represented as 0-dimensional float tensors.
- The calculated metrics are:
rgb_huber: A robust huber loss between image_pred and image. rgb_mse: Mean squared error between image_pred and image. rgb_psnr: Peak signal-to-noise ratio between image_pred and image. rgb_psnr_fg: Peak signal-to-noise ratio between the foreground
region of image_pred and image as defined by mask.
- rgb_mse_fg: Mean squared error between the foreground
region of image_pred and image as defined by mask.
- mask_neg_iou: (1 - intersection-over-union) between mask_pred
and mask.
mask_bce: Binary cross entropy between mask_pred and mask. mask_beta_prior: A loss enforcing strictly binary values
of mask_pred: log(mask_pred) + log(1-mask_pred)
- depth_abs: Mean per-pixel L1 distance between
depth_pred and depth.
- depth_abs_fg: Mean per-pixel L1 distance between the foreground
region of depth_pred and depth as defined by mask.
eikonal: Eikonal regularizer (||grad_theta|| - 1)**2. density_tv: The Total Variation regularizer of density
values in density_grid (sum of L1 distances of values of all 4-neighbouring cells).
- depth_neg_penalty: min(depth_pred, 0)**2 penalizing negative
predicted depth values.