pytorch3d.structures¶

class
pytorch3d.structures.
Meshes
(verts=None, faces=None, textures=None)[source]¶ This class provides functions for working with batches of triangulated meshes with varying numbers of faces and vertices, and converting between representations.
Within Meshes, there are three different representations of the faces and verts data:
 List
 only used for input as a starting point to convert to other representations.
 Padded
 has specific batch dimension.
 Packed
 no batch dimension.
 has auxillary variables used to index into the padded representation.
Example:
Input list of verts V_n = [[V_1], [V_2], … , [V_N]] where V_1, … , V_N are the number of verts in each mesh and N is the numer of meshes.
Input list of faces F_n = [[F_1], [F_2], … , [F_N]] where F_1, … , F_N are the number of faces in each mesh.
From the faces, edges are computed and have packed and padded representations with auxillary variables.
E_n = [[E_1], … , [E_N]] where E_1, … , E_N are the number of unique edges in each mesh. Total number of unique edges = sum(E_n)

__init__
(verts=None, faces=None, textures=None)[source]¶ Parameters:  verts –
Can be either
 List where each element is a tensor of shape (num_verts, 3) containing the (x, y, z) coordinates of each vertex.
 Padded float tensor with shape (num_meshes, max_num_verts, 3). Meshes should be padded with fill value of 0 so they all have the same number of vertices.
 faces –
Can be either
 List where each element is a tensor of shape (num_faces, 3) containing the indices of the 3 vertices in the corresponding mesh in verts which form the triangular face.
 Padded long tensor of shape (num_meshes, max_num_faces, 3). Meshes should be padded with fill value of 1 so they have the same number of faces.
 textures – Optional instance of the Textures class with mesh texture properties.
Refer to comments above for descriptions of List and Padded representations.
 verts –

__getitem__
(index)[source]¶ Parameters: index – Specifying the index of the mesh to retrieve. Can be an int, slice, list of ints or a boolean tensor. Returns: Meshes object with selected meshes. The mesh tensors are not cloned.

isempty
() → bool[source]¶ Checks whether any mesh is valid.
Returns: bool indicating whether there is any data.

verts_list
()[source]¶ Get the list representation of the vertices.
Returns: list of tensors of vertices of shape (V_n, 3).

faces_list
()[source]¶ Get the list representation of the faces.
Returns: list of tensors of faces of shape (F_n, 3).

verts_packed
()[source]¶ Get the packed representation of the vertices.
Returns: tensor of vertices of shape (sum(V_n), 3).

verts_packed_to_mesh_idx
()[source]¶ Return a 1D tensor with the same first dimension as verts_packed. verts_packed_to_mesh_idx[i] gives the index of the mesh which contains verts_packed[i].
Returns: 1D tensor of indices.

mesh_to_verts_packed_first_idx
()[source]¶ Return a 1D tensor x with length equal to the number of meshes such that the first vertex of the ith mesh is verts_packed[x[i]].
Returns: 1D tensor of indices of first items.

num_verts_per_mesh
()[source]¶ Return a 1D tensor x with length equal to the number of meshes giving the number of vertices in each mesh.
Returns: 1D tensor of sizes.

faces_packed
()[source]¶ Get the packed representation of the faces. Faces are given by the indices of the three vertices in verts_packed.
Returns: tensor of faces of shape (sum(F_n), 3).

faces_packed_to_mesh_idx
()[source]¶ Return a 1D tensor with the same first dimension as faces_packed. faces_packed_to_mesh_idx[i] gives the index of the mesh which contains faces_packed[i].
Returns: 1D tensor of indices.

mesh_to_faces_packed_first_idx
()[source]¶ Return a 1D tensor x with length equal to the number of meshes such that the first face of the ith mesh is faces_packed[x[i]].
Returns: 1D tensor of indices of first items.

verts_padded
()[source]¶ Get the padded representation of the vertices.
Returns: tensor of vertices of shape (N, max(V_n), 3).

faces_padded
()[source]¶ Get the padded representation of the faces.
Returns: tensor of faces of shape (N, max(F_n), 3).

num_faces_per_mesh
()[source]¶ Return a 1D tensor x with length equal to the number of meshes giving the number of faces in each mesh.
Returns: 1D tensor of sizes.

edges_packed
()[source]¶ Get the packed representation of the edges.
Returns: tensor of edges of shape (sum(E_n), 2).

edges_packed_to_mesh_idx
()[source]¶ Return a 1D tensor with the same first dimension as edges_packed. edges_packed_to_mesh_idx[i] gives the index of the mesh which contains edges_packed[i].
Returns: 1D tensor of indices.

mesh_to_edges_packed_first_idx
()[source]¶ Return a 1D tensor x with length equal to the number of meshes such that the first edge of the ith mesh is edges_packed[x[i]].
Returns: 1D tensor of indices of first items.

faces_packed_to_edges_packed
()[source]¶ Get the packed representation of the faces in terms of edges. Faces are given by the indices of the three edges in the packed representation of the edges.
Returns: tensor of faces of shape (sum(F_n), 3).

num_edges_per_mesh
()[source]¶ Return a 1D tensor x with length equal to the number of meshes giving the number of edges in each mesh.
Returns: 1D tensor of sizes.

verts_padded_to_packed_idx
()[source]¶ Return a 1D tensor x with length equal to the total number of vertices such that verts_packed()[i] is element x[i] of the flattened padded representation. The packed representation can be calculated as follows.
p = verts_padded().reshape(1, 3) verts_packed = p[x]
Returns: 1D tensor of indices.

verts_normals_packed
()[source]¶ Get the packed representation of the vertex normals.
Returns: tensor of normals of shape (sum(V_n), 3).

verts_normals_list
()[source]¶ Get the list representation of the vertex normals.
Returns: list of tensors of normals of shape (V_n, 3).

verts_normals_padded
()[source]¶ Get the padded representation of the vertex normals.
Returns: tensor of normals of shape (N, max(V_n), 3).

faces_normals_packed
()[source]¶ Get the packed representation of the face normals.
Returns: tensor of normals of shape (sum(F_n), 3).

faces_normals_list
()[source]¶ Get the list representation of the face normals.
Returns: list of tensors of normals of shape (F_n, 3).

faces_normals_padded
()[source]¶ Get the padded representation of the face normals.
Returns: tensor of normals of shape (N, max(F_n), 3).

faces_areas_packed
()[source]¶ Get the packed representation of the face areas.
Returns: tensor of areas of shape (sum(F_n),).

clone
()[source]¶ Deep copy of Meshes object. All internal tensors are cloned individually.
Returns: new Meshes object.

detach
()[source]¶ Detach Meshes object. All internal tensors are detached individually.
Returns: new Meshes object.

to
(device, copy: bool = False)[source]¶ Match functionality of torch.Tensor.to() If copy = True or the self Tensor is on a different device, the returned tensor is a copy of self with the desired torch.device. If copy = False and the self Tensor already has the correct torch.device, then self is returned.
Parameters:  device – Device id for the new tensor.
 copy – Boolean indicator whether or not to clone self. Default False.
Returns: Meshes object.

get_mesh_verts_faces
(index: int)[source]¶ Get tensors for a single mesh from the list representation.
Parameters: index – Integer in the range [0, N). Returns: verts – Tensor of shape (V, 3). faces: LongTensor of shape (F, 3).

split
(split_sizes: list)[source]¶ Splits Meshes object of size N into a list of Meshes objects of size len(split_sizes), where the ith Meshes object is of size split_sizes[i]. Similar to torch.split().
Parameters: split_sizes – List of integer sizes of Meshes objects to be returned. Returns: list[Meshes].

offset_verts_
(vert_offsets_packed)[source]¶ Add an offset to the vertices of this Meshes. In place operation.
Parameters: vert_offsets_packed – A Tensor of the same shape as self.verts_packed giving offsets to be added to all vertices. Returns: self.

offset_verts
(vert_offsets_packed)[source]¶ Out of place offset_verts.
Parameters: vert_offsets_packed – A Tensor of the same shape as self.verts_packed giving offsets to be added to all vertices. Returns: new Meshes object.

scale_verts_
(scale)[source]¶ Multiply the vertices of this Meshes object by a scalar value. In place operation.
Parameters: scale – A scalar, or a Tensor of shape (N,). Returns: self.

scale_verts
(scale)[source]¶ Out of place scale_verts.
Parameters: scale – A scalar, or a Tensor of shape (N,). Returns: new Meshes object.

update_padded
(new_verts_padded)[source]¶ This function allows for an update of verts_padded without having to explicitly convert it to the list representation for heterogeneous batches. Returns a Meshes structure with updated padded tensors and copies of the auxiliary tensors at construction time. It updates self._verts_padded with new_verts_padded, and does a shallow copy of (faces_padded, faces_list, num_verts_per_mesh, num_faces_per_mesh). If packed representations are computed in self, they are updated as well.
Parameters: new_points_padded – FloatTensor of shape (N, V, 3) Returns: Meshes with updated padded representations

get_bounding_boxes
()[source]¶ Compute an axisaligned bounding box for each mesh in this Meshes object.
Returns: bboxes – Tensor of shape (N, 3, 2) where bbox[i, j] gives the min and max values of mesh i along the jth coordinate axis.

pytorch3d.structures.
join_meshes_as_batch
(meshes: List[pytorch3d.structures.meshes.Meshes], include_textures: bool = True)[source]¶ Merge multiple Meshes objects, i.e. concatenate the meshes objects. They must all be on the same device. If include_textures is true, they must all be compatible, either all or none having textures, and all the Textures objects being the same type. If include_textures is False, textures are ignored.
If the textures are TexturesAtlas then being the same type includes having the same resolution. If they are TexturesUV then it includes having the same align_corners and padding_mode.
Parameters:  meshes – list of meshes.
 include_textures – (bool) whether to try to join the textures.
Returns: new Meshes object containing all the meshes from all the inputs.

pytorch3d.structures.
join_meshes_as_scene
(meshes: Union[pytorch3d.structures.meshes.Meshes, List[pytorch3d.structures.meshes.Meshes]], include_textures: bool = True) → pytorch3d.structures.meshes.Meshes[source]¶ Joins a batch of meshes in the form of a Meshes object or a list of Meshes objects as a single mesh. If the input is a list, the Meshes objects in the list must all be on the same device. Unless include_textures is False, the meshes must all have the same type of texture or must all not have textures.
If textures are included, then the textures are joined as a single scene in addition to the meshes. For this, texture types have an appropriate method called join_scene which joins mesh textures into a single texture. If the textures are TexturesAtlas then they must have the same resolution. If they are TexturesUV then they must have the same align_corners and padding_mode. Values in verts_uvs outside [0, 1] will not be respected.
Parameters:  meshes – Meshes object that contains a batch of meshes, or a list of Meshes objects.
 include_textures – (bool) whether to try to join the textures.
Returns: new Meshes object containing a single mesh

class
pytorch3d.structures.
Pointclouds
(points, normals=None, features=None)[source]¶ This class provides functions for working with batches of 3d point clouds, and converting between representations.
Within Pointclouds, there are three different representations of the data.
 List
 only used for input as a starting point to convert to other representations.
 Padded
 has specific batch dimension.
 Packed
 no batch dimension.
 has auxillary variables used to index into the padded representation.
Example
Input list of points = [[P_1], [P_2], … , [P_N]] where P_1, … , P_N are the number of points in each cloud and N is the number of clouds.

__init__
(points, normals=None, features=None)[source]¶ Parameters:  points –
Can be either
 List where each element is a tensor of shape (num_points, 3) containing the (x, y, z) coordinates of each point.
 Padded float tensor with shape (num_clouds, num_points, 3).
 normals –
Can be either
 List where each element is a tensor of shape (num_points, 3) containing the normal vector for each point.
 Padded float tensor of shape (num_clouds, num_points, 3).
 features –
Can be either
 List where each element is a tensor of shape (num_points, C) containing the features for the points in the cloud.
 Padded float tensor of shape (num_clouds, num_points, C).
where C is the number of channels in the features. For example 3 for RGB color.
Refer to comments above for descriptions of List and Padded representations.
 points –

__getitem__
(index)[source]¶ Parameters: index – Specifying the index of the cloud to retrieve. Can be an int, slice, list of ints or a boolean tensor. Returns: Pointclouds object with selected clouds. The tensors are not cloned.

isempty
() → bool[source]¶ Checks whether any cloud is valid.
Returns: bool indicating whether there is any data.

points_list
()[source]¶ Get the list representation of the points.
Returns: list of tensors of points of shape (P_n, 3).

normals_list
()[source]¶ Get the list representation of the normals.
Returns: list of tensors of normals of shape (P_n, 3).

features_list
()[source]¶ Get the list representation of the features.
Returns: list of tensors of features of shape (P_n, C).

points_packed
()[source]¶ Get the packed representation of the points.
Returns: tensor of points of shape (sum(P_n), 3).

normals_packed
()[source]¶ Get the packed representation of the normals.
Returns: tensor of normals of shape (sum(P_n), 3).

features_packed
()[source]¶ Get the packed representation of the features.
Returns: tensor of features of shape (sum(P_n), C).

packed_to_cloud_idx
()[source]¶ Return a 1D tensor x with length equal to the total number of points. packed_to_cloud_idx()[i] gives the index of the cloud which contains points_packed()[i].
Returns: 1D tensor of indices.

cloud_to_packed_first_idx
()[source]¶ Return a 1D tensor x with length equal to the number of clouds such that the first point of the ith cloud is points_packed[x[i]].
Returns: 1D tensor of indices of first items.

num_points_per_cloud
()[source]¶ Return a 1D tensor x with length equal to the number of clouds giving the number of points in each cloud.
Returns: 1D tensor of sizes.

points_padded
()[source]¶ Get the padded representation of the points.
Returns: tensor of points of shape (N, max(P_n), 3).

normals_padded
()[source]¶ Get the padded representation of the normals.
Returns: tensor of normals of shape (N, max(P_n), 3).

features_padded
()[source]¶ Get the padded representation of the features.
Returns: tensor of features of shape (N, max(P_n), 3).

padded_to_packed_idx
()[source]¶ Return a 1D tensor x with length equal to the total number of points such that points_packed()[i] is element x[i] of the flattened padded representation. The packed representation can be calculated as follows.
p = points_padded().reshape(1, 3) points_packed = p[x]
Returns: 1D tensor of indices.

clone
()[source]¶ Deep copy of Pointclouds object. All internal tensors are cloned individually.
Returns: new Pointclouds object.

detach
()[source]¶ Detach Pointclouds object. All internal tensors are detached individually.
Returns: new Pointclouds object.

to
(device, copy: bool = False)[source]¶ Match functionality of torch.Tensor.to() If copy = True or the self Tensor is on a different device, the returned tensor is a copy of self with the desired torch.device. If copy = False and the self Tensor already has the correct torch.device, then self is returned.
Parameters:  device – Device id for the new tensor.
 copy – Boolean indicator whether or not to clone self. Default False.
Returns: Pointclouds object.

get_cloud
(index: int)[source]¶ Get tensors for a single cloud from the list representation.
Parameters: index – Integer in the range [0, N). Returns: points – Tensor of shape (P, 3). normals: Tensor of shape (P, 3) features: LongTensor of shape (P, C).

split
(split_sizes: list)[source]¶ Splits Pointclouds object of size N into a list of Pointclouds objects of size len(split_sizes), where the ith Pointclouds object is of size split_sizes[i]. Similar to torch.split().
Parameters:  split_sizes – List of integer sizes of Pointclouds objects to be
 returned. –
Returns: list[PointClouds].

offset_
(offsets_packed)[source]¶ Translate the point clouds by an offset. In place operation.
Parameters: offsets_packed – A Tensor of the same shape as self.points_packed giving offsets to be added to all points. Returns: self.

offset
(offsets_packed)[source]¶ Out of place offset.
Parameters: offsets_packed – A Tensor of the same shape as self.points_packed giving offsets to be added to all points. Returns: new Pointclouds object.

scale_
(scale)[source]¶ Multiply the coordinates of this object by a scalar value.  i.e. enlarge/dilate In place operation.
Parameters: scale – A scalar, or a Tensor of shape (N,). Returns: self.

scale
(scale)[source]¶ Out of place scale_.
Parameters: scale – A scalar, or a Tensor of shape (N,). Returns: new Pointclouds object.

get_bounding_boxes
()[source]¶ Compute an axisaligned bounding box for each cloud.
Returns: bboxes – Tensor of shape (N, 3, 2) where bbox[i, j] gives the min and max values of cloud i along the jth coordinate axis.

estimate_normals
(neighborhood_size: int = 50, disambiguate_directions: bool = True, assign_to_self: bool = False)[source]¶ Estimates the normals of each point in each cloud and assigns them to the internal tensors self._normals_list and self._normals_padded
The function uses ops.estimate_pointcloud_local_coord_frames to estimate the normals. Please refer to this function for more detailed information about the implemented algorithm.
Args: neighborhood_size: The size of the neighborhood used to estimate the
geometry around each point. disambiguate_directions: If True, uses the algorithm from [1] to
 ensure sign consistency of the normals of neigboring points.
 normals: A tensor of normals for each input point
 of shape (minibatch, num_point, 3). If pointclouds are of Pointclouds class, returns a padded tensor.
 assign_to_self: If True, assigns the computed normals to the
 internal buffers overwriting any previously stored normals.
References
[1] Tombari, Salti, Di Stefano: Unique Signatures of Histograms for Local Surface Description, ECCV 2010.

extend
(N: int)[source]¶ Create new Pointclouds which contains each cloud N times.
Parameters: N – number of new copies of each cloud. Returns: new Pointclouds object.

update_padded
(new_points_padded, new_normals_padded=None, new_features_padded=None)[source]¶ Returns a Pointcloud structure with updated padded tensors and copies of the auxiliary tensors. This function allows for an update of points_padded (and normals and features) without having to explicitly convert it to the list representation for heterogeneous batches.
Parameters:  new_points_padded – FloatTensor of shape (N, P, 3)
 new_normals_padded – (optional) FloatTensor of shape (N, P, 3)
 new_features_padded – (optional) FloatTensors of shape (N, P, C)
Returns: Pointcloud with updated padded representations

inside_box
(box)[source]¶ Finds the points inside a 3D box.
Parameters: box – FloatTensor of shape (2, 3) or (N, 2, 3) where N is the number of clouds.
box[…, 0, :] gives the min x, y & z. box[…, 1, :] gives the max x, y & z.Returns: idx –  BoolTensor of length sum(P_i) indicating whether the packed points are
 within the input box.

pytorch3d.structures.
list_to_packed
(x: List[torch.Tensor])[source]¶ Transforms a list of N tensors each of shape (Mi, K, …) into a single tensor of shape (sum(Mi), K, …).
Parameters: x – list of tensors. Returns: 4element tuple containing  x_packed: tensor consisting of packed input tensors along the 1st dimension.
 num_items: tensor of shape N containing Mi for each element in x.
 item_packed_first_idx: tensor of shape N indicating the index of the first item belonging to the same element in the original list.
 item_packed_to_list_idx: tensor of shape sum(Mi) containing the index of the element in the list the item belongs to.

pytorch3d.structures.
list_to_padded
(x: List[torch.Tensor], pad_size: Union[list, tuple, None] = None, pad_value: float = 0.0, equisized: bool = False) → torch.Tensor[source]¶ Transforms a list of N tensors each of shape (Mi, Ki) into a single tensor of shape (N, pad_size(0), pad_size(1)), or (N, max(Mi), max(Ki)) if pad_size is None.
Parameters:  x – list of Tensors
 pad_size – list(int) specifying the size of the padded tensor
 pad_value – float value to be used to fill the padded tensor
 equisized – bool indicating whether the items in x are of equal size (sometimes this is known and if provided saves computation)
Returns: x_padded – tensor consisting of padded input tensors

pytorch3d.structures.
packed_to_list
(x: torch.Tensor, split_size: Union[list, int])[source]¶ Transforms a tensor of shape (sum(Mi), K, L, …) to N set of tensors of shape (Mi, K, L, …) where Mi’s are defined in split_size
Parameters:  x – tensor
 split_size – list, tuple or int defining the number of items for each tensor in the output list.
Returns: x_list – A list of Tensors

pytorch3d.structures.
padded_to_list
(x: torch.Tensor, split_size: Union[list, tuple, None] = None)[source]¶ Transforms a padded tensor of shape (N, M, K) into a list of N tensors of shape (Mi, Ki) where (Mi, Ki) is specified in split_size(i), or of shape (M, K) if split_size is None. Support only for 3dimensional input tensor.
Parameters:  x – tensor
 split_size – list, tuple or int defining the number of items for each tensor in the output list.
Returns: x_list – a list of tensors