pytorch3d.structures

class pytorch3d.structures.Meshes(verts=None, faces=None, textures=None)[source]

This class provides functions for working with batches of triangulated meshes with varying numbers of faces and vertices, and converting between representations.

Within Meshes, there are three different representations of the faces and verts data:

List
  • only used for input as a starting point to convert to other representations.
Padded
  • has specific batch dimension.
Packed
  • no batch dimension.
  • has auxillary variables used to index into the padded representation.

Example:

Input list of verts V_n = [[V_1], [V_2], … , [V_N]] where V_1, … , V_N are the number of verts in each mesh and N is the numer of meshes.

Input list of faces F_n = [[F_1], [F_2], … , [F_N]] where F_1, … , F_N are the number of faces in each mesh.

From the faces, edges are computed and have packed and padded representations with auxillary variables.

E_n = [[E_1], … , [E_N]] where E_1, … , E_N are the number of unique edges in each mesh. Total number of unique edges = sum(E_n)

__init__(verts=None, faces=None, textures=None)[source]
Parameters:
  • verts

    Can be either

    • List where each element is a tensor of shape (num_verts, 3) containing the (x, y, z) coordinates of each vertex.
    • Padded float tensor with shape (num_meshes, max_num_verts, 3). Meshes should be padded with fill value of 0 so they all have the same number of vertices.
  • faces

    Can be either

    • List where each element is a tensor of shape (num_faces, 3) containing the indices of the 3 vertices in the corresponding mesh in verts which form the triangular face.
    • Padded long tensor of shape (num_meshes, max_num_faces, 3). Meshes should be padded with fill value of -1 so they have the same number of faces.
  • textures – Optional instance of the Textures class with mesh texture properties.

Refer to comments above for descriptions of List and Padded representations.

__getitem__(index)[source]
Parameters:index – Specifying the index of the mesh to retrieve. Can be an int, slice, list of ints or a boolean tensor.
Returns:Meshes object with selected meshes. The mesh tensors are not cloned.
isempty() → bool[source]

Checks whether any mesh is valid.

Returns:bool indicating whether there is any data.
verts_list()[source]

Get the list representation of the vertices.

Returns:list of tensors of vertices of shape (V_n, 3).
faces_list()[source]

Get the list representation of the faces.

Returns:list of tensors of faces of shape (F_n, 3).
verts_packed()[source]

Get the packed representation of the vertices.

Returns:tensor of vertices of shape (sum(V_n), 3).
verts_packed_to_mesh_idx()[source]

Return a 1D tensor with the same first dimension as verts_packed. verts_packed_to_mesh_idx[i] gives the index of the mesh which contains verts_packed[i].

Returns:1D tensor of indices.
mesh_to_verts_packed_first_idx()[source]

Return a 1D tensor x with length equal to the number of meshes such that the first vertex of the ith mesh is verts_packed[x[i]].

Returns:1D tensor of indices of first items.
num_verts_per_mesh()[source]

Return a 1D tensor x with length equal to the number of meshes giving the number of vertices in each mesh.

Returns:1D tensor of sizes.
faces_packed()[source]

Get the packed representation of the faces. Faces are given by the indices of the three vertices in verts_packed.

Returns:tensor of faces of shape (sum(F_n), 3).
faces_packed_to_mesh_idx()[source]

Return a 1D tensor with the same first dimension as faces_packed. faces_packed_to_mesh_idx[i] gives the index of the mesh which contains faces_packed[i].

Returns:1D tensor of indices.
mesh_to_faces_packed_first_idx()[source]

Return a 1D tensor x with length equal to the number of meshes such that the first face of the ith mesh is faces_packed[x[i]].

Returns:1D tensor of indices of first items.
verts_padded()[source]

Get the padded representation of the vertices.

Returns:tensor of vertices of shape (N, max(V_n), 3).
faces_padded()[source]

Get the padded representation of the faces.

Returns:tensor of faces of shape (N, max(F_n), 3).
num_faces_per_mesh()[source]

Return a 1D tensor x with length equal to the number of meshes giving the number of faces in each mesh.

Returns:1D tensor of sizes.
edges_packed()[source]

Get the packed representation of the edges.

Returns:tensor of edges of shape (sum(E_n), 2).
edges_packed_to_mesh_idx()[source]

Return a 1D tensor with the same first dimension as edges_packed. edges_packed_to_mesh_idx[i] gives the index of the mesh which contains edges_packed[i].

Returns:1D tensor of indices.
mesh_to_edges_packed_first_idx()[source]

Return a 1D tensor x with length equal to the number of meshes such that the first edge of the ith mesh is edges_packed[x[i]].

Returns:1D tensor of indices of first items.
faces_packed_to_edges_packed()[source]

Get the packed representation of the faces in terms of edges. Faces are given by the indices of the three edges in the packed representation of the edges.

Returns:tensor of faces of shape (sum(F_n), 3).
num_edges_per_mesh()[source]

Return a 1D tensor x with length equal to the number of meshes giving the number of edges in each mesh.

Returns:1D tensor of sizes.
verts_padded_to_packed_idx()[source]

Return a 1D tensor x with length equal to the total number of vertices such that verts_packed()[i] is element x[i] of the flattened padded representation. The packed representation can be calculated as follows.

p = verts_padded().reshape(-1, 3)
verts_packed = p[x]
Returns:1D tensor of indices.
verts_normals_packed()[source]

Get the packed representation of the vertex normals.

Returns:tensor of normals of shape (sum(V_n), 3).
verts_normals_list()[source]

Get the list representation of the vertex normals.

Returns:list of tensors of normals of shape (V_n, 3).
verts_normals_padded()[source]

Get the padded representation of the vertex normals.

Returns:tensor of normals of shape (N, max(V_n), 3).
faces_normals_packed()[source]

Get the packed representation of the face normals.

Returns:tensor of normals of shape (sum(F_n), 3).
faces_normals_list()[source]

Get the list representation of the face normals.

Returns:list of tensors of normals of shape (F_n, 3).
faces_normals_padded()[source]

Get the padded representation of the face normals.

Returns:tensor of normals of shape (N, max(F_n), 3).
faces_areas_packed()[source]

Get the packed representation of the face areas.

Returns:tensor of areas of shape (sum(F_n),).
laplacian_packed()[source]
clone()[source]

Deep copy of Meshes object. All internal tensors are cloned individually.

Returns:new Meshes object.
to(device, copy: bool = False)[source]

Match functionality of torch.Tensor.to() If copy = True or the self Tensor is on a different device, the returned tensor is a copy of self with the desired torch.device. If copy = False and the self Tensor already has the correct torch.device, then self is returned.

Parameters:
  • device – Device id for the new tensor.
  • copy – Boolean indicator whether or not to clone self. Default False.
Returns:

Meshes object.

cpu()[source]
cuda()[source]
get_mesh_verts_faces(index: int)[source]

Get tensors for a single mesh from the list representation.

Parameters:index – Integer in the range [0, N).
Returns:verts – Tensor of shape (V, 3). faces: LongTensor of shape (F, 3).
split(split_sizes: list)[source]

Splits Meshes object of size N into a list of Meshes objects of size len(split_sizes), where the i-th Meshes object is of size split_sizes[i]. Similar to torch.split().

Parameters:split_sizes – List of integer sizes of Meshes objects to be returned.
Returns:list[Meshes].
offset_verts_(vert_offsets_packed)[source]

Add an offset to the vertices of this Meshes. In place operation.

Parameters:vert_offsets_packed – A Tensor of the same shape as self.verts_packed giving offsets to be added to all vertices.
Returns:self.
offset_verts(vert_offsets_packed)[source]

Out of place offset_verts.

Parameters:vert_offsets_packed – A Tensor of the same shape as self.verts_packed giving offsets to be added to all vertices.
Returns:new Meshes object.
scale_verts_(scale)[source]

Multiply the vertices of this Meshes object by a scalar value. In place operation.

Parameters:scale – A scalar, or a Tensor of shape (N,).
Returns:self.
scale_verts(scale)[source]

Out of place scale_verts.

Parameters:scale – A scalar, or a Tensor of shape (N,).
Returns:new Meshes object.
update_padded(new_verts_padded)[source]

This function allows for an pdate of verts_padded without having to explicitly convert it to the list representation for heterogeneous batches. Returns a Meshes structure with updated padded tensors and copies of the auxiliary tensors at construction time. It updates self._verts_padded with new_verts_padded, and does a shallow copy of (faces_padded, faces_list, num_verts_per_mesh, num_faces_per_mesh). If packed representations are computed in self, they are updated as well.

Parameters:new_points_padded – FloatTensor of shape (N, V, 3)
Returns:Meshes with updated padded representations
get_bounding_boxes()[source]

Compute an axis-aligned bounding box for each mesh in this Meshes object.

Returns:bboxes – Tensor of shape (N, 3, 2) where bbox[i, j] gives the min and max values of mesh i along the jth coordinate axis.
extend(N: int)[source]

Create new Meshes class which contains each input mesh N times

Parameters:N – number of new copies of each mesh.
Returns:new Meshes object.
pytorch3d.structures.join_meshes_as_batch(meshes: List[pytorch3d.structures.meshes.Meshes], include_textures: bool = True)[source]

Merge multiple Meshes objects, i.e. concatenate the meshes objects. They must all be on the same device. If include_textures is true, they must all be compatible, either all or none having textures, and all the Textures objects having the same members. If include_textures is False, textures are ignored.

Parameters:
  • meshes – list of meshes.
  • include_textures – (bool) whether to try to join the textures.
Returns:

new Meshes object containing all the meshes from all the inputs.

class pytorch3d.structures.Pointclouds(points, normals=None, features=None)[source]

This class provides functions for working with batches of 3d point clouds, and converting between representations.

Within Pointclouds, there are three different representations of the data.

List
  • only used for input as a starting point to convert to other representations.
Padded
  • has specific batch dimension.
Packed
  • no batch dimension.
  • has auxillary variables used to index into the padded representation.

Example

Input list of points = [[P_1], [P_2], … , [P_N]] where P_1, … , P_N are the number of points in each cloud and N is the number of clouds.

__init__(points, normals=None, features=None)[source]
Parameters:
  • points

    Can be either

    • List where each element is a tensor of shape (num_points, 3) containing the (x, y, z) coordinates of each point.
    • Padded float tensor with shape (num_clouds, num_points, 3).
  • normals

    Can be either

    • List where each element is a tensor of shape (num_points, 3) containing the normal vector for each point.
    • Padded float tensor of shape (num_clouds, num_points, 3).
  • features

    Can be either

    • List where each element is a tensor of shape (num_points, C) containing the features for the points in the cloud.
    • Padded float tensor of shape (num_clouds, num_points, C).

    where C is the number of channels in the features. For example 3 for RGB color.

Refer to comments above for descriptions of List and Padded representations.

__getitem__(index)[source]
Parameters:index – Specifying the index of the cloud to retrieve. Can be an int, slice, list of ints or a boolean tensor.
Returns:Pointclouds object with selected clouds. The tensors are not cloned.
isempty() → bool[source]

Checks whether any cloud is valid.

Returns:bool indicating whether there is any data.
points_list()[source]

Get the list representation of the points.

Returns:list of tensors of points of shape (P_n, 3).
normals_list()[source]

Get the list representation of the normals.

Returns:list of tensors of normals of shape (P_n, 3).
features_list()[source]

Get the list representation of the features.

Returns:list of tensors of features of shape (P_n, C).
points_packed()[source]

Get the packed representation of the points.

Returns:tensor of points of shape (sum(P_n), 3).
normals_packed()[source]

Get the packed representation of the normals.

Returns:tensor of normals of shape (sum(P_n), 3).
features_packed()[source]

Get the packed representation of the features.

Returns:tensor of features of shape (sum(P_n), C).
packed_to_cloud_idx()[source]

Return a 1D tensor x with length equal to the total number of points. packed_to_cloud_idx()[i] gives the index of the cloud which contains points_packed()[i].

Returns:1D tensor of indices.
cloud_to_packed_first_idx()[source]

Return a 1D tensor x with length equal to the number of clouds such that the first point of the ith cloud is points_packed[x[i]].

Returns:1D tensor of indices of first items.
num_points_per_cloud()[source]

Return a 1D tensor x with length equal to the number of clouds giving the number of points in each cloud.

Returns:1D tensor of sizes.
points_padded()[source]

Get the padded representation of the points.

Returns:tensor of points of shape (N, max(P_n), 3).
normals_padded()[source]

Get the padded representation of the normals.

Returns:tensor of normals of shape (N, max(P_n), 3).
features_padded()[source]

Get the padded representation of the features.

Returns:tensor of features of shape (N, max(P_n), 3).
padded_to_packed_idx()[source]

Return a 1D tensor x with length equal to the total number of points such that points_packed()[i] is element x[i] of the flattened padded representation. The packed representation can be calculated as follows.

p = points_padded().reshape(-1, 3)
points_packed = p[x]
Returns:1D tensor of indices.
clone()[source]

Deep copy of Pointclouds object. All internal tensors are cloned individually.

Returns:new Pointclouds object.
to(device, copy: bool = False)[source]

Match functionality of torch.Tensor.to() If copy = True or the self Tensor is on a different device, the returned tensor is a copy of self with the desired torch.device. If copy = False and the self Tensor already has the correct torch.device, then self is returned.

Parameters:
  • device – Device id for the new tensor.
  • copy – Boolean indicator whether or not to clone self. Default False.
Returns:

Pointclouds object.

cpu()[source]
cuda()[source]
get_cloud(index: int)[source]

Get tensors for a single cloud from the list representation.

Parameters:index – Integer in the range [0, N).
Returns:points – Tensor of shape (P, 3). normals: Tensor of shape (P, 3) features: LongTensor of shape (P, C).
split(split_sizes: list)[source]

Splits Pointclouds object of size N into a list of Pointclouds objects of size len(split_sizes), where the i-th Pointclouds object is of size split_sizes[i]. Similar to torch.split().

Parameters:
  • split_sizes – List of integer sizes of Pointclouds objects to be
  • returned.
Returns:

list[PointClouds].

offset_(offsets_packed)[source]

Translate the point clouds by an offset. In place operation.

Parameters:offsets_packed – A Tensor of the same shape as self.points_packed giving offsets to be added to all points.
Returns:self.
offset(offsets_packed)[source]

Out of place offset.

Parameters:offsets_packed – A Tensor of the same shape as self.points_packed giving offsets to be added to all points.
Returns:new Pointclouds object.
scale_(scale)[source]

Multiply the coordinates of this object by a scalar value. - i.e. enlarge/dilate In place operation.

Parameters:scale – A scalar, or a Tensor of shape (N,).
Returns:self.
scale(scale)[source]

Out of place scale_.

Parameters:scale – A scalar, or a Tensor of shape (N,).
Returns:new Pointclouds object.
get_bounding_boxes()[source]

Compute an axis-aligned bounding box for each cloud.

Returns:bboxes – Tensor of shape (N, 3, 2) where bbox[i, j] gives the min and max values of cloud i along the jth coordinate axis.
estimate_normals(neighborhood_size: int = 50, disambiguate_directions: bool = True, assign_to_self: bool = False)[source]

Estimates the normals of each point in each cloud and assigns them to the internal tensors self._normals_list and self._normals_padded

The function uses ops.estimate_pointcloud_local_coord_frames to estimate the normals. Please refer to this function for more detailed information about the implemented algorithm.

Args: neighborhood_size: The size of the neighborhood used to estimate the

geometry around each point.
disambiguate_directions: If True, uses the algorithm from [1] to
ensure sign consistency of the normals of neigboring points.
normals: A tensor of normals for each input point
of shape (minibatch, num_point, 3). If pointclouds are of Pointclouds class, returns a padded tensor.
assign_to_self: If True, assigns the computed normals to the
internal buffers overwriting any previously stored normals.

References

[1] Tombari, Salti, Di Stefano: Unique Signatures of Histograms for Local Surface Description, ECCV 2010.

extend(N: int)[source]

Create new Pointclouds which contains each cloud N times.

Parameters:N – number of new copies of each cloud.
Returns:new Pointclouds object.
update_padded(new_points_padded, new_normals_padded=None, new_features_padded=None)[source]

Returns a Pointcloud structure with updated padded tensors and copies of the auxiliary tensors. This function allows for an update of points_padded (and normals and features) without having to explicitly convert it to the list representation for heterogeneous batches.

Parameters:
  • new_points_padded – FloatTensor of shape (N, P, 3)
  • new_normals_padded – (optional) FloatTensor of shape (N, P, 3)
  • new_features_padded – (optional) FloatTensors of shape (N, P, C)
Returns:

Pointcloud with updated padded representations

inside_box(box)[source]

Finds the points inside a 3D box.

Parameters:box

FloatTensor of shape (2, 3) or (N, 2, 3) where N is the number of clouds.

box[…, 0, :] gives the min x, y & z. box[…, 1, :] gives the max x, y & z.
Returns:idx
BoolTensor of length sum(P_i) indicating whether the packed points are
within the input box.
class pytorch3d.structures.Textures(maps: Union[List[T], torch.Tensor, None] = None, faces_uvs: Optional[torch.Tensor] = None, verts_uvs: Optional[torch.Tensor] = None, verts_rgb: Optional[torch.Tensor] = None)[source]
__init__(maps: Union[List[T], torch.Tensor, None] = None, faces_uvs: Optional[torch.Tensor] = None, verts_uvs: Optional[torch.Tensor] = None, verts_rgb: Optional[torch.Tensor] = None)[source]
Parameters:
  • maps – texture map per mesh. This can either be a list of maps [(H, W, 3)] or a padded tensor of shape (N, H, W, 3).
  • faces_uvs – (N, F, 3) tensor giving the index into verts_uvs for each vertex in the face. Padding value is assumed to be -1.
  • verts_uvs – (N, V, 2) tensor giving the uv coordinate per vertex.
  • verts_rgb – (N, V, 3) tensor giving the rgb color per vertex. Padding value is assumed to be -1.

Note: only the padded representation of the textures is stored and the packed/list representations are computed on the fly and not cached.

clone()[source]
to(device)[source]
faces_uvs_padded() → torch.Tensor[source]
faces_uvs_list() → Optional[List[torch.Tensor]][source]
faces_uvs_packed() → Optional[torch.Tensor][source]
verts_uvs_padded() → Optional[torch.Tensor][source]
verts_uvs_list() → Optional[List[torch.Tensor]][source]
verts_uvs_packed() → Optional[torch.Tensor][source]
verts_rgb_padded() → Optional[torch.Tensor][source]
verts_rgb_list() → Optional[List[torch.Tensor]][source]
verts_rgb_packed() → Optional[torch.Tensor][source]
maps_padded() → Optional[torch.Tensor][source]
extend(N: int) → pytorch3d.structures.textures.Textures[source]

Create new Textures class which contains each input texture N times

Parameters:N – number of new copies of each texture.
Returns:new Textures object.
pytorch3d.structures.list_to_packed(x: List[torch.Tensor])[source]

Transforms a list of N tensors each of shape (Mi, K, …) into a single tensor of shape (sum(Mi), K, …).

Parameters:x – list of tensors.
Returns:4-element tuple containing
  • x_packed: tensor consisting of packed input tensors along the 1st dimension.
  • num_items: tensor of shape N containing Mi for each element in x.
  • item_packed_first_idx: tensor of shape N indicating the index of the first item belonging to the same element in the original list.
  • item_packed_to_list_idx: tensor of shape sum(Mi) containing the index of the element in the list the item belongs to.
pytorch3d.structures.list_to_padded(x: List[torch.Tensor], pad_size: Union[list, tuple, None] = None, pad_value: float = 0.0, equisized: bool = False) → torch.Tensor[source]

Transforms a list of N tensors each of shape (Mi, Ki) into a single tensor of shape (N, pad_size(0), pad_size(1)), or (N, max(Mi), max(Ki)) if pad_size is None.

Parameters:
  • x – list of Tensors
  • pad_size – list(int) specifying the size of the padded tensor
  • pad_value – float value to be used to fill the padded tensor
  • equisized – bool indicating whether the items in x are of equal size (sometimes this is known and if provided saves computation)
Returns:

x_padded – tensor consisting of padded input tensors

pytorch3d.structures.packed_to_list(x: torch.Tensor, split_size: Union[list, int])[source]

Transforms a tensor of shape (sum(Mi), K, L, …) to N set of tensors of shape (Mi, K, L, …) where Mi’s are defined in split_size

Parameters:
  • x – tensor
  • split_size – list, tuple or int defining the number of items for each tensor in the output list.
Returns:

x_list – A list of Tensors

pytorch3d.structures.padded_to_list(x: torch.Tensor, split_size: Union[list, tuple, None] = None)[source]

Transforms a padded tensor of shape (N, M, K) into a list of N tensors of shape (Mi, Ki) where (Mi, Ki) is specified in split_size(i), or of shape (M, K) if split_size is None. Support only for 3-dimensional input tensor.

Parameters:
  • x – tensor
  • split_size – list, tuple or int defining the number of items for each tensor in the output list.
Returns:

x_list – a list of tensors