pytorch3d.renderer.mesh.clip

clip

class pytorch3d.renderer.mesh.clip.ClippedFaces(face_verts: Tensor, mesh_to_face_first_idx: Tensor, num_faces_per_mesh: Tensor, faces_clipped_to_unclipped_idx: Tensor | None = None, barycentric_conversion: Tensor | None = None, faces_clipped_to_conversion_idx: Tensor | None = None, clipped_faces_neighbor_idx: Tensor | None = None)[source]

Bases: object

Helper class to store the data for the clipped version of a Meshes object (face_verts, mesh_to_face_first_idx, num_faces_per_mesh) along with conversion information (faces_clipped_to_unclipped_idx, barycentric_conversion, faces_clipped_to_conversion_idx, clipped_faces_neighbor_idx) required to convert barycentric coordinates from rasterization of the clipped Meshes to barycentric coordinates in terms of the unclipped Meshes.

Parameters:

  • face_verts – FloatTensor of shape (F_clipped, 3, 3) giving the verts of each of the clipped faces

  • mesh_to_face_first_idx – an tensor of shape (N,), where N is the number of meshes in the batch. The ith element stores the index into face_verts of the first face of the ith mesh.

  • num_faces_per_mesh – a tensor of shape (N,) storing the number of faces in each mesh.

  • faces_clipped_to_unclipped_idx – (F_clipped,) shaped LongTensor mapping each clipped face back to the face in faces_unclipped (i.e. the faces in the original meshes obtained using meshes.faces_packed())

  • barycentric_conversion – (T, 3, 3) FloatTensor, where barycentric_conversion[i, :, k] stores the barycentric weights in terms of the world coordinates of the original (big) unclipped triangle for the kth vertex in the clipped (small) triangle. If the rasterizer then expresses some NDC coordinate in terms of barycentric world coordinates for the clipped (small) triangle as alpha_clipped[i,:], alpha_unclipped[i, :] = barycentric_conversion[i, :, :]*alpha_clipped[i, :]

  • faces_clipped_to_conversion_idx – (F_clipped,) shaped LongTensor mapping each clipped face to the applicable row of barycentric_conversion (or set to -1 if conversion is not needed).

  • clipped_faces_neighbor_idx – LongTensor of shape (F_clipped,) giving the index of the neighboring face for each case 4 triangle. e.g. for a case 4 face with f split into two triangles (t1, t2): clipped_faces_neighbor_idx[t1_idx] = t2_idx. Faces which are not clipped and subdivided are set to -1 (i.e cases 1/2/3).

face_verts
mesh_to_face_first_idx
num_faces_per_mesh
faces_clipped_to_unclipped_idx
barycentric_conversion
faces_clipped_to_conversion_idx
clipped_faces_neighbor_idx
class pytorch3d.renderer.mesh.clip.ClipFrustum(left: float | None = None, right: float | None = None, top: float | None = None, bottom: float | None = None, znear: float | None = None, zfar: float | None = None, perspective_correct: bool = False, cull: bool = True, z_clip_value: float | None = None)[source]

Bases: object

Helper class to store the information needed to represent a view frustum (left, right, top, bottom, znear, zfar), which is used to clip or cull triangles. Values left as None mean that culling should not be performed for that axis. The parameters perspective_correct, cull, and z_clip_value are used to define behavior for clipping triangles to the frustum.

Parameters:

  • left – NDC coordinate of the left clipping plane (along x axis)

  • right – NDC coordinate of the right clipping plane (along x axis)

  • top – NDC coordinate of the top clipping plane (along y axis)

  • bottom – NDC coordinate of the bottom clipping plane (along y axis)

  • znear – world space z coordinate of the near clipping plane

  • zfar – world space z coordinate of the far clipping plane

  • perspective_correct – should be set to True for a perspective camera

  • cull – if True, triangles outside the frustum should be culled

  • z_clip_value – if not None, then triangles should be clipped (possibly into smaller triangles) such that z >= z_clip_value. This avoids projections that go to infinity as z->0

left
right
top
bottom
znear
zfar
perspective_correct
cull
z_clip_value
pytorch3d.renderer.mesh.clip.clip_faces(face_verts_unclipped: Tensor, mesh_to_face_first_idx: Tensor, num_faces_per_mesh: Tensor, frustum: ClipFrustum) → ClippedFaces[source]

Clip a mesh to the portion contained within a view frustum and with z > z_clip_value.

There are two types of clipping:

  1. Cull triangles that are completely outside the view frustum. This is purely to save computation by reducing the number of triangles that need to be rasterized.

  2. Clip triangles into the portion of the triangle where z > z_clip_value. The clipped region may be a quadrilateral, which results in splitting a triangle into two triangles. This does not save computation, but is necessary to correctly rasterize using perspective cameras for triangles that pass through z <= 0, because NDC/screen coordinates go to infinity at z=0.

Parameters:

  • face_verts_unclipped – An (F, 3, 3) tensor, where F is the number of faces in the packed representation of Meshes, the 2nd dimension represents the 3 vertices of the triangle, and the 3rd dimension stores the xyz locations of each vertex. The z-coordinates must be represented in world coordinates, while the xy-coordinates may be in NDC/screen coordinates

  • mesh_to_face_first_idx – an tensor of shape (N,), where N is the number of meshes in the batch. The ith element stores the index into face_verts_unclipped of the first face of the ith mesh.

  • num_faces_per_mesh – a tensor of shape (N,) storing the number of faces in each mesh.

  • frustum – a ClipFrustum object defining the frustum used to cull faces.

Returns:

clipped_faces –

ClippedFaces object storing a clipped version of the Meshes

along with tensors that can be used to convert barycentric coordinates returned by rasterization of the clipped meshes into a barycentric coordinates for the unclipped meshes.

pytorch3d.renderer.mesh.clip.convert_clipped_rasterization_to_original_faces(pix_to_face_clipped, bary_coords_clipped, clipped_faces: ClippedFaces) → Tuple[Tensor, Tensor][source]

Convert rasterization Fragments (expressed as pix_to_face_clipped, bary_coords_clipped, dists_clipped) of clipped Meshes computed using clip_faces() to the corresponding rasterization Fragments where barycentric coordinates and face indices are in terms of the original unclipped Meshes. The distances are handled in the rasterizer C++/CUDA kernels (i.e. for Cases 1/3 the distance can be used directly and for Case 4 triangles the distance of the pixel to the closest of the two subdivided triangles is used).

Parameters:

  • pix_to_face_clipped – LongTensor of shape (N, image_size, image_size, faces_per_pixel) giving the indices of the nearest faces at each pixel, sorted in ascending z-order. Concretely pix_to_face_clipped[n, y, x, k] = f means that faces_verts_clipped[f] is the kth closest face (in the z-direction) to pixel (y, x). Pixels that are hit by fewer than faces_per_pixel are padded with -1.

  • bary_coords_clipped – FloatTensor of shape (N, image_size, image_size, faces_per_pixel, 3) giving the barycentric coordinates in world coordinates of the nearest faces at each pixel, sorted in ascending z-order. Concretely, if pix_to_face_clipped[n, y, x, k] = f then [w0, w1, w2] = bary_coords_clipped[n, y, x, k] gives the barycentric coords for pixel (y, x) relative to the face defined by unproject(face_verts_clipped[f]). Pixels hit by fewer than faces_per_pixel are padded with -1.

  • clipped_faces – an instance of ClippedFaces class giving the auxillary variables for converting rasterization outputs from clipped to unclipped Meshes.

Returns:

3-tuple – (pix_to_face_unclipped, bary_coords_unclipped, dists_unclipped) that have the same definition as (pix_to_face_clipped, bary_coords_clipped, dists_clipped) except that they pertain to faces_verts_unclipped instead of faces_verts_clipped (i.e the original meshes as opposed to the modified meshes)