pytorch3d.renderer.mesh.textures
textures
- class pytorch3d.renderer.mesh.textures.TexturesBase[source]
Bases:
object- sample_textures() Tensor[source]
Different texture classes sample textures in different ways e.g. for vertex textures, the values at each vertex are interpolated across the face using the barycentric coordinates. Each texture class should implement a sample_textures method to take the fragments from rasterization. Using fragments.pix_to_face and fragments.bary_coords this function should return the sampled texture values for each pixel in the output image.
- submeshes(vertex_ids_list: List[List[LongTensor]], faces_ids_list: List[List[LongTensor]]) TexturesBase[source]
Extract sub-textures used for submeshing.
- faces_verts_textures_packed() Tensor[source]
Returns the texture for each vertex for each face in the mesh. For N meshes, this function returns sum(Fi)x3xC where Fi is the number of faces in the i-th mesh and C is the dimensional of the feature (C = 3 for RGB textures). You can use the utils function in structures.utils to convert the packed representation to a list or padded.
- clone() TexturesBase[source]
Each texture class should implement a method to clone all necessary internal tensors.
- detach() TexturesBase[source]
Each texture class should implement a method to detach all necessary internal tensors.
- __getitem__(index) TexturesBase[source]
Each texture class should implement a method to get the texture properties for the specified elements in the batch. The TexturesBase._getitem(i) method can be used as a helper function to retrieve the class attributes for item i. Then, a new instance of the child class can be created with the attributes.
- pytorch3d.renderer.mesh.textures.Textures(maps: List[Tensor] | Tensor | None = None, faces_uvs: Tensor | None = None, verts_uvs: Tensor | None = None, verts_rgb: Tensor | None = None) TexturesBase[source]
Textures class has been DEPRECATED. Preserving Textures as a function for backwards compatibility.
- Parameters:
maps – texture map per mesh. This can either be a list of maps [(H, W, C)] or a padded tensor of shape (N, H, W, C).
faces_uvs – (N, F, 3) tensor giving the index into verts_uvs for each vertex in the face. Padding value is assumed to be -1.
verts_uvs – (N, V, 2) tensor giving the uv coordinate per vertex.
verts_rgb – (N, V, C) tensor giving the color per vertex. Padding value is assumed to be -1. (C=3 for RGB.)
- Returns:
a Textures class which is an instance of TexturesBase e.g. TexturesUV, TexturesAtlas, TexturesVertex
- class pytorch3d.renderer.mesh.textures.TexturesAtlas(atlas: Tensor | List[Tensor])[source]
Bases:
TexturesBase- __init__(atlas: Tensor | List[Tensor]) None[source]
A texture representation where each face has a square texture map. This is based on the implementation from SoftRasterizer [1].
- Parameters:
atlas – (N, F, R, R, C) tensor giving the per face texture map. The atlas can be created during obj loading with the pytorch3d.io.load_obj function - in the input arguments set create_texture_atlas=True. The atlas will be returned in aux.texture_atlas.
The padded and list representations of the textures are stored and the packed representations is computed on the fly and not cached.
- [1] Liu et al, ‘Soft Rasterizer: A Differentiable Renderer for Image-based
3D Reasoning’, ICCV 2019 See also https://github.com/ShichenLiu/SoftRas/issues/21
- clone() TexturesAtlas[source]
- detach() TexturesAtlas[source]
- extend(N: int) TexturesAtlas[source]
- sample_textures(fragments, **kwargs) Tensor[source]
This is similar to a nearest neighbor sampling and involves a discretization step. The barycentric coordinates from rasterization are used to find the nearest grid cell in the texture atlas and the RGB is returned as the color. This means that this step is differentiable with respect to the RGB values of the texture atlas but not differentiable with respect to the barycentric coordinates.
TODO: Add a different sampling mode which interpolates the barycentric coordinates to sample the texture and will be differentiable w.r.t the barycentric coordinates.
- Parameters:
fragments –
The outputs of rasterization. From this we use
pix_to_face: LongTensor of shape (N, H, W, K) specifying the indices
of the faces (in the packed representation) which overlap each pixel in the image. - barycentric_coords: FloatTensor of shape (N, H, W, K, 3) specifying the barycentric coordinates of each pixel relative to the faces (in the packed representation) which overlap the pixel.
- Returns:
texels – (N, H, W, K, C)
- submeshes(vertex_ids_list: List[List[LongTensor]], faces_ids_list: List[List[LongTensor]]) TexturesAtlas[source]
Extract a sub-texture for use in a submesh.
If the meshes batch corresponding to this TextureAtlas contains n = len(faces_ids_list) meshes, then self.atlas_list() will be of length n. After submeshing, we obtain a batch of k = sum(len(v) for v in atlas_list submeshes (see Meshes.submeshes). This function creates a corresponding TexturesAtlas object with atlas_list of length k.
- faces_verts_textures_packed() Tensor[source]
Samples texture from each vertex for each face in the mesh. For N meshes with {Fi} number of faces, it returns a tensor of shape sum(Fi)x3xC (C = 3 for RGB). You can use the utils function in structures.utils to convert the packed representation to a list or padded.
- join_batch(textures: List[TexturesAtlas]) TexturesAtlas[source]
Join the list of textures given by textures to self to create a batch of textures. Return a new TexturesAtlas object with the combined textures.
- Parameters:
textures – List of TexturesAtlas objects
- Returns:
new_tex – TexturesAtlas object with the combined textures from self and the list textures.
- join_scene() TexturesAtlas[source]
Return a new TexturesAtlas amalgamating the batch.
- class pytorch3d.renderer.mesh.textures.TexturesUV(maps: Tensor | List[Tensor], faces_uvs: Tensor | List[Tensor] | Tuple[Tensor], verts_uvs: Tensor | List[Tensor] | Tuple[Tensor], *, maps_ids: Tensor | List[Tensor] | Tuple[Tensor] | None = None, padding_mode: str = 'border', align_corners: bool = True, sampling_mode: str = 'bilinear')[source]
Bases:
TexturesBase- __init__(maps: Tensor | List[Tensor], faces_uvs: Tensor | List[Tensor] | Tuple[Tensor], verts_uvs: Tensor | List[Tensor] | Tuple[Tensor], *, maps_ids: Tensor | List[Tensor] | Tuple[Tensor] | None = None, padding_mode: str = 'border', align_corners: bool = True, sampling_mode: str = 'bilinear') None[source]
Textures are represented as a per mesh texture map and uv coordinates for each vertex in each face. NOTE: this class only supports one texture map per mesh.
- Parameters:
maps –
- Either (1) a texture map per mesh. This can either be a list of maps
[(H, W, C)] or a padded tensor of shape (N, H, W, C). For RGB, C = 3. In this case maps_ids must be None.
- Or (2) a set of M texture maps per mesh. This can either be a list of sets
[(M, H, W, C)] or a padded tensor of shape (N, M, H, W, C). For RGB, C = 3. In this case maps_ids must be provided to identify which is relevant to each face.
faces_uvs – (N, F, 3) LongTensor giving the index into verts_uvs for each face
verts_uvs – (N, V, 2) tensor giving the uv coordinates per vertex (a FloatTensor with values between 0 and 1).
maps_ids – Used if there are to be multiple maps per face. This can be either a list of map_ids [(F,)] or a long tensor of shape (N, F) giving the id of the texture map for each face. If maps_ids is present, the maps has an extra dimension M (so maps_padded is (N, M, H, W, C) and maps_list has elements of shape (M, H, W, C)). Specifically, the color of a vertex V is given by an average of maps_padded[i, maps_ids[i, f], u, v, :] over u and v integers adjacent to _verts_uvs_padded[i, _faces_uvs_padded[i, f, 0], :] .
align_corners – If true, the extreme values 0 and 1 for verts_uvs indicate the centers of the edge pixels in the maps.
padding_mode – padding mode for outside grid values (“zeros”, “border” or “reflection”).
sampling_mode – type of interpolation used to sample the texture. Corresponds to the mode parameter in PyTorch’s grid_sample (“nearest” or “bilinear”).
The align_corners and padding_mode arguments correspond to the arguments of the grid_sample torch function. There is an informative illustration of the two align_corners options at https://discuss.pytorch.org/t/22663/9 .
An example of how the indexing into the maps, with align_corners=True, works is as follows. If maps[i] has shape [1001, 101] and the value of verts_uvs[i][j] is [0.4, 0.3], then a value of j in faces_uvs[i] means a vertex whose color is given by maps[i][700, 40]. padding_mode affects what happens if a value in verts_uvs is less than 0 or greater than 1. Note that increasing a value in verts_uvs[…, 0] increases an index in maps, whereas increasing a value in verts_uvs[…, 1] _decreases_ an _earlier_ index in maps.
If align_corners=False, an example would be as follows. If maps[i] has shape [1000, 100] and the value of verts_uvs[i][j] is [0.405, 0.2995], then a value of j in faces_uvs[i] means a vertex whose color is given by maps[i][700, 40]. When align_corners=False, padding_mode even matters for values in verts_uvs slightly above 0 or slightly below 1. In this case, the padding_mode matters if the first value is outside the interval [0.0005, 0.9995] or if the second is outside the interval [0.005, 0.995].
- clone() TexturesUV[source]
- detach() TexturesUV[source]
- extend(N: int) TexturesUV[source]
- sample_textures(fragments, **kwargs) Tensor[source]
Interpolate a 2D texture map using uv vertex texture coordinates for each face in the mesh. First interpolate the vertex uvs using barycentric coordinates for each pixel in the rasterized output. Then interpolate the texture map using the uv coordinate for each pixel.
- Parameters:
fragments –
The outputs of rasterization. From this we use
pix_to_face: LongTensor of shape (N, H, W, K) specifying the indices
of the faces (in the packed representation) which overlap each pixel in the image. - barycentric_coords: FloatTensor of shape (N, H, W, K, 3) specifying the barycentric coordinates of each pixel relative to the faces (in the packed representation) which overlap the pixel.
- Returns:
texels – tensor of shape (N, H, W, K, C) giving the interpolated texture for each pixel in the rasterized image.
- faces_verts_textures_packed() Tensor[source]
Samples texture from each vertex and for each face in the mesh. For N meshes with {Fi} number of faces, it returns a tensor of shape sum(Fi)x3xC (C = 3 for RGB). You can use the utils function in structures.utils to convert the packed representation to a list or padded.
- join_batch(textures: List[TexturesUV]) TexturesUV[source]
Join the list of textures given by textures to self to create a batch of textures. Return a new TexturesUV object with the combined textures.
- Parameters:
textures – List of TexturesUV objects
- Returns:
new_tex – TexturesUV object with the combined textures from self and the list textures.
- join_scene() TexturesUV[source]
Return a new TexturesUV amalgamating the batch.
We calculate a large single map which contains the original maps, and find verts_uvs to point into it. This will not replicate behavior of padding for verts_uvs values outside [0,1].
If align_corners=False, we need to add an artificial border around every map.
We use the function pack_unique_rectangles to provide a layout for the single map. This means that if self was created with a list of maps, and to() has not been called, and there were two maps which were exactly the same tensor object, then they will become the same data in the unified map. _place_map_into_single_map is used to copy the maps into the single map. The merging of verts_uvs and faces_uvs is handled locally in this function.
- centers_for_image(index: int) Tensor[source]
Return the locations in the texture map which correspond to the given verts_uvs, for one of the meshes. This is potentially useful for visualizing the data. See the texturesuv_image_matplotlib and texturesuv_image_PIL functions.
- Parameters:
index – batch index of the mesh whose centers to return.
- Returns:
centers –
- coordinates of points in the texture image
a FloatTensor of shape (V,2)
- check_shapes(batch_size: int, max_num_verts: int, max_num_faces: int) bool[source]
Check if the dimensions of the verts/faces uvs match that of the mesh
- submeshes(vertex_ids_list: List[List[LongTensor]], faces_ids_list: List[List[LongTensor]]) TexturesUV[source]
Extract a sub-texture for use in a submesh.
If the meshes batch corresponding to this TexturesUV contains n = len(faces_ids_list) meshes, then self.faces_uvs_padded() will be of length n. After submeshing, we obtain a batch of k = sum(len(f) for f in faces_ids_list submeshes (see Meshes.submeshes). This function creates a corresponding TexturesUV object with faces_uvs_padded of length k.
- Parameters:
vertex_ids_list – Not used when submeshing TexturesUV.
face_ids_list – A list of length equal to self.faces_uvs_padded. Each element is a LongTensor listing the face ids that the submesh keeps in each respective mesh.
- Returns:
A “TexturesUV in which faces_uvs_padded, verts_uvs_padded, and maps_padded have length sum(len(faces) for faces in faces_ids_list)
- class pytorch3d.renderer.mesh.textures.TexturesVertex(verts_features: Tensor | List[Tensor] | Tuple[Tensor])[source]
Bases:
TexturesBase- __init__(verts_features: Tensor | List[Tensor] | Tuple[Tensor]) None[source]
Batched texture representation where each vertex in a mesh has a C dimensional feature vector.
- Parameters:
verts_features – list of (Vi, C) or (N, V, C) tensor giving a feature vector with arbitrary dimensions for each vertex.
- clone() TexturesVertex[source]
- detach() TexturesVertex[source]
- extend(N: int) TexturesVertex[source]
- sample_textures(fragments, faces_packed=None) Tensor[source]
Determine the color for each rasterized face. Interpolate the colors for vertices which form the face using the barycentric coordinates. :param fragments: The outputs of rasterization. From this we use
pix_to_face: LongTensor of shape (N, H, W, K) specifying the indices
of the faces (in the packed representation) which overlap each pixel in the image. - barycentric_coords: FloatTensor of shape (N, H, W, K, 3) specifying the barycentric coordinates of each pixel relative to the faces (in the packed representation) which overlap the pixel.
- Returns:
texels – An texture per pixel of shape (N, H, W, K, C). There will be one C dimensional value for each element in fragments.pix_to_face.
- submeshes(vertex_ids_list: List[List[LongTensor]], faces_ids_list: List[List[LongTensor]]) TexturesVertex[source]
Extract a sub-texture for use in a submesh.
If the meshes batch corresponding to this TexturesVertex contains n = len(vertex_ids_list) meshes, then self.verts_features_list() will be of length n. After submeshing, we obtain a batch of k = sum(len(v) for v in vertex_ids_list submeshes (see Meshes.submeshes). This function creates a corresponding TexturesVertex object with verts_features_list of length k.
- Parameters:
vertex_ids_list – A list of length equal to self.verts_features_list. Each element is a LongTensor listing the vertices that the submesh keeps in each respective mesh.
face_ids_list – Not used when submeshing TexturesVertex.
- Returns:
A TexturesVertex in which verts_features_list has length sum(len(vertices) for vertices in vertex_ids_list). Each element contains vertex features corresponding to the subset of vertices in that submesh.
- faces_verts_textures_packed(faces_packed=None) Tensor[source]
Samples texture from each vertex and for each face in the mesh. For N meshes with {Fi} number of faces, it returns a tensor of shape sum(Fi)x3xC (C = 3 for RGB). You can use the utils function in structures.utils to convert the packed representation to a list or padded.
- join_batch(textures: List[TexturesVertex]) TexturesVertex[source]
Join the list of textures given by textures to self to create a batch of textures. Return a new TexturesVertex object with the combined textures.
- Parameters:
textures – List of TexturesVertex objects
- Returns:
new_tex – TexturesVertex object with the combined textures from self and the list textures.
- join_scene() TexturesVertex[source]
Return a new TexturesVertex amalgamating the batch.