pytorch3d.implicitron.dataset in general

Basics of data for implicitron

class pytorch3d.implicitron.dataset.dataset_base.DatasetBase[source]

Bases: GenericWorkaround, Dataset[FrameData]

Base class to describe a dataset to be used with Implicitron.

The dataset is made up of frames, and the frames are grouped into sequences. Each sequence has a name (a string). (A sequence could be a video, or a set of images of one scene.)

This means they have a __getitem__ which returns an instance of a FrameData, which will describe one frame in one sequence.

get_frame_numbers_and_timestamps(idxs: Sequence[int], subset_filter: Sequence[str] | None = None) List[Tuple[int, float]][source]

If the sequences in the dataset are videos rather than unordered views, then the dataset should override this method to return the index and timestamp in their videos of the frames whose indices are given in idxs. In addition, the values in _seq_to_idx should be in ascending order. If timestamps are absent, they should be replaced with a constant.

This is used for letting SceneBatchSampler identify consecutive frames.

Parameters:
  • idxs – frame index in self

  • subset_filter – If given, an index in idxs is ignored if the corresponding frame is not in any of the named subsets.

Returns:

tuple of
  • frame index in video

  • timestamp of frame in video

join(other_datasets: Iterable[DatasetBase]) None[source]

Joins the current dataset with a list of other datasets of the same type.

get_eval_batches() List[List[int]] | None[source]
sequence_names() Iterable[str][source]

Returns an iterator over sequence names in the dataset.

category_to_sequence_names() Dict[str, List[str]][source]

Returns a dict mapping from each dataset category to a list of its sequence names.

Returns:

category_to_sequence_names – Dict {category_i: […, sequence_name_j, …]}

sequence_frames_in_order(seq_name: str, subset_filter: Sequence[str] | None = None) Iterator[Tuple[float, int, int]][source]

Returns an iterator over the frame indices in a given sequence. We attempt to first sort by timestamp (if they are available), then by frame number.

Parameters:

seq_name – the name of the sequence.

Returns:

an iterator over triplets (timestamp, frame_no, dataset_idx),

where frame_no is the index within the sequence, and dataset_idx is the index within the dataset. None timestamps are replaced with 0s.

sequence_indices_in_order(seq_name: str, subset_filter: Sequence[str] | None = None) Iterator[int][source]

Same as sequence_frames_in_order but returns the iterator over only dataset indices.

frame_data_type

alias of FrameData

class pytorch3d.implicitron.dataset.dataset_map_provider.DatasetMap(train: DatasetBase | None, val: DatasetBase | None, test: DatasetBase | None)[source]

Bases: object

A collection of datasets for implicitron.

Members:

train: a dataset for training val: a dataset for validating during training test: a dataset for final evaluation

train: DatasetBase | None
val: DatasetBase | None
test: DatasetBase | None
__getitem__(split: str) DatasetBase | None[source]

Get one of the datasets by key (name of data split)

iter_datasets() Iterator[DatasetBase][source]

Iterator over all datasets.

join(other_dataset_maps: Iterable[DatasetMap]) None[source]

Joins the current DatasetMap with other dataset maps from the input list.

For each subset of each dataset map (train/val/test), the function omits joining the subsets that are None.

Note the train/val/test datasets of the current dataset map will be modified in-place.

Parameters:

other_dataset_maps – The list of dataset maps to be joined into the current dataset map.

class pytorch3d.implicitron.dataset.dataset_map_provider.DatasetMapProviderBase(*args, **kwargs)[source]

Bases: ReplaceableBase

Base class for a provider of training / validation and testing dataset objects.

get_dataset_map() DatasetMap[source]
Returns:

An object containing the torch.Dataset objects in train/val/test fields.

get_all_train_cameras() CamerasBase | None[source]

DEPRECATED! The function will be removed in future versions. If the data is all for a single scene, returns a list of the known training cameras for that scene, which is used for evaluating the difficulty of the unknown cameras. Otherwise return None.

class pytorch3d.implicitron.dataset.dataset_map_provider.PathManagerFactory(*args, **kwargs)[source]

Bases: ReplaceableBase

Base class and default implementation of a tool which dataset_map_provider implementations may use to construct a path manager if needed.

Parameters:

silence_logs – Whether to reduce log output from iopath library.

silence_logs: bool = True
get() PathManager | None[source]

Makes a PathManager if needed. For open source users, this function should always return None. Internally, this allows manifold access.

class pytorch3d.implicitron.dataset.data_loader_map_provider.DataLoaderMap(train: DataLoader[FrameData] | None, val: DataLoader[FrameData] | None, test: DataLoader[FrameData] | None)[source]

Bases: object

A collection of data loaders for Implicitron.

Members:

train: a data loader for training val: a data loader for validating during training test: a data loader for final evaluation

train: DataLoader[FrameData] | None
val: DataLoader[FrameData] | None
test: DataLoader[FrameData] | None
__getitem__(split: str) DataLoader[FrameData] | None[source]

Get one of the data loaders by key (name of data split)

class pytorch3d.implicitron.dataset.data_loader_map_provider.DataLoaderMapProviderBase(*args, **kwargs)[source]

Bases: ReplaceableBase

Provider of a collection of data loaders for a given collection of datasets.

get_data_loader_map(datasets: DatasetMap) DataLoaderMap[source]

Returns a collection of data loaders for a given collection of datasets.

class pytorch3d.implicitron.dataset.data_loader_map_provider.SimpleDataLoaderMapProvider(*args, **kwargs)[source]

Bases: DataLoaderMapProviderBase

Trivial implementation of DataLoaderMapProviderBase.

If a dataset returns batches from get_eval_batches(), then they will be what the corresponding dataloader returns, independently of any of the fields on this class.

Otherwise, returns shuffled batches.

batch_size: int = 1
num_workers: int = 0
dataset_length_train: int = 0
dataset_length_val: int = 0
dataset_length_test: int = 0
get_data_loader_map(datasets: DatasetMap) DataLoaderMap[source]

Returns a collection of data loaders for a given collection of datasets.

class pytorch3d.implicitron.dataset.data_loader_map_provider.DoublePoolBatchSampler(first_indices: List[int], rest_indices: List[int], batch_size: int, replacement: bool, num_batches: int | None = None)[source]

Bases: Sampler[List[int]]

Batch sampler for making random batches of a single frame from one list and a number of known frames from another list.

__init__(first_indices: List[int], rest_indices: List[int], batch_size: int, replacement: bool, num_batches: int | None = None) None[source]
Parameters:
  • first_indices – indexes of dataset items to use as the first element of each batch.

  • rest_indices – indexes of dataset items to use as the subsequent elements of each batch. Not used if batch_size==1.

  • batch_size – The common size of any batch.

  • replacement – Whether the sampling of first items is with replacement.

  • num_batches – The number of batches in an epoch. If 0 or None, one epoch is the length of first_indices.

class pytorch3d.implicitron.dataset.data_loader_map_provider.BatchConditioningType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]

Bases: Enum

Ways to add conditioning frames for the val and test batches.

SAME: Use the corresponding dataset for all elements of val batches

without regard to frame type.

TRAIN: Use the corresponding dataset for the first element of each
batch, and the training dataset for the extra conditioning

elements. No regard to frame type.

KNOWN: Use frames from the corresponding dataset but separate them

according to their frame_type. Each batch will contain one UNSEEN frame followed by many KNOWN frames.

SAME = 'same'
TRAIN = 'train'
KNOWN = 'known'
class pytorch3d.implicitron.dataset.data_loader_map_provider.SequenceDataLoaderMapProvider(*args, **kwargs)[source]

Bases: DataLoaderMapProviderBase

Default implementation of DataLoaderMapProviderBase.

If a dataset returns batches from get_eval_batches(), then they will be what the corresponding dataloader returns, independently of any of the fields on this class.

If conditioning is not required, then the batch size should be set as 1, and most of the fields do not matter.

If conditioning is required, each batch will contain one main frame first to predict and the, rest of the elements are for conditioning.

If images_per_seq_options is left empty, the conditioning frames are picked according to the conditioning type given. This does not have regard to the order of frames in a scene, or which frames belong to what scene.

If images_per_seq_options is given, then the conditioning types must be SAME and the remaining fields are used.

Members:

batch_size: The size of the batch of the data loader. num_workers: Number of data-loading threads in each data loader. dataset_length_train: The number of batches in a training epoch. Or 0 to mean

an epoch is the length of the training set.

dataset_length_val: The number of batches in a validation epoch. Or 0 to mean

an epoch is the length of the validation set.

dataset_length_test: The number of batches in a testing epoch. Or 0 to mean

an epoch is the length of the test set.

train_conditioning_type: Whether the train data loader should use

only known frames for conditioning. Only used if batch_size>1 and train dataset is present and does not return eval_batches.

val_conditioning_type: Whether the val data loader should use

training frames or known frames for conditioning. Only used if batch_size>1 and val dataset is present and does not return eval_batches.

test_conditioning_type: Whether the test data loader should use

training frames or known frames for conditioning. Only used if batch_size>1 and test dataset is present and does not return eval_batches.

images_per_seq_options: Possible numbers of frames sampled per sequence in a batch.

If a conditioning_type is KNOWN or TRAIN, then this must be left at its initial value. Empty (the default) means that we are not careful about which frames come from which scene.

sample_consecutive_frames: if True, will sample a contiguous interval of frames

in the sequence. It first sorts the frames by timestimps when available, otherwise by frame numbers, finds the connected segments within the sequence of sufficient length, then samples a random pivot element among them and ideally uses it as a middle of the temporal window, shifting the borders where necessary. This strategy mitigates the bias against shorter segments and their boundaries.

consecutive_frames_max_gap: if a number > 0, then used to define the maximum

difference in frame_number of neighbouring frames when forming connected segments; if both this and consecutive_frames_max_gap_seconds are 0s, the whole sequence is considered a segment regardless of frame numbers.

consecutive_frames_max_gap_seconds: if a number > 0.0, then used to define the

maximum difference in frame_timestamp of neighbouring frames when forming connected segments; if both this and consecutive_frames_max_gap are 0s, the whole sequence is considered a segment regardless of frame timestamps.

batch_size: int = 1
num_workers: int = 0
dataset_length_train: int = 0
dataset_length_val: int = 0
dataset_length_test: int = 0
train_conditioning_type: BatchConditioningType = 'same'
val_conditioning_type: BatchConditioningType = 'same'
test_conditioning_type: BatchConditioningType = 'known'
images_per_seq_options: Tuple[int, ...] = ()
sample_consecutive_frames: bool = False
consecutive_frames_max_gap: int = 0
consecutive_frames_max_gap_seconds: float = 0.1
get_data_loader_map(datasets: DatasetMap) DataLoaderMap[source]

Returns a collection of data loaders for a given collection of datasets.

class pytorch3d.implicitron.dataset.data_source.DataSourceBase(*args, **kwargs)[source]

Bases: ReplaceableBase

Base class for a data source in Implicitron. It encapsulates Dataset and DataLoader configuration.

get_datasets_and_dataloaders() Tuple[DatasetMap, DataLoaderMap][source]
property all_train_cameras: CamerasBase | None

DEPRECATED! The property will be removed in future versions. If the data is all for a single scene, a list of the known training cameras for that scene, which is used for evaluating the viewpoint difficulty of the unseen cameras.

class pytorch3d.implicitron.dataset.data_source.ImplicitronDataSource(*args, **kwargs)[source]

Bases: DataSourceBase

Represents the data used in Implicitron. This is the only implementation of DataSourceBase provided.

Members:
dataset_map_provider_class_type: identifies type for dataset_map_provider.

e.g. JsonIndexDatasetMapProvider for Co3D.

data_loader_map_provider_class_type: identifies type for data_loader_map_provider.

dataset_map_provider: DatasetMapProviderBase
dataset_map_provider_class_type: str
data_loader_map_provider: DataLoaderMapProviderBase
data_loader_map_provider_class_type: str = 'SequenceDataLoaderMapProvider'
classmethod pre_expand() None[source]
get_datasets_and_dataloaders() Tuple[DatasetMap, DataLoaderMap][source]
property all_train_cameras: CamerasBase | None

DEPRECATED! The property will be removed in future versions.

class pytorch3d.implicitron.dataset.scene_batch_sampler.SceneBatchSampler(dataset: DatasetBase, batch_size: int, num_batches: int, images_per_seq_options: Sequence[int], sample_consecutive_frames: bool = False, consecutive_frames_max_gap: int = 0, consecutive_frames_max_gap_seconds: float = 0.1, category_aware: bool = True)[source]

Bases: Sampler[List[int]]

A class for sampling training batches with a controlled composition of sequences.

dataset: DatasetBase
batch_size: int
num_batches: int
images_per_seq_options: Sequence[int]
sample_consecutive_frames: bool = False
consecutive_frames_max_gap: int = 0
consecutive_frames_max_gap_seconds: float = 0.1
category_aware: bool = True
seq_names: List[str]
category_to_sequence_names: Dict[str, List[str]]
categories: List[str]