Python API for bob.ip.pytorch_extractor¶

class bob.ip.pytorch_extractor.CNN8Extractor(model_file=None, num_classes=10575)¶

Bases: bob.bio.base.extractor.Extractor

The class implementing the feature extraction of CASIA-Net embeddings.

network¶

The network architecture

Type:	`torch.nn.Module`

to_tensor¶

The transform from numpy.array to torch.Tensor

Type:	`torchvision.transforms`

norm¶

The transform to normalize the input

Type:	`torchvision.transforms`

__call__(image)[source]¶

Extract features from an image

Parameters:	image (3D `numpy.ndarray` (floats)) – The image to extract the features from. Its size must be 3x128x128
Returns:	feature – The extracted features as a 1d array of size 320
Return type:	2D `numpy.ndarray` (floats)

__init__(model_file=None, num_classes=10575)[source]¶

Init method

Parameters:	model_file (str) – The path of the trained network to load drop_rate (float) – The number of classes.

class bob.ip.pytorch_extractor.CasiaNetExtractor(model_file=None, num_classes=10575)¶

Bases: bob.bio.base.extractor.Extractor

The class implementing the feature extraction of CASIA-Net embeddings.

network¶

The network architecture

Type:	`torch.nn.Module`

to_tensor¶

The transform from numpy.array to torch.Tensor

Type:	`torchvision.transforms`

norm¶

The transform to normalize the input

Type:	`torchvision.transforms`

__call__(image)[source]¶

Extract features from an image

Parameters:	image (3D `numpy.ndarray` (floats)) – The image to extract the features from. Its size must be 3x128x128
Returns:	feature – The extracted features as a 1d array of size 320
Return type:	2D `numpy.ndarray` (floats)

__init__(model_file=None, num_classes=10575)[source]¶

Init method

Parameters:	model_file (str) – The path of the trained network to load drop_rate (float) – The number of classes.

class bob.ip.pytorch_extractor.LightCNN9Extractor(model_file=None, num_classes=79077)¶

Bases: bob.bio.base.extractor.Extractor

The class implementing the feature extraction of LightCNN9 embeddings.

network¶

The network architecture

Type:	`torch.nn.Module`

to_tensor¶

The transform from numpy.array to torch.Tensor

Type:	`torchvision.transforms`

norm¶

The transform to normalize the input

Type:	`torchvision.transforms`

__call__(image)[source]¶

Extract features from an image

Parameters:	image (2D `numpy.ndarray` (floats)) – The grayscale image to extract the features from. Its size must be 128x128
Returns:	feature – The extracted features as a 1d array of size 320
Return type:	`numpy.ndarray` (floats)

__init__(model_file=None, num_classes=79077)[source]¶

Init method

Parameters:	model_file (str) – The path of the trained network to load num_classes (int) – The number of classes.

class bob.ip.pytorch_extractor.MLPAlgorithm(network, model_file=None, url=None, archive_extension='.tar.gz', frame_level_scores_flag=True, mean_std_norm_flag=True)¶

Bases: bob.pad.base.algorithm.Algorithm

This class is an MLP-based PAD algorithm, allowing to pass input feature vectors (1D numpy arrays) via a pretrained MLP.

network¶

An instance of an MLP Network to be used for score computation. Note: in current algorith the forward() method of the Network is used for score computation. For example, if you don’t want to use a sigmoid in the output of an MLP, set the kwargs accodingly.

Type:	object

model_file¶

A paths to the model file to be used for network initialization.

Type:	str

url¶

URL to download the pretrained models from. If models are not available in the locations specified in the model_file sting, the system will try to download them from url. The downloaded models will be placed to the locations specified in model_file argument. Default: None

Type:	str

archive_extension¶

Extension of the archived file to download from above url. Default: ‘.tar.gz’

Type:	str

frame_level_scores_flag¶

Return scores for each frame individually if True. Otherwise, return a single score per video. Default: True.

Type:	bool

mean_std_norm_flag¶

Perform mean-std normalization of data if set to True. Note: make sure an MLP was trained on mean-std normalizaed features, if this flag is set to True. The tutorial on MLP training in bob.learn.pytorch provides an example on how to train network on normalized features. Default: True.

Type:	bool

__init__(network, model_file=None, url=None, archive_extension='.tar.gz', frame_level_scores_flag=True, mean_std_norm_flag=True)[source]¶: Initialize self. See help(type(self)) for accurate signature.

load_projector(projector_file)[source]¶

Loads features mean and std from the hdf5 file. The absolute name of the file is specified in projector_file string.

Parameters:	projector_file (str) – Absolute name of the file to load the trained projector from, as returned by `bob.pad.base` framework.

project(feature)[source]¶

This function computes a vector of scores, one score for each sample in the input array of features.

Set performs_projection = True in the constructor to enable this function.

Parameters:	feature (FrameContainer or `numpy.ndarray`) – Two types of inputs are accepted. A Frame Container containing the features of an individual frmaes, see `bob.bio.video.utils.FrameContainer`. Or a ND feature array of the size (n_samples x n_features).
Returns:	scores – Vector of scores. Scores for the bona-fide class are expected to be higher, than the scores of the negative / attack class.
Return type:	1D `numpy.ndarray`

score(toscore)[source]¶

Returns a probability of a sample being a real class.

Parameters:	toscore (1D `numpy.ndarray`) – Vector with scores for each frame/sample defining the probability of the frame being a sample of the real class.
Returns:	score – If `frame_level_scores_flag = False` a single score is returned. One score per video. This score is placed into a list, because the `score` must be an iterable. Score is a probability of a sample being a real class. If `frame_level_scores_flag = True` a list of scores is returned. One score per frame/sample.
Return type:	[`float`]

score_for_multiple_projections(toscore)[source]¶

Returns a list of scores computed by the score method of this class.

Parameters:	toscore (1D `numpy.ndarray`) – Vector with scores for each frame/sample defining the probability of the frame being a sample of the real class.
Returns:	list_of_scores – A list containing the scores.
Return type:	[`float`]

train_projector(training_features, projector_file)[source]¶

Compute mean-std normalizers using samples of the real class only.

Parameters:	training_features ([[object], [object]]) – A list containing two elements: [0] - a list of Frame Containers with feature vectors for the real class; [1] - a list of Frame Containers with feature vectors for the attack class. projector_file (str) – The file to save the trained projector to, as returned by the `bob.pad.base` framework.

class bob.ip.pytorch_extractor.MultiNetPatchExtractor(transform, network, model_file, patches_num, patch_reshape_parameters=None, color_input_flag=False, urls=None, archive_extension='.tar.gz')¶

Bases: bob.bio.base.extractor.Extractor, object

This class is designed to pass a set of patches through a possibly multiple networks and compute a feature vector combining outputs of all networks.

The functional work-flow is the following:

First, an array of flattened input patches is converted to a list of patches with original dimensions (2D or 3D arrays).

Second, each patch is passed through an individual network, for example an auto-encoder pre-trained for each patch type (left eye, for example).

Third, outputs of all networks are concatenated into a single feature vector.

transform¶

Function namely transform, which is a Compose transformation of torchvision package, to be applied to the input samples.

Type:	object

network¶

An instance of the Network to be used for feature extraction. Note: in current extractor the forward() method of the Network is used for feature extraction. For example, if you want to use the latent embeddings of the autoencoder class, initialize the network accordingly.

Type:	object

model_file¶

A list of paths to the model files to be used for network initialization.

Type:	[str]

patches_num¶

A list of inices specifying which patches will be selected for processing/feature vector extraction.

Type:	[int]

patch_reshape_parameters¶

The parameters to be used for patch reshaping. The loaded patch is vectorized. Example: patch_reshape_parameters = [4, 8, 8], then the patch of the size (256,) will be reshaped to (4,8,8) dimensions. Only 2D and 3D patches are supported. Default: None.

Type:	[int] or None

color_input_flag¶

If set to True, the input is considered to be a color image of the size (3, H, W). The tensor to be passed through the net will be of the size (1, 3, H, W). If set to False, the input is considered to be a set of BW images of the size (n_samples, H, W). The tensor to be passed through the net will be of the size (n_samples, 1, H, W). Default: False.

Type:	bool

urls¶

List of URLs to download the pretrained models from. If models are not available in the locations specified in the model_file list, the system will try to download them from urls. The downloaded models will be placed to the locations specified in model_file list.

For example, a model for an autoencoder pre-trained on RGB faces of the size (3(channels) x 128 x 128) and fine-tuned on the BW-NIR-D data can be found here: [“https://www.idiap.ch/software/bob/data/bob/bob.ip.pytorch_extractor/master/” “conv_ae_model_pretrain_celeba_tune_batl_full_face.pth.tar.gz”]

Default: None

Type:	[str]

archive_extension¶

Extension of the archived files to download from above urls.

Default: ‘.tar.gz’

Type:	str

__call__(patches)[source]¶

Extract features combining outputs of multiple networks.

Parameters:	patches (2D `numpy.ndarray`) – An array containing flattened patches. The dimensions are: `num_patches x len_of_flat_patch`
Returns:	features – Feature vector.
Return type:	`numpy.ndarray`

__init__(transform, network, model_file, patches_num, patch_reshape_parameters=None, color_input_flag=False, urls=None, archive_extension='.tar.gz')[source]¶: Init method.