Python API

Classes

bob.ip.facedetect.BoundingBox	A bounding box class storing top, left, height and width of an
bob.ip.facedetect.FeatureExtractor	This class extracts LBP features of several types from a given image

Functions

Detailed Information

bob.ip.facedetect.get_config()

Returns a string containing the configuration information.

class bob.ip.facedetect.BoundingBox

Bases: object

A bounding box class storing top, left, height and width of an rectangle

Constructor Documentation:

• bob.ip.facedetect.BoundingBox (topleft, size)
• bob.ip.facedetect.BoundingBox (bounding_box)

Constructs a new Bounding box from the given top-left position and the size of the rectangle

Parameters:

topleft : (float, float)

The top-left position of the bounding box

size : (float, float)

The size of the bounding box

bounding_box : BoundingBox

The BoundingBox object to use for copy-construction

Class Members:

area

float <– The area (height x width) of the bounding box, read access only

bottom

int <– The bottom position of the bounding box (which is just outside the bounding box) as int, read access only

bottom_f

float <– The bottom position of the bounding box (which is just outside the bounding box) as float, read access only

bottomright

(int, int) <– The bottom-right position of the bounding box (which is just outside the bounding box) as integral values, read access only

bottomright_f

(float, float) <– The bottom-right position of the bounding box (which is just outside the bounding box) as float values, read access only

center

(float, float) <– The center of the bounding box (as float values), read access only

contains(point) → contained

Checks if the bounding box contains the given point

Parameters:

point : (float, float)

The point to test

Returns:

contained : bool

True if the bounding box contains the given point, False otherwise

is_valid_for(size) → valid

Checks if the bounding box is inside the given image size

Parameters:

size : (int, int)

The size of the image to test

Returns:

valid : bool

True if the bounding box is inside the image boundaries, False otherwise

left

int <– The left position of the bounding box as int, read access only

left_f

float <– The left position of the bounding box as float, read access only

mirror_x(width) → bounding_box

This function returns a horizontally mirrored version of this BoundingBox

Parameters:

width : int

The width of the image at which this bounding box should be mirrored

Returns:

bounding_box : BoundingBox

The mirrored version of this bounding box

overlap(other) → bounding_box

This function returns the overlapping bounding box between this and the given bounding box

Parameters:

other : BoundingBox

The other bounding box to compute the overlap with

Returns:

bounding_box : BoundingBox

The overlap between this and the other bounding box

right

int <– The right position of the bounding box (which is just outside the bounding box) as int, read access only

right_f

float <– The right position of the bounding box (which is just outside the bounding box) as float, read access only

scale(scale[, centered]) → bounding_box

This function returns a scaled version of this BoundingBox

When the centered parameter is set to True, the transformation center will be in the center of this bounding box, otherwise it will be at (0,0)

Parameters:

scale : float

The scale with which this bounding box should be shifted

centered : bool

[Default: False] : Should the scaling done with repect to the center of the bounding box?

Returns:

bounding_box : BoundingBox

The scaled version of this bounding box

shift(offset) → bounding_box

This function returns a shifted version of this BoundingBox

Parameters:

offset : (float, float)

The offset with which this bounding box should be shifted

Returns:

bounding_box : BoundingBox

The shifted version of this bounding box

similarity(other) → sim

This function computes the Jaccard similarity index between this and the given BoundingBox

Parameters:

other : BoundingBox

The other bounding box to compute the overlap with

Returns:

sim : float

The Jaccard similarity index between this and the given BoundingBox

size

(int, int) <– The size of the bounding box as integral values, read access only

size_f

(float, float) <– The size of the bounding box as float values, read access only

top

int <– The top position of the bounding box as int, read access only

top_f

float <– The top position of the bounding box as float, read access only

topleft

(int, int) <– The top-left position of the bounding box as integral values, read access only

topleft_f

(float, float) <– The top-left position of the bounding box as float values, read access only

class bob.ip.facedetect.Cascade(cascade_file=None, feature_extractor=None)

This class defines a cascade of strong classifiers bob.learn.boosting.BoostedMachine.

For each strong classifier, a threshold exists. When the weighted sum of predictions of classifiers gets below this threshold, the classification is stopped.

Constructor Documentation:

The constructor has two different ways to be called. The first and most obvious way is to load the cascade from the given cascade_file.

The second way instantiates an empty cascade, with the given feature_extractor. Please use the add() function to add new strong classifiers with according thresholds.

Keyword parameters:

cascade_file : bob.io.base.HDF5File

An HDF5 file open for reading

feature_extractor : FeatureExtractor

A feature extractor that will be used to extract features for the strong classifiers.

add(classifier, threshold, begin=None, end=None)

Adds a new strong classifier with the given threshold to the cascade.

Keyword parameters:

classifier : bob.learn.boosting.BoostedMachine

A strong classifier to add

threshold : float

The classification threshold for this cascade step

begin, end : int or None

If specified, only the weak machines with the indices range(begin,end) will be added.

create_from_boosted_machine(boosted_machine, classifiers_per_round, classification_thresholds=-5.0)

Creates this cascade from the given boosted machine, by simply splitting off strong classifiers that have classifiers_per_round weak classifiers.

Keyword parameters:

boosted_machine : bob.learn.boosting.BoostedMachine

The strong classifier to split into a regular cascade.

classifiers_per_round : int

The number of classifiers that each cascade step should contain.

classification_threshold : float

A single threshold that will be applied in all rounds of the cascade.

generate_boosted_machine()

Creates a single strong classifier from this cascade by concatenating all strong classifiers.

Returns:

bob.learn.boosting.BoostedMachine

The strong classifier as a combination of all classifiers in this cascade.

load(hdf5)

Loads this cascade from the given HDF5 file.

Keyword parameters:

hdf5 : bob.io.base.HDF5File

An HDF5 file open for reading

prepare(image, scale)

Prepares the cascade for extracting features of the given image in the given scale.

Keyword parameters:

image : array_like (2D, float)

The image from which features will be extracted

scale : float

The scale of the image, for which features will be extracted

save(hdf5)

Saves this cascade into the given HDF5 file.

Keyword parameters:

hdf5 : bob.io.base.HDF5File

An HDF5 file open for writing

class bob.ip.facedetect.FeatureExtractor

Bases: object

This class extracts LBP features of several types from a given image patch of a certain size

LBP features are extracted using different variants of bob.ip.base.LBP feature extractors. All LBP features of one patch are stored in a single long feature vector of type numpy.uint16.

Constructor Documentation:

• bob.ip.facedetect.FeatureExtractor (patch_size)
• bob.ip.facedetect.FeatureExtractor (patch_size, extractors)
• bob.ip.facedetect.FeatureExtractor (patch_size, template, [overlap], [square], [min_size], [max_size])
• bob.ip.facedetect.FeatureExtractor (other)
• bob.ip.facedetect.FeatureExtractor (hdf5)

Generates a new feature extractor for the given patch_size using one or several feature extractors

The constructor can be called in different ways:

• The first constructor initializes a feature extractor with no LBP extractor. Please use the append() function to add LBP extractors.
• In the second constructor, a given list of LBP extractors is specified.
• The third constructor initializes a tight set of LBP extractors for different bob.ip.base.LBP.radii, by adding all possible combinations of x- and y- radii, until the patch_size is too small, or min_size (start) or max_size (end) is reached.
• The fourth constructor copies all LBP extractors from the given FeatureExtractor
• The last constructor read the configuration from the given bob.io.base.HDF5File.

Parameters:

patch_size : (int, int)

The size of the patch to extract from the images

extractors : [bob.ip.base.LBP]

The LBP classes to use as extractors

template : bob.ip.base.LBP

The LBP classes to use as template for all extractors

overlap : bool

[default: False] Should overlapping LBPs be created?

square : bool

[default: False] Should only square LBPs be created?

min_size : int

[default: 1] The minimum radius of LBP

max_size : int

[default: MAX_INT] The maximum radius of LBP (limited by patch size)

other : FeatureExtractor

The feature extractor to use for copy-construction

hdf5 : bob.io.base.HDF5File

The HDF5 file to read the extractors from

Class Members:

append()

• append(other) -> None
• append(lbp, offsets) -> None

Appends the given feature extractor or LBP class to this one

With this function you can either append a complete feature extractor, or a partial axtractor (i.e., a single LBP class) including the offset positions for them

Parameters:

other : FeatureExtractor

All LBP classes and offset positions of the given extractor will be appended

lbp : bob.ip.base.LBP

The LBP extractor that will be added

offsets : [(int,int)]

The offset positions at which the given LBP will be extracted

extract_all(bounding_box, dataset, dataset_index) → None

Extracts all features into the given dataset of (training) features at the given index

This function exists to extract training features for several training patches. To avoid data copying, the full training dataset, and the current training feature index need to be provided.

Parameters:

bounding_box : BoundingBox

The bounding box for which the features should be extracted

dataset : array_like <2D, uint16>

The (training) dataset, into which the features should be extracted; must be of shape (#training_patches, number_of_features)

dataset_index : int

The index of the current training patch

extract_indexed(bounding_box, feature_vector[, indices]) → None

Extracts the features only at the required locations, which defaults to model_indices

Parameters:

bounding_box : BoundingBox

The bounding box for which the features should be extracted

feature_vector : array_like <1D, uint16>

The feature vector, into which the features should be extracted; must be of size number_of_features

indices : array_like<1D,int32>

The indices, for which the features should be extracted; if not given, model_indices is used (must be set beforehands)

extractor(index) → lbp

Get the LBP feature extractor associated with the given feature index

Parameters:

index : int

The feature index for which the extractor should be retrieved

Returns:

lbp : bob.ip.base.LBP

The feature extractor for the given feature index

extractors

[bob.ip.base.LBP] <– The LBP extractors, read access only

image

array_like <2D, uint8> <– The (prepared) image the next features will be extracted from, read access only

load(hdf5) → None

Loads the extractors from the given HDF5 file

Parameters:

hdf5 : bob.ip.base.HDF5File

The file to read from

mean_variance(bounding_box[, compute_variance]) → mv

Computes the mean (and the variance) of the pixel gray values in the given bounding box

Parameters:

bounding_box : BoundingBox

The bounding box for which the mean (and variance) shoulf be calculated

compute_variance : bool

[Default: False] If enabled, the variance is computed as well; requires the compute_integral_square_image enabled in the prepare() function

Returns:

mv : float or (float, float)

The mean (or the mean and the variance) of the pixel gray values for the given bounding box

model_indices

array_like <1D, int32> <– The indices at which the features are extracted, read and write access

number_of_features

int <– The length of the feature vector that will be extracted by this class, read access only

number_of_labels

int <– The maximum label for the features in this class, read access only

offset(index) → offset

Get the offset position associated with the given feature index

Parameters:

index : int

The feature index for which the extractor should be retrieved

Returns:

offset : (int,int)

The offset position for the given feature index

patch_size

(int, int) <– The expected size of the patch that this extractor can handle, read access only

prepare(image, scale[, compute_integral_square_image]) → None

Take the given image to perform the next extraction steps for the given scale

If compute_integral_square_image is enabled, the (internally stored) integral square image is computed as well. This image is required to compute the variance of the pixels in a given patch, see mean_variance()

Parameters:

image : array_like <2D, uint8 or float>

The image that should be used in the next extraction step

scale : float

The scale of the image to extract

compute_integral_square_image : bool

[Default: False] : Enable the computation of the integral square image

save(hdf5) → None

Saves the extractors to the given HDF5 file

Parameters:

hdf5 : bob.ip.base.HDF5File

The file to write to

class bob.ip.facedetect.Sampler(patch_size=(24, 20), scale_factor=0.9576032806985737, lowest_scale=0.015625, distance=2)

This class generates (samples) bounding boxes for different scales and locations in the image.

It computes different scales of the image and provides a tight set of BoundingBox of a given patch size for the given image.

Constructor Documentation:

Generates a patch-sampler, which will scan images and sample bounding boxes.

Parameters:

patch_size : (int, int)

the size of the patch (i.e., the bounding box) to sample

scale_factor : float

image pyramids are computed using the given scale factor between two scales

lowest_scale : float or None

patches which will be lower than the given scale times the image resolution will not be taken into account; if 0. all possible patches will be considered

distance : int

the distance in both horizontal and vertical direction to generate samples

iterate(image, feature_extractor, feature_vector)

Scales the given image and extracts bounding boxes, computes the features for the given feature extractor and returns an ITERATOR returning a the bounding_box.

iterate_cascade(cascade, image, threshold=None)

Iterates over the given image and computes the cascade of classifiers.

sample(image)

Returns an iterator over all bounding boxes that are sampled for the given image.

sample_scaled(scaled_image_shape)

Returns an iterator that iterates over all sampled bounding boxes in the given (scaled) image shape.

scales(image)

Computes the all possible scales for the given image and returns a tuple of the scale and the scaled image shape as an iterator.

class bob.ip.facedetect.TrainingSet(feature_directory=None, list_file=None)

A set of images including bounding boxes that are used as a training set

add_from_db(database, files)

Adds image path and bounding boxes from the given annotations.

add_image(image_path, annotations)

extract(sampler, feature_extractor, number_of_examples_per_scale=None, similarity_thresholds=(0.5, 0.8), parallel=None, mirror=False, use_every_nth_negative_scale=1)

Extracts all features from all images in all scales and writes them to file.

similarity_thresholds : (float, float)

two patches will be compared: the (scaled) annotated patch and the (shifted) extracted patch if the similarity is lower than the first value of the similarity_thresholds tuple, it will be accepted as negative example, if the similarity is higher than the second value of the similarity_thresholds tuple, it will be accepted as positive example, otherwise the patch will be rejected.

mirror_samples : bool

extract also mirrored patches

feature_extractor()

Returns the feature extractor used to extract the positive and negative features.

iterate(max_number_of_files=None)

Returns the image name and the bounding boxes stored in the training set as an iterator.

load(list_file)

Loads the list of annotations from the given file and appends it to the current list.

sample(model=None, maximum_number_of_positives=None, maximum_number_of_negatives=None, positive_indices=None, negative_indices=None)

Returns positive and negative samples from the set of positives and negatives.

save(list_file)

Saves the current list of annotations to the given file.

bob.ip.facedetect.best_detection(detections, predictions[, minimum_overlap]) → bounding_box, prediction

Computes the best detection for the given detections and according predictions.

This is acheived by computing a weighted sum of detections that overlap with the best detection (the one with the highest prediction), where the weights are based on the predictions. Only detections with according prediction values > 0 are considered.

Keyword Parameters:

detections : [bob.ip.facedetect.BoundingBox]

The detected bounding boxes.

predictions : [float]

The predictions for the detections.

minimum_overlap : float between 0 and 1

The minimum overlap of bounding boxes with the best detection to be considered.

Returns:

bounding_box : bob.ip.facedetect.BoundingBox

The bounding box which has been merged from the detections

prediction : float

The prediction value of the bounding box, which is a weighted sum of the predictions with minimum overlap

bob.ip.facedetect.bounding_box_from_annotation(source=None, padding=None, **kwargs)

Creates a bounding box from the given parameters. If ‘sources’ are specified, the according keywords (see available_sources) must be given as well. Otherwise, the source is estimated from the given keyword parameters if possible.

If ‘topleft’ and ‘bottomright’ are given (i.e., the ‘direct’ source), they are taken as is. Note that the ‘bottomright’ is NOT included in the bounding box.

For source ‘ellipse’, the bounding box is computed to capture the whole ellipse, even if it is rotated.

For other sources (i.e., ‘eyes’), the center of the two given positions is computed, and the ‘padding’ is applied. If ‘padding ‘ is None (the default) then the default_paddings of this source are used instead.

bob.ip.facedetect.default_cascade()

Returns the Cascade that is loaded from the pre-trained cascade file provided by this package.

bob.ip.facedetect.detect_all_faces(image[, cascade][, sampler][, minimum_overlap]) → bounding_boxes, qualities

Detects a single face in the given image, i.e., the one with the highest prediction value.

Keyword Parameters:

image : array_like (2D aka gray or 3D aka RGB)

The image to detect a face in.

cascade : str or bob.ip.facedetect.Cascade or None

If given, the cascade file name or the loaded cascase to be used. If not given, the bob.ip.facedetect.default_cascade() is used.

sampler : bob.ip.facedetect.Sampler or None

The sampler that defines the sampling of bounding boxes to search for the face. If not specified, a default Sampler is instantiated.

threshold : float

The threshold of the quality of detected faces. Detections with a quality lower than this value will not be considered. Higher thresholds will not detect all faces, while lower thresholds will generate false detections.

minimum_overlap : float between 0 and 1

Computes the best detection using the given minimum overlap, see bob.ip.facedetect.best_detection()

Return value:

bounding_boxes : [bob.ip.facedetect.BoundingBox]

The bounding box containing the detected face.

quality : [float]

The quality of the detected face, a value greater than 0.

bob.ip.facedetect.detect_single_face(image[, cascade][, sampler][, minimum_overlap]) → bounding_box, quality

Detects a single face in the given image, i.e., the one with the highest prediction value.

Keyword Parameters:

image : array_like (2D aka gray or 3D aka RGB)

The image to detect a face in.

cascade : str or bob.ip.facedetect.Cascade or None

If given, the cascade file name or the loaded cascase to be used. If not given, the bob.ip.facedetect.default_cascade() is used.

sampler : bob.ip.facedetect.Sampler or None

The sampler that defines the sampling of bounding boxes to search for the face. If not specified, a default Sampler is instantiated, which will perform a tight sampling.

minimum_overlap : float between 0 and 1

Computes the best detection using the given minimum overlap, see bob.ip.facedetect.best_detection()

Return value:

bounding_box : bob.ip.facedetect.BoundingBox

The bounding box containing the detected face.

quality : float

The quality of the detected face, a value greater than 0.

bob.ip.facedetect.expected_eye_positions(bounding_box)

Computes the expected eye positions based on the relative coordinates of the bounding box.

bob.ip.facedetect.overlapping_detections(detections, predictions, threshold) → overlapped_detections, overlapped_predictions

Returns the detections and predictions that overlap with the best detection

For threshold >= 1., all detections will be returned (i.e., no pruning is performed), but the list will be sorted with descendingly predictions.

Parameters:

detections : [BoundingBox]

A list of detected bouding boxes

predictions : array_like <1D, float>

The prediction (quality, weight, ...) values for the detections

threshold : float

The overlap threshold (Jaccard similarity) which should be considered

Returns:

overlapped_detections : [BoundingBox]

The list of overlapping bounding boxes

overlapped_predictions : array_like <float, 1D>

The according predictions (qualities, weights, ...)

bob.ip.facedetect.prune_detections(detections, predictions, threshold[, number_of_detections]) → pruned_detections, pruned_predictions

Prunes the given detected bounding boxes according to their predictions and returns the pruned bounding boxes and their predictions

For threshold >= 1., all detections will be returned (i.e., no pruning is performed), but the list will be sorted with descendingly predictions.

Parameters:

detections : [BoundingBox]

A list of detected bouding boxes

predictions : array_like <1D, float>

The prediction (quality, weight, ...) values for the detections

threshold : float

The overlap threshold (Jaccard similarity), for which detections should be pruned

number_of_detections : int

[default: MAX_INT] The number of detections that should be returned

Returns:

pruned_detections : [BoundingBox]

The list of pruned bounding boxes

pruned_predictions : array_like <float, 1D>

The according predictions (qualities, weights, ...)

bob.ip.facedetect.read_annotation_file(annotation_file, annotation_type)

Reads annotations from the given annotation file.

The way, how annotations are read depends on the given annotation_type. Depending on the type, one or several annotations might be present in the annotation file. Currently, these variants are implemented:

• 'lr-eyes': Only the eye positions are stored, in a single row, like: le_x le_y re_x re_y, comment lines starting with '#' are ignored.
• 'named': Each line of the file contains a name and two floats, like reye x y; empty lines separate between sets of annotations.
• 'idiap': A special 22 point format, where each line contains the index and the locations, like 1 x y.
• 'fddb': a special format for the FDDB database; empty lines separate between sets of annotations

Finally, a list of annotations is returned in the format: [{name: (y,x)}].