Li’s CVPR14 Python API¶
Signals extraction¶

bob.rppg.cvpr14.extract_utils.
kp66_to_mask
(image, keypoints, indent=10, plot=False)[source]¶ builds a mask on the lower part of the face
The mask is built using selected keypoints retrieved by a Discriminative Response Map Fitting (DRMF) algorithm. Note that the DRMF is not implemented here, and that the keypoints are loaded from file (and are not provided in the package).
Note also that this function is explicitly made for the keypoints set generated by the Matlab software downloaded from http://ibug.doc.ic.ac.uk/resources/drmfmatlabcodecvpr2013/
Update: this function also works when using
bob.ip.dlib.DlibLandmarkExtraction
 Parameters
image (numpy.ndarray) – The current frame.
keypoints (numpy.ndarray) – the set of 66 keypoints retrieved by DRMF.
indent (int) – The percentage of the facewidth [in pixels] by which selected keypoints are shifted inside the face to build the mask. THe facewidth is defined by the distance between the two keypoints located on the right and left edge of the face, at the eyes’ height.
plot (bool) – If set to True, plots the current face with the selected keypoints and the built mask.
 Returns

bob.rppg.cvpr14.extract_utils.
get_mask
(image, mask_points)[source]¶ returns a boolean array where the mask is True.
It turns mask points into a region of interest and returns the corresponding boolean array, of the same size as the image.
Taken from https://github.com/jdoepfert/roipoly.py/blob/master/roipoly.py
 Parameters
image (numpy.ndarray) – The current frame.
mask_points (
list
oftuple
) – The points corresponding to vertices of the mask.
 Returns
mask – A boolean array of the size of the original image, where the region corresponding to the mask is True.
 Return type

bob.rppg.cvpr14.extract_utils.
get_good_features_to_track
(face, npoints, quality=0.01, min_distance=10, plot=False)[source]¶ applies the openCV function “good features to track”
 Parameters
face (numpy.ndarray) – The cropped face image
npoints (int) – The maximum number of strong corners you want to detect
quality (
float
) – The minimum relative quality of the detected corners. Note that increasing this value decreases the number of detected corners. Defaluts to 0.01.min_distance (int) – minimum euclidean distance between detected corners.
plot (bool) – if we should plot the currently selected features to track.
 Returns
corners – the detected strong corners.
 Return type

bob.rppg.cvpr14.extract_utils.
track_features
(previous, current, previous_points, plot=False)[source]¶ projects the features from the previous frame in the current frame.
 Parameters
previous (numpy.ndarray) – the previous frame.
current (numpy.ndarray) – the current frame.
previous_points (numpy.ndarray) – the set of keypoints to track (in the previous frame).
plot (bool) – Plots the keypoints projected on the current frame.
 Returns
current_points – the set of keypoints in the current frame.
 Return type

bob.rppg.cvpr14.extract_utils.
find_transformation
(previous_points, current_points)[source]¶ finds the transformation matrix from previous points to current points.
The transformation matrix is found using estimateRigidTransform (fancier alternatives have been tried, but are not that stable).
 Parameters
previous_points (numpy.ndarray) – Set of ‘starting’ 2d points
current_points (numpy.ndarray) – Set of ‘destination’ 2d points
 Returns
transformation_matrix – the affine transformation matrix between the two sets of points.
 Return type

bob.rppg.cvpr14.extract_utils.
get_current_mask_points
(previous_mask_points, transfo_matrix)[source]¶ projects the previous mask points to get the current mask.
 Parameters
previous_mask_points (numpy.ndarray) – The points forming the mask in the previous frame
transformation_matrix (numpy.ndarray) – the affine transformation matrix between the two sets of points.
 Returns
current_mask_points – The points forming the mask in the current frame
 Return type

bob.rppg.cvpr14.extract_utils.
compute_average_colors_mask
(image, mask, plot=False)[source]¶ computes the average green color within a given mask.
 Parameters
image (numpy.ndarray) – The image containing the face.
mask (numpy.ndarray) – A boolean array of the size of the original image, where the region corresponding to the mask is True.
plot (bool) – Plot the mask as an overlay on the original image.
 Returns
color – The average RGB colors inside the mask ROI.
 Return type

bob.rppg.cvpr14.extract_utils.
compute_average_colors_wholeface
(image, plot=False)[source]¶ computes the average green color within the provided face image
 Parameters
image (numpy.ndarray) – The cropped face image
plot (bool) – Plot the mask as an overlay on the original image.
 Returns
color – The average green color inside the face
 Return type
Illumination rectification¶

bob.rppg.cvpr14.illum_utils.
rectify_illumination
(face_color, bg_color, step, length)[source]¶ performs illumination rectification.
The correction is made on the face green values using the background green values, so as to remove global illumination variations in the face green color signal.
 Parameters
face_color (numpy.ndarray) – The mean green value of the face across the video sequence.
bg_color (numpy.ndarray) – The mean green value of the background across the video sequence.
step (float) – Step size in the filter’s weight adaptation.
length (int) – Length of the filter.
 Returns
rectified color – The mean green values of the face, corrected for illumination variations.
 Return type

bob.rppg.cvpr14.illum_utils.
nlms
(signal, desired_signal, n_filter_taps, step, initCoeffs=None, adapt=True)[source]¶ Normalized least mean square filter.
Based on adaptfilt 0.2: https://pypi.python.org/pypi/adaptfilt/0.2
 Parameters
signal (numpy.ndarray) – The signal to be filtered.
desired_signal (numpy.ndarray) – The target signal.
n_filter_taps (int) – The number of filter taps (related to the filter order).
step (float) – Adaptation step for the filter weights.
initCoeffs (numpy.ndarray) – Initial values for the weights. Defaults to zero.
adapt (bool) – If True, adapt the filter weights. If False, only filters.
 Returns
y (numpy.ndarray) – The filtered signal.
e (numpy.ndarray) – The error signal (difference between filtered and desired)
w (numpy.ndarray) – The found weights of the filter.
Motion correction¶

bob.rppg.cvpr14.motion_utils.
build_segments
(signal, length)[source]¶ builds an array containing segments of the signal.
The signal is divided into segments of provided length (no overlap) and the different segments are stacked.
 Parameters
signal (numpy.ndarray) – The signal to be processed.
length (int) – The length of the segments.
 Returns
segments (numpy.ndarray) – the segments composing the signal.
end_index (int) – The length of the signal (there may be a trail smaller than a segment at the end of the signal, that will be discarded).

bob.rppg.cvpr14.motion_utils.
prune_segments
(segments, threshold)[source]¶ remove segments.
Segments are removed if their standard deviation is higher than the provided threshold.
 Parameters
segments (numpy.ndarray) – The set of segments.
threshold (float) – Threshold on the standard deviation.
 Returns
pruned_segments (numpy.ndarray) – The set of “stable” segments.
gaps (list of length (# of retained segments)) – Boolean list that tells if a gap should be accounted for when building the final signal.
cut_index (list of tuples) – Contains the start and end index of each removed segment. Used for plotting purposes.

bob.rppg.cvpr14.motion_utils.
build_final_signal
(segments, gaps)[source]¶ builds the final signal with remaining segments.
 Parameters
segments (numpy.ndarray) – The set of remaining segments.
gaps (list) – Boolean list that tells if a gap should be accounted for when building the final signal.
 Returns
final_signal – The final signal.
 Return type

bob.rppg.cvpr14.motion_utils.
build_final_signal_cvpr14
(segments, gaps)[source]¶ builds the final signal with remaining segments.
Warning
This contains a bug !
Builds the final signal, but reproducing the bug found in the code provided by the authors of [licvpr2014]. The bug is in the ‘collage’ of remaining segments. The gap is not always properly accounted for…
 Parameters
segments (numpy.ndarray) – The set of remaining segments.
gaps (list) – Boolean list that tells if a gap should be accounted for when building the final signal.
 Returns
final_signal – The final signal.
 Return type
Filtering¶

bob.rppg.cvpr14.filter_utils.
detrend
(signal, Lambda)[source]¶ applies a detrending filter.
This code is based on the following article “An advanced detrending method with application to HRV analysis”. Tarvainen et al., IEEE Trans on Biomedical Engineering, 2002.
 Parameters
signal (numpy.ndarray) – The signal where you want to remove the trend.
Lambda (int) – The smoothing parameter.
 Returns
filtered_signal – The detrended signal.
 Return type

bob.rppg.cvpr14.filter_utils.
average
(signal, window_size)[source]¶ Moving average filter.
 Parameters
signal (numpy.ndarray) – The signal to filter.
window_size (int) – The size of the window to compute the average.
 Returns
filtered_signal – The averaged signal.
 Return type