=======================================================
Running baseline experiments in Cross database settings
=======================================================

This section explains how to run and evaluate the cross database experiments in OULU-NPU database and REPLAY-MOBILE database. 

.. note::

  For the experiments discussed in this section, the databases OULU-NPU and REPLAY-MOBILE needs to be downloaded and installed in your system. Please refer to :ref:`bob.pad.face.baselines` section in the documentation of ``bob.pad.face`` package for more details on how to run the face PAD experiments and setup the databases.

Training on REPLAY-MOBILE and testing on OULU-Protocol_1
========================================================


1. IQM-SVM train on REPLAY-MOBILE test on OULU-Protocol_1
---------------------------------------------------------


The entire pipeline for IQM-SVM on ``grandtest`` of REPLAY-MOBILE database can be launched with the following command

.. code-block:: sh

    bin/spoof.py \                                              # spoof.py is used to run the preprocessor
    replay-mobile \                                             # run for Replay-Mobile database
    iqm-svm \                                                   # IQM-SVM configuration
    --groups train dev eval \                                   # groups
    --protocol grandtest  \                                     # Protocol to use
    --allow-missing-files \                                     # allow failed files
    --grid idiap \                                              # use grid, only for Idiap users, REMOVE otherwise
    --sub-directory <PATH_TO_STORE_IQM_RM_BASELINE_RESULTS>        # define your path here

Now to test this trained model on OULU-Protocol; run the command below

.. code-block:: sh

    bin/spoof.py \
    oulunpu \
    iqm-svm \
    --groups train dev eval \
    --protocol Protocol_1 \
    --skip-projector-training \
    --projector-file <PATH_TO_STORE_IQM_RM_BASELINE_RESULTS>/grandtest/Projector.hdf5 \
    --allow-missing-files \
    --grid idiap \
    --sub-directory <PATH_TO_STORE_IQM_TRAIN_RM_TEST_OULU> -vv


2. LBP-SVM train on REPLAY-MOBILE test on OULU-Protocol_1
---------------------------------------------------------


The entire pipeline for LBP-SVM on ``grandtest`` of REPLAY-MOBILE database can be launched with the following command

.. code-block:: sh

    bin/spoof.py \                                              # spoof.py is used to run the preprocessor
    replay-mobile \                                             # run for Replay-Mobile database
    glbp-svm \                                                  # LBP-SVM configuration
    --groups train dev eval \                                   # groups
    --protocol grandtest  \                                     # Protocol to use
    --allow-missing-files \                                     # allow failed files
    --grid idiap \                                              # use grid, only for Idiap users, REMOVE otherwise
    --sub-directory <PATH_TO_STORE_LBP_RM_BASELINE_RESULTS>        # define your path here

Now to test this trained model on OULU-Protocol; run the command below

.. code-block:: sh

    bin/spoof.py \
    oulunpu \
    qlbp-svm \
    --groups train dev eval \
    --protocol Protocol_1 \
    --skip-projector-training \
    --projector-file <PATH_TO_STORE_LBP_RM_BASELINE_RESULTS>/grandtest/Projector.hdf5 \
    --allow-missing-files \
    --grid idiap \
    --sub-directory <PATH_TO_STORE_LBP_TRAIN_RM_TEST_OULU> -vv



3. Evaluating results of face PAD Experiments
---------------------------------------------

The scores obtained can be evaluated with the following command. 

For LBP baselines

.. code-block:: sh

    bob pad metrics \
    <PATH_TO_STORE_LBP_TRAIN_RM_TEST_OULU>/scores/scores-dev \
    <PATH_TO_STORE_LBP_TRAIN_RM_TEST_OULU>/scores/scores-eval -e \

For IQM baselines

.. code-block:: sh

    bob pad metrics \
    <PATH_TO_STORE_IQM_TRAIN_RM_TEST_OULU>/scores/scores-dev \
    <PATH_TO_STORE_IQM_TRAIN_RM_TEST_OULU>/scores/scores-eval -e \


Training on OULU-Protocol_1 and and testing on REPLAY-MOBILE
============================================================


1. IQM-SVM train on OULU-Protocol_1 and and testing on REPLAY-MOBILE
--------------------------------------------------------------------


The entire pipeline for IQM-SVM on ``Protocol_1`` of OULU database can be launched with the following command

.. code-block:: sh

    bin/spoof.py \                                              # spoof.py is used to run the preprocessor
    oulunpu \                                                   # run for Replay-Mobile database
    iqm-svm \                                                   # IQM-SVM configuration
    --groups train dev eval \                                   # groups
    --protocol Protocol_1 \                                     # Protocol to use
    --allow-missing-files \                                     # allow failed files
    --grid idiap \                                              # use grid, only for Idiap users, REMOVE otherwise
    --sub-directory <PATH_TO_STORE_IQM_OULU_BASELINE_RESULTS>        # define your path here

Now to test this trained model on OULU-Protocol; run the command below

.. code-block:: sh

    bin/spoof.py \
    replay-mobile \
    iqm-svm \
    --groups train dev eval \
    --protocol grandtest   \
    --skip-projector-training \
    --projector-file <PATH_TO_STORE_IQM_OULU_BASELINE_RESULTS>/Protocol_1/Projector.hdf5 \
    --allow-missing-files \
    --grid idiap \
    --sub-directory <PATH_TO_STORE_IQM_TRAIN_OULU_TEST_RM> -vv


2. LBP-SVM train on OULU-Protocol_1 and and testing on REPLAY-MOBILE
--------------------------------------------------------------------



The entire pipeline for LBP-SVM on ``Protocol_1`` of OULU database can be launched with the following command

.. code-block:: sh

    bin/spoof.py \                                              # spoof.py is used to run the preprocessor
    oulunpu \                                                   # run for Replay-Mobile database
    glbp-svm \                                                  # LBP-SVM configuration
    --groups train dev eval \                                   # groups
    --protocol Protocol_1 \                                     # Protocol to use
    --allow-missing-files \                                     # allow failed files
    --grid idiap \                                              # use grid, only for Idiap users, REMOVE otherwise
    --sub-directory <PATH_TO_STORE_LBP_OULU_BASELINE_RESULTS>        # define your path here

Now to test this trained model on OULU-Protocol; run the command below

.. code-block:: sh

    bin/spoof.py \
    replay-mobile \
    glbp-svm \
    --groups train dev eval \
    --protocol grandtest   \
    --skip-projector-training \
    --projector-file <PATH_TO_STORE_LBP_OULU_BASELINE_RESULTS>/Protocol_1/Projector.hdf5 \
    --allow-missing-files \
    --grid idiap \
    --sub-directory <PATH_TO_STORE_LBP_TRAIN_OULU_TEST_RM> -vv



3. Evaluating results of face PAD Experiments
---------------------------------------------

The scores obtained can be evaluated with the following command. 

For LBP baselines

.. code-block:: sh

    bob pad metrics \
    <PATH_TO_STORE_LBP_TRAIN_OULU_TEST_RM>/scores/scores-dev \
    <PATH_TO_STORE_LBP_TRAIN_OULU_TEST_RM>/scores/scores-eval -e \

For IQM baselines

.. code-block:: sh

    bob pad metrics \
    <PATH_TO_STORE_IQM_TRAIN_OULU_TEST_RM>/scores/scores-dev \
    <PATH_TO_STORE_IQM_TRAIN_OULU_TEST_RM>/scores/scores-eval -e \