# Support Vector Machines and Trainers¶

A **Support vector machine** (SVM) 1 is a very popular supervised learning
technique. Bob provides a bridge to LIBSVM which allows you to train
such a machine and use it for classification. This section contains a
tutorial on how to use Bob’s Pythonic bindings to LIBSVM. It starts by
introducing the support vector `bob.learn.libsvm.Machine`

followed
by the trainer usage.

## Machines¶

The functionality of this bridge includes loading and saving SVM data files and
machine models, which you can produce or download following the instructions
found on LIBSVM’s home page. Bob bindings to LIBSVM **do not**
allow you to explicitly set the machine’s internal values. You must use the
a trainer to create a machine first, as explained further down. Once you have
followed those instructions, you can come back to this point and follow the
remaining examples here.

Note

Our current `svm`

object was trained with the file called `heart_scale`

,
distributed with LIBSVM and available here.
This dataset proposes a binary classification problem (i.e., 2 classes of
features to be discriminated). The number of features is 13.

Our extensions to LIBSVM also allows you to feed data through a
`bob.learn.libsvm.Machine`

using `numpy.ndarray`

objects
and collect results in that format. For the following lines, we assume you have
available a `bob.learn.libsvm.Machine`

named `svm`

. (For this
example, the variable `svm`

was generated from the `heart_scale`

dataset
using the application `svm-train`

with default parameters). The `shape`

attribute, indicates how many features a machine from this module can input and
how many it outputs (typically, just 1):

```
>>> svm.shape
(13, 1)
```

To run a single example through the SVM, just use the `()`

operator:

```
>> svm(numpy.ones((13,), 'float64'))
1
>> svm(numpy.ones((10,13), 'float64'))
(1, 1, 1, 1, 1, 1, 1, 1, 1, 1)
```

Visit the documentation for `bob.learn.libsvm.Machine`

to find more
information about these bindings and methods you can call on such a machine.
Visit the documentation for `bob.learn.libsvm.File`

for information
on loading LIBSVM data files directly into python and producing
`numpy.ndarray`

objects.

Below is a quick example: Suppose the variable `f`

contains an object of
type `bob.learn.libsvm.File`

. Then, you could read data (and labels)
from the file like this:

```
>>> labels, data = f.read_all()
>>> data = numpy.vstack(data) #creates a single 2D array
```

Then you can throw the data into the `svm`

machine you trained earlier like
this:

```
>>> predicted_labels = svm(data)
```

## Training¶

The training set for SVM’s consists of a list of 2D NumPy arrays, one for each class. The first dimension of each 2D NumPy array is the number of training samples for the given class and the second dimension is the dimensionality of the feature. For instance, let’s consider the following training set for a two class problem:

```
>>> pos = numpy.array([[1,-1,1], [0.5,-0.5,0.5], [0.75,-0.75,0.8]], 'float64')
>>> neg = numpy.array([[-1,1,-0.75], [-0.25,0.5,-0.8]], 'float64')
>>> data = [pos,neg]
>>> print(data)
```

Note

Please note that in the above training set, the data is pre-scaled so features remain in the range between -1 and +1. libsvm, apparently, suggests you do that for all features. Our bindings to libsvm do not include scaling. If you want to implement that generically, please do it.

Then, an SVM 1 can be trained easily using the
`bob.learn.libsvm.Trainer`

class.

```
>>> trainer = bob.learn.libsvm.Trainer()
>>> machine = trainer.train(data) #ordering only affects labels
```

This returns a `bob.learn.libsvm.Machine`

which can later be used
for classification, as explained before.

```
>>> predicted_label = machine(numpy.array([1.,-1.,1.]))
>>> print(predicted_label)
[1]
```

The training procedure allows setting several different options. For
instance, the default kernel is an RBF. If we would like a linear SVM
instead, this can be set before calling the
`bob.learn.libsvm.Trainer.train()`

method.

```
>>> trainer.kernel_type = 'LINEAR'
```

### One Class SVM¶

On the other hand, the package allows you to train a One Class Support Vector Machine. For training this kind of classifier take into account the following example.

```
>>> oc_pos = 0.4 * numpy.random.randn(100, 2).astype(numpy.float64)
>>> oc_data = [oc_pos]
>>> print(oc_data)
```

As the above example, an SVM 1 for one class problem can be trained easily using the
`bob.learn.libsvm.Trainer`

class and selecting the appropiete machine_type (ONE_CLASS).

```
>>> oc_trainer = bob.learn.libsvm.Trainer(machine_type='ONE_CLASS')
>>> oc_machine = oc_trainer.train(oc_data)
```

Then, as explained before, a `bob.learn.libsvm.Machine`

can be used for classify the new entries.

```
>>> oc_test = 0.4 * numpy.random.randn(20, 2).astype(numpy.float64)
>>> oc_outliers = numpy.random.uniform(low=-4, high=4, size=(20, 2)).astype(numpy.float64)
>>> predicted_label_oc_test = oc_machine(oc_test)
>>> predicted_label_oc_outliers = oc_machine(oc_outliers)
>>> print(predicted_label_oc_test)
>>> print(predicted_label_oc_outliers)
```

## Acknowledgements¶

As a final note, if you decide to use our LIBSVM bindings for your publication, be sure to also cite:

```
@article{CC01a,
author = {Chang, Chih-Chung and Lin, Chih-Jen},
title = {{LIBSVM}: A library for support vector machines},
journal = {ACM Transactions on Intelligent Systems and Technology},
volume = {2},
issue = {3},
year = {2011},
pages = {27:1--27:27},
note = {Software available at \url{http://www.csie.ntu.edu.tw/~cjlin/libsvm}}
}
```