Databases

This package describes the database API for Bob. Database APIs establish how your programs can query for file lists using known pre-coded protocols that assure reproducibility. This package contains only the base API for you to create and distribute new databases and a single, very simple example using the publicly available Iris Flower Dataset.

Build a database package for Bob goes pretty much like building a satellite package. For examples and details, have a look at our satellite package portal.

The db package contains simplified APIs to access data for various databases that can be used in Biometry, Machine Learning or Pattern Classification.

General Utilities

Some utilities shared by many of the databases.

class bob.db.utils.null

Bases: object

A look-alike stream that discards the input

write(s)

Writes contents of string s on this stream

flush()

Flushes the stream

bob.db.utils.apsw_is_available()

Checks lock-ability for SQLite on the current file system

class bob.db.utils.SQLiteConnector(filename, readonly=False, lock=None)

Bases: object

An object that handles the connection to SQLite databases.

Initializes the connector

Keyword arguments

filename

The name of the file containing the SQLite database

readonly

Should I try and open the database in read-only mode?

lock

Any vfs name as output by apsw.vfsnames()

static filesystem_is_lockable(database)

Checks if the filesystem is lockable

APSW_IS_AVAILABLE = True

create_engine(echo=False)

Returns an SQLAlchemy engine

session(echo=False)

Returns an SQLAlchemy session

bob.db.utils.session(dbtype, dbfile, echo=False)

Creates a session to an SQLite database

bob.db.utils.session_try_readonly(dbtype, dbfile, echo=False)

Creates a read-only session to an SQLite database. If read-only sessions are not supported by the underlying sqlite3 python DB driver, then a normal session is returned. A warning is emitted in case the underlying filesystem does not support locking properly.

Raises a NotImplementedError if the dbtype is not supported.

bob.db.utils.create_engine_try_nolock(dbtype, dbfile, echo=False)

Creates an engine connected to an SQLite database with no locks. If engines without locks are not supported by the underlying sqlite3 python DB driver, then a normal engine is returned. A warning is emitted if the underlying filesystem does not support locking properly in this case.

Raises a NotImplementedError if the dbtype is not supported.

bob.db.utils.session_try_nolock(dbtype, dbfile, echo=False)

Creates a session to an SQLite database with no locks. If sessions without locks are not supported by the underlying sqlite3 python DB driver, then a normal session is returned. A warning is emitted if the underlying filesystem does not support locking properly in this case.

Raises a NotImplementedError if the dbtype is not supported.

bob.db.utils.connection_string(dbtype, dbfile, opts={})

Returns a connection string for supported platforms

Keyword parameters

dbtype

The type of database (only ‘sqlite’ is supported for the time being)

dbfile

The location of the file to be used

Central Database Manager API

This module defines, among other less important constructions, a management interface that can be used by Bob to display information about the database and manage installed files.

class bob.db.driver.Interface

Bases: object

Base manager for Bob databases

name()

Returns a simple name for this database, w/o funny characters, spaces

files()

Returns a python iterable with all auxiliary files needed.

The values should be take w.r.t. where the python file that declares the database is sitting at.

version()

Returns the current version number defined in setup.py

type()

Returns the type of auxiliary files you have for this database

If you return ‘sqlite’, then we append special actions such as ‘dbshell’ on ‘bob_dbmanage.py’ automatically for you. Otherwise, we don’t.

If you use auxiliary text files, just return ‘text’. We may provide special services for those types in the future.

Use the special name ‘builtin’ if this database is an integral part of Bob.

setup_parser(parser, short_description, long_description)

Sets up the base parser for this database.

Keyword arguments:

short_description

A short description (one-liner) for this database

long_description

A more involved explanation of this database

Returns a subparser, ready to be added commands on

add_commands(parser)

Adds commands to a given (argparse) parser.

This method, effectively, allows you to define special commands that your database will be able to perform when called from the common driver like for example create or checkfiles.

You are not obliged to overwrite this method. If you do, you will have the chance to establish your own commands. You don’t have to worry about stock commands such as files or version. They will be automatically hooked-in depending on the values you return for type() and files().

Keyword arguments

parser

An instance of a argparse.Parser that you can customize, i.e., call add_argument() on.

Iris Flower Dataset

The Iris flower data set or Fisher’s Iris data set is a multivariate data set introduced by Sir Ronald Aylmer Fisher (1936) as an example of discriminant analysis. It is sometimes called Anderson’s Iris data set because Edgar Anderson collected the data to quantify the geographic variation of Iris flowers in the Gaspe Peninsula.

For more information: http://en.wikipedia.org/wiki/Iris_flower_data_set

References:

1. Fisher,R.A. “The use of multiple measurements in taxonomic problems”, Annual Eugenics, 7, Part II, 179-188 (1936); also in “Contributions to Mathematical Statistics” (John Wiley, NY, 1950).

2. Duda,R.O., & Hart,P.E. (1973) Pattern Classification and Scene Analysis. (Q327.D83) John Wiley & Sons. ISBN 0-471-22361-1. See page 218.

3. Dasarathy, B.V. (1980) “Nosing Around the Neighborhood: A New System Structure and Classification Rule for Recognition in Partially Exposed Environments”. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. PAMI-2, No. 1, 67-71.

4. Gates, G.W. (1972) “The Reduced Nearest Neighbor Rule”. IEEE Transactions on Information Theory, May 1972, 431-433.

bob.db.iris.names = ['Sepal Length', 'Sepal Width', 'Petal Length', 'Petal Width']

Names of the features for each entry in the dataset.

bob.db.iris.stats = {'Petal Length': [1.0, 6.9, 3.76, 1.76, 0.949], 'Petal Width': [0.1, 2.5, 1.2, 0.76, 0.9565], 'Sepal Width': [2.0, 4.4, 3.05, 0.43, -0.4194], 'Sepal Length': [4.3, 7.9, 5.84, 0.83, 0.7826]}

These are basic statistics for each of the features in the whole dataset.

bob.db.iris.stat_names = ['Minimum', 'Maximum', 'Mean', 'Std.Dev.', 'Correlation']

These are the statistics available in each column of the stats variable.

bob.db.iris.data()

Loads from (text) file and returns Fisher’s Iris Dataset.

This set is small and simple enough to require an SQL backend. We keep the single file it has in text and load it on-the-fly every time this method is called.

We return a dictionary containing the 3 classes of Iris plants catalogued in this dataset. Each dictionary entry contains an 2D numpy.ndarray of 64-bit floats and 50 entries. Each entry is an Array with 4 features as described by “names”.