7. Backend Maintenance

This guide contains information about administrative tasks related to the backend. Activities include cache clean-up actions and backend (workers, environments, queues) reconfiguration.

7.1. Configuration

The administrative command qsetup can be used to configure or re-configure the backend:

$ ./bin/django qsetup --help
usage: django qsetup [-h] [--dump] [--reset] [--version] [-v {0,1,2,3}]
                     [--settings SETTINGS] [--pythonpath PYTHONPATH]
                     [--traceback] [--no-color] [--force-color]
                     [--skip-checks]
                     [config]

Sets and resets queue configurations

positional arguments:
  config                Optional custom queue configuration to use. If not
                        passed, uses an internal default with a single
                        queue/worker/40 slots

optional arguments:
  -h, --help            show this help message and exit
  --dump                Dump all environment/worker/queues configuration and
                        exits
  --reset               Delete all environment/worker/queues before setting
                        the given configuration
  --version             show program's version number and exit
  -v {0,1,2,3}, --verbosity {0,1,2,3}
                        Verbosity level; 0=minimal output, 1=normal output,
                        2=verbose output, 3=very verbose output
  --settings SETTINGS   The Python path to a settings module, e.g.
                        "myproject.settings.main". If this isn't provided, the
                        DJANGO_SETTINGS_MODULE environment variable will be
                        used.
  --pythonpath PYTHONPATH
                        A directory to add to the Python path, e.g.
                        "/home/djangoprojects/myproject".
  --traceback           Raise on CommandError exceptions
  --no-color            Don't colorize the command output.
  --force-color         Force colorization of the command output.
  --skip-checks         Skip system checks.

This command will check current queue usage w.r.t. submitted experiments and will modify the queue/worker/environment configuration on-the-fly. Running schedulers and workers will not be affected.

By default, all new workers inserted are made inactive. This flag should be turned-on by the worker process itself once it is started.

7.2. Cache Maintenance

If your cache becomes very large, you may have to delete old files. The administative command cleanup_cache can help you in that task:

$ ./bin/django cleanup_cache --help
usage: django cleanup_cache [-h] [--olderthan MINUTES] [--delete]
                            [--path PATH] [--version] [-v {0,1,2,3}]
                            [--settings SETTINGS] [--pythonpath PYTHONPATH]
                            [--traceback] [--no-color] [--force-color]
                            [--skip-checks]

Cleans-up the cache, removing old files

optional arguments:
  -h, --help            show this help message and exit
  --olderthan MINUTES   All files which are older than this value in *minutes*
                        and are not locked or being used by active experiments
                        (running or scheduled) will be deleted [default: 0]
  --delete              By default we only list cache files that will be
                        erased. If you pass this flag, then we really erase
                        them
  --path PATH           By default, we erase the CACHE path on your settings.
                        Set this flag if you want to operate on a different
                        path [default:
                        /scratch/builds/iJyPBhV4/1/beat/beat.web/prefix/cache]
  --version             show program's version number and exit
  -v {0,1,2,3}, --verbosity {0,1,2,3}
                        Verbosity level; 0=minimal output, 1=normal output,
                        2=verbose output, 3=very verbose output
  --settings SETTINGS   The Python path to a settings module, e.g.
                        "myproject.settings.main". If this isn't provided, the
                        DJANGO_SETTINGS_MODULE environment variable will be
                        used.
  --pythonpath PYTHONPATH
                        A directory to add to the Python path, e.g.
                        "/home/djangoprojects/myproject".
  --traceback           Raise on CommandError exceptions
  --no-color            Don't colorize the command output.
  --force-color         Force colorization of the command output.
  --skip-checks         Skip system checks.

This command will first check running experiments and database files and make sure not to remove any files currently used by those are removed. You may then specify the amount of time in minutes from the current time, cache files should be considered outdated and must be removed.

Notice the command, by default, does not remove any files - it only lists the files it would remove. In order to actually remove the files, you must also specify the option --delete.