13. Frequently Asked Questions

13.1. What kind of experiments does the BEAT platform support?

The BEAT platform can run software-based experiments on any kind of raw data, for as long as the following broad conditions are met:

  1. You can define a dataset for the evaluation task, with usage protocols for its data (e.g. train and test subsets)

  2. You can read the raw data from the dataset using Python (any further dependence on external packages is OK)

  3. You can define a workflow (or a toolchain in BEAT jargon, see Toolchains) that express at least in one way to perform the evaluation task

  4. You can define a metric for the performance evaluation of the task

  5. You can write Algorithms for any of the available environments to create valid Experiments for the task at hand.

13.2. I can’t write Python code. Can I insert <my preferred programming language> code instead?

In principle, yes. In practice, this is currently not possible at our instance of the platform.

More information: Our backend executes user algorithms in a caged-environment, which is swappable. Each block is executed as an isolated process, which communicates with the backend processing infrastructure through an agnostic message passing library called ZeroMQ. For as long as you can write the API for the user algorithms that is able to communicate with the backend, then you could enable users to program using that new language.

13.3. I want to write a new backend for <my preferred programming language>. How should I proceed?

Start by contacting our development mailing list for help on how to proceed.

13.4. Can I run a version of the BEAT platform on my premises?

Yes, you can download the core components of the BEAT platform and install it on your servers. This version will, of course, not contain algorithms, datasets, toolchains and experiments available on other instances of the BEAT platform.

The source-code is at our Gitlab repository.

You may contact our development mailing list for help to install the platform, in case you don’t manage it from the provided instructions.

13.5. I’m getting an error, or the platform is not functioning as expected, or the platform does not do something I need. How can I submit a feature request or a bug report?

It is great you asked! Always start by posting your question at our development mailing list. If the new feature or bug report is confirmed, we will add it to our bug tracker. This strategy allows us to help you in case what you want to do is already possible!

Additionally, sometimes after updates to the BEAT Platform, there are still old files left in your browser’s cache. Clearing your caching and reloading the page may fix the issue!

13.6. How can I have my dataset available on your platform?

There are 4 steps:

  1. License agreement

    Formally we need to sign your End User License Agreement (EULA) explicitly mentioning the use on the BEAT platform – ideally this EULA should contain this statement or similar:

    Subject to prior identification of the End User and signature of this License, the Corpus is available to the benefit of the End User. The Licensor grants to the End User the right to use the Corpus, for use on the BEAT platform as described in the terms of service and for the purpose of scientific research only.

    The Licensor grants to the End User the right to reproduce temporarily, to adapt, arrange and modify the Corpus for use on the BEAT platform. The Licensor grants to the End User the right to rework and build upon the Corpus, or any component thereof, as necessary or desirable for research or technology development activity and create derivative products or services for the End User’s use of the Corpus on the BEAT platform. The End User is permitted to make a copy of the Corpus for archiving only.

    This License gives no right of any kind to the End User over the Corpus. The License is deemed non-exclusive and non-transferable to third parties.

    This step allow us to download and copy the dataset on the platform disks.

  2. Bob database interface and protocol

    The BEAT platform is using some elements of the Bob library and, in particular, Bob-styled database interfaces. That allows to describe data accessors and an experiment protocol (e.g, train/dev/test subsets). Some examples:

    Someone (you) needs to write a simple piece of python code for that. Support and examples can be provided on request.

  3. The BEAT platform needs a small python interface to know how to read the data from the disk (through the bob.db accessor) and feed it into the toolchains during experiment execution.

    Someone (you) needs to write a simple piece of python code for that. Support and examples can be provided on request.

  4. The final step is to provide examples to others on how to use the database for projects, lab exercises and/or challenges/competitions.

    Someone (you) needs to write an example toolchain that can execute on your database to produce example results.

On request we may be able to complete steps 2-4 for you.

13.7. Are there restrictions on dataset size? Are you ready for Big-Data?

There are no restrictions on dataset size, for as long as we can store it on the server and read the data for executing the experiments. See above for instructions on how to request the inclusion of a dataset in an instance of the BEAT platform.

13.8. The computing power is too limited for my needs. Can I get more CPU/Memory/Processing Time?

The BEAT platform is capable of controlling computing power that is allocated to users or groups of users running experiments. It is possible to assign exclusive or non-exclusive usage of computing resources on demand. Extra-computing power for a particular purpose may be available on request. You may contact the specific platform instance administrators for more information.

13.9. Why can’t I un-share a contribution of mine in the platform?

Sharing at the BEAT platform is an irreversible procedure. For example, public objects cannot be made private again. If you share an object with a user or team and change your mind, you can still delete the object, for as long as it is not being used by you or another colleagues with access.

This is implemented this way to avoid complications when users use objects that where shared with them, but then these objects disappear as the result of a un-sharing action.

Suppose you’re using algorithm user/algo/1, that was shared with you. You run an experiment with it, things look good. Next day you come back and you realize you cannot re-run the same experiment because user/algo/1 is gone. It disappeared from your list because the user user decided to un-share it with you. Now you have an irreproducible experiment at hand, one with results and so on, but without means to repeat it. In a way, un-sharing is equivalent to deleting the contribution for a particular user. In order to guarantee reproducibility, the platform must ensure that, once you used a contribution, it is not going away. If the contribution appeared to you as the result of sharing by another user, then only by making it irreversible, is the platform able to guarantee reproducibility.

13.10. What is unique about the BEAT platform? Why shouldn’t I use Spark/Hadoop/HDFS/<solution of choice here> instead?

Conducting analysis and research is not just about running an application. While these frameworks can do this much, you’re still left with huge tasks at hand: porting applications and learning techniques, enforcing protocol usage, publishing and comparing results. The BEAT platform provides an answer to those issues.

You can write applications using the native framework you’re used to - if you’d like to use Spark, so be it. If another framework, we can install it for you. No need to port the core of your work to another language or re-implement it, just slightly adapt your algorithm I/O to work against the BEAT stack. Once you encode the algorithm, the platform can parallelize it automatically.

Once the experiment is done, it is directly comparable to other experiments available in the platform. No need to worry about constraints - they have all been enforced by the platform already. Just sit back and prepare reports as you wish to analyze results. Reports are readily available on the internet so they can be easily shared with other parties. Reports and Attestations guarantee experiments in the platform are reproducible.

Want to write code in <language of choice here>? Not a problem! The BEAT platform can handle algorithms in different programming languages (for as long as there is a compatible backend installed). An experiment can be formed by putting together the best of each world. Feature extractors written in C, machine learning code written in Python and running on GPUs - you name it.

That said, it is possible to run the BEAT platform on any cloud provider and have your algorithms using whatever frameworks you’d like to. Just make sure to respect legal constraints on privacy-sensitive datasets, if that is the case for you.

13.11. Can the platform run on a cloud provider?

Yes, it is possible to make the BEAT platform to run on Amazon EC2 (or the like) for as long as legal constraints are respected. Services like this are just a virtualization solution. Your datasets can be stored on Amazon S3 as well. The only requirement is that the backend can access them transparently.

13.12. Can I install the BEAT platform on my premises? Is it free?

Yes, you can. We distribute the BEAT platform code as an open-source project, under the GNU Affero GPL v3 license. To get started, download the package beat.web and read its README.rst file located on its root for quick start instructions:

git clone https://gitlab.idiap.ch/beat/beat.web

In case of issues, please share your questions through our development mailing list.

13.13. What is a supervisor?

A supervisor is a recognized person on the platform. The supervisor is generally a person that holds a senior position in a laboratory/research institute or company. It is important to note that your email should be institutional!

13.14. Why do I need a supervisor to start using the BEAT platform?

Supervisors ensure you belong to a work group. They are institutionally responsible that you follow the BEAT platform terms of use. To successfully register, you’ll need to specify the username of a currently existing supervisor on the platform. After you have validated your institutional e-mail address, a request will be made to such an user in order to confirm your registration. Once the supervisor has confirmed your affiliation, your account becomes active and you can use the platform. Re-activation is required after a period of time (usually every year).

It is important to note that your email should be institutional!

If your supervisor is not registered on the BEAT platform, ask them to register, and to make a request to become a supervisor (see text below). Once their account is created, you will then be able to make a supervision request to them.

13.15. My account got blocked for various reasons, how do I unblock it?

The procedure for re-activating your account is the following:

  • Go to the sign in page and follow the reactivation process or directly click on the following link to reactivate an account.

  • Fill in the form: username, password (the ones used when creating your profile), and supervisor username: any supervisor registered on the BEAT platform (note that it should be someone willing to supervise you). Please refer to the paragraph above for why this necessary.

  • Once the form is completed click on the Reactivation request button.

  • An email will be sent to the supervisor who has a few days to accept your supervison request.

  • You will be informed if your supervisor has accepted or rejected your application.

  • If no information is given by the assigned supervisor, you will be allowed to make a new supervision request after a few days.

13.16. How do I become a supervisor on the BEAT platform?

As stated above, a supervisor ensures that their supervisees belong to a research group and follow the BEAT platform terms of use.

To become a supervisor, there are two ways.

  • Sign up on the platform and register as a supervisor.

    After you have validated that your institutional email is correct, a request will then be made to the BEAT admins in order to validate your account. After checking, that you hold a position in your university/research institute or laboratory, an admin of the platform will accept your request.

  • If your position has changed in your university/research institute or laboratory, please do contact us following the contact link on the main page.

    A BEAT admin will then check that you really hold a position and will turn your account into a supervisor account in order to let you manage supervisees.