The metadata server

The metadata server is a small web application that keeps the metadata files of your experiments (the tables that describe each animal, as used by ethoscopy and rethomics) organised by project, and links each file to the result databases it refers to. It runs in a Docker container built on the ethoscope_metadata_server package.

Building and running the container

Build the image from this Dockerfile:

# Dockerfile for ethoscope_metadata_server
FROM python:3.8

# prerequisites
RUN apt-get update && \
    apt-get install -y --no-install-recommends fonts-dejavu nano git gfortran file gcc cmake libcurl4-openssl-dev && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

RUN pip install bottle pandas

WORKDIR /opt
RUN git clone https://github.com/gilestrolab/ethoscope_metadata_server.git

WORKDIR /opt/ethoscope_metadata_server
CMD ["python", "/opt/ethoscope_metadata_server/metadata_db_server.py", "--refresh", "--metadata", "/opt/ethoscope_metadata"]

sudo docker build -t ethoscope-metadata-server . --no-cache

Then start a container. Mount the folder of SQLite result files read-only, and a writable folder for the metadata, their descriptions and the list of matching result files:

sudo docker run -d -p 8081:8081 \
    --name ethoscope-metadata-server \
    --volume /ethoscope_data/results:/mnt/ethoscope_results:ro \
    --volume /ethoscope_data/metadata:/opt/ethoscope_metadata \
    --restart=unless-stopped \
    ethoscope-metadata-server

The server listens on port 8081; put it behind a reverse proxy if it should be reachable under a host name. To update the server without rebuilding the image:

sudo docker exec ethoscope-metadata-server bash -c 'cd /opt/ethoscope_metadata_server && git pull'

Using it

1. Open the server’s address in a browser.
2. Drag your metadata files into the top box. Comma-separated (.csv, preferred) and tab-separated (.tsv) files are accepted.
3. Give a project name; several metadata files can belong to the same project. If the metadata belong to published work, add the DOI.
4. Fill in the authors and as many tags as you can think of, such as genotypes, treatments or ages, add a description, and click the save icon.

The file now appears under its project in the Browse tab (refresh the page if it does not). Next to each file are two icons, edit (not yet implemented) and delete.

Select a file from the list to see its information in a table. The first time, link the file to the result databases: click the refresh icon (two round arrows). After a few seconds the server reports how many databases were found and not found, and their size; both lists can be downloaded.