ethoscopy
ethoscopy is a Python package to load, curate, analyse and plot the behavioural data recorded by ethoscopes. Its central object, behavpy, is a pandas DataFrame that carries the experiment metadata with it, so selecting animals by genotype, sex or treatment filters data and metadata together. On top of pandas, behavpy adds sleep annotation, circadian analysis (actograms, anticipation scores, periodograms) and hidden Markov models of sleep and wake states. Plots can be drawn with Plotly (interactive) or Seaborn (static).
Install
Section titled “Install”pip install ethoscopyThe recommended way to run ethoscopy is ethoscope-lab, a Docker image that bundles JupyterHub, ethoscopy and the rethomics R packages. Installed on one workstation next to the ethoscope data, it lets every member of a lab run analyses from a browser. Installation covers both routes and the tutorial datasets.
If ethoscopy or ethoscope-lab contributes to your work, please cite:
Blackhurst L. and Gilestro G.F. (2023). Ethoscopy and ethoscope-lab: a framework for behavioural analysis to lower entrance barrier and aid reproducibility. Bioinformatics Advances 3(1): vbad132. doi:10.1093/bioadv/vbad132
BibTeX for this and the other ethoscope papers is on the publications page.
In these docs
Section titled “In these docs”Getting set up
Section titled “Getting set up”- Installation: Install ethoscopy with pip, or run it with JupyterHub and the rethomics R packages in the ethoscope-lab Docker container.
- Metadata design: How to write the metadata .csv file that ethoscopy uses to find, load and group the data of each animal.
- Loading the data: Download ethoscope databases, link them to the metadata and load them into a DataFrame, with sleep or stimulus annotation applied on the way.
behavpy
Section titled “behavpy”- About behavpy: behavpy keeps ethoscope data and metadata together in one pandas DataFrame subclass; this page shows how to create one, also from other data.
- Basic methods: Filter, remove, concatenate, bin, interpolate and curate behavpy data, and annotate sleep bouts, with behavpy’s built-in methods.
- Visualising your data: Summarise behavpy data and plot heatmaps, time courses and quantifications of sleep, then test group differences with SciPy.
- Visualising mAGO data: Plot how flies respond to odour puffs delivered by the mAGO module, as overall response rates and over time.
Circadian analysis
Section titled “Circadian analysis”- Circadian analysis: ethoscopy’s methods for circadian analysis: actograms, anticipation scores, periodograms and wavelets.
- Circadian methods and plots: Plot grouped and tiled actograms and compute anticipation scores to describe the circadian rhythm of each group.
- Periodograms: Compute chi-squared, Lomb-Scargle, Fourier and Welch periodograms and wavelets, then plot and quantify rhythmicity per animal and per group.
Hidden Markov models
Section titled “Hidden Markov models”- Hidden Markov models: Use hidden Markov models in ethoscopy to segment fly movement into sleep and wake states.
- Training an HMM: Curate movement data and train a categorical hidden Markov model of sleep and wake states with hmmlearn.
- Visualising with the HMM: Decode data with a trained HMM and plot the hidden states over time, their duration, their transitions and raw decoded sequences.
Tutorials
Section titled “Tutorials”- Jupyter tutorials: Jupyter notebooks that walk through ethoscopy analyses on example data, rendered on this site and available on GitHub.
- Tutorial 1 — Overview: working with ethoscope data: Load the tutorial dataset, curate it, plot sleep over time and quantify the differences between groups.
- Tutorial 2 — HMM: hidden behavioural states: Train a four-state hidden Markov model on fly movement, then decode and plot hidden sleep and wake states.
- Tutorial 3 — Circadian analysis: Actograms, anticipation scores, periodograms and wavelets on a dataset of flies with short, normal and long circadian periods.
- Tutorial 4 — Navigating the ethoscope databases: Match a metadata file to the ethoscope database files of a lab archive, list the ones that are missing and package them for sharing on Zenodo.
- Tutorial 5 — catch22 feature extraction: Extract catch22 time-series features from behavpy data, explore them with PCA and t-SNE, and classify groups with SVM and random forests.
- Tutorial 6 — Exporting to hctsa (MATLAB): Curate and normalise behavpy time series and save them as a .mat file for the hctsa toolbox in MATLAB.
- Paper companion notebook: Reproduces Figure 1 of the ethoscopy and ethoscope-lab paper (Blackhurst and Gilestro, 2023) from the tutorial dataset.