Skip to content
LAPIS

Start LAPIS and RhyDB

Every LAPIS instance needs to be backed by a RhyDB instance, that acts as data source. RhyDB could be operated stand-alone. LAPIS is meant as a layer of convenience and abstraction around RhyDB.

We provide Docker images of RhyDB and LAPIS that are ready to use. We recommend using those Docker images, so in this tutorial, we explain how to use them. You will build a Docker Compose file step by step.

Prerequisites

Section titled “Prerequisites”
  • You have Docker installed.
  • Some knowledge on how to use Docker and Docker Compose.
  • Make sure you have the latest Docker images:
Terminal window
docker pull ghcr.io/genspectrum/lapis
docker pull ghcr.io/genspectrum/lapis-silo
  • Create a directory for the tutorial:
Terminal window
mkdir ~/lapisExample
cd ~/lapisExample

Writing Configuration

Section titled “Writing Configuration”

Both LAPIS and RhyDB need to know which metadata columns are available in the dataset. Furthermore, you need to define which column acts as primary key and which column should be used to generate partitions in RhyDB. Also, LAPIS is configured to be an open instance, meaning that the underlying data requires no visibility restrictions.

~/lapisExample/config/database_config.yaml
schema:
    instanceName: testInstance
    metadata:
        - name: primaryKey
          type: string
        - name: date
          type: date
        - name: region
          type: string
          generateIndex: true
        - name: country
          type: string
          generateIndex: true
        - name: division
          type: string
          generateIndex: true
        - name: pangoLineage
          type: pango_lineage
        - name: age
          type: int
        - name: qc_value
          type: float
    opennessLevel: OPEN
    primaryKey: primaryKey
    dateToSortBy: date
    partitionBy: pangoLineage

Starting RhyDB Preprocessing

Section titled “Starting RhyDB Preprocessing”

Download the example dataset from the end-to-end tests:

  • pangolineage_alias.json
  • reference_genomes.json
  • small_metadata_set.tsv
  • all fasta files for the sequences

RhyDB expects fasta files (possibly compressed via zstandard or xz) in the same directory with naming scheme nuc_<sequence_index>.fasta for nucleotide sequences or gene_<sequence_index>.fasta for amino acid sequences. The sequence_indexs have to match the indexes of the arrays defined in the reference_genomes.json.

Put those files into the folder ~/lapisExample/data/.

Now RhyDB needs to know where it can find those files. You have to provide a “preprocessing config” for that. Note that you need to provide the paths where the files will be stored in the Docker container. Filenames are relative to the input directory. Since you don’t provide the input directory explicitly, RhyDB will fall back to the default /data.

~/lapisExample/config/preprocessing_config.yaml
metadataFilename: 'small_metadata_set.tsv'
pangoLineageDefinitionFilename: 'pangolineage_alias.json'
referenceGenomeFilename: 'reference_genomes.json'

To start the preprocessing, you have to:

  • start RhyDB in the preprocessing mode
  • mount the data into the container to the default location
  • mount the preprocessing config into the container to the default location
  • mount the database config into the container to the default location
  • mount the output directory into the container to the default location

Add a corresponding service to the docker-compose.yaml:

~/lapisExample/docker-compose.yaml
version: '3.9'


services:
    silo-preprocessing:
        image: ghcr.io/genspectrum/lapis-silo
        command: --preprocessing
        volumes:
            - ~/lapisExample/data:/preprocessing/input
            - ~/lapisExample/config/preprocessing_config.yaml:/app/preprocessing_config.yaml
            - ~/lapisExample/config/database_config.yaml:/app/database_config.yaml
            - ~/lapisExample/output:/preprocessing/output

After this has completed, the output directory should contain the result of the preprocessing. That result has to be provided to RhyDB in the next step.

Starting RhyDB

Section titled “Starting RhyDB”

To start the RhyDB api, you have to:

  • start RhyDB in the api mode,
  • expose port 8081,
  • mount the preprocessing result into the container,
  • wait for the preprocessing to complete.

Add a corresponding service to the docker-compose.yaml:

~/lapisExample/docker-compose.yaml
version: '3.9'


services:
    silo-preprocessing:
        image: ghcr.io/genspectrum/lapis-silo
        command: --preprocessing
        volumes:
            - ~/lapisExample/data:/preprocessing/input
            - ~/lapisExample/config/preprocessing_config.yaml:/app/preprocessing_config.yaml
            - ~/lapisExample/config/database_config.yaml:/app/database_config.yaml
            - ~/lapisExample/output:/preprocessing/output


    silo-api:
        image: ghcr.io/genspectrum/lapis-silo
        command: --api
        ports:
            - '8081:8081'
        volumes:
            - ~/lapisExample/output:/data
        depends_on:
            silo-preprocessing:
                condition: service_completed_successfully

Execute

Start the services
docker compose up

Now RhyDB should be available at http://localhost:8081 and http://localhost:8081/info should show that RhyDB contains sequences.

Starting LAPIS

Section titled “Starting LAPIS”

Now you can start LAPIS. You have to:

  • expose port 8080 to the host.
  • mount the database configuration and the reference genomes to the default locations in the Docker container.
  • provide LAPIS with the RhyDB URL.

Add a corresponding service to the docker-compose.yaml:

~/lapisExample/docker-compose.yaml
version: '3.9'


services:
    lapis:
        image: ghcr.io/genspectrum/lapis
        command: --silo.url=http://silo-api:8081
        ports:
            - '8080:8080'
        volumes:
            - ~/lapisExample/config/database_config.yaml:/workspace/database_config.yaml
            - ~/lapisExample/data/reference_genomes.json:/workspace/reference_genomes.json


    silo-preprocessing:
        image: ghcr.io/genspectrum/lapis-silo
        command: --preprocessing
        volumes:
            - ~/lapisExample/data:/preprocessing/input
            - ~/lapisExample/config/preprocessing_config.yaml:/app/preprocessing_config.yaml
            - ~/lapisExample/config/database_config.yaml:/app/database_config.yaml
            - ~/lapisExample/output:/preprocessing/output


    silo-api:
        image: ghcr.io/genspectrum/lapis-silo
        command: --api
        ports:
            - '8081:8081'
        volumes:
            - ~/lapisExample/output:/data
        depends_on:
            silo-preprocessing:
                condition: service_completed_successfully

Execute

Start the services
docker compose up

again. Now LAPIS should be available at http://localhost:8080. LAPIS offers a Swagger UI that serves as a good starting point for exploring its functionalities.

Further Reading

Section titled “Further Reading”