configuration.rst

Configuration

This chapter details how a running VS stack can be configured. And what steps are necessary to deploy the configuration.

In order for these configuration changes to be picked up by a running VS stack and to take effect some steps need to be performed. These steps are either a "re-deploy" of the running stack or a complete re-creation of it.

Stack Re-deploy

As will be further described, for some configurations it is sufficient to "re-deploy" the stack which automatically re-starts any service with changed configuration. This is done re-using the stack deployment command:

docker stack deploy -c docker-compose.<name>.yml -c docker-compose.<name>.dev.yml <stack-name>

Warning

When calling the docker stack deploy command, it is vital to use the command with the same files and name the stack was originally created.

Stack Re-creation

In some cases a stack re-redeploy is not enough, as the configuration was used for a materialized instance which needs to be reverted. The easiest way to do this is to delete the volume in question. If, for example, the renderer/registrar configuration was updated, the instance-data volume needs to be re-created.

First, the stack needs to be shut down. This is done using the following command:

docker stack rm <stack-name>

When that command has completed (it is advisable to wait for some time until all containers have actually stopped) the next step is to delete the instance-data volume:

docker volume rm <stack-name>_instance-data

Note

It is possible that this command fails, with the error message that the volume is still in use. In this case, it is advisable to wait for a minute and to try the deletion again.

Now that the volume was deleted, the stack can be re-deployed as described above, which will trigger the automatic re-creation and initialization of the volume. For the instance-data, it means that the instance will be re-created and all database models with it.

Docker Compose Settings

These configurations are altering the behavior of the stack itself and its contained services. A complete reference of the configuration file structure can be found in the Docker Compose documentation.

Environment Variables

These variables are passed to their respective containers environment and change the behavior of certain functionality. They can be declared in the Docker Compose configuration file directly, but typically they are bundled by field of interest and then placed into .env files and then passed to the containers. So for example, there will be a <stack-name>_obs.env file to store the access parameters for the object storage. All those files are placed in the env/ directory in the instances directory.

Environment variables and .env files are passed to the services via the docker-compose.yml directives. The following example shows how to pass .env files and direct environment variables:

services:
  # ....
  registrar:
    env_file:
      - env/stack.env
      - env/stack_db.env
      - env/stack_obs.env
      - env/stack_redis.env
    environment:
      INSTANCE_ID: "prism-view-server_registrar"
      INSTALL_DIR: "/var/www/pvs/dev/"
      SCALEFACTOR: "1"
      IN_MEMORY: "false"
      INIT_SCRIPTS: "/configure.sh /init-db.sh"
    # ...

.env Files

The following .env files are typically used:

• <stack-name>.env: The general .env file used for all services
• <stack-name>_db.env: The database access credentials, for all services interacting with the database.
• <stack-name>_django.env: This env files defines the credentials for the django admin user to be used with the admin GUI.
• <stack-name>_obs.env: This contains access parameters for the object storage(s).
• <stack-name>_preprocessor.env: Preprocessor related environment variables
• <stack-name>_redis.env: Redis access credentials and queue names

Groups of Environment Variables

GDAL Environment Variables

This group of environment variables controls the intricacies of GDAL. They control how GDAL interacts with its supported files. As GDAL supports a variety of formats and backend access, most of the full list of env variables are not applicable and only a handful are actually relevant for the VS.

• GDAL_DISABLE_READDIR_ON_OPEN - Especially when using an Object Storage backend with a very large number of files, it is vital to activate this setting (=TRUE) in order to suppress to read the whole directory contents which is very slow for some OBS backends.
• CPL_VSIL_CURL_ALLOWED_EXTENSIONS - This limits the file extensions to disable the lookup of so called sidecar files which are not used for VS. By default this value is used: =.TIF,.tif,.xml.

OpenStack Swift Environment Variables

These variables define the access coordinates and credentials for the OpenStack Swift Object storage backend.

This set of variables define the credentials for the object storage to place the preprocessed results:

• ST_AUTH_VERSION
• OS_AUTH_URL_SHORT
• OS_AUTH_URL
• OS_USERNAME
• OS_PASSWORD
• OS_TENANT_NAME
• OS_TENANT_ID
• OS_REGION_NAME
• OS_USER_DOMAIN_NAME

This set of variables define the credentials for the object storage to retrieve the original product files:

• OS_USERNAME_DOWNLOAD
• OS_PASSWORD_DOWNLOAD
• OS_TENANT_NAME_DOWNLOAD
• OS_TENANT_ID_DOWNLOAD
• OS_REGION_NAME_DOWNLOAD
• OS_AUTH_URL_DOWNLOAD
• ST_AUTH_VERSION_DOWNLOAD

VS Environment Variables

These environment variables are used by the VS itself to configure various parts.

Note

These variables are used during the initial stack setup. When these variables are changed, they will not be reflected unless the instance volume is re-created.

• COLLECTION - This defines the main collections name. This is used in various parts of the VS and serves as the layer base name.
• UPLOAD_CONTAINER - This controls the bucket name where the preprocessed images are uploaded to.
• DJANGO_USER, DJANGO_MAIL, DJANGO_PASSWORD - The Django admin user account credentials to use the Admin GUI.

Note

These variables are used during the initial stack setup. When these variables are changed, they will not be reflected unless the database volume is re-created.

These are the internal access credentials for the database:

• POSTGRES_USER
• POSTGRES_PASSWORD
• POSTGRES_DB
• DB
• DB_USER
• DB_PW
• DB_HOST
• DB_PORT
• DB_NAME

Configuration Files

Such files are passed to the containers in a similar way as environment variables, but usually contain more settings at once and are placed at a specific path in the container at runtime.

Configuration files are passed into the containers using the configs section of the docker-compose.yaml file. The following example shows how such a configuration file is defined and the used in a service:

# ...
configs:
  my-config:
    file: ./config/example.cfg
# ...
services:
  myservice:
    # ...
    configs:
    - source: my-config
      target: /example.cfg

The following configuration files are used throughout the VS:

• <stack-name>_init-db.sh: This shell script file's purpose is to set up the EOxServer instance used by both the renderer and registrar.
• <stack-name>_index-dev.html/<stack-name>_index-ops.html: The clients main HTML page, containing various client settings. The dev one is used for development only, whereas the ops one is used for operational deployment.
• <stack-name>_mapcache-dev.xml/<stack-name>_mapcache-ops.xml: The configuration file for MapCache, the software powering the cache service. Similarly to the client configuration files, the dev and ops files used for development and operational usage respectively. Further documentation can be found at the official site.

The next section :ref:`management` describes how an operator interacts with a deployed VS stack.