EOX GitLab Instance

Skip to content
Snippets Groups Projects
Commit 6b291e1b authored by Fabian Schindler's avatar Fabian Schindler
Browse files

Updating configuration section

parent 8f7f6d3c
No related branches found
No related tags found
1 merge request!8Docs
Hide whitespace changes
Inline Side-by-side
documentation/operator-guide/configuration.rst
+ 221
− 89
View file @ 6b291e1b
......@@ -6,108 +6,240 @@ Configuration
This chapter details how a running VS service can be configured. And what steps
are necessary to deploy the configuration.
Concepts
--------
This section explains the various kinds of configuration items and where to
apply them.
In the following sections, it is indicated, that for an update of the
configuration to take effect some steps need to be taken.
As will be further described, for some configurations it is only required to
"re-deploy" the stack and re-start the services. This is done using the
following commands:
.. code-block:: bash
# scale all services to 0 replicas, effectively pausing the stack
docker service scale ...
# re-deploy the stack (update its configuration)
docker stack deploy -c ...
# scale the services to their original replica count
docker service scale
.. warning::
When calling the ``docker stack deploy`` command, it is vital to use the
command with that the stack was originally created.
In some cases this is not enough, as the configuration was used to for a
metarialized instance which needs to be reverted. The easiest way to do this is
to delete the volume in question. If, for example, the renderer/registrar
configuration was updated, the ``instance-data`` volume needs to be re-created.
First, the stack needs to be shut down. This is done using the following
command:
.. code-block:: bash
docker stack rm <stack-name>
When that command has completed (it is advisable to wait for some time until
all containers have actually stopped) the next step is to delete the
``instance-data`` volume:
.. code-block:: bash
docker volume rm <stack-name>_instance-data
.. note::
It is possible that this command fails, with the error message that the
volume is still in use. In this case, it is advisable to wait for a minute
and to try the deletion again.
Now that the volume was deleted, the stack can be re-deployed, which will
trigger the automatic re-creation and initialization of the volume. For the
``instance-data``, it means that the instance will be re-created and all
database models with it.
Docker Compose Settings
These configurations are altering the behavior of the stack itself and its
contained services. A complete reference of the configuration file structure
can be found in `Docker Compose documentation
<https://docs.docker.com/compose/compose-file/>`_.
-----------------------
These configurations are altering the behavior of the stack itself and its
contained services. A complete reference of the configuration file structure
can be found in `Docker Compose documentation
<https://docs.docker.com/compose/compose-file/>`_.
Environment Variables
These variables are passed to their respective containers environment and
change the behavior of certain functionality. They can be declared in the
Docker Compose configuration file directly, but typically they are bundled by
field of interest and then placed into ``.env`` files and then passed to the
containers. So for example, there will be a ``<stack-name>_obs.env`` file
to store the access parameters for the object storage.
All those files are placed in the ``env/`` directory in the instances
directory.
Environment variables and ``.env`` files are passed to the services via the
``docker-compose.yml`` directives. The following example shows how to pass
``.env`` files and direct environment variables:
.. code-block:: yaml
services:
# ....
registrar:
env_file:
- env/stack.env
- env/stack_db.env
- env/stack_obs.env
- env/stack_redis.env
environment:
INSTANCE_ID: "prism-view-server_registrar"
INSTALL_DIR: "/var/www/pvs/dev/"
SCALEFACTOR: "1"
IN_MEMORY: "false"
INIT_SCRIPTS: "/configure.sh /init-db.sh"
# ...
The following ``.env`` files are typically used:
* ``<stack-name>.env``: The general ``.env`` file used for all services
* ``<stack-name>_db.env``: The database access credentials, for all services
interacting with the database.
* ``<stack-name>_django.env``: This env files defines the credentials for the
django admin user to be used with the admin GUI.
* ``<stack-name>_obs.env``: This contains access parameters for the object
storage(s).
* ``<stack-name>_preprocessor.env``: Preprocessor related environment
variables
* ``<stack-name>_redis.env``: Redis access credentials and queue names
.. Usually, it is not possible to override the values in running containers.
.. So whenever such a value is changed, the stack has to be re-deployed in
.. order to push the changes to the services.
---------------------
These variables are passed to their respective containers environment and
change the behavior of certain functionality. They can be declared in the
Docker Compose configuration file directly, but typically they are bundled by
field of interest and then placed into ``.env`` files and then passed to the
containers. So for example, there will be a ``<stack-name>_obs.env`` file
to store the access parameters for the object storage.
All those files are placed in the ``env/`` directory in the instances
directory.
Environment variables and ``.env`` files are passed to the services via the
``docker-compose.yml`` directives. The following example shows how to pass
``.env`` files and direct environment variables:
.. code-block:: yaml
services:
# ....
registrar:
env_file:
- env/stack.env
- env/stack_db.env
- env/stack_obs.env
- env/stack_redis.env
environment:
INSTANCE_ID: "prism-view-server_registrar"
INSTALL_DIR: "/var/www/pvs/dev/"
SCALEFACTOR: "1"
IN_MEMORY: "false"
INIT_SCRIPTS: "/configure.sh /init-db.sh"
# ...
The following ``.env`` files are typically used:
* ``<stack-name>.env``: The general ``.env`` file used for all services
* ``<stack-name>_db.env``: The database access credentials, for all services
interacting with the database.
* ``<stack-name>_django.env``: This env files defines the credentials for the
django admin user to be used with the admin GUI.
* ``<stack-name>_obs.env``: This contains access parameters for the object
storage(s).
* ``<stack-name>_preprocessor.env``: Preprocessor related environment
variables
* ``<stack-name>_redis.env``: Redis access credentials and queue names
Groups of environment variables
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
GDAL environment variables
This group of environment variables controls the intricacies of GDAL. They
control how GDAL interacts with its supported files. As GDAL supports a
variety of formats and backend access, most of the full `list of env
variables <https://gdal.org/user/configoptions.html>`_ are not applicable and
only a handful are actually relevant for VS.
``GDAL_DISABLE_READDIR_ON_OPEN``
Especially when using an Object Storage backend with a very large number of
files, it is vital to activate this setting (``=TRUE``) in order to
suppress to read the whole directory contents which is very slow for some
OBS backends.
``CPL_VSIL_CURL_ALLOWED_EXTENSIONS``
This limits the file extensions to disable the lookup of so called sidecar
files which are not used for VS. By default this value is used:
``=.TIF,.tif,.xml``.
OpenStack Swift environment variables
These variables define the access coordinates and credentials for the
OpenStack Swift Object storage backend.
``ST_AUTH_VERSION``
``OS_AUTH_URL_SHORT``
``OS_AUTH_URL``
``OS_USERNAME``
``OS_PASSWORD``
``OS_TENANT_NAME``
``OS_TENANT_ID``
``OS_REGION_NAME``
``OS_USER_DOMAIN_NAME``
This set of variables define the credentials for the object storage to
place the preprocessed results.
``OS_USERNAME_DOWNLOAD``
``OS_PASSWORD_DOWNLOAD``
``OS_TENANT_NAME_DOWNLOAD``
``OS_TENANT_ID_DOWNLOAD``
``OS_REGION_NAME_DOWNLOAD``
``OS_AUTH_URL_DOWNLOAD``
``ST_AUTH_VERSION_DOWNLOAD``
This set of variables define the credentials for the object storage to
retrieve the original product files.
VS environment variables
These environment variables are used by VS itself to configure various parts.
.. note::
These variables are used during the initial stack setup. When these
variables are changed, they will not be reflected unless the instance
volume is re-created.
``COLLECTION``
This defines the main collections name. This is used in various parts of
the VS and serves as the layer base name.
``UPLOAD_CONTAINER``
This controls the bucket name where the preprocessed images are uploaded
to.
``DJANGO_USER``, ``DJANGO_MAIL``, ``DJANGO_PASSWORD``
The Django admin user account credentials to use the Admin GUI.
.. note::
These variables are used during the initial stack setup. When these
variables are changed, they will not be reflected unless the database
volume is re-created.
``POSTGRES_USER``
``POSTGRES_PASSWORD``
``POSTGRES_DB``
``DB``
``DB_USER``
``DB_PW``
``DB_HOST``
``DB_PORT``
``DB_NAME``
These are the internal access credentials for the database.
Configuration Files
Such files are passed to the containers in a similar way as environment
variables, but usually contain more settings at once and are placed at a
specific path in the container at runtime.
Configuration files are passed into the containers using the ``configs``
section of the ``docker-compose.yaml`` file. The following example shows how
such a configuration file is defined and the used in a service:
Configuration Files
-------------------
Such files are passed to the containers in a similar way as environment
variables, but usually contain more settings at once and are placed at a
specific path in the container at runtime.
.. code-block:: yaml
Configuration files are passed into the containers using the ``configs``
section of the ``docker-compose.yaml`` file. The following example shows how
such a configuration file is defined and the used in a service:
# ...
configs:
my-config:
file: ./config/example.cfg
# ...
services:
myservice:
# ...
configs:
- source: my-config
target: /example.cfg
.. code-block:: yaml
The following configuration files are used throughout the VS:
# ...
configs:
my-config:
file: ./config/example.cfg
# ...
services:
myservice:
# ...
configs:
- source: my-config
target: /example.cfg
* ``<stack-name>_init-db.sh``: This shell script files purpose is to set up
the EOxServer instance used by both the renderer and registrar.
* ``<stack-name>_index-dev.html``/``<stack-name>_index-ops.html``: The
clients main HTML page, containing various client settings. The ``dev`` one
is used for development only, whereas the ``ops`` one is used for operational
deployment.
* ``<stack-name>_mapcache-dev.xml``/``<stack-name>_mapcache-ops.xml``: The
configuration file for MapCache, the software powering the cache service.
Similarly to the client configuration files, the ``dev`` and ``ops`` files
used for development and operational usage respectively. Further
documentation can be found at `the official site
<https://mapserver.org/mapcache/config.html>`_.
The following configuration files are used throughout the VS:
* ``<stack-name>_init-db.sh``: This shell script files purpose is to set up
the EOxServer instance used by both the renderer and registrar.
* ``<stack-name>_index-dev.html``/``<stack-name>_index-ops.html``: The
clients main HTML page, containing various client settings. The ``dev`` one
is used for development only, whereas the ``ops`` one is used for operational
deployment.
* ``<stack-name>_mapcache-dev.xml``/``<stack-name>_mapcache-ops.xml``: The
configuration file for MapCache, the software powering the cache service.
Similarly to the client configuration files, the ``dev`` and ``ops`` files
used for development and operational usage respectively. Further
documentation can be found at `the official site
<https://mapserver.org/mapcache/config.html>`_.
https://gdal.org/user/configoptions.html
\ No newline at end of file
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment