EOX GitLab Instance

Skip to content
Snippets Groups Projects
Commit 94568337 authored by Stephan's avatar Stephan
Browse files

mostly formatting

parent ba5dc832
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
documentation/operator-guide/configuration.rst
+ 137
− 126
View file @ 94568337
.. Configuration:
.. _configuration:
Configuration
=============
This chapter details how a running VS service can be configured. And what steps
This chapter details how a running VS stack can be configured. And what steps
are necessary to deploy the configuration.
In the following sections, it is indicated, that for an update of the
configuration to take effect some steps need to be taken.
In order for these configuration changes to be picked up by a running VS
stack and to take effect some steps need to be performed. These steps are
either a "re-deploy" of the running stack or a complete re-creation of it.
As will be further described, for some configurations it is only required to
"re-deploy" the stack and re-start the services. This is done using the
following commands:
Stack Re-deploy
---------------
.. code-block:: bash
# scale all services to 0 replicas, effectively pausing the stack
docker service scale ...
As will be further described, for some configurations it is sufficient to
"re-deploy" the stack which automatically re-starts any service with changed
configuration. This is done re-using the stack deployment command:
# re-deploy the stack (update its configuration)
docker stack deploy -c ...
.. code-block:: bash
# scale the services to their original replica count
docker service scale
docker stack deploy -c docker-compose.<name>.yml -c docker-compose.<name>.dev.yml <stack-name>
.. warning::
When calling the ``docker stack deploy`` command, it is vital to use the
command with that the stack was originally created.
command with the same files and name the stack was originally created.
Stack Re-creation
-----------------
In some cases this is not enough, as the configuration was used to for a
metarialized instance which needs to be reverted. The easiest way to do this is
to delete the volume in question. If, for example, the renderer/registrar
configuration was updated, the ``instance-data`` volume needs to be re-created.
In some cases a stack re-redeploy is not enough, as the configuration was used
for a materialized instance which needs to be reverted. The easiest way to do
this is to delete the volume in question. If, for example, the
renderer/registrar configuration was updated, the ``instance-data`` volume
needs to be re-created.
First, the stack needs to be shut down. This is done using the following
command:
......@@ -57,10 +57,10 @@ all containers have actually stopped) the next step is to delete the
volume is still in use. In this case, it is advisable to wait for a minute
and to try the deletion again.
Now that the volume was deleted, the stack can be re-deployed, which will
trigger the automatic re-creation and initialization of the volume. For the
``instance-data``, it means that the instance will be re-created and all
database models with it.
Now that the volume was deleted, the stack can be re-deployed as described
above, which will trigger the automatic re-creation and initialization of the
volume. For the ``instance-data``, it means that the instance will be re-created
and all database models with it.
Docker Compose Settings
......@@ -68,7 +68,7 @@ Docker Compose Settings
These configurations are altering the behavior of the stack itself and its
contained services. A complete reference of the configuration file structure
can be found in `Docker Compose documentation
can be found in the `Docker Compose documentation
<https://docs.docker.com/compose/compose-file/>`_.
Environment Variables
......@@ -105,100 +105,110 @@ Environment variables and ``.env`` files are passed to the services via the
INIT_SCRIPTS: "/configure.sh /init-db.sh"
# ...
``.env`` Files
~~~~~~~~~~~~~~
The following ``.env`` files are typically used:
* ``<stack-name>.env``: The general ``.env`` file used for all services
* ``<stack-name>_db.env``: The database access credentials, for all services
interacting with the database.
* ``<stack-name>_django.env``: This env files defines the credentials for the
django admin user to be used with the admin GUI.
* ``<stack-name>_obs.env``: This contains access parameters for the object
storage(s).
* ``<stack-name>_preprocessor.env``: Preprocessor related environment
variables
* ``<stack-name>_redis.env``: Redis access credentials and queue names
* ``<stack-name>.env``: The general ``.env`` file used for all services
* ``<stack-name>_db.env``: The database access credentials, for all services
interacting with the database.
* ``<stack-name>_django.env``: This env files defines the credentials for the
django admin user to be used with the admin GUI.
* ``<stack-name>_obs.env``: This contains access parameters for the object
storage(s).
* ``<stack-name>_preprocessor.env``: Preprocessor related environment
variables
* ``<stack-name>_redis.env``: Redis access credentials and queue names
Groups of environment variables
Groups of Environment Variables
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
GDAL environment variables
This group of environment variables controls the intricacies of GDAL. They
control how GDAL interacts with its supported files. As GDAL supports a
variety of formats and backend access, most of the full `list of env
variables <https://gdal.org/user/configoptions.html>`_ are not applicable and
only a handful are actually relevant for VS.
``GDAL_DISABLE_READDIR_ON_OPEN``
Especially when using an Object Storage backend with a very large number of
files, it is vital to activate this setting (``=TRUE``) in order to
suppress to read the whole directory contents which is very slow for some
OBS backends.
``CPL_VSIL_CURL_ALLOWED_EXTENSIONS``
This limits the file extensions to disable the lookup of so called sidecar
files which are not used for VS. By default this value is used:
``=.TIF,.tif,.xml``.
OpenStack Swift environment variables
These variables define the access coordinates and credentials for the
OpenStack Swift Object storage backend.
``ST_AUTH_VERSION``
``OS_AUTH_URL_SHORT``
``OS_AUTH_URL``
``OS_USERNAME``
``OS_PASSWORD``
``OS_TENANT_NAME``
``OS_TENANT_ID``
``OS_REGION_NAME``
``OS_USER_DOMAIN_NAME``
This set of variables define the credentials for the object storage to
place the preprocessed results.
``OS_USERNAME_DOWNLOAD``
``OS_PASSWORD_DOWNLOAD``
``OS_TENANT_NAME_DOWNLOAD``
``OS_TENANT_ID_DOWNLOAD``
``OS_REGION_NAME_DOWNLOAD``
``OS_AUTH_URL_DOWNLOAD``
``ST_AUTH_VERSION_DOWNLOAD``
This set of variables define the credentials for the object storage to
retrieve the original product files.
VS environment variables
These environment variables are used by VS itself to configure various parts.
.. note::
These variables are used during the initial stack setup. When these
variables are changed, they will not be reflected unless the instance
volume is re-created.
``COLLECTION``
This defines the main collections name. This is used in various parts of
the VS and serves as the layer base name.
``UPLOAD_CONTAINER``
This controls the bucket name where the preprocessed images are uploaded
to.
``DJANGO_USER``, ``DJANGO_MAIL``, ``DJANGO_PASSWORD``
The Django admin user account credentials to use the Admin GUI.
.. note::
These variables are used during the initial stack setup. When these
variables are changed, they will not be reflected unless the database
volume is re-created.
``POSTGRES_USER``
``POSTGRES_PASSWORD``
``POSTGRES_DB``
``DB``
``DB_USER``
``DB_PW``
``DB_HOST``
``DB_PORT``
``DB_NAME``
These are the internal access credentials for the database.
GDAL Environment Variables
^^^^^^^^^^^^^^^^^^^^^^^^^^
This group of environment variables controls the intricacies of GDAL. They
control how GDAL interacts with its supported files. As GDAL supports a
variety of formats and backend access, most of the full `list of env
variables <https://gdal.org/user/configoptions.html>`_ are not applicable and
only a handful are actually relevant for the VS.
* ``GDAL_DISABLE_READDIR_ON_OPEN`` -
Especially when using an Object Storage backend with a very large number of
files, it is vital to activate this setting (``=TRUE``) in order to
suppress to read the whole directory contents which is very slow for some
OBS backends.
* ``CPL_VSIL_CURL_ALLOWED_EXTENSIONS`` -
This limits the file extensions to disable the lookup of so called sidecar
files which are not used for VS. By default this value is used:
``=.TIF,.tif,.xml``.
OpenStack Swift Environment Variables
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
These variables define the access coordinates and credentials for the
OpenStack Swift Object storage backend.
This set of variables define the credentials for the object storage to
place the preprocessed results:
* ``ST_AUTH_VERSION``
* ``OS_AUTH_URL_SHORT``
* ``OS_AUTH_URL``
* ``OS_USERNAME``
* ``OS_PASSWORD``
* ``OS_TENANT_NAME``
* ``OS_TENANT_ID``
* ``OS_REGION_NAME``
* ``OS_USER_DOMAIN_NAME``
This set of variables define the credentials for the object storage to
retrieve the original product files:
* ``OS_USERNAME_DOWNLOAD``
* ``OS_PASSWORD_DOWNLOAD``
* ``OS_TENANT_NAME_DOWNLOAD``
* ``OS_TENANT_ID_DOWNLOAD``
* ``OS_REGION_NAME_DOWNLOAD``
* ``OS_AUTH_URL_DOWNLOAD``
* ``ST_AUTH_VERSION_DOWNLOAD``
VS Environment Variables
^^^^^^^^^^^^^^^^^^^^^^^^
These environment variables are used by the VS itself to configure various parts.
.. note::
These variables are used during the initial stack setup. When these
variables are changed, they will not be reflected unless the instance
volume is re-created.
* ``COLLECTION`` -
This defines the main collections name. This is used in various parts of
the VS and serves as the layer base name.
* ``UPLOAD_CONTAINER`` -
This controls the bucket name where the preprocessed images are uploaded
to.
* ``DJANGO_USER``, ``DJANGO_MAIL``, ``DJANGO_PASSWORD`` -
The Django admin user account credentials to use the Admin GUI.
.. note::
These variables are used during the initial stack setup. When these
variables are changed, they will not be reflected unless the database
volume is re-created.
These are the internal access credentials for the database:
* ``POSTGRES_USER``
* ``POSTGRES_PASSWORD``
* ``POSTGRES_DB``
* ``DB``
* ``DB_USER``
* ``DB_PW``
* ``DB_HOST``
* ``DB_PORT``
* ``DB_NAME``
Configuration Files
-------------------
......@@ -229,17 +239,18 @@ such a configuration file is defined and the used in a service:
The following configuration files are used throughout the VS:
* ``<stack-name>_init-db.sh``: This shell script files purpose is to set up
the EOxServer instance used by both the renderer and registrar.
* ``<stack-name>_index-dev.html``/``<stack-name>_index-ops.html``: The
clients main HTML page, containing various client settings. The ``dev`` one
is used for development only, whereas the ``ops`` one is used for operational
deployment.
* ``<stack-name>_mapcache-dev.xml``/``<stack-name>_mapcache-ops.xml``: The
configuration file for MapCache, the software powering the cache service.
Similarly to the client configuration files, the ``dev`` and ``ops`` files
used for development and operational usage respectively. Further
documentation can be found at `the official site
<https://mapserver.org/mapcache/config.html>`_.
* ``<stack-name>_init-db.sh``: This shell script file's purpose is to set up
the EOxServer instance used by both the renderer and registrar.
* ``<stack-name>_index-dev.html``/``<stack-name>_index-ops.html``: The
clients main HTML page, containing various client settings. The ``dev`` one
is used for development only, whereas the ``ops`` one is used for operational
deployment.
* ``<stack-name>_mapcache-dev.xml``/``<stack-name>_mapcache-ops.xml``: The
configuration file for MapCache, the software powering the cache service.
Similarly to the client configuration files, the ``dev`` and ``ops`` files
used for development and operational usage respectively. Further
documentation can be found at `the official site
<https://mapserver.org/mapcache/config.html>`_.
The next section :ref:`management` describes how an operator interacts with a
deployed VS stack.
documentation/operator-guide/setup.rst
+ 1
− 1
View file @ 94568337
......@@ -112,7 +112,7 @@ service identifier:
.. code-block:: bash
docker stack deploy -c docker-compose.<name>.yml -c docker-compose.<name>.dev.yml <name>-pdas
docker stack deploy -c docker-compose.<name>.yml -c docker-compose.<name>.dev.yml <stack-name>
This command actually performs a variety of tasks. First off, it obtains any
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment