diff --git a/documentation/operator-guide/configuration.rst b/documentation/operator-guide/configuration.rst index 4d40c6950de5dc43f7bdcc8bf4b8053b183805ca..2d62918dcdb1b86925d501fe6ed72da2cba9967f 100644 --- a/documentation/operator-guide/configuration.rst +++ b/documentation/operator-guide/configuration.rst @@ -1,39 +1,39 @@ -.. Configuration: +.. _configuration: Configuration ============= -This chapter details how a running VS service can be configured. And what steps +This chapter details how a running VS stack can be configured. And what steps are necessary to deploy the configuration. -In the following sections, it is indicated, that for an update of the -configuration to take effect some steps need to be taken. +In order for these configuration changes to be picked up by a running VS +stack and to take effect some steps need to be performed. These steps are +either a "re-deploy" of the running stack or a complete re-creation of it. -As will be further described, for some configurations it is only required to -"re-deploy" the stack and re-start the services. This is done using the -following commands: +Stack Re-deploy +--------------- -.. code-block:: bash - - # scale all services to 0 replicas, effectively pausing the stack - docker service scale ... +As will be further described, for some configurations it is sufficient to +"re-deploy" the stack which automatically re-starts any service with changed +configuration. This is done re-using the stack deployment command: - # re-deploy the stack (update its configuration) - docker stack deploy -c ... +.. code-block:: bash - # scale the services to their original replica count - docker service scale + docker stack deploy -c docker-compose.<name>.yml -c docker-compose.<name>.dev.yml <stack-name> .. warning:: When calling the ``docker stack deploy`` command, it is vital to use the - command with that the stack was originally created. + command with the same files and name the stack was originally created. +Stack Re-creation +----------------- -In some cases this is not enough, as the configuration was used to for a -metarialized instance which needs to be reverted. The easiest way to do this is -to delete the volume in question. If, for example, the renderer/registrar -configuration was updated, the ``instance-data`` volume needs to be re-created. +In some cases a stack re-redeploy is not enough, as the configuration was used +for a materialized instance which needs to be reverted. The easiest way to do +this is to delete the volume in question. If, for example, the +renderer/registrar configuration was updated, the ``instance-data`` volume +needs to be re-created. First, the stack needs to be shut down. This is done using the following command: @@ -57,10 +57,10 @@ all containers have actually stopped) the next step is to delete the volume is still in use. In this case, it is advisable to wait for a minute and to try the deletion again. -Now that the volume was deleted, the stack can be re-deployed, which will -trigger the automatic re-creation and initialization of the volume. For the -``instance-data``, it means that the instance will be re-created and all -database models with it. +Now that the volume was deleted, the stack can be re-deployed as described +above, which will trigger the automatic re-creation and initialization of the +volume. For the ``instance-data``, it means that the instance will be re-created +and all database models with it. Docker Compose Settings @@ -68,7 +68,7 @@ Docker Compose Settings These configurations are altering the behavior of the stack itself and its contained services. A complete reference of the configuration file structure -can be found in `Docker Compose documentation +can be found in the `Docker Compose documentation <https://docs.docker.com/compose/compose-file/>`_. Environment Variables @@ -105,100 +105,110 @@ Environment variables and ``.env`` files are passed to the services via the INIT_SCRIPTS: "/configure.sh /init-db.sh" # ... +``.env`` Files +~~~~~~~~~~~~~~ + The following ``.env`` files are typically used: - * ``<stack-name>.env``: The general ``.env`` file used for all services - * ``<stack-name>_db.env``: The database access credentials, for all services - interacting with the database. - * ``<stack-name>_django.env``: This env files defines the credentials for the - django admin user to be used with the admin GUI. - * ``<stack-name>_obs.env``: This contains access parameters for the object - storage(s). - * ``<stack-name>_preprocessor.env``: Preprocessor related environment - variables - * ``<stack-name>_redis.env``: Redis access credentials and queue names +* ``<stack-name>.env``: The general ``.env`` file used for all services +* ``<stack-name>_db.env``: The database access credentials, for all services + interacting with the database. +* ``<stack-name>_django.env``: This env files defines the credentials for the + django admin user to be used with the admin GUI. +* ``<stack-name>_obs.env``: This contains access parameters for the object + storage(s). +* ``<stack-name>_preprocessor.env``: Preprocessor related environment + variables +* ``<stack-name>_redis.env``: Redis access credentials and queue names -Groups of environment variables +Groups of Environment Variables ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -GDAL environment variables - This group of environment variables controls the intricacies of GDAL. They - control how GDAL interacts with its supported files. As GDAL supports a - variety of formats and backend access, most of the full `list of env - variables <https://gdal.org/user/configoptions.html>`_ are not applicable and - only a handful are actually relevant for VS. - - ``GDAL_DISABLE_READDIR_ON_OPEN`` - Especially when using an Object Storage backend with a very large number of - files, it is vital to activate this setting (``=TRUE``) in order to - suppress to read the whole directory contents which is very slow for some - OBS backends. - ``CPL_VSIL_CURL_ALLOWED_EXTENSIONS`` - This limits the file extensions to disable the lookup of so called sidecar - files which are not used for VS. By default this value is used: - ``=.TIF,.tif,.xml``. - -OpenStack Swift environment variables - These variables define the access coordinates and credentials for the - OpenStack Swift Object storage backend. - - ``ST_AUTH_VERSION`` - ``OS_AUTH_URL_SHORT`` - ``OS_AUTH_URL`` - ``OS_USERNAME`` - ``OS_PASSWORD`` - ``OS_TENANT_NAME`` - ``OS_TENANT_ID`` - ``OS_REGION_NAME`` - ``OS_USER_DOMAIN_NAME`` - This set of variables define the credentials for the object storage to - place the preprocessed results. - - - ``OS_USERNAME_DOWNLOAD`` - ``OS_PASSWORD_DOWNLOAD`` - ``OS_TENANT_NAME_DOWNLOAD`` - ``OS_TENANT_ID_DOWNLOAD`` - ``OS_REGION_NAME_DOWNLOAD`` - ``OS_AUTH_URL_DOWNLOAD`` - ``ST_AUTH_VERSION_DOWNLOAD`` - This set of variables define the credentials for the object storage to - retrieve the original product files. - -VS environment variables - These environment variables are used by VS itself to configure various parts. - - .. note:: - These variables are used during the initial stack setup. When these - variables are changed, they will not be reflected unless the instance - volume is re-created. - - ``COLLECTION`` - This defines the main collections name. This is used in various parts of - the VS and serves as the layer base name. - ``UPLOAD_CONTAINER`` - This controls the bucket name where the preprocessed images are uploaded - to. - ``DJANGO_USER``, ``DJANGO_MAIL``, ``DJANGO_PASSWORD`` - The Django admin user account credentials to use the Admin GUI. - - .. note:: - These variables are used during the initial stack setup. When these - variables are changed, they will not be reflected unless the database - volume is re-created. - - ``POSTGRES_USER`` - ``POSTGRES_PASSWORD`` - ``POSTGRES_DB`` - ``DB`` - ``DB_USER`` - ``DB_PW`` - ``DB_HOST`` - ``DB_PORT`` - ``DB_NAME`` - These are the internal access credentials for the database. +GDAL Environment Variables +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +This group of environment variables controls the intricacies of GDAL. They +control how GDAL interacts with its supported files. As GDAL supports a +variety of formats and backend access, most of the full `list of env +variables <https://gdal.org/user/configoptions.html>`_ are not applicable and +only a handful are actually relevant for the VS. + +* ``GDAL_DISABLE_READDIR_ON_OPEN`` - + Especially when using an Object Storage backend with a very large number of + files, it is vital to activate this setting (``=TRUE``) in order to + suppress to read the whole directory contents which is very slow for some + OBS backends. +* ``CPL_VSIL_CURL_ALLOWED_EXTENSIONS`` - + This limits the file extensions to disable the lookup of so called sidecar + files which are not used for VS. By default this value is used: + ``=.TIF,.tif,.xml``. + +OpenStack Swift Environment Variables +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +These variables define the access coordinates and credentials for the +OpenStack Swift Object storage backend. + +This set of variables define the credentials for the object storage to +place the preprocessed results: + +* ``ST_AUTH_VERSION`` +* ``OS_AUTH_URL_SHORT`` +* ``OS_AUTH_URL`` +* ``OS_USERNAME`` +* ``OS_PASSWORD`` +* ``OS_TENANT_NAME`` +* ``OS_TENANT_ID`` +* ``OS_REGION_NAME`` +* ``OS_USER_DOMAIN_NAME`` + +This set of variables define the credentials for the object storage to +retrieve the original product files: + +* ``OS_USERNAME_DOWNLOAD`` +* ``OS_PASSWORD_DOWNLOAD`` +* ``OS_TENANT_NAME_DOWNLOAD`` +* ``OS_TENANT_ID_DOWNLOAD`` +* ``OS_REGION_NAME_DOWNLOAD`` +* ``OS_AUTH_URL_DOWNLOAD`` +* ``ST_AUTH_VERSION_DOWNLOAD`` + +VS Environment Variables +^^^^^^^^^^^^^^^^^^^^^^^^ + +These environment variables are used by the VS itself to configure various parts. +.. note:: + These variables are used during the initial stack setup. When these + variables are changed, they will not be reflected unless the instance + volume is re-created. + +* ``COLLECTION`` - + This defines the main collections name. This is used in various parts of + the VS and serves as the layer base name. +* ``UPLOAD_CONTAINER`` - + This controls the bucket name where the preprocessed images are uploaded + to. +* ``DJANGO_USER``, ``DJANGO_MAIL``, ``DJANGO_PASSWORD`` - + The Django admin user account credentials to use the Admin GUI. + +.. note:: + These variables are used during the initial stack setup. When these + variables are changed, they will not be reflected unless the database + volume is re-created. + +These are the internal access credentials for the database: + +* ``POSTGRES_USER`` +* ``POSTGRES_PASSWORD`` +* ``POSTGRES_DB`` +* ``DB`` +* ``DB_USER`` +* ``DB_PW`` +* ``DB_HOST`` +* ``DB_PORT`` +* ``DB_NAME`` Configuration Files ------------------- @@ -229,17 +239,18 @@ such a configuration file is defined and the used in a service: The following configuration files are used throughout the VS: - * ``<stack-name>_init-db.sh``: This shell script files purpose is to set up - the EOxServer instance used by both the renderer and registrar. - * ``<stack-name>_index-dev.html``/``<stack-name>_index-ops.html``: The - clients main HTML page, containing various client settings. The ``dev`` one - is used for development only, whereas the ``ops`` one is used for operational - deployment. - * ``<stack-name>_mapcache-dev.xml``/``<stack-name>_mapcache-ops.xml``: The - configuration file for MapCache, the software powering the cache service. - Similarly to the client configuration files, the ``dev`` and ``ops`` files - used for development and operational usage respectively. Further - documentation can be found at `the official site - <https://mapserver.org/mapcache/config.html>`_. - - +* ``<stack-name>_init-db.sh``: This shell script file's purpose is to set up + the EOxServer instance used by both the renderer and registrar. +* ``<stack-name>_index-dev.html``/``<stack-name>_index-ops.html``: The + clients main HTML page, containing various client settings. The ``dev`` one + is used for development only, whereas the ``ops`` one is used for operational + deployment. +* ``<stack-name>_mapcache-dev.xml``/``<stack-name>_mapcache-ops.xml``: The + configuration file for MapCache, the software powering the cache service. + Similarly to the client configuration files, the ``dev`` and ``ops`` files + used for development and operational usage respectively. Further + documentation can be found at `the official site + <https://mapserver.org/mapcache/config.html>`_. + +The next section :ref:`management` describes how an operator interacts with a +deployed VS stack. diff --git a/documentation/operator-guide/setup.rst b/documentation/operator-guide/setup.rst index b570d9a40146583b1c4ebea1d5ee42c389a79a10..6d8b570ea9df8326ae5b80c1bad6b3b2177e7cd7 100644 --- a/documentation/operator-guide/setup.rst +++ b/documentation/operator-guide/setup.rst @@ -112,7 +112,7 @@ service identifier: .. code-block:: bash - docker stack deploy -c docker-compose.<name>.yml -c docker-compose.<name>.dev.yml <name>-pdas + docker stack deploy -c docker-compose.<name>.yml -c docker-compose.<name>.dev.yml <stack-name> This command actually performs a variety of tasks. First off, it obtains any