diff --git a/documentation/operator-guide/configuration.rst b/documentation/operator-guide/configuration.rst index 327832e46be3b71202293e4df20d59354518231b..e9ac61e2b5b2cd48ed15fbc6984e087df105eab6 100644 --- a/documentation/operator-guide/configuration.rst +++ b/documentation/operator-guide/configuration.rst @@ -240,18 +240,252 @@ such a configuration file is defined and the used in a service: The following configuration files are used throughout the VS: -* ``<stack-name>_init-db.sh``: This shell script file's purpose is to set up - the EOxServer instance used by both the renderer and registrar. -* ``<stack-name>_index-dev.html``/``<stack-name>_index-ops.html``: The - clients main HTML page, containing various client settings. The ``dev`` one - is used for development only, whereas the ``ops`` one is used for operational - deployment. -* ``<stack-name>_mapcache-dev.xml``/``<stack-name>_mapcache-ops.xml``: The - configuration file for MapCache, the software powering the cache service. - Similarly to the client configuration files, the ``dev`` and ``ops`` files - used for development and operational usage respectively. Further - documentation can be found at `the official site - <https://mapserver.org/mapcache/config.html>`_. +``<stack-name>_init-db.sh`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This shell script file's purpose is to set up the EOxServer instance used by +both the renderer and registrar. + +``<stack-name>_index-dev.html``/``<stack-name>_index-ops.html`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The clients main HTML page, containing various client settings. The ``dev`` one +is used for development only, whereas the ``ops`` one is used for operational +deployment. + +``<stack-name>_mapcache-dev.xml``/``<stack-name>_mapcache-ops.xml`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The configuration file for MapCache, the software powering the cache service. +Similarly to the client configuration files, the ``dev`` and ``ops`` files +used for development and operational usage respectively. Further +documentation can be found at `the official site +<https://mapserver.org/mapcache/config.html>`_. + +``<stack-name>_preprocessor-config.yaml`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The configuration for the proprocessing service to use to process to be +ingested files. + +The files are using YAML as a format and are structured in the following +fashion: + +source/target + + Here, the source file storage and the target file storage are configured. + This can either be a local directory or an OpenStack Swift object storage. + +workdir + + The workdir can be configured, to determine where the intermediate files are + placed. This can be convenient for debugging and development. + +keep_temp + + This boolean decides if the temporary directory for the preprocessing will be + cleaned up after being finished. Also, convenient for development. + +metadata_glob + + This file glob is used to determine the main metadata file to extract the + product type from. This file will be searched in the downloaded package. + +type_extractor + + This setting configures how the product type is extracted from the previously + extracted metadata. In the ``xpath`` setting one or more XPath expressions + can supplied to fetch the product type. Each XPath will be tried until one is + found that produces a result. These results can then be mapped using the + ``map`` dictionary. + +level_extractor + + This section works very similar to the ``type_extractor`` but only for the + product level. The product level is currently not used. + +preprocessing + + This is the actual preprocessing configuration setting. It is split in + defaults and product type specific settings. The defaults are applied + where there is no setting supplied for that specific type. The product type + is the one extracted earlier. + + defaults + + This section allows to configure any one of the available steps. Each step + configuration can be overridden in a specific product type configuration. + + The available steps are as follows: + + custom_preprocessor + + A custom python function to be called. + + path + + The Python module path to the function to call. + + args + + A list of arguments to pass to the function. + + kwargs + + A dictionary of keyword arguments to pass to the function. + + subdatasets + + What subdatasets to extract and how to name them. + + data_file_glob + + A file glob pattern to select files to extract from. + + subdataset_types + + Mapping of subdataset identifier to output filename postfix for + subdatasets to be extracted for each data file. + + georeference + + How the extracted files shall be georeferenced. + + type + + The type of georeferencing to apply. One of ``gcp``, ``rpc``, + ``corner``, ``world``. + + options + + Additional options for the georeferencing. Depends on the type of + georeferencing. + + order + + The polynomial order to use for GCP related georeferencing. + + projection + + The projection to use for ungeoreferenced images. + + rpc_file_template + + The file glob template to use to find the RPC file. Template + parameters are {filename}, {fileroot}, and {extension}. + + warp_options + + Warp options. See + https://gdal.org/python/osgeo.gdal-module.html#WarpOptions for + details + + corner_names + + The metadata field name including the corner names. Tuple of four: + bottom-left, bottom-right, top-left and top-right + + orbit_direction_name + + The metadata field name containing the orbit direction + + force_north_up + + TODO + + tps + + Whether to use TPS transformation instead of GCP polynomials. + + calc + + Calculate derived data using formulas. + + formulas + + A list of formulas to use to calculate derived data. Each has the + following fields + + inputs + + A map of characters in the range of A-Z to respective inputs. Each + has the following properties + + glob + + The input file glob + + band + + The input file band index (1-based) + + data_type + + The GDAL data type name for the output + + formula + + The formula to apply. See + https://gdal.org/programs/gdal_calc.html#cmdoption-calc for details. + + output_postfix + + The postfix to apply for the filename of the created file. + + nodata_value + + The nodata value to be used. + + stack_bands + + Concatenate bands and arrange them in a single file. + + group_by + + A regex to group the input datasets, if consisting of multiple file. + The first regex group is used for the grouping. + + sort_by + + A regex to select a portion of the filename to be used for sorting. The + first regex group is used. + + order + + The order of the extracted item used in 'sort_by'. When the value + extracted by ``sort_by`` is missing, then that file will be dropped. + + output + + Final adjustments to generate an output file. Add overviews, reproject to + a common projection, etc. + + options + + Options to be passed to `gdal.Warp`. See + https://gdal.org/python/osgeo.gdal-module.html#WarpOptions for details. + + custom_preprocessor + + A custom python function to be called. + + path + + The Python module path to the function to call. + + args + + A list of arguments to pass to the function. + + kwargs + + A dictionary of keyword arguments to pass to the function. + + types + + This mapping of product type identifier to step configuration allows to + define specific step settings, even overriding the values from the + defaults. The next section :ref:`management` describes how an operator interacts with a deployed VS stack.