EOX GitLab Instance
Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
C
core
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Package registry
Container Registry
Model registry
Operate
Environments
Terraform modules
Monitor
Incidents
Service Desk
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Terms and privacy
Keyboard shortcuts
?
Snippets
Groups
Projects
This is an archived project. Repository and other project resources are read-only.
Show more breadcrumbs
Data Handling
core
Commits
bd23ca8c
Commit
bd23ca8c
authored
3 years ago
by
Fabian Schindler
Browse files
Options
Downloads
Patches
Plain Diff
Adding registrar documentation to README
parent
a61f8df4
No related branches found
Branches containing commit
No related tags found
Tags containing commit
1 merge request
!1
Adding path registration capabilities
Pipeline
#19112
failed
3 years ago
Stage: test
Changes
1
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
README.md
+138
-0
138 additions, 0 deletions
README.md
with
138 additions
and
0 deletions
README.md
+
138
−
0
View file @
bd23ca8c
...
...
@@ -34,3 +34,141 @@ black file.py # to autoformat a file. also possible with a path
Or these tools can also be integrated into your IDE/Editor.
-
[
VSCODE
](
https://code.visualstudio.com/docs/python/linting
)
## Registrar
### Configuration
The
`registrar`
module is configured using a YAML based configuration file. It must be
declared using the
`--config-file`
parameter.
The file contains the following sections:
#### `sources`
In this section, the data sources can be declared. These are the references to the
storage location where the data to be registered is stored. This includes local (or
locally mounted but remote) directories, S3 buckets or OpenStack Swift storages.
Examples:
```
yaml
sources
:
-
name
:
my-local-source
type
:
local
kwargs
:
root_directory
:
/opt/data
-
name
:
my-s3-source
type
:
s3
kwargs
:
bucket_name
:
mybucket
secret_access_key
:
mysecret
access_key_id
:
myaccess
endpoint_url
:
data.storage.com
strip_bucket
:
True
validate_bucket_name
:
False
region_name
:
RegionA
public
:
False
-
name
:
my-swift-source
type
:
swift
kwargs
:
username
:
myusername
password
:
mypassword
tenant_name
:
mytenant_name
tenant_id
:
mytenant_id
region_name
:
myregion_name
user_domain_id
:
myuser_domain_id
user_domain_name
:
myuser_domain_name
auth_url
:
myauth_url
auth_url_short
:
myauth_url_short
auth_version
:
myauth_version
container
:
mycontainer
```
#### `backends`
A backend is where the objects are registered _into_. This part of the
registrar uses a plugin architecture, as in the future, additional backends
may need to be added.
There are two distinct types of backends: STAC Item based backends, registering
products described by STAC items and path based backends, referencing files or
directories to be registered.
Each backend supports both registration and de-registration.
The only built-in backend is the EOxServer.
Example configuration:
```
yaml
backends
:
-
path
:
registrar.backends.eoxserver.EOxServerBackend
kwargs
:
instance_base_path
:
/opt/instances
instance_bane
:
myproj
product_types
:
-
name
:
Sentinel-2 Level 1C
product_type
:
S2L1C
filter
:
s2:product_type
:
S2MSI1C
coverages
:
# Mapping EOxServer coverage type to assets
S2L1C_B01
:
assets
:
[
B01
]
# ...
browses
:
TRUE_COLOR
:
TCI
metadata_assets
:
[
metadata
]
collections
:
[
S2L1C
,
S2
]
-
name
:
Sentinel-2 Level 2A
product_type
:
S2L2A
filter
:
s2:product_type
:
S2MSI2A
coverages
:
# Mapping EOxServer coverage type to assets
S2L1C_B01
:
assets
:
[
B01
]
# ...
browses
:
TRUE_COLOR
:
TCI_10m
metadata_assets
:
[
metadata
]
collections
:
[
S2L2A
,
S2
]
path_backends
:
-
path
:
registrar_pycsw.backends.PycswBackend
kwargs
:
repository_database_uri
:
username:pwd@host
ows_url
:
https://access.service.org/ows
public_s3_url
:
https://storage.service.org
```
#### Handlers
These are hooks to handle registration events. There are three distinct kinds
of events:
-
pre-(de-)registration
-
post-(de-)registration
-
errors
Each hook can be supplied a list of handlers, which is python path to a
callable object.
These handlers exist for both item and path based registrations, resulting
in the following list of configurations to be set (with the passed parameters in
brackets):
-
`pre_handlers`
(config, item)
-
`post_handlers`
(config, item)
-
`error_handlers`
(config, item, exception)
-
`path_pre_handlers`
(config, path)
-
`path_post_handlers`
(config, path)
-
`path_error_handlers`
(config, path, exception)
-
`deregister_pre_handlers`
(config, identifier, item)
-
`deregister_post_handlers`
(config, identifier, item)
-
`deregister_error_handlers`
(config, identifier, item, exception)
-
`deregister_path_pre_handlers`
(config, path)
-
`deregister_path_post_handlers`
(config, path)
-
`deregister_path_error_handlers`
(config, path, exception)
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment