core merge requestshttps://gitlab.eox.at/vs/core/-/merge_requests2023-05-30T12:07:53+02:00https://gitlab.eox.at/vs/core/-/merge_requests/23feat: add storage streaming2023-05-30T12:07:53+02:00Lubomir Doležalfeat: add storage streaming- depends on https://github.com/EOxServer/eoxserver/pull/557 and new EOxServer release- depends on https://github.com/EOxServer/eoxserver/pull/557 and new EOxServer releaseLubomir DoležalLubomir Doležalhttps://gitlab.eox.at/vs/core/-/merge_requests/22adding simplify_footprint_tolerance option2023-05-30T12:07:54+02:00Lubomir Doležaladding simplify_footprint_tolerance optionhttps://gitlab.eox.at/vs/core/-/merge_requests/21update register_stac_product params2023-05-30T12:07:54+02:00Lubomir Doležalupdate register_stac_product paramsdepends on https://github.com/EOxServer/eoxserver/pull/553 being merged and a new release donedepends on https://github.com/EOxServer/eoxserver/pull/553 being merged and a new release donehttps://gitlab.eox.at/vs/core/-/merge_requests/20replacing in registrar only handled in config.yaml2023-05-30T12:07:54+02:00Lubomir Doležalreplacing in registrar only handled in config.yamlcloses #26closes #26https://gitlab.eox.at/vs/core/-/merge_requests/19Zarr registration temp2023-05-30T12:07:54+02:00Nikola JankovicZarr registration temphttps://gitlab.eox.at/vs/core/-/merge_requests/18Core documentation2023-05-30T12:07:54+02:00Anna RomanovaCore documentationLubomir DoležalLubomir Doležalhttps://gitlab.eox.at/vs/core/-/merge_requests/17Use `exec` instead of `eval` in entrypoint2023-05-30T12:07:53+02:00Bernhard MallingerUse `exec` instead of `eval` in entrypointThis way, gunicorn replaces the initial bash process and receives
signals, such that it terminates quickly on SIGTERM.This way, gunicorn replaces the initial bash process and receives
signals, such that it terminates quickly on SIGTERM.https://gitlab.eox.at/vs/core/-/merge_requests/16updated configuration and cli2022-09-01T14:31:15+02:00Nikola Jankovicupdated configuration and cliNikola JankovicNikola Jankovichttps://gitlab.eox.at/vs/core/-/merge_requests/15Registration routes2022-08-03T14:19:16+02:00Fabian SchindlerRegistration routesAdding implementation routes
A `Route` is bound to a `queue` and operates on a particular data-type (such as STAC Item, Collection, paths, arbitrary JSON structures).
`Backends` are also now generic according to its datatype and are no...Adding implementation routes
A `Route` is bound to a `queue` and operates on a particular data-type (such as STAC Item, Collection, paths, arbitrary JSON structures).
`Backends` are also now generic according to its datatype and are now part of the route.
Each route also encapsulates the `pre`, `post`, and `error` handlers.
Example configuration:
```yaml
routes:
items:
path: registrar.route.stac.ItemRoute
queue: register
output_queue: next_queue
backends:
- path: registrar.backend.eoxserver.ItemBackend
kwargs:
instance_base_path: /some/dir
instance_name: name
handlers:
pre:
- path: some.pre.handler
post:
- path: some.post.handler
error:
- path: some.error.handler
collections:
path: registrar.route.stac.CollectionRoute
queue: register-collections
backends:
- path: registrar.backend.eoxserver.CollectionBackend
path:
path: registrar.route.path.PathRoute
queue: register-paths
backends:
- path: some.path.backend
sources: [...]
```
This is how a STAC Item route can be implemented:
```python
import json
import pystac
class ItemRoute(Route[pystac.Item]):
"""A route implementation for STAC Items"""
def parse(self, raw: str) -> pystac.Item:
return pystac.Item.from_dict(json.loads(raw))
def get_source(
self, source_cfgs: List[SourceConfig], item: pystac.Item
) -> Optional[Source]:
return get_source(
source_cfgs, [asset.href for asset in item.assets.values()]
)
```
This is how the `Backend` interface can be satisfied:
```python
from pystac import Item
# ...
class MyItemBackend(Backend[Item]):
def exists(self, source: Optional[Source], item: Item) -> bool:
...
def register(self, source: Optional[Source], item: Item, replace: bool):
...
def deregister(self, source: Optional[Source], item: Item):
...
def deregister_identifier(self, identifier: str):
...
```https://gitlab.eox.at/vs/core/-/merge_requests/14test fixed datetime and pystac 1.62022-07-26T15:14:03+02:00Lubomir Doležaltest fixed datetime and pystac 1.6https://gitlab.eox.at/vs/core/-/merge_requests/13remove redis sets from registrar daemon2022-03-11T11:42:15+01:00Lubomir Doležalremove redis sets from registrar daemon- solves the memory issues when items registered were saved to redis
- all necessary info can be inferred from logs- solves the memory issues when items registered were saved to redis
- all necessary info can be inferred from logshttps://gitlab.eox.at/vs/core/-/merge_requests/12use structlog2022-02-28T13:59:14+01:00Bernhard Mallingeruse structloghttps://gitlab.eox.at/vs/core/-/merge_requests/11http access links in OpenSearch2022-02-11T18:03:06+01:00Fabian Schindlerhttp access links in OpenSearchEnabling feed result link generator for http accessEnabling feed result link generator for http accesshttps://gitlab.eox.at/vs/core/-/merge_requests/10Make worker timeout configurable using WORKER_TIMEOUT2022-02-07T15:57:33+01:00Bernhard MallingerMake worker timeout configurable using WORKER_TIMEOUTThis is necessary because downloads can take way longer than 120 secThis is necessary because downloads can take way longer than 120 sechttps://gitlab.eox.at/vs/core/-/merge_requests/9allow overwrite deployment env vars by values2022-01-31T13:39:30+01:00Lubomir Doležalallow overwrite deployment env vars by valuesChanging the order of definition, it actually allows to overwrite the `INIT_SCRIPTS` etc. `env` variables by values.
Not sure if this does not break any existing deployments though.Changing the order of definition, it actually allows to overwrite the `INIT_SCRIPTS` etc. `env` variables by values.
Not sure if this does not break any existing deployments though.https://gitlab.eox.at/vs/core/-/merge_requests/8registrar config remove browses and default_browse2022-01-26T11:09:29+01:00Lubomir Doležalregistrar config remove browses and default_browse- removed from supplied values when creating registrar config
@jankovicn @fabian.schindler Could you please confirm, if this change does not break any of existing deployments. That would mean where browse types would not be created in i...- removed from supplied values when creating registrar config
@jankovicn @fabian.schindler Could you please confirm, if this change does not break any of existing deployments. That would mean where browse types would not be created in init_db and the deployment depends on registrar creating the browse type definitions.
Fixes #10https://gitlab.eox.at/vs/core/-/merge_requests/7Configure http access2022-01-21T12:46:05+01:00Nikola JankovicConfigure http accessLubomir DoležalLubomir Doležalhttps://gitlab.eox.at/vs/core/-/merge_requests/6Hotfix product type found2022-01-13T14:54:41+01:00Nikola JankovicHotfix product type foundFabian SchindlerFabian Schindlerhttps://gitlab.eox.at/vs/core/-/merge_requests/5syncing registrar config values2021-12-10T14:55:08+01:00Nikola Jankovicsyncing registrar config valuesFabian SchindlerFabian Schindlerhttps://gitlab.eox.at/vs/core/-/merge_requests/4set optional to alpha2021-12-01T12:18:21+01:00Nikola Jankovicset optional to alpha