EOX GitLab Instance

Skip to content
Snippets Groups Projects

Draft: refactoring of registrar

Open Nikola Jankovic requested to merge configuration-performance-updates into main
10 unresolved threads
  • changed eoxserver
  • added app concept for daemon mode
  • brought in vs common
  • fixed configuration handling
  • removed traces of creating types and storages in favor of adding values directly
  • moved to src
Edited by Nikola Jankovic

Merge request reports

Pipeline #37549 failed

Pipeline failed for e396dd0a on configuration-performance-updates

Approval is optional
Merge unavailable: merge requests are read-only on archived projects.

Merge details

  • The source branch is 14 commits behind the target branch.
  • 8 commits and 1 merge commit will be added to main.
  • Source branch will be deleted.

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
39 """Run the registrar daemon, listening on registrar redis queues for jobs."""
40 # initialize the queue client
41
42 queue_map = {cfg.queue: name for name, cfg in self.config.registrars.items()}
43 queues = list(self.config.queues.keys())
44 LOGGER.debug("Waiting on queues...", queues=queues)
45 result = self.client.brpop(queues, timeout=TIMEOUT)
46
47 if result is not None:
48 queue, value = result
49 name = queue_map[queue]
50 try:
51 main(self.config, name, value)
52 except Exception as e:
53 LOGGER.exception(e)
54 if self.config.queues[queue] is not None:
  • 6
    7
    8 class Endpoint(ABC):
    9 """An endpoint handles the given value with"""
    10
    11 def __init__(self, *args, **kwargs) -> None:
    12 ...
    13
    14 @property
    15 @abstractmethod
    16 def command_input_matrix(
    17 self,
    18 ) -> Dict[Command, Dict[InputType, EndpointHandleInputFunction]]:
    19 """Commands and inputs the endpoint implements, mapping to internal handler
    20 functions"""
    21 return {}
  • 30 @dataclass
    31 class CommandConfig:
    32 type: Command
    33 replace: bool = False
    34
    35
    36 @dataclass
    37 class CustomInputConfig:
    38 path: str
    39 args: List[Any]
    40 kwargs: Dict[str, Any]
    41
    42
    43 @dataclass
    44 class JsonInputConfig:
    45 href: str
  • 75 handlers: Optional[HandlersConfig] = None
    76
    77
    78 @dataclass
    79 class BandConfig:
    80 expression: str
    81 nodata: int
    82 range: List[int]
    83
    84
    85 @dataclass
    86 class BrowseConfig:
    87 red: Optional[BandConfig] = None
    88 green: Optional[BandConfig] = None
    89 blue: Optional[BandConfig] = None
    90 grey: Optional[BandConfig] = None
  • 93 @dataclass
    94 class EOXServerFilesystemConfig:
    95 container: str
    96 name: str
    97
    98
    99 @dataclass
    100 class ProductConfig:
    101 product_name: str
    102 filter: Dict[str, Any]
    103
    104
    105 @dataclass
    106 class EOXServerEndpointConfig:
    107 coverage_asset_map: Dict[str, Any]
    108 collection_storage_map: Dict[str, str]
  • 86 class BrowseConfig:
    87 red: Optional[BandConfig] = None
    88 green: Optional[BandConfig] = None
    89 blue: Optional[BandConfig] = None
    90 grey: Optional[BandConfig] = None
    91
    92
    93 @dataclass
    94 class EOXServerFilesystemConfig:
    95 container: str
    96 name: str
    97
    98
    99 @dataclass
    100 class ProductConfig:
    101 product_name: str
  • 92
    93 @dataclass
    94 class EOXServerFilesystemConfig:
    95 container: str
    96 name: str
    97
    98
    99 @dataclass
    100 class ProductConfig:
    101 product_name: str
    102 filter: Dict[str, Any]
    103
    104
    105 @dataclass
    106 class EOXServerEndpointConfig:
    107 coverage_asset_map: Dict[str, Any]
  • 120 handle_function = self.command_input_matrix[command_config.type][
    121 input_config.type
    122 ]
    123 prepared_input = parse_input(input_config, value)
    124 handle_function(command_config, prepared_input)
    125
    126 @transaction.atomic
    127 def _register_stac_item(
    128 self,
    129 command_config: CommandConfig,
    130 item: "Item",
    131 ) -> None:
    132 collection_name = item.collection_id
    133
    134 if not collection_name:
    135 raise ValueError("Collection name must be set")
  • 139 return
    140
    141 storage_name = self.collection_storage_map[collection_name]
    142 product_filters = self.collection_product_map[collection_name]
    143 if len(product_filters) == 1:
    144 product_type_name = product_filters[0].product_name
    145 else:
    146 for pf in product_filters:
    147 if pf.filter.items() <= item.to_dict().items():
    148 product_type_name = pf.product_name
    149 break
    150 else:
    151 raise ValueError(
    152 f"""Filter error, couldn't match any
    153 {product_filters} to {item.to_dict()}"""
    154 )
  • 94 class EOXServerFilesystemConfig:
    95 container: str
    96 name: str
    97
    98
    99 @dataclass
    100 class ProductConfig:
    101 product_name: str
    102 filter: Dict[str, Any]
    103
    104
    105 @dataclass
    106 class EOXServerEndpointConfig:
    107 coverage_asset_map: Dict[str, Any]
    108 collection_storage_map: Dict[str, str]
    109 collection_product_map: Dict[str, List[ProductConfig]]
    Please register or sign in to reply
    Loading