Ingestor status
Merge request reports
Activity
- Resolved by Fabian Schindler-Strauss
- Resolved by Fabian Schindler-Strauss
- Resolved by Fabian Schindler-Strauss
- Resolved by Fabian Schindler-Strauss
- Resolved by Fabian Schindler-Strauss
- Resolved by Fabian Schindler-Strauss
- Resolved by Fabian Schindler-Strauss
94 94 try: 95 95 client.lpush(queue_name, queue_content) 96 96 97 save_Json(filename, 'SUCSSES') 97 save_json(filename, queue_content, True) The original browse report (not the JSON encoded one sent to the queue) shall be written to the SUCCESS/FAILURE directories. Also, the
save_json
is not really descriptive (apart from the fact that we want to save XML). What we do, semantically, is to report on the ingestion of a Browse report. Maybe a better name can be found here.Also, in case of SFTP upload, the file should actually be removed from the input directory and placed into the SUCCESS/FAILURE dirs.
changed this line in version 3 of the diff
added 17 commits
-
81d5a1e1...d9be4296 - 15 commits from branch
staging
- b92a74ba - fixing typo
- e219d7d4 - Merge branch 'staging' into ingestor_status
-
81d5a1e1...d9be4296 - 15 commits from branch
38 38 ADD install.sh requirements.txt \ 39 39 / 40 40 RUN /install.sh 41 RUN mkdir /ingestor 41 RUN mkdir /ingestor changed this line in version 5 of the diff
37 40 def converter(o): 38 41 if isinstance(o, datetime): 39 42 return o.isoformat() 43 44 def save_report(filename:str, data, success: bool): changed this line in version 5 of the diff
37 40 def converter(o): 38 41 if isinstance(o, datetime): 39 42 return o.isoformat() 43 44 def save_report(filename:str, data, success: bool): 45 46 if success: 47 save_dir = os.environ['INGESTOR_SUCCESS_DIR'] 48 else: 49 save_dir = os.environ['INGESTOR_FAIL_DIR'] 50 51 if isinstance(data, str): 52 shutil.move(data, save_dir) 53 else : changed this line in version 5 of the diff
37 40 def converter(o): 38 41 if isinstance(o, datetime): 39 42 return o.isoformat() 43 44 def save_report(filename:str, data, success: bool): 45 46 if success: 47 save_dir = os.environ['INGESTOR_SUCCESS_DIR'] 48 else: 49 save_dir = os.environ['INGESTOR_FAIL_DIR'] 50 51 if isinstance(data, str): 52 shutil.move(data, save_dir) 53 else : 54 55 with open(os.path.join(save_dir, '%s.xml' % filename), "w") as outfile: changed this line in version 5 of the diff
95 ) 86 87 with open(event.pathname) as f: 88 browse_report = parse_browse_report(f) 89 logger.debug(browse_report) 90 if isinstance(browse_report, str): 91 queue_content = browse_report 92 93 else: 94 queue_content = json.dumps( 95 browse_report, default=converter 96 ) 97 try: 96 98 client.lpush(queue_name, queue_content) 99 100 save_report('', event.pathname, True) changed this line in version 5 of the diff
added 20 commits
-
e219d7d4...a5c1d3c3 - 18 commits from branch
staging
- 9597fc08 - Merge branch 'staging' into ingestor_status
- 8aa81aad - using 2 saving functions
-
e219d7d4...a5c1d3c3 - 18 commits from branch
added 39 commits
-
8aa81aad...074541a5 - 37 commits from branch
staging
- 2d1f6097 - Merge branch 'staging' into ingestor_status
- 55932afb - enabling more error catching
-
8aa81aad...074541a5 - 37 commits from branch
mentioned in commit 72f9ba19