EOX GitLab Instance
Skip to content
GitLab
Projects
Groups
Snippets
/
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Sign in
Toggle navigation
Menu
Open sidebar
View Server 2
preprocessor
Commits
bd462eaa
Commit
bd462eaa
authored
Jan 19, 2022
by
Nikola Jankovic
💻
Browse files
updated product_exists to match new scheme
parent
3ae9a71f
Pipeline
#20776
passed with stage
in 1 minute and 9 seconds
Changes
1
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
preprocessor/transfer/s3.py
View file @
bd462eaa
import
os
from
functools
import
cached_property
from
typing
import
TYPE_CHECKING
,
Union
,
Dict
,
List
from
typing
import
TYPE_CHECKING
,
Union
,
Dict
,
List
,
Tuple
import
boto3
import
botocore
...
...
@@ -92,9 +92,11 @@ class S3Uploader(Base, Uploader):
return
local_remote_mapping
def
product_exists
(
self
,
remote_dir
:
str
)
->
bool
:
def
product_exists
(
self
,
remote_dir
:
str
,
expected_files_count
:
int
=
2
)
->
Tuple
[
bool
,
List
[
str
]]:
objects
=
self
.
client
.
list_objects_v2
(
Bucket
=
self
.
bucket
,
Prefix
=
remote_dir
)
if
len
(
objects
[
"Contents"
])
>=
2
:
return
True
if
len
(
objects
[
"Contents"
])
>=
expected_files_count
:
return
True
,
[
obj
[
"Key"
]
for
obj
in
objects
[
"Contents"
]]
return
False
return
False
,
[]
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment