Module util

Structs§

copy_file
copy_files
copy_recursively: Copies all files in the directory from the source store to the destination store.
delete_files
delete_recursively
exists
find_all_dirs_with_epoch_prefix: This function will find all child directories in the input store which are of the form “epoch_num” and return a map of epoch number to the directory path
find_all_files_with_epoch_prefix: This function will find all child directories in the input store which are of the form “epoch_num” and return a map of epoch number to the directory path
find_missing_epochs_dirs: This function will find missing epoch directories in the input store and return a list of such epoch numbers. If the highest epoch directory in the store is epoch_N then it is expected that the store will have all epoch directories from epoch_0 to epoch_N. Additionally, any epoch directory should have the passed in marker file present or else that epoch number is already considered as missing. The returned list will contain epoch_N+1.
get
get_path
list_all_epochs: Finds all epochs in the store and returns them as a sorted list.
path_to_filesystem
put: Writes bytes in the store with specified path.
run_manifest_update_loop: Writes the epochs existed in the store to the root MANIFEST (contains only a list of epochs in the store) every 300 seconds.
write_snapshot_manifest