ModulesΒ§
- checkpoint π
- delta π
- size_
tracker π - staging π
- utils π
StructsΒ§
- CHECKPOINT_
RE π - DELTA_
RE π - Manifest
Object Store
ConstantsΒ§
- DEFAULT_
MANIFEST_ πCOMPRESSION_ TYPE - FALL_
BACK_ πCOMPRESS_ TYPE - Due to backward compatibility, it is possible that the userβs manifest file has not been compressed.
So when we encounter problems, we need to fall back to
FALL_BACK_COMPRESS_TYPEfor processing. - FETCH_
MANIFEST_ πPARALLELISM - LAST_
CHECKPOINT_ FILE
FunctionsΒ§
- checkpoint_
checksum π - checkpoint_
file - delta_
file - file_
compress_ type - Returnβs the file compress algorithm by file extension.
- file_
version - Returnβs the file manifest version from path
- gen_
path - is_
checkpoint_ file - is_
delta_ file - manifest_
compress_ type - Returns the [CompressionType] according to whether to compress manifest files.
- manifest_
dir - Returns the directory to the manifest files.
- verify_
checksum π