buildrepo without needing to have all apks locally
We need to be able to build a repository without needing to have all files locally.
Here is what needs to be happen:
- get the APKINDEX from remote repository using
rsync
. - get the filelist of files on remote repository
rsync --list-only
- compare remote filelist with aports in git. Generate a list of files that differs. lets call it
rsync-list.txt
for now. - build the packages that are missing in the remote filelist. (eg new/updated packages)
- when generating local index, use
apk index --merge --prune-origin --index APKINDEX *.apk
to update the APKINDEX from remote. (and sign it) - add APKINDEX to
rsync-list.txt
- upload packages, delete old remote files (updated or those that have been removed from git) and delete files locally once they are transferred. The
rsync
command for this is:rsync --archive --update --verbose --delete-after --delay-updates \ --include-from=rsync-list.txt \ --exclude='*' \ --remove-source-files \ --mkpath \ $source $dest
The --update
will make sure that the updated APKINDEX overwrites the remote.
The --delete-after
will make sure that files in rsync-list.txt
that does not exist locally (eg aports that has been deleted, files that has been updated with newer versions), are deleted on the remote, and that deletion happens after new files are transferred.
The --delay-updates
will make sure that the transfer is as close to atomic as possible. files are transferred to temp files and renamed after evertying is transferred.
The --include-from
and --exclude='*'
makes sure that we only copy built packages and delete only what is is rsync-list.txt
.
The --remove-source-files
remove the packages locally once they are transferred so we save space on local builder.
Currently we do the rsync from aports-build
. We need to change so we do that from buildrepo.lua. Possibly via a hook.