Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
aports
aports
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 642
    • Issues 642
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Merge Requests 195
    • Merge Requests 195
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
  • Operations
    • Operations
    • Incidents
    • Environments
  • Analytics
    • Analytics
    • CI / CD
    • Repository
    • Value Stream
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
  • alpine
  • aportsaports
  • Issues
  • #1240

Closed
Open
Opened Jul 03, 2012 by aa latchm@aalatchm

busybox xargs causes lbu to truncate file list when committing large number of files

On Alpine 2.4.3, tested busybox-1.19.4-r4 with alpine-conf-2.11.2-r0. Also tested busybox-1.20.1-r1.apk from edge repository.

With a large file list of >32768 bytes, I found that lbu would only commit the files in the list from approximately the first 32768 bytes.

Since http://git.alpinelinux.org/cgit/alpine-conf/commit/?id=4e857d18684667e7d9df3a0256e0b6e446284641, lbu provides the included file list via stdin to xargs, rather than a shell variable:

                        _gen_filelist_0 | xargs -0 $tar_create \
                                | gzip -c  >"$tmppkg"

which expands to:

apk audit --backup --quiet --recursive | tr '\n' '\0' |\
     xargs -0 tar -c --no-recursion | gzip -c >"$tmppkg"

Therefore this bug can now be triggered when using lbu include to store a large list of files.

To easily duplicate the fundamental problem in busybox xargs:

mkdir test
cd test
for a in `seq 1000 2000`; do
  echo 'hello' > aaaaabbbbbcccccdddddeeeeefffffggggghhhhh-$a
done
ls -1 | xargs tar -c --no-recursion | tar -t

At first I thought this commit might have been at fault http://git.busybox.net/busybox/commit/findutils/xargs.c?h=1\_19\_stable&id=4a96617095f40e3cfcf148a8a7d5d83f71079aa1, but I also tested Alpine 1.10 with busybox-1.16.2-r2 and duplicated the problem.

Running apk add findutils works around the problem on all systems tested.

(Running apk add tar or removing the —no-recursion option has no effect on this problem.)

(from redmine: issue id 1240, created on 2012-07-03, closed on 2012-07-08)

  • Changesets:
    • Revision 15245c4a0eb62b0e1bcbac5063391e0d57a53799 by Natanael Copa on 2012-07-04T08:09:21Z:
lbu: do not use xargs

xargs will execute the command twice if the buffer becomes long enough.
This will break tar.

We also use -z instead of piping via gzip while here.

ref #1240
  • Revision 1ab89daa by Natanael Copa on 2012-07-04T15:07:45Z:
main/alpine-conf: lbu: do not use xargs

ref #1240
  • Revision b16dbb24 by Natanael Copa on 2012-07-04T15:09:30Z:
main/alpine-conf: lbu: do not use xargs

fixes #1240
(cherry picked from commit 1ab89daa7c0d4b46230a2608f04151214a266101)
To upload designs, you'll need to enable LFS and have admin enable hashed storage. More information
Assignee
Assign to
Alpine 2.4.5
Milestone
Alpine 2.4.5 (Past due)
Assign milestone
Time tracking
None
Due date
None
Reference: alpine/aports#1240