libvirt getting locked while forking when taking a domain screenshot on Alpine 3.12 x86_64
This issue is related to 11602. While taking a domain screenshot libvirt process gets locked while forking. Unfortunately the issue happens randomly so it's not that easy to reproduce but when it happens the only way to move on is to terminate the child process created.
Please find below gbd output:
GNU gdb (GDB) 9.2
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-alpine-linux-musl".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word".
Attaching to process 18541
Reading symbols from /usr/sbin/libvirtd...
(No debugging symbols found in /usr/sbin/libvirtd)
Reading symbols from /usr/lib/libvirt-lxc.so.0...
(No debugging symbols found in /usr/lib/libvirt-lxc.so.0)
Reading symbols from /usr/lib/libvirt-qemu.so.0...
(No debugging symbols found in /usr/lib/libvirt-qemu.so.0)
Reading symbols from /usr/lib/libvirt.so.0...
(No debugging symbols found in /usr/lib/libvirt.so.0)
Reading symbols from /usr/lib/libtirpc.so.3...
(No debugging symbols found in /usr/lib/libtirpc.so.3)
Reading symbols from /usr/lib/libdbus-1.so.3...
(No debugging symbols found in /usr/lib/libdbus-1.so.3)
Reading symbols from /usr/lib/libgobject-2.0.so.0...
(No debugging symbols found in /usr/lib/libgobject-2.0.so.0)
Reading symbols from /usr/lib/libglib-2.0.so.0...
(No debugging symbols found in /usr/lib/libglib-2.0.so.0)
Reading symbols from /usr/lib/libintl.so.8...
(No debugging symbols found in /usr/lib/libintl.so.8)
Reading symbols from /usr/lib/libgcc_s.so.1...
(No debugging symbols found in /usr/lib/libgcc_s.so.1)
Reading symbols from /lib/ld-musl-x86_64.so.1...
Reading symbols from /usr/lib/debug//lib/ld-musl-x86_64.so.1.debug...
--Type <RET> for more, q to quit, c to continue without paging--
Reading symbols from /usr/lib/libcap-ng.so.0...
(No debugging symbols found in /usr/lib/libcap-ng.so.0)
Reading symbols from /usr/lib/libyajl.so.2...
(No debugging symbols found in /usr/lib/libyajl.so.2)
Reading symbols from /usr/lib/libnl-3.so.200...
(No debugging symbols found in /usr/lib/libnl-3.so.200)
Reading symbols from /usr/lib/libxml2.so.2...
(No debugging symbols found in /usr/lib/libxml2.so.2)
Reading symbols from /usr/lib/libgio-2.0.so.0...
(No debugging symbols found in /usr/lib/libgio-2.0.so.0)
Reading symbols from /usr/lib/libsasl2.so.3...
(No debugging symbols found in /usr/lib/libsasl2.so.3)
Reading symbols from /usr/lib/libgnutls.so.30...
(No debugging symbols found in /usr/lib/libgnutls.so.30)
Reading symbols from /usr/lib/libcurl.so.4...
(No debugging symbols found in /usr/lib/libcurl.so.4)
Reading symbols from /usr/lib/libgssapi_krb5.so.2...
(No debugging symbols found in /usr/lib/libgssapi_krb5.so.2)
Reading symbols from /usr/lib/libffi.so.7...
(No debugging symbols found in /usr/lib/libffi.so.7)
Reading symbols from /usr/lib/libpcre.so.1...
(No debugging symbols found in /usr/lib/libpcre.so.1)
Reading symbols from /lib/libz.so.1...
(No debugging symbols found in /lib/libz.so.1)
Reading symbols from /usr/lib/liblzma.so.5...
(No debugging symbols found in /usr/lib/liblzma.so.5)
Reading symbols from /usr/lib/libgmodule-2.0.so.0...
(No debugging symbols found in /usr/lib/libgmodule-2.0.so.0)
Reading symbols from /lib/libmount.so.1...
(No debugging symbols found in /lib/libmount.so.1)
Reading symbols from /usr/lib/libp11-kit.so.0...
(No debugging symbols found in /usr/lib/libp11-kit.so.0)
Reading symbols from /usr/lib/libunistring.so.2...
(No debugging symbols found in /usr/lib/libunistring.so.2)
Reading symbols from /usr/lib/libtasn1.so.6...
(No debugging symbols found in /usr/lib/libtasn1.so.6)
Reading symbols from /usr/lib/libnettle.so.7...
(No debugging symbols found in /usr/lib/libnettle.so.7)
--Type <RET> for more, q to quit, c to continue without paging--
Reading symbols from /usr/lib/libhogweed.so.5...
(No debugging symbols found in /usr/lib/libhogweed.so.5)
Reading symbols from /usr/lib/libgmp.so.10...
(No debugging symbols found in /usr/lib/libgmp.so.10)
Reading symbols from /usr/lib/libnghttp2.so.14...
(No debugging symbols found in /usr/lib/libnghttp2.so.14)
Reading symbols from /lib/libssl.so.1.1...
(No debugging symbols found in /lib/libssl.so.1.1)
Reading symbols from /lib/libcrypto.so.1.1...
(No debugging symbols found in /lib/libcrypto.so.1.1)
Reading symbols from /usr/lib/libkrb5.so.3...
(No debugging symbols found in /usr/lib/libkrb5.so.3)
Reading symbols from /usr/lib/libk5crypto.so.3...
(No debugging symbols found in /usr/lib/libk5crypto.so.3)
Reading symbols from /lib/libcom_err.so.2...
(No debugging symbols found in /lib/libcom_err.so.2)
Reading symbols from /usr/lib/libkrb5support.so.0...
(No debugging symbols found in /usr/lib/libkrb5support.so.0)
Reading symbols from /lib/libblkid.so.1...
(No debugging symbols found in /lib/libblkid.so.1)
Reading symbols from /usr/lib/libkeyutils.so.1...
(No debugging symbols found in /usr/lib/libkeyutils.so.1)
Reading symbols from /usr/lib/libvirt/connection-driver/libvirt_driver_network.so...
(No debugging symbols found in /usr/lib/libvirt/connection-driver/libvirt_driver_network.so)
Reading symbols from /usr/lib/libvirt/connection-driver/libvirt_driver_interface.so...
(No debugging symbols found in /usr/lib/libvirt/connection-driver/libvirt_driver_interface.so)
Reading symbols from /usr/lib/libnetcf.so.1...
(No debugging symbols found in /usr/lib/libnetcf.so.1)
Reading symbols from /lib/libudev.so.1...
(No debugging symbols found in /lib/libudev.so.1)
Reading symbols from /usr/lib/libaugeas.so.0...
(No debugging symbols found in /usr/lib/libaugeas.so.0)
Reading symbols from /usr/lib/libexslt.so.0...
(No debugging symbols found in /usr/lib/libexslt.so.0)
Reading symbols from /usr/lib/libxslt.so.1...
(No debugging symbols found in /usr/lib/libxslt.so.1)
Reading symbols from /usr/lib/libnl-route-3.so.200...
(No debugging symbols found in /usr/lib/libnl-route-3.so.200)
--Type <RET> for more, q to quit, c to continue without paging--
Reading symbols from /usr/lib/libfa.so.1...
(No debugging symbols found in /usr/lib/libfa.so.1)
Reading symbols from /usr/lib/libgcrypt.so.20...
(No debugging symbols found in /usr/lib/libgcrypt.so.20)
Reading symbols from /usr/lib/libgpg-error.so.0...
(No debugging symbols found in /usr/lib/libgpg-error.so.0)
Reading symbols from /usr/lib/libvirt/connection-driver/libvirt_driver_secret.so...
(No debugging symbols found in /usr/lib/libvirt/connection-driver/libvirt_driver_secret.so)
Reading symbols from /usr/lib/libvirt/connection-driver/libvirt_driver_storage.so...
(No debugging symbols found in /usr/lib/libvirt/connection-driver/libvirt_driver_storage.so)
Reading symbols from /usr/lib/libvirt/storage-backend/libvirt_storage_backend_fs.so...
(No debugging symbols found in /usr/lib/libvirt/storage-backend/libvirt_storage_backend_fs.so)
Reading symbols from /usr/lib/libvirt/storage-backend/libvirt_storage_backend_logical.so...
(No debugging symbols found in /usr/lib/libvirt/storage-backend/libvirt_storage_backend_logical.so)
Reading symbols from /usr/lib/libvirt/storage-backend/libvirt_storage_backend_scsi.so...
(No debugging symbols found in /usr/lib/libvirt/storage-backend/libvirt_storage_backend_scsi.so)
Reading symbols from /usr/lib/libvirt/storage-backend/libvirt_storage_backend_mpath.so...
(No debugging symbols found in /usr/lib/libvirt/storage-backend/libvirt_storage_backend_mpath.so)
Reading symbols from /lib/libdevmapper.so.1.02...
(No debugging symbols found in /lib/libdevmapper.so.1.02)
Reading symbols from /usr/lib/libvirt/storage-backend/libvirt_storage_backend_disk.so...
(No debugging symbols found in /usr/lib/libvirt/storage-backend/libvirt_storage_backend_disk.so)
Reading symbols from /usr/lib/libvirt/storage-backend/libvirt_storage_backend_zfs.so...
(No debugging symbols found in /usr/lib/libvirt/storage-backend/libvirt_storage_backend_zfs.so)
Reading symbols from /usr/lib/libvirt/connection-driver/libvirt_driver_nodedev.so...
(No debugging symbols found in /usr/lib/libvirt/connection-driver/libvirt_driver_nodedev.so)
Reading symbols from /usr/lib/libpciaccess.so.0...
(No debugging symbols found in /usr/lib/libpciaccess.so.0)
Reading symbols from /usr/lib/libvirt/connection-driver/libvirt_driver_nwfilter.so...
(No debugging symbols found in /usr/lib/libvirt/connection-driver/libvirt_driver_nwfilter.so)
Reading symbols from /usr/lib/libpcap.so.1...
(No debugging symbols found in /usr/lib/libpcap.so.1)
Reading symbols from /usr/lib/libvirt/connection-driver/libvirt_driver_qemu.so...
(No debugging symbols found in /usr/lib/libvirt/connection-driver/libvirt_driver_qemu.so)
__wait (addr=addr@entry=0x7fe200bfba88 <mal+8>, waiters=waiters@entry=0x7fe200bfba8c <mal+12>, val=val@entry=1, priv=128, priv@entry=1)
at ./arch/x86_64/syscall_arch.h:40
40 ./arch/x86_64/syscall_arch.h: No such file or directory.
(gdb) backtrace
#0 __wait (addr=addr@entry=0x7fe200bfba88 <mal+8>, waiters=waiters@entry=0x7fe200bfba8c <mal+12>, val=val@entry=1, priv=128, priv@entry=1)
at ./arch/x86_64/syscall_arch.h:40
#1 0x00007fe200b8c3ca in lock (lk=0x7fe200bfba88 <mal+8>) at src/malloc/malloc.c:31
#2 lock_bin (i=0) at src/malloc/malloc.c:46
#3 malloc (n=<optimized out>, n@entry=8) at src/malloc/malloc.c:320
#4 0x00007fe200b8c584 in calloc (m=<optimized out>, n=8) at src/malloc/malloc.c:361
#5 0x00007fe20062212e in g_malloc0 () from /usr/lib/libglib-2.0.so.0
#6 0x00007fe20082c732 in virAllocN () from /usr/lib/libvirt.so.0
#7 0x00007fe20092fa50 in ?? () from /usr/lib/libvirt.so.0
#8 0x00007fe200882704 in virProcessRunInFork () from /usr/lib/libvirt.so.0
#9 0x00007fe20092fd22 in ?? () from /usr/lib/libvirt.so.0
#10 0x00007fe200930dbd in virSecurityManagerTransactionCommit () from /usr/lib/libvirt.so.0
#11 0x00007fe20092d8ed in ?? () from /usr/lib/libvirt.so.0
#12 0x00007fe200930dbd in virSecurityManagerTransactionCommit () from /usr/lib/libvirt.so.0
#13 0x00007fe1ff1adad0 in qemuSecurityDomainSetPathLabel () from /usr/lib/libvirt/connection-driver/libvirt_driver_qemu.so
#14 0x00007fe1ff197810 in ?? () from /usr/lib/libvirt/connection-driver/libvirt_driver_qemu.so
#15 0x00007fe2009fbfda in virDomainScreenshot () from /usr/lib/libvirt.so.0
#16 0x0000563c9047f1b8 in ?? ()
#17 0x00007fe200951554 in virNetServerProgramDispatch () from /usr/lib/libvirt.so.0
#18 0x00007fe200955a41 in ?? () from /usr/lib/libvirt.so.0
#19 0x00007fe20089b74c in ?? () from /usr/lib/libvirt.so.0
#20 0x00007fe20089b0cb in ?? () from /usr/lib/libvirt.so.0
#21 0x00007fe200bba7b7 in start (p=0x7fe1ff738858) at src/thread/pthread_create.c:195
#22 0x00007fe200bbc8f0 in __clone () at src/thread/x86_64/clone.s:22
Backtrace stopped: frame did not save the PC
(gdb) Thanks.