pip package "python-keystoneclient" causing a segmentation fault on Linux Alpine (docker and vagrant image)
Hello everyone and thanks in advance for your kind help,
I am currently experiencing a repeated bug (I think reproducible) with the python-keystoneclient module specifically on Alpine linux. Having already tried keystoneclient on debian and ubuntu, I never had these problems before.
Here are the 3 steps to reproduce the bug:
Step 1Choose your Alpine version: docker, vagrant or other
A- If you choose docker;
A.1 - Create a Dockerfile as follows:
FROM gliderlabs/alpine:3.3
# Download the necessary packages
RUN apk add --update \
gcc \
mariadb-dev \
build-base \
linux-headers \
python \
python-dev \
py-pip \
&& pip install virtualenv
# Install the pip requirements
RUN virtualenv /env && \
/env/bin/pip install mysql-python python-keystoneclient python-heatclient
# Clean package cache
RUN rm -rf /var/cache/apk/* /tmp/* /var/tmp/*
*PS: I chose to leave my dependencies installed as in my environment,
even if not necessary for the bug tests, in case a conflict with those
packages is something that could give a hint to a possible solution.
*
A.2 - Build the docker image by running this command (I chose the
initials for the name “alpine keystone client bug test”):
docker build -t=alpKSc:bugTest .
A.3 - Run the image with the command:
docker run -ti alpKSc:bugTest /bin/sh
B- If you choose vagrant as your provider for your virtualbox;
B.1 - Create the following Vagrantfile (comment the vbguest line if you
do not have the plugin onyour vagrant install):
# -*- mode: ruby -*-
# vi: set ft=ruby :
# All Vagrant configuration is done below. The "2" in Vagrant.configure
# configures the configuration version (we support older styles for
# backwards compatibility). Please don't change it unless you know what
# you're doing.
Vagrant.configure(2) do |config|
config.vm.box = "maier/alpine-3.1.3-x86_64"
config.vbguest.auto_update = false
config.vm.network "forwarded_port", guest: 8000, host: 8001
config.vm.network "private_network", ip: "192.168.30.2"
config.vm.provision "shell", inline: <<-SHELL
apk add --update \
gcc \
mariadb-dev \
build-base \
linux-headers \
python \
python-dev \
py-pip \
&& pip install virtualenv
virtualenv /env &&
/env/bin/pip install mysql-python python-keystoneclient python-heatclient
SHELL
end
B.2 - You have to run the command:
vagrant up --provision
B.2bis - If you have problems for package installations, try to update the /etc/apk/repositories to look like this:
http://dl-1.alpinelinux.org/alpine/v3.1/main
http://dl-2.alpinelinux.org/alpine/v3.1/main
http://dl-3.alpinelinux.org/alpine/v3.1/main
http://dl-4.alpinelinux.org/alpine/v3.1/main
http://dl-5.alpinelinux.org/alpine/v3.1/main
http://dl-6.alpinelinux.org/alpine/v3.1/main
http://dl-6.alpinelinux.org/alpine/edge/main
http://dl-6.alpinelinux.org/alpine/edge/testing
- Note that you can also install the provisioning part after you start your machine with “vagrant up”
B.3 - SSH into you virtual machine with one the commands
# No password, private keys are used
vagrant ssh
or :
# The password is vagrant
ssh vagrant@192.168.30.2
End of step 1
Step 2 The bug itself
-
Now you should be in front of a command line on an alpine linux with keystoneclient installed. Check with the following (+ expected result):
Python 2.7.11
Babel (2.2.0) debtcollector (1.2.0) funcsigs (0.4) futures (3.0.4) iso8601 (0.1.11) keystoneauth1 (2.2.0) monotonic (0.5) msgpack-python (0.4.6) MySQL-python (1.2.5) netaddr (0.7.18) netifaces (0.10.4) oslo.config (3.3.0) oslo.i18n (3.2.0) oslo.serialization (2.2.0) oslo.utils (3.4.0) pbr (1.8.1) pip (8.0.0) prettytable (0.7.2) python-heatclient (0.8.1) python-keystoneclient (2.1.1) python-swiftclient (2.7.0) pytz (2015.7) PyYAML (3.11) requests (2.9.1) setuptools (19.4) six (1.10.0) stevedore (1.10.0) wheel (0.26.0) wrapt (1.10.6)
- Now run python and do a simple invocation of the keystoneclient library to get the client module. Here is the command (+expected result)
# /env/bin/python
Python 2.7.11 (default, Dec 17 2015, 21:35:03)
[GCC 5.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from keystoneclient import client
Segmentation fault
# /env/bin/python
Python 2.7.11 (default, Dec 17 2015, 21:35:03)
[GCC 5.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from keystoneclient.v2_0 import client
Segmentation fault
# /env/bin/python
Python 2.7.11 (default, Dec 17 2015, 21:35:03)
[GCC 5.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from keystoneclient.v3 import client
Segmentation fault
End of step 2
Step 3 A (very) limited debugging of the issue
The only debugging I done was using pdb and then gdb.
- To use pdb, write a “test.py” file with only one line:
from keystoneclient import client
and then run:
# /env/bin/python -m pdb test.py
> /home/vagrant/test.py(1)<module>()
-> from keystoneclient import client
(Pdb) continue
Segmentation fault
Pdb also exits so no stack to read to debug or further understand the problem.
- For gdb, here is what I’ve done (+results).
# gdb /env/bin/python
GNU gdb (GDB) 7.10.1
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-alpine-linux-musl".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /env/bin/python...(no debugging symbols found)...done.
(gdb) run
Starting program: /env/bin/python
warning: Cannot call inferior functions, Linux kernel PaX protection forbids return to non-executable pages!
Python 2.7.11 (default, Dec 17 2015, 21:35:03)
[GCC 5.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from keystoneclient import client
Program received signal SIGSEGV, Segmentation fault.
0x0000768a04d1d7b1 in __vdsosym () from /lib/libc.musl-x86_64.so.1
(gdb) backtrace
#0 0x0000768a04d1d7b1 in __vdsosym () from /lib/libc.musl-x86_64.so.1
#1 0x0000768a04d54ff5 in clock_gettime () from /lib/libc.musl-x86_64.so.1
#2 0x00007f0c11b523b8 in ?? ()
#3 0x00007f0c11b52220 in ?? ()
#4 0x00007f0c11b52170 in ?? ()
#5 0x0000768a04f8f37e in ffi_call_unix64 () from /usr/lib/libffi.so.6
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
(gdb) quit
This is the only output that I was capable of extracting from the stack. From this I can see that “/lib/libc.musl-x86_64.so.1” and “/usr/lib/libffi.so.6” were invoked at some point.
End of step 3
I hope this will come of help to anyone trying to debugging the issue,
or propose a fix/patch.
I will try and provide any additional information if needed. Note that
I’ve only seen this bug in python-keystoneclient pip package for the
moment. It is possible for other packages to be affected by the use of
(what seems to be) primitives from musl-x86_64.
I also put a description to this bug here: https://bitbucket.org/pypa/pypi/issues/380/pip-package-python-keystoneclient-causing
Thanks in advance for any suggestion/help.
Sofiane Imadali.
(from redmine: issue id 5038, created on 2016-01-22, closed on 2017-05-19)
- Relations:
- relates #4512 (closed)
- relates #5219 (closed)
- Changesets:
- Revision edb625cc by Natanael Copa on 2016-01-23T12:01:38Z:
main/python: fix find_library
Bad things happens When libc is dlopened with full path so we only
return the filename without path with find_library.
ref #5038
- Revision 16fad622 by Natanael Copa on 2016-01-23T12:33:32Z:
main/python: fix find_library
Bad things happens When libc is dlopened with full path so we only
return the filename without path with find_library.
fixes #5038