fixes #10688

Ariadne Conill authored 4 years ago and Timo Teräs committed 4 years ago

If getservbyname() fails, libfetch will attempt to connect to port 0.

This creates main help like:
--
usage: apk [<OPTIONS>...] COMMAND [<ARGUMENTS>...]

Package installation and removal:
  add        Add packages to WORLD and commit changes
  del        Remove packages from WORLD and commit changes

System maintenance:
  fix        Check WORLD against the system and ensure consistency
  update     Update repository indexes
  upgrade    Install upgrades available from repositories
  cache      Commands related to the management of an offline package cache

Querying package information:
  info       Give detailed information about packages or repositories
  list       List packages matching a pattern or other criteria
  dot        Generate graphviz graphs
  policy     Show repository policy for packages

Repository maintenance:
  index      Create repository index file from packages
  fetch      Download packages from global repositories to a local directory
  manifest   Show checksums of package contents
  verify     Verify package integrity and signature

Miscellaneous:
  audit      Audit directories for changes
  stats      Show statistics about repositories and installations
  version    Compare package versions or perform tests on version strings

This apk has coffee making abilities.
--

And applet specific help like:
--
usage: apk add [<OPTIONS>...] PACKAGES...

Description:
  apk add adds the requested packages to WORLD and installs (or upgrades)
  them if not already present, ensuring all dependencies are met.

Options:
  --initdb         Initialize a new package database
  -l, --latest     Disables normal heuristics for choosing which repository to install a
  -u, --upgrade    When adding packages which are already installed, upgrade them rather
  -t, --virtual NAME
                   Instead of adding the specified packages to WORLD, create a new
  --no-chown       Do not change file owner or group
--

Fix comparing of the hostname portion that matches exactly.
The no_proxy matching is pretty rudimentary though and probably
could go through a bit of additional rework.

Fixes #10681

- split the code to a helper function
- do not set sockets to corked state when putting back to
  cache so socket state is always deterministic
- cork/uncork also when sending CONNECT to a proxy, this
  can reduce a little bit the latency how fast the packet
  gets sent out
- also pair corking with uncorking in http_request to make
  it more obvious pairing

Alex Wauck authored 4 years ago and Timo Teräs committed 4 years ago

The recent TCP_CORK change missed this bit of code.  This change
should improve performance a bit when making HTTP requests by calling
http_cmd only once instead of three times.

Unfortunately libfetch operates on raw sockets and is sending
each HTTP request line using separate syscall which causes the
HTTP request to be sent as multiple packets over the wire in most
configurations. This is not good for performance, but can also
cause subtle breakage if there's DPI firewall that does not get
the Host header.

Incidentally, it seems that on BSDs libfetch already sets
TCP_NOPUSH optimize the packetization. This commit adds same
logic for using TCP_CORK if available. When using TCP_CORK
there is no requirement to set TCP_NODELAY as uncorking will
also cause immediate send. Keep TCP_NODELAY in the fallback
codepaths.

Long term, it might make sense to replace or rewrite libfetch
to use application level buffering.

error: 'strncpy' specified bound 4096 equals destination size [-Werror=stringop-truncation]

Based on patch by Elan Ruusamäe <glen@delfi.ee>

This is unwanted information disclosure.

Reported-by: Max Justicz <max@justi.cz>

TLS_client_method is a LibreSSL extension.
SSLv23_client_method is generic, and doesn't mean SSL v2/v3 only.

ref #8161

fixes #8160

OpenSSL allows passing zero-length to indicate "use strlen".
LibreSSL requires using the real length always, so pass the length.

X509_check_host() is introduced in libressl-2.5.0 and openssl-1.0.2
which are not yet universally available. Add support for building
against the older versions.

loosely based on the freebsd implementation, implement https
connection settings to override CA, and use client certificate.

new files supported in /etc/apk/:
  ca.pem   - if exists, it contains CAs acceptable for https
	     (otherwise system wide CAs are used)
  crl.pem  - if ca.pem is used, this is the (optional) CRL for it
  cert.pem - used as client authentication certificate (+ key)
  cert.key - used as client key (can be also inside cert.pem)

from freebsd

ftp://ftp.fu-berlin.de/unix/NetBSD/packages/current-src/pkgsrc/net/libfetch/files

libfetch comes (at least) in netbsd and freebsd flavors which
differing functionality. Alpine and Arch package netbsd one,
but it's not widely packaged across other distributions.

We need NetBSD version as it does not use funopen(3) which is not
supported in musl, and supports connection pooling.

FreeBSD seems to be the orignal and better maintained version
with support for SSL CAs, client certificate authentication,
proxy authentication, and improved http redirect handling.

So this imports NetBSD version, and future commits will pick up
the needed improvements from FreeBSD tree.

Incidentally, this also fixes #7857 and likes for good.