there's no nice syscall to manipulate xattrs with atfd and
or open the symlink without dereferencing it (and having fd
that we can do xattrs with)