Partly fixes <https://issues.guix.gnu.org/77862>.
Fixes a bug whereby, when running guix-daemon unprivileged, /etc/group
would contain the wrong GID for the “nixbld” group. This inconsistency
would lead to failures in the Coreutils test suite, for instance.
* nix/libstore/build.cc (DerivationGoal::startBuilder): Use ‘guestGID’
when writing /etc/group.
* tests/store.scm ("/etc/passwd and /etc/group"): New test.
Reported-by: keinflue <keinflue@posteo.net>
Change-Id: I739bc96c4c935fd9015a45e2bfe5b3e3f90554a9
Fixes <https://issues.guix.gnu.org/77570>.
Commit 40f69b586a made chroot root
directory read-only; as a consequence, build processes attempting to
write to the root directory would now get EROFS instead of EACCES.
It turns out that a number of test suites (Go, Ruby, SCons, Shepherd)
would fail because of this observable difference.
To restore previous behavior in build environments while still
preventing build processes from exposing their root directory to outside
processes, this patch (1) keeps the root writable but #o555 by default,
thereby restoring the EACCES behavior, and (2) ensures that the parent
of the chroot root directory is itself user-accessible only.
* nix/libstore/build.cc (class DerivationGoal)[chrootRootTop]: New
field.
(DerivationGoal::startBuilder): Initialize ‘chrootRootTop’ and make it
‘AutoDelete’. Replace ‘mount’ call that made the root directory
read-only by a mere ‘chmod_’ call.
* tests/store.scm ("build root cannot be made world-readable"): Remove.
("writing to build root leads to EACCES"): New test.
Reported-by: Ada Stevenson <adanskana@gmail.com>
Reported-by: keinflue <keinflue@posteo.net>
Suggested-by: Reepca Russelstein <reepca@russelstein.xyz>
Change-Id: I5912e8b3b293f8242a010cfc79255fc981314445
* build-aux/test-env.in: Pass ‘--disable-chroot’ only when unprivileged
user namespace support is lacking and warn in that case.
* tests/store.scm ("build-things, check mode"): Use ‘gettimeofday’
rather than a shared file as a source of entropy.
("symlink is symlink")
("isolated environment", "inputs are read-only")
("inputs cannot be remounted read-write")
("build root cannot be made world-readable")
("/tmp, store, and /dev/{null,full} are writable")
("network is unreachable"): New tests.
* tests/processes.scm ("client + lock"): Skip when
‘unprivileged-user-namespace-supported?’ returns true.
Change-Id: I3b3c3ebdf6db5fd36ee70251d07b893c17ca1b84
Fixes <https://issues.guix.gnu.org/46212>.
Reported by Christopher Baines <mail@cbaines.net>.
Previously, the nar size returned by 'guix substitute' would be read as
an 'int'; thus, values above 2^31 - 1 would be read and then stored as
negative integers in the database.
Regression introduced in 9dfa20a22a.
* nix/libstore/build.cc (SubstitutionGoal::finished): Use templatized
'string2Int' instead of 'std::atoi' to get an 'unsigned long long',
which is the type of 'hash.second'.
* tests/store.scm ("substitute and large size"): New test.
Fixes <https://issues.guix.gnu.org/51983>.
Reported by Tobias Geerinckx-Rice <me@tobias.gr>.
* nix/libstore/local-store.cc (LocalStore::querySubstitutablePathInfos):
Expect 'unsigned long long' for 'downloadSize' and 'narSize'.
* tests/store.scm ("substitute query and large size"): New test.
Files smaller than 8 KiB typically represent ~70% of the entries in
/gnu/store/.links but only contribute to ~4% of the space savings
afforded by deduplication.
Not considering these files for deduplication speeds up file insertion
in the store and, more importantly, leaves 'removeUnusedLinks' with
fewer entries to traverse, thereby speeding it up proportionally.
Partly fixes <https://issues.guix.gnu.org/24937>.
* config-daemon.ac: Remove symlink hard link check and CAN_LINK_SYMLINK
definition.
* guix/store/deduplication.scm (%deduplication-minimum-size): New
variable.
(deduplicate)[loop]: Do not recurse when FILE's size is below
%DEDUPLICATION-MINIMUM-SIZE.
(dump-port): New procedure.
(dump-file/deduplicate)[hash]: Turn into...
[dump-and-compute-hash]: ... this thunk.
Call 'deduplicate' only when SIZE is greater than
%DEDUPLICATION-MINIMUM-SIZE; otherwise call 'dump-port'.
* nix/libstore/gc.cc (LocalStore::removeUnusedLinks): Drop files where
st.st_size < deduplicationMinSize.
* nix/libstore/local-store.hh (deduplicationMinSize): New declaration.
* nix/libstore/optimise-store.cc (deduplicationMinSize): New variable.
(LocalStore::optimisePath_): Return when PATH is a symlink or smaller
than 'deduplicationMinSize'.
* tests/derivations.scm ("identical files are deduplicated"): Produce
files bigger than %DEDUPLICATION-MINIMUM-SIZE.
* tests/nar.scm ("restore-file-set with directories (signed, valid)"):
Likewise.
* tests/store-deduplication.scm ("deduplicate, below %deduplication-minimum-size"):
New test.
("deduplicate", "deduplicate, ENOSPC"): Produce files bigger than
%DEDUPLICATION-MINIMUM-SIZE.
* tests/store.scm ("substitute, deduplication"): Likewise.
This is a followup to b19250eec6,
providing a proper fix for <https://issues.guix.gnu.org/46756>.
* guix/remote.scm (remote-eval): Revert b19250eec6.
* guix/store.scm (build-accumulator): Turn into a procedure. Call
CONTINUE when the store is not eq? to the initial store.
(map/accumulate-builds): Adjust accordingly.
* tests/store.scm ("map/accumulate-builds and different store"): New test.
Fixes <https://issues.guix.gnu.org/50264>.
Reported by Lars-Dominik Braun <lars@6xq.net>.
This fixes a regression introduced in
fa81971cba whereby 'map/accumulate-builds'
would return REST (the tail of LST) without applying PROC on it. The
effect would be that 'lower-inputs' in (guix gexp) would dismiss those
elements, leading to derivations with correct builders but only a subset
of the inputs they should have had.
* guix/store.scm (map/accumulate-builds): Add #:cutoff parameter and
remove 'accumulation-cutoff' variable. Call PROC on the elements of
REST.
* tests/store.scm ("map/accumulate-builds cutoff"): New test.
This typo had always been there since the test was introduced in commit
ce72c78074. Presumably, it became visible
with 9e5812ac59, where caching was no
longer global and thus the 'store' argument of 'package-derivation' was
actually being used.
* tests/store.scm ("current-build-output-port, UTF-8"): Refer to
'%store' rather than 's' in 'package-derivation' call.
This procedure lost its only user in commit
710854304b.
* guix/store.scm (references/substitutes): Remove.
* tests/store.scm ("references/substitutes missing reference info")
("references/substitutes with substitute info"): Remove.
Otherwise, the test crashes (not fails) when run in `guix environment --pure guix`.
Fixes <https://bugs.gnu.org/46445>.
* tests/store.scm (%shell): Fallback to "/bin/sh".
* guix/serialization.scm (read-string-pairs): New procedure.
* guix/store.scm (read-arg): Add support for 'string-pairs'.
(find-roots): New procedure.
* tests/store.scm ("add-indirect-root and find-roots"): New test.
* tests/store.scm ("substitute")
("substitute + build-things with output path")
("substitute + build-things with specific output"): Call 'canonical-file?'.
* tests/substitute.scm ("substitute, authorized key"): Check the mtime
and permissions of "substitute-retrieved".
This makes the test slightly less expensive.
* tests/store.scm ("add-text-to-store vs. delete-paths")
("add-to-store vs. delete-paths"): Delete and merge into...
("add-text-to-store/add-to-store vs. delete-paths"): ... this test.
Previously, we'd spawn 'guix authenticate' once for each item that has
to be signed (when exporting) or authenticated (when importing). Now,
we spawn it once for all and then follow a request/reply protocol. This
reduces the wall-clock time of:
guix archive --export -r $(guix build coreutils -d)
from 30s to 2s.
* guix/scripts/authenticate.scm (sign-with-key): Return the signature
instead of displaying it. Raise a &formatted-message instead of calling
'leave'.
(validate-signature): Likewise.
(read-command): New procedure.
(define-enumerate-type, reply-code): New macros.
(guix-authenticate)[send-reply]: New procedure.
Change to read commands from current-input-port.
* nix/libstore/local-store.cc (runAuthenticationProgram): Remove.
(authenticationAgent, readInteger, readAuthenticateReply): New
functions.
(signHash, verifySignature): Rewrite in terms of the agent.
* tests/store.scm ("import not signed"): Remove 'pk' call.
("import signed by unauthorized key"): Check the error message of C.
* tests/guix-authenticate.sh: Rewrite using the new protocol.
fixlet
Fixes <https://bugs.gnu.org/40428>.
Reported by Marius Bakke <mbakke@fastmail.com> and 白い熊.
Regression introduced with the first uses of 'with-build-handler' in
commit 62195b9a8f and subsequent.
* guix/store.scm (call-with-store): Use 'catch #t' instead of
'dynamic-wind'. This ensures STORE remains open when a non-local exit
other than an exception occurs, such as an abort to the build handler
prompt.
* tests/store.scm ("with-build-handler + with-store"): New test.
* guix/store.scm (<unresolved>): New record type.
(build-accumulator, map/accumulate-builds, mapm/accumulate-builds): New
procedures.
* tests/store.scm ("map/accumulate-builds", "mapm/accumulate-builds"):
New tests.
* guix/store.scm (current-build-prompt): New variable.
(call-with-build-handler, invoke-build-handler): New procedures.
(with-build-handler): New macro.
* tests/store.scm ("with-build-handler"): New test.
Fixes <https://bugs.gnu.org/37744>.
Reported at <https://www.openwall.com/lists/oss-security/2019/10/09/4>.
Based on Nix commit 5a303093dcae1e5ce9212616ef18f2ca51020b0d
by Eelco Dolstra <edolstra@gmail.com>.
* nix/libstore/local-store.cc (LocalStore::LocalStore): Set 'perUserDir'
to #o755 instead of #o1777.
(LocalStore::createUser): New function.
* nix/libstore/local-store.hh (LocalStore): Add it.
* nix/libstore/store-api.hh (StoreAPI): Add it.
* nix/nix-daemon/nix-daemon.cc (performOp): In 'wopSetOptions', add
condition to handle "user-name" property and honor it.
(processConnection): Add 'userId' parameter. Call 'store->createUser'
when userId is not -1.
* guix/profiles.scm (ensure-profile-directory): Note that this is now
handled by the daemon.
* guix/store.scm (current-user-name): New procedure.
(set-build-options): Add #:user-name parameter and pass it to the daemon.
* tests/guix-daemon.sh: Test the creation of 'profiles/per-user' when
listening on a TCP socket.
* tests/store.scm ("profiles/per-user exists and is not writable")
("profiles/per-user/$USER exists"): New tests.
This allows callers to request the substitution of a single derivation
output.
* guix/store.scm (build-things): Accept derivation/output pairs among
THINGS.
* guix/derivations.scm (build-derivations): Likewise.
* tests/store.scm ("substitute + build-things with specific output"):
New test.
* tests/derivations.scm ("build-derivations with specific output"):
New test.
* doc/guix.texi (The Store): Adjust accordingly.
Until now, something like "guix build sed -v1 --check" would not get a
'build-succeeded' event, which in turn meant that the spinner would not
be erased upon build completion.
* nix/libstore/build.cc (DerivationGoal::registerOutputs): When
'buildMode' is bmCheck and 'settings.printBuildTrace' emit a
"@ build-succeeded" trace upon success.
* tests/store.scm ("build-succeeded trace in check mode"): New test.
This allows clients to tell whether output comes from the daemon or, if
it comes from a builder, from which builder it comes. The latter is
particularly useful when MAX-BUILD-JOBS > 1.
* nix/libstore/build.cc (DerivationGoal::tryBuildHook)
(DerivationGoal::startBuilder): Print the child's PID in "@ build-started"
traces.
(DerivationGoal::handleChildOutput): Define 'prefix', pass it to
'writeToStderr'.
* nix/libstore/globals.cc (Settings:Settings): Initialize
'multiplexedBuildOutput'.
(Settings::update): Likewise.
* nix/libstore/globals.hh (Settings)[multiplexedBuildOutput]: New field.
Update 'printBuildTrace' documentation.
* nix/libstore/worker-protocol.hh (PROTOCOL_VERSION): Bump to 0.163.
* nix/nix-daemon/nix-daemon.cc (performOp) <wopSetOptions>: Special-case
"multiplexed-build-output" and remove "use-ssh-substituter".
* guix/store.scm (set-build-options): Add #:multiplexed-build-output?
and honor it.
(%protocol-version): Bump to #x163.
* tests/store.scm ("multiplexed-build-output"): New test.
fixlet
* guix/store.scm (%not-slash): New variable.
(add-file-tree-to-store, interned-file-tree): New procedures.
* tests/store.scm ("add-file-tree-to-store"): New test.
This is a follow-up to commit 40e89f5be6.
* tests/store.scm ("dead path can be explicitly collected"): Don't check
that (> freed 0).
* gnu/packages/package-management.scm (guix)[arguments]<#:phases>: Remove
workaround for the same problem.
Fixes a mistake in a9a0227c01.
* tests/store.scm ("current-build-output-port, UTF-8 + garbage"):
Inverse the 'cond-expand' clauses since the 'guile-2.0' feature doesn't
exist.
* guix/serialization.scm (write-bytevector): New procedure.
(write-string): Rewrite in terms of 'write-bytevector'.
* guix/store.scm (write-arg): Add 'bytevector' case.
(add-data-to-store): New procedure, from former 'add-text-to-store'.
(add-text-to-store): Rewrite in terms of 'add-data-to-store'.
* tests/store.scm ("add-data-to-store"): New test.
Fixes <http://bugs.gnu.org/20217>.
* nix/libstore/worker-protocol.hh (PROTOCOL_VERSION): Bump to 0x161.
* nix/nix-daemon/nix-daemon.cc (performOp): "build-max-jobs",
"build-max-silent-time", and "build-cores" are no longer read upfront;
instead, read them from the key/value list at the end.
* nix/nix-daemon/guix-daemon.cc (main): Explicitly set
'settings.maxBuildJobs'.
* guix/store.scm (%protocol-version): Bump to #x161.
(set-build-options): #:max-build-jobs, #:max-silent-time, and
#:build-cores now default to #f. Adjust handshake to new protocol.
* tests/store.scm ("build-cores"): New test.
* tests/guix-daemon.sh: Add test for default "build-cores" value.
* guix/store.scm (open-connection): Guard body against 'nar-error?' and
re-raise as '&nix-connection-error'.
* tests/store.scm ("connection handshake error"): New test.
Reported by Federico Beffa <beffa@ieee.org>
at <https://lists.gnu.org/archive/html/guix-devel/2016-05/msg00928.html>.
* nix/libstore/local-store.cc (LocalStore::querySubstitutablePaths)
(LocalStore::querySubstitutablePathInfos): Return when
'settings.useSubstitutes' is false.
* tests/store.scm ("references/substitutes missing reference info"):
Make sure to return #f on failure.
* tests/store.scm ("substitutable-path-info when substitutes are turned off"):
("substitutable-paths when substitutes are turned off"): New tests.
