1
Fork 0
mirror of https://git.savannah.gnu.org/git/guile.git synced 2025-07-08 02:20:26 +02:00
Commit graph

22041 commits

Author SHA1 Message Date
Andy Wingo
0210a8caf0 Refactor out-of-memory detection
Firstly, we add a priority evacuation reserve to prioritize having a few
evacuation blocks on hand.  Otherwise if we give them all to big
allocations first and we have a fragmented heap, we won't be able to
evacuate that fragmented heap to give more blocks to the large
allocations.

Secondly, we remove `enum gc_reason`.  The issue is that with multiple
mutator threads, the precise thread triggering GC does not provide much
information.  Instead we should make choices on how to collect based on
the state of the heap.

Finally, we move detection of out-of-memory inside the collector,
instead of the allocator.

Together, these changes let mt-gcbench (with fragmentation) operate in
smaller heaps.
2022-08-03 10:10:33 +02:00
Andy Wingo
1358d99abc Fix yield calculation after evacuating collections 2022-08-02 22:15:13 +02:00
Andy Wingo
a4e1f55f37 Implement generational collection
Not really battle-tested but it seems to work.  Need to implement
heuristics for when to do generational vs full-heap GC.
2022-08-02 15:37:02 +02:00
Andy Wingo
13b3bb5b24 Update barrier functions to also have the object being written
Also remove read barriers, as they were unused, and we have no plans to
use them.
2022-08-02 15:37:02 +02:00
Andy Wingo
7f405c929e Initial live mask does not include young allocations
After rotation, the young bit wasn't being included anyway.  This just
improves the first collection.
2022-08-02 15:37:02 +02:00
Andy Wingo
1781c5aed4 Fix evacuation allocator to clear any holes 2022-08-02 15:36:59 +02:00
Daniel Llorens
50d4b50203 Check for gperf at configure time
Edited Arne Babenhauserheide's patch
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=51129;att=0;filename=0001-autoconf-Check-for-gperf-if-running-from-git.patch;msg=5.

Fixes https://debbugs.gnu.org/cgi/bugreport.cgi?bug=51129.
2022-08-02 15:08:57 +02:00
Andy Wingo
22a9cc87a0 Update TODO 2022-07-20 14:40:47 +02:00
Andy Wingo
279309b821 mt-gcbench allocates garbage between live data
This obviously invalidates previous benchmark results; perhaps we should
make this optional.
2022-07-20 14:40:47 +02:00
Andy Wingo
d106f3ca71 Mutator collects evacuation target blocks 2022-07-20 14:40:47 +02:00
Andy Wingo
92b05a6310 Add implementation of parallel evacuation 2022-07-20 14:40:47 +02:00
Andy Wingo
4a9908bc4d Refactor evacuation vs pinning support
Marking conservative roots in place effectively prohibits them from
being moved, and we need to trace the roots anyway to discover
conservative roots.  No need therefore for a pin bit.
2022-07-20 14:40:47 +02:00
Andy Wingo
a16bb1833c Add logic to compute evacuation candidate blocks 2022-07-20 14:40:47 +02:00
Andy Wingo
c7c8fa2d32 Refactor to add "block_list" type 2022-07-20 14:40:47 +02:00
Andy Wingo
a8214af467 Whippet reserves a bit in object kind for forwarding
Tags without the bit are forwarding addresses.
2022-07-20 14:40:47 +02:00
Andy Wingo
8409383ee1 Refactor post-collection for mark space 2022-07-20 14:40:47 +02:00
Andy Wingo
69caead182 Add heuristics to choose when to compact or mark in place
We can choose to compact (evacuate) or mark in place.  What we choose
has some effects on how we mark.
2022-07-20 14:40:47 +02:00
Andy Wingo
09d2df1626 Compute GC yield as fraction of total heap size 2022-07-20 14:40:47 +02:00
Andy Wingo
c998f1cd5c Measure fragmentation as fraction of total heap size
This allows a relatively more fragmented mark space if the majority of
the heap is taken up by lospace.
2022-07-20 14:40:47 +02:00
Andy Wingo
7af8bb6bd0 Add machinery to disable ragged-stop marking
We'll need to disable the optimization that mutators mark their own
stacks once we support evacuation.
2022-07-20 14:40:47 +02:00
Andy Wingo
e4342f6c45 Add helper for yielding in a spinlock 2022-07-20 14:40:47 +02:00
Andy Wingo
52166fe286 Add gc_edge data structure
Less casting in user programs, and it's a step on the way to evacuation
in whippet.
2022-07-20 14:40:47 +02:00
Andy Wingo
808d365f4b We identify empty blocks lazily now 2022-07-20 14:40:47 +02:00
Andy Wingo
bc73c5ad02 Whitespace fix 2022-07-20 14:40:47 +02:00
Andy Wingo
157d40466b mark_space_reacquire_memory updates pending_unavailable_bytes 2022-07-20 14:40:47 +02:00
Andy Wingo
33a3af2c73 Large object space properly acquires blocks from mark space
If the mutator finds completely empty blocks, it puts them on the side.
The large object space acquires empty blocks, sweeping if needed, and
causes them to be unmapped, possibly causing GC.
2022-07-20 14:40:47 +02:00
Andy Wingo
71b656bca4 When sweeping, return empty blocks to global freelist
This will facilitate management of overhead for defragmentation as well
as blocks to unmap, for compensating large object allocations.
2022-07-20 14:40:47 +02:00
Andy Wingo
8f06b914b0 Refactor to allow "next" pointer embedded in block summary 2022-07-20 14:40:47 +02:00
Andy Wingo
7d80d45c79 Rename mark-sweep.h to whippet.h 2022-07-20 14:40:44 +02:00
Ludovic Courtès
8c976c6a1f Update NEWS.
* NEWS: Update.
2022-07-04 11:52:14 +02:00
Ludovic Courtès
abd39f3328 Define IPPROTO_IPV6 and IPV6_V6ONLY.
* libguile/socket.c (scm_init_socket): Define IPPROTO_IPV6 and IPV6_V6ONLY.
* doc/ref/posix.texi (Network Sockets and Communication): Document them.
2022-07-04 11:49:31 +02:00
Ludovic Courtès
6da4be170a Define IN6ADDR_ANY and IN6ADDR_LOOPBACK.
* libguile/socket.c (scm_init_socket): Define IN6ADDR_ANY and IN6ADDR_LOOPBACK.
* doc/ref/posix.texi (Network Address Conversion): Document them.
2022-07-04 11:39:14 +02:00
Christopher Baines
baa1424335 web: Don't hide missing data in the chunked input port.
This port is of limited use if it cannot be used reliably. Rather than
behaving as if the input has finished when it ends unexpectedly, instead
raise an exception.

* module/web/http.scm (make-chunked-input-port): Raise an exception on
premature termination.
(&chunked-input-ended-prematurely): New exception type.
(chunked-input-ended-prematurely-error?): New procedure.
* test-suite/tests/web-http.test (pass-if-named-exception): Rename to
pass-if-named-exception.
(pass-if-named-exception): New syntax.
("Exception on premature chunk end"): New test for this behaviour.

Signed-off-by: Ludovic Courtès <ludo@gnu.org>
2022-07-04 11:22:13 +02:00
Christopher Baines
9a3353a993 web: Handle ending CRLF (\r\n) for chunked input and output ports.
The chunked transfer encoding specifies the chunked body ends with
CRLF. This is in addition to the CRLF at the end of the last chunk, so
there should be CRLF twice at the end of the chunked body:

  https://datatracker.ietf.org/doc/html/rfc2616#section-3.6.1

* module/web/http.scm (make-chunked-input-port): Read two extra bytes at
the end of the chunked input.
(make-chunked-output-port): Write the missing \r\n when closing the
port.
* test-suite/tests/web-http.test (chunked encoding): Add missing \r\n to
test data.

Signed-off-by: Ludovic Courtès <ludo@gnu.org>
2022-07-04 11:22:11 +02:00
Aleix Conchillo Flaqué
a84d8f6473 web: send capitalized authorization header scheme
* module/web/http.scm (write-credentials): capitalize authorization
header scheme. The standard allows the scheme to be case-insensitive,
however most libraries out there expect the scheme to be capitalized,
which is what it is actually used in RFC
docs (e.g. https://datatracker.ietf.org/doc/html/rfc7617#section-2). Some
libraries even reject lowercase scheme making Guile incompatible.

Signed-off-by: Ludovic Courtès <ludo@gnu.org>
2022-07-04 11:16:36 +02:00
Ludovic Courtès
7e048c6c51 Update NEWS.
* NEWS: Update.
2022-06-16 10:06:32 +02:00
Jessica Tallon
4456245753 Add missing #:modules argument for coverage-data->lcov.
The code coverage function `coverage-data->lcov` has a documented
`modules` argument, however that was missing from the source. I have
added it so when supplied it only converts the coverage data for the
supplied modules. If not supplied it defaults the old behavour of
including all the modules currently loaded.

* module/system/vm/coverage.scm (coverage-data->lcov): Add #:modules
parameter and honor it.

Signed-off-by: Ludovic Courtès <ludo@gnu.org>
2022-06-16 10:01:06 +02:00
Liliana Marie Prikler
01b686b701 Allow null bytes in UNIX sockets.
The current socket address constructors all assume, that there are no
null bytes in the socket path.  This assumption does not hold in Linux,
which uses an initial null byte to demarcate abstract sockets and
ignores all further null bytes [1].

[1] https://www.man7.org/linux/man-pages/man7/unix.7.html

* libguile/sockets.c (scm_fill_sockaddr)[HAVE_UNIX_DOMAIN_SOCKETS]:
Use scm_to_locale_stringn to construct c_address.
Use memcpy instead of strcpy and calculate size directly instead of
using SUN_LEN.
(_scm_from_sockaddr): Copy the entire path up to the limits imposed by
addr_size.
* test-suite/tests/00-socket.test: ("make-socket-address"): Add case for
abstract unix sockets.
("AF_UNIX/SOCK_STREAM"): Add abstract socket versions of bind, listen,
connect and accept.

Signed-off-by: Ludovic Courtès <ludo@gnu.org>
2022-06-16 09:54:29 +02:00
Taylor R Campbell
e4e8afd6c8 Allow empty vendor string in GNU target triplets.
NetBSD and pkgsrc have been using an empty vendor string since the
mid-'90s, such as x86_64--netbsd.  pkgsrc has been carrying around a
workaround just the guile build for a long time.  (Before that,
NetBSD omitted the vendor altogether, so if x86_64 existed then it
might have been `x86_64-netbsd', but that caused more problems.)
This change makes Guile accept an empty vendor string so workarounds
are no longer necessary.

* module/system/base/target.scm (validate-target): Allow empty vendor
string in GNU target triplets.
* test-suite/tests/cross-compilation.test ("cross-compilation"): Add
tests for "x86_64--netbsd".

Co-authored-by: Ludovic Courtès <ludo@gnu.org>
2022-06-16 09:42:05 +02:00
Mihail Iosilevich
158da6966b Allow (library ...) form in cond-expand inside R7RS define-library.
Guile (3.0.8) reports a compilation error when cond-expand tries to
check existence of a missing library:

    scheme@(guile-user)> (define-library (test)
                           (cond-expand
                            ((library (scheme sort))
                             (import (scheme sort)))))
    While compiling expression:
    no code for module (scheme sort)

It looks like bug #40252 was not fully eliminated.

Also, (library ...) cannot handle module names like (srfi 1), though
(import (srfi 1)) works fine. For example, this code fails:

    scheme@(guile-user)> (define-library (test)
                           (cond-expand
                            ((library (srfi 1))
                             (import (srfi 1)))))
    While compiling expression:
    In procedure symbol->string: Wrong type argument in position 1
    (expecting symbol): 1

There are probably other cases when (library ...) and (import ...) does
not work identically: (library ...) uses resolve-interface while
(import ...) uses resolve-r6rs-interface.

This patch fixes both issues.

* module/ice-9/r7rs-libraries.scm (define-library): Replace
'resolve-interface' call by 'resolve-r6rs-interface', wrapped in
'cond-expand'.

Signed-off-by: Ludovic Courtès <ludo@gnu.org>
2022-06-16 09:36:31 +02:00
Ludovic Courtès
e8a41aee29 'module-define!' honors the 'module' parameter.
* module/ice-9/boot-9.scm (module-define!): In 'module-add!' call, pass
MODULE rather than (current-module).
2022-06-16 09:31:36 +02:00
Ludovic Courtès
64449a835a 'connect' handles EAGAIN like EINPROGRESS.
* libguile/socket.c (scm_connect): Handle EAGAIN the same way as
EINPROGRESS (connect(2) returns EAGAIN for Unix-domain sockets and
socketpairs).
2022-06-16 09:31:36 +02:00
Helmut Eller
f31fb0044d Fix some problems with callr and calli.
The problem with callr is that the register that contains the
function to be called, can be overwritten by the logic that moves
the values into argument registers.  To fix this, I added a
get_callr_temp function that should return a platform specific
register that is not used to pass arguments.  For Aarch64/Arm the
link registers seems to work; for Amd64/i686 the RAX register.
The function/tmp pair becomes an additional argument to the
parallel assigment; this way the original function register is not
accidentally overwritten.

The problem with calli is that it may not have enough temp
registers to move arguments.  The windmill paper says that at most
one temporary register is needed for the parallel assignment.
However, we also need a temp register for mem-to-mem moves.  So it
seems that we need a second temporary.  For Amd64/i686 we have
only one temporary GPR and one temporary FPR.  To fix this, I
modified the algorithm from the paper a bit: we perform the
mem-to-mem moves before the other moves.  Later when we need the
temp to break cycles, there shouldn't be any mem-to-mem moves
left.  So we should never need two temps at the same time.

* lightening/lightening.c: (get_callr_temp): New function; need
for each platform.
(prepare_call_args): Include the function/callr_temp pair in the
arguments for the parallel assignment.

* lightening/x86.c, lightening/arm.c, lightening/aarch64.c
(get_callr_temp): Implementation for each platform.

* lightening/arm.c (next_abi_arg): Fix the stack size for doubles.

* tests/call_10_2.c, tests/callr_10.c: New tests.
* tests/regarrays.inc: New file. Common code between the above two
tests that would be tedious to duplicate.
2022-06-08 16:20:42 +02:00
Andy Wingo
061d92d125 Update README 2022-05-15 22:06:41 +02:00
Andy Wingo
69d7ff83dd More wording 2022-05-11 22:29:37 +02:00
Andy Wingo
c39e26159d Some README updates 2022-05-11 22:25:09 +02:00
Andy Wingo
7ac0b5bb4b More precise heap size control
No longer clamped to 4 MB boundaries.  Not important in production but
very important for comparing against other collectors.
2022-05-11 21:19:26 +02:00
Andy Wingo
fa3b7bd1b3 Add global yield and fragmentation computation 2022-05-09 22:03:50 +02:00
Andy Wingo
3bc81b1654 Collect per-block statistics
This will let us compute fragmentation.
2022-05-09 21:46:27 +02:00
Andy Wingo
7461b2d5c3 Be more permissive with heap multiplier
Also if there's an error, print the right argument
2022-05-06 15:08:24 +02:00