Previously 'load-thunk-from-memory' would often throw to 'system-error'
when passed an incorrect ELF file, leading to incorrect error messages.
* libguile/loader.c (load_thunk_from_memory): Reset 'errno' when
'check_elf_header' returns non-NULL.
* test-suite/tests/vm.test: New file.
* test-suite/Makefile.am (SCM_TESTS): Add it.
* libguile/posix.c (scm_crypt): Take 'scm_i_misc_mutex' right before
calling 'crypt'. Move 'SCM_SYSERROR' call after 'scm_dynwind_end'.
* test-suite/tests/posix.test ("crypt"): New test prefix.
* libguile/vm-engine.c (RUN_HOOK0, RUN_HOOK1): Remove.
(RUN_HOOK): Take hook name.
(APPLY_HOOK, RETURN_HOOK, NEXT_HOOK, ABORT_CONTINUATION_HOOK): Use
RUN_HOOK.
* libguile/vm.c (vm_dispatch_hook): Remove value count arg; hooks no
longer receive values (e.g. the return hook now uses
frame-return-values).
(vm_dispatch_abort_hook): Remove value count, which was bogus because
the active frame was the continuation which might contain other
locals, potentially unboxed, not the implicit return-values frame. In
the future we could push on an implicit return-values frame instead.
* module/system/vm/traps.scm (trap-in-procedure, trap-frame-finish):
(trap-in-dynamic-extent, trap-calls-to-procedure): Adapt abort hooks
to not take values. They weren't being used anyway!
* libguile/frames.c (scm_frame_return_values): New function, for use
when a frame is at "return-values".
(scm_init_frames_builtins): Register frame-return-values.
* libguile/vm-engine.c (RETURN_HOOK): Rename from POP_CONTINUATION_HOOK.
(call, call-label): Remove PUSH_CONTINUATION_HOOK; it's unneeded, as
you can always check the FP from an apply hook.
(return-values): Run return hook before popping frame.
* libguile/vm.c (vm_dispatch_return_hook): Rename from
vm_dispatch_pop_continuation_hook. Remove push continuation hook.
(scm_vm_return_hook):
* libguile/vm.h (SCM_VM_PUSH_CONTINUATION_HOOK): Remove.
(SCM_VM_RETURN_HOOK): Rename from SCM_VM_POP_CONTINUATION_HOOK.
* module/system/vm/frame.scm (frame-return-values): Export.
* module/system/vm/trace.scm (print-return, trace-calls-to-procedure)
(trace-calls-in-procedure): Adapt to not receiving values as
arguments.
* module/system/vm/traps.scm (trap-in-procedure, trap-frame-finish):
Adapt to return hook coming from returning frame.
(program-sources-by-line): Update to use match instead of pmatch.
* module/system/vm/traps.scm (trap-in-dynamic-extent)
(trap-calls-to-procedure): Adapt to return hook not receiving values.
* module/system/vm/vm.scm: Remove push continuation hook and rename
return hook.
* libguile/continuations.c (goto_continuation_code):
* libguile/control.c (compose_continuation_code): Add JIT
instrumentation. Unlike foreign or subr routines, we leave these as
closures; it doesn't make sense to JIT for any particular
continuation.
* libguile/foreign.c (get_foreign_stub_code): Use
scm_i_alloc_primitive_code_with_instrumentation. This will allow FFI
procedures to be JITted, and allow us to move the apply hook to
instrument-entry.
* lib/Makefile.am: Explicitly add flexmember. Already included though.
* libguile/gsubr.c (alloc_subr_idx, record_subr_name): Factor out an
"expected subr count" variable.
(alloc_chunk, alloc): Rework so that when the arena grows, old code is
still recognized as primitive.
(scm_i_alloc_primitive_code_with_instrumentation): Fix bug whereby the
JIT function data was initialized at a bogus address, because we were
adding to a uint32_t* instead of a char*. Make internally public.
(is_primitive_code): Recognize all allocated code as primitive.
(alloc_subr_code): Adapt to
scm_i_alloc_primitive_code_with_instrumentation rename.
(primitive_subr_idx): If the call IP isn't a subr-call, return a
sentinel value.
(scm_i_primitive_name, scm_subr_function): Allow for primitives that
aren't subrs.
* libguile/gsubr.h: Decalre
scm_i_alloc_primitive_code_with_instrumentation.
* libguile/gsubr.c: Reimplement to store subr names and procedures in a
side table, and to allocate fresh vcode for each subr. This allows
JIT of subrs, moves to a uniform all-code-starts-with-instrument-entry
regime, and also allows statprof to distinguish between subrs based on
IP.
* libguile/gsubr.h (SCM_SUBRF, SCM_SUBR_NAME): Call out to functions,
now that these are in a side table.
(scm_subr_function, scm_subr_name): New exports.
(scm_i_primitive_name): New internal function, for looking up a
primitive name based on IP.
(scm_apply_subr): Take the subr index.
* libguile/vm-engine.c (subr-call):
* libguile/jit.c (compile_subr_call): Adapt to take index as arg.
* module/statprof.scm (sample-stack-procs, count-call):
(stack-samples->procedure-data): Update to always record IP in stack
samples and call counts.
* module/system/vm/frame.scm (frame-procedure-name): Simplify.
(frame-instruction-pointer-or-primitive-procedure-name): Removed.
* libguile/programs.h:
* libguile/programs.c (scm_primitive_code_name): New function.
* module/system/vm/program.scm (primitive-code-name): New export.
* module/system/vm/assembler.scm (<jit-data>, <meta>): Rework to have
<meta> create the <jit-data> in the end-program, so that jit-data
isn't mutable. Record start and end PC values relative to '.rtl-text
so that we don't need any more linker symbols.
(emit-instrument-entry*, emit-instrument-loop*, begin-program):
(end-program): Adapt.
(begin-kw-arity): Include the initial instrument-entry in the first
arity.
(link-data, link-constants): Write the init routine before interning
constants so that we correctly emit the jit-data for the init
routine.
* libguile/programs.c (try_parse_arity): Skip over a
scm_op_instrument_entry, if any.
* am/bootstrap.am (SOURCES):
* module/Makefile.am (SOURCES): Handle renamve of handle-interrupts.scm
to loop-instrumentation.scm.
* libguile/jit.h (SCM_JIT_COUNTER_ENTRY_INCREMENT): Rename from
SCM_JIT_COUNTER_CALL_INCREMENT.
* libguile/vm-engine.c (instrument-entry): Rename from instrument-call.
* module/language/cps/compile-bytecode.scm (compile-function): Add
handle-interrupts code before calls and returns. Compile the
"instrument-loop" primcall to an "instrument-loop" instruction and a
"handle-interrupts" instruction.
(lower-cps): Adapt to add-loop-instrumentation name change.
* module/language/cps/loop-instrumentation.scm: Rename from
handle-interrupts.scm and just add "instrument-loop" primcalls in
loops. The compiler will add handle-interrupts primcalls as
appropriate.
* module/system/vm/assembler.scm (<jit-data>): New data type, for
emitting embedded JIT data.
(<meta>): Add field for current JIT data.
(make-meta): Initialize current JIT data.
(emit-instrument-entry*, emit-instrument-loop*): New instruction
emitters that reference the current JIT data.
(end-program): Now that all labels are known, arrange to serialize the
JIT data.
(link-data): Reserve space for JIT data, and add relocs to initialize
the "start" / "end" fields.
* libguile/jit.h (struct scm_jit_function_data)
(enum scm_jit_counter_value): New data types.
* libguile/jit.c (scm_jit_compute_mcode, scm_jit_enter_mcode): New
function stubs. Adapt label/offset compilers to take pointers.
* libguile/vm-engine.c (instrument-call, instrument-loop): New
instructions.
* libguile/vm.c: Add jit.h include.
* module/system/vm/assembler.scm (emit-instrument-call)
(emit-instrument-loop): New exports.
This should reduce frame sizes.
* libguile/vm-engine.c (halt): Adapt to multiple-values change. Also
adapt to not having the boot closure on the stack.
(receive, receive-values, subr-call, foreign-call): Adapt to expect
values one slot down.
(prompt): Capture one less word for the values return.
* libguile/vm.c (vm_dispatch_pop_continuation_hook):
(vm_dispatch_abort_hook): Adapt for where to expect values.
(vm_builtin_values_code): Add a call to shuffle-down before
returning. This is more overhead than what existed before, but the
hope is that the savings elsewhere pay off.
(vm_builtin_values_code): Adapt to different values location.
(reinstate_continuation_x, compose_continuation): Adapt to place
resume args at right position.
(capture_delimited_continuation): Remove unused sp and ip arguments.
(abort_to_prompt): Adapt to capture_delimited_continuation change.
(scm_call_n): Adapt to not reserve space for the boot closure.
* module/language/cps/compile-bytecode.scm (compile-function): When
returning values, adapt reset-frame call for return calling convention
change. Adapt truncating or rest returns to expect values in the
right place.
* module/language/cps/slot-allocation.scm (compute-shuffles):
(allocate-lazy-vars, allocate-slots): Allocate values from the "proc
slot", not proc-slot + 1.
* module/system/vm/assembler.scm (emit-init-constants): Reset the frame
before returning so that the return value is in the right place.
* test-suite/tests/rtl.test: Update for return convention change.
* libguile/foreign.c (get_foreign_stub_code): Update for return calling
convention change.
* libguile/frames.h: Add machine return address to diagram.
(SCM_FRAME_MACHINE_RETURN_ADDRESS):
(SCM_FRAME_SET_MACHINE_RETURN_ADDRESS): New macros.
(SCM_FRAME_PREVIOUS_SP):
(SCM_FRAME_DYNAMIC_LINK):
(SCM_FRAME_SET_DYNAMIC_LINK): Adapt for new frame size.
* libguile/vm-engine.c (halt): Set frame size to 3.
(call, call-label): Set mRA to 0.
* libguile/vm.c (push_interrupt_frame, reinstate_continuation_x):
(scm_call_n): Set frame size to 3. In push_interrupt_frame, init the
mRA of the frame.
(vm_builtin_call_with_values_code, vm_handle_interrupt_code): Allocate
larger frames.
* module/language/cps/slot-allocation.scm (allocate-slots): Frame size
is 3.
* module/system/vm/disassembler.scm (define-clobber-parser): Bump frame
size.
* libguile/frames.c (scm_frame_return_address): Use
SCM_FRAME_VIRTUAL_RETURN_ADDRESS.
(scm_c_frame_previous): Likewise.
* libguile/frames.h: Update diagram for new names.
(union scm_vm_stack_element): Rename "as_ip" to "as_vcode", and
add "as_mcode" for machine code pointers.
(SCM_FRAME_VIRTUAL_RETURN_ADDRESS)
(SCM_FRAME_SET_VIRTUAL_RETURN_ADDRESS): Rename to these, from
SCM_FRAME_RETURN_ADDRESS and SCM_FRAME_SET_RETURN_ADDRESS.
* libguile/vm-engine.c (halt, call, call-label, return-values)
(return-from-interrupt): Adapt to renamings. Make "halt" have frame
size as a parameter.
* libguile/vm.c (scm_i_vm_mark_stack): Adapt to renaming.
(push_interrupt_frame): Take mRA as additional argument. In future we
will set it as frame mRA.
(capture_continuation): Adapt to renaming.
(scm_call_n): Adapt to renaming and make frame size adjustable.
(push_interrupt_frame, reinstate_continuation_x): Make frame size
adjustable.
* module/language/cps/slot-allocation.scm (allocate-slots): Make frame
size adjustable.
* libguile/intrinsics.h (scm_t_thread_mra_intrinsic): New type; use for
push_interrupt_frame.
(scm_t_thread_u8_scm_sp_vra_intrinsic): Rename from the same but was
"ra" instead of "vra", and change type to uint32_t*.
* module/system/vm/disassembler.scm (define-clobber-parser):
Parameterize clobber set for calls by frame size.
* libguile/loader.c (scm_i_program_address_range): New function.
Currently unused, will wire up once we correct the compiler to emit
symbols whose values are relative to the base addr. Will be used by
JIT so it needs to be fast.
(register_elf): Rework to be less verbose, and record symtab/strtab.
(struct mapped_elf_image): Add symtab and strtab members.
(load_thunk_from_memory, process_dynamic_segment): Parse out DT_SYMTAB
and DT_STRTAB.
(Elf_Sym): New definition.
* libguile/values.c (scm_c_value_ref): Fix a case in which a request for
the 0th value of a zero-valued object would return the object instead
of erroring.
* libguile/vm-engine.c (halt): Fix construction of a multiple-valued
return (off-by-one error). Fixes a crash introduced in
4a2d78b4d4.
* libguile/instructions.c (DOP1, DOP2, DOP3, DOP4, DOP5): New parts of
the DSL, equivalent to e.g. OP1(x) | OP_DST. Will allow other
definitions of OP* that use token pasting.
* libguile/vm-engine.c: Adapt to use new DOP descriptors.
* libguile/Makefile.am (AM_CPPFLAGS):
(libguile_@GUILE_EFFECTIVE_VERSION@_la_SOURCES): If ENABLE_JIT, build
lightning.
(EXTRA_DIST): Add lightning files.
* libguile/lightning/lightning.am (lightning_extra_files): Add COPYING
and related files to the dist.
* libguile/jit.c:
* libguile/jit.h: New files.
* libguile/lightning/configure.ac: Start trimming down for inclusion in
configure.ac. Remove check for getopt things; we don't include bits
of lightning that use getopt. Remove check for stdint, as we will have
it.
* libguile/lightning/include/lightning.h: Unconditionally include
stdint.h.
* libguile/lightning/Makefile.am:
* libguile/lightning/lib/Makefile.am: Remove.
* libguile/lightning/lightning.am: Add all the C and H files here.
* libguile/lightning/configure.ac: Remove AC_OUTPUT clause.
* libguile/lightning/lightning.am: New file. libguile will include
this. Contents initially from include/Makefile.am and
include/lightning/Makefile.am.
* libguile/lightning/include/Makefile.am:
* libguile/lightning/include/lightning/Makefile.am: Remove.
* libguile/lightning/configure.ac: Remove output of include Makefiles.
* libguile/lightning/configure.ac:
* libguile/lightning/Makefile.am:
* libguile/lightning/check: Remove unit tests. If there is a problem
with lightning on this target, users should check if the upstream tests
pass. Otherwise we will find errors via the Guile test suite.
* libguile/lightning/: New directory, made by the following commands:
git remote add lightning https://git.savannah.gnu.org/git/lightning.git
git merge -s ours --no-commit --allow-unrelated-histories lightning/master
git read-tree --prefix=libguile/lightning/ -u lightning/master
In theory we will be able to update via:
git merge -s subtree lightning/master
This reverts commit 593e2db1dd. The goto
strategy works for hooks that run just before a "next"; it doesn't work
for ones that run at the beginning of opcodes.
