* libguile/intrinsics.h: Add "intrinsic" for handle-interrupts code.
Unlike the other intrinsics, this one isn't a function.
* libguile/programs.c (try_parse_arity): Add cases for instructions used
in VM builtins.
(scm_primitive_call_ip): Return #f if call-ip not found.
* libguile/vm-engine.c (handle-interrupts): Get code from intrinsics.
* libguile/vm.c
* libguile/vm.c (instrumented_code, define_vm_builtins): Add
instrumentation to the builtins, so that they can be JIT-compiled.
(INIT_BUILTIN): Remove min-arity setting; the fallback min-arity
interpreter should figure it out.
(scm_bootstrap_vm): Call the new define_vm_builtins function.
* libguile/gsubr.c (primitive_call_ip): Return 0 if call IP not found.
(primitive_subr_idx): Interpret call ip == 0 as not-a-subr.
* module/system/vm/program.scm (program-arguments-alist): Allow a #f
call-ip.
* libguile/intrinsics.h:
* libguile/intrinsics.c (atomic_ref_scm, atomic_set_scm):
(atomic_swap_scm, atomic_compare_and_swap_scm): New intrinsics, given
that lightning doesn't know atomics.
(scm_bootstrap_intrinsics): Init new intrinsics.
* libguile/vm-engine.c (atomic-scm-ref/immediate)
(atomic-scm-set!/immediate, atomic-scm-swap!/immediate)
(atomic-scm-compare-and-swap!/immediate): Use intrinsics, to be like
the JIT.
* libguile/intrinsics.h (INDIRECT_INT64_INTRINSICS): New definition. If
true, int64 args and return values are passed by reference. Here to
make JIT easier.
* libguile/intrinsics.c (indirect_scm_to_int64, indirect_scm_to_uint64):
(indirect_scm_to_uint64_truncate, indirect_scm_from_int64):
(indirect_scm_from_uint64, indirect_lsh, indirect_rsh): New indirect
variants.
(scm_bootstrap_intrinsics): Use indirect variants as appropriate.
* libguile/vm-engine.c: Update to call indirect intrinsics if
appropriate.
* libguile/intrinsics.h:
* libguile/intrinsics.c (string_set_x): Change to take size_t and u32 as
args.
(allocate_words): Change to take size_t as arg.
* libguile/vm.c (expand_apply_argument): Rename from rest_arg_length,
and also handle the stack manipulation.
* libguile/vm-engine.c (expand-apply-argument): Update for intrinsic
change.
(call-scm-sz-u32): Rename from call-scm-u64-u64, as it matches its
uses and will compile better on 32-bit systems.
* module/system/vm/assembler.scm (define-scm-sz-u32-intrinsic):
(string-set!): Update for new instrinsic call inst.
* libguile/jit.c (compile_call_scm_sz_u32): Adapt.
* libguile/intrinsics.h (SCM_FOR_ALL_VM_INTRINSICS):
* libguile/intrinsics.c (error_wrong_num_args): Take the thread as an
arg, instead of the ostensible callee.
* libguile/vm-engine.c: Update callers of wrong-num-args intrinsic to
pass a thread instead.
* libguile/intrinsics.h (SCM_FOR_ALL_VM_INTRINSICS): Update prototype of
capture-continuation.
* libguile/jit.h:
* libguile/jit.c (scm_jit_enter_mcode): Return void, not the vra.
Instead, we expect the code to set vp->ip for the vra.
* libguile/vm-engine.c (instrument-entry, instrument-loop)
(return-values, abort): Adapt scm_jit_enter_mcode calling convention.
(capture-continuation): No need to pass an mra; the intrinsic will
read it from the stack.
* libguile/vm.c (capture_continuation): Remove mra arg, as we take mra
from the continuation.
(scm_call_n): Adapt to scm_jit_enter_mcode change.
* libguile/continuations.c (scm_i_continuation_to_frame): Adapt to vra
field renaming.
(scm_i_reinstate_continuation, grow_stack, copy_stack_and_call)
(scm_dynthrow): Take mra of continuation. Set on the vp before the
longjmp.
* libguile/continuations.h: Update scm_i_reinstate_continuation
prototype.
* libguile/dynstack.h:
* libguile/control.c (scm_suspendable_continuation_p):
* libguile/dynstack.c (PROMPT_WORDS, PROMPT_VRA, PROMPT_MRA):
(PROMPT_JMPBUF, scm_dynstack_push_prompt, scm_dynstack_find_prompt)
(scm_dynstack_wind_prompt): Store both virtual and machine return
addresses on the dynstack, for prompts.
* libguile/eval.c (eval): Pass NULL for mra.
* libguile/intrinsics.c (push_prompt): Add mra arg, and pass it to the
dynstack.
* libguile/intrinsics.h: Update prototypes so that continuation-related
intrinsics can save and restore the MRA.
* libguile/jit.h:
* libguile/jit.c: Return VRA when JIT code needs to tier down.
* libguile/stacks.c (find_prompt, scm_make_stack)
* libguile/throw.c (catch): Adapt find-prompt calls.
* libguile/vm-engine.c (instrument-entry, instrument-loop): Add logic to
continue with vcode after the mcode finishes.
(compose-continuation, capture-continuation, abort, prompt): Add logic
to pass NULL as captured MRA, but continue with mcode from new
continuations, if appropriate.
* libguile/vm.c (scm_i_vm_cont_to_frame, capture_stack)
(scm_i_capture_current_stack, reinstate_continuation_x)
(capture_continuation, compose_continuation_inner, compose_continuation)
(capture_delimited_continuation, abort_to_prompt): Adapt to plumb
around machine code continuations.
(scm_call_n): Check "mra_after_abort" field for machine code
continuation, if any.
* libguile/vm.h (struct scm_vm): Add "mra_after_abort" field.
(struct scm_vm_cont): Rename "ra" field to "vra" and add "mra" field.
* libguile/foreign.c (scm_i_foreign_call): Rename back from
foreign_call. Need a new trampoline that's easier to call from JIT,
until we actually rewrite the FFI in terms of the JIT.
(scm_register_foreign): Remove foreign_call intrinsic init.
* libguile/intrinsics.h (SCM_FOR_ALL_VM_INTRINSICS): Foreign-call
intrinsic sets return directly on stack.
* libguile/vm-engine.c (foreign-call): Adapt to new intrinsic behavior.
* libguile/vm.c (foreign_call, scm_bootstrap_vm): Add new intrinsic
wrapper.
* libguile/intrinsics.h (SCM_FOR_ALL_VM_INTRINSICS): Make intrinsics for
the VM hooks.
* libguile/vm-engine.c (RUN_HOOK): Run hooks through the intrinsics
table.
* libguile/vm.c (invoke_apply_hook, invoke_return_hook):
(invoke_next_hook, invoke_abort_hook): Rename, remove NOINLINE
attribute (as we call them through the intrinsics table).
(scm_bootstrap_vm): Init new intrinsics.
* libguile/intrinsics.h (SCM_FOR_ALL_VM_INTRINSICS): Add
unpack-values-object.
* libguile/vm-engine.c (subr-call): If the object is a values object,
call out to unpack-values-object. This is to avoid reifying
allocate-frame code in each jitted subr.
* libguile/vm.c (unpack_values_object, scm_bootstrap_vm): Define new
intrinsic.
* libguile/jit.c (compile_capture_continuation): Rename from call_cc now
that the call is elsewhere.
* libguile/vm-engine.c (call, tail-call): Remove needless SYNC_IP before
get-callee-vcode; the intrinsic can sync the ip if needed from the
frame.
(capture-continuation): Rename from call/cc, and leave the call itself
to tail-call.
* libguile/vm.c (vm_builtin_call_with_current_continuation_code): Update
to put the continuation in a local and then tail call.
(get_callee_vcode): Sync vp->ip if we error.
* libguile/intrinsics.h:
* libguile/vm.c (get_callee_vcode): Rename from apply_non_program, and
instead return the IP for the callee of a frame.
(scm_call_n, scm_bootstrap_vm): Adapt to get_callee_vcode change.
* libguile/vm-engine.c (call, tail-call, call/cc): Use
get_callee_vcode unconditionally. JIT will do this to avoid so much
code generation for calls.
Reported by Ricardo Wurmus <rekado@elephly.net>.
Fixes <https://bugs.gnu.org/32161>.
* libguile/r6rs-ports.c (custom_binary_port_seek): Use 'scm_to_off_t'
instead of 'scm_to_int'.
* test-suite/tests/r6rs-ports.test ("8.2.7 Input Ports")["custom binary
input port position, long offset"]: New test.
Fixes <https://bugs.gnu.org/28211>.
* libguile/vm-engine.c (call, call_label, handle_interrupts): Add
'new_fp' variable; set the dynamic link and return address of the frame
at NEW_FP before setting 'vp->fp'. This fixes a bug whereby, in a
multi-threaded context, the stack-marking code could run after vp->fp
has been set but before its dynamic link has been set, leading the
stack-walking code in 'scm_i_vm_mark_stack' to exit early on.
Previously 'vm-frame-older' would fail to traverse the chain of frames.
* libguile/libguile-2.2-gdb.scm (uint-type): New variable
(vm-frame): Fix "saved ip" and "saved fp" computation. The latter had
been broken roughly since commit
72353de77d.
(vm-frame-older): Return #f when IP is zero, not when FP is zero.
(vm-frame-function-name): Wrap 'vm-frame-program-debug-info' in
'false-if-exception'
Fixes <https://bugs.gnu.org/31879>.
* libguile/modules.c (import_obarray_mutex): New variable.
(resolve_duplicate_binding, module_imported_variable): Acquire it before
accessing an obarray.
* libguile/r6rs-ports.c (scm_get_bytevector_n, scm_get_bytevector_n_x):
Turn 'c_count' and related variables into a 'size_t', and use
'scm_to_size_t' instead of 'scm_to_uint'.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
Previously 'load-thunk-from-memory' would often throw to 'system-error'
based on a stale value in 'errno', leading to incorrect error messages.
* libguile/loader.c (load_thunk_from_memory): Set 'errno' to 0 before
jumping to cleanup in the ABORT preprocessor macro, and also in the case
when 'process_dynamic_segment' reports an error.
Previously 'load-thunk-from-memory' would often throw to 'system-error'
when passed an incorrect ELF file, leading to incorrect error messages.
* libguile/loader.c (load_thunk_from_memory): Reset 'errno' when
'check_elf_header' returns non-NULL.
* test-suite/tests/vm.test: New file.
* test-suite/Makefile.am (SCM_TESTS): Add it.
* libguile/posix.c (scm_crypt): Take 'scm_i_misc_mutex' right before
calling 'crypt'. Move 'SCM_SYSERROR' call after 'scm_dynwind_end'.
* test-suite/tests/posix.test ("crypt"): New test prefix.
* libguile/vm-engine.c (RUN_HOOK0, RUN_HOOK1): Remove.
(RUN_HOOK): Take hook name.
(APPLY_HOOK, RETURN_HOOK, NEXT_HOOK, ABORT_CONTINUATION_HOOK): Use
RUN_HOOK.
* libguile/vm.c (vm_dispatch_hook): Remove value count arg; hooks no
longer receive values (e.g. the return hook now uses
frame-return-values).
(vm_dispatch_abort_hook): Remove value count, which was bogus because
the active frame was the continuation which might contain other
locals, potentially unboxed, not the implicit return-values frame. In
the future we could push on an implicit return-values frame instead.
* module/system/vm/traps.scm (trap-in-procedure, trap-frame-finish):
(trap-in-dynamic-extent, trap-calls-to-procedure): Adapt abort hooks
to not take values. They weren't being used anyway!
* libguile/frames.c (scm_frame_return_values): New function, for use
when a frame is at "return-values".
(scm_init_frames_builtins): Register frame-return-values.
* libguile/vm-engine.c (RETURN_HOOK): Rename from POP_CONTINUATION_HOOK.
(call, call-label): Remove PUSH_CONTINUATION_HOOK; it's unneeded, as
you can always check the FP from an apply hook.
(return-values): Run return hook before popping frame.
* libguile/vm.c (vm_dispatch_return_hook): Rename from
vm_dispatch_pop_continuation_hook. Remove push continuation hook.
(scm_vm_return_hook):
* libguile/vm.h (SCM_VM_PUSH_CONTINUATION_HOOK): Remove.
(SCM_VM_RETURN_HOOK): Rename from SCM_VM_POP_CONTINUATION_HOOK.
* module/system/vm/frame.scm (frame-return-values): Export.
* module/system/vm/trace.scm (print-return, trace-calls-to-procedure)
(trace-calls-in-procedure): Adapt to not receiving values as
arguments.
* module/system/vm/traps.scm (trap-in-procedure, trap-frame-finish):
Adapt to return hook coming from returning frame.
(program-sources-by-line): Update to use match instead of pmatch.
* module/system/vm/traps.scm (trap-in-dynamic-extent)
(trap-calls-to-procedure): Adapt to return hook not receiving values.
* module/system/vm/vm.scm: Remove push continuation hook and rename
return hook.
* libguile/continuations.c (goto_continuation_code):
* libguile/control.c (compose_continuation_code): Add JIT
instrumentation. Unlike foreign or subr routines, we leave these as
closures; it doesn't make sense to JIT for any particular
continuation.
* libguile/foreign.c (get_foreign_stub_code): Use
scm_i_alloc_primitive_code_with_instrumentation. This will allow FFI
procedures to be JITted, and allow us to move the apply hook to
instrument-entry.
* lib/Makefile.am: Explicitly add flexmember. Already included though.
* libguile/gsubr.c (alloc_subr_idx, record_subr_name): Factor out an
"expected subr count" variable.
(alloc_chunk, alloc): Rework so that when the arena grows, old code is
still recognized as primitive.
(scm_i_alloc_primitive_code_with_instrumentation): Fix bug whereby the
JIT function data was initialized at a bogus address, because we were
adding to a uint32_t* instead of a char*. Make internally public.
(is_primitive_code): Recognize all allocated code as primitive.
(alloc_subr_code): Adapt to
scm_i_alloc_primitive_code_with_instrumentation rename.
(primitive_subr_idx): If the call IP isn't a subr-call, return a
sentinel value.
(scm_i_primitive_name, scm_subr_function): Allow for primitives that
aren't subrs.
* libguile/gsubr.h: Decalre
scm_i_alloc_primitive_code_with_instrumentation.
* libguile/gsubr.c: Reimplement to store subr names and procedures in a
side table, and to allocate fresh vcode for each subr. This allows
JIT of subrs, moves to a uniform all-code-starts-with-instrument-entry
regime, and also allows statprof to distinguish between subrs based on
IP.
* libguile/gsubr.h (SCM_SUBRF, SCM_SUBR_NAME): Call out to functions,
now that these are in a side table.
(scm_subr_function, scm_subr_name): New exports.
(scm_i_primitive_name): New internal function, for looking up a
primitive name based on IP.
(scm_apply_subr): Take the subr index.
* libguile/vm-engine.c (subr-call):
* libguile/jit.c (compile_subr_call): Adapt to take index as arg.
* module/statprof.scm (sample-stack-procs, count-call):
(stack-samples->procedure-data): Update to always record IP in stack
samples and call counts.
* module/system/vm/frame.scm (frame-procedure-name): Simplify.
(frame-instruction-pointer-or-primitive-procedure-name): Removed.
* libguile/programs.h:
* libguile/programs.c (scm_primitive_code_name): New function.
* module/system/vm/program.scm (primitive-code-name): New export.
* module/system/vm/assembler.scm (<jit-data>, <meta>): Rework to have
<meta> create the <jit-data> in the end-program, so that jit-data
isn't mutable. Record start and end PC values relative to '.rtl-text
so that we don't need any more linker symbols.
(emit-instrument-entry*, emit-instrument-loop*, begin-program):
(end-program): Adapt.
(begin-kw-arity): Include the initial instrument-entry in the first
arity.
(link-data, link-constants): Write the init routine before interning
constants so that we correctly emit the jit-data for the init
routine.
* libguile/programs.c (try_parse_arity): Skip over a
scm_op_instrument_entry, if any.
* am/bootstrap.am (SOURCES):
* module/Makefile.am (SOURCES): Handle renamve of handle-interrupts.scm
to loop-instrumentation.scm.
* libguile/jit.h (SCM_JIT_COUNTER_ENTRY_INCREMENT): Rename from
SCM_JIT_COUNTER_CALL_INCREMENT.
* libguile/vm-engine.c (instrument-entry): Rename from instrument-call.
* module/language/cps/compile-bytecode.scm (compile-function): Add
handle-interrupts code before calls and returns. Compile the
"instrument-loop" primcall to an "instrument-loop" instruction and a
"handle-interrupts" instruction.
(lower-cps): Adapt to add-loop-instrumentation name change.
* module/language/cps/loop-instrumentation.scm: Rename from
handle-interrupts.scm and just add "instrument-loop" primcalls in
loops. The compiler will add handle-interrupts primcalls as
appropriate.
* module/system/vm/assembler.scm (<jit-data>): New data type, for
emitting embedded JIT data.
(<meta>): Add field for current JIT data.
(make-meta): Initialize current JIT data.
(emit-instrument-entry*, emit-instrument-loop*): New instruction
emitters that reference the current JIT data.
(end-program): Now that all labels are known, arrange to serialize the
JIT data.
(link-data): Reserve space for JIT data, and add relocs to initialize
the "start" / "end" fields.
* libguile/jit.h (struct scm_jit_function_data)
(enum scm_jit_counter_value): New data types.
* libguile/jit.c (scm_jit_compute_mcode, scm_jit_enter_mcode): New
function stubs. Adapt label/offset compilers to take pointers.
* libguile/vm-engine.c (instrument-call, instrument-loop): New
instructions.
* libguile/vm.c: Add jit.h include.
* module/system/vm/assembler.scm (emit-instrument-call)
(emit-instrument-loop): New exports.
