* libguile/jit.c (jit_log_level): New local.
(_LOG, INFO, DEBUG, LOG): New macros.
(compile1):
(compute_mcode):
(scm_jit_enter_mcode): Use new logging macros.
* libguile/jit.c (jit_stop_after, jit_pause_when_stopping): New locals.
(scm_jit_compute_mcode): Add ability to stop after N compilations.
(scm_jit_enter_mcode): Comment out printfs for the time being.
(scm_init_jit): Init locals from environment variables.
* libguile/jit.c (compile1): Add debug for when instructions are first
compiled. Will be removed when all is working.
(compute_mcode): Add debugging about what code is compiled.
(scm_sys_jit_compile): Remove per-instruction output.
(scm_jit_compute_mcode): Actually compile JIT code. Use
GUILE_JIT_COUNTER_THRESHOLD to control when JIT happens.
* libguile/jit.c (scm_jit_counter_threshold): Make a static variable
instead of a compile-time constant.
(scm_init_jit): Init scm_jit_counter_threshold from
GUILE_JIT_COUNTER_THRESHOLD environment variable. Default is -1
indicating "never JIT".
* libguile/vm-engine.c (instrument-entry, instrument-loop): Adapt to new
variable.
* libguile/control.c (compose_continuation_code): Fix offset of code
end.
* libguile/jit.c (compile_compose_continuation): Fix test for mcode not
null.
* libguile/jit.c (compile_prompt): Actually push the MRA arg.
(analyze): Mark call continuations as entries, as both FP and SP are
set then, and also mark prompt handlers as entries (and blocks).
* libguile/jit.c (struct scm_jit_state): Add beginnings of a little
local register allocator.
(reset_register_state): New helper.
(clear_scratch_register_state): Use new helper.
(record_gpr_clobber, record_fpr_clobber): New helpers, used when there
may be cached variables in registers, called when registers are
written.
(set_sp_cache_gpr, set_sp_cache_fpr): New helpers, called when results
are written to the stack.
(emit_retval, emit_movi, emit_ldxi, DEFINE_CLOBBER_RECORDING_EMITTER_R)
(DEFINE_CLOBBER_RECORDING_EMITTER_P, DEFINE_CLOBBER_RECORDING_EMITTER_R_I)
(DEFINE_CLOBBER_RECORDING_EMITTER_R_R): New wrappers for Lightning API
that also records register clobbers. Update callers.
(save_reloadable_register_state): New helper.
(restore_reloadable_register_state): Rename from
ensure_register_state.
* libguile/jit.c (scm_jit_state): Add op_attrs array, for a pre-pass,
and store state of what's in registers.
(SP, FP): Reassign to scratch registers, as in general these need to
be reloaded anyway after callouts.
(die, DIE, ASSERT, UNREACHABLE): Add better invariant-testing.
(clear_register_state, clear_scratch_register_state)
(set_register_state, has_register_state, ASSERT_HAS_REGISTER_STATE):
Add machinery to track state of SP and FP. Can eventually track
scratch register assignments as well. Adapt code to use these.
(compile_atomic_ref_scm_immediate): Compile to a vanilla load on x86.
(compile_handle_interrupts): Analogous atomic-ref changes here.
(analyze): New helper, a simple once-through pre-pass to identify
branch targets.
(compile): Only generate labels for branch targets. Reset register
state at branch targets.
(compute_mcode): Initialize j->op_attrs appropriately.
* libguile/jit.c (emit_push_frame): Simplification; we never need to
store old_fp and new_fp at once.
(compile_alloc_frame): Fix to not keep a pointer into the stack across
a stack expansion.
* libguile/jit.c (struct scm_jit_state): Store next ip, so that
compilers can fuse opcodes.
(op_lengths): New static variable.
(emit_direct_tail_call): Add a fast case for self-recursion.
(compile1): Move IP advancement out of the specific arity compilers;
instead precompute a "next_ip", that can be incremented.
* libguile/jit.c (add_inter_instruction_patch, compile, compute_mcode):
Add support for labels.
(compile_uadd_immediate, compile_usub_immediate): Fix cases where we
were adding the wrong operand as an immediate.
* libguile/jit.c (emit_handle_interrupts_trampoline)
(initialize_handle_interrupts_trampoline)
(compile_handle_interrupts): Handle most of the slow case of
handle-interrupts out-of-line, to avoid code bloat.
* libguile/jit.c (emit_entry_trampoline): Don't bother hackily trying to
save registers; the "jit_frame" call handles that.
(compile_return_values, compile_return_from_interrupt): Fix bug when
computing previous FP: no need to add frame_overhead_slots.
(emit_load_prev_fp_offset, emit_store_prev_fp_offset): Rename from
emit_load_prev_frame_size, emit_store_prev_frame_size.
(emit_push_frame): Adapt to emit_store_prev_frame_size. Don't
subtract off the frame_overhead_slots.
(scm_jit_enter_mcode): Comment out a printf for the time being.
* libguile/jit.c (emit_alloc_frame_for_sp, compile_current_thread): Fix
some ldxr/ldxi stxr/stxi confusions.
(compile_alloc_frame): Omit if the frame size is already correct.
(compile, compute_mcode, scm_sys_jit_compile, scm_jit_enter_mcode):
Add a bit more debugging.
* libguile/init.c (scm_i_init_guile): Call scm_init_jit ().
* libguile/jit.c (enter_mcode, exit_mcode): New static members; code
pointers for the JIT trampoline.
(emit_exit): New helper. The Lightning tramp/frame mechanism that we
use needs to exit via a jmp instead of a return. Adapt callers of
jit_ret.
(emit_entry_trampoline): Use the "frame" mechanism to enter the JIT.
(compile1): Add missing "break" after case statements. Oops!
(compile): Add prolog and "tramp" to compiled functions.
(initialize_jit): New local routine to init the JIT on demand.
(compute_mcode): New helper, to compile a function.
(scm_sys_jit_compile): New function, exported to Scheme as
%jit-compile.
(scm_jit_compute_mcode): Return the existing mcode if the function is
at the start.
(scm_jit_enter_mcode): Call the enter_mcode trampoline.
* libguile/jit.h (struct scm_jit_state): Declare, so we can make
pointers to it.
* libguile/threads.h (struct scm_thread): Add jit_state member.
* libguile/threads.c (on_thread_exit): Free the jit state.
* libguile/intrinsics.h:
* libguile/intrinsics.c (string_set_x): Change to take size_t and u32 as
args.
(allocate_words): Change to take size_t as arg.
* libguile/vm.c (expand_apply_argument): Rename from rest_arg_length,
and also handle the stack manipulation.
* libguile/vm-engine.c (expand-apply-argument): Update for intrinsic
change.
(call-scm-sz-u32): Rename from call-scm-u64-u64, as it matches its
uses and will compile better on 32-bit systems.
* module/system/vm/assembler.scm (define-scm-sz-u32-intrinsic):
(string-set!): Update for new instrinsic call inst.
* libguile/jit.c (compile_call_scm_sz_u32): Adapt.
* libguile/intrinsics.h (SCM_FOR_ALL_VM_INTRINSICS): Update prototype of
capture-continuation.
* libguile/jit.h:
* libguile/jit.c (scm_jit_enter_mcode): Return void, not the vra.
Instead, we expect the code to set vp->ip for the vra.
* libguile/vm-engine.c (instrument-entry, instrument-loop)
(return-values, abort): Adapt scm_jit_enter_mcode calling convention.
(capture-continuation): No need to pass an mra; the intrinsic will
read it from the stack.
* libguile/vm.c (capture_continuation): Remove mra arg, as we take mra
from the continuation.
(scm_call_n): Adapt to scm_jit_enter_mcode change.
* libguile/continuations.c (scm_i_continuation_to_frame): Adapt to vra
field renaming.
(scm_i_reinstate_continuation, grow_stack, copy_stack_and_call)
(scm_dynthrow): Take mra of continuation. Set on the vp before the
longjmp.
* libguile/continuations.h: Update scm_i_reinstate_continuation
prototype.
* libguile/dynstack.h:
* libguile/control.c (scm_suspendable_continuation_p):
* libguile/dynstack.c (PROMPT_WORDS, PROMPT_VRA, PROMPT_MRA):
(PROMPT_JMPBUF, scm_dynstack_push_prompt, scm_dynstack_find_prompt)
(scm_dynstack_wind_prompt): Store both virtual and machine return
addresses on the dynstack, for prompts.
* libguile/eval.c (eval): Pass NULL for mra.
* libguile/intrinsics.c (push_prompt): Add mra arg, and pass it to the
dynstack.
* libguile/intrinsics.h: Update prototypes so that continuation-related
intrinsics can save and restore the MRA.
* libguile/jit.h:
* libguile/jit.c: Return VRA when JIT code needs to tier down.
* libguile/stacks.c (find_prompt, scm_make_stack)
* libguile/throw.c (catch): Adapt find-prompt calls.
* libguile/vm-engine.c (instrument-entry, instrument-loop): Add logic to
continue with vcode after the mcode finishes.
(compose-continuation, capture-continuation, abort, prompt): Add logic
to pass NULL as captured MRA, but continue with mcode from new
continuations, if appropriate.
* libguile/vm.c (scm_i_vm_cont_to_frame, capture_stack)
(scm_i_capture_current_stack, reinstate_continuation_x)
(capture_continuation, compose_continuation_inner, compose_continuation)
(capture_delimited_continuation, abort_to_prompt): Adapt to plumb
around machine code continuations.
(scm_call_n): Check "mra_after_abort" field for machine code
continuation, if any.
* libguile/vm.h (struct scm_vm): Add "mra_after_abort" field.
(struct scm_vm_cont): Rename "ra" field to "vra" and add "mra" field.
* libguile/jit.c (compile_capture_continuation): Rename from call_cc now
that the call is elsewhere.
* libguile/vm-engine.c (call, tail-call): Remove needless SYNC_IP before
get-callee-vcode; the intrinsic can sync the ip if needed from the
frame.
(capture-continuation): Rename from call/cc, and leave the call itself
to tail-call.
* libguile/vm.c (vm_builtin_call_with_current_continuation_code): Update
to put the continuation in a local and then tail call.
(get_callee_vcode): Sync vp->ip if we error.
* libguile/gsubr.c: Reimplement to store subr names and procedures in a
side table, and to allocate fresh vcode for each subr. This allows
JIT of subrs, moves to a uniform all-code-starts-with-instrument-entry
regime, and also allows statprof to distinguish between subrs based on
IP.
* libguile/gsubr.h (SCM_SUBRF, SCM_SUBR_NAME): Call out to functions,
now that these are in a side table.
(scm_subr_function, scm_subr_name): New exports.
(scm_i_primitive_name): New internal function, for looking up a
primitive name based on IP.
(scm_apply_subr): Take the subr index.
* libguile/vm-engine.c (subr-call):
* libguile/jit.c (compile_subr_call): Adapt to take index as arg.
* module/statprof.scm (sample-stack-procs, count-call):
(stack-samples->procedure-data): Update to always record IP in stack
samples and call counts.
* module/system/vm/frame.scm (frame-procedure-name): Simplify.
(frame-instruction-pointer-or-primitive-procedure-name): Removed.
* libguile/programs.h:
* libguile/programs.c (scm_primitive_code_name): New function.
* module/system/vm/program.scm (primitive-code-name): New export.
* am/bootstrap.am (SOURCES):
* module/Makefile.am (SOURCES): Handle renamve of handle-interrupts.scm
to loop-instrumentation.scm.
* libguile/jit.h (SCM_JIT_COUNTER_ENTRY_INCREMENT): Rename from
SCM_JIT_COUNTER_CALL_INCREMENT.
* libguile/vm-engine.c (instrument-entry): Rename from instrument-call.
* module/language/cps/compile-bytecode.scm (compile-function): Add
handle-interrupts code before calls and returns. Compile the
"instrument-loop" primcall to an "instrument-loop" instruction and a
"handle-interrupts" instruction.
(lower-cps): Adapt to add-loop-instrumentation name change.
* module/language/cps/loop-instrumentation.scm: Rename from
handle-interrupts.scm and just add "instrument-loop" primcalls in
loops. The compiler will add handle-interrupts primcalls as
appropriate.
* module/system/vm/assembler.scm (<jit-data>): New data type, for
emitting embedded JIT data.
(<meta>): Add field for current JIT data.
(make-meta): Initialize current JIT data.
(emit-instrument-entry*, emit-instrument-loop*): New instruction
emitters that reference the current JIT data.
(end-program): Now that all labels are known, arrange to serialize the
JIT data.
(link-data): Reserve space for JIT data, and add relocs to initialize
the "start" / "end" fields.
* libguile/jit.h (struct scm_jit_function_data)
(enum scm_jit_counter_value): New data types.
* libguile/jit.c (scm_jit_compute_mcode, scm_jit_enter_mcode): New
function stubs. Adapt label/offset compilers to take pointers.
* libguile/vm-engine.c (instrument-call, instrument-loop): New
instructions.
* libguile/vm.c: Add jit.h include.
* module/system/vm/assembler.scm (emit-instrument-call)
(emit-instrument-loop): New exports.
