* module/scripts/disassemble.scm (disassemble): Update to work with
RTl (and only RTL, as that's the future).
* module/system/vm/debug.scm (for-each-elf-symbol): New public
interface.
(debug-context-from-image): New helper.
(find-debug-context): Use the helper.
* module/system/vm/disassembler.scm (disassemble-image): New public
interface.
* module/system/vm/assembler.scm (link-data): Give stringbufs the
"shared" flag already, so we don't attempt to set it at runtime. Give
.data sections the SHF_WRITE flag.
* libguile/instructions.c (FOR_EACH_INSTRUCTION_WORD_TYPE): Allow for
five-word instructions, and for new instruction word types.
* libguile/vm-engine.c (RETURN_ONE_VALUE): Instead of returning the
value in the fixed part of the call frame, return it in the same place
multiple-value returns go: from slot 1.
(BR_ARITHMETIC): Allow arithmetic tests to be negated.
(rtl_vm_engine): Change calling convention to use the same location
for single and multiple-value returns. Renumber all instructions.
(halt, halt/values): Fold into a single instruction (halt).
(call): Take the location of the procedure instead of the location of
the call frame. Also take the number of args, and reset the sp before
jumping to the procedure, so as to indicate the number of arguments.
(call/values): Remove, as the new calling convention has RA == MVRA.
(tail-call): Require the procedure to be shuffled down already, and
take "nlocals" as an arg instead of "nargs".
(receive, receive-values): New instructions, for receiving returned
values from calls.
(return-values): Rename from return/values. Remove "values".
(alloc-frame): Rename from reserve-locals.
(reset-frame): New instruction.
(drop-locals): Remove.
(br-if-=, br-if-<, br-if-<=): Allow these instructions to be
negatable.
(br-if->, br-if->=): Remove. Probably a bad idea, given NaN.
(box-ref): Don't bother trying to do a reverse lookup -- the
toplevel-box, module-box, and resolve instructions should handle
that.
(resolve): Add arg to check that the variable is bound.
(toplevel-box, module-box): New instructions, replacing toplevel-ref,
toplevel-set, module-ref, and module-set.
* libguile/vm.c (rtl_boot_continuation_code, rtl_values_code): Adapt to
instruction set changes.
* module/Makefile.am: Make the assembler and disassembler dependent on
vm-operations.h.
* module/system/vm/assembler.scm:
* module/system/vm/disassembler.scm: Adapt to instruction changes and
new instruction word kinds.
* test-suite/tests/rtl.test: Adapt to instruction set changes.
* libguile/vm-engine.c: Change the RTL VM to number the procedure as
local 0, and other locals from 1. In the future we will want the FP
to point to local 0 instead of local 1. In the future also we can
elide the procedure for well-known closures (closures in which all
references are known call sites).
(make_closure, free_set): Instead of taking rest arguments, we add a
new free-set! op that initializes closures.
(free_ref): Take the closure as an argument.
* libguile/vm.c (rtl_boot_continuation_code): Remove comments, which
were out of date.
(rtl_apply_code, rtl_values_code): Update comments.
* module/system/vm/assembler.scm (intern-constant, emit-init-constants):
Adapt to locals numbering change.
(begin-kw-arity): For assert-nargs-ee purposes, nreq includes the
procedure.
* module/system/vm/disassembler.scm (code-annotation): Adapt annotation
for assert-nargs-ee/locals.
* test-suite/tests/rtl.test: Adapt tests.
* libguile/instructions.c (scm_rtl_instruction_list): Add an element to
the list to indicate that an instruction outputs to its first
argument.
* module/system/vm/assembler.scm:
* module/system/vm/disassembler.scm: Adapt.
* libguile/programs.c (parse_arity): Lookup rtl-program-minimum-arity
from (system vm debug).
* module/system/vm/debug.scm (find-first-arity): Fix the linear search.
Whoops!
* module/system/vm/assembler.scm (link-procprops, link-objects): Arrange
to write procedure property links out to a separate section.
* libguile/procprop.c (scm_procedure_properties):
* libguile/programs.h:
* libguile/programs.c (scm_i_rtl_program_properties):
* module/system/vm/debug.scm (find-program-properties): Wire up
procedure-properties for RTL procedures. Yeah! Fistpumps! :)
* module/system/vm/debug.scm (find-program-debug-info): Return #f if the
string is "", as it is if we don't have a name. Perhaps
elf-symbol-name should return #f in that case...
* test-suite/tests/rtl.test: Add some tests.
* libguile/procprop.h:
* libguile/procprop.c (scm_procedure_documentation): Move here from
procs.c, and to make the logic more similar to that of procedure-name,
which allows RTL programs to dispatch to rtl-program-documentation.
* libguile/programs.c (scm_i_rtl_program_documentation):
* libguile/programs.h:
* module/system/vm/program.scm (rtl-program-documentation): New
plumbing.
* module/system/vm/debug.scm (find-program-docstring): New interface to
grovel ELF for a docstring.
* libguile/procprop.c (scm_i_procedure_arity): Allow RTL programs to
dispatch to scm_i_program_arity.
* libguile/programs.c (scm_i_program_print): Refactor reference to
write-program.
(scm_i_rtl_program_minimum_arity): New procedure, dispatches to
Scheme.
(scm_i_program_arity): Dispatch to scm_i_rtl_program_minimum_arity if
appropriate.
* module/system/vm/debug.scm (program-minimum-arity): New export.
* module/system/vm/program.scm (rtl-program-minimum-arity): New internal
function.
(program-arguments-alists): New helper, implemented also for RTL
procedures.
(write-program): Refactor a bit, and call program-arguments-alists.
* test-suite/tests/rtl.test ("simply procedure arity"): Add tests that
arities make it all the way to cold ELF and back to warm Guile.
* module/system/vm/debug.scm (<arity>): New object, for reading
arities. Unlike <arity> in the assembler, this one only holds on to a
couple of pointers, and doesn't even load in argument names. Unlike
the arity lists in (system vm program), it can load in names. Very
early days but it does seem to work.
(find-program-arities, arity-arguments-alist): New higher-level
interfaces.
* module/system/vm/assembler.scm (<meta>, <arity>): Assembler now tracks
arities of a function.
(begin-standard-arity, begin-opt-arity, begin-kw-arity, end-arity):
New macro-assemblers.
* test-suite/tests/rtl.test: Adapt all tests to use begin-standard-arity
and end-arity.
* module/system/vm/assembler.scm (pack-flags): New helper.
(standard-prelude, opt-prelude, kw-prelude): New macro-instructions.
* test-suite/tests/rtl.test: Update tests to use standard-prelude.
* module/system/vm/assembler.scm (assert-match): New helper macro to
check argument types.
(<meta>): Add properties field. Rename name field to "label" to
indicate that it should be unique.
(make-meta, meta-name): New helpers.
(begin-program): Take additional properties argument.
(emit-init-constants): Adapt to begin-program change.
(link-symtab): Allow for anonymous procedures.
* test-suite/tests/rtl.test: Adapt tests.
* libguile/print.c (iprin1): Use scm_i_program_print for RTL programs
too.
* libguile/procprop.c (scm_procedure_name): For RTL programs, call
scm_i_rtl_program_name if there is no override.
* libguile/programs.h:
* libguile/programs.c (scm_i_rtl_program_name): New helper, dispatches
to (system vm program).
(scm_i_program_print): For RTL programs, the fallback prints the code
pointer too.
* module/system/vm/program.scm (rtl-program-name): Use the debug info to
get an RTL program name.
(write-program): Work with RTL programs too.
* test-suite/tests/rtl.test ("procedure name"): Add test.
* module/Makefile.am:
* module/system/vm/debug.scm: New module.
* module/system/vm/elf.scm (elf-section-by-name): New helper.
(elf-symbol-table-len): New helper.
* test-suite/tests/rtl.test: Add test for finding debug info.
* module/Makefile.am:
* module/system/vm/assembler.scm: New module, implementing an assembler
for RTL.
* test-suite/Makefile.am:
* test-suite/tests/rtl.test: New test suite.
* module/system/vm/elf.scm (make-elf-symbol*): Add constructor; export
as make-elf-symbol.
(elf-symbol-len): New export.
(write-elf32-symbol, write-elf64-symbol): New helpers.
(write-elf-symbol): New export.
* module/system/vm/linker.scm (make-string-table): Rework to be a
stateful object instead of a function object. Works better in this
case. Adapt users.
(string-table-intern!): Rename from string-table-intern, and just
return the index of the string.
(link-string-table!): Rename from link-string-table, and set a flag to
prevent interning strings after linking, as that's not going to work
well.
* module/language/objcode/elf.scm (bytecode->elf): Adapt.
* libguile/vm-engine.c (rtl_vm_engine): Add new VM.
(vm_engine): Add support for calling RTL programs.
* libguile/tags.h (scm_tc7_rtl_program): New type for procedures that
run on the new VM.
* libguile/evalext.c (scm_self_evaluating_p):
* libguile/goops.c (scm_class_of):
* libguile/print.c (iprin1):
* libguile/procprop.c (scm_i_procedure_arity):
* libguile/procs.c (scm_procedure_p): Add hooks for the new tc7.
* libguile/programs.h:
* libguile/programs.c (scm_make_rtl_program, scm_i_rtl_program_print)
(scm_rtl_program_p, scm_rtl_program_code):
* module/system/vm/program.scm: Add constructors and accessors for the
new "RTL programs".
* libguile/vm.c (rtl_boot_continuation): Define a boot program.
(rtl_apply, rtl_values): New static RTL programs.
* libguile/frames.c (scm_frame_num_locals): Adapt for frames of RTL
programs.
* libguile/frames.h: Add description of RTL frames.
* libguile/Makefile.am: Add rules to generate vm-operations.h.
* .gitignore: Ignore vm-operations.h.
* module/system/vm/instruction.scm:
* libguile/instructions.c:
* libguile/instructions.h: Use vm-operations.h to define enumerated
values for the new RTL opcodes. Define some helper macros to pack and
unpack 32-bit instruction words.
(rtl-instruction-list): New function, exported by (system vm
instruction).
* libguile/objcodes.c: Wire up the bits needed to detect the new RTL
bytecode and load it, as appropriate.
* doc/ref/api-debug.texi (VM Hooks): Update documentation.
* libguile/vm.c (vm_dispatch_hook):
* libguile/vm-engine.c: Rework the hook machinery so that they can
receive an arbitrary number of arguments. The return and abort
hooks will pass the values that they return to their continuations.
(vm_engine): Adapt to ABORT_CONTINUATION_HOOK change.
* libguile/vm-i-system.c (return, return/values): Adapt to
POP_CONTINUATION_HOOK change.
* module/system/vm/frame.scm (frame-return-values): Remove. The
pop-continuation-hook will pass the values directly.
* module/system/vm/trace.scm (print-return):
(trace-calls-to-procedure):
(trace-calls-in-procedure): Update to receive return values
directly.
* module/system/vm/traps.scm (trap-in-procedure)
(trap-in-dynamic-extent): Ignore return values.
(trap-frame-finish, trap-calls-in-dynamic-extent)
(trap-calls-to-procedure): Pass return values to the handlers.
* libguile/objcodes.c (register_elf, scm_find_mapped_elf_image): New
interfaces that keep a list of all ELF mappings. Exported from the
(system vm objcode) module.
* module/system/vm/objcode.scm: Export find-mapped-elf-image.
* module/system/vm/linker.scm (make-linker-object):
(linker-object-section-symbol):
(linker-object-symbols*): Create a symbol to the start of a linker
object. Hide it from the external linker-object-symbols* accessor.
(segment-kind, count-segments): Sections without SHF_ALLOC don't get
segments.
(collate-objects-into-segments): Allow for #f segment types. If two
sections have the same type and flags, leave them in the same order.
(align): Allow for 0 alignment.
(add-elf-objects): New helper: puts the ELF data structures (header,
segment table, and section table) in sections of their own. This
lends a nice clarity and conceptual unity to the linker.
(relocate-section-header, allocate-segment): Lay out segments with
congruent, contiguous addresses, so that we can just mmap the file and
if debugging sections that are not in segments are present, they can
be lazily paged in if needed by the kernel's VM system.
(link-elf): Refactor to use the new interfaces.
* test-suite/tests/linker.test: Update to expect the additional sections
for the header and section table.
* module/system/vm/elf.scm: Add commentary.
(make-elf): Add a constructor similar to make-elf-segment and
make-elf-section.
(write-elf32-header, write-elf64-header, write-elf-header): Take an
<elf> instead of all the fields separately.
(<elf-segment>, <elf-section>): Add "index" property. Adapt
constructors accordingly.
* module/language/objcode/elf.scm (bytecode->elf): Arrange to set the
section indexes when creating ELF sections.
* module/system/vm/linker.scm (fold-values): New helper.
(alloc-segment, relocate-section-header): Arrange to set segment and
section indexes.
(find-shstrndx): New helper, replaces compute-sections-by-name. Now
that sections know their indexes, this is easier.
(allocate-elf, write-elf): New helpers, factored out of link-elf.
Easier now that sections have indexes.
(link-elf): Simplify. Check that the incoming objects have sensible
numbers.
* test-suite/tests/linker.test: Update to set #:index on the linker
objects.
* module/Makefile.am:
* module/system/vm/linker.scm: New file, split out of (system vm elf).
(make-string-table, string-table-intern): Export under their bare
names, instead of make-elf-string-table and elf-string-table-intern.
* module/system/vm/elf.scm: Remove linking capabilities.
(string-table-ref): Export.
* module/language/objcode/elf.scm (bytecode->elf): Adapt to use (system
vm linker).
* test-suite/tests/linker.test: New test.
* module/system/vm/trace.scm (build-prefix): New helper.
(print-application, print-return): Use the helper.
(trace-calls-to-procedure, trace-calls-in-procedure):
(trace-instructions-in-procedure, call-with-trace): Add #:max-indent
argument, defaulting to the terminal width less 40 characters.
* doc/ref/scheme-using.texi: Update `trace' docs.
Based on a patch by Nala Ginrut.
* doc/ref/api-procedures.texi (Compiled Procedures): Expand
program-arguments-alist and program-lambda-list documentation.
* module/system/vm/program.scm (arity->arguments-alist): Fix the rest
arg if there are also keyword args, a bug found while documenting!
* test-suite/tests/session.test ("procedure-arguments"): Update.
* libguile/objcodes.c: Change to expect objcode on disk to be embedded
in ELF instead of having the funky cookie.
(to_native_order): Use already existing SCM_BYTE_ORDER style byte
order instead of chars.
(bytecode_to_objcode): No need for word_size arg.
(scm_bytecode_to_objcode, scm_objcode_to_bytecode): Take optional
endianness arg instead of sometimes using target-endianness.
(scm_load_objcode, scm_write_objcode, scm_bytecode_to_native_objcode):
Remove.
* libguile/objcodes.h: Adapt.
* libguile/vm.c (scm_load_compiled_with_vm): Use
scm_load_thunk_from_file.
(make_boot_program): Adapt to use scm_bytecode_to_objcode with
endianness arg.
* module/Makefile.am (OBJCODE_LANG_SOURCES): Add (language objcode
elf).
* module/language/objcode/elf.scm: New module, embeds objcode in ELF.
* module/language/bytecode/spec.scm (compile-objcode):
(decompile-objcode): Use (target-endianness).
* module/language/objcode/spec.scm: use (language objcode elf) for
write-objcode.
* module/scripts/disassemble.scm (disassemble):
* module/system/repl/command.scm (disassemble-file): Use
load-thunk-from-file.
* module/system/vm/objcode.scm: Remove load-objcode and write-objcode.
* test-suite/tests/asm-to-bytecode.test (test-target): Adapt to the new
ELF world.
* libguile/frames.c (scm_frame_source, scm_frame_previous):
* libguile/stacks.c (scm_make_stack):
* module/ice-9/boot-9.scm (exception-printers):
* module/system/vm/frame.scm (frame-call-representation): Fix more
assumptions that frame-procedure is a program, or even a procedure.
* libguile/frames.c (scm_frame_instruction_pointer):
* module/system/vm/frame.scm (frame-bindings):
(frame-next-source, frame-call-representation): Fix a few locations
that thought that the frame-procedure will always be a VM
procedure. This will not not be the case when traversing the stack of
an application of a non-procedure.
* libguile/vm-i-system.c (call, tail-call, mv-call): Instead of
special-casing structs and smobs at these call sites, just set up the
stack, and jump to a generic apply loop if the proc is not a program.
* libguile/vm-engine.c: The generic apply loop is here. Also, the boot
program is now simply a boot continuation, and can handle any number
of arguments.
* libguile/vm.c (make_boot_program): Update the code that makes the boot
continuation.
* libguile/frames.c (scm_frame_source): Don't call out to (system vm
frames), as this routine is used when printing exceptions. Make
available in the default environment (ugh).
* module/system/vm/frame.scm: Remove frame-source definition and
export.
* libguile/programs.h:
* libguile/programs.c (scm_program_source): Add an optional arg, the
sources table to traverse. Defaults to the result of
scm_program_sources.
* module/system/vm/program.scm (program-sources-pre-retire): Move
definition here from (system vm traps), and export.
* module/system/vm/traps.scm: Adapt.
* module/system/vm/frame.scm (frame-next-source): New exported binding,
returns the source line corresponding to the next instruction instead
of the previous instruction.
* module/system/vm/trace.scm (print-application, print-return): Change
to add more whitespace, as (ice-9 debug) did.
(call-with-trace): Rename from vm-trace, and make the vm a keyword
argument.
* module/system/repl/command.scm: Don't autoload (system vm profile).
(trace): Update for call-with-trace name change.
* module/system/vm/traps.scm (trap-matching-instructions): New trap,
just installs a next hook and runs the handler when a predicate
succeeds.
* module/system/vm/trap-state.scm (add-ephemeral-stepping-trap!): New
procedure, uses trap-matching-instructions with an appropriate
predicate to handle step, stepi, next, and nexti repl metacommands.
* module/system/repl/command.scm (step, step-instruction, next)
(next-instruction): New repl debugger commands.
