* module/system/repl/describe.scm (format-documentation): Remove bad
string syntax. This file doesn't compile though, due to define-macro
being a procedure->syntax macro.
* module/language/scheme/translate.scm (lookup-transformer): Allow for
undefined variables when doing the transformation -- it's possible that
they come from a module definition's forward declaration.
* module/system/repl/command.scm (import): Make into legal Scheme, caught
by the compiler :-)
* module/system/vm/assemble.scm (<vlink-now>): Remove the module field.
Immediate bindings will now always be relative to the current module.
Fixes some mess about process-define-module not being defined when
loading modules, probably because we destructively modified the
ghil-env.
(codegen, dump-object!): Don't dump a module name.
* src/vm_loader.c (link-now): Just use scm_lookup.
comments in ghil-lookup are pertinent.
* module/system/il/compile.scm (make-glil-var): Require that ghil vars
have environments. Remove the 'unresolved case -- we'll treat all
module-level variables as late bound.
* module/system/il/ghil.scm (ghil-lookup): Treat all module level vars as
late bound.
* module/system/vm/assemble.scm: Instead of vlink and vlate-bound, have
vlink-now and vlink-later.
(codegen): Add a bunch of crap to get the various cases right.
(object-assoc, dump-object!): Handle the new cases, remove the old
cases.
* src/vm_loader.c (link-now, link-later): Change from link and lazy-bind.
Include the module in which the link is to be done, so that callers
from other modules get the right behavior.
* src/vm_system.c (late-variable-ref, late-variable-set): Instead of a
sym, the unbound representation is a module name / symbol pair.
* testsuite/run-vm-tests.scm (run-vm-tests): Remove some debugging.
Fixes the mutually-recursive toplevel definitions case. This could be
fixed by rewriting bodies as letrecs, as r6 does, but that's not really
repl-compatible.
* module/system/il/ghil.scm (ghil-lookup): Ok, if we can't locate a
variable, mark it as unresolved.
* module/system/il/compile.scm (make-glil-var): Compile unresolved
variables as <glil-late-bound> objects.
* module/system/il/glil.scm: Add <glil-late-bound> definition.
* module/system/vm/assemble.scm (codegen): And, finally, when we see a
<vlate-bound> object, allocate a slot for it in the object vector,
setting it to a symbol. Add a new pair of instructions to resolve that
symbol to a variable at the last minute.
* src/vm_loader.c (load-number): Bugfix: the radix argument should be
SCM_UNDEFINED in order to default to 10.
(late-bind): Add an unresolved symbol to the object vector. Could be
replaced with load-symbol I guess.
* src/vm_system.c (late-variable-ref, late-variable-set): New
instructions to do late symbol binding.
* testsuite/Makefile.am (vm_test_files):
* testsuite/t-mutual-toplevel-defines.scm: New test, failing for some
reason involving the core even? and odd? definitions.
* module/language/scheme/translate.scm (lookup-transformer): When
expanding syncase macros, use the eval closure from the ghil-env.
Probably doesn't make any difference whatsoever.
* module/system/base/Makefile.am (SOURCES): Compile pmatch.scm, now that
it works :-))
* module/system/base/compile.scm (compile-in): Compile inside a
save-module-excursion, so that side effects of evaluation don't leak
out.
* module/system/base/pmatch.scm: Change from :use-syntax/:export-syntax
to simply :use-modules/:export. Also probably has no effect.
* module/system/il/ghil.scm (fix-ghil-mod!): Suppress warnings resulting
from compilation of define-module.
* src/vm_loader.c (link): So, referencing variables defined but not
exported from the current module didn't work. Fixed that, but it's
hacky. There are still some uncaught cases.
* module/system/base/compile.scm: Also import load-objcode from (system
vm core).
* module/language/scheme/translate.scm (lookup-transformer): Use
sc-expand3 in compilation mode when compiling macros. Yay, syncase
macros compile!
* module/system/vm/assemble.scm (dump-object!):
* src/vm_loader.c (VM_DEFINE_LOADER): Use scm_from_locale_keywordn, not
the krazy dash symbol stuff.
* module/language/scheme/translate.scm (lookup-transformer): Add a
special case for syncase macros.
* module/system/base/syntax.scm (define-record): Again, don't unquote in
actual objects, because this is uncompilable. Ah well. At least now all
of base/ is compiling.
* module/system/vm/assemble.scm (dump-object!): More debug info.
* module/language/scheme/translate.scm (primitive-syntax-table):
Translate the `else' clause of a cond as (begin ...). We used to use
trans-body, which processes internal defines, which are not legal
syntax here.
* module/system/base/syntax.scm (define-record): Unfortunately, we can't
unquote in the actual procedure for `%compute-initargs', because that
doesn't work with compilation. So reference %compute-initargs by name,
and export it.
* module/system/il/ghil.scm (apopq!): Gaaaaar. The order of the arguments
to assq-remove! was reversed, which was the badness, causing corruption
to the env after calling call-with-ghil-bindings. Grrrrrr.
(fix-ghil-mod!, ghil-lookup, ghil-define): As amply commented in the
code, deal with compile-time side effects to the current module by
lazily noticing and patching up the compile-time environment. A hacky
solution until such a time as we special-case something for
`define-module'.
* module/system/il/compile.scm (make-glil-var): Only dump the module if
we actually have one.
* module/system/il/ghil.scm (ghil-define): Make sure that ghil-var-env is
a ghil-env.
* src/vm_loader.c (link):
* module/system/vm/assemble.scm (dump-object!): Rewrite `link' to take
two Scheme arguments on the stack: the symbol, as before, and the
module in which the symbol was found at compile time. This introduces
some undesireable early binding, but it does let the vm load up
modules, and (potentially) have multiple modules in one .go file. On a
practical level, I can now compile modules and have their .go files
load up the modules' dependencies as necessary.
* module/system/il/compile.scm (make-glil-var): Make the :mod of the
glil-var actually a guile module, not a ghil-env.
* module/system/il/ghil.scm (module-lookup, ghil-lookup): For module
variables, encode the location where we found the variable in the
ghil-var.
* module/language/scheme/translate.scm (trans):
* module/system/il/compile.scm (codegen): When making records where a
value can be a keyword, make sure to use the keyword initialization
form, so that the record initializer doesn't interpret the keyword as a
slot name.
* module/system/base/Makefile.am (vm_DATA): For now, don't compile
pmatch.
* module/language/scheme/translate.scm (*the-compile-toplevel-symbol*)
(primitive-syntax-table): Existing eval-case invocations in boot-9.scm
only have `load-toplevel', not `load-toplevel' and `compile-toplevel'
as they should. Allow for interpreting `load-toplevel' as
`compile-toplevel'.
* module/language/scheme/translate.scm (trans): Remove the hacky case for
the unspecified value, not needed any more.
(primitive-syntax-table): Rework eval-case to understand
compile-toplevel and evaluate contexts, as in common lisp's eval-when:
http://www.lisp.org/HyperSpec/Body/speope_eval-when.html
This is the Right Thing.
* testsuite/t-match.scm:
* testsuite/t-records.scm: While the attempt to redefine use-syntax as
being "use during compilation" was cute, it does not reflect the
historical usage of use-syntax, nor does it correspond to existing code
that includes other modules and uses them during compilation.
So use-syntax has been replaced with use-modules. The test suites now
pass. In the future, compilation phases should be done on whole
modules, I think; r5rs-style computation does not have phases.
* module/language/scheme/translate.scm (eval-at-compile-time)
(&compile-time-module, expand-macro): Remove this attempt at dealing
with macros. Instead, we're going to rely on macros being first-class,
and just catch eval-case at the bottom.
(lookup-transformer): Lookup all syntax transformers in the module's
eval closure. We catch the primitive-macros, compiling them to ghil,
and expand the rest.
(lookup-transformer): Fold in trans-pair here. Add a hacky case for the
unspecified value; the problem shows up when compiling e.g.
(define-macro (plus! x) `(set! ,x (1+ x))), as a fallout from
eval-case.
(make-pmatch-transformers, primitive-syntax-table): Define the
primitive syntax transformers as a data-driven table instead of a
function. There's a bit of syntax, too. Eval-case was rewritten to use
pmatch.
* module/system/base/compile.scm (scheme): Define as a thunk instead
of a value, so as to allow (language scheme translate) to be imported
in the repl. Still, a hack.
* module/language/scheme/translate.scm (expand-macro, trans-pair): Remove
support for the scheme form, '(void). Replace it by (begin). What was
Keisuke thinking? :)
* Specific problems in IA64 make check
** test-unwind
Representation of the relevant dynamic context:
non-rewindable
catch frame make cont.
o----o-----a----------b-------------c
\
\ call cont.
o-----o-----------d
A continuation is captured at (c), with a non-rewindable frame in the
dynamic context at (b). If a rewind through that frame was attempted,
Guile would throw to the catch at (a). Then the context unwinds back
past (a), then winds forwards again, and the captured continuation is
called at (d).
We should end up at the catch at (a). On ia64, we get an "illegal
instruction".
The problem is that Guile does not restore the ia64 register backing
store (RBS) stack (which is saved off when the continuation is
captured) until all the unwinding and rewinding is done. Therefore,
when the rewind code (scm_i_dowinds) hits the non-rewindable frame at
(b), the RBS stack hasn't yet been restored. The throw finds the
jmp_buf (for the catch at (a)) correctly from the dynamic context, and
jumps back to (a), but the RBS stack is invalid, hence the illegal
instruction.
This could be fixed by restoring the RBS stack earlier, at the same
point (copy_stack) where the normal stack is restored. But that
causes a problem in the next test...
** continuations.test
The dynamic context diagram for this case is similar:
non-rewindable
catch frame make cont.
a----x-----o----------b-------------c
\
\ call cont.
o-------d
The only significant difference is that the catch point (a) is
upstream of where the dynamic context forks. This means that the RBS
stack at (d) already contains the correct RBS contents for throwing
back to (a), so it doesn't matter whether the RBS stack that was saved
off with the continuation gets restored.
This test passes with the Guile 1.8.4 code, but fails (with an
"illegal instruction") when the code is changed to restore the RBS
stack earlier as described above.
The problem now is that the RBS stack is being restored _too_ early;
specifically when there is still stuff to do that relies on the old
RBS contents. When a continuation is called, the sequence of relevant
events is:
(1) Grow the (normal) stack until it is bigger than the (normal)
stack saved off in the continuation. (scm_dynthrow, grow_stack)
(2) scm_i_dowinds calls itself recursively, such that
(2.1) for each rewind (from (x) to (c)) that will be needed,
another frame is added to the stack (both normal and RBS),
with local variables specifying the required rewind; the
rewinds don't actually happen yet, they will happen when
the stack unwinds again through these frames
(2.2) required unwinds - back from where the continuation was
called (d) to the fork point (x) - are done immediately.
(3) The normal (i.e. non-RBS) stack that was stored in the
continuation is restored (i.e. copied on top of the actual
stack).
Note that this doesn't overwrite the frames that were added in
(2.1), because the growth in (1) ensures that the added frames
are beyond the end of the restored stack.
(4) ? Restore the RBS stack here too ?
(5) Return (from copy_stack) through the (2.1) frames, which means
that the rewinds now happen.
(6) setcontext (or longjmp) to the context (c) where the
continuation was captured.
The trouble is that step (1) does not create space in the RBS stack in
the same kind of way that it does for the normal stack. Therefore, if
the saved (in the continuation) RBS stack is big enough, it can
overwrite the RBS of the (2.1) frames that still need to complete.
This causes an illegal instruction when we return through those frames
and try to perform the rewinds.
* Fix
The key to the fix is that the saved RBS stack only needs to be
restored at some point before the next setcontext call, and that doing
it as close to the setcontext call as possible will avoid bad
interactions with the pre-setcontext stack. Therefore we do the
restoration at the last possible point, immediately before the next
setcontext call.
The situation is complicated by there being two ways that the next
setcontext call can happen.
- If the unwinding and rewinding is all successful, the next
setcontext will be the one from step (6) above. This is the
"normal" continuation invocation case.
- If one of the rewinds throws an error, the next setcontext will
come from the throw implementation code. (And the one in step (6)
will never happen.) This is the rewind error case.
In the rewind error case, the code calling setcontext knows nothing
about the continuation. So to cover both cases, we:
- copy (in step (4) above) the address and length of the
continuation's saved RBS stack to the current thread state
(SCM_I_CURRENT_THREAD)
- modify all setcontext callers so that they check the current
thread state for a saved RBS stack, and restore it if so before
calling setcontext.
* Notes
** I think rewinders cannot rely on using any stack data
Unless it can be guaranteed that the data won't go into a register.
I'm not 100% sure about this, but I think it follows from the fact
that the RBS stack is not restored until after the rewinds have
happened.
Note that this isn't a regression caused by the current fix. In Guile
1.8.4, the RBS stack was restored _after_ the rewinds, and this is
still the case now.
** Most setcontext calls for `throw' don't need to change the RBS stack
In the absence of continuation invocation, the setcontext call in the
throw implementation code always sets context to a place higher up the
same stack (both normal and RBS), hence no stack restoration is
needed.
* Other changes
** Using setcontext for all non-local jumps (for __ia64__)
Along the way, I read a claim somewhere that setcontext was more
reliable than longjmp, in cases where the stack has been manipulated.
I don't now have any reason to believe this, but it seems reasonable
anyway to leave the __ia64__ code using getcontext/setcontext, instead
of setjmp/longjmp.
(I think the only possible argument against this would be performance -
if getcontext was significantly slower than setjmp. It that proves to
be the case, we should revisit this.)
** Capping RBS base for non-main threads
Somewhere else along the way, I hit a problem in GC, involving the RBS
stack of a non-main thread. The problem was, in
SCM_MARK_BACKING_STORE, that scm_ia64_register_backing_store_base was
returning a value that was massively greater than the value of
scm_ia64_ar_bsp, leading to a seg fault. This is because the
implementation of scm_ia64_register_backing_store_base is only valid
for the main thread. I couldn't find a neat way of getting the true
RBS base of a non-main thread, but one idea is simply to call
scm_ia64_ar_bsp when guilifying a thread, and use the value returned
as an upper bound for that thread's RBS base. (Note that the RBS
stack grows upwards.)
(Were it not for scm_init_guile, we could be much more definitive
about this. We could take the value of scm_ia64_ar_bsp as a
definitive base address for the part of the RBS stack that Guile cares
about. We could also then discard
scm_ia64_register_backing_store_base.)
* Makefile.am:
* module/Makefile.am:
* module/language/scheme/Makefile.am:
* module/system/Makefile.am:
* module/system/base/Makefile.am:
* module/system/il/Makefile.am:
* module/system/repl/Makefile.am:
* module/system/vm/Makefile.am: Cleaned up to be more complete, if not
completely working.
* module/guile/slib.scm:
* module/slib/: Removed the slib import; it's a bit out of place here,
and bitrotten at that.
* module/system/repl/repl.scm (default-pre-unwind-handler)
(default-catch-handler): New procedures, to do some error handling in
the repl.
(start-repl): Catch errors in the repl loop.
* module/system/base/compile.scm (<cenv>): No more cenv, it was a useless
data structure.
* module/system/repl/command.scm (*command-table*): Remove `use', it's
the same as `import'. Otherwise in this file, adapt to the repl having
direct pointers to the vm and the language, and to the module being in
the current-module fluid.
* module/system/repl/repl.scm (prompting-meta-read):
* module/system/repl/common.scm (<repl>): The repl now has a direct
pointer to the vm and language. Adapt accordingly.
Before:
> ,c (set! x 3)
0 (make-int8 3) ;; 3
2 (link "x")
5 (variable-set)
> ,c (define x 3)
0 (make-int8 3) ;; 3
2 (link "x")
5 (variable-set)
After:
> ,c (define x 3)
0 (make-int8 3) ;; 3
2 (define "x")
5 (variable-set)
* src/vm_loader.c (link): `link' now errors if the variable is undefined.
This corresponds with desired behavior, for both `ref' and `set'
operations, for scheme. It's not what elisp wants, though. Perhaps
elisp linking needs another instruction.
(define): New instruction, the same as calling scm_define(), basically.
* module/language/scheme/translate.scm (trans-pair): Don't try to look up
an existing variable definition when translating `define'; instead use
the special-purpose lookup from ghil.scm's `ghil-define'.
* module/system/il/compile.scm (codegen): Compile to a different kind of
variable access from `set!', specifically via passing 'define as the op
to `make-glil-var'.
* module/system/il/ghil.scm (ghil-lookup): Don't add to the module table
when compiling variable sets via `set!'.
(ghil-define): New procedure, for looking up variables for `define'.
* module/system/vm/assemble.scm (<vdefine>): New record: a new
instruction type.
(codegen): Compile `define' module vars into <vdefine>.
(dump-object!): <vdefine> == `define'.
* module/language/scheme/translate.scm (trans-pair): Add a guard to only
allow `define' at the top level; other defines are already filtered out
via trans-body.
* module/system/il/ghil.scm (ghil-env-toplevel?): Export, and fix.
* module/system/base/compile.scm:
* module/system/il/ghil.scm:
* module/system/repl/describe.scm:
* module/system/vm/core.scm:
* module/system/vm/frame.scm:
* module/system/vm/trace.scm: Explicitly list exports in the module
declaration instead of using define-public.
* module/system/base/compile.scm: Export cenv? also.
* module/system/base/syntax.scm: Clean up vestiges of the old structure
code. Make accessors defined as foo? instead of <foo>?.
* module/system/il/glil.scm:
* module/system/il/ghil.scm: Remove <foo>-1 accessors, since we have
named accessors.
* module/system/repl/common.scm (repl-prompt): Return a string instead of
outputting to the port, for better readline integration.
* module/system/repl/repl.scm (meta-reader, prompting-meta-read)
(start-repl): Integrate with (ice-9 readline) via the current-reader
fluid and the repl-reader function, both from boot-9.scm.
* module/system/repl/command.scm (system): Declare exports in the module
declaration.
* module/system/repl/repl.scm (start-repl): If the evaluation returns
multiple values, print them separately.
* module/system/base/syntax.scm (define-record): Rebase to implement on
top of Guile's records, which are the substrate of srfi-9's records.
(%compute-initargs): Rename from %make-struct, just return the list of
values.
(get-slot, set-slot!, slot): Removed, no longer used.
(record-case): Allow slots of the form (MYNAME SLOTNAME), which binds
SLOTNAME to MYNAME (instead of SLOTNAME to SLOTNAME).
(record-case, record?): No more ice-9 match!
* module/system/il/compile.scm (codegen): Tweaks so that the new record
code works.
* module/system/il/ghil.scm: Fix some slot references.
* module/system/vm/assemble.scm (preprocess, codegen): Remove calls to
`slot'.
(codegen): Fix some slot references.
