* libguile/vm.h (struct scm_vm_cont): Instead of saving the "IP", save
"RA" and "MVRA". That is, save singly-valued and multiply-valued
return addresses, so that we can return multiple values on the stack.
(scm_i_vm_reinstate_continuation): Remove.
* libguile/vm.c (vm_capture_continuation): Rename from capture_vm_cont,
and change the prototype so we can capture the RA and MVRA, and so
that tail calls to call/cc can capture a continuation without the
call/cc application frame.
(vm_return_to_continuation): Rename from reinstate_vm_cont, and take
arguments to return to the continuation. Handles returning to single
or multiple-value RA.
(scm_i_vm_capture_continuation): Change to invoke
vm_capture_continuation. Kept around for the benefit of make-stack.
* libguile/vm-i-system.c (continuation-call): Handle reinstatement of
the VM stack, with arguments.
(call/cc, tail-call/cc): Adapt to new vm_capture_continuation
prototype. tail-call/cc captures tail continuations.
* libguile/stacks.c (scm_make_stack): Update for scm_vm_cont structure
change.
* libguile/continuations.h (struct scm_contregs): Remove throw_value
member, which was used to return a value to a continuation.
(scm_i_check_continuation): New internal function, checks that a
continuation may be reinstated.
(scm_i_reinstate_continuation): Replaces scm_i_continuation_call; just
reinstates the C stack.
(scm_i_contregs_vm, scm_i_contregs_vm_cont): New internal accessors.
* libguile/continuations.c (scm_i_make_continuation): Return
SCM_UNDEFINED if we are returning again.
(grow_stack, copy_stack_and_call, scm_dynthrow): Remove extra arg, as
vm opcodes handle value returns.
(copy_stack): No need to instate VM continuation.
(scm_i_reinstate_continuation): Adapt.
* libguile/continuations.h:
* libguile/continuations.c (scm_i_make_continuation): Take VM and VM
continuation arguments as well; I'm not convinced that saving all VM
continuations was the right thing, and in any case we only ever saved
the latest. Running a new VM should create a continuation barrier.
* libguile/stacks.c (scm_make_stack):
* libguile/vm-i-system.c (call/cc, tail-call/cc): Adapt callers.
* libguile/vm.h (scm_i_vm_capture_continuation)
(scm_i_vm_reinstate_continuation): Change to be internal, and to only
capture and reinstate continuations for a particular VM.
* libguile/continuations.h
* libguile/continuations.c (scm_i_call_with_current_continuation): New
internal function. Not exported because I'm not sure whether or not
this should have a continuation barrier in the future. Uses a
hand-coded VM procedure.
* libguile/eval.c (eval): Use scm_i_call_with_current_continuation.
* libguile/vm-i-system.c (continuation-call): New op, like subr-call or
foreign-call, but for continuations.
* libguile/continuations.h: Add scm_i_continuation_call internal
declaration.
(SCM_CONTINUATIONP): Reimplement in terms of
SCM_PROGRAM_IS_CONTINUATION.
(scm_tc16_continuation, SCM_CONTREGS, SCM_CONTINUATION_LENGTH)
(SCM_SET_CONTINUATION_LENGTH, SCM_JMPBUF, SCM_DYNENV, SCM_THROW_VALUE)
(SCM_CONTINUATION_ROOT, SCM_DFRAME): Remove these from the exposed
API.
(scm_i_continuation_to_frame): New internal declaration.
* libguile/continuations.c
* libguile/continuations.c: Add trickery like in foreign.c, smob.c, and
gsubr.c, so that we can make procedural trampolines for continuations.
(scm_i_continuation_to_frame): New internal function, from stacks.c.
* libguile/programs.h (SCM_F_PROGRAM_IS_CONTINUATION)
(SCM_PROGRAM_IS_CONTINUATION): Add a flag for programs that are
continuations. Probably should add flags for the other trampoline
types too.
* libguile/programs.c (scm_i_program_print): Print continuations as
before.
* libguile/stacks.c (scm_stack_id, scm_make_stack): Use
scm_i_continuation_to_frame in the continuation case.
* libguile/debug.h (scm_t_debug_frame): Remove this type, as it was
internal to the old evaluator.
(SCM_EVALFRAME, SCM_APPLYFRAME, SCM_VOIDFRAME, SCM_MACROEXPF)
(SCM_TAILREC, SCM_TRACED_FRAME, SCM_ARGS_READY, SCM_DOVERFLOW)
(SCM_MAX_FRAME_SIZE, SCM_FRAMETYPE)
(SCM_EVALFRAMEP, SCM_APPLYFRAMEP, SCM_VOIDFRAMEP, SCM_MACROEXPFP)
(SCM_TAILRECP, SCM_TRACED_FRAME_P, SCM_ARGS_READY_P, SCM_OVERFLOWP)
(SCM_SET_MACROEXP, SCM_SET_TAILREC, SCM_SET_TRACED_FRAME)
(SCM_SET_ARGSREADY, SCM_SET_OVERFLOW)
(SCM_CLEAR_MACROEXP, SCM_CLEAR_TRACED_FRAME, SCM_CLEAR_ARGSREADY):
Remove macro accessors to scm_t_debug_frame.
(SCM_DEBUGOBJP, SCM_DEBUGOBJ_FRAME, SCM_SET_DEBUGOBJ_FRAME):
(scm_debug_object_p, scm_make_debugobj): Remove debugobj accessors.
(scm_i_unmemoize_expr): Remove unused declaration.
* libguile/debug.c (scm_debug_options): No more max limit on frame
sizes.
(scm_start_stack): Just call out to scm_vm_call_with_new_stack.
(scm_debug_object_p, scm_make_debugobj, scm_init_debug): No more
debugobj smob type.
* libguile/deprecated.h:
* libguile/deprecated.c (scm_i_deprecated_last_debug_frame)
(scm_last_debug_frame): Remove deprecated debug-frame bits.
* libguile/stacks.c (scm_make_stack): Rework this function and its
dependents to only walk VM frames.
(scm_stack_id): Call out to the holder of the VM frame in question,
which should be a VM or a VM continuation, for the stack ID. Currently
this bit is stubbed out.
(scm_last_stack_frame): Removed. It seems this is mainly useful for a
debugger, and we need to rewrite the debugger to work on the Scheme
level.
* test-suite/tests/continuations.test ("continuations"): Remove test for
last-stack-frame.
* libguile/continuations.h (struct scm_t_contregs):
* libguile/continuations.c (scm_make_continuation):
(copy_stack_and_call, scm_i_with_continuation_barrier): No need to
save and restore debug frames.
* libguile/threads.h (scm_i_thread): Don't track debug frames.
(scm_i_last_debug_frame, scm_i_set_last_debug_frame): Remove macro
accessors.
* libguile/threads.c (guilify_self_1): Don't track debug frames.
* libguile/throw.c: No need to track debug frames in a jmpbuf.
* libguile/vm-engine.c (vm_engine, VM_PUSH_DEBUG_FRAMES): Don't push
debug frames.
* libguile/vm.h:
* libguile/vm.c (scm_vm_call_with_new_stack): New function. Currently
stubbed out though.
The crux of this problem was that the thread doing a throw, and so
checking scm_i_critical_section_level, was different from the thread
that was in a critical section.
* libguile/async.h (scm_i_critical_section_level): Removed, replaced
by per-thread critical_section_level.
(SCM_CRITICAL_SECTION_START, SCM_CRITICAL_SECTION_END): Use
per-thread critical_section_level.
* libguile/continuations.c (scm_dynthrow): Check per-thread
critical_section_level.
* libguile/threads.c (guilify_self_1): Init per-thread
critical_section_level.
(scm_i_critical_section_level): Removed.
* libguile/threads.h (scm_i_thread): New critical_section_level field.
* libguile/throw.c (scm_ithrow): Check per-thread critical_section_level.
Problem was that if an application includes both libguile.h and the
system's setjmp.h, and is compiled on IA64, it gets compile errors
because of jmp_buf, setjmp and longjmp being multiply defined.
* libguile/__scm.h (__ia64__): Define scm_i_jmp_buf, SCM_I_SETJMP and
SCM_I_LONGJMP instead of jmp_buf, setjmp and longjmp.
(all other platforms): Map scm_i_jmp_buf, SCM_I_SETJMP and
SCM_I_LONGJMP to jmp_buf, setjmp and longjmp.
* libguile/continuations.c (scm_make_continuation): Use `SCM_I_SETJMP'
instead of `setjmp'.
(copy_stack_and_call): Use `SCM_I_LONJMP' instead of `longjmp'.
(scm_ia64_longjmp): Use type `scm_i_jmp_buf' instead of `jmp_buf'.
* libguile/continuations.h (scm_t_contregs): Use type `scm_i_jmp_buf'
instead of `jmp_buf'.
* libguile/threads.c (suspend): Use `SCM_I_SETJMP' instead of
`setjmp'.
* libguile/threads.h (scm_i_thread): Use type `scm_i_jmp_buf' instead
of `jmp_buf'.
* libguile/throw.c (JBJMPBUF, make_jmpbuf, jmp_buf_and_retval): Use
type `scm_i_jmp_buf' instead of `jmp_buf'.
(scm_c_catch): Use `SCM_I_SETJMP' instead of `setjmp'.
(scm_ithrow): Use `SCM_I_LONGJMP' instead of `longjmp'.
Thanks to Juhani Rantanen for the report.
* libguile/continuations.c (scm_make_continuation): Delay making the smob
until the data is fully initialized. Fixes a bug whereby a GC in
scm_vm_capture_continuations would catch the us with an undefined
continuation->vm_conts, leading to marking badness.
* libguile/vm.c (vm_cont_free): Report the correct size to scm_gc_free.
* libguile/continuations.c (continuation_mark): Mark the vm
continuations.
* libguile/vm.c (vm_cont_mark): Fix the marking function.
(vm_mark): Fix this one too -- the size is a number of STACKITEMS,
which we foolishly assume are the same size as SCM.
* test-suite/tests/ftw.test: Make our stat hacks verifyable without
assuming that they are interpreted.
* test-suite/tests/r5rs_pitfall.test: Re-indent.
* libguile/continuations.c (scm_make_continuation): Capture VM
continuations as well, as their stack is outside the C stack.
(copy_stack): Reinstate VM stacks with the C stack.
* libguile/continuations.h (scm_t_contregs): Add a pointer for VM stacks.
A binary-incompatible change -- hopefully not too many people were
messing around with this struct, though.
* libguile/vm-engine.c (vm_run): Add a note about possibly maintaining a
stack of vms.
* libguile/vm.c (struct scm_vm_cont): New struct, distinct from scm_vm.
(vm_cont_mark, vm_cont_free, capture_vm_cont, reinstate_vm_cont):
Reorder some code, and fix some bad assumptions about what part of the
stack to copy; obviously this code was never used.
* libguile/vm.h:
* libguile/vm.c (scm_vm_capture_continuations)
(scm_vm_reinstate_continuations): New public functions, used by
continuations.c.
* Specific problems in IA64 make check
** test-unwind
Representation of the relevant dynamic context:
non-rewindable
catch frame make cont.
o----o-----a----------b-------------c
\
\ call cont.
o-----o-----------d
A continuation is captured at (c), with a non-rewindable frame in the
dynamic context at (b). If a rewind through that frame was attempted,
Guile would throw to the catch at (a). Then the context unwinds back
past (a), then winds forwards again, and the captured continuation is
called at (d).
We should end up at the catch at (a). On ia64, we get an "illegal
instruction".
The problem is that Guile does not restore the ia64 register backing
store (RBS) stack (which is saved off when the continuation is
captured) until all the unwinding and rewinding is done. Therefore,
when the rewind code (scm_i_dowinds) hits the non-rewindable frame at
(b), the RBS stack hasn't yet been restored. The throw finds the
jmp_buf (for the catch at (a)) correctly from the dynamic context, and
jumps back to (a), but the RBS stack is invalid, hence the illegal
instruction.
This could be fixed by restoring the RBS stack earlier, at the same
point (copy_stack) where the normal stack is restored. But that
causes a problem in the next test...
** continuations.test
The dynamic context diagram for this case is similar:
non-rewindable
catch frame make cont.
a----x-----o----------b-------------c
\
\ call cont.
o-------d
The only significant difference is that the catch point (a) is
upstream of where the dynamic context forks. This means that the RBS
stack at (d) already contains the correct RBS contents for throwing
back to (a), so it doesn't matter whether the RBS stack that was saved
off with the continuation gets restored.
This test passes with the Guile 1.8.4 code, but fails (with an
"illegal instruction") when the code is changed to restore the RBS
stack earlier as described above.
The problem now is that the RBS stack is being restored _too_ early;
specifically when there is still stuff to do that relies on the old
RBS contents. When a continuation is called, the sequence of relevant
events is:
(1) Grow the (normal) stack until it is bigger than the (normal)
stack saved off in the continuation. (scm_dynthrow, grow_stack)
(2) scm_i_dowinds calls itself recursively, such that
(2.1) for each rewind (from (x) to (c)) that will be needed,
another frame is added to the stack (both normal and RBS),
with local variables specifying the required rewind; the
rewinds don't actually happen yet, they will happen when
the stack unwinds again through these frames
(2.2) required unwinds - back from where the continuation was
called (d) to the fork point (x) - are done immediately.
(3) The normal (i.e. non-RBS) stack that was stored in the
continuation is restored (i.e. copied on top of the actual
stack).
Note that this doesn't overwrite the frames that were added in
(2.1), because the growth in (1) ensures that the added frames
are beyond the end of the restored stack.
(4) ? Restore the RBS stack here too ?
(5) Return (from copy_stack) through the (2.1) frames, which means
that the rewinds now happen.
(6) setcontext (or longjmp) to the context (c) where the
continuation was captured.
The trouble is that step (1) does not create space in the RBS stack in
the same kind of way that it does for the normal stack. Therefore, if
the saved (in the continuation) RBS stack is big enough, it can
overwrite the RBS of the (2.1) frames that still need to complete.
This causes an illegal instruction when we return through those frames
and try to perform the rewinds.
* Fix
The key to the fix is that the saved RBS stack only needs to be
restored at some point before the next setcontext call, and that doing
it as close to the setcontext call as possible will avoid bad
interactions with the pre-setcontext stack. Therefore we do the
restoration at the last possible point, immediately before the next
setcontext call.
The situation is complicated by there being two ways that the next
setcontext call can happen.
- If the unwinding and rewinding is all successful, the next
setcontext will be the one from step (6) above. This is the
"normal" continuation invocation case.
- If one of the rewinds throws an error, the next setcontext will
come from the throw implementation code. (And the one in step (6)
will never happen.) This is the rewind error case.
In the rewind error case, the code calling setcontext knows nothing
about the continuation. So to cover both cases, we:
- copy (in step (4) above) the address and length of the
continuation's saved RBS stack to the current thread state
(SCM_I_CURRENT_THREAD)
- modify all setcontext callers so that they check the current
thread state for a saved RBS stack, and restore it if so before
calling setcontext.
* Notes
** I think rewinders cannot rely on using any stack data
Unless it can be guaranteed that the data won't go into a register.
I'm not 100% sure about this, but I think it follows from the fact
that the RBS stack is not restored until after the rewinds have
happened.
Note that this isn't a regression caused by the current fix. In Guile
1.8.4, the RBS stack was restored _after_ the rewinds, and this is
still the case now.
** Most setcontext calls for `throw' don't need to change the RBS stack
In the absence of continuation invocation, the setcontext call in the
throw implementation code always sets context to a place higher up the
same stack (both normal and RBS), hence no stack restoration is
needed.
* Other changes
** Using setcontext for all non-local jumps (for __ia64__)
Along the way, I read a claim somewhere that setcontext was more
reliable than longjmp, in cases where the stack has been manipulated.
I don't now have any reason to believe this, but it seems reasonable
anyway to leave the __ia64__ code using getcontext/setcontext, instead
of setjmp/longjmp.
(I think the only possible argument against this would be performance -
if getcontext was significantly slower than setjmp. It that proves to
be the case, we should revisit this.)
** Capping RBS base for non-main threads
Somewhere else along the way, I hit a problem in GC, involving the RBS
stack of a non-main thread. The problem was, in
SCM_MARK_BACKING_STORE, that scm_ia64_register_backing_store_base was
returning a value that was massively greater than the value of
scm_ia64_ar_bsp, leading to a seg fault. This is because the
implementation of scm_ia64_register_backing_store_base is only valid
for the main thread. I couldn't find a neat way of getting the true
RBS base of a non-main thread, but one idea is simply to call
scm_ia64_ar_bsp when guilifying a thread, and use the value returned
as an upper bound for that thread's RBS base. (Note that the RBS
stack grows upwards.)
(Were it not for scm_init_guile, we could be much more definitive
about this. We could take the value of scm_ia64_ar_bsp as a
definitive base address for the part of the RBS stack that Guile cares
about. We could also then discard
scm_ia64_register_backing_store_base.)
* threads.c (SCM_MARK_BACKING_STORE): Use scm_ia64_ar_bsp() and
scm_ia64_register_backing_store_base() instead of Linux-specific
implementations.
* gc.h (scm_ia64_register_backing_store_base, scm_ia64_ar_bsp):
New declarations.
* gc.c (__libc_ia64_register_backing_store_base): Declaration
removed.
(scm_ia64_register_backing_store_base, scm_ia64_ar_bsp): New, with
implementations for Linux and HP-UX.
* coop-pthreads.c (SCM_MARK_BACKING_STORE): Use scm_ia64_ar_bsp()
and scm_ia64_register_backing_store_base() instead of
Linux-specific implementations.
* continuations.h (__libc_ia64_register_backing_store_base):
Declaration removed.
(scm_t_contregs): New "fresh" field.
* continuations.c (ia64_getcontext): Removed.
(scm_make_continuation): Use continuation fresh field instead of
interpreting getcontext return values (which isn't portable). Use
scm_ia64_ar_bsp() and scm_ia64_register_backing_store_base()
instead of Linux-specific implementations.
(copy_stack_and_call): Use scm_ia64_register_backing_store_base()
instead of Linux-specific implementation.
* _scm.h (__ia64__): Also detect __ia64.
scm_catch_with_pre_unwind_handler, scm_with_throw_handler): New.
* throw.c (SCM_JBPREUNWIND, SCM_SETJBPREUNWIND): New.
(struct pre_unwind_data): New, replaces struct lazy_catch.
(scm_c_catch): New, replaces scm_internal_catch as the primary
catch API for C code; adds pre-unwind handler support.
(scm_internal_catch): Now just a wrapper for scm_c_catch, for back
compatibility.
(tc16_pre_unwind_data, pre_unwind_data_print,
make_pre_unwind_data, SCM_PRE_UNWIND_DATA_P): Renamed from
"lazy_catch" equivalents.
(scm_c_with_throw_handler): New, replaces scm_internal_lazy_catch
as the primary C API for a "lazy" catch.
(scm_internal_lazy_catch): Now just a wrapper for
scm_c_with_throw_handler, for back compatibility.
(scm_catch_with_pre_unwind_handler): Renamed from scm_catch; adds
pre-unwind handler support.
(scm_catch): Now just a wrapper for
scm_catch_with_pre_unwind_handler, for back compatibility.
(scm_with_throw_handler): New.
(scm_lazy_catch): Update comment to say that the handler can
return, and what happens if it does.
(toggle_pre_unwind_running): New.
(scm_ithrow): When identifying the throw target, take running
flags into account. In general, change naming of things from
"lazy_catch" to "pre_unwind". When throwing to a throw handler,
don't unwind the dynamic context first. Add dynwind framing to
manage the running flag of a throw handler. If a lazy catch or
throw handler returns, rethrow the same exception again. Add
pre-unwind support to the normal catch case (SCM_JMPBUFP).
* root.c (scm_internal_cwdr): Add NULL args to
scm_i_with_continuation_barrier call.
* dynwind.c: Change comment mentioning lazy-catch to mention
pre-unwind data and throw handler also.
* continuations.h (scm_i_with_continuation_barrier): Add
pre-unwind handler args.
* continuations.c (scm_i_with_continuation_barrier): Add
pre-unwind handler args, and pass on to scm_c_catch (changed from
scm_internal_catch).
(c_handler): Remove scm_handle_by_message_noexit call.
(scm_c_with_continuation_barrier): Call
scm_i_with_continuation_barrier with scm_handle_by_message_noexit
as the pre-unwind handler.
(scm_handler): Remove scm_handle_by_message_noexit call.
(s_scm_with_continuation_barrier): Call
scm_i_with_continuation_barrier with scm_handle_by_message_noexit
as the pre-unwind handler.
print.c, ports.c, mallocs.c, hooks.c, hashtab.c, fports.c,
guardians.c, filesys.c, coop-pthreads.c, continuations.c: Use
scm_uintprint to print unsigned integers, raw heap words, and
adresses, using a cast to scm_t_bits to turn pointers into
integers.
part of copy_stack_and_call.
(copy_stack_and_call): Copy the stack after unwinding and before
rewinding.
(scm_dynthrow): Do not call scm_dowinds, this is now done by
copy_stack_and_call.
* continuations.c: Redeclaration of getcontext() via the
__asm__ ("getcontext") directive.
* continuations.h: Include <ucontext.h> instead of
<sys/ucontext.h>.
non-zero is returned from a port or smob free function.
(scm_malloc, scm_realloc, scm_strndup, scm_strdup,
scm_gc_register_collectable_memory,
scm_gc_unregister_collectable_memory, scm_gc_malloc,
scm_gc_realloc, scm_gc_free, scm_gc_strndup, scm_gc_strdup): New.
* backtrace.c, continuations.c, convert.i.c, coop-threads.c,
debug-malloc.c, dynl.c, environments.c, environments.h,
extensions.c, filesys.c, fports.c, gc.c, gc.h, gh_data.c, goops.c,
guardians.c, hooks.c, init.c, keywords.c, load.c, numbers.c,
ports.c, posix.c, procs.c, rdelim.c, regex-posix.c, root.c,
smob.c, stime.c, strings.c, struct.c, struct.h, symbols.c, unif.c,
vectors.c, weaks.c: Use scm_gc_malloc/scm_malloc and
scm_gc_free/free instead of scm_must_malloc and scm_must_free, as
appropriate. Return zero from smob and port free functions.
* debug-malloc.c (scm_malloc_reregister): Handle "old == NULL".
* fports.c (scm_setvbuf): Reset read buffer to saved values when
it is pointing to the putback buffer.
