* Specific problems in IA64 make check
** test-unwind
Representation of the relevant dynamic context:
non-rewindable
catch frame make cont.
o----o-----a----------b-------------c
\
\ call cont.
o-----o-----------d
A continuation is captured at (c), with a non-rewindable frame in the
dynamic context at (b). If a rewind through that frame was attempted,
Guile would throw to the catch at (a). Then the context unwinds back
past (a), then winds forwards again, and the captured continuation is
called at (d).
We should end up at the catch at (a). On ia64, we get an "illegal
instruction".
The problem is that Guile does not restore the ia64 register backing
store (RBS) stack (which is saved off when the continuation is
captured) until all the unwinding and rewinding is done. Therefore,
when the rewind code (scm_i_dowinds) hits the non-rewindable frame at
(b), the RBS stack hasn't yet been restored. The throw finds the
jmp_buf (for the catch at (a)) correctly from the dynamic context, and
jumps back to (a), but the RBS stack is invalid, hence the illegal
instruction.
This could be fixed by restoring the RBS stack earlier, at the same
point (copy_stack) where the normal stack is restored. But that
causes a problem in the next test...
** continuations.test
The dynamic context diagram for this case is similar:
non-rewindable
catch frame make cont.
a----x-----o----------b-------------c
\
\ call cont.
o-------d
The only significant difference is that the catch point (a) is
upstream of where the dynamic context forks. This means that the RBS
stack at (d) already contains the correct RBS contents for throwing
back to (a), so it doesn't matter whether the RBS stack that was saved
off with the continuation gets restored.
This test passes with the Guile 1.8.4 code, but fails (with an
"illegal instruction") when the code is changed to restore the RBS
stack earlier as described above.
The problem now is that the RBS stack is being restored _too_ early;
specifically when there is still stuff to do that relies on the old
RBS contents. When a continuation is called, the sequence of relevant
events is:
(1) Grow the (normal) stack until it is bigger than the (normal)
stack saved off in the continuation. (scm_dynthrow, grow_stack)
(2) scm_i_dowinds calls itself recursively, such that
(2.1) for each rewind (from (x) to (c)) that will be needed,
another frame is added to the stack (both normal and RBS),
with local variables specifying the required rewind; the
rewinds don't actually happen yet, they will happen when
the stack unwinds again through these frames
(2.2) required unwinds - back from where the continuation was
called (d) to the fork point (x) - are done immediately.
(3) The normal (i.e. non-RBS) stack that was stored in the
continuation is restored (i.e. copied on top of the actual
stack).
Note that this doesn't overwrite the frames that were added in
(2.1), because the growth in (1) ensures that the added frames
are beyond the end of the restored stack.
(4) ? Restore the RBS stack here too ?
(5) Return (from copy_stack) through the (2.1) frames, which means
that the rewinds now happen.
(6) setcontext (or longjmp) to the context (c) where the
continuation was captured.
The trouble is that step (1) does not create space in the RBS stack in
the same kind of way that it does for the normal stack. Therefore, if
the saved (in the continuation) RBS stack is big enough, it can
overwrite the RBS of the (2.1) frames that still need to complete.
This causes an illegal instruction when we return through those frames
and try to perform the rewinds.
* Fix
The key to the fix is that the saved RBS stack only needs to be
restored at some point before the next setcontext call, and that doing
it as close to the setcontext call as possible will avoid bad
interactions with the pre-setcontext stack. Therefore we do the
restoration at the last possible point, immediately before the next
setcontext call.
The situation is complicated by there being two ways that the next
setcontext call can happen.
- If the unwinding and rewinding is all successful, the next
setcontext will be the one from step (6) above. This is the
"normal" continuation invocation case.
- If one of the rewinds throws an error, the next setcontext will
come from the throw implementation code. (And the one in step (6)
will never happen.) This is the rewind error case.
In the rewind error case, the code calling setcontext knows nothing
about the continuation. So to cover both cases, we:
- copy (in step (4) above) the address and length of the
continuation's saved RBS stack to the current thread state
(SCM_I_CURRENT_THREAD)
- modify all setcontext callers so that they check the current
thread state for a saved RBS stack, and restore it if so before
calling setcontext.
* Notes
** I think rewinders cannot rely on using any stack data
Unless it can be guaranteed that the data won't go into a register.
I'm not 100% sure about this, but I think it follows from the fact
that the RBS stack is not restored until after the rewinds have
happened.
Note that this isn't a regression caused by the current fix. In Guile
1.8.4, the RBS stack was restored _after_ the rewinds, and this is
still the case now.
** Most setcontext calls for `throw' don't need to change the RBS stack
In the absence of continuation invocation, the setcontext call in the
throw implementation code always sets context to a place higher up the
same stack (both normal and RBS), hence no stack restoration is
needed.
* Other changes
** Using setcontext for all non-local jumps (for __ia64__)
Along the way, I read a claim somewhere that setcontext was more
reliable than longjmp, in cases where the stack has been manipulated.
I don't now have any reason to believe this, but it seems reasonable
anyway to leave the __ia64__ code using getcontext/setcontext, instead
of setjmp/longjmp.
(I think the only possible argument against this would be performance -
if getcontext was significantly slower than setjmp. It that proves to
be the case, we should revisit this.)
** Capping RBS base for non-main threads
Somewhere else along the way, I hit a problem in GC, involving the RBS
stack of a non-main thread. The problem was, in
SCM_MARK_BACKING_STORE, that scm_ia64_register_backing_store_base was
returning a value that was massively greater than the value of
scm_ia64_ar_bsp, leading to a seg fault. This is because the
implementation of scm_ia64_register_backing_store_base is only valid
for the main thread. I couldn't find a neat way of getting the true
RBS base of a non-main thread, but one idea is simply to call
scm_ia64_ar_bsp when guilifying a thread, and use the value returned
as an upper bound for that thread's RBS base. (Note that the RBS
stack grows upwards.)
(Were it not for scm_init_guile, we could be much more definitive
about this. We could take the value of scm_ia64_ar_bsp as a
definitive base address for the part of the RBS stack that Guile cares
about. We could also then discard
scm_ia64_register_backing_store_base.)
1000000000.
(unchecked_unlock_sym, allow_external_unlock_sym,
recursive_sym): Use SCM_SYMBOL.
(scm_make_mutex_with_flags): When raising unsupported option
error, report what the unsupported option was.
(fat_mutex_unlock): When raising errors, unlock m->lock first.
(fat_cond_timedwait): Removed.
(scm_timed_wait_condition_variable): Call fat_mutex_unlock
directly instead of via fat_cond_timedwait.
scm_set_thread_cleanup_x, scm_thread_cleanup): Lock on thread-specific
admin mutex instead of `thread_admin_mutex'.
* threads.h (scm_i_thread)[admin_mutex]: New field.
* throw.c (make_jmpbuf): Don't enter critical section during thread
spawn -- there is a possibility of deadlock if other threads are
exiting.
* threads.c (SCM_MARK_BACKING_STORE): Use scm_ia64_ar_bsp() and
scm_ia64_register_backing_store_base() instead of Linux-specific
implementations.
* gc.h (scm_ia64_register_backing_store_base, scm_ia64_ar_bsp):
New declarations.
* gc.c (__libc_ia64_register_backing_store_base): Declaration
removed.
(scm_ia64_register_backing_store_base, scm_ia64_ar_bsp): New, with
implementations for Linux and HP-UX.
* coop-pthreads.c (SCM_MARK_BACKING_STORE): Use scm_ia64_ar_bsp()
and scm_ia64_register_backing_store_base() instead of
Linux-specific implementations.
* continuations.h (__libc_ia64_register_backing_store_base):
Declaration removed.
(scm_t_contregs): New "fresh" field.
* continuations.c (ia64_getcontext): Removed.
(scm_make_continuation): Use continuation fresh field instead of
interpreting getcontext return values (which isn't portable). Use
scm_ia64_ar_bsp() and scm_ia64_register_backing_store_base()
instead of Linux-specific implementations.
(copy_stack_and_call): Use scm_ia64_register_backing_store_base()
instead of Linux-specific implementation.
* _scm.h (__ia64__): Also detect __ia64.
(SCM_I_PTHREAD_RECURSIVE_MUTEX_INITIALIZER): Removed.
(scm_i_pthread_mutexattr_recursive): New.
* threads.c (scm_i_pthread_mutexattr_recursive): Declare.
(scm_i_critical_section_mutex): Do not initialize statically.
(scm_threads_prehistory): Initialize
scm_i_pthread_mutexattr_recursive and scm_i_critical_section_mutex
here.
* eval.c (source_mutex): Do not initialiaze statically.
(scm_init_eval): Do it here, using
scm_i_pthread_mutexattr_recursive.
code.
* gc.c (s_scm_gc_live_object_stats): Don't declare alist in middle
of code.
* gc-card.c (scm_i_card_statistics): Don't declare tag in middle
of code.
(scm_i_card_statistics): Add block for declarations of tag_as_scm
and current.
(scm_i_thread): Removed unused signal_asyncs field.
(threads_mark): Do not mark it.
(guilify_self_1): Do not initialize it. Do initialize
continuation_root field.
(do_thread_exit): Do not remove thread from all_threads list.
(on_thread_exit): Do it here, after leaving guile mode.
(sleep_level): Removed.
(scm_i_thread_put_to_sleep): Leave thread_admin_mutex locked when
returning. Do not support recursive sleeps.
(scm_i_thread_wake_up): Expect thread_admin_mutex to be locked on
entry. Do not support recursive sleeps.
SCM_CRITICAL_SECTION_END): Moved here from threads.h since now
they also block/unblock execution of asyncs and call
scm_async_click which is declared in async.h but threads.h can not
include async.h since async.h already includes threads.h.
(scm_i_critical_section_level): New, for checking mistakes in the
use of the SCM_CRITICAL_SECTION_* macros.
(scm_i_critical_section_mutex): Make it a recursive mutex so that
critical sections can be nested.
* threads.h, threads.c (scm_frame_lock_mutex): New.
(scm_frame_critical_section): Take mutex as argument.
(framed_critical_section_mutex): New, used as default for above.
(scm_init_threads): Initialize it.
(scm_threads_prehistory): Do not initialize thread_admin_mutex and
scm_i_critical_section_mutex; both are initialized statically.
print.c, ports.c, mallocs.c, hooks.c, hashtab.c, fports.c,
guardians.c, filesys.c, coop-pthreads.c, continuations.c: Use
scm_uintprint to print unsigned integers, raw heap words, and
adresses, using a cast to scm_t_bits to turn pointers into
integers.
* configure.in (isinf): Let configure find the isinf() function
on MinGW32 systems.
2004-09-03 Stefan Jahn <stefan@lkcc.org>
* threads.c (scm_threads_mark_stacks): Fixed local variable
definitions.
* strings.c (scm_i_substring_copy, s_scm_string_append): Fixed
local variable definitions.
* stime.c (_POSIX_C_SOURCE): Do not define this item on
MinGW32 because it conflicts with its pthread headers.
(s_scm_mktime): Consider the HAVE_STRUCT_TM_TM_ZONE define.
(s_scm_strftime): Using scm_from_locale_string() instead of
scm_makfrom0str().
* posix.c (s_scm_putenv): Fixed typo in the !HAVE_UNSETENV
part.
* numbers.c (scm_init_numbers): Removed check_sanity() call
inside GUILE_DEBUG. The function has been removed somewhen...
* filesys.c (_POSIX_C_SOURCE): Do not define this item on
MinGW32 because it conflicts with its pthread headers.
2004-09-03 Stefan Jahn <stefan@lkcc.org>
* srfi-1.c, srfi-1.h: Renamed any 'lst1' into 'list1' because
lst1 is a #define on Win32 systems.
be nested.
(scm_i_thread_put_to_sleep): Call scm_i_leave_guile before locking
admin mutex so that we can be put to sleep by other threads while
blocking on that mutex. Lock all the heap mutex of all threads,
including ourselves.
(scm_i_thread_wake_up): Unlock all threads, including ourselves,
call scm_i_enter_guile.
(scm_thread_mark_stacks): Expect all threads to be suspended.
SCM_NEGATE_BOOL, SCM_BOOLP): Deprecated by moving into "deprecated.h".
Replaced all uses with scm_is_false, scm_is_true, scm_from_bool, and
scm_is_bool, respectively.
* configure.in: Checking for __int64 as possible candidate for
the SCM_I_GSC_T_INT64 define.
2003-06-14 Stefan Jahn <stefan@lkcc.org>
* threads.h: Redefined scm_getspecific() and scm_setspecific()
to be functions instead of macros.
* threads.c: Conditionalized inclusion of <sys/time.h> and
<unistd.h>.
(scm_getspecific, scm_setspecific): Made these two function
real part of the API.
* posix.c (s_scm_putenv): Added some code to make a
(putenv "FOO="), i.e. setting an empty string, work also on
Win32 systems. Thanks to Kevin Ryde for the proposal.
(void *) in order to avoid an aliasing warning; thanks to Bruce
Korb.)
* stackchk.h (SCM_STACK_OVERFLOW_P): Use SCM_STACK_PTR.
* threads.c (suspend, launch_thread, scm_threads_mark_stacks): Use
SCM_STACK_PTR.
* threads.c (scm_threads_mark_stacks): Bugfix: Changed
thread->base --> t->base.
* eval.c (SCM_CEVAL): Don't cast argument of SCM_STACK_OVERFLOW_P.
symbols to expect 0 or 1 values rather than 1 or undefined.
i.e. change #ifdef to #if, etc.
* null-threads.c: fix various preprocessor usages of new public
symbols to expect 0 or 1 values rather than 1 or undefined.
i.e. change #ifdef to #if, etc.
* coop-threads.c: fix various preprocessor usages of new public
symbols to expect 0 or 1 values rather than 1 or undefined.
i.e. change #ifdef to #if, etc.
* coop-pthreads.c: fix various preprocessor usages of new public
symbols to expect 0 or 1 values rather than 1 or undefined.
i.e. change #ifdef to #if, etc.
any critical defines like _GNU_SOURCE early. Replace usage of
struct timespect with scm_t_timespec. Replace usage of
STACK_GROWS_UP with SCM_STACK_GROWS_UP. Replace usage of
USE_PTHREAD_THREADS with SCM_USE_PTHREAD_THREADS.
* gc.c (scm_gc_for_newcell), gc-malloc.c (scm_realloc,
scm_gc_register_collectable_memory): Substitute locking of
scm_i_sweep_mutex for calls to scm_i_thread_put_to_sleep.
(scm_igc): Lock sweep mutex here instead of in callers; Calls to
scm_i_thread_put_to_sleep/scm_i_thread_wake_up used to demarkate
the single-thread section (which now only contains the mark
phase).
(scm_gc): Don't lock sweeo mutex here since scm_igc locks it;
Removed SCM_DEFER/ALLOW_INTS. Simply call scm_igc directly.
* threads.c (gc_section_mutex): Removed.
thread_admin_mutex in order not to risk being joined.
(scm_i_thread_put_to_sleep, scm_i_thread_wake_up): Keep
thread_admin_mutex locked during GC.
* pthread-threads.c, pthread-threads.h: Improvements to debugging
functions.
