* Specific problems in IA64 make check
** test-unwind
Representation of the relevant dynamic context:
non-rewindable
catch frame make cont.
o----o-----a----------b-------------c
\
\ call cont.
o-----o-----------d
A continuation is captured at (c), with a non-rewindable frame in the
dynamic context at (b). If a rewind through that frame was attempted,
Guile would throw to the catch at (a). Then the context unwinds back
past (a), then winds forwards again, and the captured continuation is
called at (d).
We should end up at the catch at (a). On ia64, we get an "illegal
instruction".
The problem is that Guile does not restore the ia64 register backing
store (RBS) stack (which is saved off when the continuation is
captured) until all the unwinding and rewinding is done. Therefore,
when the rewind code (scm_i_dowinds) hits the non-rewindable frame at
(b), the RBS stack hasn't yet been restored. The throw finds the
jmp_buf (for the catch at (a)) correctly from the dynamic context, and
jumps back to (a), but the RBS stack is invalid, hence the illegal
instruction.
This could be fixed by restoring the RBS stack earlier, at the same
point (copy_stack) where the normal stack is restored. But that
causes a problem in the next test...
** continuations.test
The dynamic context diagram for this case is similar:
non-rewindable
catch frame make cont.
a----x-----o----------b-------------c
\
\ call cont.
o-------d
The only significant difference is that the catch point (a) is
upstream of where the dynamic context forks. This means that the RBS
stack at (d) already contains the correct RBS contents for throwing
back to (a), so it doesn't matter whether the RBS stack that was saved
off with the continuation gets restored.
This test passes with the Guile 1.8.4 code, but fails (with an
"illegal instruction") when the code is changed to restore the RBS
stack earlier as described above.
The problem now is that the RBS stack is being restored _too_ early;
specifically when there is still stuff to do that relies on the old
RBS contents. When a continuation is called, the sequence of relevant
events is:
(1) Grow the (normal) stack until it is bigger than the (normal)
stack saved off in the continuation. (scm_dynthrow, grow_stack)
(2) scm_i_dowinds calls itself recursively, such that
(2.1) for each rewind (from (x) to (c)) that will be needed,
another frame is added to the stack (both normal and RBS),
with local variables specifying the required rewind; the
rewinds don't actually happen yet, they will happen when
the stack unwinds again through these frames
(2.2) required unwinds - back from where the continuation was
called (d) to the fork point (x) - are done immediately.
(3) The normal (i.e. non-RBS) stack that was stored in the
continuation is restored (i.e. copied on top of the actual
stack).
Note that this doesn't overwrite the frames that were added in
(2.1), because the growth in (1) ensures that the added frames
are beyond the end of the restored stack.
(4) ? Restore the RBS stack here too ?
(5) Return (from copy_stack) through the (2.1) frames, which means
that the rewinds now happen.
(6) setcontext (or longjmp) to the context (c) where the
continuation was captured.
The trouble is that step (1) does not create space in the RBS stack in
the same kind of way that it does for the normal stack. Therefore, if
the saved (in the continuation) RBS stack is big enough, it can
overwrite the RBS of the (2.1) frames that still need to complete.
This causes an illegal instruction when we return through those frames
and try to perform the rewinds.
* Fix
The key to the fix is that the saved RBS stack only needs to be
restored at some point before the next setcontext call, and that doing
it as close to the setcontext call as possible will avoid bad
interactions with the pre-setcontext stack. Therefore we do the
restoration at the last possible point, immediately before the next
setcontext call.
The situation is complicated by there being two ways that the next
setcontext call can happen.
- If the unwinding and rewinding is all successful, the next
setcontext will be the one from step (6) above. This is the
"normal" continuation invocation case.
- If one of the rewinds throws an error, the next setcontext will
come from the throw implementation code. (And the one in step (6)
will never happen.) This is the rewind error case.
In the rewind error case, the code calling setcontext knows nothing
about the continuation. So to cover both cases, we:
- copy (in step (4) above) the address and length of the
continuation's saved RBS stack to the current thread state
(SCM_I_CURRENT_THREAD)
- modify all setcontext callers so that they check the current
thread state for a saved RBS stack, and restore it if so before
calling setcontext.
* Notes
** I think rewinders cannot rely on using any stack data
Unless it can be guaranteed that the data won't go into a register.
I'm not 100% sure about this, but I think it follows from the fact
that the RBS stack is not restored until after the rewinds have
happened.
Note that this isn't a regression caused by the current fix. In Guile
1.8.4, the RBS stack was restored _after_ the rewinds, and this is
still the case now.
** Most setcontext calls for `throw' don't need to change the RBS stack
In the absence of continuation invocation, the setcontext call in the
throw implementation code always sets context to a place higher up the
same stack (both normal and RBS), hence no stack restoration is
needed.
* Other changes
** Using setcontext for all non-local jumps (for __ia64__)
Along the way, I read a claim somewhere that setcontext was more
reliable than longjmp, in cases where the stack has been manipulated.
I don't now have any reason to believe this, but it seems reasonable
anyway to leave the __ia64__ code using getcontext/setcontext, instead
of setjmp/longjmp.
(I think the only possible argument against this would be performance -
if getcontext was significantly slower than setjmp. It that proves to
be the case, we should revisit this.)
** Capping RBS base for non-main threads
Somewhere else along the way, I hit a problem in GC, involving the RBS
stack of a non-main thread. The problem was, in
SCM_MARK_BACKING_STORE, that scm_ia64_register_backing_store_base was
returning a value that was massively greater than the value of
scm_ia64_ar_bsp, leading to a seg fault. This is because the
implementation of scm_ia64_register_backing_store_base is only valid
for the main thread. I couldn't find a neat way of getting the true
RBS base of a non-main thread, but one idea is simply to call
scm_ia64_ar_bsp when guilifying a thread, and use the value returned
as an upper bound for that thread's RBS base. (Note that the RBS
stack grows upwards.)
(Were it not for scm_init_guile, we could be much more definitive
about this. We could take the value of scm_ia64_ar_bsp as a
definitive base address for the part of the RBS stack that Guile cares
about. We could also then discard
scm_ia64_register_backing_store_base.)
scm_set_thread_cleanup_x, scm_thread_cleanup): Lock on thread-specific
admin mutex instead of `thread_admin_mutex'.
* threads.h (scm_i_thread)[admin_mutex]: New field.
* throw.c (make_jmpbuf): Don't enter critical section during thread
spawn -- there is a possibility of deadlock if other threads are
exiting.
(scm_i_thread): Removed unused signal_asyncs field.
(threads_mark): Do not mark it.
(guilify_self_1): Do not initialize it. Do initialize
continuation_root field.
(do_thread_exit): Do not remove thread from all_threads list.
(on_thread_exit): Do it here, after leaving guile mode.
(sleep_level): Removed.
(scm_i_thread_put_to_sleep): Leave thread_admin_mutex locked when
returning. Do not support recursive sleeps.
(scm_i_thread_wake_up): Expect thread_admin_mutex to be locked on
entry. Do not support recursive sleeps.
SCM_CRITICAL_SECTION_END): Moved here from threads.h since now
they also block/unblock execution of asyncs and call
scm_async_click which is declared in async.h but threads.h can not
include async.h since async.h already includes threads.h.
(scm_i_critical_section_level): New, for checking mistakes in the
use of the SCM_CRITICAL_SECTION_* macros.
(scm_i_critical_section_mutex): Make it a recursive mutex so that
critical sections can be nested.
* threads.h, threads.c (scm_frame_lock_mutex): New.
(scm_frame_critical_section): Take mutex as argument.
(framed_critical_section_mutex): New, used as default for above.
(scm_init_threads): Initialize it.
(scm_threads_prehistory): Do not initialize thread_admin_mutex and
scm_i_critical_section_mutex; both are initialized statically.
* configure.in: Checking for __int64 as possible candidate for
the SCM_I_GSC_T_INT64 define.
2003-06-14 Stefan Jahn <stefan@lkcc.org>
* threads.h: Redefined scm_getspecific() and scm_setspecific()
to be functions instead of macros.
* threads.c: Conditionalized inclusion of <sys/time.h> and
<unistd.h>.
(scm_getspecific, scm_setspecific): Made these two function
real part of the API.
* posix.c (s_scm_putenv): Added some code to make a
(putenv "FOO="), i.e. setting an empty string, work also on
Win32 systems. Thanks to Kevin Ryde for the proposal.
* configure.in: Removed -lm check and added a cached check for
__libc_stack_end to get it building for mingw32 hosts.
2003-05-29 Stefan Jahn <stefan@lkcc.org>
* win32-dirent.c: Use malloc() instead of scm_malloc().
* stime.c (s_scm_strftime): Add a type cast to avoid compiler
warning.
* posix.c (s_scm_putenv): Disable use of unsetenv() for the
mingw32 build.
* modules.c (s_scm_module_import_interface): Renamed local
variable interface to _interface. Seems like 'interface'
is a special compiler directive for the mingw32 compiler.
* mkstemp.c: Provide prototype to avoid compiler warning.
* load.c (s_scm_search_path): Fixed absolute and relative
path detections for native Windows platforms.
* gc.h, threads.h: Export some more symbols using SCM_API
(necessary to build on mingw32).
* gc-freelist.c ("s_scm_map_free_list",
"s_scm_gc_set_debug_check_freelist_x"): Fixed use of FUNC_NAME.
* fports.c (fport_fill_input): Disable use of
fport_wait_for_input() on Win32 platforms.
* filesys.c (s_scm_basename): Fixed __MINGW32__ code.
* Makefile.am: Modified some rules for cross compiling.
2003-05-29 Stefan Jahn <stefan@lkcc.org>
* raw-ltdl.c: Some more modifications for mingw32 platforms.
2003-05-29 Stefan Jahn <stefan@lkcc.org>
* Makefile.am (libguile_srfi_srfi_1_la_LDFLAGS,
libguile_srfi_srfi_4_la_LDFLAGS,
libguile_srfi_srfi_13_14__la_LDFLAGS): Added the -no-undefined
option for the mingw32 build.
2003-05-29 Stefan Jahn <stefan@lkcc.org>
* standalone/Makefile.am: Setup to build on mingw32.
scm_t_timespec. Replace usage of USE_PTHREAD_THREADS with
SCM_USE_PTHREAD_THREADS. Remove typedef for struct timespec in
favor of scm_t_timespec from scmconfig.h.
Simply lock a thread C API recursive mutex.
(SCM_NONREC_CRITICAL_SECTION_START,
SCM_NONREC_CRITICAL_SECTION_END, SCM_REC_CRITICAL_SECTION_START,
SCM_REC_CRITICAL_SECTION_END): Removed.
* eval.c: Replaced SOURCE_SECTION_START / SOURCE_SECTION_END with
direct calls to scm_rec_mutex_lock / unlock around the three calls
to scm_m_expand_body.
* eval.c, eval.h (promise_free): New function.
(scm_force): Rewritten; Now thread-safe; Removed
SCM_DEFER/ALLOW_INTS.
* pthread-threads.h: Added partially implemented plugin interface
for recursive mutexes. These are, for now, only intended to be
used internally within the Guile implementation.
* pthread-threads.c: New file.
* threads.c: Conditionally #include "pthread-threads.c".
* eval.c, eval.h (scm_makprom, scm_force): Rewritten to be
thread-safe;
* snarf.h (SCM_MUTEX, SCM_GLOBAL_MUTEX, SCM_REC_MUTEX,
SCM_GLOBAL_REC_MUTEX): New macros.
* eval.c, threads.c, threads.h, snarf.h: Rewrote critical section
macros---use mutexes instead.
* tags.h (SCM_IM_FUTURE): New tag.
* eval.c (scm_m_future): New primitive macro.
(SCM_CEVAL): Support futures.
(unmemocopy): Support unmemoization of futures.
* print.c (scm_isymnames): Name of future isym.
(SCM_NONREC_CRITICAL_SECTION_START,
SCM_NONREC_CRITICAL_SECTION_END, SCM_REC_CRITICAL_SECTION_START,
SCM_REC_CRITICAL_SECTION_END): New macros.
(SCM_CRITICAL_SECTION_START/END): Defined here.
* eval.c: Insert SOURCE_SECTION_START / SOURCE_SECTION_END around
the three calls to scm_m_expand_body.
* gc.h: #include "libguile/pthread-threads.h";
(SCM_FREELIST_CREATE, SCM_FREELIST_LOC): New macros.
* gc.c (scm_i_freelist, scm_i_freelist2): Defined to be of type
scm_t_key;
* gc.c, gc-freelist.c, inline.h: Use SCM_FREELIST_LOC for freelist
access.
* gc-freelist.c (scm_gc_init_freelist): Create freelist keys.
* gc-freelist.c, threads.c (really_launch): Use
SCM_FREELIST_CREATE.
* gc-malloc.c (scm_realloc, scm_gc_register_collectable_memory):
* gc.c (scm_i_expensive_validation_check, scm_gc,
scm_gc_for_newcell): Put threads to sleep before doing GC-related
heap administration so that those pieces of code are executed
single-threaded. We might consider rewriting these code sections
in terms of a "call_gc_code_singly_threaded" construct instead of
calling the pair of scm_i_thread_put_to_sleep () and
scm_i_thread_wake_up (). Also, we would want to have as many of
these sections eleminated.
* init.c (scm_init_guile_1): Call scm_threads_prehistory.
* inline.h: #include "libguile/threads.h"
* pthread-threads.h: Macros now conform more closely to the
pthreads interface. Some of them now take a second argument.
* threads.c, threads.h: Many changes.
* configure.in: Temporarily replaced "copt" threads option with new
option "pthreads".
(USE_PTHREAD_THREADS): Define if pthreads configured.
"libguile/pthread-threads.h" for USE_COPT_THREADS. Removed
(previously deprecated) C level thread API prototypes. They are
now in the thread package specific headers, "null-threads.h" and
"pthread-threads.h".
(SCM_VALIDATE_THREAD, SCM_VALIDATE_MUTEX, SCM_VALIDATE_CONDVAR):
New.
(scm_threads_init): Removed.
(SCM_CRITICAL_SECTION_START, SCM_CRITICAL_SECTION_END,
SCM_THREAD_SWITCHING_CODE, scm_i_switch_counter,
SCM_I_THREAD_SWITCH_COUNT): Define here.
(scm_single_thread_p): Removed.
(scm_call_with_new_thread): Take two args directly instead of list
of two args.
(scm_i_thread_data, scm_i_set_thread_data, SCM_THREAD_LOCAL_DATA,
SCM_SET_THREAD_LOCAL_DATA): Define here.
* threads.c: Merged with "coop-pthreads.c".
(scm_threads_make_mutex, scm_threads_lock_mutex,
scm_threads_unlock_mutex, scm_threads_monitor): Removed, they were
not implemented anyway.
(scm_init_thread_procs, scm_try_mutex,
scm_timed_condition_variable_wait,
scm_broadcast_condition_variable, scm_c_thread_exited_p,
scm_thread_exited_p): New prototypes.
(struct timespec): Define if not already defined.
(scm_t_mutex, scm_mutex_init, scm_mutex_lock, scm_mutex_trylock,
scm_mutex_unlock, scm_mutex_destroy, scm_t_cond, scm_cond_init,
scm_cond_wait, scm_cond_timedwait, scm_cond_signal,
scm_cond_broadcast, scm_cond_destroy): Declarations moved here and
deprecated.
* threads.c: Include <errno.h>. Include "coop-pthreads.c" when
requested.
(scm_thread_exited_p): New.
(scm_try_mutex, scm_broadcast_condition_variable): Newly
registered procedures.
(scm_wait_condition_variable, scm_timed_wait_condition_variable):
Use the latter as the procedure for "wait-condition-variable",
thus offering a optional timeout parameter to Scheme.
(scm_wait_condition_variable): Implement in terms of
scm_timed_wait_condition_variable.
(scm_mutex_init, scm_mutex_lock, scm_mutex_trylock,
scm_mutex_unlock, scm_mutex_destroy, scm_cond_init,
scm_cond_wait, scm_cond_timedwait, scm_cond_signal,
scm_cond_broadcast, scm_cond_destroy): Implement in terms of
scm_make_mutex, etc, and deprecate.
(scm_init_threads): Do not create smobs, leave this to
scm_threads_init. Do not include "threads.x" file.
(scm_init_thread_procs): New, include "threads.x" here.
* null-threads.h (scm_null_mutex, scm_null_mutex_init,
scm_null_mutex_lock, scm_null_mutex_unlock,
scm_null_mutex_destroy, scm_null_condvar, scm_null_condvar_init,
scm_null_condvar_wait, scm_null_condvar_signal,
scm_null_condvar_destroy): Removed.
(scm_mutex_init, scm_mutex_lock, scm_mutex_unlock, scm_cond_init,
scm_cond_wait, scm_cond_signal, scm_cond_broadcast,
scm_cond_destory): Do not define, they are now deprecated and
handled by threads.{h,c}.
* null-threads.c (scm_null_mutex, scm_null_cond): Define here.
(scm_threads_init): Create smobs here, using the appropriate
sizes.
(block): Removed, now unused.
(scm_c_thread_exited_p): New.
(scm_null_mutex_init, scm_null_mutex_lock, scm_null_mutex_unlock,
scm_null_mutex_destroy, scm_null_condvar_init,
scm_null_condvar_wait, scm_null_condvar_signal,
scm_null_condvar_destroy): Removed and updated users to do their
task directly.
(scm_try_mutex, timeval_subtract,
scm_timed_wait_condition_variable,
scm_broadcast_condition_variable): New.
(scm_wait_condition_variable): Removed.
* coop-threads.c (scm_threads_init): Create smobs here, using the
appropriate sizes.
(scm_c_thread_exited_p, scm_try_mutex,
scm_timed_wait_condition_variable,
scm_broadcast_condition_variable): New.
(scm_wait_condition_variable): Removed.
* threads.c: Include null-threads.c when !USE_COOP_THREADS.
(scm_init_threads): Use generic type names scm_t_mutex and
scm_t_coop instead of coop_m and coop_t.
added append docs from R4RS.
* strings.c: Docstring typo fix, + eliminate unneeded IMP tests.
Thanks Dirk Hermann!
* chars.h: Provide SCM_CHARP, SCM_CHAR, SCM_MAKE_CHAR and
deprecate SCM_ICHRP, SCM_ICHR, SCM_MAKICHR. Thanks Dirk Hermann!
* *.h, *.c: Use SCM_CHARP, SCM_CHAR, SCM_MAKE_CHAR throughout.
Drop use of SCM_P for function prototypes... assume an ANSI C
compiler. Thanks Dirk Hermann!
* scmsigs.c (usleep): Clean up oddities declaring usleep; since
we're just using it, not redefining it, we can use a K&R style
declaration here.
(sleep): Declare this, too, if the system hasn't.
(scm_sleep, scm_usleep): Use scm_thread_sleep and
scm_uthread_sleep if they're available; otherwise, just call the
system functions.
* scmconfig.h.in: Regenerated.
* threads.h (scm_single_thread_p, scm_yield,
scm_call_with_new_thread, scm_join_thread, scm_make_mutex,
scm_lock_mutex, scm_unlock_mutex, scm_make_condition_variable,
scm_wait_condition_variable, scm_signal_condition_variable): Add
prototypes for these Scheme-visible functions.
* coop-defs.h (coop_next_runnable_thread,
coop_wait_for_runnable_thread_now, coop_wait_for_runnable_thread):
Prototypes for these here, even though they're from iselect.c.
(coop_condition_variable_wait, coop_join): Add prototypes.
* coop-threads.c (scm_threads_free_thread, scm_threads_free_mutex,
scm_threads_free_condvar): Make these smob functions static.
* coop-threads.h (coop_init): Give this a real prototype.
* coop.c: #include <unistd.h>, if we have it, for `usleep' and `sleep'.
(coop_next_runnable_thread): No need to provide prototype; it's in
coop-defs.h.
