* module/ice-9/psyntax.scm (syntax-violation): Use lambda* for subform
arg. Instead of using scm-error, just throw to 'syntax-error with all
we got.
* module/ice-9/psyntax-pp.scm (#{and-map*\ 37}): Regenerated.
* module/ice-9/psyntax.scm (lookup): Reflow comment.
(chi-top, syntax): Add comments about mod for lookup.
(set!): Lookup the identifier in the module attached to its syntax
object. In the (set! MACRO foo) case, after expanding the macro, chi
the resulting expression with the empty wrap, as syntax-type
does. Seems to fix the case where the expansion references
lexically-bound variables.
* module/ice-9/psyntax-pp.scm: Regenerated.
* test-suite/tests/syncase.test: Add a bunch of tests.
* module/ice-9/psyntax.scm (underscore?): New helper, like ellipsis?.
(syntax-case): Treat the _ pattern as a matches-all pattern,
disallowing it from the keyword list. Another well-thought part of the
R6RS.
($sc-dispatch): Dispatch _ patterns.
* module/ice-9/psyntax-pp.scm: Regenerated.
* test-suite/tests/syncase.test ("_ is a placeholder"): Add a test.
* module/ice-9/psyntax.scm (build-lexical-assignment)
(build-global-assignment): Maybe name the RHS.
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/ice-9/boot-9.scm (catch, with-throw-handler, throw): Rework to
use set! instead of define! so that we get names.
* module/ice-9/psyntax.scm (set!): Handle variable transformers; though,
they cannot produce definitions.
(make-variable-transformer): New procedure.
(identifier-syntax): Allow the R6RS form that makes variable
transformers.
* module/ice-9/psyntax-pp.scm: Regenerated.
* doc/ref/r6rs.texi (R6RS Incompatibilities): Remove letrec* item, and
add set! restriction.
* module/ice-9/psyntax.scm (chi-body): Whoops, actually render internal
definitions into a letrec* in the right order.
* module/ice-9/psyntax-pp.scm: Regenerate.
* test-suite/tests/syntax.test: Add some letrec* tests.
* module/ice-9/psyntax.scm (chi-body): Internal definitions are now
letrec* instead of letrec, following the R6RS.
* module/ice-9/psyntax-pp.scm: Regenerated.
* libguile/expand.c (expand_letrec_helper): Factor out common code.
(expand_letrec): Use expand_letrec_helper.
(expand_letrec_star): New primitive syntax: letrec*.
* libguile/memoize.c (memoize): Add memoizer support for in-order letrec
(letrec*).
* module/ice-9/psyntax.scm (build-letrec): Another arg, `in-order?'.
(chi-body): Adapt to build-letrec change. We don't yet use letrec* for
internal definitions.
(letrec): Adapt to build-letrec change.
(letrec*): New expander.
* module/ice-9/psyntax-pp.scm: Regenerated.
* libguile/expand.h (SCM_EXPANDED_LETREC_IN_ORDER_P)
(SCM_MAKE_EXPANDED_LETREC): Add a new field to letrec, in-order?. Will
be used to support letrec*.
* libguile/expand.c (LETREC, expand_named_let, expand_letrec): Adapt
code.
* module/language/elisp/compile-tree-il.scm (compile-pair):
* module/ice-9/psyntax.scm (build-named-let, build-letrec): Pass #f for
in-order? to `make-letrec'.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/language/tree-il.scm: Add letrec-in-order? accessor.
(parse-tree-il, unparse-tree-il): Parse and unparse an in-order?
letrec as `letrec*'.
(tree-il->scheme): Serialize letrec*.
* module/ice-9/psyntax.scm: Fix bug in which the whole syntax object was
being recorded as the name of the operator in a lexical call, instead
of just the symbolic name.
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/ice-9/psyntax.scm (chi-install-global, chi-macro)
(eval-local-transformer): Now that we handle module hygiene through
syntax objects, there is no more need to record the current module
when installing syntax transformers.
* module/ice-9/psyntax-pp.scm: Regenerated (trickily).
* libguile/_scm.h: Bump objcode version for macro representation change.
* libguile/macros.c (scm_macro_transformer): Adapt to change in macro
representation.
* module/ice-9/psyntax.scm (chi-macro): Instead of assuming that output
of a macro should be scoped relative to the module that was current
when the macro was defined, allow the module information associated
with the syntax object itself to pass through unmolested. Fixes bug
29860.
(datum->syntax): Propagate the module of the identifier through to the
new syntax object, so that datum->syntax preserves module hygiene in
addition to lexical hygiene.
(include, include-from-path): Refactor to plumb though the hygiene
information from the filename instead of the `include', allowing
hygiene from the original caller of include-from-path to propagate
through.
* module/ice-9/psyntax-pp.scm: Regenerated.
* test-suite/tests/syncase.test ("macro-generating macro"): Add test for
bug 29860.
* module/ice-9/psyntax.scm (syntax-type): Add some comments about source
locations. In calls to chi-macro, pass the source location
information. Propagate source location information when unpacking
syntax objects.
(chi-macro): Add a comment about source locations. Decorate output
introduced by the macro with the source location of macro use.
* module/ice-9/psyntax-pp.scm: Regenerate.
Fixes bug #29817.
* module/ice-9/psyntax.scm (define-expansion-accessors): New helper, to
define accesors for a particular expansion data structure. Use it
later to define lambda?, lambda-meta, and set-lambda-meta!.
(maybe-name-value): Update to work with the newly defined accessors.
(build-global-reference, build-let, build-named-let, build-letrec):
Re-enable naming of procedures.
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/ice-9/psyntax.scm: Remove the *mode* fluid. Whee :)
(macroexpand): No need to bind aa *mode* fluid.
* module/ice-9/psyntax-pp.scm: Regenerated.
* libguile/expand.c (expand_lambda_star_case): Harmonize with tree-il,
expanding keywords to (aok? (kw name gensym) ...), not
(aok? (kw . index) ...).
* libguile/memoize.c (memoize): Process the (kw name gensym) format into
(kw . index).
* module/ice-9/psyntax.scm (build-lambda-case): Remove a
compile-versus-eval special case.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/ice-9/eval.scm (primitive-eval):
* libguile/eval.c (scm_c_primitive_eval): Don't expect a memoized
expression -- expect either raw source or an *expanded* expression. We
handle memoization ourselves.
* libguile/expand.c (scm_macroexpand): Settle down into its proper name,
"macroexpand", even as we comment that it is but a fleeting boot
expander.
(scm_macroexpanded_p): New predicate for expanded code.
* libguile/expand.h: Add scm_macroexpanded_p.
* libguile/memoize.c (scm_memoize_expression): Require that the
expression be expanded.
(scm_init_memoize): Don't alias memoize-expression to macroexpand.
* module/ice-9/psyntax-pp.scm:
* module/ice-9/psyntax.scm: Always produce macroexpanded expressions,
and hand them to primitive-eval. No more calls to memoize-expression
here.
* test-suite/tests/optargs.test: Remove some tests, as unfortunately we
have no way to invoke the boot expander after boot.
* module/ice-9/psyntax.scm: Remove a number of eval-versus-compile
cases, instead producing "expanded" structures directly, which happen
to coincide with tree-il.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/ice-9/psyntax.scm (build-case-lambda, build-lambda-case):
Actually build case-lambda expressions, knowing that the memoizer will
do the right thing.
* module/ice-9/psyntax-pp.scm: Regenerated.
* libguile/eval.c (scm_c_primitive_eval):
* module/ice-9/eval.scm (primitive-eval): Rely on the expander to
produce a memoized expression. If the expression is already memoized,
just pass it through (the equivalent of the old "noexpand" hack).
* libguile/memoize.c (scm_init_memoize): Initialize `memoize-expression'
as the initial binding of `macroexpand'.
* libguile/modules.c (scm_module_transformer): Before modules are
booted, look for `macroexpand', not `%pre-modules-transformer'.
* module/ice-9/boot-9.scm: No more %pre-modules-transformer. Loading
psyntax extends `macroexpand'.
(make-module): `macroexpand' is the default transformer.
* module/ice-9/psyntax.scm: No more `noexpand'.
(top-level-eval-hook, local-eval-hook): Instead of annotating with
noexpand, memoize the expression before handing it to primitive-eval.
(macroexpand): No more noexpand hack -- in its place we have another
hack, to memoize the result when running in eval mode.
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/ice-9/psyntax.scm: Revert tracking of the current module while
expanding toplevel sequences. Better to preserve lexical scoping
(relative to the module of the original expression) and allow for @@
to override it.
* module/ice-9/psyntax-pp.scm: Regenerated.
* test-suite/tests/syncase.test: Test module-changing with @@.
* module/ice-9/psyntax.scm (@@): The RHS of an @@ is no longer
restricted to be an identifier, it can be an arbitrary expression.
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/ice-9/psyntax.scm (chi-when-list, chi-top): Add an undocumented
`expand' situation to eval-when, which will trigger at expand-time for
toplevel expressions. Useful when you are expanding for eval, but need
to do some side effect to affect the expander -- like defining a
module.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/ice-9/psyntax.scm (chi-top): Only modify the compile-time
environment to indicate that there is a local binding if we are
running in compile mode. Otherwise some eval-when things might not
have gotten executed, and our current-module could be wrong.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/ice-9/psyntax.scm (syntax-rules, identifier-syntax):
* module/ice-9/boot-9.scm (define-macro): Embed metadata into the macro
transformer for use by documentation tools and the like.
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/ice-9/psyntax.scm (chi-lambda-case, lambda): If we find a
vector of pairs in the docstring position, interpret those pairs as
additional metadata.
* module/ice-9/psytntax-pp.scm: Regenerated.
* module/ice-9/psyntax.scm (build-simple-lambda, build-case-lambda):
Don't take a docstring, take a metadata alist. Don't bother annotating
interpreted expressions with docstrings.
(chi-simple-lambda): Take already-parsed metadata.
(lambda): Parse out multiple docstrings as separate "documentation"
entries in the "meta".
(chi-lambda-case): Merge multiple "meta" entries among the various
cases.
(lambda*, case-lambda, case-lambda*): Receive "meta" clauses, not
docstrings.
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/ice-9/boot-9.scm (macroexpand): Rename from sc-expand.
(%pre-modules-transformer): Adapt to name change.
* module/ice-9/compile-psyntax.scm: Adapt to name change.
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/ice-9/psyntax.scm: Rename sc-expand to macroexpand.
* module/language/scheme/compile-tree-il.scm (compile-tree-il): Adapt to
name change.
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/ice-9/psyntax.scm (chi-top-sequence): Track changes to the
current module. Allows e.g. top-level `define-module' within a begin
to work.
* test-suite/tests/syncase.test ("changes to expansion environment"):
Enable test.
* module/ice-9/psyntax.scm (build-dynlet, with-fluids): Use psyntax to
recognize `with-fluids' as a core form, producing <dynlet> if we are
compiling.
(sc-expand): To spice up the mix, use with-fluids here in the
implementation.
* module/ice-9/psyntax-pp.scm: Bootstrapped twice (!).
* module/ice-9/boot-9.scm: Remove with-fluids definition, it's in the
core now.
There are some incompatible changes here, but only to interfaces that
were introduced earlier in 1.9, or interfaces which have been broken
since early in 1.9.
* libguile/_scm.h (SCM_OBJCODE_MINOR_VERSION): Bump, as the macro
changes affect the interface that is called by psyntax-generated macro
definitions.
* libguile/inline.h (scm_words): New function, allocates a variable
number of contiguous scm_t_bits locations, with a given value in the
0th word, and 0 in the rest of the words.
* libguile/macros.h: Rework interface to correspond more closely, and
minimally, to the needs of memoize.c and psyntax.
(SCM_ASSYNT, SCM_MACRO_TYPE_BITS, SCM_MACRO_TYPE_MASK)
(SCM_F_MACRO_EXTENDED, SCM_MACROP, SCM_MACRO_TYPE)
(SCM_MACRO_IS_EXTENDED, SCM_BUILTIN_MACRO_P, SCM_SYNCASE_MACRO_P)
(SCM_MACRO_CODE, scm_tc16_macro): Remove CPP macros related to the
representation of Scheme macros.
(scm_i_make_primitive_macro): Renamed from scm_i_makbimacro.
(scm_i_macro_primitive): New accessor so that memoize.c can get to the
primitive syntax transformer.
(scm_make_syncase_macro, scm_make_extended_syncase_macro)
(scm_syncase_macro_type, scm_syncase_macro_binding): Removed these
functions, replaced by make-syntax-transformer and its accessors.
(scm_macro_binding): New accessor, the same as what
scm_syncase_macro_binding was.
* libguile/macros.c: All representation details of syntax transformers
are private to this file now.
(macro_print): Print macros as #<syntax-transformer ...>, or
#<primitive-syntax-transformer ...> if psyntax has not attached a
transformer of its own.
(scm_i_make_primitive_macro): Represent macros as 5-word smobs.
(scm_make_syntax_transformer): New constructor for syntax transformers
(macros), exported to scheme. Takes a name, and looks it up in the
current module to determine the previous primitive transformer, if
any.
(scm_macro_type): Instead of returning 'builtin-macro!, etc, return
the type as set by psyntax, or #f if it's a primitive.
(scm_macro_name): Return the stored macro name.
(scm_macro_transformer): Return the psyntax-set syntax transformer.
Hacky, but should help introspection somewhat.
* libguile/memoize.c (memoize_env_ref_transformer): Use the new
scm_i_macro_primitive, and adapt to other macro API changes.
* module/ice-9/psyntax.scm (put-global-definition-hook)
(get-global-definition-hook, chi-install-global): Call (and generate
calls to) the new macro constructors and accessors.
* module/ice-9/psyntax-pp.scm: Doubly regenerated.
* module/ice-9/debugging/traps.scm (trap-here): Comment out this
definition and export, while it's not working.
* libguile/vm-engine.c (VM_NAME): Keyword arg errors are now thrown to
'keyword-argument-error.
* libguile/vm.c: Define sym_keyword_argument_error, and statically
allocate some other symbols.
* module/ice-9/optargs.scm (parse-lambda-case): Throw to
'keyword-argument-error in kwarg error cases.
* module/ice-9/psyntax.scm (build-lambda-case): Remove a couple
workarounds for the old memoizer. Throw to 'wrong-number-of-args if
the lambda-case fails to parse.
* module/ice-9/psyntax-pp.scm: Regenerated.
* test-suite/tests/optargs.test: Update expected exceptions.
* module/ice-9/psyntax.scm (gen-label, new-mark): Labels and marks need
to be unique across read/write boundaries -- it's not sufficient for
them to simply be unique within a process by virtue of (string #\i)
constructing a new object. This used to mostly work before, because
the collapsing of duplicate constants didn't catch many syntax-object
cases -- but for some reason the attached test case brings out the
problem. So switch to use gensyms. Potentially more costly, but it's
what upstream psyntax does now.
This bug took me fully two days to figure out.
* module/ice-9/psyntax-pp.scm: Regenerated.
* test-suite/tests/syncase.test ("serializable labels and marks"): Add
test case.
I've prepared a patch that adds partial support for tail patterns.
Things like the the SRFI-34 `guard' macro from [0] are supported, but
you still can't combine dotted patterns with tail patterns, e.g.
(syntax-rules (else)
((foo bar ... (else something) . rest)
<TEMPLATE-HERE>))
will *not* work; there's the issue that one can't just transcribe
the implementation of this feature from the latest version of psyntax,
as I've done for non-dotted tail patterns, as it's implemented using a
dotted pattern like the above. Alas!
[0] <http://article.gmane.org/gmane.lisp.guile.devel/9442>
* module/ice-9/psyntax.scm (syntax-case, $sc-dispatch): Add support for
tail patterns, transcribed from the latest psyntax.
* module/ice-9/psyntax-pp.scm: Regenerated.
* test-suite/tests/syncase.test: Add tests for tail patterns.
* libguile/eval.c (scm_primitive_eval, scm_c_primitive_eval):
(scm_init_eval): Rework so that scm_primitive_eval always calls out to
the primitive-eval variable. The previous definition is the default
value, which is probably overridden by scm_init_eval_in_scheme.
* libguile/init.c (scm_i_init_guile): Move ports and load-path up, so we
can debug when initing eval. Call scm_init_eval_in_scheme. Awesome.
* libguile/load.h:
* libguile/load.c (scm_init_eval_in_scheme): New procedure, loads up
ice-9/eval.scm to replace the primitive-eval definition, if everything
is there and up-to-date.
* libguile/modules.c (scm_module_transformer): Export to Scheme, so it's
there for eval.go.
* module/ice-9/boot-9.scm: No need to define module-transformer.
* module/ice-9/eval.scm (capture-env): Only reference the-root-module if
modules are booted.
(primitive-eval): Inline a definition for identity. Throw a more
standard error for "wrong number of arguments".
* module/ice-9/psyntax.scm (chi-install-global): The macro binding for a
syncase macro is now a pair: the transformer, and the module that was
current when the transformer was installed. The latter is used for
hygiene purposes, replacing the use of procedure-module, which didn't
work with the interpreter's shared-code closures.
(chi-macro): Adapt for the binding being a pair, and get the hygiene
from the cdr.
(eval-local-transformer): Adapt to new form of macro bindings.
* module/ice-9/psyntax-pp.scm: Regenerated.
* .gitignore: Ignore eval.go.stamp.
* module/Makefile.am: Reorder for fastest serial compilation, now that
there are no ordering constraints. I did a number of experiments here
and this seems to be the best; but the bulk of the time is compiling
psyntax-pp.scm with eval.scm. Not so great.
* libguile/vm-engine.c (vm-engine): Throw a more standard error for
"wrong type to apply".
* test-suite/tests/gc.test ("gc"): Remove a hack that shouldn't affect
the new evaluator, and throw in another (gc) for good measure.
* test-suite/tests/goops.test ("defining classes"):
* test-suite/tests/hooks.test (proc1): We can't currently check what the
arity is of a closure made by eval.scm -- or more accurately all
closures have 0 required args and no rest args. So punt for now.
* test-suite/tests/syntax.test ("letrec"): The scheme evaluator can't
check that a variable is unbound, currently; perhaps the full "fixing
letrec" expansion could fix this. But barring that, punt.
* libguile/eval.c: So, ladies & gents, a new evaluator. It's similar to
the old one, in that we memoize and then evaluate, but in this
incarnation, memoization of an expression happens before evaluation,
not lazily as the expression is evaluated. This makes the evaluation
itself much cleaner, in addition to being threadsafe. In addition,
since this C evaluator will in the future just serve to bootstrap the
Scheme evaluator, we don't have to pay much concern for debugging
conveniences. So the environment is just a list of values, and the
memoizer pre-computes where it's going to find each individual value
in the environment.
Interface changes are commented below, with eval.h.
(scm_evaluator_traps): No need to reset the debug mode after rnning te
traps thing. But really, the whole traps system needs some love.
* libguile/memoize.h:
* libguile/memoize.c: New memoizer, which runs before evaluation,
checking all syntax before evaluation begins. Significantly, no
debugging information is left for lexical variables, which is not so
great for interactive debugging; perhaps we should change this to have
a var list in the future as per the classic interpreters. But it's
quite fast, and the resulting code is quite good. Also note that it
doesn't produce ilocs, memoized code is a smob whose type is in the
first word of the smob itself.
* libguile/eval.h (scm_sym_and, scm_sym_begin, scm_sym_case)
(scm_sym_cond, scm_sym_define, scm_sym_do, scm_sym_if, scm_sym_lambda)
(scm_sym_let, scm_sym_letstar, scm_sym_letrec, scm_sym_quote)
(scm_sym_quasiquote, scm_sym_unquote, scm_sym_uq_splicing, scm_sym_at)
(scm_sym_atat, scm_sym_atapply, scm_sym_atcall_cc)
(scm_sym_at_call_with_values, scm_sym_delay, scm_sym_eval_when)
(scm_sym_arrow, scm_sym_else, scm_sym_apply, scm_sym_set_x)
(scm_sym_args): Remove public declaration of these symbols.
(scm_ilookup, scm_lookupcar, scm_eval_car, scm_eval_body)
(scm_eval_args, scm_i_eval_x, scm_i_eval): Remove public declaration
of these functions.
(scm_ceval, scm_deval, scm_ceval_ptr): Remove declarations of these
deprecated functions.
(scm_i_print_iloc, scm_i_print_isym, scm_i_unmemocopy_expr)
(scm_i_unmemocopy_body): Remove declarations of these internal
functions.
(scm_primitive_eval_x, scm_eval_x): Redefine as macros for their less
destructive siblings.
* libguile/Makefile.am: Add memoize.[ch] to the build.
* libguile/debug.h (scm_debug_mode_p, scm_check_entry_p)
(scm_check_apply_p, scm_check_exit_p, scm_check_memoize_p)
(scm_debug_eframe_size): Remove these vars that were tied to the old
evaluator's execution model.
(SCM_RESET_DEBUG_MODE): Remove, no more need for this.
(SCM_MEMOIZEDP, SCM_MEMOIZED_EXP, SCM_MEMOIZED_ENV): Remove macros
referring to old memoized code representation.
(scm_local_eval, scm_procedure_environment, scm_memoized_environment)
(scm_make_memoized, scm_memoized_p): Remove functions operating on old
memoized code representation.
(scm_memcons, scm_mem_to_proc, scm_proc_to_mem): Remove debug-only
code for old evaluator.
* libguile/debug.c: Remove code to correspond with debug.h removals.
(scm_debug_options): No need to set the debug mode or frame limit
here, as we don't have C stack limits any more. Perhaps this is a bug,
but as long as we can compile eval.scm, we should be fine.
* libguile/init.c (scm_i_init_guile): Init memoize.c.
* libguile/modules.c (scm_top_level_env, scm_env_top_level)
(scm_env_module, scm_system_module_env_p): Remove these functions.
* libguile/print.c (iprin1): No more need to handle isyms. Adapt to new
form of interpreted procedures.
* libguile/procprop.c (scm_i_procedure_arity): Adapt to new form of
interpreted procedures.
* libguile/procs.c (scm_thunk_p): Adapt to new form of interpreted
procedures.
* libguile/procs.h (SCM_CLOSURE_FORMALS): Removed, this exists no more.
(SCM_CLOSURE_NUM_REQUIRED_ARGS, SCM_CLOSURE_HAS_REST_ARGS): New
accessors.
* libguile/srcprop.c (scm_source_properties, scm_source_property)
(scm_set_source_property_x): Remove special cases for memoized code.
* libguile/stacks.c (read_frame): Remove a source-property case for
interpreted code.
(NEXT_FRAME): Remove a case that I don't fully understand, that seems
to be designed to skip over apply frames. Will be obsolete in the
futures.
(read_frames): Default source value for interpreted frames to #f.
(narrow_stack): Don't pay attention to the system_module thing.
* libguile/tags.h: Remove isyms and ilocs. Whee!
* libguile/validate.h (SCM_VALIDATE_MEMOIZED): Fix to use the new
MEMOIZED_P formulation.
* module/ice-9/psyntax-pp.scm (do, quasiquote, case): Adapt for these no
longer being primitive macros.
* module/ice-9/boot-9.scm: Whitespace change, but just a poke to force a
rebuild due to and/or/cond/... not being primitives any more.
* module/ice-9/deprecated.scm (unmemoize-expr): Deprecate, it's
unmemoize-expression now.
* test-suite/tests/eval.test ("define set procedure-name"): XFAIL a
couple of tests here; I don't know what to do about them. I reckon the
expander should ensure that defined values are named.
* test-suite/tests/chars.test ("basic char handling"): Fix expected
exception when trying to apply a char.
