* libguile/expand.c (expand_lambda_star_case): Harmonize with tree-il,
expanding keywords to (aok? (kw name gensym) ...), not
(aok? (kw . index) ...).
* libguile/memoize.c (memoize): Process the (kw name gensym) format into
(kw . index).
* module/ice-9/psyntax.scm (build-lambda-case): Remove a
compile-versus-eval special case.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/ice-9/eval.scm (primitive-eval):
* libguile/eval.c (scm_c_primitive_eval): Don't expect a memoized
expression -- expect either raw source or an *expanded* expression. We
handle memoization ourselves.
* libguile/expand.c (scm_macroexpand): Settle down into its proper name,
"macroexpand", even as we comment that it is but a fleeting boot
expander.
(scm_macroexpanded_p): New predicate for expanded code.
* libguile/expand.h: Add scm_macroexpanded_p.
* libguile/memoize.c (scm_memoize_expression): Require that the
expression be expanded.
(scm_init_memoize): Don't alias memoize-expression to macroexpand.
* module/ice-9/psyntax-pp.scm:
* module/ice-9/psyntax.scm: Always produce macroexpanded expressions,
and hand them to primitive-eval. No more calls to memoize-expression
here.
* test-suite/tests/optargs.test: Remove some tests, as unfortunately we
have no way to invoke the boot expander after boot.
* module/ice-9/psyntax.scm: Remove a number of eval-versus-compile
cases, instead producing "expanded" structures directly, which happen
to coincide with tree-il.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/ice-9/psyntax.scm (build-case-lambda, build-lambda-case):
Actually build case-lambda expressions, knowing that the memoizer will
do the right thing.
* module/ice-9/psyntax-pp.scm: Regenerated.
* libguile/eval.c (scm_c_primitive_eval):
* module/ice-9/eval.scm (primitive-eval): Rely on the expander to
produce a memoized expression. If the expression is already memoized,
just pass it through (the equivalent of the old "noexpand" hack).
* libguile/memoize.c (scm_init_memoize): Initialize `memoize-expression'
as the initial binding of `macroexpand'.
* libguile/modules.c (scm_module_transformer): Before modules are
booted, look for `macroexpand', not `%pre-modules-transformer'.
* module/ice-9/boot-9.scm: No more %pre-modules-transformer. Loading
psyntax extends `macroexpand'.
(make-module): `macroexpand' is the default transformer.
* module/ice-9/psyntax.scm: No more `noexpand'.
(top-level-eval-hook, local-eval-hook): Instead of annotating with
noexpand, memoize the expression before handing it to primitive-eval.
(macroexpand): No more noexpand hack -- in its place we have another
hack, to memoize the result when running in eval mode.
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/ice-9/psyntax.scm: Revert tracking of the current module while
expanding toplevel sequences. Better to preserve lexical scoping
(relative to the module of the original expression) and allow for @@
to override it.
* module/ice-9/psyntax-pp.scm: Regenerated.
* test-suite/tests/syncase.test: Test module-changing with @@.
* module/ice-9/psyntax.scm (@@): The RHS of an @@ is no longer
restricted to be an identifier, it can be an arbitrary expression.
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/ice-9/psyntax.scm (chi-when-list, chi-top): Add an undocumented
`expand' situation to eval-when, which will trigger at expand-time for
toplevel expressions. Useful when you are expanding for eval, but need
to do some side effect to affect the expander -- like defining a
module.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/ice-9/psyntax.scm (chi-top): Only modify the compile-time
environment to indicate that there is a local binding if we are
running in compile mode. Otherwise some eval-when things might not
have gotten executed, and our current-module could be wrong.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/ice-9/psyntax.scm (syntax-rules, identifier-syntax):
* module/ice-9/boot-9.scm (define-macro): Embed metadata into the macro
transformer for use by documentation tools and the like.
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/ice-9/psyntax.scm (chi-lambda-case, lambda): If we find a
vector of pairs in the docstring position, interpret those pairs as
additional metadata.
* module/ice-9/psytntax-pp.scm: Regenerated.
* module/ice-9/psyntax.scm (build-simple-lambda, build-case-lambda):
Don't take a docstring, take a metadata alist. Don't bother annotating
interpreted expressions with docstrings.
(chi-simple-lambda): Take already-parsed metadata.
(lambda): Parse out multiple docstrings as separate "documentation"
entries in the "meta".
(chi-lambda-case): Merge multiple "meta" entries among the various
cases.
(lambda*, case-lambda, case-lambda*): Receive "meta" clauses, not
docstrings.
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/ice-9/boot-9.scm (macroexpand): Rename from sc-expand.
(%pre-modules-transformer): Adapt to name change.
* module/ice-9/compile-psyntax.scm: Adapt to name change.
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/ice-9/psyntax.scm: Rename sc-expand to macroexpand.
* module/language/scheme/compile-tree-il.scm (compile-tree-il): Adapt to
name change.
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/ice-9/psyntax.scm (chi-top-sequence): Track changes to the
current module. Allows e.g. top-level `define-module' within a begin
to work.
* test-suite/tests/syncase.test ("changes to expansion environment"):
Enable test.
* module/ice-9/psyntax.scm (build-dynlet, with-fluids): Use psyntax to
recognize `with-fluids' as a core form, producing <dynlet> if we are
compiling.
(sc-expand): To spice up the mix, use with-fluids here in the
implementation.
* module/ice-9/psyntax-pp.scm: Bootstrapped twice (!).
* module/ice-9/boot-9.scm: Remove with-fluids definition, it's in the
core now.
There are some incompatible changes here, but only to interfaces that
were introduced earlier in 1.9, or interfaces which have been broken
since early in 1.9.
* libguile/_scm.h (SCM_OBJCODE_MINOR_VERSION): Bump, as the macro
changes affect the interface that is called by psyntax-generated macro
definitions.
* libguile/inline.h (scm_words): New function, allocates a variable
number of contiguous scm_t_bits locations, with a given value in the
0th word, and 0 in the rest of the words.
* libguile/macros.h: Rework interface to correspond more closely, and
minimally, to the needs of memoize.c and psyntax.
(SCM_ASSYNT, SCM_MACRO_TYPE_BITS, SCM_MACRO_TYPE_MASK)
(SCM_F_MACRO_EXTENDED, SCM_MACROP, SCM_MACRO_TYPE)
(SCM_MACRO_IS_EXTENDED, SCM_BUILTIN_MACRO_P, SCM_SYNCASE_MACRO_P)
(SCM_MACRO_CODE, scm_tc16_macro): Remove CPP macros related to the
representation of Scheme macros.
(scm_i_make_primitive_macro): Renamed from scm_i_makbimacro.
(scm_i_macro_primitive): New accessor so that memoize.c can get to the
primitive syntax transformer.
(scm_make_syncase_macro, scm_make_extended_syncase_macro)
(scm_syncase_macro_type, scm_syncase_macro_binding): Removed these
functions, replaced by make-syntax-transformer and its accessors.
(scm_macro_binding): New accessor, the same as what
scm_syncase_macro_binding was.
* libguile/macros.c: All representation details of syntax transformers
are private to this file now.
(macro_print): Print macros as #<syntax-transformer ...>, or
#<primitive-syntax-transformer ...> if psyntax has not attached a
transformer of its own.
(scm_i_make_primitive_macro): Represent macros as 5-word smobs.
(scm_make_syntax_transformer): New constructor for syntax transformers
(macros), exported to scheme. Takes a name, and looks it up in the
current module to determine the previous primitive transformer, if
any.
(scm_macro_type): Instead of returning 'builtin-macro!, etc, return
the type as set by psyntax, or #f if it's a primitive.
(scm_macro_name): Return the stored macro name.
(scm_macro_transformer): Return the psyntax-set syntax transformer.
Hacky, but should help introspection somewhat.
* libguile/memoize.c (memoize_env_ref_transformer): Use the new
scm_i_macro_primitive, and adapt to other macro API changes.
* module/ice-9/psyntax.scm (put-global-definition-hook)
(get-global-definition-hook, chi-install-global): Call (and generate
calls to) the new macro constructors and accessors.
* module/ice-9/psyntax-pp.scm: Doubly regenerated.
* module/ice-9/debugging/traps.scm (trap-here): Comment out this
definition and export, while it's not working.
* libguile/vm-engine.c (VM_NAME): Keyword arg errors are now thrown to
'keyword-argument-error.
* libguile/vm.c: Define sym_keyword_argument_error, and statically
allocate some other symbols.
* module/ice-9/optargs.scm (parse-lambda-case): Throw to
'keyword-argument-error in kwarg error cases.
* module/ice-9/psyntax.scm (build-lambda-case): Remove a couple
workarounds for the old memoizer. Throw to 'wrong-number-of-args if
the lambda-case fails to parse.
* module/ice-9/psyntax-pp.scm: Regenerated.
* test-suite/tests/optargs.test: Update expected exceptions.
* module/ice-9/psyntax.scm (gen-label, new-mark): Labels and marks need
to be unique across read/write boundaries -- it's not sufficient for
them to simply be unique within a process by virtue of (string #\i)
constructing a new object. This used to mostly work before, because
the collapsing of duplicate constants didn't catch many syntax-object
cases -- but for some reason the attached test case brings out the
problem. So switch to use gensyms. Potentially more costly, but it's
what upstream psyntax does now.
This bug took me fully two days to figure out.
* module/ice-9/psyntax-pp.scm: Regenerated.
* test-suite/tests/syncase.test ("serializable labels and marks"): Add
test case.
I've prepared a patch that adds partial support for tail patterns.
Things like the the SRFI-34 `guard' macro from [0] are supported, but
you still can't combine dotted patterns with tail patterns, e.g.
(syntax-rules (else)
((foo bar ... (else something) . rest)
<TEMPLATE-HERE>))
will *not* work; there's the issue that one can't just transcribe
the implementation of this feature from the latest version of psyntax,
as I've done for non-dotted tail patterns, as it's implemented using a
dotted pattern like the above. Alas!
[0] <http://article.gmane.org/gmane.lisp.guile.devel/9442>
* module/ice-9/psyntax.scm (syntax-case, $sc-dispatch): Add support for
tail patterns, transcribed from the latest psyntax.
* module/ice-9/psyntax-pp.scm: Regenerated.
* test-suite/tests/syncase.test: Add tests for tail patterns.
* libguile/eval.c (scm_primitive_eval, scm_c_primitive_eval):
(scm_init_eval): Rework so that scm_primitive_eval always calls out to
the primitive-eval variable. The previous definition is the default
value, which is probably overridden by scm_init_eval_in_scheme.
* libguile/init.c (scm_i_init_guile): Move ports and load-path up, so we
can debug when initing eval. Call scm_init_eval_in_scheme. Awesome.
* libguile/load.h:
* libguile/load.c (scm_init_eval_in_scheme): New procedure, loads up
ice-9/eval.scm to replace the primitive-eval definition, if everything
is there and up-to-date.
* libguile/modules.c (scm_module_transformer): Export to Scheme, so it's
there for eval.go.
* module/ice-9/boot-9.scm: No need to define module-transformer.
* module/ice-9/eval.scm (capture-env): Only reference the-root-module if
modules are booted.
(primitive-eval): Inline a definition for identity. Throw a more
standard error for "wrong number of arguments".
* module/ice-9/psyntax.scm (chi-install-global): The macro binding for a
syncase macro is now a pair: the transformer, and the module that was
current when the transformer was installed. The latter is used for
hygiene purposes, replacing the use of procedure-module, which didn't
work with the interpreter's shared-code closures.
(chi-macro): Adapt for the binding being a pair, and get the hygiene
from the cdr.
(eval-local-transformer): Adapt to new form of macro bindings.
* module/ice-9/psyntax-pp.scm: Regenerated.
* .gitignore: Ignore eval.go.stamp.
* module/Makefile.am: Reorder for fastest serial compilation, now that
there are no ordering constraints. I did a number of experiments here
and this seems to be the best; but the bulk of the time is compiling
psyntax-pp.scm with eval.scm. Not so great.
* libguile/vm-engine.c (vm-engine): Throw a more standard error for
"wrong type to apply".
* test-suite/tests/gc.test ("gc"): Remove a hack that shouldn't affect
the new evaluator, and throw in another (gc) for good measure.
* test-suite/tests/goops.test ("defining classes"):
* test-suite/tests/hooks.test (proc1): We can't currently check what the
arity is of a closure made by eval.scm -- or more accurately all
closures have 0 required args and no rest args. So punt for now.
* test-suite/tests/syntax.test ("letrec"): The scheme evaluator can't
check that a variable is unbound, currently; perhaps the full "fixing
letrec" expansion could fix this. But barring that, punt.
Turns out this was not a very useful idea, and semantically tricky to
boot.
This reverts commit 24bf130fd1, and makes
the following additional changes:
* module/ice-9/optargs.scm (parse-lambda-case, let-optional)
(let-optional*, let-keywords, let-keywords*):
* module/language/tree-il.scm: (<lambda-case>, parse-tree-il)
(unparse-tree-il, tree-il->scheme, tree-il-fold,
make-tree-il-folder)
(post-order!, pre-order!):
* module/language/tree-il/analyze.scm (analyze-lexicals):
* module/language/tree-il/compile-glil.scm (compile-glil):
* module/language/tree-il/inline.scm (inline!): Remove all traces of
#:predicate from tree-il.
* module/ice-9/psyntax.scm (build-simple-lambda, build-lambda-case)
(chi-lambda-case): Adapt to tree-il change.
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/language/brainfuck/compile-tree-il.scm (compile-body):
* module/language/ecmascript/compile-tree-il.scm (comp, comp-body):
* test-suite/tests/tree-il.test: Adapt to tree-il change.
* doc/ref/api-procedures.texi (Case-lambda): Remove mention of
#:predicate.
will be useful for making e.g. typecase-lambda. Tough to tell though.
* module/ice-9/psyntax.scm (lambda-formals, lambda*-formals): Parse out
a #:predicate, which goes right before the rest args. The vanilla
lambda doesn't parse it out of course, but it does return another
value.
(chi-lambda-case, lambda*, lambda): Expand and pass the predicate on
to build-lambda-case.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/language/tree-il/compile-glil.scm (flatten): Compile a failing
predicate without an else clause into a call to `error'. Also, fix
something the compile warnings caught.
* module/ice-9/psyntax.scm (case-lambda, case-lambda*): Add
implementations of these, present in the base environment.
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/srfi/srfi-16.scm (case-lambda): Replace the core's case-lambda
definition with our own. We're not quite ready to switch yet.
* module/ice-9/psyntax.scm (lambda-formals, chi-simple-lambda)
(lambda*-formals, chi-lambda-case): Refactor the lambda
transformations with an eye to being able to do case-lambda.
* module/ice-9/psyntax-pp.scm: Regenerated.
* test-suite/tests/syntax.test: Adapted tests so that the errors we
expect match what is given by psyntax.
* module/ice-9/optargs.scm:
* module/ice-9/psyntax-pp.scm:
* module/ice-9/psyntax.scm: Make define* available in the default
environment, as lambda* is available there.
* module/ice-9/psyntax.scm (build-lambda-case): Take an "inits" arg.
Also work around a nasty memoizer bug: see
http://article.gmane.org/gmane.lisp.guile.devel/9561.
(lambda*): Implement in psyntax, in the default environment. Exciting
stuff!
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/ice-9/optargs.scm (parse-lambda-case): Helper for lambda* when
we're running under the interpreter.
* module/language/tree-il.scm (<lambda-case>): Add "inits" field, so we
don't have to parse it out of opt and kw. Adapt the traversal
procedures.
* module/language/tree-il/analyze.scm (analyze-lexicals): Analyze
lexicals in the <lambda-case> init expressions as well. Fix keyword
allocation.
* module/language/tree-il/compile-glil.scm (compile-glil): Adapt to
make-lambda-case change.
(flatten): Adapt to "inits" slot, actually init uninitialized args,
and fix bugs related to keyword arguments.
* module/language/tree-il/inline.scm (inline!): Adapt a little bit --
but with no effect.
* module/language/glil/compile-assembly.scm (glil->assembly): Flesh out
<glil-kw-prelude> compilation some more. Add a "bound?" op for
<glil-lexical>, which will push #t if the local is bound.
* module/ice-9/psyntax.scm (build-simple-lambda, build-lambda-case):
Update for new signature of make-lambda-case.
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/language/brainfuck/compile-tree-il.scm (compile-body):
* module/language/ecmascript/compile-tree-il.scm (comp):
* test-suite/tests/tree-il.test ("lambda"): Update for new lambda-case
syntax.
* libguile/_scm.h (SCM_OBJCODE_MINOR_VERSION): Bumparoo
* libguile/vm-i-system.c (push-rest, bind-rest): Logically there are
actually two rest binders -- one that pops, conses, and pushes, and
one that pops, conses, and local-sets. The latter is used on keyword
arguments, because the keyword arguments themselves have been shuffled
up on the stack. Renumber ops again.
* module/language/tree-il/compile-glil.scm (flatten): Attempt to handle
compilation of lambda-case with keyword arguments. Might need some
help.
* module/ice-9/psyntax.scm (build-lambda-case): An attempt to handle the
interpreted case correctly. This might need a couple iterations, but
at least it looks like the compile-glil code.
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/language/glil.scm (<glil>): Rename "rest?" to "rest" in
<glil-opt-prelude> and <glil-kw-prelude>, as it is no longer a simple
boolean, but if true is an integer: the index of the local variable to
which the rest should be bound.
* module/language/glil/compile-assembly.scm (glil->assembly): Adapt to
"rest" vs "rest?". In the keyword case, use "bind-rest" instead of
"push-rest".
* test-suite/tests/tree-il.test: Update for opt-prelude change.
* module/language/tree-il.scm (<lambda>, <lambda-case>): Split lambda
into the lambda itself, denoting the procedure, and lambda-case,
denoting a particular arity case. Lambda-case is fairly featureful,
and has not yet been fully tested.
(<let-values>): Use a <lambda-case> as the binding expression. Seems
to suit the purpose well.
Adapt parsers, unparsers, traversal operators, etc. Sometimes in this
first version we assume there are no optional args, rest args, or a
predicate.
* module/language/tree-il/analyze.scm (analyze-lexicals): Adapt for the
new case-lambda regime. Fairly well commented. It actually simplifies
things.
(report-unused-variables): Update for new tree-il.
* module/language/tree-il/compile-glil.scm: Adapt for the new tree-il.
There are some first stabs here at proper case-lambda compilation, but
they are untested as of yet.
* module/language/tree-il/inline.scm (inline!): Rework so we can
recurse on a single node; though these transformations are strictly
reductive, so they should complete in bounded time. Simplify
accordingly, and adapt to case-lambda. Oh, and we handle lambda->let
in not just the nullary case.
* module/ice-9/psyntax.scm (build-simple-lambda, build-case-lambda)
(build-lambda-case): New constructors. The idea is that after syntax
expansion, we shouldn't have to deal with improper lists any more.
Build-simple-lambda is a shortcut for the common case. The others are
not fully exercised yet. Adapt callers.
(syntax): Add some debugging in the lambda case. I don't fully
understand this, but in practice we don't seem to see rest args here.
(lambda): Inline chi-lambda-clause, and adapt for build-simple-lambda.
* module/ice-9/psyntax-pp.scm: Regenerated.
* test-suite/tests/tree-il.test: Update tests for new tree-il lambda
format, and to expect post-prelude labels for all glil programs.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/ice-9/psyntax.scm (chi-top)[define-form]: If a same-named
imported variable exists, take its value instead of `#f'.
* test-suite/tests/compiler.test ("psyntax")["redefinition"]: New tests.
* module/ice-9/psyntax.scm (build-lexical-var): Make our gensyms really
unique. Before, there was a chance that different lexicals could
result in the same gensym.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/ice-9/psyntax.scm (source-wrap): Use decorate-source, for
clarity.
(syntax-type): When turning the RHS of (define (foo) ...) into a
lambda, decorate the resulting lambda expression with source
information, as the RHS later goes to chi-expr, which receives no
source information. Perhaps that is a bug. In any case, fixes some
source location lossage, reported by Jao.
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/Makefile.am (ice-9/psyntax-pp.scm): Don't try autocompiling when
making psyntax-pp.scm.
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/ice-9/psyntax.scm (chi-top): Only affect the compile-time
environment if modules have booted.
* libguile/vm-i-scheme.c (FUNC2): Use a signed value for the intermediate
result here. Not sure what the effect is, though.
* module/ice-9/psyntax.scm (chi-top): Toplevel definitions ensure that
variables are defined in the current module. Fixes the specific case of
guile-lib's md5.scm, which redefines + -- this code is needed so that
we don't incorrectly open-code +.
* module/language/tree-il/primitives.scm (resolve-primitives!): I think
there were some cases in which vars and names would not resolve
properly here. Fix those.
* module/ice-9/psyntax.scm (syntax-type): Remove `external-macro', not
used any more. Take an extra arg, `for-car?', indicating that we're
checking on the type of a form in the car position. In the case that
the expression is a pair, do a full recursion on the car, which allows
us to catch the fact that the car of the following form is a macro:
((@ (ice-9 optargs) let-optional) ...)
and thus the form itself should be macroexpanded.
But, since we want to distingush `lambda' from `(lambda ...)', just as
we have global and global-call, we have core to the new `core-form'.
(chi-top, chi, chi-expr, chi-body, set!): Adapt to changes to
syntax-type.
* module/ice-9/psyntax.scm (source-annotation): Return #f if
source-properties returns null.
(source-wrap): Rework a bit.
(syntax-type): Don't throw away source info for wrapped expressions.
Can has source location info, fools!
(chi-body): Correctly propagate source info for body subforms.
(syntax): Remove special case for map, it doesn't apply (ahem) for
Guile.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/ice-9/psyntax.scm: Remove references to annotation objects,
we're just going to try and use Guile's source properties now. It works
until `syntax' reconstructs output, at which point it seems we lose it.
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/ice-9/psyntax.scm (chi-body): Fix a problem introduced in
dc1eed52f7, that internal syntax definitions were included in the id
lis along with value definitions. Only showed up on a second bootstrap.
Psyntax, how I love thee.
* module/ice-9/psyntax-pp.scm
