* module/language/tree-il/optimize.scm (peval)[maybe-unlambda]: New
procedures.
Use it to de-duplicate named lambdas. This fixes the scoping bug
described at <https://lists.gnu.org/archive/html/bug-guile/2011-09/msg00019.html>.
* test-suite/tests/tree-il.test ("partial evaluation"): Add tests to
reproduce the bug.
* module/language/tree-il/optimize.scm (peval): Propagate ARGS to BODY
only when all of ARGS are pure. Change APP to use `maybe-unconst' for
its arguments.
* test-suite/tests/tree-il.test ("partial evaluation"): Add tests for
mutability preservation and non-propagation of non-constant arguments
to lambdas.
* module/language/tree-il/optimize.scm (peval)[make-values]: Distinguish
between 1 or another number of values.
[mutable?, make-value-construction, maybe-unconst]: New procedures.
Use it in <let>, <letrec>, <toplevel-define>, and <lambda-case>.
* test-suite/tests/tree-il.test ("partial evaluation"): Add tests
for mutability preservation.
Thanks to William R. Cook for his excellent tutorial,
<http://softlang.uni-koblenz.de/dsl11/>.
* module/language/tree-il/optimize.scm (optimize!): Call `peval' unless
the #:partial-eval? option asks otherwise.
(peval): New procedure.
* module/language/tree-il/inline.scm: Add comment.
* module/language/tree-il/primitives.scm (*primitive-constructors*): New
variable.
(*effect-free-primitives*): Use it.
(constructor-primitive?): New primitive.
* test-suite/tests/tree-il.test (assert-tree-il->glil): Extend to
support `with-partial-evaluation', `without-partial-evaluation', and
`with-options'.
(peval): New binding.
(pass-if-peval): New macro.
("lexical refs"): Run tests without partial evaluation.
("letrec"): Likewise.
("the or hack"): Likewise.
("conditional"): Likewise, for some tests.
("sequence"): Adjust to new generated code.
("partial evaluation"): New test prefix.
* module/language/tree-il.scm (tree-il->scheme): Fix incorporation of
`lambda' in a `case-lambda'.
* test-suite/tests/tree-il.test ("tree-il->scheme"): Add a test.
* module/language/tree-il/primitives.scm (+, *, cons*): In the case of
just one argument (the identity case), expand to (values x) instead of
just x. Fixes values truncation in that case.
(values): Likewise remove (values x) -> x translation, as the compiler
will do it for us, and this fixes (values (values 1 2)).
* module/language/tree-il/compile-glil.scm (flatten-lambda-case): Handle
`values' in a push context here.
* test-suite/tests/tree-il.test ("values"): Add some tests.
* module/language/tree-il/fix-letrec.scm (fix-letrec!): When X is a
`letrec*' with only lambdas and simple expressions, analyze it as if
it were a `letrec'.
* test-suite/tests/tree-il.test ("letrec"): Add test for
`(letrec* (x y) (xx yy) ((const 1) (const 2)) (lexical y yy))'.
* module/language/tree-il/analyze.scm (gensym?): New procedure.
(unused-variable-analysis): Ignore variables whose name passes
`gensym?' or is `_'.
(unused-toplevel-analysis): Ignore variables whose name passes
`gensym?'.
* test-suite/tests/tree-il.test ("warnings")["unused-variable"]("special
variable names"): New test.
["unused-toplevel"]("special variable names"): New test.
* module/language/tree-il/analyze.scm (&syntax-error): New variable.
(format-string-argument-count): Throw to &SYNTAX-ERROR when a syntax
error in a format string is encountered.
(format-analysis): Catch &SYNTAX-ERROR and convert as a warning of the
appropriate type.
* module/system/base/message.scm (%warning-types)[format]: Handle
`syntax-error' warnings.
* test-suite/tests/tree-il.test
("warnings")["conditionals"]("unterminated", "unexpected ~;",
"unexpected ~]"): New tests.
["unterminated ~{...~}"]: New test.
* module/language/tree-il/analyze.scm (format-analysis): Add new
sub-warnings: `wrong-port', `wrong-format-string',
`non-literal-format-string', and `wrong-num-args'.
* module/system/base/message.scm (%warning-types)[format]: Handle
them.
* test-suite/tests/tree-il.test ("warnings")["wrong port arg",
"wrong format string", "non-literal format string",
"wrong number of args"]: New tests.
* module/language/tree-il/analyze.scm (format-string-argument-count):
Return two values, the minimum and maximum number of arguments.
Add support for most of `format' escapes, including conditionals.
(format-analysis): Adjust accordingly.
* module/system/base/message.scm (%warning-types)[format]: Take two
arguments, MIN and MAX, instead of EXPECTED. Display warning
accordingly.
* test-suite/tests/tree-il.test ("warnings")["format"]("~%, ~~, ~&, ~t,
~_, and ~\\n", "~{...~}", "~{...~}, too many args", "~@{...~}",
"~@{...~}, too few args", "~(...~)", "~v", "~v:@y", "~*", "~?",
"complex 1", "complex 2", "complex 3"): New tests.
("conditionals"): New test prefix.
* test-suite/tests/tree-il.test ("letrec"): Add some tests.
* module/language/tree-il/compile-glil.scm (flatten): Add support for
compiling letrec* in its unoptimized form.
* module/language/tree-il/fix-letrec.scm (simple-expression?):
Parameterize, so that letrec* will not treat `(car x)' as primitive
(because it could raise an exception).
(partition-vars): Lump unreferenced vars in with complex vars, when
compiling letrec*.
(fix-letrec!): No need to evaluate inits within a let for letrec*.
* module/language/tree-il/inline.scm (boolean-value, inline!): Add a
case for inlining conditional expressions.
* test-suite/tests/tree-il.test: Fix conditional tests to have a
non-inlinable condition.
* module/language/tree-il/compile-glil.scm (flatten): Change to use the
anonymous <glil-mv-bind> form when truncating to 0 or 1 values. In
those cases, remove the <glil-unbind> statements. As a side effect,
fixes compilation of abort in a "values" context.
Thanks to Tristan Colgate for the bug report.
* test-suite/tests/tree-il.test: Update to expect anonymous mv-bind.
* module/ice-9/boot-9.scm (export, re-export): Evaluate at compile-time
in addition to load/eval.
* test-suite/tests/tree-il.test ("warnings")["unused-toplevel"]("unused
but define-public"): Remove throw to `unresolved'.
* module/language/tree-il/analyze.scm (<reference-dag>): New record
type.
(dag-reachable-nodes, dag-reachable-nodes*, unused-variable-analysis):
New variables.
(unbound-variable-analysis): Slightly simplify the `up' procedure.
* module/language/tree-il/compile-glil.scm (%warning-passes): Add
`unused-toplevel'.
* module/system/base/message.scm (%warning-types): Likewise.
* test-suite/tests/tree-il.test (%opts-w-unused-toplevel): New variable.
("warnings")["unused-toplevel"]: New test prefix.
* libguile/_scm.h (SCM_OBJCODE_MINOR_VERSION): Bump.
* libguile/programs.h (SCM_PROGRAM_FREE_VARIABLES)
(SCM_PROGRAM_FREE_VARIABLE_REF, SCM_PROGRAM_FREE_VARIABLE_SET)
(SCM_PROGRAM_NUM_FREE_VARIABLES):
* libguile/programs.c (scm_make_program, scm_program_num_free_variables)
(scm_program_free_variable_ref, scm_program_free_variable_set_x):
Allocate free variables inline with programs, instead of being in a
vect. Should improve locality, and require fewer local variables in
the VM.
* libguile/vm-engine.c (vm_engine): Remove free_vars and free_vars_count
variables.
* libguile/vm-engine.h (CACHE_PROGRAM): No need to muck with free_vars
and free_vars_count.
(CHECK_FREE_VARIABLE): Update for inline free vars.
* libguile/vm-i-system.c (FREE_VARIABLE_REF): Update for inline free
vars.
(make-closure, fix-closure): Take the closure vals as separate stack
args, and copy or fix them inline into the appropriate closure.
* module/language/objcode/spec.scm (program-free-variables): Define a
local version of this removed function.
* module/language/tree-il/compile-glil.scm (flatten): Adjust to not make
a vector when making closures.
* module/system/vm/program.scm: Export program-num-free-variables,
program-free-variable-ref, program-free-variable-set!, and remove
program-free-variables.
* test-suite/tests/tree-il.test ("lambda"): Update to not make vectors
when making closures.
* libguile/vm-i-system.c (tail-call, tail-call/nargs, tail-apply)
(tail-call/cc): Rename these back to tail-* from goto/*. We should
reserve the rename-then-goto name for when you actually do a rename
and goto, not when you shuffle the stack.
* doc/ref/vm.texi:
* module/language/glil/decompile-assembly.scm:
* module/language/tree-il/compile-glil.scm:
* test-suite/tests/tree-il.test: Adapt all callers and documentation.
Turns out this was not a very useful idea, and semantically tricky to
boot.
This reverts commit 24bf130fd1, and makes
the following additional changes:
* module/ice-9/optargs.scm (parse-lambda-case, let-optional)
(let-optional*, let-keywords, let-keywords*):
* module/language/tree-il.scm: (<lambda-case>, parse-tree-il)
(unparse-tree-il, tree-il->scheme, tree-il-fold,
make-tree-il-folder)
(post-order!, pre-order!):
* module/language/tree-il/analyze.scm (analyze-lexicals):
* module/language/tree-il/compile-glil.scm (compile-glil):
* module/language/tree-il/inline.scm (inline!): Remove all traces of
#:predicate from tree-il.
* module/ice-9/psyntax.scm (build-simple-lambda, build-lambda-case)
(chi-lambda-case): Adapt to tree-il change.
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/language/brainfuck/compile-tree-il.scm (compile-body):
* module/language/ecmascript/compile-tree-il.scm (comp, comp-body):
* test-suite/tests/tree-il.test: Adapt to tree-il change.
* doc/ref/api-procedures.texi (Case-lambda): Remove mention of
#:predicate.
* module/language/tree-il/analyze.scm (validate-arity)[arity]: Rename
to...
[arities]: ... this. Return all the arities of PROC.
Update caller accordingly.
* test-suite/tests/tree-il.test ("warnings")["arity
mismatch"]("case-lambda", "case-lambda with wrong number of
arguments", "case-lambda*", "case-lambda* with wrong arguments"): New
tests.
* module/language/tree-il/analyze.scm
(validate-arity)[filter-keyword-args]: New procedure.
[arity]: Get accurate arity for programs, return ALLOW-OTHER-KEYS? as
an additional value.
Update to `arity' change; use `filter-keyword-args'.
* test-suite/tests/tree-il.test ("warnings")["arity mismatch"]("keyword
not passed and quiet", "keyword passed and quiet", "keyword passed to
global and quiet", "extra keyword", "extra keywords allowed"): New
tests.
* module/language/tree-il/analyze.scm (<arity-info>): New record type.
(validate-arity, arity-analysis): New variables.
* module/language/tree-il/compile-glil.scm (%warning-passes): Add
`arity-mismatch'.
* module/system/base/message.scm (%warning-types): Likewise.
* test-suite/tests/tree-il.test (read-and-compile): Remove, as it's now
public.
(%opts-w-arity): New.
("warnings")["arity mismatch"]: New test prefix.
* module/language/tree-il.scm (<lambda-case>): Add "inits" field, so we
don't have to parse it out of opt and kw. Adapt the traversal
procedures.
* module/language/tree-il/analyze.scm (analyze-lexicals): Analyze
lexicals in the <lambda-case> init expressions as well. Fix keyword
allocation.
* module/language/tree-il/compile-glil.scm (compile-glil): Adapt to
make-lambda-case change.
(flatten): Adapt to "inits" slot, actually init uninitialized args,
and fix bugs related to keyword arguments.
* module/language/tree-il/inline.scm (inline!): Adapt a little bit --
but with no effect.
* module/language/glil/compile-assembly.scm (glil->assembly): Flesh out
<glil-kw-prelude> compilation some more. Add a "bound?" op for
<glil-lexical>, which will push #t if the local is bound.
* module/ice-9/psyntax.scm (build-simple-lambda, build-lambda-case):
Update for new signature of make-lambda-case.
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/language/brainfuck/compile-tree-il.scm (compile-body):
* module/language/ecmascript/compile-tree-il.scm (comp):
* test-suite/tests/tree-il.test ("lambda"): Update for new lambda-case
syntax.
* libguile/_scm.h (SCM_OBJCODE_MINOR_VERSION): Bumparoo
* libguile/vm-i-system.c (push-rest, bind-rest): Logically there are
actually two rest binders -- one that pops, conses, and pushes, and
one that pops, conses, and local-sets. The latter is used on keyword
arguments, because the keyword arguments themselves have been shuffled
up on the stack. Renumber ops again.
* module/language/tree-il/compile-glil.scm (flatten): Attempt to handle
compilation of lambda-case with keyword arguments. Might need some
help.
* module/ice-9/psyntax.scm (build-lambda-case): An attempt to handle the
interpreted case correctly. This might need a couple iterations, but
at least it looks like the compile-glil code.
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/language/glil.scm (<glil>): Rename "rest?" to "rest" in
<glil-opt-prelude> and <glil-kw-prelude>, as it is no longer a simple
boolean, but if true is an integer: the index of the local variable to
which the rest should be bound.
* module/language/glil/compile-assembly.scm (glil->assembly): Adapt to
"rest" vs "rest?". In the keyword case, use "bind-rest" instead of
"push-rest".
* test-suite/tests/tree-il.test: Update for opt-prelude change.
* module/language/tree-il.scm (<lambda>, <lambda-case>): Split lambda
into the lambda itself, denoting the procedure, and lambda-case,
denoting a particular arity case. Lambda-case is fairly featureful,
and has not yet been fully tested.
(<let-values>): Use a <lambda-case> as the binding expression. Seems
to suit the purpose well.
Adapt parsers, unparsers, traversal operators, etc. Sometimes in this
first version we assume there are no optional args, rest args, or a
predicate.
* module/language/tree-il/analyze.scm (analyze-lexicals): Adapt for the
new case-lambda regime. Fairly well commented. It actually simplifies
things.
(report-unused-variables): Update for new tree-il.
* module/language/tree-il/compile-glil.scm: Adapt for the new tree-il.
There are some first stabs here at proper case-lambda compilation, but
they are untested as of yet.
* module/language/tree-il/inline.scm (inline!): Rework so we can
recurse on a single node; though these transformations are strictly
reductive, so they should complete in bounded time. Simplify
accordingly, and adapt to case-lambda. Oh, and we handle lambda->let
in not just the nullary case.
* module/ice-9/psyntax.scm (build-simple-lambda, build-case-lambda)
(build-lambda-case): New constructors. The idea is that after syntax
expansion, we shouldn't have to deal with improper lists any more.
Build-simple-lambda is a shortcut for the common case. The others are
not fully exercised yet. Adapt callers.
(syntax): Add some debugging in the lambda case. I don't fully
understand this, but in practice we don't seem to see rest args here.
(lambda): Inline chi-lambda-clause, and adapt for build-simple-lambda.
* module/ice-9/psyntax-pp.scm: Regenerated.
* test-suite/tests/tree-il.test: Update tests for new tree-il lambda
format, and to expect post-prelude labels for all glil programs.
* libguile/vm-i-system.c (bind-rest): Renamed from push-rest-list.
(reserve-locals): Change so that instead of reserving space for some
additional number of locals, reserve-locals takes the absolute number
of locals, including the arguments.
* module/language/glil.scm (<glil-std-prelude>, <glil-opt-prelude>)
(<glil-kw-prelude>): New GLIL constructs, to replace <glil-arity>.
* module/language/glil/compile-assembly.scm (glil->assembly): Compile
the new preludes. Some instructions are not yet implemented, though.
* module/language/tree-il/analyze.scm (analyze-lexicals): The nlocs for
a lambda will now be the total number of locals, including arguments.
* module/language/tree-il/compile-glil.scm (flatten-lambda): Update to
write the new prelude.
* module/system/vm/program.scm (program-bindings-for-ip): If a given
index doesn't have a binding at the ip given, don't cons it on the
resulting list.
* test-suite/tests/tree-il.test: Update for GLIL changes.
* libguile/objcodes.h (struct scm_objcode): Remove nargs, nrest, and
nlocs, as they are no longer needed. Also obviates the need for a
padding word.
* libguile/procs.c (scm_thunk_p): Use scm_i_program_arity for programs.
* libguile/procprop.c (scm_i_procedure_arity): Use scm_i_program_arity
for programs.
(scm_procedure_properties, scm_set_procedure_properties_x)
(scm_procedure_property, scm_set_procedure_property_x): Rework so that
non-closure properties are stored directly in a weak hash, instead of
needing a weak hash of "stand-in" closures to hold the properties. Fix
docstrings also.
* libguile/root.h (scm_stand_in_procs): Remove from the scm_sys_protects
set. Actually with libGC, we should be able to store the elements of
scm_sys_protects directly as global variables.
* libguile/gc.c (scm_init_storage): Remove scm_stand_in_procs
initialization.
* libguile/programs.c (scm_i_program_arity): New private accessor, tries
to determine the "minimum arity" of a program.
* libguile/vm.c (really_make_boot_program): Adapt to changes in
struct scm_objcode.
* module/language/assembly.scm (*program-header-len*, byte-length):
* module/language/assembly/compile-bytecode.scm (write-bytecode):
* module/language/assembly/decompile-bytecode.scm (decode-load-program):
* module/language/assembly/disassemble.scm (disassemble-load-program):
Adapt to changes in objcode.
* module/system/xref.scm (program-callee-rev-vars): Adapt to changes in
assembly.
* module/language/glil.scm: Remove nargs, nrest, and nlocs from
glil-program.
* module/language/glil/compile-assembly.scm (make-meta, glil->assembly):
* module/language/glil/decompile-assembly.scm (decompile-toplevel):
(decompile-load-program): Adapt to changes in GLIL and assembly.
* module/language/tree-il/compile-glil.scm (flatten-lambda): Adapt to
changes in GLIL.
* test-suite/tests/asm-to-bytecode.test: Adapt to assembly and bytecode
changes.
* test-suite/tests/tree-il.test: Adapt to GLIL changes.
* libguile/_scm.h (SCM_OBJCODE_MINOR_VERSION): Bump.
* libguile/vm-i-system.c (reserve-locals): New instruction, to reserve
space for local vars. Other ops renumbered.
* module/language/tree-il/compile-glil.scm (flatten-lambda): Emit
reserve-locals instructions as necessary.
* test-suite/tests/tree-il.test: Update to expect reserve-locals as
appropriate.
* libguile/_scm.h (SCM_OBJCODE_MINOR_VERSION): Bump.
* libguile/vm-i-system.c (assert-nargs-ee, assert-nargs-ge)
(push-rest-list): New instructions, which for now don't actually do
anything. Renumber the rest of the ops in this file.
* module/language/glil.scm (<glil-arity>): New GLIL type, an entity that
checks the number of args for a block, optionally consing a rest list,
and either branching or erroring if the arity doesn't match.
* module/language/glil/compile-assembly.scm (glil->assembly): Compile
<glil-arity> to assembly. Some of these VM ops are not implemented --
notably the branching case.
* module/language/tree-il/compile-glil.scm (flatten-lambda): Emit
<glil-arity>.
* test-suite/tests/tree-il.test: Update.
* module/language/tree-il/analyze.scm (goops-toplevel-definition): New
procedure.
(report-possibly-unbound-variables): Check for GOOPS top-level
definitions.
* test-suite/tests/tree-il.test ("warnings")["GOOPS definitions are
visible"]: New test.
* libguile/_scm.h (SCM_OBJCODE_MINOR_VERSION): Bump
* libguile/vm-engine.c (vm_engine): Push a frame corresponding to the
mv-call.
* libguile/vm-i-system.c: Renumber ops.
(new-frame): New op, pushes a frame.
(call, mv-call): No need to shuffle args, though we do need to pop the
frame in the non-vm call case.
(goto/args): Inconsequential tweaks.
(call/cc): Push a frame if needed.
* module/language/tree-il/compile-glil.scm (flatten): Emit `new-frame'
as appropriate.
* test-suite/tests/tree-il.test: Fix to expect new-frame.
* module/language/tree-il/compile-glil.scm (compile-glil): Compute
warnings before optimizing, as unreferenced variables will be
optimized out.
* libguile/_scm.h: Fix C99 comment.
* module/language/tree-il/fix-letrec.scm (partition-vars): Also analyze
let-bound vars.
(fix-letrec!): Fix a bug whereby a set! to an unreffed var would be
called for value, not effect. Also "fix" <let>-bound lambda
expressions -- really speeds up pmatch.
* test-suite/tests/tree-il.test ("lexical sets", "the or hack"): Update
to take into account the new optimizations.
* libguile/vm-i-scheme.c: Add add1 and sub1 instructions.
* module/language/tree-il/compile-glil.scm: Compile 1+ and 1- to add1
and sub1.
* module/language/tree-il/primitives.scm (define-primitive-expander):
Add support for `if' statements in the consequent.
(+, -): Compile (- x 1), (+ x 1), and (+ 1 x) to 1- or 1+ as
appropriate.
(1-): Remove this one. Seems we forgot 1+ before, but we weren't
compiling it nicely anyway.
* test-suite/tests/tree-il.test ("void"): Fix expected compilation of (+
(void) 1) to allow for add1.
* module/language/tree-il/analyze.scm (<binding-info>): New record type.
(report-unused-variables): New procedure.
* module/language/tree-il/compile-glil.scm (%warning-passes): New
variable.
(compile-glil): Honor `#:warnings' from OPTS.
* test-suite/tests/tree-il.test (call-with-warnings): New procedure.
(%opts-w-unused): New variable.
("warnings"): New test prefix.
* libguile/programs.h:
* libguile/programs.c: (SCM_PROGRAM_FREE_VARIABLES): Rename from
SCM_PROGRAM_FREE_VARS. Callers changed.
* libguile/programs.c (scm_make_program): Rename arg to
"free_variables".
(scm_program_free_variables): Rename from program-free-vars.
* libguile/vm-engine.h:
* libguile/vm-engine.c (VM_CHECK_FREE_VARIABLES): Rename from
VM_CHECK_CLOSURE.
(vm_engine, CACHE_PROGRAM): Rename closure and closure_count to free_vars and
free_vars_vount.
* libguile/vm-i-system.c (FREE_VARIABLE_REF): Rename from CLOSURE_REF.
(free-ref, free-boxed-ref, free-boxed-set): Rename from closure-ref,
closure-boxed-ref, closure-boxed-set.
(make-closure): Renamed from make-closure2.
* module/language/glil/compile-assembly.scm (glil->assembly): Hack to
never write out the the old "make-closure" instruction. Will fix
better later. Change to emit free-ref etc instead of closure-ref.
* module/language/tree-il/compile-glil.scm (flatten): Emit make-closure
instead of make-closure2, now that the old make-closure is gone.
* module/system/vm/program.scm (system): Rename program-free-vars to
program-free-variables.
* test-suite/tests/tree-il.test ("lambda"): Update for make-closure.
