* module/ice-9/psyntax.scm (analyze-variable): Previously, a reference
to a top-level variable in a module other than the current module would
be silently rewritten to reference the current module, if the variable
was unbound in its original module. This was a hack from the early days
of when we extended psyntax to know about the module system. Fix to
properly use the scope of the introduced binding instead of the scope of
the macro use site.
* test-suite/tests/syntax.test ("macro-introduced cross-module unbound
identifiers"): Add test.
* module/ice-9/psyntax-pp.scm: Regenerate.
Commit 54bbe0b284 inadvertently led
psyntax to dismiss source location info for data returned by read hash
extensions, because read hash extensions return plain data with
associated source properties, even when called from 'read-syntax'.
This change reverts part of this commit to restore that behavior.
Fixes <https://issues.guix.gnu.org/54003>.
* module/ice-9/psyntax.scm (datum-sourcev): New procedure.
(source-annotation): Fall back to 'datum-sourcev'.
* module/ice-9/psyntax-pp.scm: Regenerate.
* test-suite/tests/compiler.test ("psyntax")["syntax-source with
read-hash-extend"]: New test.
Avoiding systematic conversion from source vectors to property alists
saves 20% on the final heap size of a process doing:
(compile-file FILE #:optimization-level 1)
where FILE is large.
* module/language/tree-il.scm (tree-il-src/ensure-alist): New procedure
with setter. Export as 'tree-il-src'.
* module/ice-9/psyntax.scm (build-void, build-call)
(build-conditional, build-lexical-reference, build-lexical-assignment)
(build-global-reference, build-global-assignment)
(build-global-definition, build-simple-lambda, build-case-lambda)
(build-lambda-case, build-primcall, build-primref)
(build-data, build-sequence, build-let, build-named-let)
(build-letrec, expand-body): Remove (sourcev->alist src) calls.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/language/tree-il/analyze.scm (shadowed-toplevel-analysis): Use
'tree-il-src' instead of accessing the 'src' slot directly.
* module/system/vm/assembler.scm (link-debug): Adjust so PC can be
followed by a vector or an alist.
* module/ice-9/psyntax.scm (source-annotation): Only return source
properties from syntax objects.
(source-wrap): Don't look for source properties.
(expand-macro): Rebuild source properties on macro output via
source-wrap, not source properties. Only annotate head of a chain of
pairs.
(strip): Here's the only use of set-source-properties!: when stripping
a syntax object to a datum.
(macroexpand): If the input expression is not a syntax object, eagerly
extract its source properties.
(datum->syntax): Fix case in which source is given as an alist.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/ice-9/psyntax.scm (syntax-violation): Pass source as an alist.
A sourcev would be better but it's incompatible.
* module/ice-9/psyntax-pp.scm: Regenerate.
Instead of defining a separate module, given that "read" calls are quite
all over the place, we're just going to replace the boot "read" binding
with read.scm. This way, we'll be able to remove support for reader
options in the boot reader, as it will only ever be used for a finite
set of files.
* NEWS: Update.
* module/Makefile.am (ice-9/boot-9.go): Depend on read.scm.
(SOURCES):
* am/bootstrap.am (SOURCES): Don't build a ice-9/read.go, as we include
it.
* module/ice-9/boot-9.scm (read-syntax): Define here, as "include" now
uses it.
(read-hash-procedures, read-hash-procedure, read-hash-extend): New
procedures. Will replace C variants.
(read, read-syntax): Include read.scm to define these.
* module/ice-9/psyntax-pp.scm (include): Regenerate.
* module/ice-9/psyntax.scm (include): Use read-syntax, so we get better
source information.
* module/ice-9/read.scm (let*-values): New local definition, to avoid
loading srfi-11.
(%read): Use list->typed-array instead of u8-list->bytevector.
* module/language/scheme/spec.scm: Remove (ice-9 read) import;
read-syntax is there in the boot environment
* module/ice-9/psyntax.scm: Use the vector representation of source
properties internally. We have to convert to alists when going to
Tree-IL, but this will be in harmony with syntax objects once the reader
switches to vectors too.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/ice-9/psyntax.scm: Add a special case for (). There are
already special cases for pairs, vectors, etc; the issue is that with
read-syntax, the () might be come into psyntax as an annotated syntax
object, which here we would want to strip, to preserve the invariant to
psyntax users that all lists are unwrapped.
* module/ice-9/psyntax.scm: With the new behavior of datum->syntax which
allows #f for the lexical context, we have the question of what module
to attach to these newly created syntax objects. In that case we'll
mark down #f as the module, indicating that we know nothing. We have to
extend a number of other cases to default to the expander's idea of the
current module, if a syntax object has no module scope.
Also, change datum->syntax to attach the empty wrap, not the top wrap.
Attaching the top wrap leads to multiply applying the top mark, as you
recurse into subexpressions.
* module/ice-9/psyntax.scm (strip): It used to be that terms in the
source program could have a "top" mark, and when stripping marks we'd
stop recursing when we see an expression with the top mark. This had
the good effect that source programs could contain quoted syntax
objects, or quoted objects with shared structure -- in theory anyway.
In practice the compiler didn't support objects with shared structure.
Anyway when we switch to "read-syntax", quoted expressions can contain
syntax objects introduced by the reader, which naturally we would want
to strip away in a (quote FOO) form. Therefore we remove the
top-marked? optimization.
* module/ice-9/psyntax.scm (quote-syntax): New core form. Usually the
expander will unwrap all syntax objects from the input term. However
sometimes you want to preserve a syntax object, as a datum. That's
when you want quote-syntax.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/ice-9/local-eval.scm (identifier-syntax-from-box): Use
quote-syntax instead of our datum->syntax trick, which relied on
psyntax's special treatment of the top mark.
* doc/ref/api-macros.texi (Syntax Case): Update documentation for
datum->syntax.
* module/ice-9/psyntax.scm (datum->syntax): Use #:source keyword for
source location info instead of an optional, and allow an alist.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/ice-9/read.scm (%read, read): Refactor to allow read and
read-syntax to share an implementation.
(read-syntax): New function.
* doc/ref/api-macros.texi (Syntax Case): Document that template-id can
be false, and document srcloc.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/ice-9/psyntax.scm (syntax?): Allow the lexical context to be
null. Allow srcloc to be a source properties alist. Inspired by
Racket.
* module/ice-9/psyntax.scm (datum->syntax): Add an additional optional
argument, to allow callers to provide source annotation information.
* module/ice-9/psyntax-pp.scm: Regenerate.
* libguile/syntax.c (scm_make_syntax): Add optional "source" argument.
Note that this function is internal.
(scm_syntax_source): New function, replacing definition in boot-9.scm.
* libguile/syntax.h: Add new declarations.
* module/ice-9/psyntax-pp.scm:
* module/ice-9/psyntax.scm (source-annotation): For syntax objects, the
source annotation comes direct from the syntax object.
* module/system/vm/assembler.scm (link-data, intern-constant): Write
5-word syntax objects.
Fixes a regression introduced in commit
fd2ffc649c whereby including a relative
file name would result in:
ice-9/psyntax.scm:3254:24: attempt to include relative file name but could not determine base dir
* module/ice-9/psyntax.scm (call-with-include-port)[syntax-dirname]:
Lookup 'filename (symbol), not filename (identifier).
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/Makefile.am (ice-9/boot-9.go, NOCOMP_SOURCES): Add
r7rs-libraries.
* module/ice-9/boot-9.scm ("ice-9/r7rs-libraries"): Include file.
* module/ice-9/psyntax.scm (call-with-include-port): New definition.
(include): Use call-with-include-port.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/ice-9/r7rs-libraries.scm: New file.
* module/scheme/base.scm (r7:include, r7:include-ci): Fix mistaken use
of core "include". Use include-ci from core.
(features): Remove features that are already part of core.
* NEWS: Update.
This change to the expander allows mixed local definitions and
expressions. The expansion turns:
(let () (a) (define (b) 42) (b) (b))
into:
(let ()
(letrec* ((t0 (begin (a) (if #f #f)))
(b (lambda () 42)))
(b)))
Which is to say, expressions that precede definitions are expanded as
definitions of a temporary via (begin EXP (if #f #f)).
* module/ice-9/psyntax.scm (expand-body): Allow mixed definitions and
expressions.
* module/ice-9/psyntax-pp.scm: Regenerate.
* test-suite/tests/syntax.test: Add a couple tests and update for new
error messages.
Add "mod" field to <toplevel-ref>, <toplevel-set>, and
<toplevel-define>, indicating the expander's idea of what the current
module is when a toplevel variable is accessed or created. This will
help in later optimizations.
* libguile/expand.c (TOPLEVEL_REF, TOPLEVEL_SET, TOPLEVEL_DEFINE)
(expand, expand_define, expand_set_x, convert_assignment):
* libguile/expand.h (SCM_EXPANDED_TOPLEVEL_REF_FIELD_NAMES):
(SCM_MAKE_EXPANDED_TOPLEVEL_REF, SCM_EXPANDED_TOPLEVEL_SET_FIELD_NAMES):
(SCM_MAKE_EXPANDED_TOPLEVEL_SET, SCM_EXPANDED_TOPLEVEL_DEFINE_FIELD_NAMES):
(SCM_MAKE_EXPANDED_TOPLEVEL_DEFINE):
* module/ice-9/compile-psyntax.scm (translate-literal-syntax-objects):
* module/ice-9/psyntax-pp.scm:
* module/ice-9/psyntax.scm:
* module/language/tree-il.scm:
* module/language/tree-il.scm (parse-tree-il, make-tree-il-folder):
(pre-post-order):
* module/language/tree-il/analyze.scm (goops-toplevel-definition):
(macro-use-before-definition-analysis, proc-ref?, format-analysis):
* module/language/tree-il/compile-cps.scm (convert):
* module/language/tree-il/debug.scm (verify-tree-il):
* module/language/tree-il/effects.scm (make-effects-analyzer):
* module/language/tree-il/fix-letrec.scm (free-variables):
* module/language/tree-il/peval.scm (peval):
* test-suite/tests/tree-il.test: Adapt uses.
* libguile/macros.c (scm_i_make_primitive_macro): Give primitive macros
a primitive-macro macro-type.
* module/ice-9/psyntax.scm (put-global-definition-hook)
(get-global-definition-hook): Inline into uses.
(make-binding): Change format of lexically defined or rebound syntax
parameters to just be the transformer, not a list of the transformer.
(resolve-identifier, expand-install-global, expand-body)
(syntax-parameterize): Adapt to use the variable object (box) holding
the top-level syntax parameter as the "key" for lookups into the
lexical environment, instead of a fresh object associated with the
syntax transformer.
* module/ice-9/psyntax-pp.scm: Regenerate.
Fixes#27476, a horrible race when one thread is expanding a
syntax-parameterize form including uses, and another thread is expanding
the corresponding define-syntax-parameter. See
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=27476#102.
* module/ice-9/psyntax.scm (define-expansion-constructors): Use
make-struct/simple, so we can inline allocations.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/ice-9/psyntax.scm (make-syntax-object): Change to make
new-style syntax objects.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/ice-9/compile-psyntax.scm (squeeze-syntax-object): Change to be
functional.
(squeeze-constant): Likewise.
(squeeze-tree-il): Likewise.
(translate-literal-syntax-objects): New pass. The compiler can embed
literal syntax objects into compiled objects, but syntax can no longer
be read/written; otherwise users could forge syntax objects. So for
the bootstrap phase, rewrite literal constants to calls to
make-syntax.
* module/ice-9/psyntax.scm (%syntax-module): Rename from syntax-module
in order to make room for a new syntax-module primitive binding.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/ice-9/boot-9.scm: Push syntax bindings into an internal
module.
* module/system/syntax.scm: New file.
* module/Makefile.am (SOURCES): Add system/syntax.scm.
Fixes <http://bugs.gnu.org/20272>.
* module/ice-9/boot-9.scm (module-generate-unique-id!)
(module-gensym): New procedures.
(module): Add 'next-unique-id' field.
(the-root-module): Inherit 'next-unique-id' value from early stub.
(make-module, make-autoload-interface): Adjust calls to
module-constructor.
* module/ice-9/psyntax.scm (gen-label, new-mark): Generate unique
identifiers from the module name and the per-module unique-id.
(build-lexical-var, generate-temporaries): Use
'module-gensym' instead of 'gensym'.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/language/tree-il/fix-letrec.scm (fix-letrec!): Use
'module-gensym' instead of 'gensym'.
* module/system/base/syntax.scm (define-record): Likewise.
(transform-record): Likewise.
Co-authored-by: Ludovic Courtès <ludo@gnu.org>
* module/ice-9/psyntax.scm (include-from-path): Canonicalize result of
%search-load-path. Otherwise a relative path passed to `include'
would be treated as relative to the directory of the file that
contains the `include-from-path'. Fixes#21347.
* module/ice-9/psyntax-pp.scm: Regenerate.
Reported by Panicz Maciej Godek <godek.maciek@gmail.com> in
<http://lists.gnu.org/archive/html/guile-user/2015-09/msg00017.html>.
* module/ice-9/psyntax.scm (match-each+): Fix the case where a non-pair
syntax object is encountered in a dotted tail.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/ice-9/ports.scm: New file.
* am/bootstrap.am (SOURCES): Add ice-9/ports.scm.
* libguile/fports.c (scm_init_ice_9_fports): New function.
(scm_init_fports): Arrange for scm_init_ice_9_fports to be called via
load-extension, and load snarfed things there. Move open-file
definition early, to allow ports to bootstrap.
* libguile/ioext.c (scm_init_ice_9_ioext): New function.
(scm_init_ioext): Similarly, register scm_init_ice_9_ioext as an
extension.
* libguile/ports.c (scm_set_current_input_port)
(scm_set_current_output_port, scm_set_current_error_port): Don't
define Scheme bindings; do that in Scheme.
* libguile/ports.c (scm_i_set_default_port_encoding):
(scm_i_default_port_encoding, scm_i_default_port_conversion_handler):
(scm_i_set_default_port_conversion_handler): Since we now init
encoding early, remove the "init" flags on these encoding/strategy
vars.
(scm_init_ice_9_ports): New function.
(scm_init_ports): Register scm_init_ice_9_ports extension, and define
some bindings needed by the bootstrap.
* module/Makefile.am (SOURCES): Add ice-9/ports.scm.
* module/ice-9/boot-9.scm: Remove code that's not on the boot path,
moving it to ice-9/ports.scm. At the end, load (ice-9 ports).
* module/ice-9/psyntax.scm (include): Use close-port instead of
close-input-port.
* module/ice-9/psyntax-pp.scm (include): Regenerate.
* module/ice-9/psyntax.scm (resolve-identifier): There is a case where a
syntax object can resolve to itself. Prevent an infinite loop in that
case by continuing to resolve by name.
* module/ice-9/psyntax-pp.scm: Regenerate.
* test-suite/tests/syncase.test ("infinite loop bug"): Add a test.
* module/ice-9/psyntax.scm (binding-type): Update the header comment
to mention the new 'ellipsis' binding type.
(macros-only-env): Preserve ellipsis bindings.
(ellipsis?): Add 'r' and 'mod' as arguments. Search the lexical
environment for an ellipsis binding, and use it.
(gen-syntax): Adapt to the additional arguments of 'ellipsis?'.
(with-ellipsis): New core syntax.
(convert-pattern): Add unary 'ellipsis?' procedure as an argument.
(gen-clause): Adapt to the additional arguments of 'ellipsis?'.
Pass unary 'ellipsis?' procedure to 'convert-pattern'.
(syntax-case): Adapt to the additional arguments of 'ellipsis?'.
(syntax-local-binding): Support new 'ellipsis' binding type.
(syntax-rules): Add support for a custom ellipsis identifier as
the first operand, as per R7RS. Collect common code within new
local procedure 'expand-syntax-rules'.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/ice-9/local-eval.scm (analyze-identifiers): Add support for
'ellipsis' binding type.
* doc/ref/api-macros.texi (Syntax Rules): Add docs for R7RS custom
ellipsis syntax. Use @dots{}.
(Syntax Case): Add docs for 'with-ellipsis'. Use @dots{}.
(Syntax Transformer Helpers): Update to include new 'ellipsis'
binding type.
* test-suite/tests/syntax.test: Add tests.
* module/ice-9/psyntax.scm (expand-top-sequence): When defining a
toplevel variable, use the value of the same-named imported
_variable_, if any. Do _not_ use an existing syntactic binding.
* module/ice-9/psyntax-pp.scm: Regenerate.
Reported by taylanbayirli@gmail.com (Taylan Ulrich B.).
* module/ice-9/psyntax.scm (cvt*): Use 'syntax-case' to destructure
the pattern tail, instead of 'pair?', 'car', and 'cdr'.
(gen-clause): When checking for errors, check for misplaced ellipsis
before duplicate pattern variables, to improve the error message in
case of multiple misplaced ellipses.
* module/ice-9/psyntax-pp.scm: Regenerate.
* test-suite/tests/syntax.test: Add tests.
Reported by Nala Ginrut <nalaginrut@gmail.com>.
* module/ice-9/psyntax.scm (include): Give a proper error message when
given a relative file name, and when the form is not in a file.
* module/ice-9/psyntax-pp.scm: Regenerate.
* libguile/vm-i-system.c (push-fluid, pop-fluid):
* doc/ref/vm.texi (Dynamic Environment Instructions): Rename wind-fluids
to push-fluid, and unwind-fluids to pop-fluid. They now only work on
one fluid binding at a time.
* module/ice-9/boot-9.scm (with-fluid*): Implement in Scheme in terms of
primcalls to push-fluid and pop-fluid.
(custom-throw-handler, catch, with-throw-handler): Use with-fluid*
instead of with-fluids, as with-fluids is no longer available before
psyntax is loaded.
(with-fluids): Define in Scheme in terms of with-fluid*.
* libguile/fluids.c (scm_with_fluid): Rename from scm_with_fluids, and
don't expose to Scheme.
* libguile/eval.c (eval): Remove SCM_M_WITH_FLUIDS case.
* libguile/expand.c (expand_with_fluids): Remove with-fluids syntax.
(DYNLET): Remove, no longer defining dynlet in the %expanded-vtables.
* libguile/expand.h: Remove dynlet definitions.
* module/ice-9/eval.scm (primitive-eval): Remove with-fluids case.
* libguile/memoize.c (do_push_fluid, do_pop_fluid): New primitive
helpers, like wind and unwind.
(memoize): Memoize wind and unwind primcalls. Don't memoize dynlet to
with-fluids.
(scm_init_memoize): Initialize push_fluid and pop_fluid here.
* libguile/memoize.h (SCM_M_WITH_FLUIDS): Remove definition.
* module/ice-9/psyntax.scm (build-dynlet): Remove; this just supported
with-fluids, which is now defined in boot-9.
* module/ice-9/psyntax-pp.scm: Regenerate.
* doc/ref/compiler.texi (Tree-IL):
* module/language/tree-il.scm:
* module/language/tree-il/analyze.scm:
* module/language/tree-il/canonicalize.scm:
* module/language/tree-il/compile-glil.scm:
* module/language/tree-il/cse.scm:
* module/language/tree-il/debug.scm:
* module/language/tree-il/effects.scm: Remove <dynlet>. Add cases for
primcalls to push-fluid and pop-fluid in compile-glil.scm and
effects.scm.
* module/language/tree-il/peval.scm (peval): Factor out
with-temporaries; probably a bad idea, but works for now. Factor out
make-begin0 (a better idea). Inline primcalls to with-fluid*, and
remove dynlet cases.
* module/language/tree-il/primitives.scm (*interesting-primitive-names*):
Add with-fluid*.
* ice-9/psyntax.scm (@@): Recognize new form, (@@ primitive NAME), which
in operator position expands to a primcall. This expansion is only
available for forms in the (guile) module. Added an argument to @@
and @ procedures, the module, for use by expanded syntax objects;
adapted callers.
(analyze-variable): Error when accessing a primitive for value.
(get-global-definition-hook): Primitives are not macros.
(syntax-type): A form with a primitive in the car is a
primitive-call.
(expand-expr): Residualize primitive calls as primcalls.
(syntax-local-binding): Return 'primitive as the type for primitives.
