* module/language/assembly/spec.scm:
* module/language/brainfuck/spec.scm:
* module/language/bytecode/spec.scm:
* module/language/ecmascript/spec.scm:
* module/language/glil/spec.scm:
* module/language/scheme/spec.scm:
* module/language/tree-il/spec.scm: Language-readers now take two
arguments: the port and the environment. This should allow for
compile-environment-specific reader behavior.
* module/system/base/compile.scm (read-and-compile):
* module/system/repl/common.scm (repl-read): Pass the environment to the
language-reader.
* module/system/repl/repl.scm (meta-reader, prompting-meta-read):
* module/system/repl/command.scm (define-meta-command): Use the second
argument to repl-reader, so we avoid frobbing current-reader.
The intent is to maintain the readability of `pmatch' invocations.
* module/language/assembly/disassemble.scm (disassemble-load-program):
Don't use wildcards in `pmatch' invocations, even when the matched
elements are unused.
* module/language/glil/decompile-assembly.scm (decompile-toplevel,
decompile-load-program): Likewise.
* module/system/xref.scm (program-callee-rev-vars): Likewise.
* module/language/assembly.scm (byte-length): Likewise.
* module/language/tree-il/compile-glil.scm (flatten): Likewise.
* module/language/scheme/spec.scm (scheme)[#:reader]: Honor the
compilation environment's `current-reader'.
* module/system/base/compile.scm (*compilation-environment*): New
fluid.
(current-compilation-environment): New procedure.
(make-compilation-module): Provide a fresh `current-reader' fluid.
(read-and-compile): Set `*compilation-environment*' appropriately.
(compile): Likewise.
* test-suite/tests/compiler.test (read-and-compile): New.
("current-reader"): New test prefix.
* module/system/base/compile.scm (make-compilation-module,
language-default-environment): New procedures.
(read-and-compile, compile): Have ENV default to
`(language-default-environment from)'.
(compile-and-load): Compile in `(current-module)'.
* module/system/repl/common.scm (repl-compile): Explicitly compile in
the current module so that macro definitions are visible.
* libguile/load.c (kw_env): New variable.
(do_try_autocompile): Call `compile-file' with `#:env (current-module)'.
* test-suite/tests/compiler.test ("psyntax")["compile uses a fresh module by
default", "compile-time definitions are isolated"]: New tests.
["compile in current module"]: Specify `#:env (current-module)'.
["redefinition"]: Adjust.
* test-suite/tests/bytevectors.test (c&e): Explicitly compile in the
current module so that its imports are visible.
Ports are given two additional properties: a character encoding and
a conversion failure strategy. These properties have getters and setters.
The new properties are used to convert any locale text to/from the
internal representation of strings.
If unspecified, ports use a default value. The default value of these
properties is held in a fluid. The default character encoding can be
modified by calling setlocale.
ISO-8859-1 is treated specially. Since it is a native encoding of
strings, it can be processed more quickly. Source code is assumed to be
ISO-8859-1 unless otherwise specified. The encoding of a source code
file can be given as 'coding: XXXXX' in a magic comment at the top of a
file.
The C functions that deal with encoding often use a null pointer
as shorthand for the native Latin-1 encoding, for efficiency's sake.
* test-suite/tests/encoding-iso88591.test: new tests
* test-suite/tests/encoding-iso88597.test: new tests
* test-suite/tests/encoding-utf8.test: new tests
* test-suite/tests/encoding-escapes.test: new tests
* test-suite/tests/numbers.test: declare 'binary' encoding
* test-suite/tests/ports.test: declare 'binary' encoding
* test-suite/tests/r6rs-ports.test: declare 'binary' encoding
* module/system/base/compile.scm (compile-file): use source-code
file's self-declared encoding when compiling files
* libguile/strports.c: store string ports in locale encoding
(scm_strport_to_locale_u8vector, scm_call_with_output_locale_u8vector)
(scm_open_input_locale_u8vector, scm_get_output_locale_u8vector):
new functions
* libguile/strings.h: new declaration for scm_i_string_contains_char
* libguile/strings.c (scm_i_string_contains_char): new function
(scm_from_stringn, scm_to_stringn): use NULL for Latin-1
(scm_from_locale_stringn, scm_to_locale_stringn): respect character
encoding of input and output ports
* libguile/read.h: declaration for scm_scan_for_encoding
* libguile/read.c:
(read_token): now takes scheme string instead of C string/length
(read_complete_token): new function
(scm_read_sexp, scm_read_number, scm_read_mixed_case_symbol)
(scm_read_number_and_radix, scm_read_quote, scm_read_semicolon_comment)
(scm_read_srfi4_vector, scm_read_bytevector, scm_read_guile_bit_vector)
(scm_read_scsh_block_comment, scm_read_commented_expression)
(scm_read_extended_symbol, scm_read_sharp_extension, scm_read_shart)
(scm_read_expression): use scm_t_wchar for char type, use read_complete_token
(scm_scan_for_encoding): new function to find a file's character encoding
(scm_file_encoding): new function to find a port's character encoding
* libguile/rdelim.c: don't unpack strings
* libguile/print.h: declaration for modified function
scm_i_charprint
* libguile/print.c: use locale when printing characters and
strings
(scm_i_charprint): input parameter is now scm_t_wchar
(scm_simple_format): don't unpack strings
* libguile/posix.h: new declaration for scm_setbinary.
* libguile/posix.c (scm_setlocale): set default and stdio port
encodings based on the locale's character encoding
(scm_setbinary): new function
* libguile/ports.h (scm_t_port): add encoding and failed
conversion handler to port type. Declarations for new or modified
functions scm_getc, scm_unget_byte, scm_ungetc,
scm_i_get_port_encoding, scm_i_set_port_encoding_x,
scm_port_encoding, scm_set_port_encoding_x,
scm_i_get_conversion_strategy, scm_i_set_conversion_strategy_x,
scm_port_conversion_strategy, scm_set_port_conversion_strategy_x.
* libguile/ports.c: assign the current ports to zero on startup so
we can see if they've been set.
(scm_current_input_port, scm_current_output_port,
scm_current_error_port): return #f if the port is not yet
initialized
(scm_new_port_table_entry): set up a new port's encoding and
illegal sequence handler based on the thread's current defaults
(scm_i_remove_port): free port encoding name when port is removed
(scm_i_mode_bits_n): now takes a scheme string instead of a c
string and length. All callers changed.
(SCM_MBCHAR_BUF_SIZE): new const
(scm_getc): new function, since the scm_getc in inline.h is now
scm_get_byte_or_eof. This pulls one codepoint from a port.
(scm_lfwrite_substr, scm_lfwrite_str): now uses port's encoding
(scm_unget_byte): new function, incorportaing the low-level functionality
of scm_ungetc
(scm_ungetc): uses scm_unget_byte
* libguile/numbers.h (scm_t_wchar): compilation order problem with
scm_t_wchar being use in functions in multiple headers. Forward
declare scm_t_wchar.
* libguile/load.c (scm_primitive_load): scan for file encoding at
top of file and use it to set the load port's encoding
* libguile/inline.h (scm_get_byte_or_eof): new function
incorporating most of the functionality of scm_getc.
* libguile/fports.c (fport_fill_input): now returns scm_t_wchar
* libguile/chars.h (scm_t_wchar): avoid compilation order problem
with declaration of scm_t_wchar
* libguile/_scm.h (SCM_OBJCODE_MINOR_VERSION): Bump.
* libguile/vm-engine.c (vm_error_bad_wide_string_length): New error
case.
* libguile/vm-i-loader.c (load-unsigned-integer, load-integer)
(load-keyword): Remove these instructions. The former two are
obsoleted by make-int64/make-uint64, the latter via make-keyword.
(load-string): Only handle narrow strings.
(load-symbol): Only handle narrow symbols. The wide case is handled
via make-symbol.
(load-wide-string): New instruction, for wide strings.
* libguile/vm-i-system.c (define): Move here from loaders.c, as now it
just takes a sym on the stack.
(make-keyword, make-symbol): New instructions.
* module/language/assembly.scm: Remove removed instructions. No more
width byte in load-string etc.
* module/language/assembly/compile-bytecode.scm (write-bytecode): Adapt
to change in instruction set.
* module/language/glil/compile-assembly.scm (glil->assembly): Compile
define by pushing the sym then emitting (define).
(dump-object): Dump narrow and wide strings differently. Use
make-keyword and make-symbol as appropriate.
* module/language/tree-il/compile-glil.scm (flatten): When compiling a
ref to a primitive (not a call), first see if the primitive is
actually bound in the root module. (That's not the case with e.g.
bytevector-u8-ref).
* module/system/xref.scm (program-callee-rev-vars): Don't parse out
"nexts".
* test-suite/tests/asm-to-bytecode.test ("compiler"): Adapt to bytecode
format change.
* module/language/tree-il.scm (tree-il-fold): Fix for let-values case.
(make-tree-il-folder): New public macro, makes a multi-valued folder
specific to the number of seeds that the user wants.
* module/language/tree-il/optimize.scm (optimize!): Reverse the order of
inline! and fix-letrec!, as the latter might expose opportunities for
the former.
* module/srfi/srfi-11.scm (let-values): Reimplement in terms of
syntax-case, so that its expressions may reference hygienically bound
variables. See the NEWS for the rationale.
(let*-values): An empty let*-values still introduces a local `let'
binding contour.
* module/system/base/syntax.scm (record-case): Yukkkk. Reimplement in
terms of syntax-case. Ug-ly, but see the NEWS again: "Lexical bindings
introduced by hygienic macros may not be referenced by nonhygienic
macros."
* libguile/programs.h:
* libguile/programs.c: (SCM_PROGRAM_FREE_VARIABLES): Rename from
SCM_PROGRAM_FREE_VARS. Callers changed.
* libguile/programs.c (scm_make_program): Rename arg to
"free_variables".
(scm_program_free_variables): Rename from program-free-vars.
* libguile/vm-engine.h:
* libguile/vm-engine.c (VM_CHECK_FREE_VARIABLES): Rename from
VM_CHECK_CLOSURE.
(vm_engine, CACHE_PROGRAM): Rename closure and closure_count to free_vars and
free_vars_vount.
* libguile/vm-i-system.c (FREE_VARIABLE_REF): Rename from CLOSURE_REF.
(free-ref, free-boxed-ref, free-boxed-set): Rename from closure-ref,
closure-boxed-ref, closure-boxed-set.
(make-closure): Renamed from make-closure2.
* module/language/glil/compile-assembly.scm (glil->assembly): Hack to
never write out the the old "make-closure" instruction. Will fix
better later. Change to emit free-ref etc instead of closure-ref.
* module/language/tree-il/compile-glil.scm (flatten): Emit make-closure
instead of make-closure2, now that the old make-closure is gone.
* module/system/vm/program.scm (system): Rename program-free-vars to
program-free-variables.
* test-suite/tests/tree-il.test ("lambda"): Update for make-closure.
* libguile/frames.c (scm_frame_external_link): Removed.
* libguile/frames.h: No need to have the "external link" in the stack
frame -- update macros to take the new situation into account.
* libguile/objcodes.h (struct scm_objcode): Rename the nexts field to
"unused". In the future we can use it for nlocs, I think.
(SCM_OBJCODE_NEXTS): removed.
* libguile/programs.h:
* libguile/programs.c (scm_make_program): Expect the third argument to
be a vector of free variables, not a list of free variables.
SCM_BOOL_F indicates no free variables, not SCM_EOL.
(program_mark): Adapt.
(scm_program_arity): No more nexts.
(scm_program_free_vars): Replaces scm_program_externals.
* libguile/vm-engine.c (VM_CHECK_EXTERNAL)
(vm_engine): No need for the "external" var.
* libguile/vm-engine.h (CACHE_PROGRAM): Update for SCM_PROGRAM_FREE_VARS
instead of SCM_PROGRAM_EXTERNALS.
(NEW_FRAME): Update for new frame size, and no need to cons up
externals. Yay :)
* libguile/vm-i-loader.c (load-program): Update for scm_make_program.
* libguile/vm-i-system.c (external-ref, external-set): No more.
(make-closure): No more.
(goto/args): No need to re-cons externals here. Update for new stack
frame size.
(mv-call, return, return/values): Update for new frame size. No need
to reinstate externals on return.
* libguile/vm.c (really_make_boot_program, scm_load_compiled_with_vm):
Update for scm_make_program.
* module/language/objcode/spec.scm (objcode-env-externals): Treat '() as
#f, for the externals. Need to clean this up later...
* module/system/vm/program.scm (arity:nexts): Remove. Rename
program-external to program-free-vars.
* module/system/repl/repl.scm (start-repl): Given that the input port of
the repl is line-buffered, it's likely we have #\newline in the input
that is strictly extraneous, an in-band indicator to the repl that it
should begin reading now. So flush out that newline, so that you can
(read-char) at the repl, and it actually does wait for you to type in
a char instead of just returning #\newline.
While it's not an overriding concern, this does fix some brainfuck
programs that want to input from the user.
* module/system/repl/command.scm: Update copyright.
(meta-command): Rework so that it's the various meta-commands that do
the reading for their arguments. This way you can compile forms that
span more than one line, and forms that need to be read with another
language's reader.
(define-meta-command): New helper macro. Update commands to use it.
(help): Allow ,help on commands too.
* module/system/repl/repl.scm: Update copyright.
(start-repl): Adjust to give meta-command what it wants.
* module/Makefile.am (BRAINFUCK_LANG_SOURCES): Compile at the end. Add
compile-tree-il.scm.
* module/language/brainfuck/compile-tree-il.scm: New compiler, compiles
to tree-il instead of scheme. I thought it would be more illustrative,
though there are some uncommented bits.
* module/language/brainfuck/parse.scm: Modify not to put a header on the
scheme representation. After all, we don't put <scheme> before scheme
code, do we? :)
* module/language/brainfuck/spec.scm: Add tree-il compiler.
* module/language/tree-il.scm: Understand (set! (lexical foo) ...).
* module/system/base/language.scm: Update license. Actually, updates
licenses on all these.
* libguile/filesys.h:
* libguile/filesys.c (scm_canonicalize_path): New function,
canonicalize-path.
* module/system/base/compile.scm (compiled-file-name): Canonicalize the
filename so that compiling e.g. ../foo.scm doesn't compile to
~/.guile-ccache/1.9/../foo.scm.
* libguile/load.c (compiled_is_fresh): Rename from compiled_is_newer.
Check that the mtines of the .go and .scm match exactly, so we don't
get fooled by rsync-like modifications of the filesystem.
* module/system/base/compile.scm (call-with-output-file/atomic): Add
optional arg, a reference file. If present we utime the output file to
match the source file, before the rename.
(compile-file): Stamp the .go file with the timestamp of the .scm.
* libguile/load.c (scm_init_load_path): Set the fallback path even if
GUILE_SYSTEM_COMPILED_PATH is set. Now that we store full paths in the
autocompiled files, and the path contains the effective Guile version,
there's no danger of accidental collisions.
(do_try_autocompile, autocompile_catch_handler, scm_try_autocompile):
Simplify again -- since there's only one place we put autocompiled
files, and compile-file finds it itself, there's no need to pass along
the compiled file path.
(scm_primitive_load_path): Don't call out to compiled-file-name to get
the fallback path, as we might not be autocompiling, and besides that
we need to check if the file exists at all.
* module/system/base/compile.scm (compiled-file-name): Simplify again.
The auto-compiled path is just fallback path + full source path + .go.
* module/system/base/compile.scm (compiled-file-name):
* libguile/load.c (scm_init_load_path, scm_try_autocompile)
(scm_primitive_load_path): Rework so that we search for .go files in
the load-compiled path and in the fallback path, but we only
autocompile to the fallback path. Should produce a more desirable experience.
* module/system/base/compile.scm (ensure-writable-dir): Rename from
ensure-directory.
(dsu-sort): Helper, does a decorate / sort / undecorate.
(compiled-file-name): Refactor to only return a writable filename. The
readable case is handled by load.c now, and the other case was silly.
Hopefully it will do the right thing.
(load-ensuring-compiled): Remove, load.c will call out to compile-file
if necessary.
(ensure-fallback-path): Remove, load.c will add the ~/.guile-ccache dir
to the load-compiled path, which will prompt its creation if necessary.
* libguile/read.c (flush_ws, scm_read_commented_expression)
(scm_read_sharp): Add support for commenting out expressions with #;.
(scm_read_syntax, scm_read_sharp): Add support for #', #`, #, and #,@.
* module/ice-9/boot-9.scm: Remove #' read-hash extension, which actually
didn't do anything at all. It's been there since 1997, but no Guile
code I've ever seen uses it, and it conflicts with #'x => (syntax x)
from modern Scheme.
* module/system/base/compile.scm (compile-and-load): Whoops, fix a number
of bugs here.
* module/ice-9/boot-9.scm (default-pre-unwind-handler): Since we were
tail-called by pre-unwind-handler-dispatch, we can't use
pre-unwind-handler-dispatch as a narrowing argument. Instead just
narrow by one frame.
(pre-unwind-handler-dispatch): Deprecate.
(error-catching-loop): Remove crack comment and code, and just use
default-pre-unwind-handler as our pre-unwind handler.
* module/ice-9/stack-catch.scm (stack-catch):
* module/system/repl/repl.scm (call-with-backtrace): Use
default-pre-unwind-handler directly.
* doc/ref/api-procedures.texi (Compiled Procedures): Fix for API changes.
* doc/ref/compiler.texi (Compiling to the Virtual Machine): Replace GHIL
docs with Tree-IL docs. Update the bits about the Scheme compiler to
talk about Tree-IL and the expander instead of GHIL. Remove
<glil-argument>. Add placeholder sections for assembly and bytecode.
* doc/ref/vm.texi: Update examples with what currently happens. Reword
some things. Fix a couple errors.
* libguile/vm-i-system.c (externals): Remove this instruction, it's not
used.
* module/ice-9/documentation.scm (object-documentation): If the object is
a macro, try to return documentation on the macro transformer.
* module/language/assembly/disassemble.scm (disassemble-load-program):
Fix problem in which we skipped the first element of the object vector,
because of changes to procedure layouts a few months ago.
* module/language/scheme/spec.scm (read-file): Remove read-file
definition.
* module/language/tree-il.scm: Reorder exports. Remove <lexical>, it was
a compat shim to something that was never released. Fix `location'.
* module/language/tree-il/primitives.scm (/): Fix expander for more than
two args to /.
* module/system/base/compile.scm (read-file-in): Remove unused
definition.
* module/system/base/language.scm (system): Remove language-read-file.
* module/language/ecmascript/spec.scm (ecmascript): Remove read-file
definition.
* module/ice-9/boot-9.scm (guile-user): Move the `compile' autoload to
the guile-user module. Remove reference to compile-time-environment.
* module/language/scheme/compile-ghil.scm:
* module/language/tree-il/compile-glil.scm:
* module/language/tree-il/optimize.scm:
* module/system/base/compile.scm:
* test-suite/tests/compiler.test: Remove definition of and references to
compile-time-environment. While I do think that recompilation based on
a lexical environment can be useful, I think it needs to be implemented
differently. So for now we've lost nothing if we take it away, as it
doesn't work with syncase anyway.
* module/system/base/pmatch.scm: Wrap consequents in (let () ) instead of
(begin ) so that they can have local definitions.
* module/language/tree-il/compile-glil.scm: Inline some calls to
primitives.
* module/system/repl/command.scm:
* module/system/repl/common.scm:
* module/system/repl/repl.scm:
* module/system/vm/debug.scm:
* module/system/vm/trace.scm: Change #:use-syntax to #:use-module, as
that's really what we want to do.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/ice-9/psyntax.scm (syntax-type): Look up the type of the car of
a form relative to its module, if it is a syntax object. Fixes hygiene
wrt modules and private macros.
* module/ice-9/syncase.scm (sc-macro): Add a comment.
* module/system/base/pmatch.scm: The big test case: just export pmatch,
not ppat too.
* module/language/ghil.scm (unparse-ghil): Fix unparsing of quasiquoted
expressions.
* module/language/ghil/spec.scm (join): Define a joiner for GHIL.
* module/language/scheme/compile-ghil.scm (cenv-ghil-env): Expand the
definition of a CENV so it can have an actual ghil-env, if available.
(compile-ghil): Return the actual ghil env in the cenv.
* module/system/base/compile.scm (compile-file): Rewrite. `output-file'
is now a keyword argument, along with the new kwargs `env' and `from'.
We now allow exceptions to propagate up, and instead of printing the
output file to the console, we return a string corresponding to its
location.
(compile-and-load): Use read-and-compile.
(compile-fold): Thread around the cenv as well. Return all three
values.
(find-language-joint, read-and-compile): New exciting helpers. The idea
is that compiling a file should be semantically equivalent to compiling
each expression in it, one by one. Compilation can have side effects,
e.g. affecting the current language or the current reader. So what we
do is find a point in the compilation path at which different
expressions of a given language can be joined into one. Expressions
from the source language are compiled to the joint language, then
joined and compiled to the target.
(compile): Just return the first value from compile-fold.
* module/system/base/language.scm (language-joiner): New optional field.
* scripts/compile: Rework for changes to compile-file.
* module/system/base/compile.scm: Expect compile passes to produce three
values, not two. The third is the "continuation environment", the
environment that can be used to compile a subsequent expression from
the same source language. For example, expansion-time side effects can
set the current module, which would be reflected appropriately in the
continuation environment.
* module/language/assembly/compile-bytecode.scm:
* module/language/bytecode/spec.scm:
* module/language/ecmascript/compile-ghil.scm:
* module/language/ghil/compile-glil.scm:
* module/language/glil/spec.scm:
* module/language/objcode/spec.scm:
* module/language/scheme/compile-ghil.scm:
* module/system/base/compile.scm: Update compile passes to return a
continuation environment.
* We cache callees in each module, and keep a list of modified
('tainted') modules, which is used to reconstruct the callers
database incrementally.
* `procedure-callers' now returns an a-list, keyed by module name.
* module/system/xref.scm (ensure-callers-db): OK! Since we can see the
same variable twice, e.g. in different modules, keep a unified hash of
seen vars and modules. Prevents duplicates in procedure-callers.
* module/system/xref.scm (program-callee-rev-vars): It's possible to get
duplicates when combining callees of inner procedures, so ignore dups.
Quadratic, boo.
* libguile/extensions.h: Define a scm_t_extension_init_func.
* libguile/frames.c:
* libguile/instructions.c:
* libguile/objcodes.c:
* libguile/programs.c:
* libguile/vm.c: Register extension init funcs. Should play nicer with a
static Guile, in addition to working on Darwin with non-default
installation prefixes without munging DYLD_LIBRARY_PATH.
* module/system/vm/frame.scm:
* module/system/vm/instruction.scm:
* module/system/vm/objcode.scm:
* module/system/vm/program.scm:
* module/system/vm/vm.scm: Use load-extension.
* module/system/xref.scm (procedure-callers): Rework to calculate the
callers of a *variable*, not of a value. This is because the
module-observers only get fired when the module changes, not with the
variables change values. Also accept either a variable, a symbol
(resolved in the current module), or a modname . symname pair.
* module/system/xref.scm: Implement procedure-callers, as the inverse of
procedure-callees, with a cache invalidated by changes in modules.
* module/ice-9/boot-9.scm (module-use!): Don't poke module observers when
module-use! is called for an already-used module.
* module/system/xref.scm (hacky-procedure-callees): Add a
procedure-callees implementation for procedures with source, that
currently does nothing. Not sure what to do, going into the future.
* module/language/scheme/expand.scm: New module, implements a separate
expansion phase, not interleaved with compilation.
* module/language/scheme/amatch.scm: Helper for expand.scm, it's pmatch
with support for annotated source.
* module/ice-9/Makefile.am (SOURCES): Add annotate.scm to build list --
early on because it will be used in the compiler.
* module/ice-9/annotate.scm: Fix the printer, default to unstripped
(whatever that is), and add a deannotator.
* module/system/base/compile.scm (call-with-compile-error-catch): Fix for
new representation of source locations.
* module/Makefile.am (SCHEME_LANG_SOURCES): Add amatch and expand.
* module/language/ghil/compile-glil.scm (optimize*): Add a note.
* module/system/base/syntax.scm (transform-record): Access the common
slots once at the beginning. Cuts down on the number of toplevel refs
needed by the generated code.
* module/system/base/syntax.scm (transform-record): Introduce a ->
binding inside the body, that produces records of the same type.
* module/language/ghil/compile-glil.scm (optimize*): Remove our ->ghil
definition, as transform-record introduces a -> binding for us. Nice.
* module/language/ghil/compile-glil.scm (optimize*): Rewritten optimizer
-- not yet in use, but it's closer to the code that I'd like to write.
* module/system/base/syntax.scm (transform-record): New crazy macro,
makes GHIL a little less painful to work with.
