* libguile/_scm.h (SCM_OBJCODE_MINOR_VERSION): Bump.
* libguile/vm-engine.c (vm_error_bad_wide_string_length): New error
case.
* libguile/vm-i-loader.c (load-unsigned-integer, load-integer)
(load-keyword): Remove these instructions. The former two are
obsoleted by make-int64/make-uint64, the latter via make-keyword.
(load-string): Only handle narrow strings.
(load-symbol): Only handle narrow symbols. The wide case is handled
via make-symbol.
(load-wide-string): New instruction, for wide strings.
* libguile/vm-i-system.c (define): Move here from loaders.c, as now it
just takes a sym on the stack.
(make-keyword, make-symbol): New instructions.
* module/language/assembly.scm: Remove removed instructions. No more
width byte in load-string etc.
* module/language/assembly/compile-bytecode.scm (write-bytecode): Adapt
to change in instruction set.
* module/language/glil/compile-assembly.scm (glil->assembly): Compile
define by pushing the sym then emitting (define).
(dump-object): Dump narrow and wide strings differently. Use
make-keyword and make-symbol as appropriate.
* module/language/tree-il/compile-glil.scm (flatten): When compiling a
ref to a primitive (not a call), first see if the primitive is
actually bound in the root module. (That's not the case with e.g.
bytevector-u8-ref).
* module/system/xref.scm (program-callee-rev-vars): Don't parse out
"nexts".
* test-suite/tests/asm-to-bytecode.test ("compiler"): Adapt to bytecode
format change.
* module/Makefile.am (ECMASCRIPT_LANG_SOURCES):
* module/language/ecmascript/compile-ghil.scm:
* module/language/ecmascript/compile-tree-il.scm: SOURCES): Replace the
GHIL compiler with a ->tree-il compiler. Not fully functional, but the
basics work.
* module/language/ecmascript/spec.scm: Only include the tree-il compiler.
* module/language/ecmascript/tokenize.scm (read-punctuation): Avoid
mutating a constant.
This adds full Unicode strings as a datatype, and it adds some
minimal functionality. The terminal and port encoding is assumed
to be ISO-8859-1. Non-ISO-8859-1 characters are written or
input as string character escapes.
The string character escapes now have 3 forms: \xXX \uXXXX and
\UXXXXXX, for unprintable characters that have 2, 4 or 6 hex digits.
The process for writing to strings has been modified. There is now a
function scm_i_string_start_writing that does the copy-on-write
conversion if necessary.
To compile strings that may be wide, the VM storage of strings and
string-likes has changed.
Most string-using functions have not yet been updated and may break
when used with wide strings.
* module/language/assembly/compile-bytecode.scm (write-bytecode):
use variable width string bytecode format
* module/language/assembly.scm (byte-length): use variable width
bytecode format
* libguile/vm-i-loader.c (load-string, load-symbol):
(load-keyword, define): use variable-width bytecode format
* libguile/vm-engine.h (FETCH_WIDTH): new macro
* libguile/strings.h: new declarations
* libguile/strings.c (make_wide_stringbuf): new function
(widen_stringbuf): new function
(scm_i_make_wide_string): new function
(scm_i_is_narrow_string): new function
(scm_i_string_wide_chars): new function
(scm_i_string_start_writing): new function
(scm_i_string_ref): new function
(scm_i_string_set_x): new function
(scm_i_is_narrow_symbol): new function
(scm_i_symbol_wide_chars, scm_i_symbol_ref): new function
(scm_string_width): new function
(unistring_escapes_to_guile_escapes): new function
(scm_to_stringn): new function
(scm_i_stringbuf_free): modify for wide strings
(scm_i_substring_copy): modify for wide strings
(scm_i_string_chars, scm_string_append): modify for wide strings
(scm_i_make_symbol, scm_to_locale_stringn): modify for wide strings
(scm_string_dump, scm_symbol_dump, scm_to_locale_stringbuf):
(scm_string, scm_i_deprecated_string_chars): modify for wide strings
(scm_from_locale_string, scm_from_locale_stringn): add null test
* libguile/srfi-13.c: add calls for scm_i_string_start_writing for
each call of scm_i_string_stop_writing
(scm_string_for_each): modify for wide strings
* libguile/socket.c: add calls for scm_i_string_start_writing for each
call of scm_i_string_stop_writing
* libguile/rw.c: add calls for scm_i_string_start_writing for each
call of scm_i_string_stop_writing
* libguile/read.c (scm_read_string): allow reading of wide strings
* libguile/print.h: add declaration for scm_charprint
* libguile/print.c (iprin1): print wide strings and add new string
escapes
(scm_charprint): new function
* libguile/ports.h: new declarations for scm_lfwrite_substr and
scm_lfwrite_str
* libguile/ports.c (update_port_lf): new function
(scm_lfwrite): use update_port_lf
(scm_lfwrite_substr): new function
(scm_lfwrite_str): new function
* test-suite/tests/asm-to-bytecode.test ("compiler"): add string
width byte to sting-like asm tests
* module/language/tree-il/analyze.scm (analyze-lexicals): Rework to
actually determine when a fixed-point procedure may be allocated as a
label.
* module/language/tree-il/compile-glil.scm (emit-bindings): Always emit
a <glil-bind>. Otherwise it's too hard to pair with unbindings.
(flatten-lambda): Consequently, here we only `bind' if there are any
vars to bind. This doesn't make any difference, given that lambdas
don't have trailing unbind instructions, but it does keep the GLIL
output the same for thunks -- no extraneous (bind) instructions. Keeps
tree-il.test happy.
(flatten): Some bugfixes. Yaaay, it works!!!
* module/language/tree-il/compile-glil.scm (flatten-lambda, flatten):
Implement compilation of label-allocated lambda expressions. Quite
tricky, we'll see if this works when the new analyzer lands.
* module/language/tree-il/analyze.scm: Add some more comments about
something that will land in a future commit: compiling fixpoint
lambdas as labels.
(analyze-lexicals): Reorder a bit, and add a label alist to procedure
allocations. Empty for now.
* module/language/tree-il/compile-glil.scm (flatten): Adapt to the free
variables being in the cddr of the allocation, not the cdr.
* libguile/vm-i-scheme.c (vector-ref, vector-set): Sync registers if we
call out to C.
* module/language/tree-il/compile-glil.scm (flatten-lambda): Add an
extra argument, the self-label, which should be the gensym under which
the procedure is bound in a <fix> expression.
(flatten): If we see a call to a lexical ref to the self-label in a
tail position, rename and goto instead of goto/args, which will tear
down the frame -- or will, in the future. It's a primitive form of
loop detection.
* module/language/tree-il/primitives.scm (zero?): Expand to (= x 0).
* module/Makefile.am (SOURCES): Reorganize so GHIL is compiled last,
along with ecmascript.
* module/language/scheme/spec.scm: Remove references to GHIL, as it's
bitrotten and obsolete..
* module/language/tree-il.scm (make-tree-il-folder): Rework so that we
only have down and up procs, and call down and up on each element.
* module/language/tree-il/analyze.scm (analyze-lexicals): Fix a thinko
handling let-values.
* module/language/tree-il/fix-letrec.scm: Actually implement fixing
letrec. The resulting code will perform better, but violations of the
letrec restriction are not detected. This behavior is allowed by the
spec, but it is undesirable. Perhaps that will be fixed later.
* module/language/tree-il/inline.scm (inline!): Fix a case in which
((lambda args foo)) would be erroneously inlined to foo. Remove empty
let, letrec, and fix statements.
* module/language/tree-il/primitives.scm (effect-free-primitive?): New
public predicate.
* module/language/tree-il.scm (tree-il-fold): Fix for let-values case.
(make-tree-il-folder): New public macro, makes a multi-valued folder
specific to the number of seeds that the user wants.
* module/language/tree-il/optimize.scm (optimize!): Reverse the order of
inline! and fix-letrec!, as the latter might expose opportunities for
the former.
* module/srfi/srfi-11.scm (let-values): Reimplement in terms of
syntax-case, so that its expressions may reference hygienically bound
variables. See the NEWS for the rationale.
(let*-values): An empty let*-values still introduces a local `let'
binding contour.
* module/system/base/syntax.scm (record-case): Yukkkk. Reimplement in
terms of syntax-case. Ug-ly, but see the NEWS again: "Lexical bindings
introduced by hygienic macros may not be referenced by nonhygienic
macros."
* libguile/vm-i-system.c (fix-closure): New instruction, for wiring
together fixpoint procedures.
* module/Makefile.am (TREE_IL_LANG_SOURCES): Add fix-letrec.scm.
* module/language/glil/compile-assembly.scm (glil->assembly): Reindent
the <glil-lexical> case, and handle 'fix for locally-bound vars.
* module/language/tree-il.scm (<fix>): Add the <fix> tree-il type and
accessors, for fixed-point bindings. This IL construct is taken from
the Waddell paper.
(parse-tree-il, unparse-tree-il, tree-il->scheme, tree-il-fold)
(pre-order!, post-order!): Update for <fix>.
* module/language/tree-il/analyze.scm (analyze-lexicals): Update for
<fix>. The difference here is that the bindings may not be assigned,
and are not marked as such. They are not boxed.
(report-unused-variables): Update for <fix>.
* module/language/tree-il/compile-glil.scm (flatten): Compile <fix> to
GLIL.
* module/language/tree-il/fix-letrec.scm: A stub implementation of
fixing letrec -- will flesh out in a separate commit.
* module/language/tree-il/inline.scm: Fix license, it was mistakenly
added with LGPL v2.1+.
* module/language/tree-il/optimize.scm (optimize!): Run the fix-letrec!
pass.
* libguile/vm-i-scheme.c: Add add1 and sub1 instructions.
* module/language/tree-il/compile-glil.scm: Compile 1+ and 1- to add1
and sub1.
* module/language/tree-il/primitives.scm (define-primitive-expander):
Add support for `if' statements in the consequent.
(+, -): Compile (- x 1), (+ x 1), and (+ 1 x) to 1- or 1+ as
appropriate.
(1-): Remove this one. Seems we forgot 1+ before, but we weren't
compiling it nicely anyway.
* test-suite/tests/tree-il.test ("void"): Fix expected compilation of (+
(void) 1) to allow for add1.
* module/language/elisp/compile-tree-il.scm: Move dynamic binding to one place
and changed names that refer to `fluids' for dynamic binding.
* module/language/elisp/bindings.scm: Changed names referring to `fluids'.
Affects so far let-bound symbols, lambda arguments to come in the future.
* module/language/elisp/README: Document it.
* module/language/elisp/compile-tree-il.scm: Add :always-lexical option.
* test-suite/tests/elisp-compiler.test: Test it.
* module/language/tree-il/analyze.scm (<binding-info>): New record type.
(report-unused-variables): New procedure.
* module/language/tree-il/compile-glil.scm (%warning-passes): New
variable.
(compile-glil): Honor `#:warnings' from OPTS.
* test-suite/tests/tree-il.test (call-with-warnings): New procedure.
(%opts-w-unused): New variable.
("warnings"): New test prefix.
* module/language/elisp/README: Document it.
* module/language/elisp/compile-tree-il.scm: Handle without-void-checks.
* test-suite/tests/elisp-compiler.test: Test it.
* module/language/elisp/README: Document new features.
* module/language/elisp/runtime/function-slot.scm: Implement funcall, apply and
eval by using the existing compiler code.
* test-suite/tests/elisp-compiler.test: Test those.
This adds the 32-bit standalone characters. Strings are still
8-bit. Characters larger than 8-bit can only be entered or
displayed in octal format at this point. At this point, the
terminal's display encoding is expected to be Latin-1.
* module/language/assembly/compile-bytecode.scm (write-bytecode):
add 32-bit char
* module/language/assembly.scm (object->assembly): add 32-bit char
(assembly->object): add 32-bit char
* libguile/vm-i-system.c (make-char32): new op
* libguile/print.c (iprin1): print 32-bit char
* libguile/numbers.h: add type scm_t_wchar
* libguile/numbers.c: add type scm_t_wchar
* libguile/chars.h: new type scm_t_wchar
(SCM_CODEPOINT_MAX): new
(SCM_IS_UNICODE_CHAR): new
(SCM_MAKE_CHAR): operate on 32-bit char
* libguile/chars.c: comparison operators now use Unicode
codepoints
(scm_c_upcase): now receives and returns scm_t_wchar
(scm_c_downcase): now receives and returns scm_t_wchar
* module/language/elisp/README: Document it.
* module/language/elisp/compile-tree-il.scm: Implement guile-primitive.
* test-suite/tests/elisp-compiler.test: Switched a usage of guile-ref to
the now available guile-primitive.
* module/language/elisp/README: Document it.
* module/language/elisp/bindings.scm: New fields in bindings data structure
to keep track of lexical bindings for symbols.
* module/language/elisp/compile-tree-il.scm: Implement lexical-let(*).
* test-suite/tests/elisp-compiler.test: Test lexical scoping with lexical-let.
* libguile/objcodes.c (OBJCODE_COOKIE): Bump again, as our jump offsets
are now multiplied by 8.
* libguile/vm-i-system.c (BR): Interpret the 16-bit offset as a relative
jump to the nearest 8-byte-aligned block -- increasing relative jump
range from +/-32K to +/-240K.
(mvra): Do the same for the mvra jump.
* libguile/vm.c (really_make_boot_program): Align the mvra.
* module/language/assembly.scm (align-block): New export, for aligning
blocks.
* module/language/assembly/compile-bytecode.scm (write-bytecode): Emit
jumps to the nearest 8-byte-aligned block. Effectively our range is 18
bits in either direction. I would like to do this differently -- have
long-br and long-br-if, and all the other br instructions go to 8 bits
only. But the assembler doesn't have an appropriate representation to
allow me to do this yet, so for now this is what we have.
* module/language/assembly/decompile-bytecode.scm (decode-load-program):
Decode the 19-bit jumps.
* libguile/objcodes.c (OBJCODE_COOKIE): Bump objcode cookie, as we added
to struct scm_objcode.
* libguile/objcodes.h (struct scm_objcode): Add a uint32 after metalen
and before base, so that if the structure has 8-byte alignment, base
will have 8-byte alignment too. (Before, base was 12 bytes from the
start of the structure, now it's 16 bytes.)
* libguile/vm-engine.h (ASSERT_ALIGNED_PROCEDURE): Add a check that can
be turned on with VM_ENABLE_PARANOID_ASSERTIONS.
(CACHE_PROGRAM): Call ASSERT_ALIGNED_PROCEDURE.
* libguile/vm-i-system.c (long-local-ref): Add a missing semicolon.
* libguile/vm.c (really_make_boot_program): Rework to operate directly
on a malloc'd buffer, so that the program will be 8-byte aligned.
* module/language/assembly.scm (*program-header-len*): Add another 4 for
the padding.
(object->assembly): Fix case in which we would return (make-int8 0)
instead of (make-int8:0). This would throw off compile-assembly.scm's
use of addr+.
* module/language/assembly/compile-bytecode.scm (write-bytecode): Write
out the padding int.
* module/language/assembly/decompile-bytecode.scm (decode-load-program):
And pop off the padding int too.
* module/language/glil/compile-assembly.scm (glil->assembly): Don't pack
the assembly, assume that assembly.scm has done it for us. If a
program has a meta, pad out the program so that meta will be aligned.
* test-suite/tests/asm-to-bytecode.test: Adapt to expect programs to
have the extra 4-byte padding int.
* module/language/glil/compile-assembly.scm (glil->assembly)
(dump-object): Fix an exciting bug! Subprograms of subprograms were
not being aligned correctly, because the code was generated too early.
So instead delay dumping the object table until the proper time.
* module/ice-9/psyntax.scm (build-lexical-var): Make our gensyms really
unique. Before, there was a chance that different lexicals could
result in the same gensym.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/language/elisp/README: Document the change.
* module/language/elisp/compile-tree-il.scm: Add disable-void-check option.
* test-suite/tests/elisp-compiler.test: Test it.
* libguile/vm-i-system.c (long-local-ref, long-local-set)
(make-variable): New intructions, for handling nlocs >= 256.
* module/language/glil/compile-assembly.scm (glil->assembly): Compile
<glil-lexical> with support for nlocs >= 256.
* libguile/objcodes.h (struct scm_objcode): Remove the "unused" field --
the old "nexts" -- and expand nlocs to 16 bits.
* module/language/assembly/compile-bytecode.scm (write-bytecode): Write
the nlocs as a uint16.
* module/language/assembly/decompile-bytecode.scm (decode-load-program):
Decompile 16-bit nlocs. It seems this decompilation is little-endian
:-/
* test-suite/tests/asm-to-bytecode.test: Fix up to understand nlocs as a
little-endian value. The test does the right thing regarding
endianness.
* module/language/elisp/README: Document it.
* module/language/elisp/compile-tree-il.scm: Implement flet and flet*.
* test-suite/tests/elisp-compiler.test: Test flet and flet*.
* libguile/programs.h:
* libguile/programs.c: (SCM_PROGRAM_FREE_VARIABLES): Rename from
SCM_PROGRAM_FREE_VARS. Callers changed.
* libguile/programs.c (scm_make_program): Rename arg to
"free_variables".
(scm_program_free_variables): Rename from program-free-vars.
* libguile/vm-engine.h:
* libguile/vm-engine.c (VM_CHECK_FREE_VARIABLES): Rename from
VM_CHECK_CLOSURE.
(vm_engine, CACHE_PROGRAM): Rename closure and closure_count to free_vars and
free_vars_vount.
* libguile/vm-i-system.c (FREE_VARIABLE_REF): Rename from CLOSURE_REF.
(free-ref, free-boxed-ref, free-boxed-set): Rename from closure-ref,
closure-boxed-ref, closure-boxed-set.
(make-closure): Renamed from make-closure2.
* module/language/glil/compile-assembly.scm (glil->assembly): Hack to
never write out the the old "make-closure" instruction. Will fix
better later. Change to emit free-ref etc instead of closure-ref.
* module/language/tree-il/compile-glil.scm (flatten): Emit make-closure
instead of make-closure2, now that the old make-closure is gone.
* module/system/vm/program.scm (system): Rename program-free-vars to
program-free-variables.
* test-suite/tests/tree-il.test ("lambda"): Update for make-closure.
* libguile/frames.c (scm_frame_external_link): Removed.
* libguile/frames.h: No need to have the "external link" in the stack
frame -- update macros to take the new situation into account.
* libguile/objcodes.h (struct scm_objcode): Rename the nexts field to
"unused". In the future we can use it for nlocs, I think.
(SCM_OBJCODE_NEXTS): removed.
* libguile/programs.h:
* libguile/programs.c (scm_make_program): Expect the third argument to
be a vector of free variables, not a list of free variables.
SCM_BOOL_F indicates no free variables, not SCM_EOL.
(program_mark): Adapt.
(scm_program_arity): No more nexts.
(scm_program_free_vars): Replaces scm_program_externals.
* libguile/vm-engine.c (VM_CHECK_EXTERNAL)
(vm_engine): No need for the "external" var.
* libguile/vm-engine.h (CACHE_PROGRAM): Update for SCM_PROGRAM_FREE_VARS
instead of SCM_PROGRAM_EXTERNALS.
(NEW_FRAME): Update for new frame size, and no need to cons up
externals. Yay :)
* libguile/vm-i-loader.c (load-program): Update for scm_make_program.
* libguile/vm-i-system.c (external-ref, external-set): No more.
(make-closure): No more.
(goto/args): No need to re-cons externals here. Update for new stack
frame size.
(mv-call, return, return/values): Update for new frame size. No need
to reinstate externals on return.
* libguile/vm.c (really_make_boot_program, scm_load_compiled_with_vm):
Update for scm_make_program.
* module/language/objcode/spec.scm (objcode-env-externals): Treat '() as
#f, for the externals. Need to clean this up later...
* module/system/vm/program.scm (arity:nexts): Remove. Rename
program-external to program-free-vars.
* module/language/glil.scm (<glil>): New GLIL type, <glil-lexical>,
which will subsume other lexical types.
* module/language/glil/compile-assembly.scm: Compile <glil-lexical>.
(make-open-binding): Change the interpretation of the second argument
-- instead of indicating an "external" var, it now indicates a boxed
var.
(open-binding): Adapt to new glil-bind format.
* module/language/tree-il/analyze.scm: Add a lot more docs.
(analyze-lexicals): Change the allocation algorithm and output format
to allow the tree-il->glil compiler to capture free variables
appropriately and to reference bound variables in boxes if necessary.
Amply documented.
* module/language/tree-il/compile-glil.scm (compile-glil): Compile
lexical variable access to <glil-lexical>. Emit variable capture and
closure creation code here, instead of leaving that task to the
GLIL->assembly compiler.
* test-suite/tests/tree-il.test: Update expected code emission.
* module/language/elisp/README: Document it.
* module/language/elisp/compile-tree-il.scm: Moved ensure-fluid! to runtime function.
* module/language/elisp/runtime.scm: Runtime functions to support dynamic value access.
* module/language/elisp/runtime/function-slot.scm: Defined the built-ins.
* test-suite/tests/elisp-compiler.test: Test them.
* module/language/elisp/README: Document this.
* module/language/elisp/compile-tree-il.scm: Implement it here, pass bindings all around the compilation.
* module/language/elisp/bindings.scm: New module with symbol-tracking abilities needed for this.
