A byte ordering error caused incorrect display of wide strings
when using the ",c" decompilation from the REPL.
* module/language/assembly/decompile-bytecode.scm (decode-bytecode):
wide strings are encoded in native endianness
The bytevector to string conversion functions were accomplished
by converting via locale strings. This did not allow conversions
of wide strings in an 8-bit locale. This is avoided by using knowledge
of the storage format of the string.
* libguile/bytevectors.c (STRING_TO_UTF, scm_string_to_utf8): modify
string to bytevector conversion to use internal string information
(UTF_TO_STRING, scm_utf8_to_string): modify bytevector to string
conversion
* Renumbers the IFLAG constants.
* Adds several macros related to boolean type tests, null tests, and
boolean-truth testing (including lisp-style boolean-truth tests).
* Adds compile-time checks to verify the necessary IFLAG numbering
properties needed for the checks to work properly.
* Changes some existing code to use the new optimized macros, without
changing the semantics of the code at all (except that scm_is_bool
is changed from a function to a macro).
I added the following macros, whose names explicitly state how %nil
should be handled. See the comments in the patch for more information
about these.
scm_is_false_assume_not_lisp_nil scm_is_true_assume_not_lisp_nil
scm_is_false_and_not_lisp_nil scm_is_true_or_lisp_nil
scm_is_false_or_lisp_nil scm_is_true_and_not_lisp_nil
scm_is_lisp_false scm_is_lisp_true
scm_is_null_assume_not_lisp_nil
scm_is_null_and_not_lisp_nil
scm_is_null_or_lisp_nil
scm_is_bool_and_not_lisp_nil
scm_is_bool_or_lisp_nil
The following already-existing macros are defined as aliases, such
that their semantics is unchanged (although scm_is_bool used to be a
function and is now a macro).
scm_is_null --> scm_is_null_and_not_lisp_nil
scm_is_false --> scm_is_false_and_not_lisp_nil
scm_is_true --> scm_is_true_or_lisp_nil
scm_is_bool --> scm_is_bool_and_not_lisp_nil
(I still believe that these should be changed to versions that handle
%nil properly, but await approval on that point, so these patches do
not make those changes)
Also, if the preprocessor macro SCM_ENABLE_ELISP is not true (this
macro already existed and was used in lang.h), all overheads
associated with %nil handling are eliminated from the above macros.
* libguile/tags.h (SCM_BOOL_F, SCM_BOOL_T, SCM_UNSPECIFIED)
(SCM_UNDEFINED, SCM_UNBOUND, SCM_ELISP_NIL): Renumber, so that a
number of important distinctions (false versus true, end-of-list, etc)
can be made by masking a single bit. Also define a number of
build-time tests to assert that this condition holds.
* libguile/boolean.h (scm_is_false_and_not_nil, scm_is_true_or_nil)
(scm_is_false_assume_not_nil, scm_is_true_assume_not_nil):
(scm_is_false_or_nil, scm_is_true_and_not_nil)
(scm_is_bool_or_nil, scm_is_bool_and_not_nil): New exciting macros to
test certain boolean/end-of-list properties.
(scm_is_false, scm_is_true): Use a restrictive definition, where only
SCM_BOOL_F is false. Should probably change in the future.
(scm_is_bool): Incompatible change: changed to be a macro. Was a
function before. Probably should allow nil as a boolean, but that will
be for a later patch.
(scm_is_lisp_false, scm_is_lisp_true): New macros, implementing the
standard Lisp boolean predicates, where '() is actually false.
* libguile/eval.i.c (CEVAL): Fix a number of false-or-nil and similar
tests to use the new macros.
* libguile/lang.h (SCM_NULL_OR_NIL_P): Use scm_is_null_or_nil.
* libguile/pairs.c: Add a compile-time check that null and nil differ by
only one bit.
* libguile/pairs.h (scm_is_null_and_not_nil, scm_is_null_assume_not_nil)
(scm_is_null_or_nil): New exciting macros!
(scm_is_null): Just be scm_is_null_and_not_nil, for now.
* libguile/print.c: Adapt to the reordering, and print suitably nasty
things for the not-to-be-used values.
* libguile/hashtab.h (scm_t_hash_fold_fn, scm_t_hash_handle_fn): New
types.
(scm_internal_hash_fold, scm_internal_hash_for_each_handle): Use them.
* libguile/hashtab.c (scm_internal_hash_fold): Take an
`scm_t_hash_fold_fn'. Update callers.
(scm_internal_hash_for_each_handle): Take an `scm_t_hash_handle_fn'.
Update callers.
This is a followup to d587c9e8b2 ("Use
proper types for hash/assoc functions in `hashtab.h'.").
* libguile/hashtab.c (scm_i_rehash, scm_hash_fn_create_handle_x,
scm_hash_fn_ref, scm_hash_fn_set_x, scm_hash_fn_remove_x): Use
`scm_t_hash_fn' and `scm_t_assoc_fn' as appropriate.
* module/system/vm/program.scm: Export the arity things again, and
program-arity. Why not.
* doc/ref/api-procedures.texi (Compiled Procedures): Update for current
API.
(Optional Arguments): Update to assume lambda* and define* are always
available, and (ice-9 optargs) is now the ghetto.
(Case-lambda): Now here, moved from SRFI-16 docs. Also docs
case-lambda*.
* doc/ref/srfi-modules.texi: Point to core case-lambda docs.
will be useful for making e.g. typecase-lambda. Tough to tell though.
* module/ice-9/psyntax.scm (lambda-formals, lambda*-formals): Parse out
a #:predicate, which goes right before the rest args. The vanilla
lambda doesn't parse it out of course, but it does return another
value.
(chi-lambda-case, lambda*, lambda): Expand and pass the predicate on
to build-lambda-case.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/language/tree-il/compile-glil.scm (flatten): Compile a failing
predicate without an else clause into a call to `error'. Also, fix
something the compile warnings caught.
* module/srfi/srfi-35.scm (%make-condition-type): New procedure.
(make-condition-type, make-compound-condition-type): Use it.
* test-suite/tests/srfi-35.test ("condition
types")["struct-vtable-name"]: New test.
* libguile/goops.c (scm_class_of): Fix second argument for
`scm_make_extended_class_from_symbol ()' for nameless structs.
* test-suite/tests/goops.test ("classes for built-in types")["struct
vtable"]: New test case.
* module/ice-9/psyntax.scm (case-lambda, case-lambda*): Add
implementations of these, present in the base environment.
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/srfi/srfi-16.scm (case-lambda): Replace the core's case-lambda
definition with our own. We're not quite ready to switch yet.
* module/language/assembly/compile-bytecode.scm (write-bytecode): Handle
br-if-nargs compilation.
* module/language/assembly/decompile-bytecode.scm (decode-load-program):
And decompile them nicely as well.
* module/language/assembly/disassemble.scm (code-annotation): And,
present the disassembly if br-if-nargs-* nicely.
* module/language/glil/compile-assembly.scm (open-arity, close-arity)
(begin-arity, glil->assembly): Refactor so that arities can have
noncontiguous starts and ends. So within a prelude there is no arity
-- only before (the previous arity) or after (the new arity).
* module/system/vm/program.scm (arity:end): Add this private accessor.
Arities are expected to be in the new format. While not a change in
objcode format, it is an incompatible change, so I'll bump the objcode
cookie.
(program-arity): Check that the ip is within both bounds of the arity.
* libguile/_scm.h (SCM_OBJCODE_MINOR_VERSION): Bump.
* libguile/programs.c (scm_i_program_arity): Update for new arity format.
* module/system/vm/frame.scm (vm-frame-arguments): Avoid throwing an
error in this function, which is called from the backtrace code.
* module/ice-9/psyntax.scm (lambda-formals, chi-simple-lambda)
(lambda*-formals, chi-lambda-case): Refactor the lambda
transformations with an eye to being able to do case-lambda.
* module/ice-9/psyntax-pp.scm: Regenerated.
* test-suite/tests/syntax.test: Adapted tests so that the errors we
expect match what is given by psyntax.
* module/ice-9/optargs.scm:
* module/ice-9/psyntax-pp.scm:
* module/ice-9/psyntax.scm: Make define* available in the default
environment, as lambda* is available there.
* goops.c (scm_sys_modify_instance, scm_sys_modify_class): Use
SCM_{,SET_}CELL_WORD_[01] instead of SCM_{,SET}C[AD]R since the
objects passed are not pairs.
* module/ice-9/optargs.scm (let-optional, let-optional*, let-keywords)
(let-keywords*): Implement in terms of parse-lambda-case, so all the
logic is in one place.
(lambda*): Re-export from the default environment -- it's all in the
VM now :-))
(define*, define*-public, defmacro*, defmacro*-public): Implement with
syntax-case.
* module/ice-9/psyntax.scm (build-lambda-case): Take an "inits" arg.
Also work around a nasty memoizer bug: see
http://article.gmane.org/gmane.lisp.guile.devel/9561.
(lambda*): Implement in psyntax, in the default environment. Exciting
stuff!
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/ice-9/optargs.scm (parse-lambda-case): Helper for lambda* when
we're running under the interpreter.
* module/language/tree-il.scm (<lambda-case>): Add "inits" field, so we
don't have to parse it out of opt and kw. Adapt the traversal
procedures.
* module/language/tree-il/analyze.scm (analyze-lexicals): Analyze
lexicals in the <lambda-case> init expressions as well. Fix keyword
allocation.
* module/language/tree-il/compile-glil.scm (compile-glil): Adapt to
make-lambda-case change.
(flatten): Adapt to "inits" slot, actually init uninitialized args,
and fix bugs related to keyword arguments.
* module/language/tree-il/inline.scm (inline!): Adapt a little bit --
but with no effect.
* module/language/glil/compile-assembly.scm (glil->assembly): Flesh out
<glil-kw-prelude> compilation some more. Add a "bound?" op for
<glil-lexical>, which will push #t if the local is bound.
* module/ice-9/psyntax.scm (build-simple-lambda, build-lambda-case):
Update for new signature of make-lambda-case.
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/language/brainfuck/compile-tree-il.scm (compile-body):
* module/language/ecmascript/compile-tree-il.scm (comp):
* test-suite/tests/tree-il.test ("lambda"): Update for new lambda-case
syntax.
* libguile/modules.c (scm_module_lookup, scm_lookup): Throw to
'unbound-variable, like eval.i.c does.
* test-suite/tests/optargs.test: Add an optargs test. Run optargs tests
under both the VM and the interpreter.
* libguile/vm-i-system.c (local-bound?, long-local-bound?)
(variable-bound?): New instructions, push #f unless the local is
bound. You can get unbound locals from optional arguments.
(bind-optionals/shuffle): A number of bugfixes.
(bind-kwargs): Bugfixes. If we enocunter an improper kwarg list but
the procedure has a rest argument, just stop kwarg processing, but
without an error.
Renumbered ops.
* libguile/_scm.h (SCM_OBJCODE_MAJOR_VERSION): Bump.
* libguile/_scm.h (SCM_OBJCODE_MINOR_VERSION): Bumparoo
* libguile/vm-i-system.c (push-rest, bind-rest): Logically there are
actually two rest binders -- one that pops, conses, and pushes, and
one that pops, conses, and local-sets. The latter is used on keyword
arguments, because the keyword arguments themselves have been shuffled
up on the stack. Renumber ops again.
* module/language/tree-il/compile-glil.scm (flatten): Attempt to handle
compilation of lambda-case with keyword arguments. Might need some
help.
* module/ice-9/psyntax.scm (build-lambda-case): An attempt to handle the
interpreted case correctly. This might need a couple iterations, but
at least it looks like the compile-glil code.
* module/ice-9/psyntax-pp.scm: Regenerated.
* module/language/glil.scm (<glil>): Rename "rest?" to "rest" in
<glil-opt-prelude> and <glil-kw-prelude>, as it is no longer a simple
boolean, but if true is an integer: the index of the local variable to
which the rest should be bound.
* module/language/glil/compile-assembly.scm (glil->assembly): Adapt to
"rest" vs "rest?". In the keyword case, use "bind-rest" instead of
"push-rest".
* test-suite/tests/tree-il.test: Update for opt-prelude change.
* libguile/_scm.h (SCM_OBJCODE_MINOR_VERSION): Bump.
* libguile/vm-i-system.c (br-if-nargs-ne, br-if-args-lt)
(br-if-nargs-gt): New instructions, for use by different lambda cases.
(bind-optionals, bind-optionals/shuffle, bind-kwargs): New
instructions, for binding optional and keyword arguments. Renumber
other ops.
* module/language/ecmascript/compile-tree-il.scm (comp, comp-body):
Update for new tree-il. Use the new optional argument mechanism
instead of emulating it with rest arguments.
* module/language/glil/compile-assembly.scm (glil->assembly): Tweaks for
optional and keyword argument compilation.
* module/language/tree-il.scm (parse-tree-il, unparse-tree-il): Make the
else case optional, in the s-expression serialization of tree-il.
* module/language/tree-il/compile-glil.scm (flatten): Handle all of the
lambda-case capabilities.
* test-suite/Makefile.am:
* test-suite/tests/brainfuck.test: Add a brainfuck test.
* module/system/base/compile.scm: Also export read-and-compile.
* module/language/tree-il/spec.scm (join): Fix the joiner in the
0-expression case.
* module/language/tree-il/primitives.scm (+): Recognize (+ x -1) as 1-.
* module/language/brainfuck/parse.scm (read-brainfuck): Return EOF if we
actually received EOF, and there were no expressions read.
* module/language/brainfuck/compile-tree-il.scm (compile-body): Fix the
compiler for the new format of "lambda" in tree-il.
* module/language/tree-il.scm (<lambda>, <lambda-case>): Split lambda
into the lambda itself, denoting the procedure, and lambda-case,
denoting a particular arity case. Lambda-case is fairly featureful,
and has not yet been fully tested.
(<let-values>): Use a <lambda-case> as the binding expression. Seems
to suit the purpose well.
Adapt parsers, unparsers, traversal operators, etc. Sometimes in this
first version we assume there are no optional args, rest args, or a
predicate.
* module/language/tree-il/analyze.scm (analyze-lexicals): Adapt for the
new case-lambda regime. Fairly well commented. It actually simplifies
things.
(report-unused-variables): Update for new tree-il.
* module/language/tree-il/compile-glil.scm: Adapt for the new tree-il.
There are some first stabs here at proper case-lambda compilation, but
they are untested as of yet.
* module/language/tree-il/inline.scm (inline!): Rework so we can
recurse on a single node; though these transformations are strictly
reductive, so they should complete in bounded time. Simplify
accordingly, and adapt to case-lambda. Oh, and we handle lambda->let
in not just the nullary case.
* module/ice-9/psyntax.scm (build-simple-lambda, build-case-lambda)
(build-lambda-case): New constructors. The idea is that after syntax
expansion, we shouldn't have to deal with improper lists any more.
Build-simple-lambda is a shortcut for the common case. The others are
not fully exercised yet. Adapt callers.
(syntax): Add some debugging in the lambda case. I don't fully
understand this, but in practice we don't seem to see rest args here.
(lambda): Inline chi-lambda-clause, and adapt for build-simple-lambda.
* module/ice-9/psyntax-pp.scm: Regenerated.
* test-suite/tests/tree-il.test: Update tests for new tree-il lambda
format, and to expect post-prelude labels for all glil programs.
* libguile/procprop.c (scm_procedure_property)
scm_set_procedure_property_x): Fix to not call
scm_procedure_properties(), and thus to avoid consing up the arity as
well.
* libguile/vm-i-system.c (bind-rest): Renamed from push-rest-list.
(reserve-locals): Change so that instead of reserving space for some
additional number of locals, reserve-locals takes the absolute number
of locals, including the arguments.
* module/language/glil.scm (<glil-std-prelude>, <glil-opt-prelude>)
(<glil-kw-prelude>): New GLIL constructs, to replace <glil-arity>.
* module/language/glil/compile-assembly.scm (glil->assembly): Compile
the new preludes. Some instructions are not yet implemented, though.
* module/language/tree-il/analyze.scm (analyze-lexicals): The nlocs for
a lambda will now be the total number of locals, including arguments.
* module/language/tree-il/compile-glil.scm (flatten-lambda): Update to
write the new prelude.
* module/system/vm/program.scm (program-bindings-for-ip): If a given
index doesn't have a binding at the ip given, don't cons it on the
resulting list.
* test-suite/tests/tree-il.test: Update for GLIL changes.
* libguile/objcodes.h (struct scm_objcode): Remove nargs, nrest, and
nlocs, as they are no longer needed. Also obviates the need for a
padding word.
* libguile/procs.c (scm_thunk_p): Use scm_i_program_arity for programs.
* libguile/procprop.c (scm_i_procedure_arity): Use scm_i_program_arity
for programs.
(scm_procedure_properties, scm_set_procedure_properties_x)
(scm_procedure_property, scm_set_procedure_property_x): Rework so that
non-closure properties are stored directly in a weak hash, instead of
needing a weak hash of "stand-in" closures to hold the properties. Fix
docstrings also.
* libguile/root.h (scm_stand_in_procs): Remove from the scm_sys_protects
set. Actually with libGC, we should be able to store the elements of
scm_sys_protects directly as global variables.
* libguile/gc.c (scm_init_storage): Remove scm_stand_in_procs
initialization.
* libguile/programs.c (scm_i_program_arity): New private accessor, tries
to determine the "minimum arity" of a program.
* libguile/vm.c (really_make_boot_program): Adapt to changes in
struct scm_objcode.
* module/language/assembly.scm (*program-header-len*, byte-length):
* module/language/assembly/compile-bytecode.scm (write-bytecode):
* module/language/assembly/decompile-bytecode.scm (decode-load-program):
* module/language/assembly/disassemble.scm (disassemble-load-program):
Adapt to changes in objcode.
* module/system/xref.scm (program-callee-rev-vars): Adapt to changes in
assembly.
* module/language/glil.scm: Remove nargs, nrest, and nlocs from
glil-program.
* module/language/glil/compile-assembly.scm (make-meta, glil->assembly):
* module/language/glil/decompile-assembly.scm (decompile-toplevel):
(decompile-load-program): Adapt to changes in GLIL and assembly.
* module/language/tree-il/compile-glil.scm (flatten-lambda): Adapt to
changes in GLIL.
* test-suite/tests/asm-to-bytecode.test: Adapt to assembly and bytecode
changes.
* test-suite/tests/tree-il.test: Adapt to GLIL changes.
* gdbinit: Ignore SIGPWR and SIGXCPU, which the BDW GC seems to use.
* libguile/vm-engine.h (FETCH_WIDTH): Remove unused macro.
(INIT_ARGS, INIT_FRAME): Remove; callees now check their args and
reserve space for their locals.
* libguile/vm-engine.c:
* libguile/vm-i-system.c: Turn on callee arg checking and local
reservation. Seems to work!
* libguile/_scm.h (SCM_OBJCODE_MINOR_VERSION): Bump.
* libguile/vm-i-system.c (reserve-locals): New instruction, to reserve
space for local vars. Other ops renumbered.
* module/language/tree-il/compile-glil.scm (flatten-lambda): Emit
reserve-locals instructions as necessary.
* test-suite/tests/tree-il.test: Update to expect reserve-locals as
appropriate.
* libguile/objcodes.h: Bump for metadata format change.
* libguile/frames.h: Rework so we don't frob the program's nargs, nlocs,
etc at runtime. Instead we don't really know what's a local var, an
argument, or an intermediate value. It's a little unfortunate, but
this will allow for case-lambda, and eventually for good polymorphic
generic dispatch; and the nlocs etc can be heuristically
reconstructed. Such a reconstruction would be better done at the
Scheme level, though.
(SCM_FRAME_STACK_ADDRESS): New macro, the pointer to the base of the
stack elements (not counting the program).
(SCM_FRAME_UPPER_ADDRESS): Repurpose to be the address of the last
element in the bookkeeping part of the stack -- i.e. to point to the
return address.
* libguile/vm-engine.h:
* libguile/vm-i-system.c: Adapt to removal of stack_base. Though we
still detect stack-smashing underflow, we don't do so as precisely as
we did before, because now we only detect overwriting of the frame
metadata.
* libguile/vm-engine.c (vm_engine): Remove the stack_base variable. It
is unnecessary, and difficult to keep track of in the face of
case-lambda. Also fix miscommented "ra" and "mvra" pushes. Push the
vp->ip as the first ra...
* libguile/vm-i-system.c (halt): ...because here we can restore the
vp->ip instead of setting ip to 0. Allows us to introspect ips all
down the stack, including in recursive VM invocations.
* libguile/frames.h:
* libguile/frames.c (scm_vm_frame_stack): Removed, because it's getting
more difficult to tell what's an argument and what's a temporary stack
element.
(scm_vm_frame_num_locals): New accessor.
(scm_vm_frame_instruction_pointer): New accessor.
(scm_vm_frame_arguments): Defer to an implementation in Scheme.
(scm_vm_frame_num_locals scm_vm_frame_local_ref)
(scm_vm_frame_local_set_x): Since we can get not-yet-active frames on
the stack now, with our current calling convention, we have to add a
heuristic here to jump over those frames -- because frames have
pointers in them, not Scheme values.
* libguile/programs.h:
* libguile/programs.c (scm_program_arity): Remove, in favor of..
(scm_program_arities): ...this, which a list of arities, in a new
format, occupying a slot in the metadata.
* module/language/assembly/decompile-bytecode.scm (decode-load-program):
Fix mv-call decompilation.
* module/system/vm/frame.scm (vm-frame-bindings, vm-frame-binding-ref)
(vm-frame-binding-set!): New functions, to access bindings by name in
a frame.
(vm-frame-arguments): Function now implemented in Scheme. Commented
fairly extensively.
* module/system/vm/program.scm (program-bindings-by-index)
(program-bindings-for-ip): New accessors, parsing the program bindings
metadata into something more useful.
(program-arities, program-arguments): In a case-lambda world, we have
to assume that programs can have multiple arities. But it's tough to
detect this algorithmically; instead we're going to require that the
program metadata include information about the arities, and the parts
of the program that that metadata applies to.
(program-lambda-list): New accessor.
(write-program): Show multiple arities.
* module/language/glil/compile-assembly.scm (glil->assembly): Add
"arities" to the state of the compiler, and add arities entries as
appropriate.
* libguile/_scm.h (SCM_OBJCODE_MINOR_VERSION): Bump.
* libguile/vm-i-system.c (assert-nargs-ee, assert-nargs-ge)
(push-rest-list): New instructions, which for now don't actually do
anything. Renumber the rest of the ops in this file.
* module/language/glil.scm (<glil-arity>): New GLIL type, an entity that
checks the number of args for a block, optionally consing a rest list,
and either branching or erroring if the arity doesn't match.
* module/language/glil/compile-assembly.scm (glil->assembly): Compile
<glil-arity> to assembly. Some of these VM ops are not implemented --
notably the branching case.
* module/language/tree-il/compile-glil.scm (flatten-lambda): Emit
<glil-arity>.
* test-suite/tests/tree-il.test: Update.
* module/language/tree-il/analyze.scm (goops-toplevel-definition): New
procedure.
(report-possibly-unbound-variables): Check for GOOPS top-level
definitions.
* test-suite/tests/tree-il.test ("warnings")["GOOPS definitions are
visible"]: New test.
Partly fixes bug #23681 ("Function declarators with empty parentheses
should not be used").
* libguile/goops.c (scm_wrap_component): Cast `scm_sloppy_assq'.
* libguile/hashtab.c (scm_hash_fn_get_handle): Update to take functions
of type `scm_t_hash_fn' and `scm_t_assoc_fn'. Update callers.
(scm_ihashx): Change to match `scm_t_hash_fn'.
(scm_sloppy_assx): Change to match `scm_t_assoc_fn'.
* libguile/hashtab.h (scm_t_hash_fn, scm_t_assoc_fn): New types.
(scm_t_hashtable)[hash_fn]: Change to `scm_t_hash_fn'.
(scm_i_rehash, scm_hash_fn_get_handle, scm_hash_fn_create_handle_x,
scm_hash_fn_ref, scm_hash_fn_set_x, scm_hash_fn_remove_x): Change to
take `scm_t_hash_fn' and `scm_t_assoc_fn' parameters.
* libguile/srcprop.h (scm_whash_get_handle, scm_whash_create_handle,
scm_whash_lookup): Implement in terms of `scm_hashq_*' instead of
`scm_hash_fn_*'.
* libguile/struct.c (scm_struct_ihashq): Change to match
`scm_t_hash_fn'.
(scm_struct_create_handle): Cast `scm_sloppy_assq'.
* libguile/struct.h (scm_struct_ihashq): Update, make internal.
