* module/system/vm/assembler.scm (check-urange): Hoist exact-integer?
check so that the side effect is entirely in this function and not in
`logand'. Allows devirtualize-integers to peel off a nice straight
trace.
* libguile/vm-engine.c (s64-imm=?, u64-imm<?, imm-u64<?, s64-imm<?)
(imm-s64<?): New instructions.
* libguile/instructions.c (FOR_EACH_INSTRUCTION_WORD_TYPE): Add new
X8_S12_Z12 word type used by the new S64/immediate instructions. A
Z12 is a 12-bit signed integer immediate.
* module/system/vm/assembler.scm: Export new instructions, and add
X8_S12_Z12 support. Also, add missing shufflers for X8_S12_C12.
* module/language/bytecode.scm (compute-instruction-arity):
* module/system/vm/disassembler.scm (unpack-s12, disassembler): Add
support for X8_S12_Z12.
* module/language/cps/types.scm (define-predicate-inferrer/param): New
helper.
(u64-=, u64-<, s64-<): Remove type checkers; this procedure does not
cause &type-check.
(u64-imm=?, s64-imm=?, u64-imm<?, imm-u64<?, s64-imm<?, imm-s64<?):
New type inferrers.
* module/language/cps/type-fold.scm (define-unary-branch-folder*): New
helper.
(u64-imm=?, s64-imm=?, u64-imm<?, imm-u64<?, s64-imm<?, imm-s64<?):
New branch folders.
* module/language/cps/reify-primitives.scm (reify-primitives): Reify
constants for new immediate branching primcalls if values out of
range.
* module/language/cps/effects-analysis.scm: Add support for new
primcalls.
* module/language/cps/compile-bytecode.scm (compile-function): Add
support for new primcalls and instructions. Compile u64-imm-= to
s64-imm=?.
* libguile/vm-engine.c (srsh, srsh/immediate): New instructions.
* module/language/cps/compile-bytecode.scm (compile-function):
* module/language/cps/effects-analysis.scm:
* module/language/cps/reify-primitives.scm (reify-primitives):
* module/language/cps/slot-allocation.scm (compute-var-representations):
* module/language/cps/specialize-primcalls.scm (specialize-primcalls):
* module/language/cps/types.scm (srsh, srsh/immediate):
* module/system/vm/assembler.scm: Add support for new instructions.
* module/language/cps/types.scm (ulsh, ursh): Remove type checkers, as
these are effect-free. Limit range of ursh count.
* module/language/cps/effects-analysis.scm: Remove case for "ash".
* module/language/cps/types.scm (ash): Remove.
* module/language/tree-il/compile-cps.scm (convert, canonicalize):
Convert "ash" to "lsh"/"rsh" early on.
* module/system/base/target.scm (target-fixnum?): New procedure.
* module/system/base/target.scm (target-most-negative-fixnum):
(target-most-positive-fixnum): New definitions.
* module/language/cps/types.scm (constant-type, define-exact-integer!)
(&min/fixnum, &max/fixnum): Use new definitions.
(&max/vector): Use target-max-vector-length.
* libguile/Makefile.am (INSTANTIATE): New variable.
(install-data-hook): Use it.
* libguile/libguile-2.2-gdb.scm: Autoload (system vm debug).
Augment %load-path and %load-compiled-path, and reload (system base
types).
* module/system/base/types.scm: Remove #:hide to be 2.0-compatible.
Use (system syntax internal) conditionally when on 2.2.
* libguile/throw.h (scm_ithrow, scm_throw): Mark as SCM_NORETURN.
* libguile/throw.c (scm_throw, scm_ithrow): Adapt to not return.
* libguile/vm-engine.c (throw, throw/value, throw/value+data): New
instructions.
* libguile/vm.c (vm_throw, vm_throw_with_value)
(vm_throw_with_value_and_data): New helpers.
* module/language/cps/compile-bytecode.scm (compile-function): Add cases
for new instructions.
* module/language/cps/prune-bailouts.scm (prune-bailouts): More simple,
now that there are no $kreceives in play.
* module/language/cps/reify-primitives.scm (reify-clause): Update
reification of no-clause functions to use new throw op.
* module/language/tree-il/compile-cps.scm (convert): Convert invocations
of the variable-arity 'throw primitive from Tree-IL to the new
fixed-arity CPS instructions.
* module/system/vm/assembler.scm (emit-throw/value*)
(emit-throw/value+data*, emit-throw): Export new instructions.
* module/system/vm/disassembler.scm (code-annotation): Add annotation.
* libguile/vm-engine.c (lsh, rsh, lsh/immediate, rsh/immediate): New
instructions taking unboxed bit counts.
* module/language/cps/compile-bytecode.scm (compile-function):
* module/language/cps/effects-analysis.scm:
* module/language/cps/specialize-numbers.scm (specialize-f64-unop):
(specialize-u64-unop): Add ability to specialize add/immediate, etc,
and add lsh/immediate as well.
(specialize-u64-binop, specialize-u64-shift): Move rsh/lsh
specialization to its own procedure, given that the bit count is
already unboxed.
(specialize-operations): Adapt to support more /immediate
instructions.
* module/language/cps/type-fold.scm (mul): Reify an lsh/immediate
instead of an ash.
* module/language/cps/types.scm (compute-ash-range): Add type inferrers
for lsh, rsh, and their immediate variants.
* module/system/vm/assembler.scm: Export emit-lsh and so on.
* module/language/tree-il/compile-cps.scm (convert): Convert "ash" on
immediates to rsh/immediate or lsh/immediate.
* module/system/base/target.scm (target-max-size-t):
(target-max-size-t/scm, target-max-vector-length): New public
functions.
* module/language/cps/types.scm (type-entry-saturating-union): Remove
restriction of polymorphic types to be within max-size-t; this could
incorrectly apply constraints on numeric values.
(&max/size, &max/scm-size): Use target-max-size-t.
(*max-size-t*): Remove; replace uses with (target-max-size-t).
* module/system/vm/assembler.scm (standard-prelude, opt-prelude):
(kw-prelude): Emit new instructions in function preludes. Now all
branches are via the new instructions. Remove exports for old
branches.
* module/system/base/types/internal.scm (heap-tags): Rename number to
heap-number.
* module/system/vm/assembler.scm: Adapt for emit-heap-number?.
* module/system/base/types.scm (%tc7-heap-number): Rename from
%tc7-number.
* module/system/vm/disassembler.scm (immediate-tag-annotations)
(heap-tag-annotations): Generate using tag visitors.
* module/system/vm/assembler.scm
(define-immediate-tag=?-macro-assembler)
(define-heap-tag=?-macro-assembler): New helpers. Use these to
generate immediate-tag=? and heap-tag=? macro assemblers.
* module/system/base/types/internal.scm (visit-immediate-tags)
(visit-heap-tags): New helpers.
* module/system/base/types/internal.scm (define-tags, define-tag): New
helpers.
(immediate-tags, heap-tags): Use define-tags to define all of the tag
values. For consistency some names are changed:
(%tc2-fixnum): Renamed from %tc2-inum.
(%tc8-flag): Removed.
(%tc16-null): Renamed from %tc16-eol.
(%tc7-weak-vector): Renamed from %tc7-wvect.
(%tc7-hash-table): Renamed from %tc7-hashtable.
(%tc7-flonum): Renamed from %tc7-real.
(visit-heap-tags, visit-immediate-tags): New exports.
* module/system/base/types.scm (cell->object): Adapt to renamings.
(match-bit-pattern): Add a case to match immediate SCM bits
literally.
(scm->object): Adapt to use the special immediate values directly.
* module/system/vm/disassembler.scm (immediate-tag-annotations):
(heap-tag-annotations): Adapt to new names.
This fixes a bug when using ",break":
system/vm/traps.scm:127:31: system/vm/traps.scm:127:31: In procedure <: Wrong type: #f
* module/system/vm/traps.scm (frame-matcher): Check whether END is
true.
* module/system/vm/assembler.scm: Export assemblers for the new
instructions.
* module/system/vm/disassembler.scm (immediate-tag-annotations)
(heap-tag-annotations, code-annotation): Add support for disassembling
the new instructions, with good annotations.
(compute-labels, instruction-has-fallthrough?, define-jump-parser):
Add support for new branching instructions.
* module/system/base/types/internal.scm: New file, extracted
from (system base types).
* module/system/base/types.scm: Use (system base types internal) and
adapt to %tc1-pair, %tc2-inum, and %tc3-heap-object name changes.
* module/Makefile.am (SOURCES):
* am/bootstrap.am (SOURCES): Add new file.
* libguile/struct.h (SCM_VTABLE_BASE_LAYOUT, scm_vtable_index_self):
Remove "self" field. Renumber the other fields.
* module/oop/goops.scm (<self-slot>): Remove.
(fold-class-slots): Adapt for "self" slot removal. Adapt all users.
(class-redefinition): Now that there is no "self" slot to update, use
%modify-instance instead of %modify-class.
* libguile/goops.c (class_self): Remove.
(scm_sys_modify_class): Remove.
* libguile/goops.h (scm_sys_modify_class): Remove.
* module/rnrs/records/procedural.scm: Import vtable-offset-user.
Renumber rtd indexes using vtable-offset-user.
* module/srfi/srfi-35.scm (%condition-type-vtable): Remove mention of
vtable fields.
* module/system/base/types.scm (address->inferior-struct): Adapt for
different vtable field layout.
* libguile/gc.c (scm_storage_prehistory): Register struct displacement
here.
* libguile/goops.c (scm_sys_modify_instance): Fix the format of a
comment.
* libguile/modules.c (scm_post_boot_init_modules): Update for new format
of struct vtable references.
* libguile/struct.c (scm_i_alloc_struct): Update to include slots
directly, instead of being indirected by an embedded pointer.
(scm_c_make_structv, scm_allocate_struct, scm_i_make_vtable_vtable):
Adapt to pass vtable bits as argument to scm_i_alloc_struct, not
vtable data bits.
(scm_init_struct): Remove two-word displacement from libgc.
* libguile/struct.h: Update comment.
(SCM_STRUCT_SLOTS, SCM_STRUCT_DATA): Update definitions.
(SCM_STRUCT_VTABLE_DATA, SCM_STRUCT_VTABLE_SLOTS): Remove.
(SCM_STRUCT_VTABLE, SCM_STRUCT_LAYOUT, SCM_STRUCT_PRINTER)
(SCM_STRUCT_FINALIZER, SCM_STRUCT_VTABLE_FLAGS)
(SCM_STRUCT_VTABLE_FLAG_IS_SET): Simplify definitions.
* module/system/base/types.scm (cell->object, address->inferior-struct):
Adapt to struct representation change.
* libguile/_scm.h (SCM_OBJCODE_MINIMUM_MINOR_VERSION): New definition,
indicating the oldest objcode version that we support.
(SCM_OBJCODE_MINOR_VERSION): Bump.
* libguile/loader.c (process_dynamic_segment): Support a range of
versions.
* module/system/vm/assembler.scm (*bytecode-minor-version*): Bump.
* libguile/array-handle.c (initialize_vector_handle): Add mutable_p
argument. Unless the vector handle is mutable, null out its
writable_elements member.
(scm_array_get_handle): Adapt to determine mutability of the various
arrays.
(scm_array_handle_elements, scm_array_handle_writable_elements):
Reverse the sense: instead of implementing read-only in terms of
read-write, go the other way around, adding an assertion in the
read-write case that the array handle is mutable.
* libguile/array-map.c (racp): Assert that the destination is mutable.
* libguile/bitvectors.c (SCM_F_BITVECTOR_IMMUTABLE, IS_BITVECTOR):
(IS_MUTABLE_BITVECTOR): Add a flag to indicate immutability.
(scm_i_bitvector_bits): Fix indentation.
(scm_i_is_mutable_bitvector): New helper.
(scm_array_handle_bit_elements)
((scm_array_handle_bit_writable_elements): Build writable_elements in
terms of elements.
(scm_bitvector_elements, scm_bitvector_writable_elements): Likewise.
(scm_c_bitvector_set_x): Require a mutable bitvector for the
fast-path.
(scm_bitvector_to_list, scm_bit_count): Use read-only elements()
function.
* libguile/bitvectors.h (scm_i_is_mutable_bitvector): New decl.
* libguile/bytevectors.c (INTEGER_ACCESSOR_PROLOGUE):
(INTEGER_GETTER_PROLOGUE, INTEGER_SETTER_PROLOGUE):
(INTEGER_REF, INTEGER_NATIVE_REF, INTEGER_SET, INTEGER_NATIVE_SET):
(GENERIC_INTEGER_ACCESSOR_PROLOGUE):
(GENERIC_INTEGER_GETTER_PROLOGUE, GENERIC_INTEGER_SETTER_PROLOGUE):
(LARGE_INTEGER_NATIVE_REF, LARGE_INTEGER_NATIVE_SET):
(IEEE754_GETTER_PROLOGUE, IEEE754_SETTER_PROLOGUE):
(IEEE754_REF, IEEE754_NATIVE_REF, IEEE754_SET, IEEE754_NATIVE_SET):
Setters require a mutable bytevector.
(SCM_BYTEVECTOR_SET_FLAG): New helper.
(SCM_BYTEVECTOR_SET_CONTIGUOUS_P, SCM_BYTEVECTOR_SET_ELEMENT_TYPE):
Remove helpers.
(SCM_VALIDATE_MUTABLE_BYTEVECTOR): New helper.
(make_bytevector, make_bytevector_from_buffer): Use
SCM_SET_BYTEVECTOR_FLAGS.
(scm_c_bytevector_set_x, scm_bytevector_fill_x)
(scm_bytevector_copy_x): Require a mutable bytevector.
* libguile/bytevectors.h (SCM_F_BYTEVECTOR_CONTIGUOUS)
(SCM_F_BYTEVECTOR_IMMUTABLE, SCM_MUTABLE_BYTEVECTOR_P): New
definitions.
* libguile/bytevectors.h (SCM_BYTEVECTOR_CONTIGUOUS_P): Just access one
bit.
* libguile/srfi-4.c (DEFINE_SRFI_4_C_FUNCS): Implement
writable_elements() in terms of elements().
* libguile/strings.c (scm_i_string_is_mutable): New helper.
* libguile/uniform.c (scm_array_handle_uniform_elements):
(scm_array_handle_uniform_writable_elements): Implement
writable_elements in terms of elements.
* libguile/vectors.c (SCM_VALIDATE_MUTABLE_VECTOR): New helper.
(scm_vector_elements, scm_vector_writable_elements): Implement
writable_elements in terms of elements.
(scm_c_vector_set_x): Require a mutable vector.
* libguile/vectors.h (SCM_F_VECTOR_IMMUTABLE, SCM_I_IS_MUTABLE_VECTOR):
New definitions.
* libguile/vm-engine.c (VM_VALIDATE_MUTABLE_BYTEVECTOR):
(VM_VALIDATE_MUTABLE_VECTOR, vector-set!, vector-set!/immediate)
(BV_BOUNDED_SET, BV_SET): Require mutable bytevector/vector.
* libguile/vm.c (vm_error_not_a_mutable_bytevector):
(vm_error_not_a_mutable_vector): New definitions.
* module/system/vm/assembler.scm (link-data): Mark residualized vectors,
bytevectors, and bitvectors as being read-only.
* libguile/Makefile.am (libguile_@GUILE_EFFECTIVE_VERSION@_la_SOURCES):
(DOT_X_FILES, DOT_DOC_FILES, noinst_HEADERS): Add syntax.c and
syntax.h.
* libguile/evalext.c (scm_self_evaluating_p):
* libguile/goops.c (class_syntax, scm_class_of, scm_goops_early_init):
* libguile/init.c (scm_init_guile):
* libguile/print.c (iprin1):
* libguile/tags.h (scm_tc7_syntax):
* module/oop/goops.scm (<syntax>):
* module/system/base/types.scm (%tc7-syntax, cell->object):
* module/system/vm/disassembler.scm (code-annotation): Wire up the new
data type.
* libguile/syntax.c:
* libguile/syntax.h: New files.
* module/ice-9/boot-9.scm: Move new definitions to (system syntax
internal).
* module/system/syntax.scm (print-syntax): New helper.
* module/system/vm/assembler.scm (statically-allocatable?)
(intern-constant, link-data): Arrange to be able to write syntax
objects into images.
* module/language/cps/types.scm (&syntax): New type. Remove
&hash-table; it was never detected, an internal binding, and we need
the bit to avoid going into bignum territory.
* module/ice-9/psyntax.scm (%syntax-module): Rename from syntax-module
in order to make room for a new syntax-module primitive binding.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/ice-9/boot-9.scm: Push syntax bindings into an internal
module.
* module/system/syntax.scm: New file.
* module/Makefile.am (SOURCES): Add system/syntax.scm.
* doc/ref/vm.texi (Inlined Scheme Instructions): Add string-set!.
* libguile/vm-engine.c (string-set!): New opcode.
* module/ice-9/rdelim.scm (read-string): Reimplement in terms of a
geometrically growing list of strings, to reduce total heap usage when
reading big files.
* module/language/cps/compile-bytecode.scm (compile-function): Add
string-set! support.
* module/language/cps/types.scm (string-set!): Update for &u64 index.
* module/language/tree-il/compile-cps.scm (convert): Unbox index to
string-set!.
* module/system/vm/assembler.scm (system): Export string-set!.
* module/system/vm/linker.scm (*lcm-page-size*): Rename
from *page-size*, change to 64 KB.
* libguile/loader.c (load_thunk_from_memory): Only require page size
alignment, knowing that although Guile might emit ELF with 64k
alignment, it only really needs page alignment.
Fixes <http://bugs.gnu.org/20272>.
* module/ice-9/boot-9.scm (module-generate-unique-id!)
(module-gensym): New procedures.
(module): Add 'next-unique-id' field.
(the-root-module): Inherit 'next-unique-id' value from early stub.
(make-module, make-autoload-interface): Adjust calls to
module-constructor.
* module/ice-9/psyntax.scm (gen-label, new-mark): Generate unique
identifiers from the module name and the per-module unique-id.
(build-lexical-var, generate-temporaries): Use
'module-gensym' instead of 'gensym'.
* module/ice-9/psyntax-pp.scm: Regenerate.
* module/language/tree-il/fix-letrec.scm (fix-letrec!): Use
'module-gensym' instead of 'gensym'.
* module/system/base/syntax.scm (define-record): Likewise.
(transform-record): Likewise.
Co-authored-by: Ludovic Courtès <ludo@gnu.org>
* module/system/base/target.scm (cpu-endianness, triplet-pointer-size):
Add case for "sh3".
Signed-off-by: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de>
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
Reported by Christopher Allan Webber <cwebber@dustycloud.org>
Co-authored-by: Ludovic Courtès <ludo@gnu.org>
This commit adds protection to Guile's REPL servers against HTTP
inter-protocol exploitation attacks, a scenario whereby an attacker can,
via an HTML page, cause a web browser to send data to TCP servers
listening on a loopback interface or private network. See
<https://en.wikipedia.org/wiki/Inter-protocol_exploitation> and
<https://www.jochentopf.com/hfpa/hfpa.pdf>, The HTML Form Protocol
Attack (2001) by Tochen Topf <jochen@remote.org>.
Here we add a procedure to 'before-read-hook' that looks for a possible
HTTP request-line in the first line of input from the client socket. If
present, the socket is drained and closed, and a loud warning is written
to stderr (POSIX file descriptor 2).
* module/system/repl/server.scm: Add 'maybe-check-for-http-request'
to 'before-read-hook' when this module is loaded.
(with-temporary-port-encoding, with-saved-port-line+column)
(drain-input-and-close, permissive-http-request-line?)
(check-for-http-request, guard-against-http-request)
(maybe-check-for-http-request): New procedures.
(serve-client): Use 'guard-against-http-request'.
* module/system/repl/coop-server.scm (start-repl-client): Use
'guard-against-http-request'.
* doc/ref/guile-invoke.texi (Command-line Options): In the description
of the --listen option, make the security warning more prominent.
Mention the new protection added here. Recommend using UNIX domain
sockets for REPL servers. "a path to" => "the file name of".
* libguile/snarf.h (SCM_IMMUTABLE_STRINGBUF): Remove shared flag.
Stringbufs are immutable by default.
* libguile/strings.c: Rewrite blurb. Change to have stringbufs be
immutable by default and mutable only when marked as such. Going
mutable means making a private copy.
(STRINGBUF_MUTABLE, STRINGBUF_F_MUTABLE): New definitions.
(SET_STRINGBUF_SHARED): Remove.
(scm_i_print_stringbuf): Simplify to just alias the stringbuf as-is.
(substring_with_immutable_stringbuf): New helper.
(scm_i_substring, scm_i_substring_read_only, scm_i_substring_copy):
use new helper.
(scm_i_string_ensure_mutable_x): New helper.
(scm_i_substring_shared): Use scm_i_string_ensure_mutable_x.
(stringbuf_write_mutex): Remove; yaaaaaaaay.
(scm_i_string_start_writing): Use scm_i_string_ensure_mutable_x. No
more mutex.
(scm_i_string_stop_writing): Now a no-op.
(scm_i_make_symbol): Use substring/copy.
(scm_sys_string_dump, scm_sys_symbol_dump): Update.
* libguile/strings.h (SCM_I_STRINGBUF_F_SHARED): Remove.
(SCM_I_STRINGBUF_F_MUTABLE): Add.
* module/system/vm/assembler.scm (link-data): Don't add shared flag any
more. Existing compiled flags are harmless tho.
* test-suite/tests/strings.test ("string internals"): Update.
* libguile/dynstack.h:
* libguile/dynstack.c (scm_dynstack_find_old_fluid_value): New
function.
* libguile/fluids.c (saved_dynamic_state_ref): New helper.
(scm_fluid_ref): Fix docstring.
(scm_fluid_ref_star): New function allowing access to previous values
for a fluid.
(scm_dynamic_state_ref): New internal function.
* libguile/fluids.h: Add scm_fluid_ref_star and scm_dynamic_state_ref.
* libguile/stacks.c (scm_stack_id): Adapt to %stacks not being a chain.
* libguile/throw.c (catch, throw_without_pre_unwind): Adapt to
%exception-handlers not being a chain.
* module/ice-9/boot-9.scm (catch, dispatch-exception): Instead of having
%exception-handlers be a chain, use fluid-ref* to access the chain
that is in place at the time the exception is thrown. Prevents
unintended undelimited capture of the current exception handler stack
by a delimited "catch".
(%start-stack): Similarly, don't be a chain.
* module/system/repl/debug.scm (frame->stack-vector):
* module/system/repl/error-handling.scm (call-with-error-handling):
* module/ice-9/save-stack.scm (save-stack): Adapt to %stacks not being a
chain.
* test-suite/tests/exceptions.test ("delimited exception handlers"): Add
tests.
* doc/ref/api-control.texi (Fluids and Dynamic States): Add docs.
