Reported by Dmitry Chestnykh <dmitry@codingrobots.com>.
Fixes <http://debbugs.gnu.org/10070>.
* libguile/bytevectors.c (scm_bytevector_copy_x): Use `memmove', not
`memcpy'.
* test-suite/tests/bytevectors.test ("2.2 General
Operations")["bytevector-copy! overlapping"]: New test.
* doc/ref/api-data.texi (Bytevector Manipulation): Mention possible
overlapping.
* libguile/ports.c (scm_putc, scm_puts):
* libguile/ports.h (scm_putc_unlocked, scm_puts_unlocked): Separate into
_unlocked and locked variants. Change all callers to use the
_unlocked versions.
* libguile/tags.h (SCM_HEAP_OBJECT_BASE): New macro. Given a SCM,
returns a pointer to the start of its memory area on the heap.
* libguile/bytevectors.c:
* libguile/fluids.c:
* libguile/foreign.c:
* libguile/gc.h:
* libguile/guardians.c:
* libguile/numbers.h:
* libguile/ports.c:
* libguile/smob.c:
* libguile/struct.c:
* libguile/weak-set.c:
* libguile/weak-table.c:
* libguile/weak-vector.c: Use it.
* libguile/tags.h (SCM_UNPACK_POINTER, SCM_PACK_POINTER): New macros.
The old SCM2PTR and PTR2SCM were defined in such a way that
round-tripping through a pointer could lose precision, even in the
case in which you weren't interested in actually dereferencing the
pointer, it was simply that you needed to plumb a SCM through APIs
that take pointers. These new macros are more like SCM_PACK and
SCM_UNPACK, but for pointer types. The bit representation of the
pointer should be the same as the scm_t_bits representation.
* libguile/gc.h (PTR2SCM, SCM2PTR): Remove support for (old) UNICOS
pointers. We are going to try tagging the SCM object itself in the
future, and I don't think that keeping this support is worth its
cost. It probably doesn't work anyway.
* libguile/backtrace.c:
* libguile/bytevectors.c:
* libguile/continuations.c:
* libguile/fluids.c:
* libguile/foreign.c:
* libguile/gc.h:
* libguile/guardians.c:
* libguile/hashtab.c:
* libguile/load.c:
* libguile/numbers.c:
* libguile/ports.c:
* libguile/smob.c:
* libguile/strings.c:
* libguile/symbols.c:
* libguile/vm.c:
* libguile/weak-set.c:
* libguile/weak-table.c:
* libguile/weak-vector.c: Update many sites to use the new macros.
* libguile/bytevectors.h:
* libguile/bytevectors.c (scm_c_take_gc_bytevector): Rename this
internal function, from scm_c_take_bytevector. This indicates that
unlike the other scm_take_* functions, this one takes GC-managed
memory.
* libguile/objcodes.c (scm_objcode_to_bytecode):
* libguile/vm.c (really_make_boot_program): Use
scm_gc_malloc_pointerless, not scm_malloc. Thanks to Stefan
Israelsson Tampe!
* libguile/r6rs-ports.c:
* libguile/strings.c: Adapt to renames.
* libguile/bytevectors.c (COMPLEX_ACCESSOR_PROLOGUE, COMPLEX_NATIVE_REF,
COMPLEX_NATIVE_SET): New macros.
(bytevector_ref_c32, bytevector_ref_c64): Defined in terms of
`COMPLEX_NATIVE_REF'.
(bytevector_set_c32, bytevector_set_c64): Defined in terms of
`COMPLEX_NATIVE_SET'.
(bytevector_ref_fns): Make `static'.
* test-suite/tests/srfi-4.test ("c32 vectors")["generalized-vector-ref",
"generalized-vector-set!", "generalized-vector-ref, out-of-range",
"generalized-vector-set!, out-of-range"]: New tests.
("c64 vectors")["generalized-vector-ref", "generalized-vector-set!",
"generalized-vector-ref, out-of-range",
"generalized-vector-set!, out-of-range"]: New tests.
* libguile/bytevectors.c (bytevector_ref_c32, bytevector_ref_c64,
bytevector_set_c32, bytevector_set_c64): Use `memcpy' to avoid
unaligned accesses. This fixes SIGBUS on SPARC and possibly other
alignment-sensitive platforms.
* libguile/__scm.h: Remove defines for SCM_I_SIZE_MAX, SCM_I_SSIZE_MIN,
and SCM_I_SSIZE_MAX, as there is no longer a scm_t_size / scm_size_t
type.
* libguile/bytevectors.c (make_bytevector): Replace a use of
SCM_I_SIZE_MAX with ((size_t) -1).
* libguile/bytevectors.h (SCM_BYTEVECTOR_HEADER_SIZE): Bump, giving
bytevectors another word: a parent pointer. Will allow for
sub-bytevectors and efficient mmap bindings.
* libguile/bytevectors.c (make_bytevector):
(make_bytevector_from_buffer): Init parent to #f.
(scm_c_take_bytevector, scm_c_take_typed_bytevector): Another
argument, the parent, which gets set in the bytevector.
* libguile/foreign.c (scm_pointer_to_bytevector): Use the parent field
instead of registering a weak reference from bytevector to foreign
pointer.
* libguile/objcodes.c (scm_objcode_to_bytecode): Use the parent field to
avoid copying the objcode.
* libguile/srfi-4.c (DEFINE_SRFI_4_C_FUNCS):
* libguile/strings.c (scm_from_stringn):
* libguile/vm.c (really_make_boot_program):
* libguile/r6rs-ports.c (scm_get_bytevector_some)
(scm_get_bytevector_all, bytevector_output_port_procedure): Set the
parent to #f.
Reported and fixed by Daniel Llorens <dll@bluewin.ch>.
* libguile/bytevectors.c (VALIDATE_REAL): Test for reals, not rationals.
* test-suite/tests/srfi-4.test (f32 vectors, f64 vectors): Add tests.
Reported and fixed by Daniel Llorens <dll@bluewin.ch>.
* libguile/bytevectors.c (VALIDATE_REAL): Test for reals, not rationals.
* test-suite/tests/srfi-4.test (f32 vectors, f64 vectors): Add tests.
* libguile/bytevectors.c:
* libguile/eval.c:
* libguile/goops.c:
* libguile/i18n.c:
* libguile/load.c:
* libguile/memoize.c:
* libguile/modules.c:
* libguile/ports.c:
* libguile/print.c:
* libguile/procs.c:
* libguile/programs.c:
* libguile/read.c:
* libguile/script.c:
* libguile/srfi-14.c:
* libguile/stacks.c:
* libguile/strings.c:
* libguile/throw.c:
* libguile/vm.c: Use scm_from_latin1_symboln to make symbols from string
literals, because they aren't in the user's locale -- they are in
ASCII, and we can optimize this case.
* libguile/vm-i-loader.c: Also use scm_from_latin1_symboln when loading
narrow symbols.
* libguile/gc-malloc.c: Add a note that the gc-malloc does not clear the
memory block, so users need to make sure it is initialized.
* libguile/bitvectors.c (scm_c_make_bitvector):
* libguile/bytevectors.c (scm_make_bytevector):
* libguile/strings.c (scm_c_make_string): If no initializer is given,
initialize the bytes to 0. Prevents information leakage if an app uses
make-string et al without initializers.
* libguile/foreign.c (make_cif): Initialize this too, to prevent leakage
in the struct holes. Paranoia...
* libguile/bytevectors.c:
* libguile/goops.c:
* libguile/instructions.c:
* libguile/numbers.c:
* libguile/random.c:
* libguile/read.c:
* libguile/vm-i-scheme.c: Fix a number of assumptions that a long could
hold an inum. This is not the case on platforms whose void* is larger
than their long.
* libguile/numbers.c (scm_i_inum2big): New helper, only implemented for
sizeof(void*) == sizeof(long); produces a compile error on other
platforms. Basically gmp doesn't have a nice interface for converting
between mpz values and intmax_t.
* libguile/array-map.c (array_compare, scm_array_equal_p): Rewrite as
something that operates on the generic array handle infrastructure.
Based on array->list.
(scm_i_array_equal_p): Change the docs, as array-equal? is now the same
as equal?, except that it typechecks its args.
* doc/ref/api-compound.texi (Array Procedures): Update array-equal?
docs.
* libguile/deprecated.h:
* libguile/deprecated.c (scm_raequal): Deprecate.
* libguile/bytevectors.c (scm_bytevector_eq_p): Bugfix: bytevectors are
bytevector=? only if their element type is the same.
* libguile/eq.c (scm_equal_p): Only dispatch to scm_array_equal_p if
both args are arrays (generically).
* test-suite/tests/arrays.test ("equal?"): Add some more tests.
* libguile/Makefile.am (lib_LTLIBRARIES): Instead of just "libguile.la",
make "libguile-@EFFECTIVE_VERSION@.la". This allows multiple versions
of Guile to be installed at once. See
http://www106.pair.com/rhp/parallel.html for a rationale.
(libguile_@GUILE_EFFECTIVE_VERSION@_la_CFLAGS):
(libguile_@GUILE_EFFECTIVE_VERSION@_la_SOURCES):
(EXTRA_libguile_@GUILE_EFFECTIVE_VERSION@_la_SOURCES):
(libguile_@GUILE_EFFECTIVE_VERSION@_la_DEPENDENCIES):
(libguile_@GUILE_EFFECTIVE_VERSION@_la_LIBADD):
(libguile_@GUILE_EFFECTIVE_VERSION@_la_LDFLAGS): Fixup automake vars
to include the effective version.
(guile_LDADD): Fix up the spelling of libguile.
* libguile/bytevectors.c (scm_bootstrap_bytevectors):
* libguile/foreign.c (scm_register_foreign):
* libguile/i18n.c (scm_bootstrap_i18n):
* libguile/instructions.c (scm_bootstrap_instructions):
* libguile/objcodes.c (scm_bootstrap_objcodes):
* libguile/programs.c (scm_bootstrap_programs):
* libguile/vm.c (scm_bootstrap_vm): Register extensions using e.g.
"libguile-2.0" as the libname -- i.e., including the effective version
in the libname.
* module/ice-9/i18n.scm:
* module/rnrs/bytevector.scm:
* module/rnrs/io/ports.scm:
* module/system/foreign.scm:
* module/system/vm/instruction.scm:
* module/system/vm/objcode.scm:
* module/system/vm/program.scm:
* module/system/vm/vm.scm: When doing a load-extension for something in
Guile, use the effective version also.
* meta/guile-2.0-uninstalled.pc.in (Libs):
* meta/guile-2.0.pc.in (Libs): Use -lguile-@EFFECTIVE_VERSION@. This
change should mean that code built against Guile should not be
affected by the libguile rename.
* guile-readline/Makefile.am (libguilereadline_v_@LIBGUILEREADLINE_MAJOR@_la_LIBADD):
* srfi/Makefile.am
(libguile_srfi_srfi_1_v_@LIBGUILE_SRFI_SRFI_1_MAJOR@_la_LIBADD):
(libguile_srfi_srfi_4_v_@LIBGUILE_SRFI_SRFI_4_MAJOR@_la_LIBADD):
(libguile_srfi_srfi_13_14_v_@LIBGUILE_SRFI_SRFI_13_14_MAJOR@_la_LIBADD):
(libguile_srfi_srfi_60_v_@LIBGUILE_SRFI_SRFI_60_MAJOR@_la_LIBADD):
* test-suite/standalone/Makefile.am (test_num2integral_LDADD):
(test_round_LDADD):
(libtest_asmobs_la_LIBADD):
(libtest_ffi_la_LIBADD):
(test_list_LDADD):
(test_unwind_LDADD):
(test_conversion_LDADD):
(test_loose_ends_LDADD):
(test_scm_c_read_LDADD):
(test_scm_take_locale_symbol_LDADD):
(test_scm_take_u8vector_LDADD):
(libtest_extensions_la_LIBADD):
(test_with_guile_module_LDADD):
(test_scm_with_guile_LDADD): Fix up the spelling of libguile.la.
* libguile/srfi-4.h:
* libguile/srfi-4.c (scm_make_srfi_4_vector): New function, exported by
(srfi srfi-4 gnu).
* libguile/srfi-4.i.c: Removed.
* module/srfi/srfi-4.scm:
* module/srfi/srfi-4/gnu.scm: Reimplement srfi-4 vectors on top of
bytevectors. The implementation is mostly in Scheme now.
* test-suite/tests/unif.test: Update to use (srfi srfi-4 gnu).
* libguile/bytevectors.c (bytevector_ref_c32, bytevector_ref_c64)
(bytevector_set_c32, bytevector_set_c64): Fix some embarrassing bugs.
Still need to do an upper bounds check.
* libguile/deprecated.h: Remove deprecated array functions:
scm_i_arrayp, scm_i_array_ndim, scm_i_array_mem, scm_i_array_v,
scm_i_array_base, scm_i_array_dims, and the deprecated macros:
SCM_ARRAYP, SCM_ARRAY_NDIM, SCM_ARRAY_CONTP, SCM_ARRAY_MEM,
SCM_ARRAY_V, SCM_ARRAY_BASE, SCM_ARRAY_DIMS.
* libguile/deprecated.c (scm_uniform_vector_read_x)
(scm_uniform_vector_write, scm_uniform_array_read_x)
(scm_uniform_array_write): Newly deprecated functions.
* libguile/generalized-arrays.c (scm_array_type): Remove the bytevector
hack.
* libguile/objcodes.c (scm_bytecode_to_objcode, scm_objcode_to_bytecode):
Rework to operate on bytevectors, as scm_make_u8vector now causes a
module lookup, which can't be done e.g. when loading the VM boot
program for psyntax-pp.go on a fresh bootstrap.
* libguile/objcodes.h (SCM_F_OBJCODE_IS_BYTEVECTOR):
(SCM_OBJCODE_IS_BYTEVECTOR): s/U8VECTOR/BYTEVECTOR/.
* module/ice-9/boot-9.scm (the-scm-module): A terrible hack to pull in
(srfi srfi-4), as the bindings are primarily there now. We'll worry
about this later.
* libguile/bytevectors.c (scm_bootstrap_bytevectors): Remove a call to
scm_gc_protect_object.
* libguile/gc.h:
* libguile/gc.c (scm_init_gc_protect_object): Rename from
scm_init_storage, and just return void. Make the GC boot procs have
internal linkage.
* libguile/init.c: Adapt to the name change.
The intent is to allow for mmap(3) bindings and to actually reuse
user-provided buffers in `scm_c_take_bytevector ()'.
* libguile/bytevectors.h (SCM_BYTEVECTOR_HEADER_SIZE): Increment.
(SCM_BYTEVECTOR_CONTENTS): Take the pointer from the second word.
(SCM_BYTEVECTOR_CONTIGUOUS_P): New macro.
(SCM_BYTEVECTOR_ELEMENT_TYPE): Adjust to live alongside the CONTIGUOUS
flag.
* libguile/bytevectors.c (SCM_BYTEVECTOR_SET_CONTENTS,
SCM_BYTEVECTOR_SET_CONTIGUOUS_P): New macros.
(SCM_BYTEVECTOR_SET_ELEMENT_TYPE): Adjust.
(SCM_BYTEVECTOR_TYPED_LENGTH): Properly parenthesize.
(make_bytevector): Adjust to new bytevector header.
(make_bytevector_from_buffer): Reuse CONTENTS.
(scm_c_shrink_bytevector): Differentiate between contiguous and
non-contiguous bytevectors.
The bytevector to string conversion functions were accomplished
by converting via locale strings. This did not allow conversions
of wide strings in an 8-bit locale. This is avoided by using knowledge
of the storage format of the string.
* libguile/bytevectors.c (STRING_TO_UTF, scm_string_to_utf8): modify
string to bytevector conversion to use internal string information
(UTF_TO_STRING, scm_utf8_to_string): modify bytevector to string
conversion
* libguile/arrays.c (scm_from_contiguous_typed_array):
* libguile/bytevectors.c (scm_uniform_array_to_bytevector): Error if the
uniform element size is more than 8 bits, but not divisible by 8 --
because our math could overflow in that case.
* module/ice-9/deprecated.scm (#\y): Indeed, #* is the valid bitvector
syntax :)
* libguile/bytevectors.c (scm_uniform_array_to_bytevector): Fix BYTE_LEN
computation for bitvectors.
* test-suite/tests/bytevectors.test ("uniform-array->bytevector"): New
test prefix.
* libguile/uniform.h (scm_array_handle_uniform_element_bit_size): New
public accessor.
* libguile/uniform.c (scm_array_handle_uniform_element_size): Better
errors in non-byte-aligned arrays.
(scm_uniform_vector_element_type, scm_uniform_vector_element_size)
(scm_c_uniform_vector_ref, scm_c_uniform_vector_set_x):
(scm_uniform_vector_to_list): Don't require byte-aligned access.
* libguile/bytevectors.c (scm_uniform_array_to_bytevector):
* libguile/arrays.c (scm_from_contiguous_typed_array): Fix for
uniform arrays whose element size is not a multiple of the byte size.
* libguile/bytevectors.c (SCM_BYTEVECTOR_INLINE_THRESHOLD,
SCM_BYTEVECTOR_INLINEABLE_SIZE_P, SCM_BYTEVECTOR_SET_CONTENTS,
SCM_BYTEVECTOR_SET_INLINE): Remove.
(SCM_BYTEVECTOR_HEADER_BYTES): New macro.
(SCM_BYTEVECTOR_SET_ELEMENT_TYPE): Adjust to new flag layout.
(make_bytevector): Remove content inlining machinery; use
`scm_gc_malloc_pointerless ()' in all cases; special-case zero-sized
vu8 buffers.
(make_bytevector_from_buffer): Simplified.
(scm_c_shrink_bytevector): New, formerly `scm_i_shrink_bytevector ()'.
Remove buffer inlining machinery.
(scm_bootstrap_bytevectors): Use `make_bytevector ()' for
SCM_NULL_BYTEVECTOR.
* libguile/bytevectors.h (SCM_BYTEVECTOR_HEADER_SIZE): New macro.
(SCM_BYTEVECTOR_CONTENTS): Adjust to new layout.
(SCM_SET_BYTEVECTOR_FLAGS): Properly cast F.
(SCM_F_BYTEVECTOR_INLINE, SCM_BYTEVECTOR_INLINE_P): Remove.
(SCM_BYTEVECTOR_ELEMENT_TYPE): Adjust.
(scm_c_shrink_bytevector): Remove macro, make a C function
declaration.
* libguile/bytevectors.c (VALIDATE_REAL): SCM_VALIDATE_REAL is not what
we need for checking values for bytevector-ieee-single-native-set! et
al, so define our own validator.
(IEEE754_SET, IEEE754_NATIVE_SET): Use it.
Bytevectors have a very close relationship to other forms of uniform
vectors. Often you want to view a u64vector as a series of bytes, for
writing over a socket; or to process an incoming stream using the
convenient and less error-prone s16vector-ref API rather than
bytevector-s16-native-ref.
The essential needs of the representation of a bytevector and an
s64vector are the same, so we take advantage of that and extend the
bytevector implementation to have a "native type" field, which defaults
to VU8.
This commit doesn't actually expose any user-noticeable changes,
however.
* libguile/bytevectors.h (SCM_BYTEVECTOR_ELEMENT_TYPE): New internal
defines.
(scm_i_make_typed_bytevector, scm_c_take_typed_bytevector): New
internal functions.
* libguile/bytevectors.c (SCM_BYTEVECTOR_SET_ELEMENT_TYPE):
(SCM_BYTEVECTOR_TYPE_SIZE):
(SCM_BYTEVECTOR_TYPED_LENGTH): New internal macros.
(make_bytevector, make_bytevector_from_buffer): Take an extra
argument, the element type. The length argument is interpreted as
being the number of elements, which corresponds to the number of bytes
in the default VU8 case. Doing it this way eliminates a class of bugs
-- e.g. a u32vector of length 3 bytes doesn't make sense. We do have
to check for another class of bugs: overflow. The length stored on the
bytevector itself is still the byte length, though.
(scm_i_make_typed_bytevector):
(scm_c_take_typed_bytevector): New internal functions.
(scm_i_shrink_bytevector): Make sure the new size is valid for the
bytevector's type.
(scm_i_bytevector_generalized_set_x): Remove this function, the
array-handle infrastructure takes care of this for us.
(print_bytevector): Print the bytevector according to its type.
(scm_make_bytevector, scm_bytevector_copy)
(scm_uniform_array_to_bytevector)
(scm_u8_list_to_bytevector, scm_bytevector_to_uint_list): Adapt to
make_bytevector extra arg.
(bv_handle_ref, bv_handle_set_x): Adapt to ref and set based on the
type of the bytevector, e.g. f64 or u8.
(bytevector_get_handle): Set the typed length of the vector, not the
byte length.
Conflicts:
libguile/bytevectors.c
* libguile/bytevectors.h (SCM_BYTEVECTOR_INLINE_P): Change to check a
flag instead of checking the length of the bytevector.
* libguile/bytevectors.c (make_bytevector_from_buffer): Handle the len
<= inline threshold case as well. Set the inline flag as appropriate.
(make_bytevector): Updat the inline flag as appropriate.
(scm_c_take_bytevector): Just dispatch to make_bytevector_from_buffer.
(scm_i_shrink_bytevector): Update the inline flag as appropriate.
Update the length when shrinking an already-inlined vector.
(STRING_TO_UTF): Fix some indentation.
* libguile/generalized-vectors.h:
* libguile/generalized-vectors.c: Add a registry of vector constructors.
(scm_make_generalized_vector): New public function, constructs a
vector of a given type.
* libguile/bitvectors.c:
* libguile/bytevectors.c:
* libguile/srfi-4.c:
* libguile/strings.c:
* libguile/vectors.c: Register vector constructors.
* libguile/extensions.c (scm_init_extensions): No need to NULL the list
of registered extensions here, the static init does it for us. Allows
scm_c_register_extension to be called before scm_init_extensions.
* libguile/init.c (scm_i_init_guile): Move array initialization earlier,
so e.g. scm_init_strings has access to a valid list of array element
types when registering its vector constructor.
* libguile/array-handle.c (scm_i_register_array_implementation):
(scm_i_array_implementation_for_obj): Add generic array facility,
which will (in a few commits) detangle the array code.
(scm_array_get_handle): Use the generic array facility. Note that
scm_t_array_handle no longer has ref and set function pointers;
instead it has a pointer to the array implementation. It is unlikely
that code out there used these functions, however, as the supported
way was through scm_array_handle_ref/set_x.
(scm_array_handle_pos): Move this function here from arrays.c.
(scm_array_handle_element_type): New function, returns a Scheme value
representing the type of element stored in this array.
* libguile/array-handle.h (scm_t_array_element_type): New enum, for
generically determining the type of an array.
(scm_array_handle_rank):
(scm_array_handle_dims): These are now just #defines.
* libguile/arrays.c:
* libguile/bitvectors.c:
* libguile/bytevectors.c:
* libguile/srfi-4.c:
* libguile/strings.c:
* libguile/vectors.c: Register array implementations for all of these.
* libguile/inline.h: Update for array_handle_ref/set change.
* libguile/deprecated.h: Need to include arrays.h now.
* libguile/bytevectors.h (scm_i_native_endianness): Allow the VM to use
scm_i_native_endianness, but still keep it marked as internal.
* libguile/bytevectors.c: Adjust to use scm_i_native_endianness instead
of native_endianness. Define it at bootstrap time.
