* libguile/ports-internal.h (scm_port_buffer_reset_end): New helper.
(scm_port_buffer_putback): New helper.
* libguile/ports.h (scm_t_port_buffer): Remove "buf" field.
(scm_get_byte_or_eof_unlocked, scm_peek_byte_or_eof_unlocked): Adapt.
* libguile/ports.c (scm_c_make_port_buffer): No more "buf" field.
(scm_i_unget_bytes_unlocked): Use helper.
* libguile/read.c (scm_i_scan_for_encoding): No more "buf" field.
* libguile/ports.h (scm_t_port_buffer): Rename has_eof member to
has_eof_p, and be a Scheme value, in anticipation of moving the port
buffers to be Scheme objects.
* module/ice-9/ports.scm: New file.
* am/bootstrap.am (SOURCES): Add ice-9/ports.scm.
* libguile/fports.c (scm_init_ice_9_fports): New function.
(scm_init_fports): Arrange for scm_init_ice_9_fports to be called via
load-extension, and load snarfed things there. Move open-file
definition early, to allow ports to bootstrap.
* libguile/ioext.c (scm_init_ice_9_ioext): New function.
(scm_init_ioext): Similarly, register scm_init_ice_9_ioext as an
extension.
* libguile/ports.c (scm_set_current_input_port)
(scm_set_current_output_port, scm_set_current_error_port): Don't
define Scheme bindings; do that in Scheme.
* libguile/ports.c (scm_i_set_default_port_encoding):
(scm_i_default_port_encoding, scm_i_default_port_conversion_handler):
(scm_i_set_default_port_conversion_handler): Since we now init
encoding early, remove the "init" flags on these encoding/strategy
vars.
(scm_init_ice_9_ports): New function.
(scm_init_ports): Register scm_init_ice_9_ports extension, and define
some bindings needed by the bootstrap.
* module/Makefile.am (SOURCES): Add ice-9/ports.scm.
* module/ice-9/boot-9.scm: Remove code that's not on the boot path,
moving it to ice-9/ports.scm. At the end, load (ice-9 ports).
* module/ice-9/psyntax.scm (include): Use close-port instead of
close-input-port.
* module/ice-9/psyntax-pp.scm (include): Regenerate.
* libguile/ports.c (scm_i_write_bytes_unlocked): Allow incomplete
writes from the implementation.
(scm_c_write_bytes_unlocked): Use scm_i_write_bytes_unlocked helper to
call the write function.
* libguile/r6rs-ports.c (custom_binary_output_port_write): Don't loop;
core Guile will do that.
This will allow better Scheme integration for ports.
* libguile/ports.h (scm_t_port_buffer): Change "holder" member to be a
bytevector defined to have "buf" as its starting point.
(scm_t_ptob_descriptor): Change read and write functions to take
bytevectors as arguments and to return the number of octets read or
written.
(scm_make_port_type): Adapt accordingly.
(scm_c_read_bytes, scm_c_write_bytes): New functions that take
bytevectors.
* libguile/ports.c (scm_make_port_type): Adapt to read/write function
prototype change.
(scm_c_make_port_buffer): Arrange to populate the "bytevector" field.
(scm_i_read_bytes_unlocked): New function.
(scm_i_read_unlocked): Use scm_i_read_bytes_unlocked.
(scm_c_read_bytes_unlocked): New function.
(scm_c_read_unlocked): Update comment, and always go through the
buffer.
(scm_c_read_bytes): New function.
(scm_flush_unlocked): Use scm_i_write_unlocked instead of the port's
write function.
(scm_i_write_bytes_unlocked): New function.
(scm_i_write_unlocked): Use scm_i_write_bytes_unlocked.
(scm_c_write_bytes_unlocked): New function.
(scm_c_write_unlocked): Always write through the buffer.
(scm_c_write_bytes): New function.
(scm_truncate_file): Remove unused variable.
(void_port_read, void_port_write): Adapt to read/write prototype
change.
* libguile/fports.c (fport_read, fport_write):
* libguile/r6rs-ports.c (bytevector_input_port_read)
(custom_binary_input_port_read, bytevector_output_port_write)
(custom_binary_output_port_write, transcoded_port_write)
(transcoded_port_read): Adapt to read/write prototype
change.
(scm_get_bytevector_n, scm_get_bytevector_n_x)
(scm_get_bytevector_all): Use scm_c_read_bytes.
(scm_put_bytevector): Use scm_c_write_bytes.
* libguile/strports.c (string_port_read, string_port_write):
* libguile/vports.c (soft_port_write, soft_port_read): Adapt to
read/write prototype change.
* test-suite/standalone/test-scm-c-read.c (custom_port_read): Fix for
read API change.
* libguile/ports.h (struct scm_t_port_buffer): New data type.
(struct scm_t_port): Refactor to use port buffers instead of
implementation-managed read and write pointers. Add "read_buffering"
member.
(SCM_INITIAL_PUTBACK_BUF_SIZE, SCM_READ_BUFFER_EMPTY_P): Remove.
(scm_t_ptob_descriptor): Rename "fill_input" function to "read", and
take a port buffer, returning void. Likewise "write" takes a port
buffer and returns void. Remove "end_input"; instead if there is
buffered input and rw_random is true, then there must be a seek
function, so just seek back if needed. Remove "flush"; instead all
calls to the "write" function implicitly include a "flush", since the
buffering happens in the generic port code now. Remove "setvbuf", but
add "get_natural_buffer_sizes"; instead the generic port code can
buffer any port.
(scm_make_port_type): Adapt to read and write prototype changes.
(scm_set_port_flush, scm_set_port_end_input, scm_set_port_setvbuf):
Remove.
(scm_slow_get_byte_or_eof_unlocked)
(scm_slow_get_peek_or_eof_unlocked): Remove; the slow path is to call
scm_fill_input.
(scm_set_port_get_natural_buffer_sizes): New function.
(scm_c_make_port_buffer): New internal function.
(scm_port_non_buffer): Remove. This was a function for
implementations that is no longer needed. Instead open with BUF0 or
use (setvbuf port 'none).
(scm_fill_input, scm_fill_input_unlocked): Return the filled port
buffer.
(scm_get_byte_or_eof_unlocked, scm_peek_byte_or_eof_unlocked): Adapt
to changes in buffering and EOF management.
* libguile/ports.c: Adapt to port interface changes.
(initialize_port_buffers): New function, using the port mode flags to
set up appropriate initial buffering for all ports.
(scm_c_make_port_with_encoding): Create port buffers here instead of
delegating to implementations.
(scm_close_port): Flush the port if needed instead of delegating to
the implementation.
* libguile/filesys.c (set_element): Adapt to buffering changes.
* libguile/fports.c (fport_get_natural_buffer_sizes): New function,
replacing scm_fport_buffer_add.
(fport_write, fport_read): Update to let the generic ports code do the
buffering.
(fport_flush, fport_end_input): Remove.
(fport_close): Don't flush in a dynwind; that's the core ports' job.
(scm_make_fptob): Adapt.
* libguile/ioext.c (scm_redirect_port): Adapt to buffering changes.
* libguile/poll.c (scm_primitive_poll): Adapt to buffering changes.
* libguile/ports-internal.h (struct scm_port_internal): Remove
pending_eof flag; this is now set on the read buffer.
* libguile/r6rs-ports.c (struct bytevector_input_port): New type. The
new buffering arrangement means that there's now an intermediate
buffer between the bytevector and the user of the port; this could
lead to a perf degradation, but on the other hand there are some other
speedups enabled by the buffering refactor, so probably the memcpy
cost is dwarfed by the cost of the other parts of the ports
machinery.
(make_bytevector_input_port, bytevector_input_port_read):
(bytevector_input_port_seek, initialize_bytevector_input_ports): Adapt
to new buffering arrangement.
(struct custom_binary_port): Remove read buffer, as Guile handles that
now.
(custom_binary_input_port_setvbuf): Remove; now handled by Guile.
(make_custom_binary_input_port, custom_binary_input_port_read)
(initialize_custom_binary_input_ports): Adapt.
(scm_get_bytevector_some): Adapt to new EOF management.
(scm_t_bytevector_output_port_buffer): Hold on to the underlying port,
so we can flush it if it's open.
(make_bytevector_output_port, bytevector_output_port_write):
(bytevector_output_port_seek): Adapt.
(bytevector_output_port_procedure): Flush the port as appropriate, so
that we get all the bytes.
(make_custom_binary_output_port, custom_binary_output_port_write):
Adapt.
(make_transcoded_port): Don't muck with buffering.
(transcoded_port_write): Simply forward the write to the underlying
port.
(transcoded_port_read): Likewise.
(transcoded_port_close): No need to flush.
(initialize_transcoded_ports): Adapt.
* libguile/read.c (scm_i_scan_for_encoding): Adapt to buffering
changes.
* libguile/rw.c (scm_write_string_partial): Adapt to buffering changes.
* libguile/strports.c: Adapt to the fact that we don't manage the
buffer. Probably room for speed improvements here...
* libguile/vports.c (soft_port_get_natural_buffer_sizes): New function.
Adapt the rest of the file for the new buffering regime.
* test-suite/tests/r6rs-ports.test ("8.2.10 Output ports"): Custom
binary output ports need to be flushed before you can rely on the
write! procedure having been called. Add necessary flush-port
invocations.
("8.2.6 Input and output ports"): Transcoded ports now have an
internal buffer by default. This test checks that the characters are
transcoded one at a time, so to do that, call setvbuf on the
transcoded port to remove the buffer.
* test-suite/tests/web-client.test (run-with-http-transcript): Fix for
different flushing regime on soft ports. (The vestigial flush
procedure is now called after each write, which is not what the test
was expecting.)
* test-suite/standalone/test-scm-c-read.c: Update for changes to the C
interface for defining port types.
* doc/ref/api-io.texi (Ports): Update to discuss buffering in a generic
way, and to remove a hand-wavey paragraph describing string ports as
"interesting and powerful".
(Reading, Writing): Remove placeholder comments. Document
`scm_lfwrite'.
(Buffering): New section.
(File Ports): Link to buffering.
(I/O Extensions): Join subnodes into parent and describe new API,
including buffering API.
* doc/ref/posix.texi (Ports and File Descriptors): Link to buffering.
Remove unread-char etc, as they are documented elsewhere.
(Pipes, Network Sockets and Communication): Link to buffering.
* libguile/ports.h (scm_t_ptob_descriptor): The port close function now
returns void.
(scm_set_port_close): Adapt prototype.
* libguile/ports.c (scm_close_port): Always return true if we managed to
call the close function. There's no other sensible result; exceptions
are handled, well, exceptionally.
* libguile/fports.c (fport_close)
* libguile/r6rs-ports.c (custom_binary_port_close, transcoded_port_close):
* libguile/vports.c (soft_port_close): Adapt.
* doc/ref/api-io.texi (Port Implementation): Update.
* libguile/ports.c (scm_seek): Flush before seeking on a buffered port.
* libguile/fports.c (fport_seek):
* libguile/strports.c (st_seek): Remove code to flush buffers.
* test-suite/tests/ports.test: Update test expectations that the
putback buffer is flushed on a seek. Previously there was a special
case for SEEK_CUR with an offset of 0 to avoid flushing buffers, but
that's an arbitrary choice that differs from all other combinations of
OFFSET and WHENCE.
* libguile/fports.c (fport_flush, fport_end_input): Move rw_active
handling to ports.c.
* libguile/ioext.c (scm_redirect_port): Use scm_flush_unlocked instead
of calling the flush function directly.
* libguile/ports.c (scm_c_make_port_with_encoding): Ports default to
"rw_random" mode when they have a seek function.
(scm_c_read_unlocked, scm_i_unget_bytes_unlocked)
(scm_slow_get_byte_or_eof_unlocked)
(scm_slow_peek_byte_or_eof_unlocked): Flush write buffer and set
rw_active always in the same way, and only if rw_random is true.
(scm_end_input_unlocked, scm_flush_unlocked): Clear rw_active here
unconditionally.
(scm_c_write_unlocked): Flush read buffer and set rw_active always in
the same way, but only if rw_random is true.
(scm_c_write, scm_lfwrite): Whitespace fixes.
(scm_lfwrite_substr): Don't flush read buffer; lower-level code will
do this.
(scm_truncate_file): Use scm_flush_unlocked instead of calling the
flush function directly.
* libguile/r6rs-ports.c (transcoded_port_flush): Don't muck with
rw_active.
* libguile/read.c (scm_i_scan_for_encoding): Flush write buffer if
needed in same way as ports.c.
* libguile/strports.c (st_end_input): Don't muck with rw_active.
(scm_mkstrport): rw_random defaults to 1 now.
* libguile/ports.h (scm_t_port_type_flags): Replace
SCM_PORT_TYPE_HAS_FLUSH with SCM_PORT_TYPE_NEEDS_CLOSE_ON_GC.
(scm_t_ptob_descriptor): Remove free function.
* libguile/ports.c (scm_set_port_needs_close_on_gc): New function.
(scm_set_port_flush): Don't set flags.
(scm_c_make_port_with_encoding, scm_close_port): Use the new flag to
determine when to add a finalizer and also when to include the port in
the weak set.
(scm_set_port_free): Remove.
(do_close, finalize_port): Close port instead of calling free
function.
* libguile/r6rs-ports.c (initialize_transcoded_ports):
* libguile/vports.c (scm_make_sfptob):
* libguile/fports.c (scm_make_fptob): Mark these ports as needing close
on GC.
* libguile/fports.c (fport_free): Remove.
* NEWS: Update.
* doc/ref/api-io.texi (Port Implementation): Update.
* libguile/ports.c (scm_current_input_port, scm_current_output_port)
(scm_current_error_port): Fix declarations to C99.
(scm_current_warning_port, scm_set_current_warning_port): Rework to
use a C fluid, like scm_current_error_port.
(scm_init_ports): Initialize and define the warning port fluid.
* libguile/init.c (scm_init_standard_ports): Init the current warning
port.
* module/ice-9/boot-9.scm: Remove definitions for current-warning-port.
Instead, steal it from the boot objtable with port-parameterize!.
Reported by Doug Evans <xdje42@gmail.com>.
* libguile/ports.c (scm_port_conversion_strategy): Don't validate
that 'port' is an open port until after the (port == #f) case
has been handled.
* libguile/ports-internal.h (struct scm_port_internal): Add
setvbuf' field. Change 'pending_eof' to a 1-bit unsigned char.
* libguile/ports.c (scm_new_port_table_entry): Clear 'pti->setvbuf'.
* libguile/fports.c (scm_setvbuf): Accept any open port, and error out
when PORT's setvbuf' field is NULL. Remove explicit 'scm_gc_free' calls.
(scm_i_fdes_to_port): Set PORT's 'setvbuf' field.
* test-suite/tests/ports.test ("setvbuf")["closed port", "string port"]:
New tests.
* doc/ref/posix.texi (Ports and File Descriptors): Suggest that
'setvbuf' works for different port types.
* libguile/ports.c (scm_i_port_property, scm_i_set_port_property_x):
Lock the port mutex while accessing the port alist.
* libguile/read.c (set_port_read_option): Lock the port mutex
while modifying port read options.
* libguile/ports.c (scm_peek_char): Don't require error cases to
set *len. Fixes a bug caught by GCC whereby an EOF while reading a
multibyte character with iconv would leave len uninitialized.
* libguile/ports.c (scm_i_port_alist, scm_i_set_port_alist_x): Removed.
(scm_i_port_property, scm_i_set_port_property_x): New procedures,
available from Scheme as '%port-property' and '%set-port-property!'.
* libguile/ports.h (scm_i_port_alist, scm_i_set_port_alist_x): Removed.
(scm_i_port_property, scm_i_set_port_property_x): New prototypes.
* libguile/read.c (set_port_read_option, init_read_options): Adapt to
use scm_i_port_property and scm_i_set_port_property_x.
* libguile/ports.c (scm_ungetc_unlocked): Inline the conversion from
codepoint to bytes for UTF-8 and latin-1 ports. Speeds up a
numbers-reading test case by 100% (!).
* libguile/ports.c (scm_i_unget_bytes): New static function.
(scm_unget_bytes): New API function.
(scm_unget_byte): Rewrite to simply call 'scm_i_unget_bytes'.
(scm_ungetc, scm_peek_char, looking_at_bytes): Use 'scm_i_unget_bytes'.
* libguile/ports.h: Add prototype for 'scm_unget_bytes'.
* libguile/fports.c (scm_setvbuf): Use 'scm_unget_bytes'.
* libguile/r6rs-ports.c (scm_unget_bytevector): New procedure.
* module/ice-9/binary-ports.scm (unget-bytevector): New export.
* doc/ref/api-io.texi (R6RS Binary Input): Add documentation.
(R6RS I/O Ports): Update brief description of (ice-9 binary-ports) to
reflect the new reality: it is no longer a subset of (rnrs io ports).
* test-suite/tests/ports.test ("unget-bytevector"): Add test.
* libguile/ports-internal.h (struct scm_port_internal): Add new members
'at_stream_start_for_bom_read' and 'at_stream_start_for_bom_write'.
(SCM_UNICODE_BOM): New macro.
(scm_i_port_iconv_descriptors): Add 'mode' parameter to prototype.
* libguile/ports.c (scm_new_port_table_entry): Initialize
'at_stream_start_for_bom_read' and 'at_stream_start_for_bom_write'.
(get_iconv_codepoint): Pass new 'mode' parameter to
'scm_i_port_iconv_descriptors'.
(get_codepoint): After reading a codepoint at stream start, record
that we're no longer at stream start, and consume a BOM where
appropriate.
(scm_seek): Set the stream start flags according to the new position.
(looking_at_bytes): New static function.
(scm_utf8_bom, scm_utf16be_bom, scm_utf16le_bom, scm_utf32be_bom,
scm_utf32le_bom): New static const arrays.
(decide_utf16_encoding, decide_utf32_encoding): New static functions.
(scm_i_port_iconv_descriptors): Add new 'mode' parameter. If the
specified encoding is UTF-16 or UTF-32, make that precise by deciding
what byte order to use, and construct iconv descriptors based on the
precise encoding.
(scm_i_set_port_encoding_x): Record that we are now at stream start.
Do not open the new iconv descriptors immediately; let them be
initialized lazily.
* libguile/print.c (display_string_using_iconv): Record that we're no
longer at stream start. Write a BOM if appropriate.
* doc/ref/api-io.texi (BOM Handling): New node.
* test-suite/tests/ports.test ("set-port-encoding!, wrong encoding"):
Adapt test to cope with the fact that 'set-port-encoding!' does not
immediately open the iconv descriptors.
(bv-read-test): New procedure.
("unicode byte-order marks (BOMs)"): New test prefix.
* libguile/ports.c (scm_i_set_port_encoding_x): Always copy the
user-provided port encoding string, so that its case will be preserved
and returned exactly by subsequent calls to 'port-encoding'.
* libguile/ports.c (scm_i_get_byte_or_eof, scm_i_peek_byte_or_eof):
Rename to 'scm_slow_get_byte_or_eof' and 'scm_slow_peek_byte_or_eof',
respectively.
* libguile/ports.h (scm_i_get_byte_or_eof, scm_i_peek_byte_or_eof):
Rename to 'scm_slow_get_byte_or_eof' and 'scm_slow_peek_byte_or_eof',
respectively, and mark them as SCM_API.
* libguile/inline.h (scm_get_byte_or_eof, scm_peek_byte_or_eof): Adjust
to use the new names.
* libguile/ports.c (get_iconv_codepoint): Rewrite to fix a bug and
improve efficiency and clarity. Previously, it incorrectly assumed
that iconv would never consume input without producing output, which
led to a buffer overrun and subsequent assertion failure. This
happens when a byte-order mark is consumed by iconv at the beginning
of the stream when using the UTF-16 or UTF-32 encodings.
* test-suite/tests/ports.test (unicode byte-order marks (BOMs)):
Add tests.
Suggested by Andy Wingo.
* libguile/inline.h (scm_get_byte_or_eof, scm_peek_byte_or_eof): Keep
only the fast path here, with fallback to 'scm_i_get_byte_or_eof' and
'scm_i_peek_byte_or_eof'.
* libguile/ports.c (scm_i_get_byte_or_eof, scm_i_peek_byte_or_eof):
New internal functions.
* libguile/ports.h (scm_i_get_byte_or_eof, scm_i_peek_byte_or_eof): Add
prototypes.
* libguile/ports.c (scm_i_fill_input): New static function, containing
the code that was previously in 'scm_fill_input'.
(scm_fill_input): Simply call 'scm_i_fill_input'.
(scm_c_read): Use 'scm_i_fill_input'.
* libguile/ports-internal.h (struct scm_port_internal): Add 'alist'
member.
* libguile/ports.c (scm_i_port_alist, scm_i_set_port_alist_x): New
internal functions.
(scm_i_port_weak_hash): Update comment: the hash table is no longer
used to store the port's alist.
(scm_new_port_table_entry): Initialize 'alist'. Store SCM_BOOL_F in
the port weak hash, not SCM_EOL.
* libguile/ports.h (scm_i_port_alist, scm_i_set_port_alist_x): Add
protoypes.
* libguile/read.c (set_port_read_option, init_read_options): Access the
port's alist via 'scm_i_port_alist' and 'scm_i_set_port_alist_x'.
Based on 6c98257f2e by Andy Wingo.
* libguile/ports-internal.h (struct scm_port_internal): Add a flag
for the port encoding mode: UTF8 or iconv. The iconv descriptors
are now in a separate structure so that we can avoid attaching
finalizers to the ports themselves in the future.
(enum scm_port_encoding_mode): New enum.
(struct scm_iconv_descriptors): New struct.
(scm_i_port_iconv_descriptors): Add prototype.
* libguile/ports.c (finalize_port): Don't close iconv descriptors here.
(scm_new_port_table_entry): Adapt to the iconv descriptors being
moved. Initialize 'encoding_mode'.
(scm_i_remove_port): Adapt to call 'close_iconv_descriptors'.
(close_iconv_descriptors): New static function.
(get_iconv_codepoint): Use 'scm_i_port_iconv_descriptors'.
(get_codepoint): Check the port 'encoding_mode'.
(finalize_iconv_descriptors, open_iconv_descriptors,
close_iconv_descriptors, scm_i_port_iconv_descriptors): New static
functions.
(scm_i_set_port_encoding_x): Adapt to iconv descriptors being moved
to separate structure, to set the 'encoding_mode' flag, and to use
'open_iconv_descriptors' and 'close_iconv_descriptors'.
* libguile/print.c (display_string_using_iconv): Use
'scm_i_port_iconv_descriptors'.
(display_string): Use 'encoding_mode' flag.
* libguile/ports-internal.h: New file.
* libguile/Makefile.am (noinst_HEADERS): Add ports-internal.h.
* libguile/ports.h (scm_t_port): Add a comment mentioning that the
'input_cd' and 'output_cd' fields of the public structure are no
longer what they seem to be.
* libguile/ports.c: Include ports-internal.h.
(finalize_port, scm_i_remove_port, get_iconv_codepoint, get_codepoint,
scm_i_set_port_encoding_x): Access 'input_cd' and 'output_cd' via the
new internal port structure.
(scm_new_port_table_entry): Allocate and initialize the internal port
structure.
* libguile/print.c: Include ports-internal.h.
(display_string_using_iconv, display_string): Access 'input_cd' and
'output_cd' via 'internal' pointer.
