* libguile/ports.c (scm_i_set_pending_eof): Remove now-unused helper.
(peek_utf8_codepoint, peek_latin1_codepoint, peek_iconv_codepoint):
(peek_codepoint): Refactor the fundamental character readers in Guile
to peek into the read buffer instead of reading then unreading. This
will allow Scheme to use the port buffer to convert, when we port this
to Scheme.
(get_codepoint): Use peek_codepoint.
(scm_getc): Adapt.
(scm_peek_char): Use peek_codepoint.
* libguile/ports.h (scm_fill_input): Add "minimum_size" argument. Adapt
all callers to pass 0 as this argument.
* libguile/ports.c (scm_i_read): Inline into scm_fill_input.
(scm_fill_input): "minimum_size" argument ensures that there are a
certain number of bytes available, or EOF. Instead of shrinking the
read buffer, only fill by the read_buffering amount, or the
minimum_size, whichever is larger.
* libguile/r6rs-ports.c:
* libguile/read.c: Adapt scm_fill_input callers.
* libguile/ports.c (trampoline_to_c_read, trampoline_to_c_write): Since
C might assume that the indices are within bounds of the bytevector,
verify them more here.
(scm_port_random_access_p, scm_port_read_buffering)
(scm_set_port_read_buffer, scm_port_read, scm_port_write): New helpers
exposed to (ice-9 ports).
(scm_port_read_buffer, scm_port_write_buffer): Don't flush or validate
port mode; we do that in Scheme.
* module/ice-9/ports.scm: Implement enough of port machinery to
implement peek-byte in Scheme. Not yet exported.
* libguile/init.c (scm_i_init_guile): Initialize ports before
strports/fports, so that we have initialized the read/write
trampolines before making port types.
* libguile/ports.c (scm_init_ice_9_ports): Define the-eof-object here.
Update a comment.
* module/ice-9/ports.scm: Use the-eof-object definition from C.
* libguile/ports.h (scm_t_ptob_descriptor): Add "scm_read" and
"scm_write" members, for calling from Scheme.
(scm_set_port_scm_read, scm_set_port_scm_write): New procedures.
* libguile/ports.c (trampoline_to_c_read_subr)
(trampoline_to_c_write_subr): New static variables.
* libguile/ports.c (scm_make_port_type): Initialize scm_read and
scm_write members to trampoline to C.
(trampoline_to_c_read, trampoline_to_scm_read)
(trampoline_to_c_write, trampoline_to_scm_write): New helpers.
(scm_set_port_scm_read, scm_set_port_scm_write): New functions.
(default_buffer_size): Move definition down.
(scm_i_read_bytes, scm_i_write_bytes): Use new names for read and
write procedures.
(scm_init_ports): Initialize trampolines.
* libguile/ports.c (scm_peek_char): Optimize. A loop calling peek-char
on a buffered string port 10e6 times goes down from 50ns/iteration to
32ns/iteration.
* libguile/print.c (scm_write, scm_display):
* libguile/read.c (set_port_read_option): Remove port locking. Reading
and writing to the same port from multiple threads just must not
crash; it doesn't have to make sense.
* libguile/ports.h (scm_unget_bytes_unlocked, scm_unget_byte_unlocked):
Remove.
* libguile/ports.c (looking_at_bytes): Use scm_unget_bytes instead of
scm_i_unget_bytes_unlocked
(scm_unget_bytes): Rename from scm_i_unget_bytes_unlocked. Remove
other implementations of this function.
(scm_unget_byte): Likewise.
(scm_ungetc_unlocked, scm_peek_char): Use scm_unget_byte.
* libguile/read.c (read_token): Use scm_unget_byte.
* libguile/ports.h (scm_getc_unlocked): Remove, or rather rename to
scm_getc. This probably introduces some thread-related bugs but we'll
fix them in a different way.
* libguile/ports.c (scm_getc): Rename from scm_getc_unlocked, replacing
the locky implementation.
(scm_read_char): Use scm_getc.
* libguile/r6rs-ports.c (scm_get_string_n_x): Use scm_getc.
* libguile/rdelim.c (scm_read_delimited_x, scm_read_line): Use
scm_getc.
* libguile/read.c: Use scm_getc.
* libguile/ports.h (scm_flush_unlocked, scm_end_input_unlocked):
Remove.
* libguile/ports.c (scm_c_read_bytes_unlocked):
(scm_i_unget_bytes_unlocked, scm_setvbuf, scm_force_output)
(scm_fill_input_unlocked, scm_c_write_bytes_unlocked)
(scm_c_write_unlocked, scm_lfwrite_unlocked, scm_seek)
(scm_truncate_file, flush_output_port): Call scm_flush / scm_end_input
instead of the _unlocked variants.
(scm_end_input): Lock while discarding the input buffer but not while
calling out to the seek function.
* libguile/filesys.c (scm_fsync):
* libguile/ioext.c (scm_redirect_port):
* libguile/read.c (scm_i_scan_for_encoding):
* libguile/rw.c (scm_write_string_partial): Use scm_flush, not
scm_flush_unlocked.
* libguile/ports.h (scm_c_read_unlocked): Remove.
* libguile/ports.c (scm_c_read): Rename from scm_c_read_unlocked.
Remove old scm_c_read. Lock around access to the rw_active flag, and
call scm_flush instead of scm_flush_unlocked, and scm_fill_input
instead of scm_fill_input_unlocked.
* libguile/read.c (scm_i_scan_for_encoding): Use scm_c_read instead of
the _unlocked function.
* libguile/ports.h (scm_get_byte_or_eof_unlocked)
(scm_peek_byte_or_eof_unlocked): Remove inline functions. The
important uses are in ports.c anyway and we will use a static function
there.
(scm_slow_get_byte_or_eof_unlocked)
(scm_slow_peek_byte_or_eof_unlocked): Remove declarations without
definitions.
* libguile/ports.c (looking_at_bytes): Use scm_peek_byte_or_eof instead
of the _unlocked variant.
(get_byte_or_eof, peek_byte_or_eof): New static functions.
(scm_get_byte_or_eof, scm_peek_byte_or_eof): Don't lock: the port
buffer mechanism means that we won't crash. More comments to come.
(get_utf8_codepoint, get_latin1_codepoint, get_iconv_codepoint): Use
new static functions.
* libguile/read.c (read_token, scm_read_semicolon_comment): Use
scm_get_byte_or_eof, not scm_get_byte_or_eof_unlocked.
* libguile/ports-internal.h (scm_port_buffer_bytevector)
(scm_port_buffer_cur, scm_port_buffer_set_cur)
(scm_port_buffer_end, scm_port_buffer_set_end)
(scm_port_buffer_has_eof_p, scm_port_buffer_set_has_eof_p): New
helpers.
* libguile/ports-internal.h (scm_port_buffer_size)
(scm_port_buffer_reset, scm_port_buffer_reset_end)
(scm_port_buffer_can_take, scm_port_buffer_can_put)
(scm_port_buffer_can_putback, scm_port_buffer_did_take)
(scm_port_buffer_did_put, scm_port_buffer_take_pointer)
(scm_port_buffer_put_pointer, scm_port_buffer_take)
(scm_port_buffer_put, scm_port_buffer_putback): Adapt to treat port
buffers as SCM values and use helpers to access them.
* libguile/ports.c (scm_i_clear_pending_eof, scm_i_set_pending_eof)
(scm_c_make_port_buffer, scm_i_read_unlocked)
(scm_c_read_bytes_unlocked, scm_i_unget_bytes_unlocked)
(scm_setvbuf, scm_fill_input, scm_take_from_input_buffers)
(scm_drain_input, scm_end_input_unlocked, scm_flush_unlocked)
(scm_fill_input_unlocked, scm_i_write_unlocked)
(scm_c_write_bytes_unlocked, scm_c_write_unlocked)
(scm_char_ready_p): Adapt to treat port buffers as SCM values and use
helpers to access them.
(scm_port_read_buffer, scm_port_write_buffer): New functions,
allowing (ice-9 ports) to access port buffers.
* libguile/ports.h: Update comments on port buffers. Replace
scm_t_port_buffer structure with a Scheme vector whose fields are
enumerated by "enum scm_port_buffer_field".
(scm_get_byte_or_eof_unlocked, scm_peek_byte_or_eof_unlocked): Adapt
these implementations to port buffer representation change.
* libguile/r6rs-ports.c (scm_get_bytevector_some):
* libguile/read.c (scm_i_scan_for_encoding):
* libguile/rw.c (scm_write_string_partial): Port buffers are Scheme
objects.
* libguile/ports-internal.h (scm_port_buffer_size): Verify that the
bytevector field is a bytevector, in anticipation of Schemification.
(scm_port_buffer_can_take, scm_port_buffer_can_put)
(scm_port_buffer_can_putback): Enforce invariants on cur and end
here.
(scm_port_buffer_did_take, scm_port_buffer_did_put): Relax to not call
other functions.
* libguile/ports.h (scm_get_byte_or_eof_unlocked)
(scm_peek_byte_or_eof_unlocked): Refactor to call no functions on the
fast path.
* libguile/ports.h (scm_t_port_buffer): Change "cur" and "end" members
to be SCM values, in preparation for changing port buffers to be
Scheme vectors.
(scm_get_byte_or_eof_unlocked, scm_peek_byte_or_eof_unlocked): Adapt.
* libguile/ports.c (scm_c_make_port_buffer): Initialize cur and end
members.
(looking_at_bytes): Use helper instead of incrementing cur.
(scm_i_read_unlocked): Adapt to end type change.
(CONSUME_PEEKED_BYTE): Use helper instead of incrementing cur.
(scm_i_unget_bytes_unlocked): Use helper instead of comparing cur.
(scm_i_write_unlocked): Fix for changing end/cur types.
* libguile/read.c (scm_i_scan_for_encoding): Use helpers instead of
addressing cursors directly.
* libguile/rw.c (scm_write_string_partial): Likewise.
* libguile/ports-internal.h (scm_port_buffer_reset):
(scm_port_buffer_reset_end, scm_port_buffer_can_take):
(scm_port_buffer_can_put, scm_port_buffer_can_putback):
(scm_port_buffer_did_take, scm_port_buffer_did_put):
(scm_port_buffer_take_pointer, scm_port_buffer_put_pointer):
(scm_port_buffer_putback): Adapt to data types.
* libguile/ports-internal.h (scm_port_buffer_reset_end): New helper.
(scm_port_buffer_putback): New helper.
* libguile/ports.h (scm_t_port_buffer): Remove "buf" field.
(scm_get_byte_or_eof_unlocked, scm_peek_byte_or_eof_unlocked): Adapt.
* libguile/ports.c (scm_c_make_port_buffer): No more "buf" field.
(scm_i_unget_bytes_unlocked): Use helper.
* libguile/read.c (scm_i_scan_for_encoding): No more "buf" field.
* libguile/ports.h (scm_t_port_buffer): Rename has_eof member to
has_eof_p, and be a Scheme value, in anticipation of moving the port
buffers to be Scheme objects.
* libguile/vports.c (struct soft_port): Inline the encoding buffer so as
to not use scm_t_port_buffer, in anticipation of changing the port
buffer representations. Adapt users.
* module/ice-9/ports.scm: New file.
* am/bootstrap.am (SOURCES): Add ice-9/ports.scm.
* libguile/fports.c (scm_init_ice_9_fports): New function.
(scm_init_fports): Arrange for scm_init_ice_9_fports to be called via
load-extension, and load snarfed things there. Move open-file
definition early, to allow ports to bootstrap.
* libguile/ioext.c (scm_init_ice_9_ioext): New function.
(scm_init_ioext): Similarly, register scm_init_ice_9_ioext as an
extension.
* libguile/ports.c (scm_set_current_input_port)
(scm_set_current_output_port, scm_set_current_error_port): Don't
define Scheme bindings; do that in Scheme.
* libguile/ports.c (scm_i_set_default_port_encoding):
(scm_i_default_port_encoding, scm_i_default_port_conversion_handler):
(scm_i_set_default_port_conversion_handler): Since we now init
encoding early, remove the "init" flags on these encoding/strategy
vars.
(scm_init_ice_9_ports): New function.
(scm_init_ports): Register scm_init_ice_9_ports extension, and define
some bindings needed by the bootstrap.
* module/Makefile.am (SOURCES): Add ice-9/ports.scm.
* module/ice-9/boot-9.scm: Remove code that's not on the boot path,
moving it to ice-9/ports.scm. At the end, load (ice-9 ports).
* module/ice-9/psyntax.scm (include): Use close-port instead of
close-input-port.
* module/ice-9/psyntax-pp.scm (include): Regenerate.
* libguile/struct.c (scm_init_struct): Use scm_from_latin1_string to
avoid locale-dependency for what is really a latin1 string. Also
avoids an early dependency on the default port conversion handler,
though I wonder if using port conversion handlers in strings is the
right thing.
* libguile/ports.c (scm_i_write_bytes_unlocked): Allow incomplete
writes from the implementation.
(scm_c_write_bytes_unlocked): Use scm_i_write_bytes_unlocked helper to
call the write function.
* libguile/r6rs-ports.c (custom_binary_output_port_write): Don't loop;
core Guile will do that.
This will allow better Scheme integration for ports.
* libguile/ports.h (scm_t_port_buffer): Change "holder" member to be a
bytevector defined to have "buf" as its starting point.
(scm_t_ptob_descriptor): Change read and write functions to take
bytevectors as arguments and to return the number of octets read or
written.
(scm_make_port_type): Adapt accordingly.
(scm_c_read_bytes, scm_c_write_bytes): New functions that take
bytevectors.
* libguile/ports.c (scm_make_port_type): Adapt to read/write function
prototype change.
(scm_c_make_port_buffer): Arrange to populate the "bytevector" field.
(scm_i_read_bytes_unlocked): New function.
(scm_i_read_unlocked): Use scm_i_read_bytes_unlocked.
(scm_c_read_bytes_unlocked): New function.
(scm_c_read_unlocked): Update comment, and always go through the
buffer.
(scm_c_read_bytes): New function.
(scm_flush_unlocked): Use scm_i_write_unlocked instead of the port's
write function.
(scm_i_write_bytes_unlocked): New function.
(scm_i_write_unlocked): Use scm_i_write_bytes_unlocked.
(scm_c_write_bytes_unlocked): New function.
(scm_c_write_unlocked): Always write through the buffer.
(scm_c_write_bytes): New function.
(scm_truncate_file): Remove unused variable.
(void_port_read, void_port_write): Adapt to read/write prototype
change.
* libguile/fports.c (fport_read, fport_write):
* libguile/r6rs-ports.c (bytevector_input_port_read)
(custom_binary_input_port_read, bytevector_output_port_write)
(custom_binary_output_port_write, transcoded_port_write)
(transcoded_port_read): Adapt to read/write prototype
change.
(scm_get_bytevector_n, scm_get_bytevector_n_x)
(scm_get_bytevector_all): Use scm_c_read_bytes.
(scm_put_bytevector): Use scm_c_write_bytes.
* libguile/strports.c (string_port_read, string_port_write):
* libguile/vports.c (soft_port_write, soft_port_read): Adapt to
read/write prototype change.
* test-suite/standalone/test-scm-c-read.c (custom_port_read): Fix for
read API change.
* libguile/ports.h (struct scm_t_port_buffer): New data type.
(struct scm_t_port): Refactor to use port buffers instead of
implementation-managed read and write pointers. Add "read_buffering"
member.
(SCM_INITIAL_PUTBACK_BUF_SIZE, SCM_READ_BUFFER_EMPTY_P): Remove.
(scm_t_ptob_descriptor): Rename "fill_input" function to "read", and
take a port buffer, returning void. Likewise "write" takes a port
buffer and returns void. Remove "end_input"; instead if there is
buffered input and rw_random is true, then there must be a seek
function, so just seek back if needed. Remove "flush"; instead all
calls to the "write" function implicitly include a "flush", since the
buffering happens in the generic port code now. Remove "setvbuf", but
add "get_natural_buffer_sizes"; instead the generic port code can
buffer any port.
(scm_make_port_type): Adapt to read and write prototype changes.
(scm_set_port_flush, scm_set_port_end_input, scm_set_port_setvbuf):
Remove.
(scm_slow_get_byte_or_eof_unlocked)
(scm_slow_get_peek_or_eof_unlocked): Remove; the slow path is to call
scm_fill_input.
(scm_set_port_get_natural_buffer_sizes): New function.
(scm_c_make_port_buffer): New internal function.
(scm_port_non_buffer): Remove. This was a function for
implementations that is no longer needed. Instead open with BUF0 or
use (setvbuf port 'none).
(scm_fill_input, scm_fill_input_unlocked): Return the filled port
buffer.
(scm_get_byte_or_eof_unlocked, scm_peek_byte_or_eof_unlocked): Adapt
to changes in buffering and EOF management.
* libguile/ports.c: Adapt to port interface changes.
(initialize_port_buffers): New function, using the port mode flags to
set up appropriate initial buffering for all ports.
(scm_c_make_port_with_encoding): Create port buffers here instead of
delegating to implementations.
(scm_close_port): Flush the port if needed instead of delegating to
the implementation.
* libguile/filesys.c (set_element): Adapt to buffering changes.
* libguile/fports.c (fport_get_natural_buffer_sizes): New function,
replacing scm_fport_buffer_add.
(fport_write, fport_read): Update to let the generic ports code do the
buffering.
(fport_flush, fport_end_input): Remove.
(fport_close): Don't flush in a dynwind; that's the core ports' job.
(scm_make_fptob): Adapt.
* libguile/ioext.c (scm_redirect_port): Adapt to buffering changes.
* libguile/poll.c (scm_primitive_poll): Adapt to buffering changes.
* libguile/ports-internal.h (struct scm_port_internal): Remove
pending_eof flag; this is now set on the read buffer.
* libguile/r6rs-ports.c (struct bytevector_input_port): New type. The
new buffering arrangement means that there's now an intermediate
buffer between the bytevector and the user of the port; this could
lead to a perf degradation, but on the other hand there are some other
speedups enabled by the buffering refactor, so probably the memcpy
cost is dwarfed by the cost of the other parts of the ports
machinery.
(make_bytevector_input_port, bytevector_input_port_read):
(bytevector_input_port_seek, initialize_bytevector_input_ports): Adapt
to new buffering arrangement.
(struct custom_binary_port): Remove read buffer, as Guile handles that
now.
(custom_binary_input_port_setvbuf): Remove; now handled by Guile.
(make_custom_binary_input_port, custom_binary_input_port_read)
(initialize_custom_binary_input_ports): Adapt.
(scm_get_bytevector_some): Adapt to new EOF management.
(scm_t_bytevector_output_port_buffer): Hold on to the underlying port,
so we can flush it if it's open.
(make_bytevector_output_port, bytevector_output_port_write):
(bytevector_output_port_seek): Adapt.
(bytevector_output_port_procedure): Flush the port as appropriate, so
that we get all the bytes.
(make_custom_binary_output_port, custom_binary_output_port_write):
Adapt.
(make_transcoded_port): Don't muck with buffering.
(transcoded_port_write): Simply forward the write to the underlying
port.
(transcoded_port_read): Likewise.
(transcoded_port_close): No need to flush.
(initialize_transcoded_ports): Adapt.
* libguile/read.c (scm_i_scan_for_encoding): Adapt to buffering
changes.
* libguile/rw.c (scm_write_string_partial): Adapt to buffering changes.
* libguile/strports.c: Adapt to the fact that we don't manage the
buffer. Probably room for speed improvements here...
* libguile/vports.c (soft_port_get_natural_buffer_sizes): New function.
Adapt the rest of the file for the new buffering regime.
* test-suite/tests/r6rs-ports.test ("8.2.10 Output ports"): Custom
binary output ports need to be flushed before you can rely on the
write! procedure having been called. Add necessary flush-port
invocations.
("8.2.6 Input and output ports"): Transcoded ports now have an
internal buffer by default. This test checks that the characters are
transcoded one at a time, so to do that, call setvbuf on the
transcoded port to remove the buffer.
* test-suite/tests/web-client.test (run-with-http-transcript): Fix for
different flushing regime on soft ports. (The vestigial flush
procedure is now called after each write, which is not what the test
was expecting.)
* test-suite/standalone/test-scm-c-read.c: Update for changes to the C
interface for defining port types.
* doc/ref/api-io.texi (Ports): Update to discuss buffering in a generic
way, and to remove a hand-wavey paragraph describing string ports as
"interesting and powerful".
(Reading, Writing): Remove placeholder comments. Document
`scm_lfwrite'.
(Buffering): New section.
(File Ports): Link to buffering.
(I/O Extensions): Join subnodes into parent and describe new API,
including buffering API.
* doc/ref/posix.texi (Ports and File Descriptors): Link to buffering.
Remove unread-char etc, as they are documented elsewhere.
(Pipes, Network Sockets and Communication): Link to buffering.
* libguile/ports.h (scm_t_ptob_descriptor): The port close function now
returns void.
(scm_set_port_close): Adapt prototype.
* libguile/ports.c (scm_close_port): Always return true if we managed to
call the close function. There's no other sensible result; exceptions
are handled, well, exceptionally.
* libguile/fports.c (fport_close)
* libguile/r6rs-ports.c (custom_binary_port_close, transcoded_port_close):
* libguile/vports.c (soft_port_close): Adapt.
* doc/ref/api-io.texi (Port Implementation): Update.
* libguile/vports.c (struct soft_port): New data structure.
(scm_make_soft_port): Unpack vector into struct when making soft
port.
(soft_port_input_waiting, soft_port_close, soft_port_fill_input):
(soft_port_write, soft_port_flush): Adapt. Remove an extraneous
errno=0 in soft_port_close.
* libguile/poll.c (scm_primitive_poll): A buffered port's buffer marks
it as writable only when writing a byte would not block, which is the
case only if there is more than one byte free in the buffer; writing
the last byte would block.
* libguile/ports.c (scm_seek): Flush before seeking on a buffered port.
* libguile/fports.c (fport_seek):
* libguile/strports.c (st_seek): Remove code to flush buffers.
* test-suite/tests/ports.test: Update test expectations that the
putback buffer is flushed on a seek. Previously there was a special
case for SEEK_CUR with an offset of 0 to avoid flushing buffers, but
that's an arbitrary choice that differs from all other combinations of
OFFSET and WHENCE.
* libguile/fports.c (fport_flush, fport_end_input): Move rw_active
handling to ports.c.
* libguile/ioext.c (scm_redirect_port): Use scm_flush_unlocked instead
of calling the flush function directly.
* libguile/ports.c (scm_c_make_port_with_encoding): Ports default to
"rw_random" mode when they have a seek function.
(scm_c_read_unlocked, scm_i_unget_bytes_unlocked)
(scm_slow_get_byte_or_eof_unlocked)
(scm_slow_peek_byte_or_eof_unlocked): Flush write buffer and set
rw_active always in the same way, and only if rw_random is true.
(scm_end_input_unlocked, scm_flush_unlocked): Clear rw_active here
unconditionally.
(scm_c_write_unlocked): Flush read buffer and set rw_active always in
the same way, but only if rw_random is true.
(scm_c_write, scm_lfwrite): Whitespace fixes.
(scm_lfwrite_substr): Don't flush read buffer; lower-level code will
do this.
(scm_truncate_file): Use scm_flush_unlocked instead of calling the
flush function directly.
* libguile/r6rs-ports.c (transcoded_port_flush): Don't muck with
rw_active.
* libguile/read.c (scm_i_scan_for_encoding): Flush write buffer if
needed in same way as ports.c.
* libguile/strports.c (st_end_input): Don't muck with rw_active.
(scm_mkstrport): rw_random defaults to 1 now.
* libguile/r6rs-ports.c (struct custom_binary_port): Use a struct
instead of a vector as the state for a custom binary port. Adapt all
callers. Some whitespace fixes as well.
* libguile/r6rs-ports.c: Expand out the acronyms "bip", "bop", "cbip",
"cbop", and "tp". They always confused me, especially that the "b" in
cbip/cbop wasn't the same as the one in bip/bop.
* libguile/ports.h (scm_t_port_type_flags): Replace
SCM_PORT_TYPE_HAS_FLUSH with SCM_PORT_TYPE_NEEDS_CLOSE_ON_GC.
(scm_t_ptob_descriptor): Remove free function.
* libguile/ports.c (scm_set_port_needs_close_on_gc): New function.
(scm_set_port_flush): Don't set flags.
(scm_c_make_port_with_encoding, scm_close_port): Use the new flag to
determine when to add a finalizer and also when to include the port in
the weak set.
(scm_set_port_free): Remove.
(do_close, finalize_port): Close port instead of calling free
function.
* libguile/r6rs-ports.c (initialize_transcoded_ports):
* libguile/vports.c (scm_make_sfptob):
* libguile/fports.c (scm_make_fptob): Mark these ports as needing close
on GC.
* libguile/fports.c (fport_free): Remove.
* NEWS: Update.
* doc/ref/api-io.texi (Port Implementation): Update.
