1
Fork 0
mirror of https://git.savannah.gnu.org/git/guile.git synced 2025-05-05 15:10:27 +02:00
Commit graph

19988 commits

Author SHA1 Message Date
Andy Wingo
1e058add7b U+FFFD is the input substitution character
* libguile/ports.c (UNICODE_REPLACEMENT_CHARACTER):
* libguile/ports.c (peek_utf8_codepoint)
  (scm_port_decode_char, peek_iconv_codepoint):
* module/ice-9/sports.scm (peek-char-and-len/utf8):
  (peek-char-and-len/iconv): Return U+FFFD when we get a decoding error
  when reading, instead of '?', in accordance with Unicode
  recommendations.
* test-suite/tests/iconv.test:
* test-suite/tests/ports.test:
* test-suite/tests/rdelim.test: Update tests.
* NEWS: Update.
2016-05-16 10:48:35 +02:00
Andy Wingo
da456d23be Bump objcode version
* libguile/_scm.h (SCM_OBJCODE_MINOR_VERSION):
* module/system/vm/assembler.scm (*bytecode-minor-version*): Bump for
  Guile 2.1.3.
2016-05-16 09:38:09 +02:00
Andy Wingo
d6922b4af4 Update NEWS for release
* NEWS: Try to tell the port story better.
2016-05-15 22:26:34 +02:00
Andy Wingo
745cbb4918 Remove unused type from ports-internal
* libguile/ports-internal.h: Remove unused scm_t_port_rw_active.
* libguile/deprecated.h (scm_port_rw_active): Remove deprecation shim,
  as this thing is just gone now.
2016-05-14 23:46:17 +02:00
Andy Wingo
556ac9777b Document scm_c_make_port and friends
* doc/ref/api-io.texi (I/O Extensions): Document scm_c_make_port and
  friends, and document "mode bits".
2016-05-14 23:27:38 +02:00
Andy Wingo
a9d0fe9ea1 Update port mode bits documentation.
* libguile/ports.h: Update documentation of port mode bits.
2016-05-14 23:26:50 +02:00
Andy Wingo
9ecf77a82d Add SCM_OPN to mode bits when making ports
* libguile/ports.c (scm_c_make_port_with_encoding): Add SCM_OPN to mode
  bits, so that users don't have to.
  (scm_i_mode_bits_n):
* libguile/print.c (scm_simple_format)
* libguile/r6rs-ports.c (make_bytevector_input_port)
  (make_custom_binary_input_port, make_bytevector_output_port)
  (make_custom_binary_output_port, make_transcoded_port)
* libguile/strports.c (scm_object_to_string, scm_open_input_string)
  (scm_open_output_string, scm_c_read_string): Remove now-unneeded
  SCM_OPN mentions.
2016-05-14 12:42:17 +02:00
Andy Wingo
9322902d02 Update port type documentation
* doc/ref/api-io.texi (I/O Extensions): Update for port type change.
2016-05-13 18:34:12 +02:00
Andy Wingo
cd51ce81d0 Use heap-allocated port types instead of ptobnums
This removes a limitation on the number of port types, simplifies the
API, and removes a central point of coordination.

* libguile/ports-internal.h (struct scm_t_port_type): Rename from
  scm_t_ptob_descriptor, now that it's private.  Add GOOPS class
  fields.
  (struct scm_t_port): Rename from struct scm_port, especially
  considering that deprecated.h redefines scm_port using the
  preprocessor :(.
* libguile/ports.h: Add definitions of SCM_PORT and SCM_PORT_TYPE,
  though the scm_t_port and scm_t_port_type types are incomplete.
  (SCM_TC2PTOBNUM, SCM_PTOBNUM, SCM_PTOBNAME): Remove, as there are no
  more typecodes for port types.
  (scm_c_num_port_types, scm_c_port_type_ref, scm_c_port_type_add_x):
  Remove.
  (scm_make_port_type): Return a scm_t_port_type*.  All methods adapted
  to take a scm_t_port_type* instead of a ptobnum.
  (scm_c_make_port_with_encoding, scm_c_make_port): Take a port type
  pointer instead of a tag.
  (scm_new_port_table_entry): Remove; not useful.
* libguile/ports.c: Remove things related to the port kind table.  Adapt
  uses of SCM_PORT_DESCRIPTOR / scm_t_ptob_descriptor to use
  SCM_PORT_TYPE and scm_t_port_type.
* libguile/deprecated.c:
* libguile/deprecated.h:
* libguile/filesys.c:
* libguile/fports.c:
* libguile/fports.h:
* libguile/print.c:
* libguile/r6rs-ports.c:
* libguile/strports.c:
* libguile/strports.h:
* libguile/tags.h:
* libguile/vports.c:
* test-suite/standalone/test-scm-c-read.c: Adapt to change.
* libguile/goops.c (scm_class_of, make_port_classes)
  (scm_make_port_classes, create_port_classes): Adapt to store the
  classes in the ptob.
2016-05-13 18:31:29 +02:00
Andy Wingo
17f90360b6 Make scm_t_ptob_descriptor private.
* libguile/goops.c: Use port internals header.
* libguile/ports-internal.h (scm_t_port_type_flags)
  (struct scm_t_ptob_descriptor): Make private.
* libguile/ports.h: Adapt.
2016-05-13 17:14:59 +02:00
Andy Wingo
af1c443f83 Update port documentation
* doc/ref/api-io.texi: Update for refactorings.
2016-05-13 16:48:02 +02:00
Andy Wingo
08574987d9 Remove scm_t_port_internal
* libguile/ports-internal.h (SCM_PORT): Rename from SCM_PTAB_ENTRY.
  (scm_t_port_internal, SCM_PORT_GET_INTERNAL): Remove.
  (SCM_FILENAME, SCM_SET_FILENAME, SCM_LINUM, SCM_COL): Adapt.
* libguile/ports.c:
* libguile/poll.c:
* libguile/ioext.c:
* libguile/fports.c:
* libguile/filesys.c:
* libguile/print.c:
* libguile/read.c:
* libguile/rw.c:
* libguile/strings.c: Adapt.
2016-05-13 11:34:52 +02:00
Andy Wingo
e5d2f4e566 Make scm_t_port private
* libguile/ports-internal.h (enum scm_port_encoding_mode): Remove unused
  enum.
  (scm_t_port_internal, scm_t_port): Make encoding and
  conversion_strategy private. Instead of scm_t_port_internal containing
  scm_t_port, now that all members are private, we can store the user's
  "stream" in a word in the port object itself and make the whole of
  scm_t_port private.  The next commit will remove scm_t_port_internal.
  (SCM_PTAB_ENTRY, SCM_PORT_DESCRIPTOR): Make private.
* libguile/ports.c (scm_c_make_port_with_encoding): Adapt to new port
  layout.
  (scm_port_print): Use SCM_PTAB_ENTRY when printing.
* libguile/ports.h: Remove scm_t_port definition.
* libguile/ioext.c (get_matching_port): Simplify.
* libguile/fports.c (scm_i_evict_port): Simplify.
2016-05-13 11:27:37 +02:00
Andy Wingo
9a9e0cceae Make port buffering fields private
* libguile/ports-internal.h (enum scm_port_buffer_field)
  (scm_t_port_internal): Make port buffering fields private.
* libguile/ports.h (scm_t_port): Adapt.
* libguile/filesys.c (set_element):
* libguile/ioext.c (scm_redirect_port):
* libguile/poll.c (scm_primitive_poll):
* libguile/ports.c:
* libguile/read.c (scm_i_scan_for_encoding):
* libguile/rw.c (scm_write_string_partial): Adapt users.
2016-05-13 10:58:19 +02:00
Andy Wingo
8af64975be Make file/line/column fields of ports private
* libguile/ports-internal.h (scm_t_port_internal): Move file_name,
  line_number, and column_number here.
  (SCM_FILENAME, SCM_SET_FILENAME, SCM_LINUM, SCM_COL, SCM_INCLINE):
  (SCM_ZEROCOL, SCM_INCCOL, SCM_DECCOL, SCM_TABCOL): Make internal.
* libguile/ports.c (scm_c_make_port_with_encoding)
  (scm_set_port_line_x, scm_set_port_column_x): Adapt to change.
2016-05-13 10:33:17 +02:00
Andy Wingo
209d50c7d8 Embed scm_t_port in scm_t_port_internal
* libguile/ports-internal.h (scm_t_port_internal)
* libguile/ports.h (scm_t_port): Embed scm_t_port in scm_t_port_internal
  so that we have just one allocation.
* libguile/ports-internal.h (SCM_PORT_GET_INTERNAL): Adapt.
* libguile/ports.c (scm_i_port_property, scm_i_set_port_property_x)
  (scm_c_make_port_with_encoding): Adapt.
2016-05-13 10:24:32 +02:00
Andy Wingo
fe7ceff969 Remove SCM backlink in port structure
* libguile/ports.h (scm_t_port): Remove port backlink.
* libguile/ports.c (scm_c_make_port_with_encoding, scm_fill_input)
  (scm_seek): Adapt.
2016-05-13 10:16:27 +02:00
Andy Wingo
6ff542ee71 Add random_access_p port type method
* doc/ref/api-io.texi (I/O Extensions): Update documentation on
  implementing port types.  Document get_natural_buffer_sizes.  Document
  the new random_access_p.
* libguile/fports.c (scm_i_fdes_to_port, fport_random_access_p):
  (scm_make_fptob): Instead of frobbing rw_random manually, implement a
  random_access_p function.
* libguile/ports.c (default_random_access_p)
  (scm_set_port_random_access_p): New functions.
  scm_make_port_type, scm_c_make_port_with_encoding): Arrange for
  random_access_p to work.
2016-05-13 10:05:23 +02:00
Andy Wingo
704c42870d Add (ice-9 sports) module
* module/ice-9/sports.scm: New module.
* module/Makefile.am (SOURCES): Add new module.
2016-05-13 09:22:36 +02:00
Andy Wingo
d1bb400c3f Beginnings of shunting ports-in-scheme off to a module
* libguile/ports.c (scm_specialize_port_encoding_x): Add some sanity
  checks.
  (scm_unget_bytes): Use scm_expand_port_read_buffer_x.
  (port_clear_stream_start_for_bom_read): Use
  scm_specialize_port_encoding_x.
  (scm_fill_input): Use scm_expand_port_read_buffer_x.
  (scm_expand_port_read_buffer_x): Rename from
  scm_set_port_read_buffer_x and actually expand the buffer.
* libguile/ports.h: Adapt to scm_expand_port_read_buffer_x change.

* module/ice-9/ports.scm: Remove ports-in-scheme stuff, and instead
  expose the ports internals via an auxiliary module.  This will let
  ports-in-scheme live in a module during Guile 2.2.
2016-05-13 09:09:42 +02:00
Andy Wingo
df0dade9b7 Implement lookahead-u8, get-u8 in Scheme
* module/ice-9/ports.scm (%lookahead-u8, %get-u8): Scheme
  implementations.
2016-05-12 08:57:01 +02:00
Andy Wingo
a8fe0f42f3 %read-char speedup
* module/ice-9/ports.scm (%read-char): Always call update-position! with
  the same continuation, so that it will contify.
2016-05-10 16:30:50 +02:00
Andy Wingo
d28d1a57bf Implement read-char in Scheme.
* module/ice-9/ports.scm (%read-char): New function.
2016-05-10 15:38:30 +02:00
Andy Wingo
ab21af544a %peek-char port argument optional.
* module/ice-9/ports.scm (%peek-char): Port argument is optional.
2016-05-10 15:38:11 +02:00
Andy Wingo
3ccfa213c1 peek-char-and-len/iconv uses both returns from fill-input
* module/ice-9/ports.scm (peek-char-and-len/iconv): Use buffered value
  from fill-input.
2016-05-10 15:37:42 +02:00
Andy Wingo
9c02ede07e Fix bad-utf8-len bug.
* module/ice-9/ports.scm (bad-utf8-len): Fix bug.
2016-05-10 15:37:11 +02:00
Andy Wingo
0dd18191bc clear-stream-start-for-bom-read refactor
* module/ice-9/ports.scm (clear-stream-start-for-bom-read): Use the
  "buffered" value that fill-input returns.
2016-05-10 15:36:31 +02:00
Andy Wingo
502e3a2213 Fix port-clear-stream-start-for-bom-read logic.
* libguile/ports.c (scm_port_clear_stream_start_for_bom_read): Fix
  logic.
2016-05-10 15:36:06 +02:00
Andy Wingo
837a7e0810 Scheme peek-char can handle iconv encodings
* module/ice-9/ports.scm (peek-char-and-len/iconv): Fully implement.
2016-05-10 12:48:10 +02:00
Andy Wingo
8ee189980d Factor out iconv to port-decode-char
* libguile/ports.c (scm_port_decode_char): New helper, exported
  to (ice-9 ports).
  (peek_iconv_codepoint): Use scm_port_decode_char.
2016-05-10 12:45:56 +02:00
Andy Wingo
08c67dbef8 Simplify decoding error handling
* libguile/ports.c (peek_utf8_codepoint, peek_latin1_codepoint):
  (peek_iconv_codepoint, peek_codepoint): Refactor to push error
  handling to the leaves, where errors happen.  Just return
  the (possibly substituted) codepoint, without an error code; if
  there's really an error, we should raise it.
  (scm_getc, scm_peek_char): Adapt.
2016-05-10 12:09:30 +02:00
Andy Wingo
1953d29038 Decoding errors do not advance read pointer
* libguile/ports.c (scm_getc): If the port conversion strategy is
  'error, signal an error before advancing the read pointer.  This is a
  change from previous behavior; before, we advanced the read pointer
  under an understanding that that was what R6RS required.  But, that
  seems to be not the case.
* test-suite/tests/ports.test ("string ports"): Update decoding-error
  tests to assume that read-char with an error doesn't advance the read
  pointer.
* test-suite/tests/rdelim.test ("read-line"): Likewise.
2016-05-10 11:36:28 +02:00
Andy Wingo
83e5ccb02f Fix bug in trampoline_to_c_read
* libguile/ports.c (trampoline_to_c_read): Fix bug comparing SCM
  values.
2016-05-10 11:08:41 +02:00
Andy Wingo
d77b50476a Speed refactors to Scheme %peek-char
* module/ice-9/ports.scm (peek-bytes): New helper.
  (peek-byte): Use peek-bytes helper.
  (decoding-error): Don't inline; no need.
  (decode-utf8, bad-utf8-len): New helpers.
  (peek-char-and-len/utf8): Use new helpers.
  (peek-char-and-len): No fast paths, and not inline.  Peek-char has its
  own fast path.
  (%peek-char): Use helpers to make fast path.
2016-05-07 12:41:07 +02:00
Andy Wingo
13f2128587 Fix port-clear-stream-start-for-bom-read bug
* libguile/ports.c (scm_port_clear_stream_start_for_bom_read): Fix to
  return SCM_BOOL_F instead of 0.
2016-05-07 12:40:46 +02:00
Andy Wingo
bed7bdc19c Fix use of port-encoding
* module/ice-9/ports.scm (clear-stream-start-for-bom-read): Fix to use
  %port-encoding, which works in symbols.
2016-05-05 23:03:04 +02:00
Andy Wingo
0b4b4db9fa Speed up peek-char
* module/ice-9/ports.scm (peek-char-and-len): Only inline fast path for
  UTF-8.
2016-05-05 22:57:33 +02:00
pcpa
aa939b8ef8 Correct wrong live information for some code patterns
* lib/lightning.c: Correct wrong movr simplification,
	remove no longer needed code to set return registers live
	and update live register set when reaching a label boundary,
	but do not descend if the block has been already visited.
	The later need some tuning for complex code generation, where
	it will still have issues.
2016-05-05 17:55:29 -03:00
Andy Wingo
6d15a71e8f Port to Scheme of new BOM handling
* libguile/ports.c (scm_specialize_port_encoding_x)
  (scm_port_clear_stream_start_for_bom_read): New functions exported
  to (ice-9 ports).
* module/ice-9/ports.scm (clear-stream-start-for-bom-read):
  (fill-input, peek-char-and-len): Rework to handle BOM in fill-input
  instead of once per peek-char.
2016-05-05 22:54:58 +02:00
pcpa
b00c750830 Correct wrong movr simplification 2016-05-05 11:19:45 -03:00
Andy Wingo
86267af8b3 Handle BOM around fill/flush instead of peek/put
* libguile/print.c (display_string_using_iconv): Remove BOM handling;
  this is now handled by scm_lfwrite.
* libguile/ports.c (open_iconv_descriptors): Refactor to take encoding
  as a symbol.
  (prepare_iconv_descriptors): New helper.
  (scm_i_port_iconv_descriptors): Remove scm_t_port_rw_active argument,
  and don't sniff UTF-16/UTF-32 byte orders here.  Instead BOM handlers
  will call prepare_iconv_descriptors.
  (scm_c_read_bytes): Call new port_clear_stream_start_for_bom_read
  helper.
  (port_maybe_consume_initial_byte_order_mark)
  (scm_port_maybe_consume_initial_byte_order_mark): Remove.  Leaves
  Scheme %peek-char broken but it's unused currently so that's OK.
  (peek_iconv_codepoint): Fetch iconv descriptors after doing fill-input
  because it's fill-input that will sniff the BOM.
  (peek_codepoint): Instead of handling BOM at every character, handle
  in fill-input instead.
  (maybe_consume_bom, port_clear_stream_start_for_bom_read)
  (port_clear_stream_start_for_bom_write): New helpers.
  (scm_fill_input): Slurp a BOM if needed.
  (scm_i_write): Clear the start-of-stream-for-bom-write flag.
  (scm_lfwrite): Write a BOM if needed.
2016-05-05 13:00:19 +02:00
Andy Wingo
d7a111b0ec Spead tweaks to Scheme peek-char
* module/ice-9/ports.scm: Speed tweaks to %peek-char.  Ultimately
  somewhat fruitless; I can get 1.4s instead of 1.5s by only
  half-inlining the UTF-8 case though.
2016-05-04 12:40:27 +02:00
Andy Wingo
f5b9a53bd0 Add integer->char and char->integer opcodes
* libguile/vm-engine.c (integer_to_char, char_to_integer): New opcodes.
* libguile/vm.c (vm_error_not_a_char): New error case.
* module/language/cps/compile-bytecode.scm (compile-function):
* module/language/cps/slot-allocation.scm (compute-var-representations):
* module/language/cps/types.scm:
* module/language/tree-il/compile-cps.scm (convert):
* doc/ref/vm.texi (Inlined Scheme Instructions):
* module/system/vm/assembler.scm: Add support for new opcodes.
2016-05-04 12:36:41 +02:00
Andy Wingo
2ba638092f Initial peek-char implementation in Scheme
* module/ice-9/ports.scm (EILSEQ, decoding-error, peek-char-and-len/utf8):
  (peek-char-and-len/iso-8859-1, peek-char-and-len/iconv):
  (peek-char-and-len, %peek-char): New definitions.  Missing iconv1 for
  peek-char, but enough to benchmark.
2016-05-04 11:48:05 +02:00
Andy Wingo
36e32138f8 Port refactors to help Scheme peek-char
* libguile/ports.h (scm_sys_port_encoding, scm_sys_set_port_encoding):
  New functions, to expose port encodings as symbols directly to (ice-9
  ports).
  (scm_port_maybe_consume_initial_byte_order_mark): New function.
* libguile/ports.c (scm_port_encoding): Dispatch to %port-encoding.
  (scm_set_port_encoding_x): Dispatch to %set-port-encoding!.
  (port_maybe_consume_initial_byte_order_mark): New helper, factored out
  of peek_codepoint.
  (scm_port_maybe_consume_initial_byte_order_mark, peek_codepoint): Call
  port_maybe_consume_initial_byte_order_mark.
* module/ice-9/ports.scm (port-encoding): Implement in Scheme.
2016-05-04 11:47:42 +02:00
Andy Wingo
383df7976f Port conversion strategies internally are symbols
* libguile/ports.h (scm_t_port): Represent the conversion strategy as a
  symbol, to make things easier for Scheme.  Rename to
  "conversion_strategy".
  (scm_c_make_port_with_encoding): Change to take encoding and
  conversion_strategy arguments as symbols.
  (scm_i_string_failed_conversion_handler): New internal helper, to turn
  a symbol to a scm_t_string_failed_conversion_handler.
  (scm_i_default_port_encoding): Return the default port encoding as a
  symbol.
  (scm_i_default_port_conversion_strategy)
  (scm_i_set_default_port_conversion_strategy): Rename from
  scm_i_default_port_conversion_handler et al.  Take and return Scheme
  symbols.
* libguile/foreign.c (scm_string_to_pointer, scm_pointer_to_string): Use
  scm_i_default_string_failed_conversion_handler instead of
  scm_i_default_port_conversion_handler.
* libguile/print.c (PORT_CONVERSION_HANDLER): Update definition.
  (print_normal_symbol): Use PORT_CONVERSION_HANDLER.
* libguile/r6rs-ports.c (make_bytevector_input_port):
  (make_custom_binary_input_port, make_bytevector_output_port): Adapt to
  changes in scm_c_make_port_with_encoding.
* libguile/strings.h:
* libguile/strings.c (scm_i_default_string_failed_conversion_handler):
  New helper.
  (scm_from_locale_stringn, scm_from_port_stringn):
  (scm_to_locale_stringn, scm_to_port_stringn): Adapt to interface
  changes.
* libguile/strports.c (scm_mkstrport): Adapt to
  scm_c_make_port_with_encoding change.
* libguile/ports.c (scm_c_make_port): Adapt to
  scm_c_make_port_with_encoding change.
  (ascii_toupper, encoding_matches, canonicalize_encoding): Move down in
  the file.
  (peek_codepoint, get_codepoint, scm_ungetc): Adapt to port conversion
  strategy change.  Remove duplicate case in get_codepoint.
  (scm_init_ports): Move symbol initializations to the same place.
2016-05-04 10:41:07 +02:00
Andy Wingo
d8711b9759 Port encoding internally represented as symbol
* libguile/ports-internal.h (scm_t_port_internal): Remove encoding_mode
  member.
* libguile/ports.h (scm_t_port): "encoding" member is now a SCM symbol.
* libguile/ports.c (scm_init_ports): Define symbols for the encodings
  that we handle explicitly.
  (encoding_matches): Adapt to check against an encoding as a symbol.
  (canonicalize_encoding): Return an encoding as a symbol.
  (scm_c_make_port_with_encoding, scm_i_set_default_port_encoding)
  (decide_utf16_encoding, decide_utf32_encoding)
  (scm_i_port_iconv_descriptors, scm_i_set_port_encoding_x)
  (scm_port_encoding, peek_codepoint, scm_ungetc): Adapt to encoding
  change.

* libguile/print.c (display_string_using_iconv, display_string):
* libguile/read.c (scm_read_character):
* libguile/strings.c (scm_from_port_stringn, scm_to_port_stringn): Adapt
  to port encoding change.
2016-05-03 10:52:54 +02:00
Andy Wingo
422f65fe09 Minor tweak to Scheme peek-byte.
* module/ice-9/ports.scm (peek-byte): Use second return from
  fill-input.
2016-05-01 22:00:37 +02:00
Andy Wingo
4ba59e94f9 Changes to Scheme fill-input corresponding to C
* module/ice-9/ports.scm (fill-input): Rewrite to make changes like the
  ones made to the C scm_fill_input: allow callers to specify a minimum
  amount of buffering.
2016-05-01 21:57:32 +02:00
Andy Wingo
1309ab8093 Char readers peek into read buffer
* libguile/ports.c (scm_i_set_pending_eof): Remove now-unused helper.
  (peek_utf8_codepoint, peek_latin1_codepoint, peek_iconv_codepoint):
  (peek_codepoint): Refactor the fundamental character readers in Guile
  to peek into the read buffer instead of reading then unreading.  This
  will allow Scheme to use the port buffer to convert, when we port this
  to Scheme.
  (get_codepoint): Use peek_codepoint.
  (scm_getc): Adapt.
  (scm_peek_char): Use peek_codepoint.
2016-05-01 16:58:57 +02:00