This mirrors Guix commit b168acae2a01fd84075cc134a6140594a978fde5.
* module/web/client.scm (tls-wrap)[unbuffered]: New procedure.
Pass the result of 'make-custom-binary-input/output-port' to
'unbuffered'.
This mirrors Guix commit 279d932b1ca7bfbb8657c41a84616dd0dfc6e0a8.
* module/web/client.scm (tls-wrap)[read!]: Read straight into BV
instead of calling 'get-bytevector-some' and 'unget-bytevector'.
This reverts commit 0f983e3db0.
After discussing with Mike we are going to punt the read-line changes
for now. Open the port in O_TEXT mode if you want to chomp the CR in
CFLF sequences.
* libguile/rdelim.c (scm_read_line): handle CRLF, LS and PS
* module/ice-9/suspendable-ports.scm (read-line): handle CRLF, LS, and PS
* module/web/http.scm (read-header-line): take advantage of CRLF in read-line
(read-header): don't need to test for \return
* test-suite/tests/rdelim.test: new tests for read-line CRLF, LS and PS
* doc/ref/api-io.texi: update doc for read-line
* module/ice-9/binary-ports.scm (call-with-input-bytevector):
(call-with-output-bytevector): New functions.
* module/ice-9/iconv.scm: Remove superfluous copies of
call-with-output-string* and call-with-output-bytevector*, now that
the former closes the port and the latter exists.
(call-with-encoded-output-string): Adapt.
* module/web/uri.scm: Use (ice-9 iconv) instead of local
bytevector/string conversion procedures.
Fixes <https://bugs.gnu.org/40582>.
Reported by Julien Lepiller <julien@lepiller.eu>.
Previously, a host part consisting of hex digits would be mistaken as an
IPv6 address and rejected by 'valid-host?'.
* module/web/uri.scm (ipv6-regexp): Add colon.
* test-suite/tests/web-uri.test ("string->uri")["xyz://abc/x/y/z"]: New
test.
* NEWS: Update.
Previously they would always assume #:verify-certificate? #t,
unless #:port was given.
Fixes <https://bugs.gnu.org/40486>.
Reported by Jan Synacek <jsynacek@redhat.com>.
* module/web/client.scm (define-http-verb): Pass #:verify-certificate?
to 'open-socket-for-uri'.
PATCH is described by RFC 5789 and CONNECT is described by RFC 7231.
* module/web/http.scm (parse-http-method): Support CONNECT and PATCH.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
This is largely based on Guix commit
bc3c41ce36349ed4ec758c70b48a7059e363043a and subsequent changes to that
code.
* module/web/client.scm (x509-certificate-directory): New variable.
(set-certificate-credentials-x509-trust-file!*)
(make-credendials-with-ca-trust-files, peer-certificate)
(assert-valid-server-certificate, print-tls-certificate-error): New
procedures.
<top level>: Add call to 'set-exception-printer!'.
(tls-wrap): Add #:verify-certificate? parameter. When it is true, call
'make-credendials-with-ca-trust-files', pass it to
'set-session-credentials!', and call 'assert-valid-server-certificate'.
(open-socket-for-uri): Add #:verify-certificate? parameter and pass it
to 'tls-wrap'.
(http-request): Add #:verify-certificate? parameter and pass it to
'open-socket-for-uri'.
(define-http-verb): Add #:verify-certificate? parameter and pass it to
'http-request'.
* doc/ref/web.texi (Web Client): Update documentation of
'open-socket-for-uri' and 'http-request'. Document
'x509-certificate-directory'.
This is a backport of Guix commit 7b9ac883ea62a816afbfa747c1377dc273c15c20.
* module/web/client.scm (tls-wrap): Catch 'gnutls-error' around
'handshake'. Upon ERROR/WARNING-ALERT-RECEIVED, print a message and
call 'handshake'.
Prior to commit cb14fd2143 (Guile 2.9.7),
autoloading a module would give you access to all its bindings. In
future versions, autoloading a module gives access only to the listed
bindings, as per #:select (see <https://bugs.gnu.org/38895>).
This commit adjusts autoloads to the new semantics, fixing a regression
introduced in cb14fd2143.
* module/web/client.scm <top level>: Remove 'module-autoload!' call.
(gnutls-module, ensure-gnutls): Remove.
(load-gnutls): New procedure.
(tls-wrap): Call it instead of 'ensure-gnutls'. Replace reference to
GNUTLS-MODULE by a call to 'resolve-interface'.
* module/web/http.scm (spaces-and-tabs, space-or-tab?): New variables.
(read-header-line): After reading a header, if a space or tab follows,
read the continuation lines and join them.
* test-suite/tests/web-http.test: Add test.
Based on a patch by Daniel Hartwig <mandyke@gmail.com>.
* NEWS: Update.
* doc/ref/web.texi (URIs): Fragments are properly part of a URI, so
remove the incorrect note. Add documentation on URI subtypes.
* module/web/uri.scm (uri-reference?): New base type predicate.
(uri?, relative-ref?): Specific predicates.
(validate-uri-reference): Strict validation.
(validate-uri, validate-relative-ref): Specific validators.
(build-uri-reference, build-relative-ref): New constructors.
(string->uri-reference): Rename from string->uri.
(string->uri, string->relative-ref): Specific constructors.
(uri->string): Add #:include-fragment? keyword argument.
* module/web/http.scm (parse-request-uri): Use `build-uri-reference',
and result is a URI-reference, not URI, object. No longer infer an
absent `uri-scheme' is `http'.
(write-uri): Just use `uri->string'.
(declare-uri-header!): Remove unused function.
(declare-uri-reference-header!): Update. Rename from
`declare-relative-uri-header!'.
* test-suite/tests/web-uri.test ("build-uri-reference"):
("string->uri-reference"): Add.
("uri->string"): Also tests for relative-refs.
* test-suite/tests/web-http.test ("read-request-line"):
("write-request-line"): Update for no scheme in some URIs.
("entity headers", "request headers"): Content-location, Referer, and
Location should also parse relative-URIs.
* test-suite/tests/web-request.test ("example-1"): Expect URI-reference
with no scheme.
* module/web/client.scm (tls-wrap): Use get-bytevector-some instead of
get-bytevector-n, to prevent Guile from attempting to read more bytes
than are available. Normally trying to read data on a shut-down
socket is fine, but but gnutls issues an error if you attempt to read
data from a shut-down socket, and that appears to be a security
property. Fixes HTTPS requests whose responses are smaller than the
port buffer.
* module/web/http.scm (header-writer): Default to calling put-string.
(put-list): Rename from write-list, take the port first, and call the
put-item function with port then value. Adapt all callers.
(write-date): Rename display-digits to put-digits.
(put-challenge): Rename from write-challenge, adapt arguments to put
convention, and adapt callers.
(declare-symbol-list-header!): Use put-symbol.
(declare-integer-header!): Use put-non-negative-integer.o
(declare-entity-tag-list-header!): Use put-entity-tag-list.
("If-Range", "Etag"): Adapt to put-entity-tag.
(make-chunked-output-port): Use put-char.
* module/web/http.scm: Use put-string and other routines from (ice-9
textual-ports) in preference to `display'. The goal is for these
operations to be suspendable.
* module/web/http.scm: Modernize the Guile Scheme by using more match,
when, unless, and non-tail conversion. No functional change, with the
exception of fixing a bug in write-key-value-list for symbols like
100-continue that shouldn't print as #{100-continue}#.
* test-suite/tests/web-http.test (pass-if-only-parse):
(pass-if-reparse, pass-if-parse): Arrange to also serialize and
reparse values from pass-if-parse. Apply to all existing tests except
fragments where we don't expect fragments to be written out.
Since importing gnutls directly would result in a dependency cycle,
we load gnutls lazily.
This uses code originally written for Guix by Ludovic Courtès.
* module/web/client.scm: (%http-receive-buffer-size)
(gnutls-module, ensure-gnutls, gnutls-ref, tls-wrap): New variables.
(open-socket-for-uri): Wrap in tls when uri scheme is https.
* doc/ref/web.texi (open-socket-for-uri): Document gnutls usage.
* module/web/http.scm (parse-entity-tag): Add #:sloppy-delimiters
keyword argument, and return a second value indicating the end
position.
(parse-entity-tag-list): Use parse-entity-tag, so that we also accept
sloppy etags that aren't qstrings.
* test-suite/tests/web-http.test ("request headers"): Add a test.
Fixes <http://bugs.gnu.org/23421>.
Reported by Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de>.
* module/web/http.scm (parse-rfc-822-date): Add two clauses for hours
with a leading space.
* test-suite/tests/web-http.test ("general headers"): Add two tests.
Fixes <http://bugs.gnu.org/22273>.
Reported by Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de>.
* module/web/http.scm (read-header-line): New procedure.
(read-response-line): Use it instead of 'read-line*'.
* test-suite/tests/web-http.test ("read-response-line"): Add test.
* module/web/response.scm (make-delimited-input-port)[close]: Replace
erroneous self-recursive call with a call to 'close-port'.
* test-suite/tests/web-response.test ("example-1")["response-body-port +
close"]: New test.
* module/web/http.scm (make-chunked-output-port): Add #:buffering
argument, defaulting to 1200 (some random value under the MTU). This
will force a flush every so often, and not every character as would
otherwise be the case after this port rewrite.
* module/web/http.scm (parse-entity-tag): Be less strict, accepting
unquoted strings as well.
* test-suite/tests/web-http.test ("response headers"): Add a test for
etag parsing.
* module/web/uri.scm (validate-uri): Add reference? keyword argument,
for validating references.
(build-uri): Clarify comments to indicate that the result is an
absolute URI.
(build-uri-reference): New interface, to build URI-references.
(string->uri-reference): Rename from string->uri*. Fix fragment
parsing to not include the #.
(string->uri): Adapt to string->uri-reference name change.
* module/web/request.scm (request-absolute-uri): Add default-scheme
optional argument. Use it if the request-uri has no scheme, or
error.
* module/web/http.scm (write-uri): Reflow to use "when". Fix writing of
URI-reference instances.
(declare-uri-reference-header!): Rename from
declare-relative-uri-header!. Use string->uri-reference.
("Location"): Declare as a URI-reference header, as per RFC 7231.
* module/web/client.scm (open-socket-for-uri): Handle the case in which
there is no URI scheme.
* test-suite/tests/web-http.test:
* test-suite/tests/web-uri.test: Add tests.
