Looking at the SRFI-19 specification, the argument is called `day', not
`date'. Even the accessor is called `date-day'. So adjust the
documentation to match.
Also adjust the (web http) module, which was using `date' as well.
* doc/ref/srfi-modules.texi (SRFI-19 Date): Use `day' instead of `date'.
* module/web/http.scm (parse-rfc-822-date, parse-rfc-850-date)
(parse-asctime-date): Same.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
Fixes <https://bugs.gnu.org/49223>.
Reported by Domagoj Stolfa <ds815@gmx.com>.
Backport of Guix commit b36267b1d96ac344d2b42c9822ce04b4c3117f85.
* guix/build/download.scm (tls-wrap): Retry up to 5 times when
'handshake' throws a non-fatal error.
The custom input/output port wrapping the TLS session record port would
introduce overhead, and it would also prevent its uses in a non-blocking
context--e.g., with Fibers. The port close mechanism added in GnuTLS
3.7.7 allows us to get rid of that wrapper.
Backported from Guix commit dd573ceea73295c7a872088ecd91e5f0fd74bf2b.
* web/client.scm (wrap-record-port-for-gnutls<3.7.7): New procedure,
with code formerly in 'tls-wrap'.
(tls-wrap): Check for 'set-session-record-port-close!' and use it when
available; otherwise call 'wrap-record-port-for-gnutls<3.7.7'.
This port is of limited use if it cannot be used reliably. Rather than
behaving as if the input has finished when it ends unexpectedly, instead
raise an exception.
* module/web/http.scm (make-chunked-input-port): Raise an exception on
premature termination.
(&chunked-input-ended-prematurely): New exception type.
(chunked-input-ended-prematurely-error?): New procedure.
* test-suite/tests/web-http.test (pass-if-named-exception): Rename to
pass-if-named-exception.
(pass-if-named-exception): New syntax.
("Exception on premature chunk end"): New test for this behaviour.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
The chunked transfer encoding specifies the chunked body ends with
CRLF. This is in addition to the CRLF at the end of the last chunk, so
there should be CRLF twice at the end of the chunked body:
https://datatracker.ietf.org/doc/html/rfc2616#section-3.6.1
* module/web/http.scm (make-chunked-input-port): Read two extra bytes at
the end of the chunked input.
(make-chunked-output-port): Write the missing \r\n when closing the
port.
* test-suite/tests/web-http.test (chunked encoding): Add missing \r\n to
test data.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
* module/web/http.scm (write-credentials): capitalize authorization
header scheme. The standard allows the scheme to be case-insensitive,
however most libraries out there expect the scheme to be capitalized,
which is what it is actually used in RFC
docs (e.g. https://datatracker.ietf.org/doc/html/rfc7617#section-2). Some
libraries even reject lowercase scheme making Guile incompatible.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
This mirrors Guix commit b168acae2a01fd84075cc134a6140594a978fde5.
* module/web/client.scm (tls-wrap)[unbuffered]: New procedure.
Pass the result of 'make-custom-binary-input/output-port' to
'unbuffered'.
This mirrors Guix commit 279d932b1ca7bfbb8657c41a84616dd0dfc6e0a8.
* module/web/client.scm (tls-wrap)[read!]: Read straight into BV
instead of calling 'get-bytevector-some' and 'unget-bytevector'.
This reverts commit 0f983e3db0.
After discussing with Mike we are going to punt the read-line changes
for now. Open the port in O_TEXT mode if you want to chomp the CR in
CFLF sequences.
* libguile/rdelim.c (scm_read_line): handle CRLF, LS and PS
* module/ice-9/suspendable-ports.scm (read-line): handle CRLF, LS, and PS
* module/web/http.scm (read-header-line): take advantage of CRLF in read-line
(read-header): don't need to test for \return
* test-suite/tests/rdelim.test: new tests for read-line CRLF, LS and PS
* doc/ref/api-io.texi: update doc for read-line
* module/ice-9/binary-ports.scm (call-with-input-bytevector):
(call-with-output-bytevector): New functions.
* module/ice-9/iconv.scm: Remove superfluous copies of
call-with-output-string* and call-with-output-bytevector*, now that
the former closes the port and the latter exists.
(call-with-encoded-output-string): Adapt.
* module/web/uri.scm: Use (ice-9 iconv) instead of local
bytevector/string conversion procedures.
Fixes <https://bugs.gnu.org/40582>.
Reported by Julien Lepiller <julien@lepiller.eu>.
Previously, a host part consisting of hex digits would be mistaken as an
IPv6 address and rejected by 'valid-host?'.
* module/web/uri.scm (ipv6-regexp): Add colon.
* test-suite/tests/web-uri.test ("string->uri")["xyz://abc/x/y/z"]: New
test.
* NEWS: Update.
Previously they would always assume #:verify-certificate? #t,
unless #:port was given.
Fixes <https://bugs.gnu.org/40486>.
Reported by Jan Synacek <jsynacek@redhat.com>.
* module/web/client.scm (define-http-verb): Pass #:verify-certificate?
to 'open-socket-for-uri'.
PATCH is described by RFC 5789 and CONNECT is described by RFC 7231.
* module/web/http.scm (parse-http-method): Support CONNECT and PATCH.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
This is largely based on Guix commit
bc3c41ce36349ed4ec758c70b48a7059e363043a and subsequent changes to that
code.
* module/web/client.scm (x509-certificate-directory): New variable.
(set-certificate-credentials-x509-trust-file!*)
(make-credendials-with-ca-trust-files, peer-certificate)
(assert-valid-server-certificate, print-tls-certificate-error): New
procedures.
<top level>: Add call to 'set-exception-printer!'.
(tls-wrap): Add #:verify-certificate? parameter. When it is true, call
'make-credendials-with-ca-trust-files', pass it to
'set-session-credentials!', and call 'assert-valid-server-certificate'.
(open-socket-for-uri): Add #:verify-certificate? parameter and pass it
to 'tls-wrap'.
(http-request): Add #:verify-certificate? parameter and pass it to
'open-socket-for-uri'.
(define-http-verb): Add #:verify-certificate? parameter and pass it to
'http-request'.
* doc/ref/web.texi (Web Client): Update documentation of
'open-socket-for-uri' and 'http-request'. Document
'x509-certificate-directory'.
This is a backport of Guix commit 7b9ac883ea62a816afbfa747c1377dc273c15c20.
* module/web/client.scm (tls-wrap): Catch 'gnutls-error' around
'handshake'. Upon ERROR/WARNING-ALERT-RECEIVED, print a message and
call 'handshake'.
Prior to commit cb14fd2143 (Guile 2.9.7),
autoloading a module would give you access to all its bindings. In
future versions, autoloading a module gives access only to the listed
bindings, as per #:select (see <https://bugs.gnu.org/38895>).
This commit adjusts autoloads to the new semantics, fixing a regression
introduced in cb14fd2143.
* module/web/client.scm <top level>: Remove 'module-autoload!' call.
(gnutls-module, ensure-gnutls): Remove.
(load-gnutls): New procedure.
(tls-wrap): Call it instead of 'ensure-gnutls'. Replace reference to
GNUTLS-MODULE by a call to 'resolve-interface'.
* module/web/http.scm (spaces-and-tabs, space-or-tab?): New variables.
(read-header-line): After reading a header, if a space or tab follows,
read the continuation lines and join them.
* test-suite/tests/web-http.test: Add test.
Based on a patch by Daniel Hartwig <mandyke@gmail.com>.
* NEWS: Update.
* doc/ref/web.texi (URIs): Fragments are properly part of a URI, so
remove the incorrect note. Add documentation on URI subtypes.
* module/web/uri.scm (uri-reference?): New base type predicate.
(uri?, relative-ref?): Specific predicates.
(validate-uri-reference): Strict validation.
(validate-uri, validate-relative-ref): Specific validators.
(build-uri-reference, build-relative-ref): New constructors.
(string->uri-reference): Rename from string->uri.
(string->uri, string->relative-ref): Specific constructors.
(uri->string): Add #:include-fragment? keyword argument.
* module/web/http.scm (parse-request-uri): Use `build-uri-reference',
and result is a URI-reference, not URI, object. No longer infer an
absent `uri-scheme' is `http'.
(write-uri): Just use `uri->string'.
(declare-uri-header!): Remove unused function.
(declare-uri-reference-header!): Update. Rename from
`declare-relative-uri-header!'.
* test-suite/tests/web-uri.test ("build-uri-reference"):
("string->uri-reference"): Add.
("uri->string"): Also tests for relative-refs.
* test-suite/tests/web-http.test ("read-request-line"):
("write-request-line"): Update for no scheme in some URIs.
("entity headers", "request headers"): Content-location, Referer, and
Location should also parse relative-URIs.
* test-suite/tests/web-request.test ("example-1"): Expect URI-reference
with no scheme.
* module/web/client.scm (tls-wrap): Use get-bytevector-some instead of
get-bytevector-n, to prevent Guile from attempting to read more bytes
than are available. Normally trying to read data on a shut-down
socket is fine, but but gnutls issues an error if you attempt to read
data from a shut-down socket, and that appears to be a security
property. Fixes HTTPS requests whose responses are smaller than the
port buffer.
* module/web/http.scm (header-writer): Default to calling put-string.
(put-list): Rename from write-list, take the port first, and call the
put-item function with port then value. Adapt all callers.
(write-date): Rename display-digits to put-digits.
(put-challenge): Rename from write-challenge, adapt arguments to put
convention, and adapt callers.
(declare-symbol-list-header!): Use put-symbol.
(declare-integer-header!): Use put-non-negative-integer.o
(declare-entity-tag-list-header!): Use put-entity-tag-list.
("If-Range", "Etag"): Adapt to put-entity-tag.
(make-chunked-output-port): Use put-char.
* module/web/http.scm: Use put-string and other routines from (ice-9
textual-ports) in preference to `display'. The goal is for these
operations to be suspendable.
* module/web/http.scm: Modernize the Guile Scheme by using more match,
when, unless, and non-tail conversion. No functional change, with the
exception of fixing a bug in write-key-value-list for symbols like
100-continue that shouldn't print as #{100-continue}#.
* test-suite/tests/web-http.test (pass-if-only-parse):
(pass-if-reparse, pass-if-parse): Arrange to also serialize and
reparse values from pass-if-parse. Apply to all existing tests except
fragments where we don't expect fragments to be written out.
Since importing gnutls directly would result in a dependency cycle,
we load gnutls lazily.
This uses code originally written for Guix by Ludovic Courtès.
* module/web/client.scm: (%http-receive-buffer-size)
(gnutls-module, ensure-gnutls, gnutls-ref, tls-wrap): New variables.
(open-socket-for-uri): Wrap in tls when uri scheme is https.
* doc/ref/web.texi (open-socket-for-uri): Document gnutls usage.
* module/web/http.scm (parse-entity-tag): Add #:sloppy-delimiters
keyword argument, and return a second value indicating the end
position.
(parse-entity-tag-list): Use parse-entity-tag, so that we also accept
sloppy etags that aren't qstrings.
* test-suite/tests/web-http.test ("request headers"): Add a test.
Fixes <http://bugs.gnu.org/23421>.
Reported by Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de>.
* module/web/http.scm (parse-rfc-822-date): Add two clauses for hours
with a leading space.
* test-suite/tests/web-http.test ("general headers"): Add two tests.
