This is largely based on Guix commit
bc3c41ce36349ed4ec758c70b48a7059e363043a and subsequent changes to that
code.
* module/web/client.scm (x509-certificate-directory): New variable.
(set-certificate-credentials-x509-trust-file!*)
(make-credendials-with-ca-trust-files, peer-certificate)
(assert-valid-server-certificate, print-tls-certificate-error): New
procedures.
<top level>: Add call to 'set-exception-printer!'.
(tls-wrap): Add #:verify-certificate? parameter. When it is true, call
'make-credendials-with-ca-trust-files', pass it to
'set-session-credentials!', and call 'assert-valid-server-certificate'.
(open-socket-for-uri): Add #:verify-certificate? parameter and pass it
to 'tls-wrap'.
(http-request): Add #:verify-certificate? parameter and pass it to
'open-socket-for-uri'.
(define-http-verb): Add #:verify-certificate? parameter and pass it to
'http-request'.
* doc/ref/web.texi (Web Client): Update documentation of
'open-socket-for-uri' and 'http-request'. Document
'x509-certificate-directory'.
This is a backport of Guix commit 7b9ac883ea62a816afbfa747c1377dc273c15c20.
* module/web/client.scm (tls-wrap): Catch 'gnutls-error' around
'handshake'. Upon ERROR/WARNING-ALERT-RECEIVED, print a message and
call 'handshake'.
Prior to commit cb14fd2143 (Guile 2.9.7),
autoloading a module would give you access to all its bindings. In
future versions, autoloading a module gives access only to the listed
bindings, as per #:select (see <https://bugs.gnu.org/38895>).
This commit adjusts autoloads to the new semantics, fixing a regression
introduced in cb14fd2143.
* module/web/client.scm <top level>: Remove 'module-autoload!' call.
(gnutls-module, ensure-gnutls): Remove.
(load-gnutls): New procedure.
(tls-wrap): Call it instead of 'ensure-gnutls'. Replace reference to
GNUTLS-MODULE by a call to 'resolve-interface'.
Based on a patch by Daniel Hartwig <mandyke@gmail.com>.
* NEWS: Update.
* doc/ref/web.texi (URIs): Fragments are properly part of a URI, so
remove the incorrect note. Add documentation on URI subtypes.
* module/web/uri.scm (uri-reference?): New base type predicate.
(uri?, relative-ref?): Specific predicates.
(validate-uri-reference): Strict validation.
(validate-uri, validate-relative-ref): Specific validators.
(build-uri-reference, build-relative-ref): New constructors.
(string->uri-reference): Rename from string->uri.
(string->uri, string->relative-ref): Specific constructors.
(uri->string): Add #:include-fragment? keyword argument.
* module/web/http.scm (parse-request-uri): Use `build-uri-reference',
and result is a URI-reference, not URI, object. No longer infer an
absent `uri-scheme' is `http'.
(write-uri): Just use `uri->string'.
(declare-uri-header!): Remove unused function.
(declare-uri-reference-header!): Update. Rename from
`declare-relative-uri-header!'.
* test-suite/tests/web-uri.test ("build-uri-reference"):
("string->uri-reference"): Add.
("uri->string"): Also tests for relative-refs.
* test-suite/tests/web-http.test ("read-request-line"):
("write-request-line"): Update for no scheme in some URIs.
("entity headers", "request headers"): Content-location, Referer, and
Location should also parse relative-URIs.
* test-suite/tests/web-request.test ("example-1"): Expect URI-reference
with no scheme.
* module/web/client.scm (tls-wrap): Use get-bytevector-some instead of
get-bytevector-n, to prevent Guile from attempting to read more bytes
than are available. Normally trying to read data on a shut-down
socket is fine, but but gnutls issues an error if you attempt to read
data from a shut-down socket, and that appears to be a security
property. Fixes HTTPS requests whose responses are smaller than the
port buffer.
Since importing gnutls directly would result in a dependency cycle,
we load gnutls lazily.
This uses code originally written for Guix by Ludovic Courtès.
* module/web/client.scm: (%http-receive-buffer-size)
(gnutls-module, ensure-gnutls, gnutls-ref, tls-wrap): New variables.
(open-socket-for-uri): Wrap in tls when uri scheme is https.
* doc/ref/web.texi (open-socket-for-uri): Document gnutls usage.
* module/web/uri.scm (validate-uri): Add reference? keyword argument,
for validating references.
(build-uri): Clarify comments to indicate that the result is an
absolute URI.
(build-uri-reference): New interface, to build URI-references.
(string->uri-reference): Rename from string->uri*. Fix fragment
parsing to not include the #.
(string->uri): Adapt to string->uri-reference name change.
* module/web/request.scm (request-absolute-uri): Add default-scheme
optional argument. Use it if the request-uri has no scheme, or
error.
* module/web/http.scm (write-uri): Reflow to use "when". Fix writing of
URI-reference instances.
(declare-uri-reference-header!): Rename from
declare-relative-uri-header!. Use string->uri-reference.
("Location"): Declare as a URI-reference header, as per RFC 7231.
* module/web/client.scm (open-socket-for-uri): Handle the case in which
there is no URI scheme.
* test-suite/tests/web-http.test:
* test-suite/tests/web-uri.test: Add tests.
Fixes <http://bugs.gnu.org/15368>.
* module/web/client.scm (open-socket-for-uri): Remove call to
'setsockopt'. Contrary to what the comment said, its effect was to
shrink the receive buffer from 124 KiB (the default size, per
/proc/sys/net/core/rmem_default on Linux-based systems) to 12 KiB.
* module/web/client.scm (sanitize-request): Add a Content-Length
header if a body if given, even if the body is empty.
* module/web/server.scm (sanitize-response): Add a Content-Length
header if a body if given, even if the body is empty.
* module/web/http.scm (http-proxy-port?, set-http-proxy-port?!): New
exported procedures.
(write-request-line): If we're using an http proxy, write an
absolute-URI in the request line.
* module/web/client.scm: Import (web http).
(current-http-proxy): New exported parameter.
(open-socket-for-uri): If 'current-http-proxy' is not false,
connect to the proxy instead of the URI host, and use
'set-http-proxy-port?!' to make note of that fact.
* doc/ref/web.texi (Web Client): Document 'current-http-proxy'.
* doc/ref/web.texi: Say `World Wide Web'; the hyphenated form is almost
never used (c.f. w3.org).
General predicate arguments are named `obj'. Fill in arguments
omitted from some procedure definitions (e.g. `request-method').
Minor tweaks, such as using en-dash and missing markup as appropriate.
Wrap very long deffn lines.
* module/web/*.scm: Expand texinfo markup in doc strings. Synchronize
with changes in web.texi.
* module/web/client.scm (ensure-uri): New helper.
(open-socket-for-uri): Accept a URI as a string or as a URI object.
(extend-request, sanitize-request): New helpers, like the
corresponding functions in (web server).
(decode-response-body): Add a reference to the HTTP/1.1 spec, and
use (ice-9 iconv).
(request): New helper, factoring all aspects of sending an HTTP
request and getting a response.
(http-get): Redefine in terms of http-get. Deprecate the
#:extra-headers argument in favor of #:headers. Allow a body. Add a
#:streaming? argument, subsuming the functionality of http-get*.
(http-get*): Deprecate.
(http-head, http-post, http-put, http-delete, http-trace)
(http-options): Define interfaces for all HTTP verbs.
* test-suite/tests/web-client.test: Add tests.
* doc/ref/web.texi: Update documentation.
Thanks to Gregory Benison for the initial patch.
* module/web/http.scm ("Connection"): Write the "close" token in
lower-case.
* module/web/client.scm (http-get): Don't shutdown the writing side of
the pipe if we are not doing a keepalive, as this may prevent the
request from being sent at all. Prevented http://friendfeed.com/ from
being correctly fetched.
* module/web/client.scm (open-socket-for-uri): Delete addrinfos
with the same address. Always open SOCK_STREAM/IPPROTO_IP sockets.
Fix the error handler's condition to determine what to do.
Reported by Nikita Karetnikov <nikita.karetnikov@gmail.com> at
<http://lists.gnu.org/archive/html/bug-guix/2012-12/msg00150.html>.
* doc/ref/web.texi: Fix spacing. Update with a few missing function
descriptions.
* module/web/client.scm:
* module/web/http.scm:
* module/web/request.scm:
* module/web/server.scm:
* module/web/uri.scm: Update docstrings from manual (reworked by Ludovic
Courtès).
