This port is of limited use if it cannot be used reliably. Rather than
behaving as if the input has finished when it ends unexpectedly, instead
raise an exception.
* module/web/http.scm (make-chunked-input-port): Raise an exception on
premature termination.
(&chunked-input-ended-prematurely): New exception type.
(chunked-input-ended-prematurely-error?): New procedure.
* test-suite/tests/web-http.test (pass-if-named-exception): Rename to
pass-if-named-exception.
(pass-if-named-exception): New syntax.
("Exception on premature chunk end"): New test for this behaviour.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
The chunked transfer encoding specifies the chunked body ends with
CRLF. This is in addition to the CRLF at the end of the last chunk, so
there should be CRLF twice at the end of the chunked body:
https://datatracker.ietf.org/doc/html/rfc2616#section-3.6.1
* module/web/http.scm (make-chunked-input-port): Read two extra bytes at
the end of the chunked input.
(make-chunked-output-port): Write the missing \r\n when closing the
port.
* test-suite/tests/web-http.test (chunked encoding): Add missing \r\n to
test data.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
* module/web/http.scm (write-credentials): capitalize authorization
header scheme. The standard allows the scheme to be case-insensitive,
however most libraries out there expect the scheme to be capitalized,
which is what it is actually used in RFC
docs (e.g. https://datatracker.ietf.org/doc/html/rfc7617#section-2). Some
libraries even reject lowercase scheme making Guile incompatible.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
* module/web/http.scm (spaces-and-tabs, space-or-tab?): New variables.
(read-header-line): After reading a header, if a space or tab follows,
read the continuation lines and join them.
* test-suite/tests/web-http.test: Add test.
Based on a patch by Daniel Hartwig <mandyke@gmail.com>.
* NEWS: Update.
* doc/ref/web.texi (URIs): Fragments are properly part of a URI, so
remove the incorrect note. Add documentation on URI subtypes.
* module/web/uri.scm (uri-reference?): New base type predicate.
(uri?, relative-ref?): Specific predicates.
(validate-uri-reference): Strict validation.
(validate-uri, validate-relative-ref): Specific validators.
(build-uri-reference, build-relative-ref): New constructors.
(string->uri-reference): Rename from string->uri.
(string->uri, string->relative-ref): Specific constructors.
(uri->string): Add #:include-fragment? keyword argument.
* module/web/http.scm (parse-request-uri): Use `build-uri-reference',
and result is a URI-reference, not URI, object. No longer infer an
absent `uri-scheme' is `http'.
(write-uri): Just use `uri->string'.
(declare-uri-header!): Remove unused function.
(declare-uri-reference-header!): Update. Rename from
`declare-relative-uri-header!'.
* test-suite/tests/web-uri.test ("build-uri-reference"):
("string->uri-reference"): Add.
("uri->string"): Also tests for relative-refs.
* test-suite/tests/web-http.test ("read-request-line"):
("write-request-line"): Update for no scheme in some URIs.
("entity headers", "request headers"): Content-location, Referer, and
Location should also parse relative-URIs.
* test-suite/tests/web-request.test ("example-1"): Expect URI-reference
with no scheme.
* module/web/http.scm: Modernize the Guile Scheme by using more match,
when, unless, and non-tail conversion. No functional change, with the
exception of fixing a bug in write-key-value-list for symbols like
100-continue that shouldn't print as #{100-continue}#.
* test-suite/tests/web-http.test (pass-if-only-parse):
(pass-if-reparse, pass-if-parse): Arrange to also serialize and
reparse values from pass-if-parse. Apply to all existing tests except
fragments where we don't expect fragments to be written out.
* module/web/http.scm (parse-entity-tag): Add #:sloppy-delimiters
keyword argument, and return a second value indicating the end
position.
(parse-entity-tag-list): Use parse-entity-tag, so that we also accept
sloppy etags that aren't qstrings.
* test-suite/tests/web-http.test ("request headers"): Add a test.
Fixes <http://bugs.gnu.org/23421>.
Reported by Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de>.
* module/web/http.scm (parse-rfc-822-date): Add two clauses for hours
with a leading space.
* test-suite/tests/web-http.test ("general headers"): Add two tests.
Fixes <http://bugs.gnu.org/22273>.
Reported by Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de>.
* module/web/http.scm (read-header-line): New procedure.
(read-response-line): Use it instead of 'read-line*'.
* test-suite/tests/web-http.test ("read-response-line"): Add test.
* module/web/http.scm (parse-entity-tag): Be less strict, accepting
unquoted strings as well.
* test-suite/tests/web-http.test ("response headers"): Add a test for
etag parsing.
* module/web/uri.scm (validate-uri): Add reference? keyword argument,
for validating references.
(build-uri): Clarify comments to indicate that the result is an
absolute URI.
(build-uri-reference): New interface, to build URI-references.
(string->uri-reference): Rename from string->uri*. Fix fragment
parsing to not include the #.
(string->uri): Adapt to string->uri-reference name change.
* module/web/request.scm (request-absolute-uri): Add default-scheme
optional argument. Use it if the request-uri has no scheme, or
error.
* module/web/http.scm (write-uri): Reflow to use "when". Fix writing of
URI-reference instances.
(declare-uri-reference-header!): Rename from
declare-relative-uri-header!. Use string->uri-reference.
("Location"): Declare as a URI-reference header, as per RFC 7231.
* module/web/client.scm (open-socket-for-uri): Handle the case in which
there is no URI scheme.
* test-suite/tests/web-http.test:
* test-suite/tests/web-uri.test: Add tests.
Fixes <http://bugs.gnu.org/14370>.
Reported by Atom X Zane <atomx@deadlyhead.com>.
* module/web/http.scm (write-credentials): Handle the Basic auth scheme
correctly.
* test-suite/tests/web-http.test (pass-if-round-trip): Use
'pass-if-equal' for better error reporting.
("request headers"): Add tests.
* THANKS: Add "Atom X Zane" to bug fix section.
* module/web/http.scm ("Content-Disposition"): Add a parser and
serializer. Defined in RFC2616 section 19.5.1.
* test-suite/tests/web-http.test ("entity headers"): New test case.
* module/web/http.scm ("Host"): Parse and write IP-literals treating
escapes as uri module does: remove brackets on parse, replace them on
write.
* test-suite/tests/web-http.test ("request headers"): Add tests.
* module/web/http.scm ("Cache-Control"): Write string values using the
default val writer, to get quoting correct.
* test-suite/tests/web-http.test (pass-if-round-trip): New helper.
("general headers"): Check that cache-extensions round trip properly.
* module/web/http.scm ("Cache-Control"): Value for `max-stale' is
optional. Strict validation for value-less directives (`no-store',
etc.). String values optional for "cache-extension" directives.
* test-suite/tests/web-http.test: Value for `max-stale' is optional.
* module/web/http.scm (declare-symbol-list-header!): New helper.
("Connection"): Redefine as a header list.
("Allow", "Content-Encoding", "Accept-Ranges"): Redefine as symbol
lists.
* test-suite/tests/web-http.test:
* test-suite/tests/web-response.test: Adapt tests.
* module/web/http.scm (parse-media-type): Parse media types as symbols.
(parse-key-value-list, parse-param-component, parse-param-list):
Change kons to val-parser. Always parse keys as symbols, and always
either cons, if there is a val, or just have the key, if there is no
val. Easier to explain and just as correct.
(declare-param-list-header!, declare-key-value-list-header!): Adapt to
key-list and param-list kons change.
("Cache-Control", "Pragma", "Transfer-Encoding", "Accept", "Expect")
("TE"): Likewise, adapt.
("Content-Type"): Param keys are symbols.
* module/web/http.scm (declare-header!): No need to specify `sym', as it
can be derived from `name'. Change to take parser, validator, and
writer as positional arguments, and multiple? as a keyword.
(parse-header): Change to take the header as a symbol already, and
just return the parsed value. All headers are symbols now, including
unknown headers. I feel OK doing this given that the symbol GC works
now.
(lookup-header-decl): Only look up headers by symbol.
(read-header): Adapt to parse-header change.
(valid-header?, write-header): Adapt to all headers being symbols.
(split-header-names, list-of-header-names?, write-header-list):
Represent all header names as symbols.
(declare-opaque-header!, declare-date-header!)
(declare-string-list-header!, declare-header-list-header!)
(declare-integer-header!, declare-uri-header!)
(declare-quality-list-header!, declare-param-list-header!)
(declare-key-value-list-header!, declare-entity-tag-list-header!):
Change to be functions instead of syntax, and no need to specify the
symbolic name. Update all header declarations accordingly.
* module/web/request.scm (validate-headers):
* module/web/response.scm (validate-headers): Adapt to all headers being
symbols.
* test-suite/tests/web-http.test (pass-if-parse, pass-if-any-error)
(pass-if-parse-error): Update for parse-header change.
("general headers"): Update header list examples to be all symbols.
* module/web/http.scm (list-of-strings?, write-list-of-strings): Move
definitions up.
(split-header-names, list-of-header-names?, write-header-list): New
helpers.
(declare-header-list-header): New helper.
(cache-control): Use split-header-names for private and no-cache.
(trailer): Use declare-header-list-header to parse known headers to
symbols.
(vary): Likewise, use split-header-names et al.
* test-suite/tests/web-http.test ("general headers"): Add a test.
* module/web/http.scm: Add commentary.
(parse-quality): Allow .NNN to be interpreted as 0.NNN.
* test-suite/tests/web-http.test ("request headers"): Add a test.
* module/web/http.scm (parse-media-type, validate-media-type,
(content-type): Change to represent media types as ("foo/bar" ("param"
. "val") ...) instead of ("foo" "bar" ("param" . "val") ...). Seems to
be more in line with what people expect.
* test-suite/tests/web-http.test ("entity headers"): Add content-type
test.
* test-suite/tests/web-response.test ("example-1"): Adapt expected
parse.
* module/web/http.scm: New module, declares known HTTP headers, and
their parsers and unparsers.
* test-suite/tests/web-http.test: Add test suite.
* module/Makefile.am:
* test-suite/Makefile.am: Adapt.
