" (string-concatenate contents) "
")) ;; Now the meat of our simple web application (define (you-said text) (para "You said: " text)) (display (you-said "Hi!")) @print{}You said: Hi!
@end example This is a perfectly valid implementation, provided that the incoming text does not contain the special HTML characters @samp{<}, @samp{>}, or @samp{&}. But this provision of a restricted character set is not reflected anywhere in the program itself: we must @emph{assume} that the programmer understands this, and performs the check elsewhere. Unfortunately, the short history of the practice of programming does not bear out this assumption. A @dfn{cross-site scripting} (@acronym{XSS}) vulnerability is just such a common error in which unfiltered user input is allowed into the output. A user could submit a crafted comment to your web site which results in visitors running malicious Javascript, within the security context of your domain: @example (display (you-said "")) @print{}You said:
@end example The fundamental problem here is that both user data and the program template are represented using strings. This identity means that types can't help the programmer to make a distinction between these two, so they get confused. There are a number of possible solutions, but perhaps the best is to treat HTML not as strings, but as native s-expressions: as SXML. The basic idea is that HTML is either text, represented by a string, or an element, represented as a tagged list. So @samp{foo} becomes @samp{"foo"}, and @samp{foo} becomes @samp{(b "foo")}. Attributes, if present, go in a tagged list headed by @samp{@@}, like @samp{(img (@@ (src "http://example.com/foo.png")))}. @xref{sxml simple}, for more information. The good thing about SXML is that HTML elements cannot be confused with text. Let's make a new definition of @code{para}: @example (define (para . contents) `(p ,@@contents)) (use-modules (sxml simple)) (sxml->xml (you-said "Hi!")) @print{}You said: Hi!
(sxml->xml (you-said "Rats, foiled again!")) @print{}You said: <i>Rats, foiled again!</i>
@end example So we see in the second example that HTML elements cannot be unwittingly introduced into the output. However it is now perfectly acceptable to pass SXML to @code{you-said}; in fact, that is the big advantage of SXML over everything-as-a-string. @example (sxml->xml (you-said (you-said "You said:
You said: <Hi!>
@end example The SXML types allow procedures to @emph{compose}. The types make manifest which parts are HTML elements, and which are text. So you needn't worry about escaping user input; the type transition back to a string handles that for you. @acronym{XSS} vulnerabilities are a thing of the past. Well. That's all very nice and opinionated and such, but how do I use the thing? Read on! @node URIs @subsection Universal Resource Identifiers Guile provides a standard data type for Universal Resource Identifiers (URIs), as defined in RFC 3986. The generic URI syntax is as follows: @example URI := scheme ":" ["//" [userinfo "@@"] host [":" port]] path \ [ "?" query ] [ "#" fragment ] @end example For example, in the URI, @indicateurl{http://www.gnu.org/help/}, the scheme is @code{http}, the host is @code{www.gnu.org}, the path is @code{/help/}, and there is no userinfo, port, query, or path. All URIs have a scheme and a path (though the path might be empty). Some URIs have a host, and some of those have ports and userinfo. Any URI might have a query part or a fragment. Userinfo is something of an abstraction, as some legacy URI schemes allowed userinfo of the form @code{@var{username}:@var{passwd}}. But since passwords do not belong in URIs, the RFC does not want to condone this practice, so it calls anything before the @code{@@} sign @dfn{userinfo}. Properly speaking, a fragment is not part of a URI. For example, when a web browser follows a link to @indicateurl{http://example.com/#foo}, it sends a request for @indicateurl{http://example.com/}, then looks in the resulting page for the fragment identified @code{foo} reference. A fragment identifies a part of a resource, not the resource itself. But it is useful to have a fragment field in the URI record itself, so we hope you will forgive the inconsistency. @example (use-modules (web uri)) @end example The following procedures can be found in the @code{(web uri)} module. Load it into your Guile, using a form like the above, to have access to them. @defun build-uri scheme [#:userinfo=@code{#f}] [#:host=@code{#f}] @ [#:port=@code{#f}] [#:path=@code{""}] [#:query=@code{#f}] @ [#:fragment=@code{#f}] [#:validate?=@code{#t}] Construct a URI object. @var{scheme} should be a symbol, and the rest of the fields are either strings or @code{#f}. If @var{validate?} is true, also run some consistency checks to make sure that the constructed URI is valid. @end defun @defun uri? x @defunx uri-scheme uri @defunx uri-userinfo uri @defunx uri-host uri @defunx uri-port uri @defunx uri-path uri @defunx uri-query uri @defunx uri-fragment uri A predicate and field accessors for the URI record type. The URI scheme will be a symbol, and the rest either strings or @code{#f} if not present. @end defun @defun string->uri string Parse @var{string} into a URI object. Return @code{#f} if the string could not be parsed. @end defun @defun uri->string uri Serialize @var{uri} to a string. If the URI has a port that is the default port for its scheme, the port is not included in the serialization. @end defun @defun declare-default-port! scheme port Declare a default port for the given URI scheme. @end defun @defun uri-decode str [#:encoding=@code{"utf-8"}] Percent-decode the given @var{str}, according to @var{encoding}, which should be the name of a character encoding. Note that this function should not generally be applied to a full URI string. For paths, use split-and-decode-uri-path instead. For query strings, split the query on @code{&} and @code{=} boundaries, and decode the components separately. Note also that percent-encoded strings encode @emph{bytes}, not characters. There is no guarantee that a given byte sequence is a valid string encoding. Therefore this routine may signal an error if the decoded bytes are not valid for the given encoding. Pass @code{#f} for @var{encoding} if you want decoded bytes as a bytevector directly. @xref{Ports, @code{set-port-encoding!}}, for more information on character encodings. Returns a string of the decoded characters, or a bytevector if @var{encoding} was @code{#f}. @end defun Fixme: clarify return type. indicate default values. type of unescaped-chars. @defun uri-encode str [#:encoding=@code{"utf-8"}] [#:unescaped-chars] Percent-encode any character not in the character set, @var{unescaped-chars}. The default character set includes alphanumerics from ASCII, as well as the special characters @samp{-}, @samp{.}, @samp{_}, and @samp{~}. Any other character will be percent-encoded, by writing out the character to a bytevector within the given @var{encoding}, then encoding each byte as @code{%@var{HH}}, where @var{HH} is the hexadecimal representation of the byte. @end defun @defun split-and-decode-uri-path path Split @var{path} into its components, and decode each component, removing empty components. For example, @code{"/foo/bar%20baz/"} decodes to the two-element list, @code{("foo" "bar baz")}. @end defun @defun encode-and-join-uri-path parts URI-encode each element of @var{parts}, which should be a list of strings, and join the parts together with @code{/} as a delimiter. For example, the list @code{("scrambled eggs" "biscuits&gravy")} encodes as @code{"scrambled%20eggs/biscuits%26gravy"}. @end defun @node HTTP @subsection The Hyper-Text Transfer Protocol The initial motivation for including web functionality in Guile, rather than rely on an external package, was to establish a standard base on which people can share code. To that end, we continue the focus on data types by providing a number of low-level parsers and unparsers for elements of the HTTP protocol. If you are want to skip the low-level details for now and move on to web pages, @pxref{Web Server}. Otherwise, load the HTTP module, and read on. @example (use-modules (web http)) @end example The focus of the @code{(web http)} module is to parse and unparse standard HTTP headers, representing them to Guile as native data structures. For example, a @code{Date:} header will be represented as a SRFI-19 date record (@pxref{SRFI-19}), rather than as a string. Guile tries to follow RFCs fairly strictly---the road to perdition being paved with compatibility hacks---though some allowances are made for not-too-divergent texts. Header names are represented as lower-case symbols. @defun string->header name Parse @var{name} to a symbolic header name. @end defun @defun header->string sym Return the string form for the header named @var{sym}. @end defun For example: @example (string->header "Content-Length") @result{} content-length (header->string 'content-length) @result{} "Content-Length" (string->header "FOO") @result{} foo (header->string 'foo @result{} "Foo" @end example Guile keeps a registry of known headers, their string names, and some parsing and serialization procedures. If a header is unknown, its string name is simply its symbol name in title-case. @defun known-header? sym Return @code{#t} iff @var{sym} is a known header, with associated parsers and serialization procedures. @end defun @defun header-parser sym Return the value parser for headers named @var{sym}. The result is a procedure that takes one argument, a string, and returns the parsed value. If the header isn't known to Guile, a default parser is returned that passes through the string unchanged. @end defun @defun header-validator sym Return a predicate which returns @code{#t} if the given value is valid for headers named @var{sym}. The default validator for unknown headers is @code{string?}. @end defun @defun header-writer sym Return a procedure that writes values for headers named @var{sym} to a port. The resulting procedure takes two arguments: a value and a port. The default writer is @code{display}. @end defun For more on the set of headers that Guile knows about out of the box, @pxref{HTTP Headers}. To add your own, use the @code{declare-header!} procedure: @defun declare-header! name parser validator writer [#:multiple?=@code{#f}] Declare a parser, validator, and writer for a given header. @end defun For example, let's say you are running a web server behind some sort of proxy, and your proxy adds an @code{X-Client-Address} header, indicating the IPv4 address of the original client. You would like for the HTTP request record to parse out this header to a Scheme value, instead of leaving it as a string. You could register this header with Guile's HTTP stack like this: @example (define (parse-ip str) (inet-aton str) (define (validate-ip ip) (define (write-ip ip port) (display (inet-ntoa ip) port)) (declare-header! "X-Client-Address" (lambda (str) (inet-aton str)) (lambda (ip) (and (integer? ip) (exact? ip) (<= 0 ip #xffffffff))) (lambda (ip port) (display (inet-ntoa ip) port))) @end example @defun valid-header? sym val Return a true value iff @var{val} is a valid Scheme value for the header with name @var{sym}. @end defun Now that we have a generic interface for reading and writing headers, we do just that. @defun read-header port Read one HTTP header from @var{port}. Return two values: the header name and the parsed Scheme value. May raise an exception if the header was known but the value was invalid. Returns the end-of-file object for both values if the end of the message body was reached (i.e., a blank line). @end defun @defun parse-header name val Parse @var{val}, a string, with the parser for the header named @var{name}. Returns the parsed value. @end defun @defun write-header name val port Write the given header name and value to @var{port}, using the writer from @code{header-writer}. @end defun @defun read-headers port Read the headers of an HTTP message from @var{port}, returning the headers as an ordered alist. @end defun @defun write-headers headers port Write the given header alist to @var{port}. Doesn't write the final @samp{\r\n}, as the user might want to add another header. @end defun The @code{(web http)} module also has some utility procedures to read and write request and response lines. @defun parse-http-method str [start] [end] Parse an HTTP method from @var{str}. The result is an upper-case symbol, like @code{GET}. @end defun @defun parse-http-version str [start] [end] Parse an HTTP version from @var{str}, returning it as a major-minor pair. For example, @code{HTTP/1.1} parses as the pair of integers, @code{(1 . 1)}. @end defun @defun parse-request-uri str [start] [end] Parse a URI from an HTTP request line. Note that URIs in requests do not have to have a scheme or host name. The result is a URI object. @end defun @defun read-request-line port Read the first line of an HTTP request from @var{port}, returning three values: the method, the URI, and the version. @end defun @defun write-request-line method uri version port Write the first line of an HTTP request to @var{port}. @end defun @defun read-response-line port Read the first line of an HTTP response from @var{port}, returning three values: the HTTP version, the response code, and the "reason phrase". @end defun @defun write-response-line version code reason-phrase port Write the first line of an HTTP response to @var{port}. @end defun @node HTTP Headers @subsection HTTP Headers The @code{(web http)} module defines parsers and unparsers for all headers defined in the HTTP/1.1 standard. This section describes the parsed format of the various headers. We cannot describe the function of all of these headers, however, in sufficient detail. The interested reader would do well to download a copy of RFC 2616 and have it on hand. To begin with, we should make a few definitions: @table @dfn @item key-value list A key-value list is a list of values. Each value may be a string, a symbol, or a pair. Known keys are parsed to symbols; otherwise keys are left as strings. Keys with values are parsed to pairs, the car of which is the symbol or string key, and the cdr is the parsed value. Parsed values for known keys have key-dependent formats. Parsed values for unknown keys are strings. @item param list A param list is a list of key-value lists. When serialized to a string, items in the inner lists are separated by semicolons. Again, known keys are parsed to symbols. @item quality A number of headers have quality values in them, which are decimal fractions between zero and one indicating a preference for various kinds of responses, which the server may choose to heed. Given that only three digits are allowed in the fractional part, Guile parses quality values to integers between 0 and 1000 instead of inexact numbers between 0.0 and 1.0. @item quality list A list of pairs, the car of which is a quality value. @item entity tag A pair, the car of which is an opaque string, and the cdr of which is true iff the entity tag is a ``strong'' entity tag. @end table @subsubsection General Headers @table @code @item cache-control A key-value list of cache-control directives. Known keys are @code{max-age}, @code{max-stale}, @code{min-fresh}, @code{must-revalidate}, @code{no-cache}, @code{no-store}, @code{no-transform}, @code{only-if-cached}, @code{private}, @code{proxy-revalidate}, @code{public}, and @code{s-maxage}. If present, parameters to @code{max-age}, @code{max-stale}, @code{min-fresh}, and @code{s-maxage} are all parsed as non-negative integers. If present, parameters to @code{private} and @code{no-cache} are parsed as lists of header names, represented as symbols if they are known headers or strings otherwise. @item connection A list of connection tokens. A connection token is a string. @item date A SRFI-19 date record. @item pragma A key-value list of pragma directives. @code{no-cache} is the only known key. @item trailer A list of header names. Known header names are parsed to symbols, otherwise they are left as strings. @item transfer-encoding A param list of transfer codings. @code{chunked} is the only known key. @item upgrade A list of strings. @item via A list of strings. There may be multiple @code{via} headers in ne message. @item warning A list of warnings. Each warning is a itself a list of four elements: a code, as an exact integer between 0 and 1000, a host as a string, the warning text as a string, and either @code{#f} or a SRFI-19 date. There may be multiple @code{warning} headers in one message. @end table @subsubsection Entity Headers @table @code @item allow A list of methods, as strings. Methods are parsed as strings instead of @code{parse-http-method} so as to allow for new methods. @item content-encoding A list of content codings, as strings. @item content-language A list of language tags, as strings. @item content-length An exact, non-negative integer. @item content-location A URI record. @item content-md5 A string. @item content-range A list of three elements: the symbol @code{bytes}, either the symbol @code{*} or a pair of integers, indicating the byte rage, and either @code{*} or an integer, for the instance length. @item content-type A pair, the car of which is the media type as a string, and the cdr is an alist of parameters, with strings as keys and values. For example, @code{"text/plain"} parses as @code{("text/plain")}, and @code{"text/plain;charset=utf-8"} parses as @code{("text/plain" ("charset" . "utf-8"))}. @item expires A SRFI-19 date. @item last-modified A SRFI-19 date. @end table @subsubsection Request Headers @table @code @item accept A param list. Each element in the list indicates one media-range with accept-params. They only known key is @code{q}, whose value is parsed as a quality value. @item accept-charset A quality-list of charsets, as strings. @item accept-encoding A quality-list of content codings, as strings. @item accept-language A quality-list of languages, as strings. @item authorization A string. @item expect A param list of expectations. The only known key is @code{100-continue}. @item from A string. @item host A pair of the host, as a string, and the port, as an integer. If no port is given, port is @code{#f}. @item if-match Either the symbol @code{*}, or a list of entity tags (see above). @item if-modified-since A SRFI-19 date. @item if-none-match Either the symbol @code{*}, or a list of entity tags (see above). @item if-range Either an entity tag, or a SRFI-19 date. @item if-unmodified-since A SRFI-19 date. @item max-forwards An exact non-negative integer. @item proxy-authorization A string. @item range A pair whose car is the symbol @code{bytes}, and whose cdr is a list of pairs. Each element of the cdr indicates a range; the car is the first byte position and the cdr is the last byte position, as integers, or @code{#f} if not given. @item referer A URI. @item te A param list of transfer-codings. The only known key is @code{trailers}. @item user-agent A string. @end table @subsubsection Response Headers @table @code @item accept-ranges A list of strings. @item age An exact, non-negative integer. @item etag An entity tag. @item location A URI. @item proxy-authenticate A string. @item retry-after Either an exact, non-negative integer, or a SRFI-19 date. @item server A string. @item vary Either the symbol @code{*}, or a list of headers, with known headers parsed to symbols. @item www-authenticate A string. @end table @node Requests @subsection HTTP Requests @example (use-modules (web request)) @end example The request module contains a data type for HTTP requests. Note that the body is not part of the request, but the port is. Once you have read a request, you may read the body separately, and likewise for writing requests. @defun build-request [#:method] [#:uri] [#:version] [#:headers] [#:port] [#:meta] [#:validate-headers?] Construct an HTTP request object. If @var{validate-headers?} is true, the headers are each run through their respective validators. @end defun @defun request? @defunx request-method @defunx request-uri @defunx request-version @defunx request-headers @defunx request-meta @defunx request-port A predicate and field accessors for the request type. The fields are as follows: @table @code @item method The HTTP method, for example, @code{GET}. @item uri The URI as a URI record. @item version The HTTP version pair, like @code{(1 . 1)}. @item headers The request headers, as an alist of parsed values. @item meta An arbitrary alist of other data, for example information returned in the @code{sockaddr} from @code{accept} (@pxref{Network Sockets and Communication}). @item port The port on which to read or write a request body, if any. @end table @end defun @defun read-request port [meta] Read an HTTP request from @var{port}, optionally attaching the given metadata, @var{meta}. As a side effect, sets the encoding on @var{port} to ISO-8859-1 (latin-1), so that reading one character reads one byte. See the discussion of character sets in "HTTP Requests" in the manual, for more information. @end defun @defun write-request r port Write the given HTTP request to @var{port}. Returns a new request, whose @code{request-port} will continue writing on @var{port}, perhaps using some transfer encoding. @end defun @defun read-request-body/latin-1 r Reads the request body from @var{r}, as a string. Assumes that the request port has ISO-8859-1 encoding, so that the number of characters to read is the same as the @code{request-content-length}. Returns @code{#f} if there was no request body. @end defun @defun write-request-body/latin-1 r body Write @var{body}, a string encodable in ISO-8859-1, to the port corresponding to the HTTP request @var{r}. @end defun @defun read-request-body/bytevector r Reads the request body from @var{r}, as a bytevector. Returns @code{#f} if there was no request body. @end defun @defun write-request-body/bytevector r bv Write @var{body}, a bytevector, to the port corresponding to the HTTP request @var{r}. @end defun The various headers that are typically associated with HTTP requests may be accessed with these dedicated accessors. @xref{HTTP Headers}, for more information on the format of parsed headers. @defun request-accept request [default='()] @defunx request-accept-charset request [default='()] @defunx request-accept-encoding request [default='()] @defunx request-accept-language request [default='()] @defunx request-allow request [default='()] @defunx request-authorization request [default=#f] @defunx request-cache-control request [default='()] @defunx request-connection request [default='()] @defunx request-content-encoding request [default='()] @defunx request-content-language request [default='()] @defunx request-content-length request [default=#f] @defunx request-content-location request [default=#f] @defunx request-content-md5 request [default=#f] @defunx request-content-range request [default=#f] @defunx request-content-type request [default=#f] @defunx request-date request [default=#f] @defunx request-expect request [default='()] @defunx request-expires request [default=#f] @defunx request-from request [default=#f] @defunx request-host request [default=#f] @defunx request-if-match request [default=#f] @defunx request-if-modified-since request [default=#f] @defunx request-if-none-match request [default=#f] @defunx request-if-range request [default=#f] @defunx request-if-unmodified-since request [default=#f] @defunx request-last-modified request [default=#f] @defunx request-max-forwards request [default=#f] @defunx request-pragma request [default='()] @defunx request-proxy-authorization request [default=#f] @defunx request-range request [default=#f] @defunx request-referer request [default=#f] @defunx request-te request [default=#f] @defunx request-trailer request [default='()] @defunx request-transfer-encoding request [default='()] @defunx request-upgrade request [default='()] @defunx request-user-agent request [default=#f] @defunx request-via request [default='()] @defunx request-warning request [default='()] Return the given request header, or @var{default} if none was present. @end defun @defun request-absolute-uri r [default-host] [default-port] A helper routine to determine the absolute URI of a request, using the @code{host} header and the default host and port. @end defun @node Responses @subsection HTTP Responses @example (use-modules (web response)) @end example As with requests (@pxref{Requests}), Guile offers a data type for HTTP responses. Again, the body is represented separately from the request. @defun response? @defunx response-version @defunx response-code @defunx response-reason-phrase response @defunx response-headers @defunx response-port A predicate and field accessors for the response type. The fields are as follows: @table @code @item version The HTTP version pair, like @code{(1 . 1)}. @item code The HTTP response code, like @code{200}. @item reason-phrase The reason phrase, or the standard reason phrase for the response's code. @item headers The response headers, as an alist of parsed values. @item port The port on which to read or write a response body, if any. @end table @end defun @defun read-response port Read an HTTP response from @var{port}, optionally attaching the given metadata, @var{meta}. As a side effect, sets the encoding on @var{port} to ISO-8859-1 (latin-1), so that reading one character reads one byte. See the discussion of character sets in "HTTP Responses" in the manual, for more information. @end defun @defun build-response [#:version] [#:code] [#:reason-phrase] [#:headers] [#:port] Construct an HTTP response object. If @var{validate-headers?} is true, the headers are each run through their respective validators. @end defun @defun extend-response r k v . additional Extend an HTTP response by setting additional HTTP headers @var{k}, @var{v}. Returns a new HTTP response. @end defun @defun adapt-response-version response version Adapt the given response to a different HTTP version. Returns a new HTTP response. The idea is that many applications might just build a response for the default HTTP version, and this method could handle a number of programmatic transformations to respond to older HTTP versions (0.9 and 1.0). But currently this function is a bit heavy-handed, just updating the version field. @end defun @defun write-response r port Write the given HTTP response to @var{port}. Returns a new response, whose @code{response-port} will continue writing on @var{port}, perhaps using some transfer encoding. @end defun @defun read-response-body/latin-1 r Reads the response body from @var{r}, as a string. Assumes that the response port has ISO-8859-1 encoding, so that the number of characters to read is the same as the @code{response-content-length}. Returns @code{#f} if there was no response body. @end defun @defun write-response-body/latin-1 r body Write @var{body}, a string encodable in ISO-8859-1, to the port corresponding to the HTTP response @var{r}. @end defun @defun read-response-body/bytevector r Reads the response body from @var{r}, as a bytevector. Returns @code{#f} if there was no response body. @end defun @defun write-response-body/bytevector r bv Write @var{body}, a bytevector, to the port corresponding to the HTTP response @var{r}. @end defun As with requests, the various headers that are typically associated with HTTP responses may be accessed with these dedicated accessors. @xref{HTTP Headers}, for more information on the format of parsed headers. @defun response-accept-ranges response [default=#f] @defunx response-age response [default='()] @defunx response-allow response [default='()] @defunx response-cache-control response [default='()] @defunx response-connection response [default='()] @defunx response-content-encoding response [default='()] @defunx response-content-language response [default='()] @defunx response-content-length response [default=#f] @defunx response-content-location response [default=#f] @defunx response-content-md5 response [default=#f] @defunx response-content-range response [default=#f] @defunx response-content-type response [default=#f] @defunx response-date response [default=#f] @defunx response-etag response [default=#f] @defunx response-expires response [default=#f] @defunx response-last-modified response [default=#f] @defunx response-location response [default=#f] @defunx response-pragma response [default='()] @defunx response-proxy-authenticate response [default=#f] @defunx response-retry-after response [default=#f] @defunx response-server response [default=#f] @defunx response-trailer response [default='()] @defunx response-transfer-encoding response [default='()] @defunx response-upgrade response [default='()] @defunx response-vary response [default='()] @defunx response-via response [default='()] @defunx response-warning response [default='()] @defunx response-www-authenticate response [default=#f] Return the given request header, or @var{default} if none was present. @end defun @node Web Server @subsection Web Server @code{(web server)} is a generic web server interface, along with a main loop implementation for web servers controlled by Guile. @example (use-modules (web server)) @end example The lowest layer is the @code{