start at documenting the compiler

* doc/ref/api-evaluation.texi: Add documentation for the standard
  compilation interface, and some notes about compiled files.

* doc/ref/api-procedures.texi (Compiled Procedures): A stub at
  documenting compiled procedures.

* doc/ref/compiler.texi (Compiling to the Virtual Machine): Flesh out
  with some structure, though much of the text remains to be written.
  This stuff is hard to write!

doc/ref/api-evaluation.texi

@@ -5,20 +5,22 @@
 @c See the file guile.texi for copying conditions.
 
 @page
-@node Read/Load/Eval
+@node Read/Load/Eval/Compile
 @section Reading and Evaluating Scheme Code
 
 This chapter describes Guile functions that are concerned with reading,
-loading and evaluating Scheme code at run time.
+loading, evaluating, and compiling Scheme code at run time.
 
 @menu
 * Scheme Syntax::               Standard and extended Scheme syntax.
 * Scheme Read::                 Reading Scheme code.
 * Fly Evaluation::              Procedures for on the fly evaluation.
+* Compilation::                 How to compile Scheme files and procedures.
 * Loading::                     Loading Scheme code from file.
 * Delayed Evaluation::          Postponing evaluation until it is needed.
 * Local Evaluation::            Evaluation in a local environment.
 * Evaluator Behaviour::         Modifying Guile's evaluator.
+* VM Behaviour::                Modifying Guile's virtual machine.
 @end menu
 
 
@@ -411,6 +413,69 @@ the current module.
 @end deffn
 
 
+@node Compilation
+@subsection Compiling Scheme Code
+
+The @code{eval} procedure directly interprets the S-expression
+representation of Scheme. An alternate strategy for evaluation is to
+determine ahead of time what computations will be necessary to
+evaluate the expression, and then use that recipe to produce the
+desired results. This is known as @dfn{compilation}.
+
+While it is possible to compile simple Scheme expressions such as
+@code{(+ 2 2)} or even @code{"Hello world!"}, compilation is most
+interesting in th context of procedures. Compiling a lambda expression
+produces a compiled procedure, which is just like a normal procedure
+except typically much faster, because it can bypass the generic
+interpreter.
+
+Functions from system modules in a Guile installation are normally
+compiled already, so they load and run quickly.
+
+Note that well-written Scheme programs will not typically call the
+procedures in this section, for the same reason that it is often bad
+taste to use @code{eval}. The normal interface to the compiler is the
+command-line file compiler, which can be invoked from the shell as
+@code{guile-tools compile @var{foo.scm}}. This interface needs more
+documentation.
+
+(Why are calls to @code{eval} and @code{compile} usually in bad taste?
+Because they are limited, in that they can only really make sense for
+top-level expressions. Also, most needs for ``compile-time''
+computation are fulfilled by macros and closures. Of course one good
+counterexample is the REPL itself, or any code that reads expressions
+from a port.)
+
+For more information on the compiler itself, @xref{Compiling to the
+Virtual Machine}. For information on the virtual machine, @xref{A
+Virtual Machine for Guile}.
+
+@deffn {Scheme Procedure} compile exp [env=#f] [from=(current-language)] [to=value] [opts=()]
+Compile the expression @var{exp} in the environment @var{env}. If
+@var{exp} is a procedure, the result will be a compiled procedure;
+otherwise @code{compile} is mostly equivalent to @code{eval}.
+
+For a discussion of languages and compiler options, @xref{Compiling to
+the Virtual Machine}.
+@end deffn
+
+@deffn {Scheme Procedure} compile-file file [to=objcode] [opts='()]
+Compile the file named @var{file}.
+
+Output will be written to a file in the current directory whose name
+is computed as @code{(compiled-file-name @var{file})}.
+@end deffn
+
+@deffn {Scheme Procedure} compiled-file-name file
+Compute an appropriate name for a compiled version of a Scheme file
+named @var{file}.
+
+Usually, the result will be the original file name with the
+@code{.scm} suffix replaced with @code{.go}, but the exact behavior
+depends on the contents of the @code{%load-extensions} and
+@code{%load-compiled-extensions} lists.
+@end deffn
+
 @node Loading
 @subsection Loading Scheme Code from File
 
@@ -435,9 +500,19 @@ procedure that will be called before any code is loaded.  See
 documentation for @code{%load-hook} later in this section.
 @end deffn
 
+@deffn {Scheme Procedure} load-compiled filename
+Load the compiled file named @var{filename}. The load paths are not
+searched.
+
+Compiling a source file (@pxref{Read/Load/Eval/Compile}) and then
+calling @code{load-compiled} on the resulting file is equivalent to
+calling @code{load} on the source file.
+@end deffn
+
 @deffn {Scheme Procedure} load-from-path filename
 Similar to @code{load}, but searches for @var{filename} in the load
-paths.
+paths. Preferentially loads a compiled version of the file, if it is
+available and up-to-date.
 @end deffn
 
 @deffn {Scheme Procedure} primitive-load filename
@@ -461,7 +536,8 @@ documentation for @code{%load-hook} later in this section.
 Search @code{%load-path} for the file named @var{filename} and
 load it into the top-level environment.  If @var{filename} is a
 relative pathname and is not found in the list of search paths,
-an error is signalled.
+an error is signalled. Preferentially loads a compiled version of the
+file, if it is available and up-to-date.
 @end deffn
 
 @deffn {Scheme Procedure} %search-load-path filename
@@ -639,6 +715,30 @@ trap handlers.
 Option interface for the evaluator trap options.
 @end deffn
 
+@node VM Behaviour
+@subsection VM Behaviour
+
+Like the procedures from the previous section that operate on the
+evaluator, there are also procedures to modify the behavior of a
+virtual machine.
+
+The most useful thing that a user can do is to add to one of the
+virtual machine's predefined hooks:
+
+@deffn {Scheme Procedure} vm-next-hook vm
+@deffnx {Scheme Procedure} vm-apply-hook vm
+@deffnx {Scheme Procedure} vm-boot-hook vm
+@deffnx {Scheme Procedure} vm-return-hook vm
+@deffnx {Scheme Procedure} vm-break-hook vm
+@deffnx {Scheme Procedure} vm-exit-hook vm
+@deffnx {Scheme Procedure} vm-halt-hook vm
+@deffnx {Scheme Procedure} vm-enter-hook vm
+Accessors to a virtual machine's hooks. Usually you pass
+@code{(the-vm)} as the @var{vm}.
+@end deffn
+
+@xref{A Virtual Machine for Guile}, for more information on Guile's
+virtual machine.
 
 @c Local Variables:
 @c TeX-master: "guile.texi"

doc/ref/api-procedures.texi

@@ -11,6 +11,7 @@
 @menu
 * Lambda::                      Basic procedure creation using lambda.
 * Primitive Procedures::        Procedures defined in C.
+* Compiled Procedures::         Scheme procedures can be compiled.
 * Optional Arguments::          Handling keyword, optional and rest arguments.
 * Procedure Properties::        Procedure properties and meta-information.
 * Procedures with Setters::     Procedures with setters.
@@ -131,6 +132,21 @@ use @code{scm_c_make_subr} and also @code{scm_makcclo} if necessary.
 It is advisable to use the gsubr variants since they provide a
 slightly higher-level abstraction of the Guile implementation.
 
+@node Compiled Procedures
+@subsection Compiled Procedures
+
+Procedures that were created when loading a compiled file are
+themselves compiled. (In contrast, procedures that are defined by
+loading a Scheme source file are interpreted, and often not as fast as
+compiled procedures.)
+
+Loading compiled files is the normal way that compiled procedures come
+to being, though procedures can be compiled at runtime as well.
+@xref{Read/Load/Eval/Compile}, for more information on runtime
+compilation.
+
+here document things from (system vm program)
+
 @node Optional Arguments
 @subsection Optional Arguments
 

doc/ref/compiler.texi

@@ -7,3 +7,126 @@
 @node Compiling to the Virtual Machine
 @section Compiling to the Virtual Machine
 
+Compilers have a mystique about them that is attractive and
+off-putting at the same time. They are attractive because they are
+magical -- they transform inert text into live results, like throwing
+the switch on Frankenstein. However, this magic is perceived by many
+to be impenetrable.
+
+This section aims to pull back the veil from over Guile's compiler
+implementation, some reference to the wizard of oz FIXME.
+
+REFFIXME, if you're lost and you just wanted to know how to compile
+your .scm file.
+
+@menu
+* Compiler Tower::                   
+* The Scheme Compiler::                   
+* GHIL::                 
+* GLIL::                
+* Object Code::                   
+@end menu
+
+@node Compiler Tower
+@subsection Compiler Tower
+
+Guile's compiler is quite simple, actually -- its @emph{compilers}, to
+put it more accurately. Guile defines a tower of languages, starting
+at Scheme and progressively simplifying down to languages that
+resemble the VM instruction set (REFFIXME).
+
+Each language knows how to compile to the next, so each step is simple
+and understandable. Furthermore, this set of languages is not
+hardcoded into Guile, so it is possible for the user to add new
+high-level languages, new passes, or even different compilation
+targets.
+
+lookup-language
+(lang xxx spec)
+
+(system-base-language)
+
+describe:
+
+(define-record <language>
+  name
+  title
+  version
+  reader
+  printer
+  (parser #f)
+  (read-file #f)
+  (compilers '())
+  (evaluator #f))
+
+(define-macro (define-language name . spec)
+
+(lookup-compilation-order from to)
+
+language definition
+
+compiling from here to there
+
+the normal tower: scheme, ghil, glil, object code
+maybe from there serialized to disk
+or if at repl, brought back to life by compiling to ``value''
+
+compile-file defaults to compiling to objcode
+compile defaults to compiling to value
+
+((lambda (x) ((compile x) x)) '(lambda (x) ((compile x) x)))
+quine
+
+@node The Scheme Compiler
+@subsection The Scheme Compiler
+
+macro expansion
+
+define-scheme-translator
+
+inlining
+
+format of the environment
+
+compile-time-environment
+
+symbols resolved as local, external, or toplevel
+
+@node GHIL
+@subsection GHIL
+
+ghil environments
+
+structured, typed intermediate language, close to scheme
+with an s-expression representation
+
+,lang ghil
+
+some pre-optimization
+
+real name of the game is closure elimination -- fixing letrec
+
+@node GLIL
+@subsection GLIL
+
+structured, typed intermediate language, close to object code
+
+passes through the env
+
+no let, no lambda, no closures, just labels and branches and constants
+and code. Well, there's a bit more, but that's the flavor of GLIL.
+
+Compiled code will effectively be a thunk, of no arguments, but
+optionally closing over some number of variables (which should be
+captured via `make-closure' REFFIXME.
+
+@node Object Code
+@subsection Object Code
+
+describe the env -- module + externals (the actual values!)
+
+The env is used when compiling to value -- effectively calling the
+thunk from objcode->program with a certain current module and with
+those externals. so you can recompile a closure at runtime, a trick
+that goops uses.
+

doc/ref/guile.texi

@@ -301,7 +301,7 @@ available through both Scheme and C interfaces.
 * Binding Constructs::          Definitions and variable bindings.
 * Control Mechanisms::          Controlling the flow of program execution.
 * Input and Output::            Ports, reading and writing.
-* Read/Load/Eval::              Reading and evaluating Scheme code.
+* Read/Load/Eval/Compile::      Reading and evaluating Scheme code.
 * Memory Management::           Memory management and garbage collection.
 * Objects::                     Low level object orientation support.
 * Modules::                     Designing reusable code libraries.

doc/ref/vm.texi

@@ -25,7 +25,7 @@ machine.
 * VM Concepts::                 
 * Stack Layout::                
 * Variables and the VM::                   
-* Compiled Procedures::         
+* VM Programs::         
 * Instruction Set::
 @end menu
 
@@ -258,8 +258,8 @@ counterintuitively, what would seem ``closer to the metal'', viz
 @code{set!}, actually forces heap allocation instead of stack
 allocation.
 
-@node Compiled Procedures
-@subsection Compiled Procedures
+@node VM Programs
+@subsection Compiled Procedures are VM Programs
 
 By default, when you enter in expressions at Guile's REPL, they are
 first compiled to VM object code, then that VM object code is executed