diff --git a/doc/ref/guile.texi b/doc/ref/guile.texi index d49c4f36a..983080a08 100644 --- a/doc/ref/guile.texi +++ b/doc/ref/guile.texi @@ -177,7 +177,7 @@ x * Guile Modules:: -* History and Implementation Details:: +* Guile Implementation:: * Autoconf Support:: @@ -367,15 +367,16 @@ available through both Scheme and C interfaces. @include autoconf.texi -@node History and Implementation Details -@chapter History and Implementation Details +@node Guile Implementation +@chapter Guile Implementation -Some mumblings about Guile as an artifact of historical processes; -knowledge of this history useful when hacking the source code. -Libguile as the end product of +Things that aren't necessary to know to use guile, but that are +interesting once you decide that Guile is interesting. + +Also Schemers as closet compiler writers. @menu -* A Brief History of Guile:: Foo. +* History:: A brief history of Guile. * Data Representation in Scheme:: Why things aren't just totally straightforward, in general terms. * The Libguile Runtime Environment:: Low-level details on Guile's C diff --git a/doc/ref/history.texi b/doc/ref/history.texi index 93d7062e0..c8c077a56 100644 --- a/doc/ref/history.texi +++ b/doc/ref/history.texi @@ -4,43 +4,280 @@ @c Free Software Foundation, Inc. @c See the file guile.texi for copying conditions. -@node A Brief History of Guile +@node History @section A Brief History of Guile +Guile is an artifact of historical processes, both as code and as a +community of hackers. It is sometimes useful to know this history when +hacking the source code, to know about past decisions and future +directions. + +Of course, the real history of Guile is written by the hackers hacking +and not the writers writing, so we round up the section with a note on +current status and future directions. + @menu -* In the Beginning There Was Emacs:: -* The Tcl Wars:: +* The Emacs Thesis:: * Early Days:: -* Adolescence:: -* Maturity:: +* A Scheme of Many Maintainers:: +* A Timeline of Selected Guile Releases:: +* Status:: @end menu -@node In the Beginning There Was Emacs -@subsection In the Beginning, There Was Emacs +@node The Emacs Thesis +@subsection The Emacs Thesis -@node The Tcl Wars -@subsection The ``Tcl Wars'' +The story of Guile is the story of bringing the development experience +of Emacs to the mass of programs on a GNU system. + +Emacs, when it was first created in its GNU form in 1984, was a new +take on the problem of ``how to make a program''. The Emacs thesis is +that it is delightful to create composite programs based on an +orthogonal kernel written in a low-level language together with a +powerful, high-level extension language. + +Extension languages foster extensible programs, programs which adapt +readily to different users and to changing times. Proof of this can be +seen in Emacs' current and continued existence, spanning more than a +quarter-century. + +Besides providing for modification of a program by others, extension +languages are good for /intension/ as well. Programs built in ``the +Emacs way'' are pleasurable and easy for their authors to flesh out +with the features that they need. + +After the Emacs experience was appreciated more widely, a number of +hackers started to consider how to spread this experience to the rest +of the GNU system. It was clear that the easiest way to Emacsify a +program would be to embed a shared language implementation into it. @node Early Days @subsection Early Days -The naming (scheme, plan, connive) +Tom Lord was the first to fully concentrate his efforts on an +embeddable language runtime, which he named ``GEL'', the GNU Extension +Language. -GEL -> GUILE -> Guile +GEL was the product of converting SCM, Aubrey Jaffer's implementation +of Scheme, into something more appropriate to embedding as a library. +(SCM was itself based on an implementation by George Carrette, SIOD). -Multilingual vision +Lord managed to convince Richard Stallman to dub GEL the official +extension language for the GNU project. It was a natural fit, given +that Scheme was a cleaner, more modern Lisp than Emacs Lisp. Part of +the argument was that eventually when GEL became more capable, it +could gain the ability to execute other languages, especially Emacs +Lisp. -@node Adolescence -@subsection Adolescence +Due to a naming conflict with another programming language, Jim Blandy +suggested a new name for GEL: ``Guile''. Besides being a recursive +acroymn, ``Guile'' craftily follows the naming of its ancestors, +``Planner'', ``Conniver'', and ``Schemer''. (The latter was truncated +to ``Scheme'' due to a 6-character file name limit on an old operating +system.) Finally, ``Guile'' suggests ``guy-ell'', or ``Guy L. +Steele'', who, together with Gerald Sussman, originally discovered +Scheme. -GOOPS +Around the same time that Guile (then GEL) was readying itself for +public release, another extension language was gaining in popularity, +Tcl. Many developers found advantages in Tcl because of its shell-like +syntax and its well-developed graphical widgets library, Tk. Also, at +the time there was a large marketing push promoting Tcl as a +``universal extension language''. -the module system +Richard Stallman, as the primary author of GNU Emacs, had a particular +vision of what extension languages should be, and Tcl did not seem to +him to be as capable as Emacs Lisp. He posted a criticism to the +comp.lang.tcl newsgroup, sparking one of the internet's legendary +flamewars. As part of these discussions, retrospectively dubbed the +``Tcl Wars'', he announced the Free Software Foundation's intent to +promote Guile as the extension language for the GNU project. -@node Maturity -@subsection Maturity +It is a common misconception that Guile was created as a reaction to +Tcl. While it is true that the public announcement of Guile happened +at the same time as the ``Tcl wars'', Guile was created out of a +condition that existed outside the polemic. Indeed, the need for a +powerful language to bridge the gap between extension of existing +applications and a more fully dynamic programming environment is still +with us today. -1.6, 1.8, ... +@node A Scheme of Many Maintainers +@subsection A Scheme of Many Mantainers -pthreads +Surveying the field, it seems that Scheme implementations correspond +with their maintainers on an N-to-1 relationship. That is to say, that +those people that implement Schemes might do so on a number of +occasions, but that the lifetime of a given Scheme is tied to the +maintainership of one individual. +Guile is atypical in this regard. + +Tom Lord maintaned Guile for its first year and a half or so, +corresponding to the end of 1994 through the middle of 1996. The +releases made in this time constitute an arc from SCM as a standalone +program to Guile as a reusable, embeddable library, but passing +through a explosion of features: embedded Tcl and Tk, a toolchain for +compiling and disassembling Java, addition of a C-like syntax, +creation of a module system, and a start at a rich POSIX interface. + +Only some of those features remain in Guile. There were ongoing +tensions between providing a small, embeddable language, and one which +had all of the features (e.g. a graphical toolkit) that a modern Emacs +might need. In the end, as Guile gained in uptake, the development +team decided to focus on depth, documentation and orthogonality rather +than on breadth. This has been the focus of Guile ever since, although +there is a wide range of third-party libraries for Guile. + +Jim Blandy presided over that period of stabilization, in the three +years until the end of 1999, when he too moved on to other projects. +Since then, Guile has had a group maintainership. The first group was +Maciej Stachowiak, Mikael Djurfeldt, and Marius Vollmer, with Vollmer +staying on the longest. By late 2007, Vollmer had mostly moved on to +other things, so Neil Jerram and Ludovic Courtès stepped up to take on +the primary maintenance responsibility. + +Of course, a large part of the actual work on Guile has come from +other contributors too numerous to mention, but without whom the world +would be a poorer place. + +@node A Timeline of Selected Guile Releases +@subsection A Timeline of Selected Guile Releases + +@table @asis +@item guile-i --- 4 February 1995 +SCM, turned into a library. + +@item guile-ii --- 6 April 1995 +A low-level module system was added. Tcl/Tk support was added, +allowing extension of Scheme by Tcl or vice versa. POSIX support was +improved, and there was an experimental stab at Java integration. + +@item guile-iii --- 18 August 1995 +The C-like syntax, ctax, was improved, but mostly this release +featured a start at the task of breaking Guile into pieces. + +@item 1.0 --- 5 January 1997 +@code{#f} was distinguished from @code{'()}. Green threads were added. +Source-level debugging became more useful, and programmer's and user's +manuals were begun. The module system gained a high-level interface, +which is still used today in more or less the same form. + +@item 1.1 --- 16 May 1997 +@itemx 1.2 --- 24 June 1997 +Support for Tcl/Tk and ctax were split off as separate packages, and +have remained there since. Guile became more compatible with SCSH, and +more useful as a UNIX scripting language. Libguile can now be built as +a shared library, and third-party extensions written in C became +loadable via dynamic linking. + +@item 1.3.0 --- 19 October 1998 +Command-line editing became much more pleasant through the use of the +readline library. The initial support for internationalization via +multi-byte strings was removed, and has yet to be added back, though +UTF-8 hacks are common. Modules gained the ability to have custom +expanders, which is still used for syntax-case macros. Ports have +better support for file descriptors, and fluids were added. + +@item 1.3.2 --- 20 August 1999 +@itemx 1.3.4 --- 25 September 1999 +@itemx 1.4 --- 21 June 2000 +A long list of lispy features were added: hooks, Common Lisp's +@code{format}, optional and keyword procedure arguments, +@code{getopt-long}, sorting, random numbers, and many other fixes and +enhancements. Guile now has an interactive debugger, interactive help, +and gives better backtraces. + +@item 1.6 --- 6 September 2002 +Guile gained support for the R5RS standard, and added a number of SRFI +modules. The module system was expanded with programmatic support for +identifier selection and renaming. The GOOPS object system was merged +into Guile core. + +@item 1.8 --- 20 February 2006 +Guile's arbitrary-precision arithmetic switched to use the GMP +library, and added support for exact rationals. Green threads were +removed in favor of POSIX threads, providing true multiprocessing. +Gettext support was added, and Guile's C API was cleaned up and +orthogonalized in a massive way. + +@item 2.0 --- thus far, only unstable snapshots available +A virtual machine was added to Guile, along with the associated +compiler and toolchain. Support for locales was added. Running Guile +instances became controllable and debuggable from within Emacs, via +GDS. GDS was backported to 1.8.5. An SRFI-compatible interface to +multithreading was added, including thread cancellation. +@end table + +@node Status +@subsection Status, or: Your Help Needed + +Guile has achieved much of what it set out to achieve, but there is +much remaining to do. + +There is still the old problem of bringing existing applications into +a more Emacs-like experience. Guile has had some successes in this +respect, but still most applications in the GNU system are without +Guile integration. + +Getting Guile to those applications takes an investment, the +``hacktivation energy'' needed to wire Guile into a program that only +pays off once it is good enough to enable new kinds of behavior. This +would be a great way for new hackers to contribute: take an +application that you use and that you know well, think of something +that it can't yet do, and figure out a way to integrate Guile and +implement that task in Guile. + +With time, perhaps this exposure can reverse itself, whereby programs +can run under Guile instead of vice versa, eventually resulting in the +Emacsification of the entire GNU system. Indeed, this is the reason +for the naming of the many Guile modules that live in the @code{ice-9} +namespace, a nod to the fictional substance in Kurt Vonnegut's +novel, Cat's Cradle, capable of acting as a seed crystal to +crystallize the mass of software. + +Implicit to this whole discussion is the idea that dynamic languages +are somehow better than languages like C. While languages like C have +their place, Guile's take on this question is that yes, Scheme is more +expressive than C, and more fun to write. This realization carries an +imperative with it to write as much code in Scheme as possible rather +than in other languages. + +These days it is possible to write extensible applications almost +entirely from high-level languages, through byte-code and native +compilation, speed gains in the underlying hardware, and foreign call +interfaces in the high-level language. Smalltalk systems are like +this, as are Common Lisp-based systems. While there already are a +number of pure-Guile applications out there, users still need to drop +down to C for some tasks: interfacing to system libraries that don't +have prebuilt Guile interfaces, and for some tasks requiring high +performance. + +The addition of the virtual machine in Guile 2.0, together with the +compiler infrastructure, should go a long way to addressing the speed +issues. But there is much optimization to be done. Interested +contributors will find lots of delightful low-hanging fruit, from +simple profile-driven optimization to hacking a just-in-time compiler +from VM bytecode to native code. + +Still, even with an all-Guile application, sometimes you want to +provide an opportunity for users to extend your program from a +language with a syntax that is closer to C, or to Python. Another +interesting idea to consider is compiling e.g. Python to Guile. It's +not that far-fetched of an idea: see for example IronPython or JRuby. + +And then there's Emacs itself. Though there is a somewhat-working +Emacs Lisp translator for Guile, it cannot yet execute all of Emacs +Lisp. A serious integration of Guile with Emacs would replace the +Elisp virtual machine with Guile, and provide the necessary C shims so +that Guile could emulate Emacs' C API. This would give lots of +exciting things to Emacs: native threads, a real object system, more +sophisticated types, cleaner syntax, and access to all of the Guile +extensions. + +Finally, there is another axis of crystallization, the axis between +different Scheme implementations. Guile does not yet support the +latest Scheme standard, R6RS, and should do so. Like all standards, +R6RS is imperfect, but supporting it will allow more code to run on +Guile without modification, and will allow Guile hackers to produce +code compatible with other schemes. Help in this regard would be much +appreciated. diff --git a/doc/ref/vm.texi b/doc/ref/vm.texi index bb5d16c52..bc8a55aee 100644 --- a/doc/ref/vm.texi +++ b/doc/ref/vm.texi @@ -7,6 +7,19 @@ @node A Virtual Machine for Guile @section A Virtual Machine for Guile +Guile has both an interpreter and a compiler. To a user, the +difference is largely transparent -- interpreted and compiled +procedures can call each other as they please. + +The difference is that the compiler creates and interprets bytecode +for a custom virtual machine, instead of interpreting the +S-expressions directly. Running compiled code is faster than running +interpreted code. + +The virtual machine that does the bytecode interpretation is a part of +Guile itself. This section describes the nature of Guile's virtual +machine. + @menu * Why a VM?:: * VM Concepts:: @@ -19,7 +32,47 @@ @node Why a VM? @subsection Why a VM? -asdfa +For a long time, Guile only had an interpreter, called the evaluator. +Guile's evaluator operates directly on the S-expression representation +of Scheme source code. + +But while the evaluator is highly optimized and hand-tuned, and +contains some extensive speed trickery (REFFIXME memoization), it +still performs many needless computations during the course of +evaluating an expression. For example, application of a function to +arguments needlessly conses up the arguments in a list. Evaluation of +an expression always has to figure out what the car of the expression +is -- a procedure, a memoized form, or something else. All values have +to be allocated on the heap. Et cetera. + +The solution to this problem is to compile the higher-level language, +Scheme, into a lower-level language for which all of the checks and +dispatching have already been done -- the code is instead stripped to +the bare minimum needed to ``do the job''. + +The question becomes then, what low-level language to choose? There +are many options. We could compile to native code directly, but that +poses portability problems for Guile, as it is a highly cross-platform +project. + +So we want the performance gains that compilation provides, but we +also want to maintain the portability benefits of a single code path. +The obvious solution is to compile to a virtual machine that is +present on all Guile installations. + +The easiest (and most fun) way to depend on a virtual machine is to +implement the virtual machine within Guile itself. This way the +virtual machine provides what Scheme needs (tail calls, multiple +values, call/cc) and can provide optimized inline instructions for +Guile (cons, struct-ref, etc.). + +So this is what Guile does. The rest of this section describes that VM +that Guile implements, and the compiled procedures that run on it. + +Note that this decision to implement a bytecode compiler does not +preclude native compilation. We can compile from bytecode to native +code at runtime, or even do ahead of time compilation. More +possibilities are discussed in REFFIXME. @node VM Concepts @subsection VM Concepts @@ -184,18 +237,17 @@ Within the lambda expression, "foo" is a top-level variable, "a" is a lexically captured variable, and "b" is a local variable. That is to say: @code{b} may safely be allocated on the stack, as -there is no enclosing lexical environment that references it, nor is -it ever mutated. +there is no enclosed procedure that references it, nor is it ever +mutated. -@code{a}, on the other hand, is referenced by an enclosed lexical -context, that of the lambda. Thus it must be allocated on the heap, as -it may (and will) outlive the dynamic extent of the invocation of -@code{foo}. +@code{a}, on the other hand, is referenced by an enclosed procedure, +that of the lambda. Thus it must be allocated on the heap, as it may +(and will) outlive the dynamic extent of the invocation of @code{foo}. @code{foo} is a toplevel variable, as mandated by Scheme's semantics: @example - (define proc (foo 'bar)) + (define proc (foo 'bar)) ; assuming prev. definition of @code{foo} (define foo 42) ; redefinition (proc 'baz) @result{} (42 bar baz) @@ -642,7 +694,7 @@ Create and fill a vector with the top @var{n} values from the stack, popping off those values and pushing on the resulting vector. @end deffn -@deffn mark +@deffn Instruction mark Pushes a special value onto the stack that other stack instructions like @code{list-mark} can use. @end deffn