Debugging Features
==================

Guile includes debugging tools to help you work out what is going wrong when a program signals an error or behaves differently to how you would expect.  This chapter describes how to use these tools.

Broadly speaking, Guile's debugging support allows you to do two things:

  * specify breakpoints --- points in the execution of a program where execution should pause so you can see what is going on

  * examine in detail the ``scene of the crime'' --- in other words, the execution context at a breakpoint, or when the last error occurred.

The details are more complex and more powerful ...

Debugging the Most Recent Error
-------------------------------

When an error is signalled, Guile remembers the execution context where the error occurred.  By default, Guile then displays only the most immediate information about where and why the error occurred, for example:

     (make-string (* 4 (+ 3 #\s)) #\space)
     -|
     standard input:2:19: In procedure + in expression (+ 3 #\s):
     standard input:2:19: Wrong type argument: #\s
     ABORT: (wrong-type-arg)

     Type "(backtrace)" to get more information or "(debug)" to enter the debugger.

However, as the message above says, you can obtain much more information about the context of the error by typing (backtrace) or (debug).

(backtrace) displays the Scheme call stack at the point where the error occurred:

     (backtrace)
     -|
     Backtrace:
     In standard input:
        2: 0* [make-string ...
        2: 1*  [* 4 ...
        2: 2*   [+ 3 #\s]

     Type "(debug-enable 'backtrace)" if you would like a backtrace
     automatically if an error occurs in the future.

In a more complex scenario than this one, this can be extremely useful for understanding where and why the error occurred.  For more on the format of the displayed backtrace, see the subsection below.

(debug) takes you into Guile's interactive debugger, which provides commands that allow you to

  * display the Scheme call stack at the point where the error occurred (the backtrace command --- see Display Backtrace)

  * move up and down the call stack, to see in detail the expression being evaluated, or the procedure being applied, in each frame (the up, down, frame, position, info args and info frame commands --- see Frame Selection and Frame Information)

  * examine the values of variables and expressions in the context of each frame (the evaluate command --- see Frame Evaluation).

Use of the interactive debugger, including these commands, is described in Interactive Debugger.

How to Interpret a Backtrace
............................

Intro to Breakpoints
--------------------

If you are not already familiar with the concept of breakpoints, the first subsection below explains how they work are why they are useful.

Broadly speaking, Guile's breakpoint support consists of

  * type-specific features for creating breakpoints of various types

  * relatively generic features for manipulating the behaviour of breakpoints once they've been created.

Different breakpoint types are implemented as different classes in a GOOPS hierarchy with common base class <breakpoint>.  The magic of generic functions then allows most of the manipulation functions to be generic by default but specializable (by breakpoint class) if the need arises.

Generic breakpoint support is provided by the (ice-9 debugger breakpoints) module, so you will almost always need to use this module in order to access the functionality described here:

     (use-modules (ice-9 debugger breakpoints))

You may like to add this to your .guile file.

How Breakpoints Work and Why They Are Useful
............................................

Often, debugging the last error is not enough to tell you what went wrong.  For example, the root cause of the error may have arisen a long time before the error was signalled, in which case the execution context of the error is too late to be useful.  Or your program might not signal an error at all, just return an unexpected result or have some incorrect side effect.

In many such cases, it's useful to pause the program at or before the point where you suspect the problem arises.  Then you can explore the stack, display the values of key variables, and generally check that the state of the program is as you expect.  If all is well, you can let the program continue running normally, or step more slowly through each expression that the Scheme interpreter evaluates.  Single-stepping may reveal that the program is going through blocks of code that you didn't intend --- a useful data point for understanding what the underlying problem is.

Telling Guile where or when to pause a program is called setting a breakpoint.  When a breakpoint is hit, Guile's default behaviour is to enter the interactive debugger, where there are now two sets of commands available:

  * all the commands as described for last error debugging (see Debug Last Error), which allow you to explore the stack and so on

  * additional commands for continuing program execution in various ways: next, step, finish, trace-finish and continue.

Use of the interactive debugger is described in Interactive Debugger.

Source Breakpoints
..................

A source breakpoint is a breakpoint that triggers whenever program execution hits a particular source location.  A source breakpoint can be conveniently set simply by evaluating code that has ## inserted into it at the position where you want the breakpoint to be.

For example, to set a breakpoint immediately before evaluation of (= n 0) in the following procedure definition, evaluate:

     (define (fact1 n)
       (if ##(= n 0)
           1
           (* n (fact1 (- n 1)))))
     -|
     Set breakpoint 1: standard input:4:9: (= n 0)

Note the message confirming that you have set a breakpoint.  If you don't see this, something isn't working.

## is provided by the (ice-9 debugger breakpoints source) module, so you must use this module before trying to set breakpoints in this way:

     (use-modules (ice-9 debugger breakpoints source))

You may like to add this to your .guile file.

The default behaviour for source breakpoints is debug-here (see Breakpoint Behaviours), which means to enter the command line debugger when the breakpoint is hit.  So, if you now use fact1, that is what happens.

     guile> (fact1 3)
     Hit breakpoint 1: standard input:4:9: (= n 0)
     Frame 3 at standard input:4:9
             (= n 0)
     debug>

Procedural Breakpoints
......................

A procedural breakpoint is a breakpoint that triggers whenever Guile is about to apply a specified procedure to its (already evaluated) arguments.  To set a procedural breakpoint, call break! with the target procedure as a single argument.  For example:

     (define (fact1 n)
       (if (= n 0)
           1
           (* n (fact1 (- n 1)))))

     (break! fact1)
     -|
     Set breakpoint 1: [fact1]
     =>
     #<<procedure-breakpoint> 808b0b0>

Alternatives to break! are trace! and trace-subtree!.  The difference is that these three calls create a breakpoint in the same place but with three different behaviours, respectively debug-here, trace-here and trace-subtree.  Breakpoint behaviours are documented fully later (see Breakpoint Behaviours), but to give a quick taste, here's an example of running code that includes a procedural breakpoint with the trace-here behaviour.

     (trace! fact1)
     -|
     Set breakpoint 1: [fact1]
     =>
     #<<procedure-breakpoint> 808b0b0>

     (fact1 4)
     -|
     |  [fact1 4]
     |  |  [fact1 3]
     |  |  |  [fact1 2]
     |  |  |  |  [fact1 1]
     |  |  |  |  |  [fact1 0]
     |  |  |  |  |  1
     |  |  |  |  2
     |  |  |  6
     |  |  24
     |  24
     =>
     24

To set and use procedural breakpoints, you will need to use the (ice-9 debugger breakpoints procedural) module:

     (use-modules (ice-9 debugger breakpoints procedural))

You may like to add this to your .guile file.

Setting Breakpoints
...................

In general, that is.  We've already seen how to set source and procedural breakpoints conveniently in practice.  This section explains how those conveniences map onto a more general mechanism.

The general mechanism for setting breakpoints is the generic function set-breakpoint!.  Different kinds of breakpoints define subclasses of the class <breakpoint> and provide their own methods for set-pbreakpoint!.

For example, (ice-9 debugger breakpoints procedural) implements the <procedure-breakpoint> subclass and provides a set-breakpoint! method that takes a procedure argument:

     (set-breakpoint! BEHAVIOR fact1)
     -|
     Set breakpoint 1: [fact1]
     =>
     #<<procedure-breakpoint> 808b0b0>

A non-type-specific set-breakpoint! method is provided by the generic module (ice-9 debugger breakpoints).  It allows you to change the behaviour of an existing breakpoint that is identified by its breakpoint number.

     (set-breakpoint! BEHAVIOR 1)

break! trace! trace-subtree!
............................

We have already talked above about the use of break!, trace! and trace-subtree! for setting procedural breakpoints.  Now that set-breakpoint! has been introduced, we can reveal that break!, trace! and trace-subtree! are in fact just wrappers for set-breakpoint! that specify particular breakpoint behaviours, respectively debug-here, trace-here and trace-subtree.

     (break! . ARGS)
         == (set-breakpoint! debug-here . ARGS)
     (trace! . ARGS)
         == (set-breakpoint! trace-here . ARGS)
     (trace-subtree! . ARGS)
         == (set-breakpoint! trace-subtree . ARGS)

This means that these three procedures can be used to set the corresponding behaviours for any type of breakpoint for which a set-breakpoint! method exists, not just procedural ones.

Accessing Breakpoints
.....................

Information about the state and behaviour of a breakpoint is stored in an instance of the appropriate breakpoint class.  To access and change that information, therefore, you need to get hold of the desired breakpoint instance.

The generic function get-breakpoint meets this need: For every set-breakpoint! method there is a corresponding get-breakpoint method.  Note especially the useful type-independent case:

     (get-breakpoint 1)
     =>
     #<<procedure-breakpoint> 808b0b0>

Breakpoint Behaviours
.....................

A breakpoint's behaviour determines what happens when that breakpoint is hit.  Several kinds of behaviour are generally useful.

debug-here
     Enter the command line debugger.  This gives the opportunity to explore the stack, evaluate expressions in any of the pending stack frames, change breakpoint properties or set new breakpoints, and continue program execution when you are done.

trace-here
     Trace the current stack frame.  For expressions being evaluated, this shows the expression.  For procedure applications, it shows the procedure name and its arguments post-evaluation.  For both expressions and applications, the indentation of the tracing indicates whether the traced items are mutually tail recursive.

trace-subtree
     Trace the current stack frame, and enable tracing for all future evaluations and applications until the current stack frame is exited. trace-subtree is a great preliminary exploration tool when all you know is that there is a bug ``somewhere in XXX or in something that XXX calls''.

(at-exit THUNK)
     Don't do anything now, but arrange for THUNK to be executed when the current stack frame is exited.  For example, the operation that most debugging tools call ``finish'' is (at-exit debug-here).

(at-next COUNT THUNK)
     ... arrange for THUNK to be executed when beginning the COUNTth next evaluation or application with source location in the current file.

(at-entry COUNT THUNK)
     ... arrange for THUNK to be executed when beginning the COUNTth next evaluation (regardless of source location).

(at-apply COUNT THUNK)
     ... arrange for THUNK to be executed just before performing the COUNTth next application (regardless of source location).

(at-step COUNT THUNK)
     Synthesis of at-entry and at-apply; counts both evaluations and applications.

Every breakpoint instance has a slot in which its behaviour is stored. If you have a breakpoint instance in hand, you can change its behaviour using the bp-behaviour accessor.

An accessor supports the setting of a property like this:

     (set! (bp-behaviour BREAKPOINT) NEW-BEHAVIOUR)

See the GOOPS manual for further information on accessors.

Alternatively, if you know how to specify the LOCATION-ARGS for the breakpoint in question, you can change its behaviour using set-breakpoint!.  For example:

     ;; Change behaviour of breakpoint number 2.
     (set-breakpoint! NEW-BEHAVIOUR 2)

     ;; Change behaviour of procedural breakpoint on [fact1].
     (set-breakpoint! NEW-BEHAVIOUR fact1)

In all cases, the behaviour that you specify should be either a single thunk, or a list of thunks, to be called when the breakpoint is hit.

The most common behaviours above are exported as thunks from the (ice-9 debugger behaviour) module.  So, if you use this module, you can use those behaviours directly like this:

     (use-modules (ice-9 debugger behaviour))
     (set-breakpoint! trace-subtree 2)
     (set! (bp-behaviour (get-breakpoint 3)) debug-here)

You can also use the list option to combine common behaviours:

     (set-breakpoint! (list trace-here debug-here) 2)

Or, for more customized behaviour, you could build and use your own thunk like this:

     (define (my-behaviour)
       (trace-here)
       (at-exit (lambda ()
                  (display "Exiting frame of my-behaviour bp\n")
                  ... do something unusual ...)))

     (set-breakpoint my-behaviour 2)

Enabling and Disabling
......................

Independently of its behaviour, each breakpoint also keeps track of whether it is currently enabled.  This is a straightforward convenience to allow breakpoints to be temporarily switched off without losing all their carefully constructed properties.

If you have a breakpoint instance in hand, you can enable or disable it using the bp-enabled? accessor.

Alternatively, you can enable or disable a breakpoint via its location args by using enable-breakpoint! or disable-breakpoint!.

     (disable-breakpoint! fact1)     ; disable the procedural breakpoint on fact1
     (enable-breakpoint! 1)          ; enable breakpoint 1

enable-breakpoint! and disable-breakpoint! are implemented using get-breakpoint and bp-enabled?, so any LOCATION-ARGS that are valid for get-breakpoint will work also for these procedures.

Deleting Breakpoints
....................

Given a breakpoint instance in hand, you can deactivate it and remove it from the global list of current breakpoints by calling bp-delete!.

Alternatively, you can delete a breakpoint by its location args:

     (delete-breakpoint! 1)         ; delete breakpoint 1

delete-breakpoint! is implemented using get-breakpoint and bp-delete!, so any LOCATION-ARGS that are valid for get-breakpoint will work also for delete-breakpoint!.

There is no way to reinstate a deleted breakpoint.  Final destruction of the breakpoint instance is determined by the usual garbage collection rules.

Breakpoint Information
......................

To get Guile to print a description of a breakpoint instance, use bp-describe:

     (bp-describe (get-breakpoint 1) #t)   ; #t specifies standard output
     -|
     Breakpoint 1: [fact1]
             enabled? = #t
             behaviour = #<procedure trace-here ()>

Following the usual model, describe-breakpoint is also provided:

     (describe-breakpoint 1)
     -|
     Breakpoint 1: [fact1]
             enabled? = #t
             behaviour = #<procedure trace-here ()>

Finally, two stragglers.  all-breakpoints returns a list of all current breakpoints.  describe-all-breakpoints combines bp-describe and all-breakpoints by printing a description of all current breakpoints to standard output.

Other Breakpoint Types
......................

Besides source and procedural breakpoints, Guile includes an early implementation of a third class of breakpoints: range breakpoints. These are breakpoints that trigger when program execution enters (or perhaps exits) a defined range of source locations.

Sadly, these don't yet work well.  The apparent problem is that the extra methods for set-breakpoint! and get-breakpoint cause some kind of explosion in the time taken by GOOPS to construct its method cache and to dispatch calls involving these generic functions. But we haven't really investigated enough to be sure that this is the real issue.

If you're interested in looking and/or investigating anyway, please feel free to check out and play with the (ice-9 debugger breakpoints range) module.

The other kind of breakpoint that we'd like to have is watchpoints, but this hasn't been implemented at all yet.  Watchpoints may turn out to be impractical for performance reasons.

Using the Interactive Debugger
------------------------------

Guile's interactive debugger is a command line application that accepts commands from you for examining the stack and, if at a breakpoint, for continuing program execution in various ways.  Unlike in the normal Guile REPL, commands are typed mostly without parentheses.

When you first enter the debugger, it introduces itself with a message like this:

     This is the Guile debugger -- for help, type `help'.
     There are 3 frames on the stack.

     Frame 2 at standard input:36:19
             [+ 3 #\s]
     debug>

``debug>'' is the debugger's prompt, and a useful reminder that you are not in the normal Guile REPL.  The available commands are described in detail in the following subsections.

Display Backtrace
.................

The backtrace command, which can also be invoked as bt or where, displays the call stack (aka backtrace) at the point where the debugger was entered:

     debug> bt
     In standard input:
       36: 0* [make-string ...
       36: 1*  [* 4 ...
       36: 2*   [+ 3 #\s]

 -- Debugger Command: backtrace [count]
 -- Debugger Command: bt [count]
 -- Debugger Command: where [count]
     Print backtrace of all stack frames, or of the innermost COUNT frames.  With a negative argument, print the outermost -COUNT frames.  If the number of frames isn't explicitly given, the debug option depth determines the maximum number of frames printed.

The format of the displayed backtrace is the same as for the backtrace procedure --- see Backtrace Format for details.

Frame Selection
...............

A call stack consists of a sequence of stack frames, with each frame describing one level of the nested evaluations and applications that the program was executing when it hit a breakpoint or an error. Frames are numbered such that frame 0 is the outermost --- i.e. the operation on the call stack that began least recently --- and frame N-1 the innermost (where N is the total number of frames on the stack).

When you enter the debugger, the innermost frame is selected, which means that the commands for getting information about the ``current'' frame, or for evaluating expressions in the context of the current frame, will do so by default with respect to the innermost frame.  To select a different frame, so that these operations will apply to it instead, use the up, down and frame commands like this:

     debug> up
     Frame 1 at standard input:36:14
             [* 4 ...
     debug> frame 0
     Frame 0 at standard input:36:1
             [make-string ...
     debug> down
     Frame 1 at standard input:36:14
             [* 4 ...

 -- Debugger Command: up [n]
     Move N frames up the stack.  For positive N, this advances toward the outermost frame, to higher frame numbers, to frames that have existed longer.  N defaults to one.

 -- Debugger Command: down [n]
     Move N frames down the stack.  For positive N, this advances toward the innermost frame, to lower frame numbers, to frames that were created more recently.  N defaults to one.

 -- Debugger Command: frame [n]
     Select and print a stack frame.  With no argument, print the selected stack frame.  (See also ``info frame''.)  An argument specifies the frame to select; it must be a stack-frame number.

Frame Information
.................

[to be completed]

 -- Debugger Command: info frame
     All about selected stack frame.

 -- Debugger Command: info args
     Argument variables of current stack frame.

 -- Debugger Command: position
     Display the position of the current expression.

Frame Evaluation
................

[to be completed]

 -- Debugger Command: evaluate expression
     Evaluate an expression. The expression must appear on the same line as the command, however it may be continued over multiple lines.

Single Stepping
...............

[to be completed]

 -- Debugger Command: step [n]
     Continue until entry to Nth next frame.

 -- Debugger Command: next [n]
     Continue until entry to Nth next frame in same file.

Run To Frame Exit
.................

[to be completed]

 -- Debugger Command: finish
     Continue until evaluation of the current frame is complete, and print the result obtained.

 -- Debugger Command: trace-finish
     Trace until evaluation of the current frame is complete.

Continue Execution
..................

[to be completed]

 -- Debugger Command: continue
     Continue program execution.

Leave Debugger
..............

[to be completed]

 -- Debugger Command: quit
     Exit the debugger.

Tracing
-------

Tracing has already been described as a breakpoint behaviour (see Breakpoint Behaviours), but we mention it again here because it is so useful, and because Guile actually now has two mechanisms for tracing, and its worth clarifying the differences between them.

Tracing Provided by (ice-9 debug)
.................................

The (ice-9 debug) module implements tracing of procedure applications.  When a procedure is traced, it means that every call to that procedure is reported to the user during a program run. The idea is that you can mark a collection of procedures for tracing, and Guile will subsequently print out a line of the form

     |  |  [PROCEDURE ARGS ...]

whenever a marked procedure is about to be applied to its arguments. This can help a programmer determine whether a function is being called at the wrong time or with the wrong set of arguments.

In addition, the indentation of the output is useful for demonstrating how the traced applications are or are not tail recursive with respect to each other.  Thus, a trace of a non-tail recursive factorial implementation looks like this:

     [fact1 4]
     |  [fact1 3]
     |  |  [fact1 2]
     |  |  |  [fact1 1]
     |  |  |  |  [fact1 0]
     |  |  |  |  1
     |  |  |  1
     |  |  2
     |  6
     24

While a typical tail recursive implementation would look more like this:

     [fact2 4]
     [facti 1 4]
     [facti 4 3]
     [facti 12 2]
     [facti 24 1]
     [facti 24 0]
     24

 -- Scheme Procedure: trace procedure
     Enable tracing for procedure.  While a program is being run, Guile will print a brief report at each call to a traced procedure, advising the user which procedure was called and the arguments that were passed to it.

 -- Scheme Procedure: untrace procedure
     Disable tracing for procedure.

Here is another example:

     (define (rev ls)
       (if (null? ls)
           '()
           (append (rev (cdr ls))
                   (cons (car ls) '())))) => rev

     (trace rev) => (rev)

     (rev '(a b c d e))
     => [rev (a b c d e)]
        |  [rev (b c d e)]
        |  |  [rev (c d e)]
        |  |  |  [rev (d e)]
        |  |  |  |  [rev (e)]
        |  |  |  |  |  [rev ()]
        |  |  |  |  |  ()
        |  |  |  |  (e)
        |  |  |  (e d)
        |  |  (e d c)
        |  (e d c b)
        (e d c b a)
        (e d c b a)

Note the way Guile indents the output, illustrating the depth of execution at each procedure call.  This can be used to demonstrate, for example, that Guile implements self-tail-recursion properly:

     (define (rev ls sl)
       (if (null? ls)
           sl
           (rev (cdr ls)
                (cons (car ls) sl)))) => rev

     (trace rev) => (rev)

     (rev '(a b c d e) '())
     => [rev (a b c d e) ()]
        [rev (b c d e) (a)]
        [rev (c d e) (b a)]
        [rev (d e) (c b a)]
        [rev (e) (d c b a)]
        [rev () (e d c b a)]
        (e d c b a)
        (e d c b a)

Since the tail call is effectively optimized to a goto statement, there is no need for Guile to create a new stack frame for each iteration.  Tracing reveals this optimization in operation.

Breakpoint-based Tracing
........................

Guile's newer mechanism implements tracing as an optional behaviour for any kind of breakpoint.

To trace a procedure (in the same kind of way as the older tracing), use the trace! procedure to set a procedure breakpoint with trace-here behaviour:

     (trace! fact1)
     -|
     Set breakpoint 1: [fact1]
     =>
     #<<procedure-breakpoint> 40337bf0>

     (fact1 4)
     -|
     |  [fact1 4]
     |  |  [fact1 3]
     |  |  |  [fact1 2]
     |  |  |  |  [fact1 1]
     |  |  |  |  |  [fact1 0]
     |  |  |  |  |  1
     |  |  |  |  2
     |  |  |  6
     |  |  24
     |  24
     =>
     24

To trace evaluation of a source expression, evaluate code containing a breakpoint marker ## in the appropriate place, then use set-breakpoint to change the behaviour of the new breakpoint to trace-here:

     (define (fact1 n)
       (if ##(= n 0)
           1
           (* n (fact1 (- n 1)))))
     -|
     Set breakpoint 4: standard input:13:9: (= n 0)

     (use-modules (ice-9 debugger behaviour))
     (set-breakpoint! trace-here 4)
     -|
     Breakpoint 4: standard input:13:9: (= n 0)
             enabled? = #t
             behaviour = #<procedure trace-here ()>

     (fact1 4)
     -|
     |  (= n 0)
     |  #f
     |  (= n 0)
     |  #f
     |  (= n 0)
     |  #f
     |  (= n 0)
     |  #f
     |  (= n 0)
     |  #t
     =>
     24

(Note --- this example reveals a bug: each occurrence of (= n 0) should be shown indented with respect to the one before it, as fact1 does not call itself tail-recursively.)

You can also give a breakpoint the trace-subtree behaviour, which means to trace the breakpoint location itself plus any evaluations and applications that occur below it in the call stack.  In the following example, this allows us to see the evaluated arguments that are being compared by the = procedure:

     (set-breakpoint! trace-subtree 4)
     -|
     Breakpoint 4: standard input:13:9: (= n 0)
             enabled? = #t
             behaviour = #<procedure trace-subtree ()>

     (fact1 4)
     -|
     |  (= n 0)
     |  [= 4 0]
     |  #f
     |  (= n 0)
     |  [= 3 0]
     |  #f
     |  (= n 0)
     |  [= 2 0]
     |  #f
     |  (= n 0)
     |  [= 1 0]
     |  #f
     |  (= n 0)
     |  [= 0 0]
     |  #t
     =>
     24

Differences Between Old and New Tracing Mechanisms
..................................................

The newer tracing mechanism is more general and so more powerful than the older one: it works for expressions as well as procedure applications, and it implements the useful trace-subtree behaviour as well as the more traditional trace-here.

The older mechanism will probably become obsolete eventually, but it's worth keeping it around for a while until we are sure that the new mechanism is correct and does what programmers need.