Tcl Tutorial Lesson CORO

Programming with coroutines

As of version 8.6, Tcl supports coroutines as a programming feature. The main advantage of coroutines is that they make certain tasks much simpler to implement, especially tasks that need to keep a lot of information around and need to cooperate with other tasks. Coroutines are useful in fact for implementing co-operative multitasking - not to be confused with multiprocessing.

Before we dive into details, here is a very simple example of how to create a coroutine:

    #
    # Define a procedure that will return a sequence of integers
    #
    proc next_integer {{step 1}} {
        yield
        set i [expr {-$step}]
        while {1} {
            yield [incr i $step]
        }
    }

    #
    # Define two coroutines - one for all integers
    # and one that returns the even ones
    #
    coroutine next1 next_integer
    coroutine next_even next_integer 2

    #
    # Print the first few
    #
    while {1} {
        set i1 [next1]
        set i2 [next_even]

        puts "$i1 -- $i2"

        if { $i2 > 10 } {
            break
        }
    }

The output is, as intended:

0 -- 0
1 -- 2
2 -- 4
3 -- 6
4 -- 8
5 -- 10
6 -- 12

The coroutine command creates a new environment in which the procedure you give it can run. But unlike the ordinary environment, you can suspend the running of the procedure via the yield command. Let us have a closer look at the procedure next_integer:

  • The first command is a plain yield, without an argument. The reason for this is that the first yield within a coroutine environment returns its argument (if any) to the coroutine command. And in general that is not what we want.
  • The procedure then sets the variable i and enters an endless loop. Normally the program would get stuck, but because of the yield command, the loop is suspended, while keeping all the local variables and the location in the procedure, and a value is returned to the caller, our outer loop.
  • If the procedure is called again, processing is resumed at the point where it left off.

That is the charm and power of coroutines. They act as procedures that can be suspended and resumed at will.

A text processing example

To illustrate how they can be used effectively, consider the following problem: we want to search text for duplicate words, like "The dog is is wagging its tail". In this sentence the word "is" is duplicated. We could do that by splitting up the text in words, put them in a long list and then compare the consecutive words in pairs. A drawback is that either we have the full text in memory or we have a fairly complex piece of code to write that keeps track of the position in the text and of the previous word. Let us instead try to do it via coroutines.

We need a procedure that takes a word and compares it to a previous word:

    proc isEqual {} {
        set word [yield]

        set previous {}
        set doubleWords {}

        while { $word ne "" } {
            set equal [expr { $word eq $previous }]

            if { $equal } {
                lappend doubleWords $word
            }

            set previous $word

            set word [yield $equal]
        }

        return $doubleWords
    }

This procedure takes no arguments, but gets it input from the return value of the yield command. So the first yield is now used to get the first word. Then we enter a loop that is exited when there is an empty word passed. Otherwise the new word is compared to the previous one and the result is passed to the caller. When the caller calls the coroutine again - with a new word - that word is caught as the return value of yield.

Note: if a coroutine returns like a normal procedure would, instead of yielding, it is automatically destroyed with all its resources. You cannot call it again - that would result in an error.

Alternatives without coroutines would have to store the previous word somewhere, perhaps as a variable in an object or as a global variable. While certainly a valid solution, when the code gets more complex, for instance if you are dealing with a recursive search through a directory structure and you need to pass information on to another procedure, this could get quite involved.

The above procedure can be used like this:

    coroutine examine isEqual

    foreach w {a b c d d e} {
        examine $w
    }
    puts "Double words: [examine {}]"

resulting in:

Double words: d

Now the other part of our programming exercise: retrieving individual words from a file. To keep the code simple, we use the split command to split up a line of text into words. The end of the file is flagged with an empty string.

    proc readWords {} {
        set file [yield]
        set infile [open $file]

        while { [gets $infile line] >= 0 } {
            #
            # Get rid of non-letters
            #
            regsub -all {[^a-zA-Z]} $line " " line
            set words [split $line]

            #
            # The examine the words
            #
            foreach word $words {
                if { $word ne "" } {
                    examine $word
                }
            }
        }

        close $infile
        return [examine {}]
    }

When used on this text:

As of version 8.6, Tcl supports ''coroutines'' as a programming feature. The main
advantage of coroutines is that they they make certain tasks much simpler to
implement, especially tasks that need to
to keep a lot of information around and need to cooperate with other tasks.
Coroutines are useful in fact for implementing co-operative multitasking - not to be
confused with multiprocessing.

via:

    coroutine examine     isEqual
    coroutine examineFile readWords

    puts "Double words: [examineFile mytext.txt]"

it produces:

Double words: they to

Passing on control

Besides the command yield to suspend the running and return as it were to the caller, you can also use the yieldto command to pass control to another coroutine (or command or procedure). This makes it possible to hand over the control to a whole chain of commands and procedures.

Generators and other programming constructs

One common task for which coroutines are quite suitable is that of a generator. A generator returns a potentially infinite list of values without actually constructing that list explicitly in memory. The first example, that of iterating over integer and even values, is a basic example of a generator. More sophisticated ones: iterating over the contents of a directory and returning the file names one by one:

    ... create a generator called "nextFile" that returns
    files with the extension ".tcl" ...

    foreach file [nextFile] {
        ... examine the contents ...
    }

Since generators are so common a programming construct, there is actually a generator package in the Tcllib library for creating and using generators. Another package in that library is the coroutine package that implements a number of utilities to make coroutines cooperate smoothly with the event loop. The ycl package is a rich collection of procedures that perform all manner of common and not-so-common tasks based on coroutines. These packages help greatly for making an application that uses co-operative multitasking, where various tasks are performed seemingly at the same time without having to use several processors as in multiprocessing.