**Running other programs from Tcl - exec, open** !!!!!! '''[Tcl Tutorial Lesson 25%|%Previous lesson%|%]''' | '''[Tcl Tutorial Index%|%Index%|%]''' | '''[Tcl Tutorial Lesson 40%|%Next lesson%|%]''' !!!!!! So far the lessons have dealt with programming within the Tcl interpreter. However, Tcl is also useful as a scripting language to tie other packages or programs together. To accomplish this function, Tcl has two ways to start another program: * `open` - run a new program and open a file-like connection to this program. * `exec` - run a new program as a more or less independent subprocess The `open` call is the same call that is used to open a file. If the first character in the file name argument is a "pipe" symbol (`|`), then `open` will treat the rest of the argument as a program name, and will run that program with the ''standard input'' or ''output'' connected to a file descriptor. This "pipe" connection can be used to read the output from that other program or to write fresh input data to it or both. If the "pipe" is opened for both reading and writing, you must be aware that the pipes are buffered. The output from a `puts` command will be saved in an I/O buffer until the buffer is full, or until you execute a `flush` command to force it to be transmitted to the other program. The output of this other program will not be available to `read` or `gets` until its output buffer is filled up or flushed explicitly. (''Note:'' As this is ''internal'' to this other program, there is no way that your Tcl script can influence that. The other program simply must cooperate. Well, that is not entirely true: the ''Expect'' extension actually works around this limitation by exploiting deep system features.) The `exec` call is similar to invoking a program (or a set of programs piped together) from the prompt in an interactive shell, a DOS batch file or a UNIX/Linux shell script. It supports several styles of output redirection, or it can return the output of the other program(s) as the return value of the `exec` call. `open |program ?access?`: Returns a file descriptor for the pipe. The `program` argument must start with the pipe symbol. If `program` is enclosed in quotes or braces, it can include arguments to the subprocess. `exec ?switches? arg1 ?arg2? ... ?argN?`: `exec` treats its arguments as the names and arguments for a set of programs to run. If the first `args` start with a `"-"`, then they are treated as `switches` to the `exec` command, instead of being invoked as subprocesses or subprocess options. `switches` are: * `-keepnewline` retains a trailing newline in the pipeline's output. Normally a trailing newline will be deleted. * `--` marks the end of the switches. The next string will be treated as `arg1`, even if it starts with a "`-`" The arguments to the `exec` command, `arg1` ... `argN` can be one of: * the name of a program to execute * a command line argument for the subprocess * an I/O redirection instruction. * an instruction to put the new program in the background: ====== exec myprog & ====== will start the program `myprog` in the background, and return immediately. There is no connection between that program and the Tcl script, both can run on independently. The `&` must be the last argument - you can use all other types of arguments in front of it. [''NOTE:'' Add information on how to wait for the program to finish?] There are many I/O redirection commands. The main subset of these commands is: `|`: Pipes the ''standard output'' of the command preceding the pipe symbol into the ''standard input'' of the command following the pipe symbol. `< filename`: The first program in the pipe will read input from `filename`. `<@ fileID`: The first program in the pipe will read input from the Tcl descriptor `fileID`. `fileID` is the value returned from an `open` `...` `"r"` command. `<< value`: The first program in the pipe will read `value` as its input. `> filename`: The output of the last program in the pipe will be sent to `filename`. Any previous contents of `filename` will be lost. `>> filename`: The output of the last program in the pipe will be appended to `filename`. `2> filename`: The ''standard error'' from all the programs in the pipe will be sent to `filename`. Any previous contents of `filename` will be lost. `2>> filename`: The ''standard error'' from all the programs in the pipe will be appended to `filename`. `>@ fileID`: The output from the last program in the pipe will be written to `fileID`. `fileID` is the value returned from an `open` `...` `"w"` command. If you are familiar with shell programming, there are a few differences to be aware of when you are writing Tcl scripts that use the `exec` and `open` calls. * You don't need the quotes that you would put around arguments to escape them from the shell expanding them. In the example, the argument to the `sed` command is not put in quotes. If it were put in quotes, the quotes would be passed to `sed`, instead of being stripped off (as theshell does), and `sed` would report an error. * If you use the `open` `|cmd` `"r+"` construct, you must follow each `puts` with a flush to force Tcl to send the command from its buffer to the program. The output from the program itself may be buffered in its output buffer. You can sometimes force the output from the external program to flush by sending an `exit` command to the process. You can also use the `fconfigure` command to make a connection (channel) unbuffered. * This will fail - there is most probably no file with the literal name "*.tcl": ====== exec ls *.tcl ====== * To pass a list of files, based on such a pattern use the `{*}` prefix, it forces the list to become individual arguments: ====== exec ls {*}[glob *.tcl] ====== * If one of the commands in an `exec` call fails to execute, the `exec` will return an error, and the error output will include the last line describing the error. The `exec` treats any output to ''standard error'' to be an indication that the external program failed. This is simply a conservative assumption: many programs behave that way and they are sloppy in setting return codes. Some programs however write to ''standard error'' without intending this as an indication of an error. You can guard against this from upsetting your script by using the `catch`command: ====== if { [catch { exec ls *.tcl } msg] } { puts "Something seems to have gone wrong but we will ignore it" } ====== As already mentioned, the ''Expect'' extension to Tcl provides a very powerful interface to other programs, which in particular handles the buffering problem. ''NOTE:'' add good reference to expect. If one of the commands in an `open |cmd` fails, the `open` does not return an error. However, attempting to read input from the file descriptor with `gets $file` will return an empty string. Using the `gets $file input` construct will return a character count of -1. To inspect the return code from a program and the possible reason for failure, you can use the global `errorInfo` variable: ====== if { [catch { exec ls *.tcl } msg] } { puts "Something seems to have gone wrong:" puts "Information about it: $::errorInfo" } ====== ---- ***Example*** ====== # # Write a Tcl script to get a platform-independent program: # # Create a unique (mostly) file name for a Tcl program set TMPDIR "." if { [info exists ::env(TMP)] } { set TMPDIR $::env(TMP) } set tempFileName [file join $TMPDIR invert_[pid].tcl] # Open the output file, and # write the program to it set outfl [open $tempFileName w] puts $outfl { set len [gets stdin line] if {$len < 5} {exit -1} for {set i [expr {$len-1}]} {$i >= 0} {incr i -1} { append invertedLine [string range $line $i $i] } puts $invertedLine exit 0 } # Close the file close $outfl # # Run the new Tcl script: # # Open a pipe to the program (for both reading and writing: r+) # set io [open "|[info nameofexecutable] $tempFileName" r+] # # Send a string to the new program # *MUST FLUSH* puts $io "This will come back backwards." flush $io # Get the reply, and display it. set len [gets $io line] puts "To invert: 'This will come back backwards.'" puts "Inverted is: $line" puts "The line is $len characters long" # Run the program with input defined in an exec call set invert [exec [info nameofexecutable] $tempFileName << \ "ABLE WAS I ERE I SAW ELBA"] # Display the results puts "The inversion of 'ABLE WAS I ERE I SAW ELBA' is \n $invert" # Clean up file delete $tempFileName ====== <> Resulting output ======none To invert: 'This will come back backwards.' Inverted is: .sdrawkcab kcab emoc lliw sihT The line is 30 characters long The inversion of 'ABLE WAS I ERE I SAW ELBA' is ABLE WAS I ERE I SAW ELBA ====== <> !!!!!! '''[Tcl Tutorial Lesson 25%|%Previous lesson%|%]''' | '''[Tcl Tutorial Index%|%Index%|%]''' | '''[Tcl Tutorial Lesson 40%|%Next lesson%|%]''' !!!!!!