Chapter 3: How to get various things to work

3.1: Why does $var where var="foo bar" not do what I expect?

In most Bourne-shell derivatives, multiple-word variables such as


    var="foo bar"
  
are split into words when passed to a command or used in a for foo in $var loop. By default, zsh does not have that behaviour: the variable remains intact. (This is not a bug! See below.) The option SH_WORD_SPLIT exists to provide compatibility.

For example, defining the function args to show the number of its arguments:


    args() { echo $#; }
  
and with our definition of `var',

    args $var
  
produces the output `1'. After

    setopt shwordsplit
  
the same function produces the output `2', as with sh and ksh.

Unless you need strict sh/ksh compatibility, you should ask yourself whether you really want this behaviour, as it can produce unexpected effects for variables with entirely innocuous embedded spaces. This can cause horrendous quoting problems when invoking scripts from other shells. The natural way to produce word-splitting behaviour in zsh is via arrays. For example,


    set -A array one two three twenty
  
(or

    array=(one two three twenty)
  
if you prefer), followed by

    args $array
  
produces the output `4', regardless of the setting of SH_WORD_SPLIT. Arrays are also much more versatile than single strings. Probably if this mechanism had always been available there would never have been automatic word splitting in scalars, which is a sort of uncontrollable poor man's array.

Note that this happens regardless of the value of the internal field separator, $IFS; in other words, with IFS=:; foo=a:b; args $foo you get the answer 1.

Other ways of causing word splitting include a judicious use of `eval':


    sentence="Longtemps, je me suis couch\+NOTRANS(é) de bonne heure."
    eval "words=($sentence)"
  
after which $words is an array with the words of $sentence (note characters special to the shell, such as the ' in this example, must already be quoted), or, less standard but more reliable, turning on SH_WORD_SPLIT for one variable only:

    args ${=sentence}
  
always returns 8 with the above definition of args. (In older versions of zsh, ${=foo} toggled SH_WORD_SPLIT; now it forces it on.)

Note also the "$@" method of word splitting is always available in zsh functions and scripts (though strictly this does array splitting, not word splitting). This is more portable than the $*, since it will work regardless of the SH_WORD_SPLIT setting; the other difference is that $* removes empty arguments from the array. You can fix the first half of that objection by using ${==*}, which turns off SH_WORD_SPLIT for the duration of the expansion.

SH_WORD_SPLIT is set when zsh is invoked with the names `ksh' or `sh', or (entirely equivalent) when emulate ksh or emulate sh is in effect.

There is one other effect of word splitting which differs between ksh and zsh. In ksh, the builtin commands that declare parameters such as typeset and export force word-splitting not to take place after on an assignment argument:


    typeset param=`echo foo bar`
  
in ksh will create a parameter with value foo bar, but in zsh will create a parameter param with value foo and a parameter bar whose value is empty. Contrast this with a normal assignment (no typeset or other command in front), which never causes a word split unless you have GLOB_ASSIGN set. From zsh version 4.0.2 the option KSH_TYPESET, set automatically in compatibility mode, fixes this problem. Note that in bash this behaviour occurs with all arguments that look like assignments, whatever the command name; to get this behaviour in zsh you have to set the option MAGIC_EQUAL_SUBST.

3.2: In which startup file do I put...?

When zsh starts up, there are four files you can change which it will run under various circumstances: .zshenv, .zprofile, .zshrc and .zlogin. They are usually in your home directory, but the variable $ZDOTDIR may be set to alter that. Here are a few simple hints about how to use them. There are also files which the system administrator can set for all shells; you can avoid running all except /etc/zshenv by starting zsh with the -f option --- for this reason it is important for administrators to make sure /etc/zshenv is as brief as possible.

The order in which the four files are searched (none of them need to exist) is the one just given. However, .zprofile and .zlogin are only run when the shell is a login shell --- when you first login, of course, and whenever you start zsh with the -l option. All login shells are interactive. The order is the only difference between those; you should decide whether you need things set before or after .zshrc. These files are a good place to set environment variables (i.e. export commands), since they are passed on to all shells without you having to set them again, and also to check that your terminal is set up properly (except that if you want to change settings for terminal emulator windows like xterm you will need to put those in .zshrc, since usually you do not get a login shell here).

The only file you can alter which is started with every zsh (unless you use the -f option) is .zshenv, so this is a good place to put things you want even if the shell is non-interactive: options for changing the syntax, like EXTENDED_GLOB, any changes to set with limit, any more variables you want to make sure are set as for example $fpath to find functions. You almost certainly do not want .zshenv to produce any output. Some people prefer not to use .zshenv for setting options, as this affects scripts; but making zsh scripts portable usually requires special handling anyway.

Finally, .zshrc is run for every interactive shell; that includes login shells, but also any other time you start up a shell, such as simply by typing zsh or opening a new terminal emulator window. This file is the place to change the editing behaviour via options or bindkey, control how your history is saved, set aliases unless you want to use them in scripts too, and for any other clutter which can't be exported but you only use when interacting directly with the shell. You probably don't want .zshrc to produce output, either, since there are occasions when this can be a problem, such as when using rsh from another host. See 3.21 for what to put in .zshrc to save your history.

3.3: What is the difference between `export' and the ALL_EXPORT option?

Normally, you would put a variable into the environment by using export var. The command setopt allexport causes all variables which are subsequently set (N.B. not all the ones which already exist) to be put into the environment.

This may seem a useful shorthand, but in practice it can have unhelpful side effects:

  1. Since every variable is in the environment as well as remembered by the shell, the memory for it needs to be allocated twice. This is bigger as well as slower.
  2. It really is every variable which is exported, even loop variables in for loops. This is probably a waste.
  3. An arbitrary variable created by the user might have a special meaning to a command. Since all shell variables are visible to commands, there is no protection against this.
For these reasons it is usually best to avoid ALL_EXPORT unless you have a specific use for it. One safe use is to set it before creating a list of variables in an initialisation file, then unset it immediately afterwards. Only those variables will be automatically exported.

3.4: How do I turn off spelling correction/globbing for a single command?

In the first case, you presumably have setopt correctall in an initialisation file, so that zsh checks the spelling of each word in the command line. You probably do not want this behaviour for commands which do not operate on existing files.

The answer is to alias the offending command to itself with nocorrect stuck on the front, e.g.


    alias mkdir='nocorrect mkdir'
  

To turn off globbing, the rationale is identical:


    alias mkdir='noglob mkdir'
  
You can have both nocorrect and noglob, if you like, but the nocorrect must come first, since it is needed by the line editor, while noglob is only handled when the command is examined.

Note also that a shell function won't work: the no... directives must be expanded before the rest of the command line is parsed.

3.5: How do I get the Meta key to work on my xterm?

The Meta key isn't present on a lot of keyboards, but on some the Alt key has the same effect. If a character is typed on the keyboard while the Meta key is held down, the characters is sent as terminal input with its eighth bit set. For example, ASCII A, hex 65, becomes hex E5. This is sometimes used to provide extra editing commands.

As stated in the manual, zsh needs to be told about the Meta key by using bindkey -me or bindkey -mv in your .zshrc or on the command line. You probably also need to tell the terminal driver to allow the `Meta' bit of the character through; stty pass8 is the usual incantation. Sample .zshrc entry:


    [[ $TERM = "xterm" ]] && stty pass8 && bindkey -me
  
or, on SYSVR4-ish systems without pass8,

    [[ $TERM = "xterm" ]] && stty -parenb -istrip cs8 && bindkey -me
  
(disable parity detection, don't strip high bit, use 8-bit characters). Make sure this comes before any bindkey entries in your .zshrc which redefine keys normally defined in the emacs/vi keymap. You may also need to set the eightBitOutput resource in your ~/.Xdefaults file, although this is on by default and it's unlikely anybody will have tinkered with it.

You don't need the bindkey to be able to define your own sequences with the Meta key, though you still need the stty.

If you are using multibyte input directly from the keyboard you probably don't want to use this feature since the eighth bit in each byte is used to indicate a part of a multibyte character. See chapter 5.

3.6: How do I automatically display the directory in my xterm title bar?

You should use the special function chpwd, which is called when the directory changes. The following checks that standard output is a terminal, then puts the directory in the title bar if the terminal is an xterm or some close relative, or a sun-cmd.


  chpwd() {
    [[ -t 1 ]] || return
    case $TERM in
      sun-cmd) print -Pn "\e]l%~\e\\"
        ;;
      *xterm*|rxvt|(dt|k|E)term) print -Pn "\e]2;%~\a"
        ;;
    esac
  }
  

Change %~ if you want the message to be different. (The -P option interprets such sequences just like in prompts, in this case producing the current directory; you can of course use $PWD here, but that won't use the ~ notation which I find clearer.) Note that when the xterm starts up you will probably want to call chpwd directly: just put chpwd in .zshrc after it is defined or autoloaded.

3.7: How do I make the completion list use eight bit characters?

If you are sure your terminal handles this, the easiest way from versions 3.0.6 and 3.1 of the shell is to set the option PRINT_EIGHT_BIT. In principle, this will work automatically if your computer uses the `locale' system and your locale variables are set properly, as zsh understands this. However, it is quite complicated, so if it isn't already set up, trying the option is a lot easier. For earlier versions of zsh 3, you are stuck with trying to understand locales, see the setlocale(3) and zshparam(1) manual pages: the simplest possibility may be to set LC_ALL=en_US. For older versions of the shell, there is no easy way out.

3.8: Why do the cursor (arrow) keys not work? (And other terminal oddities.)

The cursor keys send different codes depending on the terminal; zsh only binds the most well known versions. If you see these problems, try putting the following in your .zshrc:


    bindkey "$(echotc kl)" backward-char
    bindkey "$(echotc kr)" forward-char
    bindkey "$(echotc ku)" up-line-or-history
    bindkey "$(echotc kd)" down-line-or-history
  

If you use vi mode, use vi-backward-char and vi-forward-char where appropriate. As of version 4.0.1, zsh attempts to look up these codes and to set the key bindings for you (both emacs and vi), but in some circumstances this may not work.

Note, however, that up to version 3.0 binding arbitrary multiple key sequences can cause problems, so check that this works with your set up first. Also, from version 3.1.3, more sequences are supported by default, namely those in the form <ESC>O followed by A, B, C or D, as well as the corresponding set beginning <ESC>[, so this may be redundant.

A particular problem which sometimes occurs is that there are two different modes for arrow keys, normal mode and keypad mode, which send different sequences. Although this is largely a historical artifact, it sometimes happens that your terminal can be switched from one mode to the other, for example by a rogue programme that sends the sequence to switch one way, but not the sequence to switch back. Thus you are stuck with the effects. Luckily in this case the arrow key sequences are likely to be standard, and you can simply bind both sets. The following code does this.


    bindkey '\e[A'  up-line-or-history
    bindkey '\e[B'  down-line-or-history
    bindkey '\e[C'  forward-char
    bindkey '\e[D'  backward-char
    bindkey '\eOA'  up-line-or-history
    bindkey '\eOB'  down-line-or-history
    bindkey '\eOC'  forward-char
    bindkey '\eOD'  backward-char
  
For most even vaguely VT100-compatible terminals, the above eight instructions are a fairly safe bet for your .zshrc. Of course you can substitute variant functions for the second argument here too.

It should be noted that the O / [ confusion can occur with other keys such as Home and End. Some systems let you query the key sequences sent by these keys from the system's terminal database, terminfo. Unfortunately, the key sequences given there typically apply to the mode that is not the one zsh uses by default (it's the "application" mode rather than the "raw" mode). Explaining the use of terminfo is outside the scope of this FAQ, but if you wish to use the key sequences given there you can tell the line editor to turn on "application" mode when it starts and turn it off when it stops:


    function zle-line-init () { echoti smkx }
    function zle-line-finish () { echoti rmkx }
    zle -N zle-line-init
    zle -N zle-line-finish
  
If you only have the predecessor to terminfo, called termcap (which is what we used to get the cursor keys above), replace echoti smkx with echotc ks and replace echoti rmkx with echotc ke.

3.9: Why does my terminal act funny in some way?

If you are using an OpenWindows cmdtool as your terminal, any escape sequences (such as those produced by cursor keys) will be swallowed up and never reach zsh. Either use shelltool or avoid commands with escape sequences. You can also disable scrolling from the cmdtool pane menu (which effectively turns it into a shelltool). If you still want scrolling, try using an xterm with the scrollbar activated.

If that's not the problem, and you are using stty to change some tty settings, make sure you haven't asked zsh to freeze the tty settings: type


    ttyctl -u
  
before any stty commands you use.

On the other hand, if you aren't using stty and have problems you may need the opposite: ttyctl -f freezes the terminal to protect it from hiccups introduced by other programmes (kermit has been known to do this).

A problem I have experienced myself (on an AIX 3.2 workstation with xterm) is that termcap deinitialization sequences sent by `less' were causing automargins to be turned off --- not actually a shell problem, but you might have thought it was. The fix is to put `X' into the environment variable LESS to stop the sequences being sent. Other programs (though not zsh) may also send that sequence.

If that's not the problem, and you are having difficulties with external commands (not part of zsh), and you think some terminal setting is wrong (e.g. ^V is getting interpreted as `literal next character' when you don't want it to be), try


    ttyctl -u
    STTY='lnext "^-"' commandname
  
(in this example). Note that zsh doesn't reset the terminal completely afterwards: just the modes it uses itself and a number of special processing characters (see the stty(1) manual page).

3.10: Why does zsh not work in an Emacs shell mode any more?

(This information comes from Bart Schaefer and other zsh-workers.)

Emacs 19.29 or thereabouts stopped using a terminal type of "emacs" in shell buffers, and instead sets it to "dumb". Zsh only kicks in its special I'm-inside-emacs initialization when the terminal type is "emacs".

Probably the most reliable way of dealing with this is to look for the environment variable $EMACS, which is set to t in Emacs' shell mode. Putting


    [[ $EMACS = t ]] && unsetopt zle
  
in your .zshrc should be sufficient.

Another method is to put


    #!/bin/sh
    TERM=emacs exec zsh
  
into a file ~/bin/eshell, then chmod +x ~/bin/eshell, and tell emacs to use that as the shell by adding

    (setenv "ESHELL" (expand-file-name "~/bin/eshell"))
  
to ~/.emacs.

3.11: Why do my autoloaded functions not autoload [the first time]?

The problem is that there are two possible ways of autoloading a function (see the AUTOLOADING FUNCTIONS section of the zsh manual page zshmisc for more detailed information):

  1. The file contains just the body of the function, i.e. there should be no line at the beginning saying function foo { or foo () {, and consequently no matching } at the end. This is the traditional zsh method. The advantage is that the file is called exactly like a script, so can double as both. To define a function xhead () { print -n "\033]2;$*\a"; }, the file would just contain print -n "\033]2;$*\a".
  2. The file contains the entire definition, and maybe even other code: it is run when the function needs to be loaded, then the function itself is called up. This is the method in ksh. To define the same function xhead, the whole of the usual definition should be in the file.

In old versions of zsh, before 3.0, only the first behaviour was allowed, so you had to make sure the file found for autoload just contained the function body. You could still define other functions in the file with the standard form for definitions, though they would be redefined each time you called the main function.

In version 3.0.x, the second behaviour is activated if the file defines the autoloaded function. Unfortunately, this is incompatible with the old zsh behaviour which allowed you to redefine the function when you called it.

From version 3.1, there is an option KSH_AUTOLOAD to allow full ksh compatiblity, i.e. the function must be in the second form above. If that is not set, zsh tries to guess which form you are using: if the file contains only a complete definition of the function in the second form, and nothing else apart from comments and whitespace, it will use the function defined in the file; otherwise, it will assume the old behaviour. The option is set if emulate ksh is in effect, of course.

(A neat trick to autoload all functions in a given directory is to include a line like autoload ~/fns/*(:t) in .zshrc; the bit in parentheses removes the directory part of the filenames, leaving just the function names.)

3.12: How does base arithmetic work?

The ksh syntax is now understood, i.e.


    let 'foo = 16#ff'
  
or equivalently

    (( foo = 16#ff ))
  
or even

    foo=$((16#ff))
  
The original syntax was

    (( foo = [16]ff ))
  
--- this was based on a misunderstanding of the ksh manual page. It still works but its use is deprecated. Then

    echo $foo
  
gives the answer `255'. It is possible to declare variables explicitly to be integers, via

    typeset -i foo
  
which has a different effect: namely the base used in the first assignment (hexadecimal in the example) is subsequently used whenever `foo' is displayed (although the internal representation is unchanged). To ensure foo is always displayed in decimal, declare it as

    typeset -i 10 foo
  
which requests base 10 for output. You can change the output base of an existing variable in this fashion. Using the $(( ... )) method will always display in decimal, except that in 3.1.9 there is a new feature for selecting a base for displaying here:

    print $(( [#16] 255 ))
  

3.13: How do I get a newline in my prompt?

You can place a literal newline in quotes, i.e.


    PROMPT="Hi Joe,
    what now?%# "
  
If you have the bad taste to set the option cshjunkiequotes, which inhibits such behaviour, you will have to bracket this with unsetopt cshjunkiequotes and setopt cshjunkiequotes, or put it in your .zshrc before the option is set.

In recent versions of zsh (not 3.0), there is a form of quoting which interprets print sequences like `\n' but otherwise acts like single quotes: surround the string with $'...'. Hence:


    PROMPT=$'Hi Joe,\nwhat now?%# '
  
is a neat way of doing what you want. Note that it is the quotes, not the prompt expansion, which turns the `\n' into a newline.

3.14: Why does bindkey ^a command-name or stty intr ^- do something funny?

You probably have the extendedglob option set in which case ^ and # are metacharacters. ^a matches any file except one called a, so the line is interpreted as bindkey followed by a list of files. Quote the ^ with a backslash or put quotation marks around ^a. See 3.27 if you want to know more about the pattern character ^.

3.15: Why can't I bind \C-s and \C-q any more?

The control-s and control-q keys now do flow control by default, unless you have turned this off with stty -ixon or redefined the keys which control it with stty start or stty stop. (This is done by the system, not zsh; the shell simply respects these settings.) In other words, \C-s stops all output to the terminal, while \C-q resumes it.

There is an option NO_FLOW_CONTROL to stop zsh from allowing flow control and hence restoring the use of the keys: put setopt noflowcontrol in your .zshrc file.

3.16: How do I execute command foo within function foo?

The command command foo does just that. You don't need this with aliases, but you do with functions. Note that error messages like


    zsh: job table full or recursion limit exceeded
  
are a good sign that you tried calling `foo' in function `foo' without using `command'. If foo is a builtin rather than an external command, use builtin foo instead.

3.17: Why do history substitutions with single bangs do something funny?

If you have a command like "echo !-2:$ !$", the first history substitution then sets a default to which later history substitutions with single unqualified bangs refer, so that !$ becomes equivalent to !-2:$. The option CSH_JUNKIE_HISTORY makes all single bangs refer to the last command.

3.18: Why does zsh kill off all my background jobs when I logout?

Simple answer: you haven't asked it not to. Zsh (unlike [t]csh) gives you the option of having background jobs killed or not: the nohup option exists if you don't want them killed. Note that you can always run programs with nohup in front of the pipeline whether or not the option is set, which will prevent that job from being killed on logout. (nohup is actually an external command.)

The disown builtin is very useful in this respect: if zsh informs you that you have background jobs when you try to logout, you can disown all the ones you don't want killed when you exit. This is also a good way of making jobs you don't need the shell to know about (such as commands which create new windows) invisible to the shell. Likewise, you can start a background job with &! instead of just & at the end, which will automatically disown the job.

3.19: How do I list all my history entries?

Tell zsh to start from entry 1: history 1. Those entries at the start which are no longer in memory will be silently omitted.

3.20: How does the alternative loop syntax, e.g. while {...} {...} work?

Zsh provides an alternative to the traditional sh-like forms with do,


    while TEST; do COMMANDS; done
  
allowing you to have the COMMANDS delimited with some other command structure, often {...}. The rules are quite complicated and in most scripts it is probably safer --- and certainly more compatible --- to stick with the sh-like rules. If you are wondering, the following is a rough guide.

To make it work you must make sure the TEST itself is clearly delimited. For example, this works:


    while (( i++ < 10 )) { echo i is $i; }
  
but this does not:

    while let "i++ < 10"; { echo i is $i; }   # Wrong!
  
The reason is that after while, any sort of command list is valid. This includes the whole list let "i++ < 10"; { echo i $i; }; the parser simply doesn't know when to stop. Furthermore, it is wrong to miss out the semicolon, as this makes the {...} part of the argument to let. A newline behaves the same as a semicolon, so you can't put the brace on the next line as in C.

So when using this syntax, the test following the while must be wrapped up: any of ((...)), [[...]], {...} or (...) will have this effect. (They have their usual syntactic meanings too, of course; they are not interchangeable.) Note that here too it is wrong to put in the semicolon, as then the case becomes identical to the preceding one:


    while (( i++ < 10 )); { echo i is $i; }   # Wrong!
  

The same is true of the if and until constructs:


    if { true } { echo yes } else { echo no }
  
but with for, which only needs a list of words, you can get away with it:

    for foo in a b; { echo foo is $a; bar=$foo; }
  
since the parser knows it only needs everything up to the first semicolon. For the same reason, there is no problem with the repeat, case or select constructs; in fact, repeat doesn't even need the semicolon since it knows the repeat count is just one word.

This is independent of the behaviour of the SHORTLOOPS option (see manual), which you are in any case encouraged even more strongly not to use in programs as it can be very confusing.

3.21: Why is my history not being saved?

In zsh, you need to set three variables to make sure your history is written out when the shell exits. For example,


    HISTSIZE=200
    HISTFILE=~/.zsh_history
    SAVEHIST=200
  
$HISTSIZE tells the shell how many lines to keep internally, $HISTFILE tells it where to write the history, and $SAVEHIST, the easiest one to forget, tells it how many to write out. The simplest possibility is to set it to the same as $HISTSIZE as above. There are also various options affecting history; see the manual.

3.22: How do I get a variable's value to be evaluated as another variable?

The problem is that you have a variable $E containing the string EDITOR, and a variable $EDITOR containing the string emacs, or something such. How do you get from $E to emacs in one easy stage?

There is no standard single-stage way of doing this. However, there is a zsh idiom (available in all versions of zsh since 3.0) for this:


    print ${(e)E:+\$$E}
  
Ignore the (e) for now. The :+ means: if the variable $E is set, substitute the following, i.e. \$$E. This is expanded to $EDITOR by the normal rules. Finally, the (e) means `evaluate the expression you just made'. This gives emacs.

For a standard shell way of doing this, you are stuck with eval:


    eval echo \$$E
  
produces the same result.

Versions since 3.1.6 allow you to do this directly with a new flag; ${(P)E}.

As a slight aside, sometimes people note that the syntax ${${E}} is valid and expect it to have this effect. It probably ought to, but in the early days of zsh it was found convenient to have this way of producing different substitutions on the same parameter; for example, ${${file##**/}%.*} removes everything up to the last slash in $file, then everything from the last dot on, inclusive (try it, this works). So in ${${E}}, the internal ${...} actually does nothing.

3.23: How do I prevent the prompt overwriting output when there is no newline?

The problem is normally limited to zsh versions prior to 4.3.0 due to the advent of the PROMPT_SP option (which is enabled by default, and eliminates this problem for most terminals). An example of the overwriting is:


    % echo -n foo
    % 
  
This shows a case where the word foo was output without a newline, and then overwritten by the prompt line %. The reason this happens is that the option PROMPT_CR is enabled by default, and it outputs a carriage return before the prompt in order to ensure that the line editor knows what column it is in (this is needed to position the right-side prompt correctly ($RPROMPT, $RPS1) and to avoid screen corruption when performing line editing). If you add unsetopt promptcr to your .zshrc, you will see any partial output, but your screen may look weird until you press return or refresh the screen.

A better solution than disabling PROMPT_CR (for most terminals) is adding a simpler version of the PROMPT_SP functionality to an older zsh using a custom precmd function, like this one:


    # Skip defining precmd if the PROMPT_SP option is available.
    if ! eval '[[ -o promptsp ]] 2>/dev/null'; then
      function precmd {
        # Output an inverse char and a bunch spaces.  We include
        # a CR at the end so that any user-input that gets echoed
        # between this output and the prompt doesn't cause a wrap.
        print -nP "%B%S%#%s%b${(l:$((COLUMNS-1)):::):-}\r"
      }
    fi
  
That precmd function will only bump the screen down to a new line if there was output on the prompt line, otherwise the extra chars get removed by the PROMPT_CR action. Although this typically looks fine, it may result in the spaces preceding the prompt being included when you select a line of preserved text with the mouse.

One final alternative is to put a newline in your prompt -- see question 3.13 for that.

3.24: What's wrong with cut and paste on my xterm?

On the majority of modern UNIX systems, cutting text from one window and pasting it into another should work fine. On a few, however, there are problems due to issues about how the terminal is handled: most programs expect the terminal to be in `canonical input mode', which means that the program is passed a whole line of input at a time, while for editing the shell needs a single character at a time and must be in `non-canonical input mode'. On the systems in question, input can be lost or re-ordered when the mode changes. There are actually two slightly different problems:

  1. When you paste something in while a programme is running, so that the shell only retrieves it later. Traditionally, there was a test which was used only on systems where the problem was known to exist, so it is possible some other systems were not handled (for example, certain versions of IRIX, it appears); also, continuation lines were not handled properly. A more reliable approach appears from versions 3.0.6 and 3.1.6.
  2. When the shell is waiting for input, and you paste in a chunk of text consisting of more than one complete set of commands. Unfortunately, this is a much harder problem: the line editor is already active, and needs to be turned off when the first command is executed. The shell doesn't even know if the remaining text is input to a command or for the shell, so there's simply nothing it can do. However, if you have problems you can trick it: type `{' on a line by itself, then paste the input, then type `}' on a line by itself. The shell will not execute anything until the final brace is read; all input is read as continuation lines (this may require the fixes referred to above in order to be reliable).

3.25: How do I get coloured prompts on my colour xterm?

(Or `color xterm', if you're reading this in black and white.)

Versions of the shell starting with the 4.3 series have this built in. Use


    PS1='%K{white}%F{red}<red on white>%f%k<default colours>'
  
to change the prompt. Names are only usable for the colours black, red, green, yellow, blue, magenta, cyan and white, understood by most terminals, but if you happen to know the details of how your terminal implements colours you can specify a number, e.g. %20F to turn the foreground into colour number 20. echotc Co will often output the number of colours the terminal supports. (Careful: echotc co is different; it also outputs a number but it's the number of columns in the terminal.) If this is 8 then probably you have the named colours and nothing more.

In older versions of the shell you need to find the sequences which generate the various colours from the manual for your terminal emulator; these are ANSI standard on those I know about which support colour. With a recent (post 3.1.6) distribution of zsh, there is a theme system to handle this for you; even if you don't see that, the installed function `colors' (meaning `colours', if you're not reading this in black and white) gives the escape sequences. You will end up with code looking like this (borrowed from Oliver Kiddle):


    PS1=$'%{\e[1;31m%}<the rest of your prompt here>%{\e[0m%}'
  
The $' form of quoting turns the `\e' into a real escape character; this only works from about version 3.1.4, so if you're using 3.0.x, you need to do something like

    PS1="$(print '%{\e[1;31m%}<the rest goes here>%{\e[0m%}')"
  
The `%{...%}' is used in prompts for strings which will not appear as characters, so that the prompt code doesn't miscalculate the length of the prompt which would have a bad effect on editing. The resulting `<ESC>[1;31m' makes the prompt red, and the `<ESC>[0m' puts printing back to normal so that the rest of the line is unchanged.

3.26: Why is my output duplicated with `foo 2>&1 >foo.out | bar'?

This is a slightly unexpected effect of the option MULTIOS, which is set by default. Let's look more closely:


    foo 2>&1 >foo.out | bar
  
What you're probably expecting is that the command foo sends its standard output to the pipe and so to the input of the command bar, while it sends its standard error to the file foo.out. What you actually see is that the output is going both to the pipe and into the file. To be more explicit, here's the same example with real commands:

    % { print output; print error >&2 } 2>&1 >foo.out | sed 's/error/erratic'
    erratic
    output
    % cat foo.out
    output
  
and you can see `output' appears twice.

It becomes clearer what's going on if we write:


    % print output >foo1.out >foo2.out
    % cat foo1.out
    output
    % cat foo2.out
    output
  
You might recognise this as a standard feature of zsh, called `multios' and controlled by the option of the same name, whereby output is copied to both files when the redirector appears twice. What's going on in the first example is exactly the same, however the second redirector is disguised as a pipe. So if you want to turn this effect off, you need to unset the option MULTIOS.

3.27: What are these `^' and `~' pattern characters, anyway?

The characters ^ and ~ are active when the option EXTENDED_GLOB is set. Both are used to exclude patterns, i.e. to say `match something other than ...'. There are some confusing differences, however. Here are the descriptions for ^ and ~.

^ means `anything except the pattern that follows'. You can think of the combination ^pat as being like a * except that it doesn't match pat. So, for example, myfile^.txt matches anything that begins with myfile except myfile.txt. Because it works with patterns, not just strings, myfile^*.c matches anything that begins with myfile unless it ends with .c, whatever comes in the middle --- so it matches myfile1.h but not myfile1.c.

Also like *, ^ doesn't match across directories if you're matching files when `globbing', i.e. when you use an unquoted pattern in an ordinary command line to generate file names. So ^dir1/^file1 matches any subdirectory of the current directory except one called dir1, and within any directory it matches it picks any file except one called file1. So the overall pattern matches dir2/file2 but not dir1/file1 nor dir1/file2 nor dir2/file1. (The rule that all the different bits of the pattern must match is exactly the same as for any other pattern character, it's just a little confusing that what does match in each bit is found by telling the shell not to match something or other.)

As with any other pattern, a ^ expression doesn't treat the character `/' specially if it's not matching files, for example when pattern matching in a command like [[ $string = ^pat1/pat2 ]]. Here the whole string pat1/pat2 is treated as the argument that follows the ^. So anything matches but that one string pat1/pat1.

It's not obvious what something like [[ $string = ^pat1^pat2 ]] means. You won't often have cause to use it, but the rule is that each ^ takes everything that follows as an argument (unless it's already inside parentheses --- I'll explain this below). To see this more clearly, put those arguments in parentheses: the pattern is equivalent to ^(pat1^(pat2)). where now you can see exactly what each ^ takes as its argument. I'll leave it as an exercise for you to work out what this does and doesn't match.

~ is always used between two patterns --- never right at the beginning or right at the end. Note that the other special meaning of ~, at the start of a filename to refer to your home directory or to another named directory, doesn't require the option EXTENDED_GLOB to be set. (At the end of an argument ~ is never special at all. This is useful if you have Emacs backup files.) It means `match what's in front of the tilde, but only if it doesn't match what's after the tilde'. So *.c~f* matches any file ending in .c except one that begins with f. You'll see that, unlike ^, the parts before and after the ~ both refer separately to the entire test string.

For matching files by globbing, ~ is the only globbing operator to have a lower precedence than /. In other words, when you have /a/path/to/match~/a/path/not/to/match the ~ considers what's before as a complete path to a file name, and what's after as a pattern to match against that file. You can put any other pattern characters in the expressions before and after the ~, but as I said the pattern after the ~ is really just a single pattern to match against the name of every file found rather than a pattern to generate a file. That means, for example, that a * after the ~ will match a /. If that's confusing, you can think of how ~ works like this: take the pattern on the left, use it as normal to make a list of files, then for each file found see if it matches the pattern on the right and if it does take that file out of the list. Note, however, that this removal of files happens immediately, before anything else happens to the file list --- before any glob qualifiers are applied, for example.

One rule that is common to both ^ and ~ is that they can be put inside parentheses and the arguments to them don't extend past the parentheses. So (^README).txt matches any file ending in .txt unless the string before that was README, the same as *.txt~README.txt or (*~README).txt. In fact, you can always turn ^something into (*~something), where something mustn't contain / if the pattern is being used for globbing.

Likewise, abc(<->~<10-100>).txt matches a file consisting of abc, then some digits, then .txt, unless the digits happen to match a number from 10 to 100 inclusive (remember the handy <-> pattern for matching integers with optional limits to the range). So this pattern matches abc1.txt or abc200.txt but not abc20.txt nor abc100.txt nor even abc0030.txt. However, if you're matching files by globbing note you can't put /s inside the parentheses since the groups can't stretch across multiple directories. (You can do that, of course, whenever the character / isn't special.) This means that you need to take care when using exclusions across multiple directories; see some examples below.

You may like to know that from zsh 5.0.3 you can disable any pattern character separately. So if you find ^ gets in your way and you're happy using ~, put disable -p "^" in ~/.zshrc. You still need to turn on EXTENDED_GLOB; the disable command only deactivates things that would otherwise be active, you can't specially enable something not allowed by the syntax options in effect.

Here are some examples with files to illustrate the points. We'll assume the option EXTENDED_GLOB is set and none of the pattern characters is disabled.

  1. **/foo~*bar* matches any file called foo in any subdirectory, except where bar occurred somewhere in the path. For example, users/barstaff/foo will be excluded by the ~ operator. As the ** operator cannot be grouped (inside parentheses it is treated as *), this is one way to exclude some subdirectories from matching a **. Note that this can be quite inefficent because the shell performs a complete search for **/foo before it uses the pattern after the ~ to exclude files from the match. The file is excluded if bar occurs anywhere, in any directory segment or the final file name.
  2. The form (^foo/)# can be used to match any hierarchy of directories where none of the path components is foo. For example, (^CVS/)# selects all subdirectories to any depth except where one component is named CVS. (The form (pat/)# is very useful in other cases; for example, (../)#.cvsignore finds the file .cvsignore if it exists in the current directory or any parent.)