Regular Expressions Reference

If you are an advanced user who understands what "regular" expressions
are, you can use this highly specific UNIX based expression to create
complex filters. (If you are a novice, you should first see the topic
Agent's Expression Language for background about expressions.)

Note: Use Regular Expressions Only When Necessary.
Because of the extra computation required to evaluate regular
expressions, we recommend that you use them only when necessary,
especially in Usenet and email filters, where the time required to
evaluate filters can significantly affect online performance. If all
you need to do is match words, then it is much faster to use Agent's
simpler word-matching syntax. For example, the following word-based
filter:
subject: this is a test
is much faster than this regular expression filter, which is roughly
equivalent
subject: { this is a test }
In general, you should find that you need to use regular expressions
only if you need to:

· match punctuation characters that are not normally treated as
"words".

· match text that appears inside of words.

How to Include Regular Expressions in Filter Expressions
Anywhere a sequence of words to be matched can appear in a filter
expression, you can include a regular expression enclosed in {..}
(curly braces). Here's an example of a regular expression, combined
with a sequence of words to be matched, as well:
subject: (this is a test and {[0-9]+})

This example matches any subject containing the phrase "this is a
test" AND a regular expression consisting of any sequence of one or
more digits.

Rules for Writing Regular Expressions
The simplest regular expression is called an "atom". An atom can be
any of the following:

· Any character that doesn't otherwise have special meaning.
For example, the letter "a" is an atom that matches "a" or "A" in the
field being tested.
· The . (dot) character. A dot matches any character.
· The ^ (carat) character, which matches the beginning of the
field being tested.
· The $ (dollar sign) character, which matches the end of the
field.
· The \ (backslash) character, followed by any character. You
can use backslash to "quote" characters that otherwise have special
meaning. For example, to match the literal character ".", you would
write "\.". Without the backslash, the "." would match any character,
not just ".".

· Any more complex regular expression enclosed in parentheses
(...).
· A "range" of characters, enclosed in [...] square brackets. A
range is defined according to the following rules:

a. A range may be a simple list of characters. For example, the
range [qwerty] matches a single occurrence of any of the characters q,
w, e, r, t, or y.
b. A range may be a character range, specified as x-z. For
example, the range [0-9] matches a single occurrence of any character
in the range 0 through 9.
c. More complex ranges may be built with the previous two rules.
For example, the range [a-z0-9] matches any letter or digit.
(Remember that Agent's regular expressions are normally
case-insensitive. That's why you don't need to include A-Z in the
range.)
d. If a range begins with a ^ character, it matches any character
not in the specified range. For example, the range [^a-z0-9] matches
any character that is not a letter or a digit.

e. To include the characters \, ^, [, ] , or - inside a range as
literals, prefix them with a \ backslash. For example, the range
[^\^\]] matches any character that is not ^ or ].

Controlling the Extent of an Atom's Match
You can modify an atom by following it with special operators that
control how much of the field the atom should match. The special
operators are:

* This matches a sequence of zero or more matches of the atom.
For example, the expression {a*} matches zero or more occurrences of
the letter a.
+ This matches a sequence of one or more matches of the atom.
For example, the expression {[0-9]+} matches one or more digits.
? This matches zero or one occurrence of the atom.

Note: These operators apply the the entire preceeding sequence of
characters. (This behavior is different from other regular expression
parsers with which you may be familiar.) For example,
{abc*} matches zero or more occurrences of the sequence "abc". Thus,
it matches nothing, "abc", "abcabc", "abcabcabc", etc.
To apply an operator to a subset of the preceeding characters, enclose
the desired subset in parentheses. For example:

{ab(c)*} matches "ab" followed by zero or more occurrences of the
letter "c". Thus, it matches "ab", "abc", "abcc", "abccc", etc.

Concatenating Atoms
You can concatenate atoms to form more complex sequences to be
matched. Some examples:

{forte} matches the sequence consisting of "f", followed by "o",
followed by "r", followed by "t", followed by "e".
{a?b} matches zero or one occurrences of the letter a, followed by
one occurrence of the letter b. Thus, it matches "ab" and "b".
{[0-9]+/[0-9]+} matches one or more digits, followed by a "/"
character, followed by another sequence of one or more digits.
{foo(bar)*} matches the sequence "foo", followed by zero or more
occurrences of the sequence "bar". Thus, it matches, "foo", "foobar",
"foobarbar", "foobarbarbar", etc.

{^foo.*bar} matches the sequence "foo", followed a sequence of
zero or more occurrences of any character, followed by the sequence
"bar". In addition, the ^ requires that the match occur at the
beginning of the field.

Alternation
You can specify alternative matches, by combining sequences of atoms
with the | operator. For example,

{a|b} matches either the letter "a" or the letter "b".

{foo|bar} matches either the sequence "foo" or the sequence
"bar".

Filter Operators Applied to Regular Expressions
You can apply the filter operators = (match entire field) and %
(case-sensitive) to regular expressions in the same way that you can
use them with word phrases.
= Exact Match Operator
If a regular expression is preceded by =, Agent must match the entire
field. For example, the following filter:
subject: ={begin.*end}
matches any subject that starts with the characters "begin", ends with
the characters "end" and contains any characters in between.
As with word phrases, when the = operator is applied to a subject
field, Agent first removes any "Re: " prefixes before performing the
match. For example, the filter
subject: ={testing}

matches both "testing" and "Re: testing". But, it does not match
"more testing", since the = operator requires an exact match of the
entire field.

% Case Sensitive Operator
The % operator causes the entire regular expression to be
case-sensitive. Otherwise, regular expressions in Agent filters are
case-insensitive. For example, the filter
subject: %{[A-Z]+}
matches a subject containing a sequence of one or more CAPITAL
LETTERS.
But, the filter
subject: {[a-z]+}

matches a subject containing a sequence of one or more letters, in any
combination of lower-case or CAPITALS.

See Also:
How to Create Message Filters
Agent's Expression Language
Characters Used in Expressions
Expression Examples
How to Filter Out Spam
How to Create View Filters