This is a guide for writing patterns for l7-filter.  If you want a HOWTO
for _using_ l7-filter, see http://l7-filter.sf.net

*** File Format ***

A pattern file consists of the name of the protocol on one line followed
by a regular expression defining that protocol on one line.  Lines
starting with '#' and blank lines are ignored.  The name of the file
must match the protocol defined in it.  i.e. if the protocol is "ftp",
the file must be called "ftp.pat".

Additionally, to be displayed nicely on the webpage, the top lines should
look like this:

# <Protocol name and some concise detail about the protocol>
# Pattern quality: <great|good|ok|marginal|poor>

In other words, the top line should start with "# " and then have a
freeform string.  The next line should start with "# Pattern quality: "
and then have one of "great", "good", "ok", "marginal" or "poor".

*** Regular Expressions ***

At the moment, l7-filter uses V8 regular expresions (see
http://www.hmug.org/man/3/regsub.html).  In addition, we have added the
capability to do perl-style hex matches using the \xHH notation (i.e. to
match a tab, use \x09).  Note that our regexps are more limited than the
ones you may be used to. Notably, you CANNOT use bounds ("foo{3}" to
match foofoofoo), character classes ("[[:punct:]]" to match any
punctuation) or backreferences.  

If you want to check for printable characters, including whitespace,
[\x09-\0d -~] is a reasonable way of doing it.  (Don't just use "." if
you can help it.)

l7-filter is case insensitive.  Using upper case in your patterns is
identical to using lower case.  This is largly because our regexp
implementation doesn't have a case-insensitivity flag.

l7-filter strips out nulls (0x00 bytes) in the packets it reads so that
it can treat them as normal C strings.  Therefore, you can't match on
nulls.  Also, fields may appear shorter than expected.  For example, if
a protocol has a 4 byte field and any of those bytes can be null, the
field can appear to be any length from 0 to 4.

*** What The Classifier Sees ***

If you have set up your computer as recommended in the documentation
(http://l7-filter.sf.net), the data to be matched is that of both the
client and the server, in the order that it passes through the computer.
For instance, in FTP, the first thing the filter sees is "221 server
ready", then "USER bob", then "331 send password", then "PASS
frogbeard", and so on.

l7-filter can match across packets.  For instance, you could match FTP
with "220.*USER.*331".

*** What Makes A Good Pattern ***

There are two general guidelines:

1) A pattern must be neither too specific nor not specific enough.  

	Example 1: The pattern "bear" for Bearshare is not specific enough. 
	This pattern could match a wide variety of non-Bearshare connections. 
	For instance a HTTP request for http://bear.com would be matched.

	Example 2: "220 .*ftp.*(\[.*\]|\(.*\))" for FTP is too specific.  
	Not all servers send ()s or []s after their 220.  In fact, servers 
	are not even required to send the string "ftp" at any time, but the 
	vast majority do.  Good judgement and testing are necessary for 
	instances such as this.

2) It should use a minimum of processing power.  Thus, if it is possible
to reduce the number of *'s, +'s and |'s in your pattern, you should do
so.  Use the performance testing program included in the patterns package 
to determine the speed of your pattern.

The recommended procedure for writing packets is this:

1) Find and read the spec for the protocol you wish to match.  If it's
an Internet standard, http://rfc-editor.org is a good place to start,
although not all standards are RFCs.  If it is a proprietary protocol,
it is likely that someone has written a reverse-engineered spec for it.
Do a general web search to find it.  Skipping this step is a good way to
write patterns that are overly specific!

2) Use Ethereal (http://www.ethereal.com/) to watch packets of this
protocol go by in a typical session of its use.  (If you failed to find
a spec for your protocol, but Ethereal can parse it, reading the
Ethereal source code may also be worth your time.)

3) Write a pattern that will reliably match one of the first few packets
that are sent in your protocol.  Test it.  Test its performance.

4) Send your pattern to l7-filter-developers@lists.sf.net for it to be
incorporated into the official pattern definitions.
