
FIREWALL LOG DAEMON for LINUX
firelogd

DISCLAIMER:
This software is offered for free and without restrictions of any sort except
that you agree not to hold me responsible if it does something you don't like.
If that does happen I would appreciate hearing about it.


THIS IS NOT A FIREWALL! It will not increase the security of your machines
except that it makes it easy to review your log data and/or respond to it.
It will not affect your existing log setup.


DESCRIPTION:
This is a program that will parse ipchains or netfilter (iptables)
log data in real time. It will queue up a small batch of alerts and mail
them to you. It can also be used to parse an existing log file and it will
take log data on standard input for formatting.


FEATURES:
It features host name resolution and port/protocol lookups as well as icmp
code/type data. It was designed to be run as a daemon process, but you can
use it to output data to a terminal or to turn a logfile into something
readable. Output preprocessor templates can be written to format the data
any way you like. Some examples are given.


HOW DOES IT WORK?
Firelogd reads a FIFO (first in, first out) file that is written by syslog.
Syslog writes to the pipe and firelogd is there to parse the log entry into
human readable format. It will save a (BUFFERSIZE) number of log alerts and
mail them to you when the threshold is reached. This will not change anything
about your current log setup, but it will make it easier to review and respond
to security threats. If nothing else, you will end up *actually reading* your
firewall log entries.


TABLES AND CHAINS TOGETHER:
Firelogd does not care what kind of log data it is given. It will figure it
out as soon as a log line matces an internal regular expression. Once a line
matches for iptables or ipchains log data, it will stop trying to match the 
other. 

If you are in a mixed logging environment (running a log server or testing 
both chain and table configurations) you can use the "-m" option to cause the
program to keep trying to match both types of data. This imposes a performance
penalty, so don't use it unless you need it.


INSTALLATION:
Read the file QUICKSTART for information on building and installing firelogd.
The sources should build on any system with a compiler and semi-recent libs.


BUILD TARGETS:
	make easy	setup, make, install, start program	
	
	make		build the executable
	make install	copy it to /usr/sbin/
	make setup	create a fifo in /var/log, add it to syslog


POST INSTALLATION
Read the documentation. Good for you! You already knew that.
You will need to add the program to your startup scripts if you want it to
continue working for you after a reboot. The easiest way to do this is to add
a line to /etc/rc.d/rc.local that will run the program after startup.


SECURITY NOTE
This program has been carefully tested in an attempt to find weaknesses, and
kernel data comes from your kernel not the net. Nevertheless, you should be
aware that ANY program could have holes in it, including the kernel. This
program can be run as a user other than root, just modify the file permissions.


RUNNING THE PROGRAM:
If you run the program without options it will open the default file (FIFO)
and block (wait) until there is log data sent by syslog. You can start this
in a terminal and watch as log data comes in. It will print to the screen.
Eventually you will want to stop watching the screen...CTRL-C will stop it.

Be aware that because firelogd will attempt to resolve all IP addresses, you
may see some delay in the output. This is normally only a problem if your
DNS system is not set up properly. Even if DNS is working properly, some
queries will have to time out.

Start the program with the "-d" flag and it will go away and watch your FIFO
for log data. When the default number of log entries is reached, it will send
you email with the formatted log data. The default email address is "root"
which will get sent to the local machine. You can specify an email address by
using the "-e" option.

Examples:
-e myaddr@other.host.net
-eian

The default number of log entries before mailing is 10. You can change it by
using the "-b<buffersize>" option.

If you want to crunch an existing log file you can specify the file by using 
the "-l" option. This will also work on a FIFO.

Example:
	(if daemon is not already running)
	firelogd -l /var/log/messages > ~/badguys.log

	-or-

	(if daemon is already running)
	firelogd - < /var/log/messages > ~/firewall_hits

The default log source is /var/log/kernelpipe

You can alter the output format by using an output preprocessor template (See
below). The default location for templates is /etc/firelog.conf. If there is
no template file there or on the command line, the internal default output
format will result.


EXTENDED PORT/SERVICES AND ICMP LOOKUP
If you used "make install" you will find three new files in your /etc directory.
The file /etc/firelog.conf is for output templates. The file iana-icmp-numbers
is for icmp type/code lookup. The file /etc/iana-port-numbers is used for port
and service lookups (you can disable this via the command line with -s). You
can replace this file with any file in the "services" format. For example, if
you have nmap, you can use nmap-services to get information on ports that
are known to be trojans/backdoors.


PROGRAM OPTIONS
There are several recognized options to the program:
	-d 		will cause it to become a daemon process with the
			default mailbuffer size (10) and the default email
			address (root)
	
	-b<size> 	will set the size of the mailbuffer, this implies -d
			example: "firelogd -b50" or "firelogd -b 4"
			
	-k 		will kill a running firelogd
	
	 -		will read from stdin and parse log data
	 		example: "cat /var/log/kernel | firelogd -"

	-l<log>		specify the log file to parse or the FIFO to watch
			The default is /var/log/kernelpipe

	-m		"mixed" logs, tables and chains log data in one stream

	-s		disable the use of 'extended' port/service lookups and
			use the system services file (getservbyport)

	-t<template>	specify the output preprocessor template (see below)
			the default location is /etc/firelog.conf

	-e<email>	specify the email address to send alerts (daemon mode)
			the default is root@localhost

Any other option will display a brief help message.


THE BUFFER FILE
If a running daemon is killed it will attempt to write a buffer file with log
data that was not yet flushed to mail. It will read and remove this file the
next time you run it in daemon mode. The will save log data until the daemon
is restarted.


THE PID FILE
In some cases (like when you use "kill -9 `pidof firelogd`) the daemon will
die without removing it's pid file. If you try to start it as a daemon and it
complains that it is already running, you can use the "-k" option and it will
either kill the running process or remove the stale pid file.


OUTPUT TEMPLATES:
Using an output preprocessor template you can have the log data formatted any
way you want. This is done by creating a template file which is read in on
program startup by specifying it with the "-t <template>" option. If you do
not specify an output template the program will use it's own default format.
You can change the default format by altering the the source and rebuilding.

A couple of example templates are included in the file TEMPLATES of the 
installation directory. This file can be copied to /etc/firelog.conf and it
will be read on startup.

See the file "decode.php" and the corresponding template in the "TEMPLATES"
file for an example of how this feature can be used to build your own custom
logger.

Output templates are simply a file with space delimited tokens. By that I mean
that any single word is a token. The token '$' is simply a three character
token with no special meaning. In the output for each log entry, the token
'$' would stand for itself with no whitespace around it. There are special
tokens that will get replaced with log data or whitespace. One example of this
is the two character token "nl" (no quotes). It will cause a newline to be
output in it's place for each log entry. The token -> sp <- will output a 
space character.

Two other special tokens are: srcip r_srcip
srcip will be replaced with the source IP address of the host that logged on
your firewall and r_srcip will output the resolved hostname (if available).

A simple output template then could consist of the following:
------------------------------------------------------------:

The sp host sp r_srcip sp - sp IP sp address sp srcip nl
hit sp me sp on sp month sp day sp at sp time sp YIKES !! ! nl nl

------------------------------------------------------------

The previous template would show the following for three firewall hits using
the command:
"tail -3 /var/log/kernel | firelogd -t test_template -"

The host badman.crack.nl - IP address 214.0.3.6
hit me on Dec 1 at 12:20:37 YIKES!!!

The host 10.0.2.10 - IP address 10.0.2.10
hit me on Dec 1 at 12:53:01 YIKES!!!

The host PAY2ppp-6.uc.infovia.com.ar - IP address 209.13.214.134
hit me on Dec 2 at 18:15:41 YIKES!!!

=============================================================

The special tokens are actually variable names in the data structure which
hold the log data. If you want to use one of the special token names you can
split it up into two tokens and they becomes just themselves. All tokens and
replacements are concatenated together on output. The special token "time"
(no quotes) can be written as two tokens: ->ti<- and  ->me<-. Together
they will print "time" in the output with no replacement as the single token
time would.

You can put comments into your template file by using the special tokens:
startcomment
endcomment

The complete list of special tokens follows:
-----------------------------------------------------------------------------

  -- structure --	-- token name --	-- replacement text --
  	n/a			sp		the space character(' ')
	n/a			nl		the new line character('\n')
	n/a			tab		the tab character('\t')

	ltype			ltype		iptables or ipchains
	char proto[11]		proto		the protocol
	
	char log[1024]		log		the actual log entry
	char month[4]		month		Jan - Dec
	char day[3]		day		1 - 31
	char time[9]		time		00:00:00
	char msg[50]		msg		log messages
	char in[5]		in		interface
	char out[5]		out		interface
	char mac[46]		mac		mac address
	char srcip[16]		srcip		source ip address
	char r_srcip[255]	r_srcip		resolved source address
	char dstip[16]		dstip		destination ip address
	char r_dstip[255]	r_dstip		resolved destination address
	char iplen[6]		iplen		ip header length
	char tos[5]		tos		hex TOS field
	char sflags[6]		sflags		TOS bits (***** - DTREC)
						Delay|Throughput|Reliability|
						ECT-ECN/Monetary|CE-ECN/Reserved
						
	char prec[5]		prec		hex TOS precedence bits
	char pflags[4]		pflags		Precedence bits (*** - 123)
	char ttl[4]		ttl		time to live
	char id[6]		id		ip ID
	char frag[145]		frag		fragment data
	char fflags[4]		fflags		IP flags (+** - CDM)
						Reserved/CE?|Don't Fragment|More
						Fragments
						
						?? Does anyone know why linux
						sets bit 0 of the IP flags as
						the congestion bit? RFC2481
						refers to the TOS bits 6&7 ???
ICMP SPECIFIC	
	char type[4]		type		icmp type number
	char code[4]		code		icmp code number
	char info[128]		info		resolved icmp information
	char trigger[260]	trigger		the triggering packet's data
						- netfilter only

	struct pktlog recursed	recursed	the resolved information for
						the triggering packet
						- netfilter only
						This token acts as a complete
						log entry with the message
						"TRIGGERING PACKET"

TCP SPECIFIC	
	char window[6]		window		TCP window
	char res[5]		res		hex TCP reserved bits
	char flags[23]		flags		TCP flags, text (SYN only for
							ipchains)
	char tflags[9]		tflags		TCP flag bits
						(******* - 12UAPRSF)
						Reserved|Reserved|Urg|Ack|Psh
						Rst|Syn|Fin
	
UDP SPECIFIC
	char ulen[6]		ulen		datagram length
	
TCP/UDP	
	char srcpt[6]		srcpt		source port number
	char r_srcpt[16]	r_srcpt		source port, resolved
	char dstpt[6]		dstpt		destination port number
	char r_dstpt[16]	r_dstpt		destination port, resolved
  
--------------------------------------------------------------------------
			
NOTE: Fields that evaluate to a blank will not be printed. For example,
the token info (resolved ICMP information) will not cause any output for a 
TCP log. The token srcpt (source port) will not print anything for an ICMP
packet.

The token "recursed" will output a complete log entry corresponding to the
information from the packet that triggered the ICMP error message. The context
information (date, time) is set to the context of the ICMP packet. The log
message "TRIGGERING PACKET" denotes these entries. This is not possible with
ipchains.

Some of the fields are specific to netfilter/iptables. Ipchains log data will
produce a blank field ("") for these tokens. Example: recursed, prec

If you write a really nifty output template, please send it to me and I will
include it in future releases. Having said that, I can't guarantee that the
output template feature will not change in future releases. It feels kind of
kludgy, so I am certainly open to ideas.
____________________________________________________________________________

Enjoy!
Let me know if you have any problems.

Ian Jones - roux@speakeasy.org






