logfilehandler.h File Reference

Detailed Description

The logfilehandler will "watch" the the tail of text file (usually a log file) and do a regex match. This is useful to watch web server log or /var/log/messages for lines that indicate events you care about (e.g., failed logins, dropped packets from iptables, su to root, etc.). You may have multiple regex's, each generating its own events. This avoids opening/reading the file for each regex, rather it is is read one time for all. You may optionally specify a subsitution string that behaves similar to 'sed' to customize the generated events. If you supply a substitution string, rather than copying the original complete matching string, the substitution string is used, replacing all occurances of the special charcters \[0-9] with the associated substring matches. Note: \0 matches the whole expression, while \1,\2 through \9 are substring matches 1, 2 through 9. For example, say you are logging events from a packet filter (e.g., Snort) that reports all sorts of alerts, and you are interested in only the denial of service alerts, further you want to format the string nicely, pulling out the from and to IP:PORT. You might use a regex like:
"DOS.* ([0-9]*\.[0-9]*\.[0-9]*\.[0-9]*:[0-9]*) -> ([0-9]*\.[0-9]*\.[0-9]*\.[0-9]*:[0-9]*)"
This has 2 substrings, one for each IP:PORT. The substitution string might be: "DOS\t\1\t\2" which would result in a tab delimited record having the 'from' IP:PORT substring followed by the 'to' IP:PORT substring. You might configure an sfilterhandler to dispatch these alerts based on the destination IP:PORT, sending notification to different sysadmins responsible for the different networks/subnets. Regex's are POSIX 1003.2 "extended" form. If the digest: keyword is supplied then duplicate events are suppressed over the time interval and a summary event is emmited at the end of the time interval. This is useful in combination with regex substitution for environments that might have a "storm" of errors in a log file. In this situation you would use the regex subsitution to remove the timestamp in log file entry and enable the digest. When you had a storm of identical errors (except for the timestamp in the log file) that you were watching for with a regex, then you would get an event for the first error and then only a summary at the end of the digest time interval, rather than the thousands of events you might have received without the digest option.

Wire keywords (standard handler keywords documented in Wire )

filename: filename
match: match-event nomatch-event regexp subst (if either match-event or nomatch-event are empty strings they are ignored)
digest: seconds Compute a summary of duplicates over time interval of seconds

You may have multiple match: lines. Matches are applied in the order they are declared. Example:

///////////////////////////////
// watch /var/log/messages for:
//     'su root' messages
//     ipchains messages
//     ftp logins
///////////////////////////////
set su create event { name: su }
set packetdrop create event { name: packetdrop }
set ftplogin create event { name: ftplogin }
create handler logfile { 
        filename: /var/log/messages
	match: $su "" "\(su\) session" ""
	match: $packetdrop "" "Packet log:.*REJECT" ""
	match: $ftplogin "" "FTP LOGIN" ""
	cycletime: 60
}

Events generated:

Event Name	Type	Description
match	AW_EVENT_TYPE_STRING	Regex match in line, send string of matched line or substitution
nomatch	AW_EVENT_TYPE_STRING	No regex match in line, send string of unmatched line

Events accepted:

Type	Description
Any	Generate an event of type requested

#include "hash.h"
#include "regexmatch.h"
#include "monitor.h"
#include "wire.h"

Go to the source code of this file.

Data Structures

struct aw_logfilehandler_digest_s

struct aw_logfilehandler_events_t

Events generated. More...

struct aw_logfilehandler_regex_t

Specifies a regex and associated events. More...

struct aw_logfilehandler_state_t

State of handler during probing. More...

struct aw_logfilehandler_t

Handler object. More...

struct aw_logfilehandler_watcher_t

Compiled regex, associated events and state. More...

Typedefs

typedef aw_logfilehandler_digest_s aw_logfilehandler_digest_t

Functions

aw_logfilehandler_t * aw_create_logfilehandler (const byte_t *filename, const u_int32_t nregexs, const aw_logfilehandler_regex_t *regexs, const aw_timestamp_t digest_interval, int32_t(*regevent)(aw_handler_t *h, aw_address_t *eventid, u_int32_t index), int32_t(*regmaskevent)(aw_handler_t *h, aw_address_t *eventid, u_int32_t index), const aw_alarm_sched_t *sched, aw_logger_t *logger)

Create a logfile handler.

void aw_free_logfilehandler (aw_logfilehandler_t *p)

Free handler and all associated resources.

aw_handler_t * aw_wire_logfilehandler (aw_wire_mkhandler_args_t *args)

Create a logfile handler using "wire". See header doc for keyword documentation.

Typedef Documentation

typedef struct aw_logfilehandler_digest_s aw_logfilehandler_digest_t

Function Documentation

aw_logfilehandler_t* aw_create_logfilehandler ( const byte_t * filename,

const u_int32_t nregexs,

const aw_logfilehandler_regex_t * regexs,

const aw_timestamp_t digest_interval,

int32_t(* regevent)(aw_handler_t *h, aw_address_t *eventid, u_int32_t index),

int32_t(* regmaskevent)(aw_handler_t *h, aw_address_t *eventid, u_int32_t index),

const aw_alarm_sched_t * sched,

aw_logger_t * logger

)

Create a logfile handler.
aw_create_logfilehandler

Parameters:

filename File to watch

nregexs How many regex strings

regexs Array of regex strings and events

digest_interval Time interval over which to compute digest of dups

regevent Function pointer for event registration

regmaskevent Function pointer for event mask registration

sched Run schedule

logger Logger object

Returns:
Handler object

void aw_free_logfilehandler ( aw_logfilehandler_t * p )

Free handler and all associated resources.
aw_free_logfilehandler

Parameters:

p The handler

aw_handler_t* aw_wire_logfilehandler ( aw_wire_mkhandler_args_t * args )

Create a logfile handler using "wire". See header doc for keyword documentation.
aw_wire_logfilehandler


Data Structures
struct	aw_logfilehandler_digest_s
struct	aw_logfilehandler_events_t
	Events generated. More...
struct	aw_logfilehandler_regex_t
	Specifies a regex and associated events. More...
struct	aw_logfilehandler_state_t
	State of handler during probing. More...
struct	aw_logfilehandler_t
	Handler object. More...
struct	aw_logfilehandler_watcher_t
	Compiled regex, associated events and state. More...
Typedefs
typedef aw_logfilehandler_digest_s	aw_logfilehandler_digest_t
Functions
aw_logfilehandler_t *	aw_create_logfilehandler (const byte_t filename, const u_int32_t nregexs, const aw_logfilehandler_regex_t regexs, const aw_timestamp_t digest_interval, int32_t(regevent)(aw_handler_t h, aw_address_t eventid, u_int32_t index), int32_t(regmaskevent)(aw_handler_t h, aw_address_t eventid, u_int32_t index), const aw_alarm_sched_t sched, aw_logger_t logger)
	Create a logfile handler.
void	aw_free_logfilehandler (aw_logfilehandler_t *p)
	Free handler and all associated resources.
aw_handler_t *	aw_wire_logfilehandler (aw_wire_mkhandler_args_t *args)
	Create a logfile handler using "wire". See header doc for keyword documentation.