grtool ReadMe
================
(C) 2005 Marc Schiffbauer

||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

grtool [1] is a small utility useful for people who are dealing with
learning logs and policy files created by and maintained for the
grsecurity [2] RBAC system by Brad Spengler.

Main features of grtool
=======================

split-policy
============
On busy machines with a lot of services running a policy file can be
really huge and hard to create or maintain. This is where grtool
can help you: It can split a policy file into many files and 
directories so that you end up with a directory structure where
every role is represented by a directory with a sub-directory
structure with each subject being a seperate file. The single
subject files are connected by the "include" statement of the
RBAC policy file. 

The structure can look like this:

policy <-- only includes "include" statements to the _role file
           of every role under the policy.d directory

policy.d/
|-- default_role
|   |-- default.subject <-- contains the "/" default subject policy
|   `-- _role           <-- contains role spec and include statements for the subjects
|-- www-user.group_role
|   |-- default.subject
|   |-- _role   
|   `-- usr
|       |-- lib
|       |   |-- cgi-bin
|       |   |   `-- php4.subject
|       |   `-- suphp
|       |       `-- suphp.subject
|       `-- sbin
|           `-- exim4
|-- cyrus.user_role
|   |-- default.subject
|   |-- etc
|   |   `-- cron.daily.subject
|   |-- _role
|   `-- usr
|       |-- lib
|       |   `-- cyrus
|       |       `-- bin
|       |           |-- imapd.subject
|       |           |-- lmtpd.subject
|       |           `-- pop3d.subject
|       `-- sbin
|           |-- ctl_cyrusdb.subject
|           |-- ctl_deliver.subject
|           |-- ipurge.subject
|           `-- tls_prune.subject
|-- list.user_role
|   |-- default.subject
|   |-- _role
|   `-- usr
|       |-- bin
|       |   `-- python2.2.subject
|       `-- lib
|           `-- mailman
|               |-- bin
|               |   `-- mailmanctl.subject
|               `-- cron
|                   |-- checkdbs.subject
|                   |-- disabled.subject
|                   |-- mailpasswds.subject
|                   `-- nightly_gzip.subject
`-- root.user_role
    |-- default.subject
    |-- bin
    |   |-- afio.subject
    |   |-- cat.subject
    |   |-- mkdir.subject
    |   |-- mktemp.subject
    |   |-- mv.subject
    |   |-- rm.subject
    |   `-- su.subject
    |-- etc
    |   |-- cron.daily.subject
    |   `-- cron.monthly.subject
    |-- home
    |   |-- backup.subject
    |   |-- chroot
    |       `-- usr
    |           `-- sbin
    |               `-- autolog.subject
    |-- _role
    |-- sbin
    |   `-- syslogd.subject
    `-- usr
        |-- bin
        |   |-- mail.subject
        |   `-- rsync.subject
        |   `-- sysstat
        |       `-- sadc.subject
        |-- local
        |   `-- sbin
        |       `-- tob.subject
        `-- sbin
            |-- apache.subject
            |-- chroot.subject
            |-- cron.subject
            |-- exim4.subject
            |-- inetd.subject
            |-- logcheck.subject
            |-- logtail.subject
            |-- monit.subject
            |-- ntpd.subject
            |-- proftpd.subject
            |-- smartd.subject
            |-- sshd.subject
            `-- tcpd.subject

||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

glob-objects
============
The grsecurity RBAC system supports object globbing which means you
can use wildcarded objects which is useful for libs in many cases
because lib names contain their version in the filename. So if you
do not want to have to update your policy everytime you bump up a
lib version you can just write "*" instead of the version part in the
filename for a lib object of a lib. 
You can use grtool to automatically glob objects that match a 
specified regular expression.

Example: You create a new policy from your full learning logs and
want to have all libs under /usr/lib to be globbed.

You want the object for /usr/lib/libfoo.so.1.2.3 to be just 
/usr/lib/libfoo.so* and so on.

Running

  grtool glob-objects -r '(/usr/lib/lib.*\.so).*' -p /path/to/policy-file

will update your policy automatically.
Any part of an object in /path/to/policy-file that matches after the
pair of braces will be globbed to "*".

The regex must contain one or two pairs of braces and the matching 
part within those will be the part of the object that is preserved.
  The matching part within the first pair of braces will be the part 
before the asterisk (*) and the matching part of the second pair the 
one after it.

more Examples:
  one pair of braces:
    -r '(/usr/lib/lib.*.so.).*'
      will replace an object like
        /usr/lib/libgnutls.so.11.1.16
      by
        /usr/lib/libgnutls.so.*

  two pairs of braces:
    -r '(/lib/ld-2.).*(.so)'
      will replace an object like
        /lib/ld-2.2.5.so
      by
        /lib/ld-2.*.so

convert-log (currently disabled!)
===========
Maybe you did role- or subject based learning on a process and you want
to do full learning again. There is no need to let the system create
learning log in full leanirng mode if you still have the log from the
role- or subject based learning process on your disk. Just use grtool
to convert the learning log to full leanring log format

shrink-log
==========
The grlean daemon tries to filter dublicate loglines where possible
while learning to keep logs small. This works only for similar loglines 
that occure within a small timeframe. 
Now if you have lots of learning logs and you want to keep them to do 
later policy generation maybe several times while tuning learn_config
or maybe to re-generate policies with later grsecutity versions you
can safe disk-space and speed-up learning by optimizing the logs with
the shrink-log command of grtool. 

The shrink-log command will eliminate all duplicate lines which 
makes the logfiles much smaller.

Example:
A big logfile with 2GB can often be shrinked by 90%. The result is a
file with round about 200MB. gradm -L will produce the same policy
with both files but with the small file it will do it *a lot* faster.

Some numbers:
On my Celeron 2GHz, 512MB RAM (2GB vs. 200MB logfile):

Policy generation with the original logfile:
# time gradm -F -L 2GB.log -O policy-from-2GBfile
[...]
Full learning complete.

real    364m4.218s
user    238m55.160s
sys     8m39.650s

(~6 hours)

Policy generation with log shrinked by grtool:
# time gradm -F -L 200MB.log -O policy-from-200MBfile
[...]
Full learning complete.

real    42m51.047s
user    36m15.670s
sys     0m29.080s

(~42 minutes)

----
[1] http://linuxcc.de/grtool/
[2] http://grsecurity.net
