mod_msg_tgl: A Module for communicating with external processes to get DNS data
version 0.01

by Todd Lewis <tlewis@mindspring.com>
Sat Aug 14 18:49:51 EDT 1999

Table of Contents
-----------------

1. INTRODUCTION
    1.1 Message formats
    1.2 Communications media
    1.3 Invocation and Configuration
2. EXCHANGES
    2.1 Queries
    2.2 Zone Loads
    2.3 Updates
3. OPTYPES
    3.1 Query
    3.2 Answer
    3.3 Finis
    3.4 Exit
    3.5 Load Zone
    3.6 Reload Zone
    3.7 Delete Zone
    3.8 Error
    3.9 SOA
    3.10 Options
4. QTYPES
    4.1 Address
    4.2 MX
    4.3 Misc names
5. ERRORS
6. FUTURE WORK



1. INTRODUCTION
---------------

mod_msg_tgl works by sending messages between the parent, which will
be a Dents server, and a child process, called a driver process.

This driver is designed to be used in conjunction with external programs,
the driver processes.  The author of the driver process bears the primary
responsibility for documenting the usage of his driver with mod_msg_tgl.
This documentation is targeted mostly at mod_msg_tgl implementors and
authors of driver processes, detailing how the interaction between the
two works.  If users are looking for information on how to use a given
driver process, then they should probably see the documentation which
accompanies that software.

Because this module works via a well-documented(sic) message passing
interface, I don't feel comfortable imposing any licensing restrictions on
driver process authors; therefore, no such restrictions are made.

    1.1 Message formats
    ===================

All messages between server and driver process in either direction are
of the general form:

	TYPE: Message
	INHERITS FROM: (none)
	FORMAT: <seq#> <optype> (<params>)
	PARAMETERS:

	seq#: sequence number
		An integer value used to relate different messages as
		part of the same query.  
		e.g., "43"

	optype: operation type
		e.g., "Q"

	params: optional further parameters
		dependent on optype.  

This format is intended to be powerful and extensible while simple to
parse.  In particular, it is amenable to my favorite perl programming
paradigm:

	while(<>){
		chomp;
		@A=split;
		(...)
	}

The details come from the various optypes, which are described below.

The meaning of parameters is implicitly conveyed positionally.  I am
tempted to label parameters explicitly, but that would require more work
on the server side, where I'm already dreading all of the text processing
that all of this is going to require.  I hate string processing in C.

All parameters are separated by a space, i.e., ASCII 0x20.  No provision
is made for escaping spaces in parameters, so no parameters may contain
spaces.

All messages are terminated by a newline, i.e., ASCII 0xA.  Receipt of
EOF/NUL (ASCII 0x0) is an error condition and can result in termination.

    1.2 Communications media
    ========================

Lost messages between the server and the driver process can result in
deadlock, so it is important that any communications channel be reliable.
(I've added timeouts to the server code, so deadlock isn't so much a
concern now, but lost messages still result in badness.)

The only communications channel specified right now is pipe-based.
The child will be born finding its stdin and stdout connected to the
server.

There is no reason that this protocol could not be used over any
other suitable transport, such as a TCP connection or a shared-memory,
mailbox-based approach.  Such fun and games are left for a future revision
of this specification.

    1.3 Invocation and Configuration
    ================================

Because of the present form of the Dents driver module system, we don't
really pass any useful information at driver process startup time.
mod_msg_tgl will only spawn off a single instance of a given driver
process within a single Dents instance, and all arguments are passed to
the driver process as part of the "Load Zone" directive.  (See below.)

2. EXCHANGES
------------

Deadlock avoidance is important in this sort of arrangement.  To describe
the responsibilities of each party, we will use the notion of "the ball".
Whichever party has the ball can disregard the other party until it needs
to interact with it.  Whichever party does not have the ball needs to pay
attention to the other party until it receives some information from it.

By default, the server has the ball.

    2.1 Queries
    ===========

This exchange passes a query from the server to the driver process and
then passes an answer from the driver process back to the server.  The
order is as follows:

	A. The server sends a message of type "Query" (see below) to
	the driver process.  Once this happens, the driver process has
	the ball.

	B. The driver process sends a sequence of zero or more "Answer"
	messages (see below) to the server.  These answers use the
	same sequence ID that the server sent in its query.  The driver
	retains the ball.

	C. The driver sends a "Finis" message (see below) to the server.
	Once this happens, the ball is returned to the server.

    2.2 Zone Loads
    ==============

This exchange passes a query from the server to the driver process and
then passes an answer from the driver process back to the server.  The
order is as follows:

	A. The server sends a message of type "Zone Load" (see below) to
	the driver process.  Once this happens, the driver process has
	the ball.

	B. The driver process optionally returns an "Error" message
	indicating that it encountered an error.  In this case, the
	server will consider the zone load a failure, will not consider
	the zone loaded, and will not answer queries for the zone.
	This terminates the dialogue and returns the ball to the
	server.

	C. If the load is a success, then the driver process should
	return an "SOA" message.  (see below.) Once this happens, the
	ball is returned to the server.

    2.3 Zone Deletes
    ================

This exchange passes a query from the server to the driver process and
then passes an answer from the driver process back to the server.  The
order is as follows:

	A. The server sends a message of type "Delete Zone" (see below) to
	the driver process.  Once this happens, the driver process has
	the ball.

	B. The driver sends a "Finis" message (see below) to the server.
	Once this happens, the ball is returned to the server.

There can be no errors in this case; since the server will not be sending
any more queries for the domain in question to the driver process,
it doesn't matter if the driver process succeeded or not.

    2.4 Updates
    ===========

This exchange will be used to update zone data.  Presently unspecified,
it will probably be a good bit more complicated than the Query exchange.



3. OPTYPES
----------

OPTYPE table:
--------+-------------
optype  |  description
--------+-------------
Q       | Query
A       | Answer
F       | Finis
X       | Exit
ZL      | Load Zone
ZR      | Reload Zone
ZD      | Delete Zone
E       | Error
SOA     | SOA
O       | Options

    3.1 Query
    =========

The query message is passed from the server to the driver process.
It conveys a DNS query to which the driver process responds with zero
or more answers followed by a "Finis" message.

	TYPE: Simple Query
	INHERITS FROM: message
	FORMAT: <seq#> Q <name> <qtype>
	PARAMETERS:

	name: DNS name
		The DNS name of the query in textual form (i.e.,
		dot-separated.)  e.g., "foo.example.dents.org.".
		This name shall be fully-qualified and shall end with
		a trailing dot ('.', ASCII 0x2E.)

	qtype: DNS query type, numeric or textual
		e.g., "5" or "CNAME"

Notes:

- The "<name>" parameter will probably follow the RFC-1035 preferred
name syntax, but you shouldn't count on that.  (see RFC-1035, section 2.3.1)

- qtype consists of all of the qtypes listed in RFC-1035, section 3.2.3.

- qclass consists of all the qclasses listed in RFC-1035, section 3.2.5.

- drivers need not consider themselves obligated to support query classes
other than 1 ("IN").

There is an alternate form of this message:

	TYPE: Extended Query
	INHERITS FROM: Simple Query
	FORMAT: <seq#> EQ <name> <qtype> <qclass> <id> <client-ip> <client-port> <opcode> <rd>
	PARAMETERS:

	qclass - DNS query class, numeric or textual e.g., "1" or "IN"

	id - the id field in the query packet; a 16-bit number represented
	here in ASCII decimal format.  E.g. "10476".

	client-ip - the IP address of the client in dotted-quad notation.
	e.g., "168.121.254.3"

	client-port - the port number used by the client for the request.
	e.g., "53"

	opcode - the DNS opcode passed by the client; a 4-bit number
	represented in ASCII decimal format.  E.g., "1", meaning
	standard query.  (See RFC-1035, section 4.1.1)

	rd - the recursion desired bit passed by the client.

    3.2 Answer
    ==========

The answer message is passed from the driver process to the server.  A DNS
query is followed with zero or more answers followed by a "Finis" message.

	TYPE: Answer
	INHERITS FROM: message
	FORMAT: <seq#> A <name> <type> <data> (<options>)
	PARAMETERS:

	name: DNS name
		The DNS name of the answer in textual form (i.e.,
		dot-separated.)  e.g., "foo.example.dents.org.". This
		name shall be fully- qualified and shall end with a
		trailing dot ('.', ASCII 0x2E.)

	type: DNS query type
		e.g., "1" or "A"

	data: answer data
		dependent on type.  
		e.g., "100.150.200.250"

	options: a series of zero or more "k=v" pairs of the following types:

	section: the section of the answer message in which to pack the RR
		Answer, NS, Additional or Cache.  Cache results in no
		packing at all, but rather merely in the server taking
		note; intended only for SOA.  (FIXME: details)
		E.g., "section=ANSWER" or "section=0"
		(default: ANSWER)

	class: DNS query class
		E.g., "class=IN" or "class=1"
		(default: IN)

	ttl: 32-bit quantity indication the ttl for this RR.
		This parameter is optional; if ommitted, then the
		ttl from the SOA will be used.  (See below, "SOA"
		for info on how to populate the SOA.)
		E.g., "ttl=3600"
		(default: value from SOA)

	authority: set the authority bit
		This parameter allows the dproc explicitly to set the
		"Authoritative Answer" bit in the response.
		E.g., "authority=0"
		(default: 1)

- data is a variable type.  For more info, see the ("QTYPES") section
below, which specifies a pile of formats which inherit from this type.

    3.3 Finis
    =========

The "Finis" message is passed from the driver process to the server.
A DNS query is followed with zero or more answers followed by a "Finis"
message.

	TYPE: Finis
	INHERITS FROM: message
	FORMAT: <seq#> F
	PARAMETERS: (none)

    3.4 Exit
    =========

The "Exit" message tells the driver process that it should exit.
The driver process should respond by, like, exiting, and stuff.

	TYPE: Exit
	INHERITS FROM: message
	FORMAT: <seq#> X
	PARAMETERS: (none)

    3.5 Load Zone
    ============

The "LoadZone" message tells the driver process that it should load a
new zone.  For this version of the system, driver processes are supposed
to know themselves how to bootstrap zones.  I hope to remedy this in the
future.

	TYPE: Load Zone
	INHERITS FROM: message
	FORMAT: <seq#> ZL <zone> <args...>
	RAISES: Error #5
	PARAMETERS:

	zone: the domain name of the zone.  This name shall be
	fully-qualified and shall have a trailing '.' (ASCII 0x2E).

	args: the arguments passed via the Dents zone loading mechanism.
	mod_msg_tgl just pases these arguments through to the driver
	process.  The driver process should document the arguments it
	accepts and requires for the user.  The first argument is always
	the name of the driver script.

	(FIXME: Once Eric finishes his doc on how the new config file
	stuff works, document here more precisely what is passed throug.)

Here is what you see in the named.boot:

mod-primary msg.dents.com mod_msg_tgl /usr/local/bin/myscript arg1 arg2 arg3

And here's what gets passed to /usr/local/bin/myscript:

	100 ZL msg.dents.com /usr/local/bin/myscript arg1 arg2 arg3

N.b., the script name must be argv[0], with the remaining arguments
starting at argv[1].

You should probably include in your documentation the form of the named.boot
entry that the user should employ.


    3.6 Reload Zone
    ===============

The "Reload Zone" message tells the driver process that it should reload
an existing zone.  The meaning of this command will vary depending on the
nature of the driver; driver authors should document what a reload command
makes happen.

	TYPE: Reload zone
	INHERITS FROM: message
	FORMAT: <seq#> ZR <zone>
	RAISES: Error #10
	PARAMETERS:

	zone: the domain name of the zone.  This name shall be
	fully-qualified and shall have a trailing '.' (ASCII 0x2E).

    3.7 Delete Zone
    ===============

The "Delete Zone" message tells the driver process that it should delete
an existing zone.

	TYPE: Delete zone
	INHERITS FROM: message
	FORMAT: <seq#> ZD <zone>
	RAISES: Error #10
	PARAMETERS:

	zone: the domain name of the zone.  This name shall be
	fully-qualified and shall have a trailing '.' (ASCII 0x2E).

    3.8 Error
    =========

The "Error" message tells the server that the command in question caused
an error the part of the driver process.

	TYPE: Error
	INHERITS FROM: message
	FORMAT: <seq#> E <ecode> <message>
	PARAMETERS:

	ecode: The error code.	See below, (Errors), for a listing of
	codes and their meaning

	message: A message describing what the problem was.  Should be
	fit for consumption by a sysadmin.  Remember, you can use spaces
	here, but not tabs.

    3.9 SOA
    =======

The "SOA" message informs the server of the SOA for the named zone.
This message may be issued at any time the driver process has the ball.
The settings communicated will be used for the answer terminated by
the next "Finis" message for that domain and for all subsequent answers
until a new SOA setting is made.

Note that an SOA message is different from an Answer message for the
SOA qtype.  The former instructs the server to cache the value of the
SOA for incidental use; the latter conveys an SOA RR for packing and
conveyance to the client.

	TYPE: SOA
	INHERITS FROM: message
	FORMAT: <seq#> SOA <domain> <mname> <rname> <serial> <refresh> <retry> <expire> <minimum>
	PARAMETERS:

	domain: The domain to which the SOA applies.  Driver processes
	can, of course, only set the SOA for domains for which they
	are responsible.

	mname:	The <domain-name> of the name server that was the original
	or primary source of data for this zone.

	rname:	A <domain-name> which specifies the mailbox of the person
	responsible for this zone.  (E.g., tlewis.gnu.org; pretty broken,
	but that's what the RFC says.)

	serial:  The unsigned 32 bit version number of the original copy
	of the zone.  Zone transfers preserve this value.  This value
	wraps and should be compared using sequence space arithmetic.

	refresh: A 32 bit time interval before the zone should be
	refreshed.

	retry:	A 32 bit time interval that should elapse before a failed
	refresh should be retried.

	expire: A 32 bit time value that specifies the upper limit on
	the time interval that can elapse before the zone is no longer
	authoritative.

	mimimum: The unsigned 32 bit minimum TTL field that should be
	exported with any RR from this zone.

    3.10 Options
    ============


	TYPE: O
	INHERITS FROM: message
	FORMAT: <seq#> O <domain> <options...>
	PARAMETERS:

	options: a list of one or more unordered options.  These are
	bare words which indicate to the server behaviour which the
	client requires of the server.  Currently supported options
	are:

	"NUMERIC" - indicates that the dproc requires class and rrtype
	to be conveyed numerically, i.e., in ASCII decimal form.  The
	default is verbal form, i.e., "CNAME" instead of "5".

	"EXTENDEDQ" - indicates that the dproc would rather receive
	"EQ" messages instead of "Q" messages.

4. QTYPES
---------

    4.1 Address
    ===========

	TYPE: Address
	INHERITS FROM: answer
	FORMAT: <seq#> A <name> 1 <address> (<options>)
	FORMAT: <seq#> A <name> A <address> (<options>)
	PARAMETERS:

	address: IP address
		The IPv4 address of the answer in dotted-quad format.
		e.g., "100.150.200.250"

    4.2 MX
    ======

	TYPE: MX
	INHERITS FROM: answer
	FORMAT: <seq#> A <name> 3 <prio> <exchange> (<options>)
	FORMAT: <seq#> A <name> MX <prio> <exchange> (<options>)
	PARAMETERS:

	prio: priority
		An integer value between 0 and 65535.
		e.g., "20"

	exchange: DNS name
		The DNS name of the exchange in textual form (i.e., dot-separated.)
		e.g., "mx1.example.dents.org."

    4.3 Misc names
    ==============

	TYPE: NS|CNAME|PTR|MD|MF|MB|MG|MR
	INHERITS FROM: answer
	FORMAT: <seq#> A <name> <type> <aname> (<options>)
	PARAMETERS:

	type: one of NS|CNAME|PTR|MD|MF|MB|MG|MR
	aname: the answered domain name

    4.4 TXT|NSAP
    ============

	TYPE: TXT|NSAME
	INHERITS FROM: answer
	FORMAT: <seq#> A <name> <type> <string> (<options>)
	PARAMETERS:

	type: one of TXT|NSAP
	string: a string; remember, no spaces!

    4.4 A-SOA
    =========

	TYPE: A-SOA
	INHERITS FROM: answer
	FORMAT: <seq#> A <name> SOA <mname> <rname> <serial> <refresh> <retry> <expire> <minimum> (<options>)
	PARAMETERS:

	All parameters have the same meaning as in the SOA message.  N.b.,
	the SOA message causes the server to cache the SOA value for future
	use, whereas the A-SOA message causes the server to pack an SOA
	RR into the reply packet sent to the client.  Two similar messages
	with very different purposes.

5. Errors
---------

2 - server vailure
	The dproc was unable to process the query because of some
	otherwise-unspecified internal error.  (An RCODE of 2 is
	what the DNS client sees for any dproc errors >15.)
3 - name no exist
	The name asked for does not exist.
4 - no support
	The dproc does not support that kind of query.
5 - refused
	The dproc refuses to answer the query.
105 - Load error
	This error is raised in response to a "ZL" message when the driver
	process is unable to load the zone for some reason.
110 - NO such zone
	This error is raised in response to a "ZD" or "Q" message when
	the driver process has no knowledge of the zone in question.



6. FUTURE WORK
--------------

The big gaping hole here is update support.  It needs to be transactional.
The existing sequence numbers can be treated as transaction ids, so it
should be a simple matter of adding "add_rr", "prepare", "commit" and
"rollback" optypes and having the driver process respond appropriately.
The hard part will be resucitating prepared transactions on restart.

The other obvious area is support for more types.

Miscellaneous thought: With the way that this design numbers queries,
there is no real need for the interface to be synchronous.  If we
could figure out how to get the file descriptors associated with the
driver within Dents, then we could make the interface asynchronous.
Really neat would be to run a recursive server using this interface to
talk to an external resolver; it shouldn't be too hard to write one up
using ADNS.  Oh, well, we'll leave that topic for later; it's certainly
not an appropriate thing to pursue right now.

TODO
====
misc. RR types

DONE
====
CHLD handler in server - DONE, needs debugging
TTL on A records - DONE
SOA - DONE
section selection support (should be easy) - DONE
support verbal as well as numeric type, class and section - DONE
general dproc option selection procedure - DONE
allow dproc to chose to receive verbal or numeric messages - DONE
optionally convey the opcode, recursion-desired in a Q message - DONE
allow dproc to set authority bit - DONE
extend error message so that dprocs can convey rcodes - DONE

RR types:
	A:	DONE
	CNAME:	DONE
	PTR:	DONE
	NS:	DONE
	MD:	DONE
	MF:	DONE
	MB:	DONE
	MG:	DONE
	MR:	DONE
	MX:	DONE
	TXT:	DONE
