
OPeNDAP C API - User's Guide
1 March 2007
John Chamberlain <jchamberlain at opendap.org>

Support address: support at opendap.org

Introduction

The OPeNDAP C API (OCAPI) is an OPeNDAP DAP 2.0 client implementation written
in generic C. The goal of this implementation is to provide a C code base
that can be compiled easily on any platform without the complexities found in
C++. There are four main ways you can use the OCAPI:

Pre-Compiled Binary - You run a program from the command line, enter commands
and get text responses.

User-Compiled Binary - You compile the source code provided in the API
resource kit. You can then run the program from the command line, enter
commands and get a text responses.

Dynamic Library - The OCAPI shared library, a single file, is dynamically
linked with a program you have written, at run time. You must distribute the
library file with your program. The shared library allows you to access
OPeNDAP servers programmatically and retrieve live data structures your
program can use.

Static Library - The OCAPI code base is statically linked with a program you
have written when it is compiled. There is no separate file--the OPeNDAP
functionality is compiled directly into your program. You include the ocapi.h
header file from the OCAPI code base and can use the functions defined there
to access OPeNDAP servers programmatically and retrieve live data structures
your program can use.

Depending on which mode you want to use the procedures are very different.
Refer to appropriate section of this document for instructions on each mode.

<< At this point we should separate the UNIX and Win32 specific portions.
These instructions won't work for the UNIX builds (which include generic
UNIX, Linux and MacOS. The build instructions and README info needs to be
separated. Tom should work on the User's Guide. jhrg 1/25/05 >>

****************************************************************************
   Pre-Compiled Binary
****************************************************************************

The OCAPI binary is currently provided for three specific computing
architectures:

	- 32-bit Windows running on x86
	- Linux x86
	- MacOS X

If you want to use the OCAPI on a different platform than one of the above
you will need to compile it. See the section User-Compiled Binary below.

You can use a pre-compiled binary either from the command line or a
script/piped program. In either case the ocapi runs as a separate process.
You send it commands and it gives a text response. This is called the
"interactive" mode. There is a help command in this mode which details the
commands available.

****************************************************************************
   User-Compiled Binary
****************************************************************************

The OCAPI source files can be compiled on any system that can run libcurl,
its HTTP support library. The C code it contains is otherwise completely
generic. The easiest way to compile the OCAPI is to use the provided
makefile.

[autoconf instructions...]

If you are are on a Windows system and want to compile it for some reason you
will probably need to create a Unix sub-environment on your machine. The
reason for this is that even though the OCAPI itself can be easily compiled
using typical Windows tools like Visual C++, libcurl has a much more
complicated structure which is difficult to compile from scratch. There are
two main sub-environments to choose from, both of which can be freely
downloaded: MSys and Cygwin. MSys is a bare bones environment to which you can
add functionality selectively and Cygwin is an attempt to create a complete
Unix environment with all its attendant tools. Cygwin is a much larger and
complex install, so unless you want to create a Unix environment for actual
day-to-day work you are better off using MSys. These tools can be found at
the following websites: cygwin.com or mingw.org.

Once you have a Unix sub-environment running on your PC you can run the
autoconf scripts as above.

****************************************************************************
   Dynamic Library
****************************************************************************

You can include the ocapi as a shared library to be dynamically loaded by
your program or script. In windows this file is called "ocapi.dll" and in a
unix-based system "libocapi.so". Pre-compiled versions of this library are
available for Win32, x86 Linux and the Macintosh.

If you want to build the library yourself see the preceding section
("User-Compiled Binary"). When you use the provided makefile the library is
generated at the same time as the standalone executable.

In Windows you can see the contents of the library by viewing the info of the
dll or using utilities like impdef or dumppe. Under Unix/Linux you can use
the command "nm" to see all the functions and symbols in the library.

Refer to the section below "Using the OCAPI Programmatically" for information
on calling OCAPI functions from your program. There is also the doxygen
online help and ocapi header files, especially ocapi.h for instructions on
calling these functions.

****************************************************************************
   Static Library
****************************************************************************

To build the OCAPI into your program you can link it statically.
Include ocapi.h in your C code, and link to ocapi.o in the link stage.

An example of how to do this is in the directory "example_static". This directory
contains a basic program which loads the data from a URL and then prints it out
and does this by statically linking the ocapi library.

There is also a makefile in the same directory which builds the static linking
example. You may want to refer to this makefile to see the dependencies for
doing a static build.

****************************************************************************
   Using the OCAPI Programmatically
****************************************************************************

General Instructions:

- To use the OCAPI programmatically you should be familiar with the C language.
- include ocapi.h in your source code
- refer to example_static directory for examples
- dynamic libraries for cURL and pthreads need to be present on your system

When using the library it should be initialized once using the
zInitializeOCAPI() function. This will initialize the communications
library libCURL which the OCAPI depends on.

You may then load a URL using one of the load functions. The URL
should point to a valid DataDDS, DDS, or DAS which is on the web.

The load function will return an OPeNDAPStructure structure which
contains the root node of the tree the parser creates. The
DDS/DataDDS/DAS is represented in memory by a tree of OPeNDAPNode
structures which are nested. Each node has a reference to its parent
and a list of references to its sub nodes. By traveling up and down
the tree you can reach any node in the dataset.

If the structure is a DDS then none of the nodes will have content, ie
the nodeContent members will always be NULL. If the structure is a
DataDDS some of the nodes will have nodeContent members. Some nodes,
for example structure nodes, might not have content.

You can access the structures directly yourself or use the accessors
provided in the library for convenience.
		
****************************************************************************
   Using the OCAPI Programmatically - Memory Management
****************************************************************************

The OCAPI allocates memory for its returned values on the heap. It is
imperative that you strictly follow correct memory management policies.
Otherwise your program will have memory leaks in it.

Whenever a parameter to an OCAPI call is a handle (a pointer to a pointer)
there is potentially a value that is being returned to your code. This value
will be in memory allocated on the heap. When you are done using the value
your program must free the handle. To do this in a consistent manner follow
these guidelines:

* If you supply a NULL handle no memory will be allocated by the OCAPI. In
  cases where the handle is required the function will return an error. In
  cases where the handle is optional, such as for an error message, the
  function will continue to operate.

* If you supply a non-NULL handle, memory may or may not be allocated
  regardless of the success or failure of the function. Therefore if you
  supply a handle you must always check to see if it points to a non-NULL
  value when the function returns and if it is non-NULL you must free it when
  you are done with it. Make no assumptions that the handle is empty. If you
  supply a handle you must always check it and free it if there is a value.

* If you may finish your use of an OCAPI value in a function with multiple
  exit points you should make a practice of using finalization code to free
  the value. If you are programming in C++ you can use the 'finally'
  construct to do this. If you are programming in C you should have the
  finalization code labeled at the end of the function and should use a goto
  statement to jump to this code when you are ready to exit. For example,

int myFunction(){
	char** hError;
	[...use OCAPI to get data_structure]
	switch( x ){
		case 1:
			[...processing...]
			goto RETURN_SUCCESS;
		case 2:
			[...processing...]
			goto RETURN_FAILURE;
		case 3:
			[...more code]
	}
	[...more code]

	/* finalization zone below */
	int iReturnValue;
RETURN_SUCCESS:
	iReturnValue = 0;
	goto CLEANUP;
RETURN_FAILURE:
	iReturnValue = 1;
	goto CLEANUP;

CLEANUP:
	if( !structure_Free( data_structure, hError ) ){
		printf( "error freeing data structure, possible memory leak: %s", *hError ) /* report error */
	}
	if( *hError != NULL ) free( *hError ); /* always check if need to free */

	return iReturnValue; /* your function has a solitary exit point */

}

* When you are working with OCAPI structure data, the entire structure tree
  must be freed. To do this use the structure_Free function as in the example
  above.

*Anytime Ocapi has an allocation call to malloc there must be a 'free' associated with that specific allocation.
 If there is not memory leaks will occur.  If you see any such occurrence please notify us by sending an email to
 support@opendap.org. 

****************************************************************************
   Using the OCAPI Programmatically - Building
****************************************************************************

Linux and Mac builds can be done with Autoconf and there are files for using autoconf
present in the source directories. For Windows builds there are two example makefiles
both of which should be fully functional. In the Win32 directory is a document 
called "build_instructions.txt". This document has step by step exact instructions for
building on Windows and is a good starting point for most users. In particular you 
should follow that document to make sure you have cURL and pthreads on your system.
The two make files are "makefile" in the win32 directory and makefile.w32 in the
example_static directory. The first builds the whole system standalone including 
the interactive component. The second is an example program showing a minimal 
case of an EXE that used the OCAPI as library functionality.

****************************************************************************
   Using the OCAPI Programmatically - Retrieving Data
****************************************************************************

The first step in getting data is to retrieve the dataset. This is typically done
with code such as the following:

	URL* pURL = NULL;
	char* sError = NULL; /* handle to the error string */
	ThreadSpecificStorage* pTSS; /* thread-specific data for thread safety */
	OPeNDAPStructure* pData; /* the data (or dds or das) we get will be in here if successful */
	if( zInitializeOCAPI( &pTSS, &sError ) == FAILURE ){ /* init and get thread object */
		fprintf( stderr, "failed to initialize OCAPI: %s\n", sError );
		return -1;
	}
	pURL = malloc( sizeof(URL) );
	pURL->sCanonicalURL = "http://test.opendap.org:8080/dods/dts/b31.dods";
	if( structure_Load_URL( pURL, &pData, pTSS, &sError ) == FAILURE ){
		... error handling code here
	}
	... rest of program
	vCleanupOCAPI( pTSS );

If successful this will create an OPeNDAPStructure structure and return a pointer to it.
If unsuccessful this pointer will be NULL and FAILURE will be returned.

Note that if an error is generated then the OCAPI has allocated the memory for the
error message and it is your responsibility to free that memory after you are done 
using the error message (sError in the example above).

For complete examples of how to use the OCAPI programmatically see the Example_Static program in the project.

****************************************************************************
   Using the OCAPI Programmatically - What is an OPeNDAP Structure?
****************************************************************************

When you load any retrievable OPeNDAP object from the internet (or a file) it will be 
stored in an OPeNDAPStructure which is a generic container capable of holding a 
variety of different OPeNDAP structure types including DDS, DataDDS and DAS.

The OPeNDAPStructure object has three fields:

	eSTRUCTURE_TYPE   eStructureType;
	int               iEstimatedSize;
	OPeNDAPNode*      nodeRoot;

The structure type field tells you whether the object is a DDS,
DataDDS or something else. This is determined, by the way, by the
content of the stream--not by the suffix on the URL you passed to the
load function. The estimated size field may or may not be filled. If
it is zero that means the size is unknown. The root is the root of the
tree representing the object. All OPeNDAP objects have a tree-like
shape.

All the nodes in the tree are of the same type, ie, OPeNDAPNode,
regardless of whether it is DDS, DataDDS, DAS, etc. The way the nodes
and there content is differentiated is by the eType field inside the
node (see the next topic "Traversing an OPeNDAP Tree").

If the structure is a DDS it will have all the structural information
but be lacking the data. A DataDDS will have both the structural
information and the data as well.

For example, if your data has a string in it the node that represents
that string will have a pointer to a NodeContent_String structure in
its nodeContent member. This structure will have length 0 and sContent
of NULL if you have retrieve a DDS because the DDS just has the
structural values. The data values are missing so even though the
content structure is there it will be devoid of the string itself.

In the case of an Array there will be a NodeContent_Array structure in
the node's content member. If the object was a DataDDS then all the
fields in this structure will be filled in, but if it was a DDS then
the "array" field will be NULL. In this case you can still tell what
the dimensions are of the array by examining the
iDimensionLengthArray1 and sDimensionName1 fields.

****************************************************************************
   Using the OCAPI Programmatically - Traversing an OPeNDAP Tree
****************************************************************************

The nodes representing the fields in an OPeNDAP object are arranged in
a tree. To access the nodes you need to traverse down the tree. The
fields in every node are as follows:

	eNODE_TYPE       eType;
	struct node     *nodeSuper;
	struct node    **nodeSub;
	int              iSubNodeCount;
	void*            nodeContent;
	char*            sName;
	long int         nEstimatedSize;
	AttributeTable*  attributes;

The eType tells you what kind of node it is, array, grid, int,
structure, etc. The nodeSuper field tells you the super node or
"parent" node of the current one. The nodeSub is a list of the node's
sub nodes or "child" nodes. This list is zero-based. So if you wanted
a pointer to the first child node of node "x" where x is a pointer to
a node you would write:

	OPeNDAPNode* pFirstChild = x->nodeSub;

The third sub node would be:

	OPeNDAPNode* pThirdChild = x->nodeSub + 2;

The last sub node would be:

	OPeNDAPNode* pLastChild = x->nodeSub + x->SubNodeCount - 1;

You can obtain any parent node by using the nodeSuper field. For example,

	pThirdChild->nodeSuper

would refer back to x.

****************************************************************************
   Using the OCAPI Programmatically - Accessing Sequences
****************************************************************************

Sequence data is tricky because in the case of nested sequences the data and 
structure are in parallel trees. This is because the structural data is a 
template. For example:

Dataset MyData
	int32 "grumpy" - has content (pointer to an int)
	float64 "dopey"	- has content  (pointer to a float)
	Sequence "Happy" - has data content for WHOLE sequence
		int32 "sneezy" - no data content
		string "sleepy" - no data content
		Sequence "Doc" - no data content
			string "apple" - no data content
			int32 "pear" - no data content
			float32 "orange" - no data content

In this example all the data for the sequence Happy is in the node of
the topmost sequence. The reason for this is that there is only one
template node for sequence Doc but there might be hundreds of
different sets of data for doc, ie, one for each row in Happy. To get
Doc's data you have to first retrieve the relevant row from Happy and
then get the data for Doc from that row.

There are some accessor functions in the library to help easy access
to sequence data and avoid pointer arithematic.

If you want to access the data manually without using accessors there
is detailed information in the header file and DOxygen documentation
on doing this. The key thing to remember is that all data is stored in
columns by field and that primitive data is stored right in the
column, whereas in the case of non-primitive data, like an array or
string in a sequence the column stores pointers to the non-primitive
content.

For example, imagine you have a sequence that has a DFLOAT64 field in
it. First you would retrieve the field column from the aFieldValue
member:

	NodeContent_Sequence* pSequenceContent = ... [code to get content]
	int xField = 3; /* we want, say, field 3 */
	DFLOAT64* pFloat64Column = (DFLOAT64*)pSequenceContent->aFieldValue + xField - 1;

This column of data will have a number of rows each containing the
DFLOAT64. To print them out you might use code like this:

	int xRow;
	int ctRows = pSequenceContent->ctRows;
	for( xRow = 1; xRow <= ctRows; xRow++ ){
		printf( "row %u has double value %f\n", xRow, *(pFloat64Column + xRow - 1) );
	}

Notice that since the field array has been cast to a type that is
64-bits wide that the pointer arithematic will be correct when you go
through the rows in the print loop, ie each row will skip by 8 bytes
as you go along.





