
xm-test README

Copyright (C) International Business Machines Corp., 2005
Author(s): Dan Smith <danms@us.ibm.com> 
  	   Woody Marvel <marvel@us.ibm.com>

Overview
========

This suite provides a framework for testing the Xen userspace tools.
The directory structure is:

  ./xm-test
   |
   +-/lib: Python support libraries
   |
   +-/ramdisk: Staging area for building the test ramdisk
   |
   +-/tests
   | |
   | +-/create: Tests for the 'xm create' command
   | +-/destroy: Tests for the 'xm destroy' command
   |  . . .
   |
   +-/utils: Utility scripts for ramdisk building

Reports are posted here:

   http://xmtest.dague.org


Building
========

Before the test suite can be used, the ramdisk must be built from
source.  All source needed for this process is automatically
downloaded, extracted, and compiled.  Due to the need to create
special files, this process must be done as root:

   # ./autogen
   # ./configure
   # make

NB: If you have the initrd.img from another installation of xm-test,
you can copy it into the ramdisk directory to eliminate the need to
rebuild it.  If you do this, there is no need to run 'make' again.
Simply copy the initrd-X.Y.img file into ramdisk/ and then run:

   # make existing

This will set up the link so that xm-test will use the existing
ramdisk.  Next, just run "runtest.sh" normally.  Note that in general,
you should not attempt to use a ramdisk from a previous minor version
of xm-test (i.e., don't use a ramdisk from 0.4.0 with 0.5.0.  0.5.0
should work for 0.5.3 though)

If you'd like to build and run this with hardware virtual machine assist
(HVM) support to test fully virtualized disk images on VMX hardware, 
please add the --enable-vmx-support option to configure:

  # ./autogen
  # ./configure --enable-vmx-support
  # make

The ramdisk/bin/create_disk_image script, which builds the full virt
disk.img, requires Lilo 22.7+ to be installed on the system. Lilo is 
used to install the bootloader on the disk.img.

If HVM / VMX support is enabled, the ramdisk/bin/create_disk_image script
will be run to create a full virt disk.img in the ramdisk directory. The
script, by default, will look in /boot for the first non-Xen kernel it
runs across. If you wish to use a different kernel or the script fails
to find a kernel, please run the script manually to make a disk.img 
using the -k option. Xm-test will look for disk.img in the ramdisk
directory when run by default.


Running
=======

To run the full test suite, do the following as root:

   # ./runtest.sh <logfile>

This will run all tests, as well as generate and submit a report at
the end.  All output files will begin with "<logfile>."  If you wish to
prevent submission of a report, add "-d" to the command line like this:

   # ./runtest.sh -d <logfile>

It may be useful to run tests without submission as above, and then
submit the report at a later time.  To do so, run runtest.sh with the
-s flag and the name of the previously-generated report:

   # ./runtest.sh -s <logfile>

For people needing a quick test run instead the full suite, a quick
mode has been added that will attempt to run a representative subset
of tests.  This is not a substitute for the whole suite, but will
verify that some of the major functions of xen and xm are working:

   # ./runtest.sh -q <logfile>

Because of the current structure of the reporting software, submission
of quick test run results is not supported.

It may be desirable to run a specific test group.  This can be
accomplished by doing the following:

   # cd tests/create
   # TEST_VERBOSE=1 make check

When developing or debugging a specific feature, a single test can be
run to avoid having to run even a whole test group:

   # cd tests/create
   # TEST_VERBOSE=1 make check TESTS=01_create_basic_pos.test

The runtest.sh script will create several files, including a .report
file, which is the cleaned up, email-friendly report of failures.
Additionally, the script will submit your results to the development
team for trend analysis.  This helps us determine the level of success
people "out there" are having with different versions of Xen.

Note: you should generally run xm-test with a minimum of memory
allocated to Dom0.  More memory available for allocation to DomUs
means a more rigorous test.

BIG FAT WARNING: The test framework assumes it is running on a
dedicated machine.  As such, the library automatically destroys any
running DomUs on the system to provide each test with a "clean slate".


Extending
=========

Additional tests may be added in existing groups to test additional
cases for a given xm subcommand.  Test programs should be named
according to the following scheme:
	
  XY_group_name_{pos,neg}.py

Where:
  XY        is the next number in line
  group     is the name of the subcommand being tested
  name      is the short name of the test
  {pos,neg} denotes whether this is a positive or negative test case

New subcommand groups should be added as directories named after the
subcommand itself.  The "Makefile.am.template" should be copied into
the new group directory as "Makefile.am".

See the Writing_Tests_HOWTO file for more detailed information on
adding tests to the suite.


Developer Notes
===============

Our library provides a DomU console abstraction for automated
execution of commands.  Please note that this is relatively fragile,
and is intended for use only with the ramdisk built by the framework.
Because the console experiences some occasional corruption, this
method is not completely perfect at the moment, although the authors
use it with relatively few problems.


Known Issues
============


Reporting Bugs
==============

If you find a bug in the test framework, report it to:

   Dan Smith <danms@us.ibm.com>

If you find a bug in a specific test case, contact the author of the
test case first.
