February 04, 2003

MPEG4IP Project
===============

The MPEG4IP project provides a standards-based system for encoding, 
streaming, and playing MPEG-4 encoded audio and video. To achieve 
this we've integrated a number of existing open source packages, and 
also created some original code to fill in the gaps.

Please note this project is intended for developers who are interested
in MPEG-4 audio and video, and Internet streaming. It is not intended
for end-users. Please read all the legal information in the file "COPYING"!

Also note that the primary development focus of this project is the 
Linux platform. If you're going to use the package on other platforms,
especially non-UNIX platforms, you'll probably have some work to do.

Please use the SourceForge site to report problems, suggest enhancements,
ask questions, etc. The URL is http://www.sourceforge.net/projects/mpeg4ip

There is also a project web site at http://www.mpeg4ip.net/ that has some
general information on MPEG4IP.   

We also have a guide to MPEG4IP donated by everwicked.  See
doc/MPEG4IP_Guide.pdf.

Overview
========

There are two ways to use MPEG4IP to create content:

The older method assumes that you've somehow managed to capture raw audio
and/or video into a file. That's the starting point from which you can use
the MPEG4IP encoding tools to create an MP4 file. The simplest method being 
to use the 'mp4encode' script. Detailed instructions for this script and the
individual tools that it uses are in doc/encoding/encoding.htm  

The newer method is an integrated live encoding tool called mp4live.  This 
program is designed to make it easy to create MP4 files or transmit live
audio/video streams over the network. It can even do both things 
simultaneously!  The key requirement to use this tool is to have a video 
capture device and a Video for Linux (v4l) driver for it. So far we've 
tested with the bttv driver for Brooktree based video capture cards, and 
the qce driver for Logitech QuickCam Express USB webcams. Both of these 
solutions can be acquired for about $50 US! Please see the file 
mpeg4ip/server/live/README for more information about mp4live.

Once an MP4 file is prepared, it can be placed in the content 
directory of a streaming server. We typically use Apple's Darwin Streaming 
Server, but any server that understands MP4 files (or hinted Quicktime 
files) can be used. 

When the content is encoded and available on the server, you can run
the player. Start 'gmp4player' and then enter the RTSP URL to the server 
and the content, (command line works too). E.g.  
    $ gmp4player rtsp://myserver.mydomain.com/mycontent.mp4

Although we're focussed on streaming, the player will also playback from
a local file. E.g
    $ gmp4player mycontent.mp4

That's not all! The player is not limited to local playback of MP4 files.
It can also read AVI, CMP, DIVX, AAC, MP3, and WAV. This is useful for 
debugging since the encoded data can be check independently of the MP4 
file container, and known good content, such as your favorite MP3, can be 
used to verify that the player is working correctly with your hardware. E.g.
    $ gmp4player mymusic.mp3

And that's still not all... You can set up your player to run a playlist
by creating a simple text file with the extension .mp4plist, with
each item (file or stream) on it's own line.

Note: If you prefer a no UI version of the player, or your system does not
include GTK or GLIB, just the bare video window, 'mp4player' is available 
to fit that need.


Legal
=====

Please see the file "COPYING".


Building and Installing
=======================

We are now (as of release 0.9.5) using make dist for our tarball.  To
build, you should only have to issue the following:
   ./bootstrap <params for configure>
   make
   make install (needs root privileges).

We no longer include lame in our distribution; however, we require
lame version 3.92 or later for mp4live.  This may change at some point, 
but for now, it is required.  If you wish to make without mp4live, you 
can give the --disable-mp4live or --disable-server arguments.  Neither
the bootstrap or cvs_bootstrap scripts will allow you to build without
lame installed.

If you don't have root privileges, but still wish to install the 
distribution to a directory to which you do have write permission,
then here is an example of how to do that:

   mkdir -p $HOME/local/bin $HOME/local/lib
   ./bootstrap --prefix=$HOME/local
   make
   make install

If this process works for you, you can skip down to the next section on 
Configure Script Options.  If this does not work, or you have downloaded 
from CVS, the below applies.

We've built the distribution using GNU autoconf, automake and libtool.
We have attempted to follow the GNU conventions for open source packages. 
This is complicated by the fact that we build on many other packages. Where 
the package was already using the GNU tools, we left things alone. Where 
the package was using it's own Makefile, we left things alone if it was a 
complicated Makefile. If it was straightforward we replaced it with an 
equivalent automake file.

If you have to install any of these tools (for example, automake is
not included with Mac OS X), find out where the others are installed
(which autoconf). If the path does not start with /usr/local, use the
"configure --prefix=<path before /bin on other packages>"

For example, if autoconf is in /usr/bin, use the "configure --prefix=/usr"
command when installing automake or libtools.

Note: libtool is frequently not installed on Linux systems (autoconf, 
and automake generally are). You can download libtool from a GNU mirror
site, or http://download.sourceforge.net/mpegip/libtool-1.3.5.tar.gz

In order to compile mpeg4ip from CVS, we require the following tools:
gcc 3.2 or greater
libtool 1.4.3
autoconf 2.53
automake 1.6.

If you don't have these tools and are trying to compile from CVS, don't
complain.

Be sure to read the OS specific section later before continuing here.

In general, the code should be portable, but as someone once said 
"There's no such thing as portable code, just code that has been ported." 
When you find problems please be sure to use the SourceForge site to tell 
us what you encountered, and hopefully how you fixed it.

To build:
    ./cvs_bootstrap <arguments to pass to configure scripts>
    make
    make install (optional, typically need root privileges)

The bootstrap script will pass any arguments to the configure scripts.

For the curious, the bootstrap script invokes the configure scripts of 
the included packages that have them, and then our own top level configure 
script is generated and run. At the end of this process all the Makefile's
are ready, and setup in the correct hierarchy. If this doesn't work for you,
you're free to hack as needed ;-)  

Configure Script Options
========================

Two options of potential interest are "--disable-server" and "--disable-player"
which disable the building of the server and player respectively. By
default both server and player are built.

If you are building on a system with an Intel x86 CPU clone, you may
need to specify the configuration option "--enable-mmx=no". The
configuration script automatically detects an x86 target CPU and 
enables MMX assembly code in the build (if the NASM assembler is available).
If your CPU doesn't support MMX instructions you will want to disable 
this feature.  The configure script will also check for the minimum
version of nasm supported; we require 0.98.19 or greater.

To build with IPv6 support, use the --enable-ipv6 command option.

Darwin Quicktime Streaming Server
=================================

Please note that the Apple Darwin Quicktime Streaming Server is NOT distributed
with mpeg4ip. It can be downloaded from Apple, http://www.apple.com/ as either
source or pre-built binaries. For those who choose the source option, below is
the express version of the build and install process for the Darwin Streaming
Server:

To build the Darwin Streaming Server:

    cd DSS4
    ./Buildit

To install the Darwin Streaming Server from the build:
    cd DSS4
    mkdir Dist
    ./DSS_MakeRoot -f Dist
    cd Dist
    ./Install    (need root privileges)

See the documentation that accompanies the server on how to configure it
for your environment. 

Note the default content directory is /usr/local/movies.

There are some sample mp4 files available on the mpeg4ip SourceForge 
download area. Also Envivio, http://www.envivio.com/, has some sample mp4 
files. We suggest first downloading one of these samples and try opening 
the file with gmp4player. If that works, then try copying the file to the 
streaming server's content directory (e.g. /usr/local/movies), and enter 
the appropriate RTSP URL in gmp4player.


OS Supported
============

Currently, we have compiled and tested on the following platforms:
linux, freeBSD, BSD/OS, Solaris and windows.  For all varieties
of *nux, X11 is required.

To date we've built on Red Hat Linux 6.1, 6.2, 7.0, 7.1, and 7.2 with
the native compilers, on 7.1 and 7.2 with gcc 3.0, and on 7.3 with gcc 3.1.  

Windows
-------

For windows, Visual Studio 6.0 projects are included.  You will need
to install nasm in the VC98/bin directory before compiling.  You should
get nasm-0.98.22-win32.zip from the nasm web site.  You will need to 
rename nasmw.exe to nasm.exe.  We recommend getting Service Pack 5.

Use the encoding60.dsw project for encoding tools, and the 
player/src/player60.dsw for the player.

We recommend installing DirectX 8.1 or later.  If that is not possible, 
and you have problems with video, try uncommenting out #define OLD_SURFACE
in player/src/video.cpp. Other than that, you'll have to figure it
out yourself - libsdl.org is a good resource.

To run mp4player other than in Visual Studio, install both mp4player.exe 
and SDL.DLL into a directory on your Window's path (install them into
the same directory).   You will also need to install all the plugin
.dlls into the same directory as well.  Look for the *_plugin projects
in player60.dsw.

In encoding60.dsw and player60.dsw, if you do a batch build, you will
see several projects that do not build correctly - these are Lame
with GTK, and common with IPv6.  This is as designed.  If you want
these to build, please see the appropriate package creator for the
correct steps - they are not supported in mpeg4ip.

For windows GUI player, see below section.

Mac OSX
-------
We haven't been able to build mpeg4ip on a MAC for quite some
time.  We just don't understand the toolchains, and it's not a
high priority for us.  One of our forum member (yakima) sent us
the following steps:

- updated automake -> 1.6.3
- installed DLCompat (dlfcn.h etc., see above)
- bootstrap w/ --disable-shared --disable-mp4live
- make w/ LDFLAGS='-flat_namespace -undefined warning' 

Version 0.9.7.2 should have the other changes required to get this
to compile.  He didn't have any luck with the player plugins, and
we hope to get that done before the next release.

This makes the below accurate again.
/*
When building on Mac OSX, you may get an error that libtoolize is
not installed correctly.  Mac OSX comes with libtool installed as
glibtool.  Automake (which doesn't come included with OSX) does not
correct its scripts.

To fix this problem, do a link of libtoolize to glibtoolize (ie:
ln -s libtoolize glibtoolize where ever glibtoolize is installed).

You should also use the --disable-shared command when issuing the
bootstrap shell.  This should be done for you if you use the project
builder project.  

We've tested the latest build on OSX, and have some strange results
at times.  Your mileage may vary.
*/

Slackware
---------
When building on Slackware-8.0, you need to consider the following 
(from maersk):
  The Slackware distribution does not as standard come with shared 
  libraries for libXv and libXxf86dga. In other cases, where you 
  have upgraded from xfree86 version 4.0 to 4.1, you may not have 
  compiled the shared versions. The fix to this is in general is this:

  # pushd /usr/X11R6/lib
  # rm -f libXv.so libXxf86dga.so
  # ld --whole-archive -shared -o \
      libXv.so libXv.a
  # ld --whole-archive -shared -o \
      libXxf86dga.so libXxf86dga.a
  # popd

  Note that you need to be root to do this.

Solaris
-------
When building on Solaris, libtool and gnu make must be installed and
used.  If libtool is installed, you make get a warning message that
common libraries made with gcc less than 3.0 might have problems, 
ignore it, but don't take binary libraries from any other machines.

General UNIX
------------
If you have built a previous version of mpeg4ip, do a make uninstall first,
or go in and remove libsndfile from your shared libraries directory, unless
you have another version installed.

Executables
===========

If you ran 'make install' with the defaults, then all the MPEG4IP executables
will end up in /usr/local/bin. 

For encoding tools, you get:

mp4live     Integrated, live encoding to file or network - Linux only

mp4encode   Front-end script to the following encoding tools:

avi2raw     Extracts raw audio/video tracks from an AVI file
lboxcrop    Vertically crops raw video to a new aspect ratio
faac        Encodes raw audio into MPEG-4 AAC encoded audio
mp4venc     Encodes raw video into MPEG-4 encoded video using ISO codec
mp4creator  Creates and hints audio/video tracks to an mp4 file
xvidenc     Encodes raw video into MPEG-4 encoded video using the Xvid codec

These are described in more detail in doc/encoding/encoding.htm

A few debugging tools are also included:

mp4extract  Utility to extract tracks from an MP4 file
mp4dump     Utility to dump MP4 file meta-information in text form
mp4info     Utility to display MP4 file summary
avidump     Utility to display AVI file summary
yuvdump     Utility to display a raw video file on the screen

For playback, you get:

gmp4player  Simple graphical interface player
mp4player   Bare video window with sync'ed audio

If you installed the Darwin Streaming Server, those executables will end
up in /usr/local/sbin.

DarwinStreamingServer    Provides streaming service for MP4 files
PlaylistBroadcaster      Provides multicast playlist service for MP4 files

Executable Notes
================

FAAC and FAAD
=============
In our desire to make the package smaller, we have removed SNDFILE.
This has the effect of allowing FAAC to only encode raw PCM files,
unless bootstrap detects that SNDFILE is installed.  We do not have
the complete version of FAAD, so the faad standalone decoder will not
build.

If the existing translation behaviour of FAAC is required, please
obtain SNDFILE and install it, then re-run bootstrap and make for
*nux base platforms.  For windows based platforms, obtain a fresh
copy of faac and use it instead of the one in our package.

An even better idea would be to obtain the complete faac/faad package
from www.audiocoding.com.

Directory Structure
===================

If you're going to start hacking, a map of the territory may prove useful:

mpeg4ip - top level project directory

    config - autoconf files

    doc - the minimal doc we've written so far
        encoding - how to encode contenet
        ietf - copies of the relevant RFC's
        mcast - how to multicast 
		mp4v2 - man pages for mp4v2 library
		programs - man pages for core programs

    include - project wide includes 

    common - shared code
        video 
	    libmpeg32 - mpeg1/2 encoder/decoder
            mpeg4 - ISO MPEG-4 video encoder/decoder

    lib - project wide libraries
        SDL - Simple DirectMedia Layer
        avi - AVI file format
        bitstream - MPEG style low level bitstream utlity
        config_file - Configuration file utility
        gnu
            getopt - gnu getopt routines
        mp4 - MP4 (aka MOV/Quicktime) file format library
        mp4v2 - new MP4 library written from scratch
            test - contains some test programs
            util - contains new mp4dump and mp4extract utilities
	mpeg2t - mpeg2 transport stream utilities
        msg_queue - SDL based Inter-thread messages utlity
        rtp - UCL RTP 
        sdp - Our own SDP
        win32 - libary files need for MS Windows
	xvid - xvid video encoder/decoder

    player - player specific code
        lib - libraries specifically for the player
            audio
                faad - FAAD AAC decoder
                mp3 - MP3 decoder
            libhttp - Our own http client
            rtsp - Our own RTSP client 
        plugin - home of player plugins
            audio - audio plugins
               raw - raw audio plugin
	    rtp - rtp bytestream plugins
	       h261 - h261 rtp plugin
               isma_audio - isma audio format plugin
            video - video plugins
               raw - raw video plugin
               mpeg3 - mpeg1/2 video plugin
 	       xvid - xvid video plugin
               h261 - h261 decoder
        src - the player executable
            codec - 
                aa - aac plugin
                mp3 - mp3 plugin
                mpeg4 - mpeg4 ISO decoder plugin
                wav - wav plugin
            osx - Mac OS X UI
	    win_common - common windows code
            win_client - windows client
            win_gui - windows gui code.

    server - server specific code
        audio
            faac - AAC encoder program
        mp4creator - create and hint A/V tracks to an mp4 file
        mp4live - mp4 live encoding interface
            gui - gtk gui for mp4 live.
        util
            avi2raw - extract raw A/V data from AVI files
            avidump - dump AVI meta information.
            lboxcrop - vertically crop raw video 
            mp4encode - front-end script to simplify encoding process
            rgb2yuv - rgb to yuv converter
            xvidenc - command line interface to Xvid MPEG-4 encoder
        video
			H26L - ITU H.26L TML 9.4 reference video encoder (EXPERIMENTAL)

    util - generally useful utilities
	iptv - read Cisco IP/TV programs from a content manager
        yuv - simple tools for examining raw video


Standards Compliance
====================

We're not only supporters of open source, we're supporters of open standards!
We've attempted to use the publically defined standards as much as possible.
Here's what we believe we are following. If you find something non-compliant,
please let us know. We certainly will want to fix it.

Here are the citations:

ISO/IEC 14496-1:2001 MPEG-4 Systems (includes MP4 file format)
ISO/IEC 14496-2:2000 MPEG-4 Video
ISO/IEC 14496-3:1999 MPEG-4 Audio (includes AAC)
ISO/IEC 11172-3:1993 MPEG-1 Audio (includes MP3)
ISO/IEC 13818-3:1998 MPEG-2 Audio (includes extensions to MP3)
ISO/IEC 13818-7:1997 MPEG-2 AAC

The ISO/IEC documents must be purchased from either ISO (www.iso.ch)
or one of the national bodies. In the US, ANSI is the representative
body, and provides an online store under www.ansi.org


IETF RFC 1889 & 1890 RTP

IETF RFC 2326 RTSP

IETF RFC 2327 SDP

IETF RFC 2250 RTP Payload for MPEG-1/2
    Note: we're just using the audio part for MP3 (and video reception
    for the player).

IETF RFC 3119 A More Loss-Tolerant RTP Payload Format for MP3 Audio

IETF RFC 3016 RTP Payload for MPEG-4 Audio/Visual
    Note: we're implementing the MPEG-4 video part

IETF draft-ietf-avt-mpeg4-simple-06.txt - work in progress
    Note: we're implementing the AAC audio part, and for the next
    release will finalize implementation.

The IETF RFCs can be found in mpeg4ip/doc/ietf


Note the MP4 file format is derived from Apple's QuickTime file format.
That specification is:

Apple Computer QuickTime File Format, June 28 2000
http://developer.apple.com/quicktime/


We're also involved in the Internet Streaming Media Alliance (ISMA) 
which seeks to standardize the protocols and formats used for streaming.
We believe MPEG4IP is interoperable with that organization's 1.0 Technical 
Specification. For more information see http://www.isma.tv

Latency in mpeg4ip
==================

There have been enough questions to merit a place in the README about
the latency in mpeg4ip.

First of all, mpeg4ip is not designed for video conferencing or real
time display of data.  Most streaming products are not (look at Real
or Quicktime's buffering schemes - QT buffers 3.0 seconds of data, 
while Real can buffer up to 30-40 seconds).

When you need to look at latency in a streaming environment, you need
to look at each potential piece.  In the simplest case (mp4live to 
mp4player), there are 5 potential places where latency can take place - 
at mp4live, in the kernel IP transmit stack, in the network, in the
kernel IP receive stack and in the player.

Mp4live has very little latency.  In video, it tends to be 1 frame.  In
audio, it tends to be 3-4 audio frames (if using AAC, frames tend to
be 1024 samples.  MP3 tends to be 1152 samples, but can change based on
sampling frequency and bit rate.  Samples can be converted to seconds by
dividing by the sample rate).

The kernal IP stacks can have some latency built in, as well.  Probably
not too much, but you should be aware that it can exist (look at a sniffer
trace of DSS output and you'll see what I mean).

Network latency can occur, as well.  Collisions, etc, can happen, especially
if server and player are not on the same network.

Finally, the player has latency as well.  For streaming, we tend to buffer
2 seconds for each stream.  This is changeable by setting the 
RtpBufferTimeMsec variable in the .gmp4player_rc file (for Windows users, 
you'll have to change the registry to do so).  The value is in milliseconds, 
and you need to set it to a non-zero value (0 indicates the default value).

Player Information
==================
See the new README in player/src.

Known Problems
==============
* There have been some changes to the MPEG4 AAC DTS headers in the
  latest version of the MPEG4 specification.  This renders previous
  .aac files pretty much obsolete.  We have NOT made this change to 
  mpeg4ip - we will make it in the next release.

* If you're playing through a NAT box, you may have to specify the
  default client IP ports.  Use the command RtpIpPortMin=<port> and
  RtpIpPortMax=<port> in the .gmp4player_rc file created in your home
  directory.  The IETF recommends a range of 6970 to 6999.

* The ISO MPEG-4 video codec bitrate control feature is broken. We've
  heard that a fix is available, so we're trying to track that down.

* The player for windows is sketchy at best.  We have included project
  files that should build for Dev Studio 6.0.  We recommend installing 
  DirectX 8.1 or later.  If that is not possible, and you have problems 
  with video, try uncommenting out #define OLD_SURFACE in player/src/video.cpp.

  If you have problems with choppy playback, and you're using Windows 98, 
  forget about it.  The timer tick time is too slow (55 msec) for us to
  use effectively.  Try Quicktime or Real with Envivio, or update to a
  newer Windows OS.

* Mac OSX player sometimes will lock up, or stop playing video while
  continuing the audio - we need someone to help us add the sound buffer 
  delay to get really good audio/video sync.

* FreeBSD based OS's have a problem with thread delays.  This can 
  cause the player to skip rendering many frames.  If you have this
  problem update to the latest version of FreeBSD - the problem still
  can occur, but is reduced. 

  This is due to an error in the thread scheduling code that causes a problem
  with a delay of less than the thread scheduler quantum (200 msec in
  some versions, 20 in others).  Since the average delay used is 9
  to 10 msec (less as we get closer to the video rendering time), this
  can have a great effect on video playback.
  
  You can get around this error by rebuilding your libc, after changing the
  THREAD_SCHED_USECS to 20000 (or lower) from 200000 in thread_private.h.

* If you're running on Linux, and trying to play a raw audio file, and 
  notice that you get garbage, try setting the LimitAudioSdlBuffer config
  variable to 1 in the .gmp4player_rc file.  This seems to occur on a 
  Soundblaster Live, Red Hat 7.3 machine.

* As the RTP standard (or new draft) suggests, the round trip delay can
  be approximated at the sender side by subtracting the intercepted time 
  of a RTCP SR and the delay at the client side from the intercepted time 
  of the corresponding RTCP RR. That is:
         RRT = Time_RR - Time_SR - Time_delay_at_client.

  However, as the Windows run-time library only provides up to 1 millisecond 
  resolution (0.001 sec), such measurement on Windows machines on a LAN may 
  be very inaccurate.

  A typical RRT between two hosts on a LAN is around 0.5 millisecond.  
  Measurements between Linux/Unix boxes have no such a problem. 

* It appears that mpeg audio created and hinted with our mp4creator do not
  work correctly with Quicktime when streaming.  This is most likely because 
  we use a dynamic RTP payload number and a RTP TS frequency that matches 
  the sampling frequency.  QT seems to only like an RTP payload of 14 and 
  a RTP TS frequency of 90000. (This is with QT 6.0).  This will work with
  mpeg 1/2 files, or if you use -timescale=90000 when using mp4creator with
  mp3 files.  If you're using mp4live to stream audio, use the 
  rtpUseMp4RtpPayload14 config file option.

  Neither QT or Real appear to understand mpeg1/2 or mp3 content in a .mp4
  container file for local playback.
=== END OF README ===
