An attempt was made to make the implementation of RBp very flexible,
modular, and robust. Thus, units keep track of their own sense of time
and record their own history of activations. This is done with a
CircBuffer object, which is derived from an Array type
(see section 8.3 Arrays). Values in this buffer can wrap-around, and can be
shifted without performing any memory copying. Thus, the unit
activation records can be quickly shifted after performing a
backpropagation enough to make room for new states to be recorded. The
trial process will shift the buffers just enough so that they will be
full again the next time a backpropagation will occur (i.e., they are
shifted by the bp_gap value converted into tick units).
However, the buffers are robust in the sense that if the bp_gap parameter is changed during processing, they will simply add new states and dynamically increase the size of the buffer if it is already full. Indeed, when a unit first starts processing, the buffer is automatically added to by the same mechanism--it is always full until some number of values have been shifted off the end.
The units only record their activation values in the buffers. Thus,
there is a StoreState function which takes a snapshot of the
current activation state. It is called at the end of the
Compute_Act function. During backpropagation, the
StepBack function is called, which will take one step back in
time. The activation state recorded in the prv_ version of the
unit variables are copied into the current variables, and the new
previous values are loaded from the buffers at the given tick.
The backpropagation loop looks as follows:
PerformBP(): {
InitForBP(); // clear current and prev error vals
int buf_sz = GetUnitBufSize(); // get unit buffer size (states in units)
for(i=buf_sz-2; i>=0; i--) { // loop backwards through unit states
Compute_dEdA_dEdNet(); // backpropagate based on current state
Compute_dWt(); // compute weight changes from error
if(i > 0) // if not all the way back yet
StepBack(i-1); // step back to previous time
}
RestoreState(buf_sz-1); // restore activations to end values
if(real_time)
ShiftBuffers(); // don't shift unless real time
}
Thus, error derivatives are computed on the current and prv_
activation state variables, and then these are shifted backwards one
step, and this continues for the entire length of stored activation
values. The above routine is called by the trial process whenever the
buffer size of the units is equal to or greater than the bp time window.
During backpropagation, the prv_dEdA and prv_dEdNet values
are kept, and are used to time-average the computations of these values,
much in the same way the activations or net inputs are time averaged
during the activation computation phase.
The following is a chart describing the flow of processing in the RBp algorithm, starting with the epoch process, since higher levels do not interact with the details of particular algorithms, and assuming sequences are being used:
SequenceEpoch: {
Init: { // at start of epoch
environment->InitEvents(); // init events (if dynamic)
event_list.Add() 0 to environment->GroupCount(); // get list of groups
if(order == PERMUTE) event_list.Permute(); // permute list if necessary
GetCurEvent(); // get pointer to current group
}
Loop (trial): { // loop over trials
SequenceProcess: { // sequence process (one sequence)
Init: { // at start of sequence
tick.max = cur_event_gp->EventCount(); // set max no of ticks
event_list.Add() 0 to tick.max; // get list of events from group
if(order == PERMUTE) event_list.Permute(); // permute if necessary
GetCurEvent(); // get pointer to current event
InitNetState() { // initialize net state at start
if(sequence_init == INIT_STATE) network->InitState();
}
}
Loop (tick): { // loop over ticks (sequence events)
RBpTrial: { // trial process (one event)
Init: { // at start of trial
cur_event = epoch_proc->cur_event; // get event from sequence
}
Loop (once): { // process this once per trial
network->InitExterns(); // init external input to units
cur_event->ApplyPatterns(network); // apply patterns from event
if(unit buffer size == 0) { // units were just reset, time starting
time = 0; // reset time
StoreState(); // store initial state at t = 0
}
Compute_Act(): { // compute acts (synchronous)
network->Compute_Net(); // first get net inputs
network->Compute_Act(); // then update acts based on nets
}
if(unit buffer size > time_win_ticks) // if act state buffers full
PerformBP(); // backpropagate through states
time += dt; // time has advanced..
}
}
if(wt_update == ON_LINE) network->UpdateWeights(); // after trial
}
}
if(wt_update == SMALL_BATCH) // end of sequence
network->UpdateWeights(); // update weights after sequence
GetCurEvent(); // get next event group
}
Final: // at end of epoch
if(wt_update == BATCH) network->UpdateWeights(); // batch mode updt
}