===============================
 Coding style and instructions
===============================

This file contains information on coding conventions, style and implementation.
If you plan to contribute, read this before making changes. It is assumed that
reader of this file understands related terminology, for reference you might
want to see `HFST glossary`_.

On coding style
---------------

Implementation language is, currently, C++. Most of the programming style
follows `C++ Programming style guidelines
<http://geosoft.no/development/cppstyle.html>`_ (version 4.5). Personal
deviations:

* Prefer tabs over spaces in beginning of line

* New page characters (^L) may be used to separate logical sections
  (prefer splitting files though; a single C++ file should not contain two
  separate logical secitions)

For parts that are more like C programming language, most of the rules apply.
Except naming, where more traditional C style, underscores instead of
camel case, is used. And yes, C must be used to interface with flex and bison,
there is no clean and portable way around it.

On stability
------------

This software uses all the typical autotools & GNU tricks, so follow the
usual practices. Do not commit anything that won’t pass ``make``. Do not
distribute any alphas that do not pass ``make check``. Do not distribute
any betas that won’t pass ``make distcheck``. Do not distribute release versions
that have not had large-scale end-user testings, if in doubt, mark them at least
release candidates.

Specifics: handling of ``0`` in lexc parsing
--------------------------------------------

Since this is a thing that I've re-implemented at least thrice during HFST lexc
development, you might want to read this before wondering why it is such a mess
currently. The problem: in lexc, ``0`` in Multichar_Symbols section means actual
ASCII 0 and does not require (but may use) escaping, i.e. both ``A0`` and
``A%0`` are same multicharacter symbols in Multichar_Symbols declarations. In
lexicon strings, ``0`` is parsed as epsilon, except for ``%0``, which is parsed
as ASCII 0, **and** if it can form multicharacter sequence, e.g. ``blaaA0``
does not contain epsilon if a multicharacter sequence that can match to 0 is
declared in Multichar_Symbols.

Now, why does this make a problem, you say? This user-friendliness of ambiguous
parsing that is dependent on set of multicharacter sequences declared previously
prevents us from parsing %-escaped characters by merely stripping %'s, as it
does prevent parsing epsilon-zeroes by merely replacing %-escaped zeros with
zeros and zeros with some epsilon markers, both will cause some
isinterpretations. The only way to get correct results is to do full-scale
tokenization of string by some greedy left-to-right longest match parser.

Example::

  Multichar_Symbols A0
  
  LEXICON Root
  A0    #   ;
  A%0   #   ;
  %%0    #   ;
  %%%0   #   ;

If you strip_percents() on this this, you end up with::

  A0 # ;
  A0 # ;
  %0 # ;
  %0 # ;

which is certainly not what we want. If you don't, you must at least inject %0
to alphabet and recode on the fly (%0 -> 0, 0 -> eps)::

  Aeps # ;
  A0 # ;
  %%eps # ;
  %%0 # ;

Which would be nice except for Xerox lexc decides to be helpful and allow A0 to
be as is on the string and furthermore A%0 not as A0... So we cannot rewrite the
0 as we need to detect it as part of the multicharacter sequences, we may
rewrite %0 and then do strip_percents::

  A0 # ;
  A@ZERO@ # ;
  %0 # ;
  %@ZERO@ # ;

This we can parse with left to right longest match without special handling for
percents.

If we don't strip_percents, by the way, we need to inject % mark and
%% mark to the alphabet, and treat the former as epsilon, which will cause
severe disalignment. Or we can declare %X for each X in non-multichar symbols,
and finally rewrite them as X, which is even slower.


.. _HFST glossary: https://kitwiki.csc.fi/twiki/bin/view/KitWiki/HfstLexC#Glossary

.. vim: set ft=rst:
