Character sets
==============


Supported character sets
------------------------
UdmSearch supports for the following character sets:

Cyrillic group:
	koi8-r, windows-1251, iso-8859-5, cp866, x-mac-cyrillic

Western group:
	iso-8859-1

Central Europe group:
	windows-1250, iso-8859-2

Arabic group:
	windows-1256 

Greek group:
	windows-1253, iso-8859-7 

Hebrew group:
	iso-8859-8, windows-1255

Baltic group:
	iso-8859-4, iso-8859-13, windows-1257

Recoding
--------
indexer recodes all documents to the character set specified
in the "LocalCharset" indexer.conf command. Recoding only inside 
the character set group is available of course. This is currently 
implemented for "Cyrillic", "Central Europe","Greek","Baltic" groups.
Hebrew iso-8859-8 and windows-1255 character sets are letters compatible,
so no recoding is required. Recoding between character sets from the
different groups, for example, from Cyrillic koi8-r into Western 
iso-8859-1 will never be done by indexer.



Document charset detection
--------------------------
indexer detects document character set in this order:

1) "Content-type: text/html; charset=xxx"
2) <META NAME="Content" CONTENT="text/html; charset=xxx">
3) Defaults from "Charset" indexer.conf command


Automatic charset guesser
-------------------------
There is also automatic cyrillic charset guesser which is not compiled
by default. You may activate it using "--with-charset-guesser" configure
arguments. If the automatic character set guesser was built at 
installation time, the above three methods of charset detection will 
be used only in the case when automatic guessing fails.


Default Language
----------------
You can set default language for Servers by using DefaultLang 
indexer.conf variable. This is useful while restricting search
by url language.
