A morph.xml file is a list of word-forms and a list of associated macros.
morph @name
> entry* @word @stem @pos @macros
> macro* @name
> fs? @id @attr @val
> lf?
The 'stem' attribute should be thought of as the semantic predicate (by default it's the same as the value of 'word'). The 'pos' attribute specifies which families are relevant in building the full lexical category. The 'id' attribute make reference to an identifier in the 'lexicon.xml' file.
macro @name > fs* > lf
private String name; private FeatureStructure[] featStrucs; private LF[] preds;
This class parses the entry elements in morph.xml files:
entry @word @pos (@stem) (@class) (@coart) (@macros) (@excluded)
There are the following fields:
private Word surfaceWord; private Word word; private Word coartIndexingWord; private String[] macros; private String[] excluded; private boolean coart = false;
The constructor is quite complicated. The value of 'macros' and 'excluded' is straightforward - you just split the value up into string tokens (macro names and excluded strings). The value of 'surfaceWord' comes from passing the value of the word attribute through the tokeniser in some strange way:
surfaceWord = Grammar.theGrammar.lexicon.tokenizer.parseToken(e.getAttributeValue("word"),coart);
The value of 'word' is derived as follows:
word = word.createFullWord(surfaceWord,
e.getAttributeValue("stem"),
e.getAttributeValue("pos"),
null,
e.getAttributeValue("class"));
This class parses member elements inside family elements inside lexicon.xml files:
member @stem (@pred)
The fields and constructor are obvious:
private String stem;
private String pred;
public DataItem(org.jdom.Element e) { ... }
If there is no 'pred' attribute, its value is the same as 'stem'.
This class is used to parse entry elements from inside family elements in lexicon.xml files:
entry @name (@stem) (@active) (@indexRel) > category
The fields for this class are:
private String name; private String stem; private Boolean active; private String indexRel; private Category cat; private Family family;
The constructor method is obvious too:
public EntriesItem(org.jdom.Element e, Family f) {
this.family = f;
...
cat = CatReader.getCat((Element) e.getChildren().get(0));
}
If there is no 'stem' attribute, this field gets the value Lexicon.DEFAULT_VAL, i.e. "[*DEFAULT*]". The default value of 'active' is Boolean.TRUE. If there is no 'indexRel' attribute, then the value of this field comes from that of the family. Obviously, the last line is the most interesting - it takes the first child only and converts it into a category? What if there are other children?
There are the usual 'get' methods, as well as a couple that get attributes of the immediate superfamily, as well a toString() method.
This class is used to parse XML family elements in OpenCCG lexicon.xml files:
family @name @pos (@closed) (@indexRel) (@coartRel) > entry* > member*
The fields of a Family object are thus as follows:
private String name; private String pos; private Boolean closed; private String indexRel; private String coartRel; private EntriesItem[] entries; private DataItem[] data; //members private String supertag;
The supertag is formed by removing slash modalities and other minor features from a category (i.e. from the family name). There are get and set accessor methods for each field. The constructor method is exactly as you'd think:
public Family(org.jdom.Element e) { ... }
See also: opennlp.ccg.lexicon.EntriesItem and opennlp.ccg.lexicon.DataItem.
The constructor method of the opennlp.ccg.grammar.Grammar class contains the following code:
public final Lexicon lexicon; lexicon = new Lexicon(this); lexicon.init(lexiconUrl,morphUrl);
Here we turn to the code for the opennlp.ccg.lexicon.Lexicon class. Here are two basic fields and the constructor:
public final Grammar grammar;
public final Tokenizer tokenizer;
public Lexicon(Grammar g) {
this.grammar = g;
this.tokenizer = new DefaultTokenizer();
}
The init method starts as follows, where there are two input parameters, lexiconUrl and morphUrl:
List<Family> lexicon = null; List<MorphItem> morph = null; List<MacroItem> macroModel = null; lexicon = getLexicon(lexiconUrl); Pair<List<MorphItem>,List<MacroItem>> morphInfo = getMorph(morphUrl); morph = morphInfo.a; macroModel = morphInfo.b;
See also opennlp.ccg.lexicon.Family, opennlp.ccg.lexicon.MorphItem, and opennlp.ccg.lexicon.MacroItem.
This class implements the notion of a (multiple inheritance) hierachy of syntactic types. It is called by the constructor method of the opennlp.ccg.grammar.Grammar class, as documented here. Each Grammar object has an instance field types of class Types. The constructor method for Grammar calls one of two constructor methods to initialise this field: (a) new Types(URL u,Grammar g); or (b) new Types(Grammar g).
The code for the
Apart from that, the code is a bit of a mess, so I'm going to ignore these classes for the moment.
This class encodes the notion of a CCG grammar, essentially a lexicon, a set of rules and a type hierarchy. Its constructor is called by the opennlp.ccg.TextCCG class, as discussed here. Here are the essentials (ignoring XSLT files for transforming from and to LFs, pitch accents, special tokenisers, supertags etc.):
public final Types types; public final Lexicon lexicon; public final RuleGroup rules; private String grammarName = null; public static Grammar theGrammar; //nasty hack!Here are the basics of the constructor, which takes a single input parameter url, of class java.net.URL:
theGrammar = this;
SAXBuilder builder = new SAXBuilder();
Document doc = builder.build(url);
Element root = doc.getRootElement();
grammarName = root.getAttributeValue("name");
...
Element typesElt = root.getChild("types");
URL typesUrl;
if (typesElt!=null)
typesUrl = new URL(url,typesElt.getAttributeValue("file"));
else typesUrl = null;
Element lexiconElt = root.getChild("lexicon");
URL lexiconUrl = new URL(url,lexiconElt.getAttributeValue("file"));
Element morphElt = root.getChild("morphology");
URL morphUrl = new URL(url,morphElt.getAttributeValue("file"));
Element rulesElt = root.getChild("rules");
URL rulesUrl = new URL(url,rulesElt.getAttributeValue("file"));
...
if (typesUrl!=null) types = new Types(typesUrl,this);
else types = new Types(this);
lexicon = new Lexicon(this);
lexicon.init(lexiconUrl,morphUrl); *****
rules = new RuleGroup(rulesUrl,this);
...
This is the top-level class for the OpenCCG parser. It contains the main method. Here is the important stuff (assuming a default grammar file and no input parameters, and ignoring special input commands when running the tccg loop):
String grammarfile = "grammar.xml";
URL grammarURL = new File(grammarFile).toURL();
Grammar grammar = new Grammar(grammarURL);
Parser parser = new Parser(grammar);
Realizer realizer = new Realizer(grammar);
LineReader lineReader = ...
while (true) {
String input = lineReader.readLine("tccg> ");
try {
parse.parse(input.trim());
List parses = parser.getResult();
...
}
catch (ParseException pe) System.out.println(pe);
}
See also opennlp.ccg.grammar.Grammar.
$ java -Xmx128m -classpath ../../lib/openccg.jar:
../../lib/trove.jar:
../../lib/jdom.jar:
../../lib/jline.jar:.
opennlp.ccg.TextCCG
The tccg command has a number of optional arguments, which are passed straight on to the TextCCG class's main method as input parameters: (a) you can get help using tccg -h; (b) you can set a grammar file to read from, i.e. tccg grammar.xml; (c) you can set a file for 'exporting preferences' to, with tccg -exportprefs blah; or (d) you can set a file for 'importing preferences' from, with tccg -importprefs blah.