=head1 NAME

intro.pod - introduction to WordNet::SenseRelate::TargetWord

=head1 CONTENTS

I<intro.pod>, I<install.pod>, I<utils.pod>, I<modules.pod>, I<developers.pod>,
I<config.pod>, L<FDL.txt>

=head1 SYNOPSIS

You can use pod2html, pod2latex, pod2man, or pod2text to get this
documentation in a different format.  See the man pages for pod2html, etc.
These translators should come with Perl, but you can also download them
from L<http://search.cpan.org>.

=head1 DESCRIPTION

This package consists of a set of Perl modules along with supporting Perl
programs that perform the task of Word Sense Disambiguation. The program(s)
attempt to disambiguate the sense of a single target word in a given
context as described by Banerjee and Pedersen (2002), Patwardhan et
al. (2003) and Banerjee and Pedersen (2003). This package separates the
disambiguation process into the following steps:

(a) Preprocessing

(b) Context Selection

(c) Postprocessing

(d) Sense Selection Algorithm

The context, which is typically 3 to 4 sentences of text, is passed through
each of these stages. Each stage does some processing of the context. The
final sense selection stage picks a sense of the target word, using the
information from the context. For example, in order to determine which
sense of a given word is being used in a particular context, the sense
having the highest relatedness with its context word senses is most likely
to be the sense being used.

Because of the wide range of possibilites for each of the stages, the
package has been made highly configurable. The user is allowed to choose
different modules for each of the stages. Additionally, various
configuration options can be passed to the substages to control their
behavior.

The WordNet::SenseRelate::TargetWord initializes the modules specified by
the user for each substage. It then accepts instances of text, consisting
of a few sentences, with one of the words marked as the target word to be
disambiguated. It passes the text through each of the stages, and gets the
chosen sense from the last (Algorithm) stage, and return the chosen sense
as the answer for that instance. The substages are applied in the following
order:

=over

=item Preprocessing

WordNet::SenseRelate::TargetWord allows the user to apply multiple
preprocessing modules to the input text. All of the preprocessing modules
specified by the user are applied to the text in the order specified by the
user.

=item Context Selection

The WordNet::SenseRelate::TargetWord package allows the user to select one
I<Context Selection> package, which when applied to the context would pick
a subset of the words around the target word for the disambiguation
algorithm to use in picking the intended sense of the target word.

=item Postprocessing

Like the preprocessing modules the user can select multiple postprocessing
modules that would be applied to the text after the context selection has
been done. Before the postprocessing modules are called the program
determines the possible senses of the target word and the selected context
words. The postprocessing stages are intended for any processing that
involves the senses of the context words. For example, applying some
heutistics to "prune" some of the senses of the target word, or the context
words.

=item Sense Selection Algorithm

The goal of the sense selection algorithm is to use the senses of the
context words to pick one of the senses of the target word as the
answer. The WordNet::SenseRelate::TargetWord module allows the user to
specify only one Algorithm module.

=back

=head1 INSTALL

The package can be installed with the following four commands:

 perl Makefile.PL

 make

 make test

 make install

For a more detailed explanation or for non-standard installations, see L<install.pod>.

=head1 MODULES

In this section, we list the modules provided in the package.

=over

=item Preprocessing modules

Currently, only one preprocessing module is provided in the package:

=over

=item WordNet::SenseRelate::Preprocess::Compounds

This module detects collocations within the text, and joins them with
underscores. For example, a piece of text such as "the board of directors"
would become "the board_of_directors".

=back

=item Context Selection modules

One context selection module is provided in the package:

=over

=item WordNet::SenseRelate::Context::NearestWords

This modules picks the N words nearest the target word as the context words
that should be used by the disambiguation algorithm. A stop list can be
specified to omit words like "a", "an, "the", etc. The value N can be set
during initialization.

=back

=item Postprocessing modules

As of this vesion, there are no postprocessing modules in the
package. However, we intend to add some in future releases.

=item Sense Selection Algorithm

Four sense selection algorithm modules are provided in the pacakge:

=over

=item WordNet::SenseRelate::Algorithm::Local

This modules selects that sense of the target word which is most related to
the senses of the context words. To do this it uses the "Local"
disambiguation algorithm as described by Banerjee and Pedersen (2002). In
order to determine the relatedness of senses, the algorithm uses one of the
WordNet::Similarity measures of relatedness. Using the configuration
options for this module, the user can specify which measure the algorithm
should use.

=item WordNet::SenseRelate::Algorithm::Global

This modules selects that sense of the target word which is most related to
the senses of the context words. To do this it uses the "Global"
disambiguation algorithm as described by Banerjee and Pedersen (2002). This
algorithm is somewhat similar to the "Local" algorithm. It differs from the
"Local" algorithm, in that it forms all possible combinations of the senses
of the context words, and evaluates the semantic relatedness for each
combination separately. The combination with the maximum score is is
selected, and the sense of the target word in that combination is returned
as the answer.

=item WordNet::SenseRelate::Algorithm::SenseOne

This module provides a baseline for the disambiguation process by always
returning the first sense of the target word as the answer.

=item WordNet::SenseRelate::Algorithm::Random

This module provides another baseline by randomly selecting one of the
senses of the target word as the answer.

=back

=back

Apart from all of the modules mentioned above that form the pieces of the
bigger structure, the package also contains the
WordNet::SenseRelate::TargetWord module which combines the above pieces.

In order to use these modules in a Perl program for Word Sense
Disambiguation, we need only create an instance of the
WordNet::SenseRelate::TargetWord module in our program, and provide it with
options that indicate which of the above modules (along with configuration
options) it should use in the disambiguation process.

Additionally, in order to be able to use this package for disambiguating
instances from the Senseval2 or the Senseval3 data sets, the
WordNet::SenseRelate::Reader::Senseval2 module has also been provided in
the package. The reader module reads in an entire Senseval2 formatted XML
file and builds a list of instances from the file. A Perl program can then
iterate over these instances and pass them to the
WordNet::SenseRelate::TargetWord object.

=head1 UTILITIES

Some Perl utilities that provide command-line and graphical interfaces to
the Perl modules are provided in the '/utils' directory of the package.

=over

=item disamb.pl

Performs Word Sense Disambiguation on Senseval-2 lexical sample data. It
uses the WordNet::SenseRelate::Reader::Senseval2 module to read a Senseval2
lexical sample file, and then disambiguates each of the instances using the
WordNet::SenseRelate::TargetWord module.

Usage: disamb.pl [ [--config FILE] [--wnpath WNPATH] [--trace] XMLFILE | --help | --version]

B<--config>=I<FILENAME>         
    Specifies a configuration file (FILENAME) to set up the various
    configuration options.

B<--wnpath>=I<WNPATH>         
    WNPATH specifies the path of the WordNet data files. Ordinarily, this
    path is determined from the $WNHOME environment variable. But this
    option overides this behavior.

B<--trace>
    Indicates that trace information be printed.

B<--help>
    Displays this help screen.

B<--version>
    Displays version information.


Example:

To disambiguate an English lexical sample file using the default options

  disamb.pl eng-lex-samp.xml

To dismabiguate an English lexical sample file, specifying configuration
options and trace output

  disamb.pl --config config.txt --trace eng-lex-sample.xml

=item wps2sk.pl

Creates a word#pos#sense to sensekey mapping of a Senseval-2 answer file
(output by disamb.pl). In order to be able to evaluate the output of
disamb.pl using software provided by the Senseval organizers, we need to
covert the output of disamb.pl to the "SenseKey" format.

Usage: wps2sk.pl [ [ --wnpath WNPATH] [FILE...] | --help | --version ]

B<--wnpath>=I<WNPATH>
    WNPATH specifies the path of the WordNet data files. Ordinarily, this
    path is determined from the $WNHOME environment variable. But this
    option overides this behavior.

B<--quiet>
    Run in quiet mode -- does not print informational messages. But it does
    print warning or error messages if any.

B<--help>
    Displays this help screen.

B<--version>
    Displays version information.

Example:

To convert a captured output from disamb.pl

  wps2sk.pl output.txt

Typically used in a pipe with disamb.pl

  disamb.pl xmlfile.xml | wps2sk.pl --quiet

=item disamb-gui.pl

Performs Word Sense Disambiguation on Senseval-2 lexical sample data
(graphical interface to WordNet::SenseRelate::TargetWord). It uses the
WordNet::SenseRelate::Reader::Senseval2 module to read a Senseval2 lexical
sample file, and then allows the user to pick the instances that she wishes
to disambiguate.

Usage: disamb-gui.pl [ [--config FILE] [--wnpath WNPATH] [XMLFILE] | --help | --version]

B<--config>=I<FILENAME>
    Specifies a configuration file (FILENAME) to set up the various
    configuration options.

B<--wnpath>=I<WNPATH>
    WNPATH specifies the path of the WordNet data files. Ordinarily, this
    path is determined from the $WNHOME environment variable. But this
    option overides this behavior.

B<--help>
    Displays this help screen.

B<--version>
    Displays version information.

=back

=head1 CONFIGURATION FILE

A configuration file can be passed to both, disamb.pl as well as to
disamb-gui.pl in order to set up the configuration parameters for the
modules. The format of this file is specified in this section.

The configuration file must start with a header, which is the string
"WordNet::SenseRelate::TargetWord" on the first byte of the first line of
the file.

It consists of a number of sections. Each section contains a list of
modules for that section, along with the configuration options for the
modules.

A section starts with the string

  [SECTION:section_name]

A new section header indicates the end of the previous section. The utilies
in this package recognize four types of sections:

  [SECTION:PREPROCESS]

  [SECTION:CONTEXT]

  [SECTION:POSTPROCESS]

  [SECTION:ALGORITHM]

Any other section will be ignored by the program, without warning. Each of
the sections is used to specify the list of modules for that section. A
module is specified by [START modname] and [END] tags appearing on separate
lines.

Configuration parameters for the module appear on separate lines in between
the start and end tags as PARAMETER=VALUE pairs.

Example config file:

I<WordNet::SenseRelate::TargetWord>

I<[SECTION:PREPROCESS]>

I<[START WordNet::SenseRelate::Preprocess::Compounds]>

I<[END]>

I<[SECTION:CONTEXT]>

I<[START WordNet::SenseRelate::Context::NearestWords]>

I<windowsize=5>

I<windowstop=/home/sid/window-stop.txt>

I<[END]>

I<[SECTION:POSTPROCESS]>

I<[SECTION:ALGORITHM]>

I<[START WordNet::SenseRelate::Algorithm::Local]>

I<[END]>

The above config file has four sections -- PREPROCESS, CONTEXT, POSTPROCESS
and ALGORITHM. The PREPROCESS section contains a single module for compound
detection, and has no config parameters. The CONTEXT section lists one
module (NearestWords), with parameters I<windowsize> and
I<windowstop>. There are no modules listed in the POSTPROCESS section. The
ALGORITHM section lists the Local disambiguation algorithm module, also
with no config parameters.

=head1 SOFTWARE COPYRIGHT AND LICENSE

Copyright (C) 2005 Siddharth Patwardhan, Ted Pedersen, and Satanjeev
Banerjee

This suite of programs is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License as published by the
Free Software Foundation; either version 2 of the License, or (at your option)
any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.  See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with
this program; if not, write to the Free Software Foundation, Inc., 59 Temple
Place - Suite 330, Boston, MA  02111-1307, USA.

Note: The text of the GNU General Public License is provided in the file
'GPL.txt' that you should have received with this distribution.

=head1 ACKNOWLEDGEMENTS

This research is partially supported by a National Science Foundation
Faculty Early CAREER Development Award (#0092784).

=head1 REFERENCES

=over

=item 1

S. Banerjee and T. Pedersen. An Adapted Lesk Algorithm for Word Sense
Disambiguation Using WordNet. In Proceedings of the Fourth International
Conference on Computational Linguistics and Intelligent Text Processing
(CICLING-02). Mexico City, Mexico. 2002.

=item 2

S. Banerjee and T. Pedersen. Extended gloss overlaps as a measure of
semantic relatedness. In Proceedings of the Eighteenth International
Conference on Artificial Intelligence (IJCAI-03). Acapulco, Mexico. 2003.

=item 3

S. Patwardhan, S. Banerjee and T. Pedersen. Using Semantic Relatedness for
Word Sense Disambiguation. In Proceedings of the Fourth International
Conference on Intelligent Text Processing and Computational Linguistics
(CiCLING-03). Mexico City, Mexico. 2003.

=item 4

S. Banerjee. Adapting the Lesk algorithm for Word Sense Disambiguation to
WordNet. Masters Thesis, University of Minnesota, Duluth. 2002.

=item 5

S. Patwardhan. Incorporating Dictionary and Corpus Information into a
Context Vector Measure of Semantic Relatedness. Masters Thesis, University
of Minnesota, Duluth. 2003.

=item 6

Fellbaum C., editor. WordNet: An electronic lexical database. MIT Press.
1998.

=back

=head1 SEE ALSO

L<http://groups.yahoo.com/group/senserelate>

L<http://search.cpan.org/dist/WordNet-SenseRelate-TargetWord>

L<http://senserelate.sourceforge.net>

=head1 AUTHORS

 Siddharth Patwardhan, University of Utah
 sidd at cs.utah.edu

 Ted Pedersen, University of Minnesota Duluth
 tpederse at d.umn.edu

 Satanjeev Banerjee, Carnegie Mellon University
 banerjee+ at cs.cmu.edu

=head1 DOCUMENTATION COPYRIGHT AND LICENSE

Copyright (C) 2005 Siddharth Patwardhan, Ted Pedersen, and Satanjeev
Banerjee

Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.2
or any later version published by the Free Software Foundation;
with no Invariant Sections, no Front-Cover Texts, and no Back-Cover
Texts.

Note: a copy of the GNU Free Documentation License is available on
the web at L<http://www.gnu.org/copyleft/fdl.html> and is included in
this distribution as FDL.txt.

=cut
