In Tutorial 1, we learned how to
do hand annotation for simple span annotations - tasks which
involve only labels and text spans. But MAT supports hand
annotation for much more complex annotation schemas, where
annotations have additional attributes; and some of those
attribute values can be other annotations. In this tutorial, we'll
learn about additional capabilities in the UI that you'll
encounter when you annotate documents according to these more
complex schemas. (For details about the impact on other MAT
capabilities, see here.)
So like tutorial 1, we're going to do this tutorial in file mode. We're going to assume that the sample task is installed (see step 1 in Tutorial 1 if it isn't).
See the section on starting the UI in Tutorial 1.
In all the tutorials so far, we've been using the named entity
task. The named entity task is a simple span annotation task. What
we didn't tell you, way back in Tutorial 1, was that when you
installed this task, you installed two other tasks, which we're
going to be using in the last two tutorials.
The first of these tasks is called "Enhanced Named Entity", and it's intended only to illustrate the enhancements we're about to show you. This task contains six labels, rather than three, and is modeled on a selection of tasks you might encounter. For this task, we're going to expand the intended meanings of the three labels in the "Named Entity" task:
Each of these labels has an attribute named "nomtype", which can
have one of three values: "Proper name" (e.g., "Hilary Clinton"),
"noun" (e.g., "the secretary"), or "pronoun" (e.g., "she"). We're
going to call this set of labels mentions, because they
mention the individual they refer to. (This term is common in
language annotation.) For illustration, we've also added a Boolean
(i.e., true/false) attribute to the LOCATION label,
"is_political_entity", which should be true for things like
"Poland" and false for things like "Asia".
In addition to these labels, we have a label named
"PERSON_COREF". The intention of PERSON_COREF is to encode the
fact that a set of annotations all refer to the same individual.
So if you have two PERSON annotations, one on "Hilary Clinton" and
one on "she", and they refer to the same person, you'd represent
this by creating a PERSON_COREF annotation, and adding each of the
PERSON annotations to the value of its "mention" attribute, which
must be a set of annotations. The intended interpretation is that
all the annotations in the "mentions" attribute value refer to the
Because the PERSON_COREF annotation doesn't correspond to a
specific chunk of text, it is spanless. (You'll learn how
to manipulate spanless annotations in a minute.)
Finally, we have two other annotations:
We don't intend these definitions to be precise; these two labels
represent two ways of representing events and relations which you
Let's see how to add these annotations.
Let's go ahead and load a document.
Your window will look like this:
You should notice three important differences from the document
view you found in Tutorial 1:
We'll learn how to use all these features in this tutorial.
You'll also notice, in the control area on the right, that this
step is completely done for this document, and the annotation
status line shows that hand annotation is not available. Back up
through this step by pressing the back button which has the
drawing hand on it; this will mark the document as not complete
for the current step, and hand annotation will be available.
The document we just loaded was annotated for the "Named Entity"
task. So the annotations it contains aren't all the annotations
we'd add, and those annotations don't yet have values for the
attributes those labels have in this task.
Click on the annotation for "North Korea" in the second line.
You'll notice that in the popup menu, in addition to the usual
actions, there are two new actions: "Edit annotation in popup" and
"Edit annotation in tab". Select "Edit annotation in popup", and
you'll get an annotation editor popup:
You can set values for the nomtype and is_political_entity
attributes here. Choose "Proper name" for nomtype, and "yes" for
is_political_entity, and then press "OK". Now, if you hover your
mouse over the annotation for "North Korea", you'll see a
description of the annotation with its attribute values in the
line immediately below the document text.
Notice that the editor has a green interior border; this
identifies it as the "active" annotation editor. This feature will
be important in a minute.
Press the "Done" button to dismiss the editor.
Next, let's create a new annotation. In the second paragraph,
create a LOCATION annotation for "it" in the sentence "It has
about 800 missiles...". An annotation editor will pop up
immediately (we've configured this task to do that). Select
"Pronoun" for nomtype, "yes" for is_political_entity, and then
In the detail pane at the bottom, you have a tab named
"Annotation tables for voa1.txt.json". These tables can be
displayed for any document, via "View -> Show annotation
tables", but if the task contains spanless annotations (e.g.,
PERSON_COREF), this tab is enabled automatically. The drop-down
menu at the top of this pane allows you to access tables for each
annotation separately. The numbers in parentheses indicate how
many of the elements are present.
Select "LOCATION" in the drop-down menu. You'll see that that
first element listed, "North Korea", has the attribute values you
set a moment ago. Click on that row. You should see a popup menu,
which again gives you the opportunity (among other things) to edit
the annotation in a popup editor. Select this option. Now change
is_political_entity from "yes" to "no". You should see the value
of that attribute change in the annotation table as you modify it
in the editor popup. Change the value back to "yes", and click
Most of the columns in the annotation tables are sortable. The
default order (for spanned annotation) is ascending by start
index. Select the "Text" column header, which shows the text span
for the annotation. You'll see the rows reorder themselves by
ascending order of the alphabet. Press the "Text" header again,
and the order will invert.
Select the "Start" header to return the table to its original
In paragraph 3, select the text "said" and label it as a
LOCATED_EVENT. You'll immediately get a popup editor which looks
There are two ways to add annotations as attribute values. The first is called choose mode. Choose mode allows you to create or select an annotation which can be the value of this attribute. Press "Choose" next to "actor" to enter choose mode. Your window will look like this:
You'll notice that the "Choose" button now reads "Choosing (press
to exit)", and the status line above the document text now reads
"Choose mode: active", along with an option to exit choose mode.
You can now either swipe text, to create a new annotation to
serve as the attribute value, or select an existing annotation, to
choose it as the value. If you're creating a new annotation, the
annotation will be created immediately (if there's only one kind
of annotation that can fill the attribute) or you'll get a popup
menu corresponding to your choices (if there are multiple kinds).
In this case, we have only the option of choosing or creating a
Move the popup editor down and to the right, until you can see
all of the third paragraph. Swipe the word "he" immediately to the
left of the "said" that you swiped to create the LOCATED_EVENT
(make sure to swipe, rather than click - normally, you can click
on an unannotated word to swipe it, but not in choose mode). A
PERSON annotation will be created immediately, and you'll
automatically exit choose mode. You'll get a second popup editor
to edit the PERSON, where you can set the nomtype (which you
should set to "Pronoun"). You'll see also that the popup editor
also has a new "References" section which shows you where the
annotation is attached:
If you click "Done" and dismiss this popup editor, you'll see in
the popup editor for "said" that the "actor" attribute is now
filled with the annotation you just created.
Another to add attributes as values is to select the annotation
you want to use as an attribute, and add it to the active
annotation editor (the one with the green border, if more than one
editor is open). Select the annotation on "Seoul" in this same
paragraph. In the popup menu, you'll see an entry "Set location
slot". Choosing this menu item will cause the annotation to be
inserted as the value of the "location" attribute in the active
annotation editor. Do this. (These are only two of the ways to
link annotations to each other; for more details, see here.)
Now that you've linked annotations, you can see how they're connected by hovering over them. When annotations are linked, when you hover over one, the other will be highlighted, along with the relationship it has to the element you're hovering over. So if you hover over "said" in the annotation text, "he" and "Seoul" will be highlighted, along with an indication that "he" is the "actor" and "Seoul" is the "location. Similarly, if you hover over "he" or "Seoul" in the annotation editor popup, they will be highlighted in the annotation text.
Finally, dismiss the popup editor for "said" by clicking "Done",
and in the annotation tables, select "LOCATED_EVENT" from the
drop-down menu. You'll see that the values you inserted are shown.
And if you select "PERSON", you'll see that there's now an entry
for "he" (which you just created), which shows its references, in
the same way that they're shown in the popup editor.
Up to this point, all the annotations we've worked with are
spanned annotations - that is, they correspond to a region of
text. MAT also supports spanless annotations, which aren't
anchored directly to the text in this way.
The region to the left of the annotation text is the spanless
sidebar. Icons for spanless annotations will appear in this
sidebar, aligned (as closely as possible) with the midpoint of the
annotations they point to. We can create spanless annotations by
clicking in this sidebar.
Click in the spanless sidebar. You should get an annotation popup
menu which gives you the opportunity to create either a
LOCATION_RELATION or a PERSON_COREF annotation. Choose
LOCATION_RELATION. You'll get an annotation editor popup, and an
icon will appear in the spanless sidebar, at the very top of the
document (because it has no annotations it points to yet). We're
going to use this relation to mark the location relation expressed
by "South Korea's Dongseo University" in the paragraph about
two-thirds of the way through the document, in the paragraph that
begins "Brian Myers specializes in...". Scroll down to this
First, make "South Korea" the value of the "location" attribute,
either via choose mode or by selecting the "Set location slot"
element from the annotation popup menu. The relation icon will now
be aligned with "South Korea", rather than at the top of the
window as it was when it was created. Complete the relation by
selecting "Dongseo University" to fill the "located" attribute.
Your window should look like this:
Click "Done" to dismiss the editor popup.
As our final hand annotation activity, we're going to use the
PERSON_COREF annotation to represent coreference between mentions.
Return to the third paragraph, which begins "David Asher is...".
Pop up an editor for the PERSON annotation on "David Asher". In
the "Actions" menu, select "Add to mentions in PERSON_COREF...".
This action will shift us into choose mode; in this case, rather
than selecting an annotation that fills an attribute value, we're
going to select or create the annotation which this annotation
fills. (We could also have initiated this process from the
spanless sidebar, but in some cases, this is a more natural way to
Click in an empty region of the spanless sidebar. This has the
same effect, in choose mode, as swiping the annotation text,
except it works for spanless annotations instead of spanned
annotations. Because there's only one possible annotation type
that can be created (namely, PERSON_COREF), this annotation will
be created immediately, and the annotation for "David Asher" will
be inserted into the "mentions" attribute. At this point, your
window will look like this:
You'll see a spanless icon to the left of "David Asher", and two
popup editors: one for "David Asher", and a new one for the
PERSON_COREF annotation, showing the mention as an element of the
"mentions" attribute. Note the curly brackets, indicating a set
(so far, with one member).
Now, we have two annotation editors open, but the coref editor is
active. Let's add another coref element. Click on "he" in the next
line, and select "Add to mentions slot" from the popup menu. The
reference will appear in the annotation editor, and if "he" is
visible in the annotation table, you'll see that it now has two
Hover over "he" in the text pane. You'll see that its two
references are highlighted. Now hover over the PERSON_COREF icon
in the spanless sidebar. Note that the mentions are shown as part
of the annotation description in the status line immediately below
the document text, and that the mentions are highlighted in the
text. In the active PERSON_COREF editor, hover your mouse over the
descriptions of the annotations in the "mentions" slot. You'll
notice that they change color, and that those annotations are
highlighted with a box in the document text. Click on the
description of "he". You'll get a popup menu which allows you to,
among other things, detach the annotation from this location.
Choose that option, and dismiss the popup editor. If you hover
over the spanless icon again, you'll see that it now only has one
Click "Done" in both popup editors to dismiss them.
For more details on editing annotations, see here. For more details about
spanless annotations, see here.
Now, you can save your file if you want. This is identical to
step 6 in Tutorial 1.
Shut down your Web server by typing "exit" in the window where
you started the Web server. More details here.
If you're not planning on doing any other tutorials, and you
don't want the "Enhanced Named Entity" task hanging around, remove
it as follows:
Unix:This concludes Tutorial 7.
% cd $MAT_PKG_HOME
% bin/MATManagePluginDirs remove $PWD/sample/ne
> cd %MAT_PKG_HOME%%
> bin\MATManagePluginDirs.cmd remove %CD%\sample\ne