Span annotation

The Web UI that comes with MAT is a rich, logging-enabled tool for viewing, processing and annotating your documents. We describe how to use it here.

We assume that you've started the Web server and the UI.

Note: while the UI attempts to minimize the redraw required for the annotation panes, there are circumstances where large numbers of annotations (e.g., thousands, including tokens) will slow down the UI visibly in specific circumstances: initial document load; displaying the result of automated steps performed in the backend; and resizing of the window width.

The file menu

The file menu looks like this:

[file menu]

The five things you can do from this menu are:

Each of these operations brings up a dialog so you can proceed. In this section, we'll focus on simple span annotation in file mode, accessible via "Open file...".

Loading and viewing files in file mode

The "Load document" dialog

When you select "File -> Open file...", you'll be presented with the following dialog:

[load document dialog]

The "Task" menu will be active, and the "Language", "Workflow" and "Input" elements will be disabled. Once you select an item from the "Task" menu, the "Language" menu will be enabled; once you select a language, the "Workflow" menu will be enabled; once you select a workflow, the "Input" element will be enabled. If MAT knows of only one element for the task, language or workflow, those elements will be automatically selected for you.

The workflows are established in your task configuration file. Choose the workflow that's appropriate for what you're going to do. E.g., in the "Named Entity" task:

In addition to those, the "Workflow" menu allows you to open a document read-only, and to open a reconciliation or comparison document:

[file menu with workflow]

Select the appropriate document type and encoding for the document. If the document is a raw document, select "raw" from the "Document type" menu; or, select "mat-json" for rich annotated documents in the MAT JSON format (other document types will also be available). The encoding for MAT JSON documents will be fixed as utf-8. You also have the option of providing an optional memo, which will be echoed in the "Tabs" menu and the document title bar. Once you've filled out the appropriate elements, the "Open" button will be enabled:

[load document open]

If you press the "Open" button, the dialog will vanish and you'll see a new tab in the upper left region of your window, and the task-related information about your document will appear in the upper right region.

The file mode document window

At this point, your UI will look like this:

The tab contains the basename of the document, and the hand annotation status is shown above the document text, below the tab (here, it says "Hand annotation unavailable"). At the right end of the title tab, there are two icons. The "-" hides the document, and the "x" closes it; you can make a hidden document visible again using the "Tabs" menu.

In the upper right region, you see the task controls on top, with a more complete description of the document name, the workflow menu, and a sequence of steps. The advance/retreat controls appear immediately below the steps; the "Reload" button allows you to reload the document. Below the task controls is the tag legend.

The "up" arrow to the right of the "Controls" and "Legend" labels allows you to contract that section of the task information.

When there's an active tab, the "File -> Save..." submenu is enabled:

[raw plus menu]

This menu allows you to save the document in various formats. Note: if you're having trouble finding the document you saved, please keep in mind that the browser, not the MAT UI, is responsible for saving your file. In particular, if you haven't configured your browser to prompt you for where to save your file, it will be saved to your browser's download directory. To fix this in Firefox, see the documentation on starting the UI.

The advance/retreat controls

The advance/retreat controls govern how you move through your workflow. The iconography for advancement is intended to be consistent:

The retreat buttons should also be consistent (these buttons will be available if your workflow is marked undoable="yes"):

There are many different possible patterns, depending on which type of step you're in, and where you are in the step (if it has multiple phases):

step type
what's been done in the step
which buttons you'll see
what you can do
what the buttons do
(all circumstances)
a button with a gear + right arrow
press the button
press to apply the automated step in the MAT backend
(all circumstances)
a button with a writing hand + right arrow
hand annotation
press to indicate that you've completed hand annotation
a button with a gear, and a grayed-out button with a writing hand + right arrow
press the button
press to apply the automated step in the MAT backend
some annotation
a grayed-out button with a gear, and a button with a writing hand + right arrow
hand annotation
press to indicate that you've completed hand annotation
a button with a gear, in parentheses, and a button with a writing hand + right arrow
press the gear button, or hand annotation
press the gear button to apply the automated step in the MAT backend, or press the hand button to indicate that you've completed annotation for this step
some annotation
a grayed-out button with a gear, in parentheses, and a button with a writing hand + right arrow hand annotation
press to indicate that you've completed annotation for this step

In the initial screen above, the "zone" step is an auto step, so you see a button with a gear + right arrow. Below, we'll see some of the other button configurations.

If you hover over one of the buttons, it will tell you what it does; e.g., if you hover over the button in the initial screen above, it'll say "Apply zone".

The legend

The legend lists the various content annotation sets and annotation labels, and also allows you to control these annotation sets and labels in various ways. In general, annotations which can be added and modified in a given step can be active, inactive, or selected, and all annotations can be invisible. Here's what each of these settings means:

what it means
annotation labels and sets in steps which add or modify the relevant sets
The label (or all the labels in the set) will be available in the annotation popup menu for addition to the document. This is the default setting for this case.
annotation labels and sets in steps which add or modify the relevant sets
The label (or all the labels in the set) will not be available for addition, even though the step adds or modifies them. The "Inactive" setting on an individual label takes precedence over the setting on the set which contains that label.
all content labels and sets in all steps
The annotations bearing the label (or any of the labels in the set) will not be rendered in the annotation pane.  The "Invisible" setting takes precedence over all other settings, either at the set or individual label level.
annotation labels (but not sets) in steps which add or modify the set which contains the label
When an annotation creation gesture is made in the annotation pane, an annotation of the selected label will be created immediately, rather than an annotation popup menu being presented.
annotation labels and sets in steps which do not add or modify the relevant sets
The annotations bearing the label (or any of the labels in the set) will be rendered in the annotation pane. This is the default setting for this case.

In the initial screen above, the "zone" step doesn't add or modify any content annotations, so all the annotation label controls are "Visible".


Here's another view of the UI, after two steps (zone and tokenize) have been applied by pressing the forward button:

The following changes are visible:

Whenever the hand annotation status line reads "Hand annotation available (swipe or left-click)", you may edit the content annotations for the current step in your document.

Adding an annotation

To add an annotation, either swipe or left-click in the document text where there are no content annotations. Swiping selects a range of tokens; left-clicking selects a single token. In this example, we've swiped:

[add annotation]

The annotation menu pops up automatically. In this example, where the text is previously unannotated, the menu contains, for each content annotation, the option to add that annotation. The menu items are styled with the same CSS that the resulting annotation will be styled with. Optionally, the menu may present a keyboard accelerator; here, if you press "O" when the menu is visible, it's identical to selecting "Add ORGANIZATION" from the annotation menu. The result is shown here:

[added annotation]

If you place your mouse over the annotation you've added, the tagging information line immediately below the text changes to show the name of the annotation:


And now that you've added an annotation, your annotation popup will provide the option of repeating the last annotation. If, say, you move on to annotate "Pakistan" as a location in the third line, and then make another selection, you'll see "Repeat LOCATION" as an option in your annotation popup:

[repeat annotation]

Note that this option always has the keyboard accelerator "=". So in this case, you can mark this next occurrence of "Pakistan" as a location either by pressing the "L" or "=" keys, or by selecting "Add LOCATION" or "Repeat LOCATION" from the annotation popup menu.

Removing an annotation

To remove an annotation, left-click the annotation:

[add or modify]

From the menu, select "Delete annotation", or press the "-" key. The annotation will be removed. If you'd rather replace the annotation with a different one, simply select that option with the mouse or via the keyboard accelerator; the current annotation will be removed and the new one added.

Adding an overlapping annotation

In some cases, you'll want to add annotations which overlap each other. To do this, swipe the region of text you wish to add an annotation to. Do not left-click on an existing annotation without swiping; the UI will interpret this as a request to perform an operation on the annotation (i.e., autotag, delete, or replace).

[add or modify annotation]

If you select "Add LOCATION" in this case, you'll now see a "stacked" display of the overlapping annotations:

[stacked annotations]

The stacking order is currently arbitrary. You may discover that when you add an annotation, it might become the annotation behind the text, and the new annotation may be moved to above the text.

For more about overlapping annotations, see here.

Modifying an annotation extent

As you can see, when you swipe over an existing annotation, you have the option of modifying the extent of that annotation. If the region contains multiple overlapping annotations, one of two things will happen. In the first case, if each "layer" has no more than one annotation on it, the popup menu will present you with the option of modifying the extent of the annotation on a particular layer:

[stacked modify]

If any layer has more than one annotation under the swipe, instead of having a separate entry for each layer, there will be a single option "Modify extent...", and when you choose it, you'll be presented with an opportunity to choose the annotation which you intend to modify. The extent of your swipe will be shown by a dotted-line box; any annotation which overlaps with this box is eligible to be selected to be modified:

[stacked modify multiple]

You may cancel the modification, either by pressing "Cancel" or closing the popup using the "x" in the upper right corner. If you select an annotation outside the boundary box, you'll be presented with an error and the modification will be cancelled.


You'll notice, above, that when you select an annotation, the popup menu contains not just the options to replace or remove, but also to "Autotag matches". If you select this option, all the eligible spans in the text which are not yet marked and match the existing span will be assigned the same annotation label as the already-annotated span. Autotagging has the following properties:

Annotation review and overlapping annotations

When hand annotation is available, overlapping annotations are always stacked, as shown above. It's possible to "collapse" this stack when hand annotation is not available, by changing the alwaysShowStackedAnnotations UI setting. When annotations are not stacked, they're layered on top of one another. This view is more compact, but less clear; but in addition to this UI setting, there are situations where all the annotations in a given document must be layered in a single line, such as in the document comparison and document reconciliation views.

When you're in these views, there are some other resources that you can rely on to understand what's in your document. First, when you hover your mouse over overlapping annotations in these views, you'll see, in the tagging information line, all the annotations at that position, and the frontmost annotation (the one you can see) will be in boldface. In addition, there are additional annotation menu options that allow you to manipulate these layered annotations. When you click (not swipe) on a position that has a visible annotation which is partially obscured elsewhere, you'll see a "Bring to front" menu action which will make that annotation frontmost along its entire extent. And when there are annotations which are not visible, there will be a "Bring <label> to front" menu action for each of those annotations.