Version history

3.3

• Minor updates to documentation and license statement.
  

3.3RC8

• Fixed a long-standing lurking bug in the XML inline writer, where crossing dependencies were not properly identified.
• Enhanced the Python warning about characters outside the Basic Multilingual Plane.
• Cleaned up updated copyrights.
  

3.3RC7

• Internal refactor to provide cleaner support for CSS-based appearance management.
  

3.3RC6

• Small improvements to documentation on similarity profiles.
• Fixed a tiny bug in computing standard deviation in bootstrapping in the scorer.
• Fixed a tiny bug in MATReport related to creating file summaries for files which could not be read.
• Fixed a type error in comparing an integer in the scorer.
• Fixed a bug in MATManagePluginDirs where a directory that no longer exists could not be removed from the plugin list.
• Fixed a very subtle bug in the interaction between read-only attributes, editable attributes, and guided navigation in the annotation editor.
  

3.3RC5

• Fixed a tiny bug in the workspace log replay tool.
• Updated copyrights and sponsor license, imported copyright marking tools.
  

3.3RC4

• Added elements to task.xml to expose documentation enhancement for tasks, refactored to support new functionality.
• Fixed a bug where static documentation was assumed to be generated from the "doc" subdirectory in the task, which was not necessarily the case.
• Fixed a bug where marking a document gold in workspaces via the UI would not trigger a scheduled reconciliation review.
  

3.3RC3

• Fixed a bug where submenus were appearing off-screen in Firefox and Safari when the parent menu was scrolled.
• Fixed a lurking typo in handling batch replacements in the UI.
  
• Enhanced the UI replacement code to properly handle replacements of annotations that are attribute values of other annotations.
• Enhanced the UI replacement code to preserve attribute values of the replaced annotation when possible.
  
• Fixed some long-standing lurking bugs in managing the internal parent-child links properly when an existing list or set value of an annotation-valued attribute is modified.
• Made improvements to ensure that jCarafe will work with Java 15.
• Added UI setting to compress the annotation edit popup menu for use in tasks with large label sets.
  

3.3RC2

• Fixed two tiny but significant bugs in the scoring code that generates decompositions by attribute. This bug is likely to have been lurking for several MAT iterations.
  

3.3RC1

• Added an option to MATAnnotationInfoToJSON to allow JSON conversion of non-installed tasks.
• Fixed a bug in the task reader where effective label UI configurations were prevented from supporting the new background_span rendering style.
• Made additional enhancements to the new debugging options in MATWeb.
• Fixed a bug in the MATWeb startup routine where errors in starting up supporting threads messed up the flow control.
• Fixed a couple bugs related to Python 3 conversion, added tests.
• Regenerated Javadoc using Java 8.
• Incorporated tweaks to the Java library to make it compatible with Maven and Java up through Java 12.
• Fixed a subtle bug in redisplaying read-only document panels in the UI.
• Made it possible to reload read-only documents in the UI (in case they'd been updated on disk using an external process).
• Fixed a typo in the documentation for the "unschedule_review" workspace operation.
• Fixed a bug where "Show annotation tables" wasn't always disabled in tasks with spanless annotations. Updated documentation to clarify.
• Fixed a bug in annotation tree construction where the ordering of nodes in the hierarchy was accidently reversed.
  

3.3beta13

• Improved handling of MAT-JSON files hand-saved on Windows after a MAT server failure.
• Added debugging option to MATWeb to help diagnose server failures.
  

3.3beta12

• Fixed a bug in the installer where JavaCarafe hadn't been inserted appropriately into the MAT package.
• Fixed a bug in the plugin manager where it was failing to check whether an installed plugin path was identical to a new path before complaining.
• Added version numbers to the CSS and JS resources for the main Web page to force reloading when a new version arrives.
• Fixed a subtle bug in the XML inline reader where it was attempting to find a defined annotation for the label the writer uses to ensure that the XML document has a toplevel spanning element.
• Fixed an omission in the backend comparison creation where it wasn't requiring the document signals to be identical.
• Fixed a bug in the workbench UI where Web clients behind a URL-rewriting proxy were not being properly supported.
• Added a toggle on the annotation pane which allows token boundary trim/extend to be temporarily suspended.
• Reorganized the UI documentation to make its structure more sensible.
• Fixed a small Python 3 compatibility bug in MATTransducer.
• Added a "Hide" button to the annotation table tab in the UI.
  
• Improved the behavior of hovers and tooltips for the tab elements in the UI.
• Improved connection error handling in the Bottle server.
  

3.3beta11

• Removed some additional instances of has_key() for Python 3 compatibility.
• Fixed a bug where the tokenless autotag delimiters were not properly preprocessed when automatically expanding and contracting swipes.
  

3.3beta10

• Removed an undocumented, broken command-line tool.
• Fixed a Python 3 incompatibility in the verbose reporting of alignments when tags are aligned to tokens.
• Fixed a Python 3 incompatibility in modifying list values of annotation attributes.
  
• Fixed a variety of formatting errors and minor typos.
  

3.3beta9

• Installer fixes to deal better with cases where Java isn't installed.
  

3.3beta8

• Fixed a typo in the install script.
• Fixed a bug introduced in the automated portion of the code transition to Python 3 compatibility.
• Removed some unused code.
• Added a test to the test suite.
• Added documentation about CSS styling for choice and boolean attributes.
  

3.3beta7

• Added notification callback for when label status updates are complete in the UI.
• Fixed an omission in both the JavaScript and Python libraries where decoding MAT-JSON was not sensitive to malformed cases where the list of attributes for an annotation was longer than the available attributes for the type.
• Added the MAT-JSON of the current document being saved to the notification that the document can't be saved because of a server failure, to allow the user a list-ditch ability to save her work.
• Improved the error messages in JavaScript wellformedness checking for attribute values to be the same as the Python error messages.
• Fixed an issue in the UI redisplay code where under certain circumstances, the palette would redraw more regions than it needed to.
  

3.3beta6

• Rationalized the presentation of commandlines in subprocess verbose contexts.
• Added output capture functionality to the test suite.
• Added <annotation_set> to the new-style annotation declarations as a way of declaring the annotation sets.
• Liberalized the requirements for referring to non-default content categories in the new-style annotation declarations.
• Fixed a missing package import in MATAnnotationIntoToJSON.
• Reorganized the documentation to better support describing the role of the jCarafe maximum-entropy classifier.
• Added the <positive> and <negative> child elements of <boolean> in the new-style annotation declaration schema to better control the UI attributes for boolean attribute values.
• Added support for CSS styling for choice and boolean attributes.
• Fixed two typos in handling effective labels for integer attributes in old-style annotation declarations.
• Rationalized internal handling of attribute-changing actions in the UI.
• Improved attribute classifier documentation.
• Added sample task for classification and improved documentation for task creation.
  

3.3beta5

• Expanded jCarafe services to expose the maximum-entropy classifier. Added documentation for how to use the classifier to train for attribute values.
• Cleaned up installation script to reflect updates for how task locations are stored.
  
• Minor bug fixes.
  

3.3beta4

• Fixed a small bug in a test for the standalone UI.
  

3.3beta3

• Fixed a bug in the standalone viewer where the new getDocument() function wasn't actually returning a populated document object.
• Added an option to jCarafe to preserve sentence annotations.
• Fixed a limitation in the new bottle Web framework on the size of requests.
  

3.3beta2

• Made MAT compatible with Python 3.
• Factored out a separate pip package which contains the majority of the MAT Python core.
• Removed the requirement that most attributes in the task.xml files be ASCII-compliant.
• Added a documentation page describing the increased likelihood of character offset issues with Python 3.
• Removed the _SUBPROCESS_DEBUG variable in favor of a more robust solution.
• Made MAT compatible with Java versions up through Java 11.
• Removed the command-line utilities MATUpdateTaskXML and MATUpdateWorkspace2To3.
• Upgraded munkres dependency to 1.0.7 (Python 3 compatible but still MIT licensed).
• Replaced CherryPy Web service framework with bottle.
• Introduced some handling to deal with how Python 3 counts the length of Unicode strings differently.

3.3beta1

• Added the annotationPopupMenuFontSizePercentage UI setting.
• Changed UI settings popup to a tabbed view to make it easier to handle.
• For spanless annotations, subdivided the actions that allow a selected annotation to be added to another annotation. There are now separate gestures for new and existing annotations, and the annotator no longer has to click in the spanless sidebar to create a new annotation in this case.
• Added the annotationEditorDismissKey UI setting.
  

3.2.1beta3

• Fixed a bug where keyboard accelerators for attributes were not recognized if there were no annotation labels to be added in the step.
• Fixed a bug where keyboard accelerators at the toplevel annotation menu were being overridden by keyboard accelerators for attribute submenus which were not currently visible.

3.2.1beta2

• Fixed a number of bugs related to the '/tasks/' URL.
  

3.2.1beta1

• Reconfigured Web server to encapsulate Web application at a single endpoint.
• Added the ability to MATWeb to customize the endpoint prefix.
  
• Replaced undocumented '/tasks/' URL with endpoint prefix + '/tasks/'.
• Fixed a bug in the installer which was failing to check to see if unzipped third-party tools were still there.
• Fixed some erroneous instructions about the standalone viewer.
• Fixed a bug where the new version of jCarafe was occasionally issuing whitespace as a token.
• Fixed a bug where certain CSS element URL paths were hard-coded to the obsolete package distribution layout.
  

3.2

• Minor enhancement to test harness.
  

3.2RC7

• Dealt with an error in the Web server where plugin errors reporting circularities in comparison were not being caught.
• Fixed a long-time lurking bug in autotagging in the Web UI where selecting an annotation for autotagging whose signal contains characters which are significant for regular expressions would result in the wrong autotagging behavior.
• Fixed a long-time lurking bug in tokenless autotagging in the Web UI where the dash character ('-') was handled incorrectly when used as a tokenless separator.
• Fixed a long-time lurking bug in tokenless autotagging in the Web UI where trailing delimiters couldn't count as leading delimiters for the next match.
  
• Fixed a bug where informational popups were not removed when a document is reloaded in the UI.
  

3.2RC6

• Updated Java requirements to clarify that nothing after Java 8 is supported.
  

3.2RC5

• Removed support for psutil (never used, only available on Linux)
• Removed some stray references to previously-included tabbed terminal executables.
• Migrated to git.
• Modified distribution layout and changed source archive to match.
• Fixed a bug introduced in 3.2RC1 where the distribution install script was failing to detect the availability of jCarafe server mode in the new jCarafe jar.
• Updated redistribute.py to work with the new archive.
• Modified installation to store configuration information outside the MAT Python library.
• Introduced option to store the plugin record outside the MAT distribution.
• Fixed a long-standing lurking bug in how the presence of unmodifiable configuration variables was checked in the Python library.
• Fixed what might have been a long-standing lurking bug on Windows when trying to do server-based jCarafe decoding.
  

3.2RC4

• Fixed a bug in the processing of <display_group> where nested groups did not report their labels appropriately.
• Fixed a bug where excluding labels in the popup menu (e.g., for replacement) could occasionally result in an empty menu.
  

3.2RC3

• Fixed an issue where both swipe and click actions were suppressed (instead of just click actions) when annotation gestures were suppressed for open annotation editors.
• Updated platform compatibility information.
• Fixed a lurking bug where keyboard arrows did not scroll in visible submenus, but rather in their parents.
• Fixed a lurking bug where <enter> did not do the expected thing when submenus were visible.
  

3.2RC2

• Fixed a long-standing issue with command-line configuration in the jCarafe tokenization wrapper.
• Fixed a bug in processing the UI settings layoutRightPanelWidth and layoutBottomPanelHeight introduced when the UI setting handling was updated.
  

3.2RC1

• Fixed a documentation typo.
• Updated to jCarafe 0.9.99.
  

3.2pre30

• Added custom_editor_multiattributes to improve handling of unset in custom editor attribute cases in the UI.
• Finished the implementation of writeable_only and attribute_writeable_only from 3.2pre29.
• Fixed a bug with the new rendering_style attribute where background spans were blocking autotagging.
• Enhanced control over choose mode to support restricting choices to either selection or creation.
• Fixed a long-standing lurking bug in the UI where copying attributes with undefined values in autotagging caused an unrecoverable error.
• Added a function to check if a reconciliation document has been modified at all.
• Fixed a bug in the creation of comparison and reconciliation documents where documents with identical labels caused fatal confusion.
  
• Updated documentation.

3.2pre29

• Added writeable_only and attribute_writeable_only to the <gesture> element to control when menu gestures can be presented (not implemented yet in the UI).
• Improved the code that simplifies the JSON version of the task, and updated the documentation.
• Cleaned up some lurking bugs where workspace database connections were not being closed, which can cause problems on Windows.
• Added the <d_gesture> element to the newer annotation definition method, to mirror the <gesture> element of the legacy method.
• Added a UI logging action for tab selection.
  
• Various documentation updates.
  

3.2pre28

• Fixed a small logic issue in excluding annotation categories from sets.
• Cleaned up the logic in the jCarafe tokenizer to ensure that zoning is not required.
• Fixed an obscure bug in the pairing algorithm where custom pairing methods defined in tasks could not be found.
• Fixed a bug in the UI where similarity profile information wasn't available for reconciliation documents in workspaces.
• Fixed a bug in the UI where custom workspace folder configurations were not managed correctly.
  
• Updated the reconciliation documentation.
• Fixed some errors in the advanced workspace documentation and improved the documentation related to workspace logging.
• Fixed a lurking typo which prevented the UI from recognizing that workspace logging was enabled.
• Fixed a bug in the UI where modified workspace documents were not being marked as modified in the UI.
• Fixed a bug where workspace logging was assumed by the UI to save UI logs with each save in any folder, but was only supported for the "core" folder.
• Fixed a bug in workspace logging where reconciliation changes were not being captured in incremental document fragments.
• Fixed a long-standing bug in how user assignment in workspaces is handled.
  

3.2pre27

• Updated documentation to better cover issues related to label order in the UI.
• Improved error display in the UI when errors are encountered while tasks are being loaded.
• Added documentation search.
• Fixed a tiny typo in the document conversion utilities.
• Fixed a tiny typo in the UI code.
• Fixed a bug in the reconciliation UI where system default similarity profiles were not available to the interface.
• Fixed a bug in the reporting of errors resulting from stratum advancement in the reconciliation UI.
• Enabled adding annotations in the reconciliation UI, and refactored and simplified how hand annotation availability is managed in the reconciliation UI.
• Changed logger gesture_type to gesture_method, and the modify_annotation event to modify_attribute, in preparation for a larger reorganization of annotation events.
• Added gesture_category and gesture_is_auto to logging.
• Applied label styling to table header in reconciliation.
• Fixed an obscure bug which blocked some relation link tooltips from being removed when an already-linked annotation is selected as an argument for another annotation.
• Completely refactored how annotation gestures are handled in reconciliation.
• Reorganized the buttons in the reconciliation table to make them more concise.
• Fixed a bug which was blocking edit_immediately from being respected during annotation in the reconciliation UI.
• Completely reorganized the logging in reconciliation to improve parallelism with normal annotation.
• Fixed a bug where offsets were not present in reconciliation log entries.
• Reduced the size of the documentation images, and eliminated the duplicate static documentation, to reduce the size of the MAT distribution.
• Added the <copy_metadata> document conversion operation, and the <discard_annot> operation within the <values> scope.
• Cleaned up the semantics of attribute value selection in the document conversion.
• Updated documentation to better explain the <display_group> element in the annotation descriptors.
  

3.2pre26

• Fixed a couple tiny lurking bugs in computing pathnames for saving in the UI.
• Made it possible to change boolean attributes in the UI using the popup menu, in the same way that choice attributes can be changed.
  

3.2pre25

• Finished improvements for the rendering_style attribute.
• Improved thoroughness of validate operation for MATManagePluginDirs, and blocked installation if Python errors are encountered.
• Expanded regular expression support in the document conversion tool, and added the ability to use regexp backreferences in the targets.
  
• Small documentation updates.
  

3.2pre24

• Refined the default handling fix.
• Added offsets to the UI log for cases where annotations are affected.
  

3.2pre23

• Fixed a long-standing lurking bug in the Python MAT-JSON reader which wasn't handling defaults appropriately. This bug can only arise in MAT-JSON implementations which implement task definitions, so Java doesn't exhibit it.
  

3.2pre22

• Fixed a long-standing lurking bug in the scorer involving zero-length spanned annotations.
• More preparations for the latest version of jCarafe.
  

3.2pre21

• Fixed some error messages.
• Improved (but did not completely fix) the behavior of non-tokenized wrapping in the face of annotations which wrap across line breaks.

3.2pre20

• Added support for new REST interface to the latest version of jCarafe.
• Other preparations for update to the latest version of jCarafe.
• Removed optional tabbed terminal packages from the distribution. Added the ability to specify your own tabbed terminal invocation script, provided an example.
• Fixed a couple of typos.
  

3.2pre19

• Fixed two subtle bugs in selecting annotations to modify their extent when the swipe covers more than one in-line annotation.
• Backed out the change in 3.2pre16 and replaced it with the more principled management rendering_style attribute.
• Fixed a bug in XML inline input where non-annotation-valued attributes were not being added soon enough to annotations which serve as values of annotation-valued attributes.
  

3.2pre18

• Fixed two bugs in the ability to modify chosen annotations in reconciliation.
• Fixed a bug where under certain circumstances in comparison and reconciliation, a popup menu was presented after a warning that hand annotation was not available.
  

3.2pre17

• Fixed (hopefully) some very subtle bugs introduced by the enhancement in 3.2pre16.
  

3.2pre16

• Added the ability to force a CSS class on an otherwise unstyled and unannotatable annotation.
  

3.2pre15

• Made it possible to arbitrarily modify chosen annotations in reconciliation.
• Made a number of internal improvements to the document display to better support task customization.
• Fixed an omission where the multiple choice strings in the annotation editor lacked an Unset button.

3.2pre14

• Small improvement to the Python library for importing annotations from one document to another.
• Small improvement to UI to support customization of annotation tables.
• Enhanced distribution build script.
  

3.2pre13

• Added UI settings to control the height and spacing of stacked annotations.
• Internal cleanup of tagger server infrastructure code.
• Added functionality to load MATWeb startup settings from a file, and to install default startup settings.
• Fixed a bug in trimming and expanding the right edge of tokenless swipes.
• Improved an error message related to generation of comparison and reconciliation documents.
• Added the ability to specify a sort function for the annotation tables for string attributes.
  

3.2pre12

• Added a flag to enable the annotation table summaries (now disabled by default, for visual backward compatibility).
• Added the ability to show a column for each attribute of each label in annotation table summaries, and a flag to control it.
  

3.2pre11

• Updated the documentation for MATWeb a bit.
• Improved a bit of error handling in PluginMgr.py.
• Enhanced the subprocess control to enable steps to import attribute values alone when their annotation hosts can be paired.
  

3.2pre10

• Refactored code internal to managing automated tagging steps to support greater flexibility.
• Refactored document reader internals to support reading from non-Unicode sources.
• Fixed a long-standing bug in the testing library where temporary directories for MAT invocations were not being retained appropriately when requested.
  

3.2pre9

• Added the ability to declare a default value for an effective label attribute in the new annotation definition format.
  

3.2pre8

• Enhanced the installer to warn and give the option of quitting, instead of just quitting, when jCarafe is included but Java isn't found on the installation machine.
  

3.2pre7

• Fixed two bad bugs in the keyboard accelerators in the UI: non-visible non-ambiguous accelerators could be triggered, and ambiguous accelerators present in the same nesting level of multiple annotation cascades were confused with each other.

3.2pre6

• Fixed a bug in the new documentation sidebar view tracking.
• Added the --detail_context_window and --normalize_detail_whitespace options to MATScore.
• Added the --normalize_whitespace option to MATReport.

3.2pre5

• Enhanced the Python library to support iterative import of annotations from one document to another.
• Cleaned up the implementation of choose mode annotation selection in the UI.
• Cleaned up the implementation of custom editors in the UI to make it a bit less confusing.
• Added better feedback in the documentation sidebar about which page is being viewed.
• Improved the annotation popup menu so that it doesn't propose to replace a selected annotation with the same label, or provide the "Repeat" option when an annotation is selected.
• Added the ability to display spans for annotation attribute values in MATReport.
• Fixed a long-standing bug in the UI where very long annotation descriptions would run off the bottom of the hover line below the annotation palette.
  

3.2pre4

• Made it possible to define task-specific readers in the MAT UI (but not in the MAT engine itself yet).
• Made it possible to attach a document convertor to a reader in a particular task, in addition to document conversion instruction sets.

3.2pre3

• Added UI settings to trim and expand tokenless swipes according to the tokenless autotag delimiters.
• Fixed a bug which was failing to trim tokened swipes down to the nearest token boundaries.

3.2pre2

• Fixed a sneaky bug which caused spanless annotations without display properties to break the document display.
• Added summary tables to annotation table tab (all spanned, all spanless).

3.2pre1

• Fixed a bug where overlap order was not captured by MATAnnotationInfoToJSON.
• Enabled specifying tokenless autotag delimiter behavior in standalone UI.
• Added more sophisticated test harness for standalone UI.
• Added capability for enhanced autotagging (delete matches, relabel matches), controlled by a UI setting.
• Fixed a sneaky bug where random complex annotation actions failed in the standalone UI because of an inappropriate variable reference.
• Refactored annotation popup creation to generalize and isolate the item options, including being able to have multiple types of annotation gestures in the same popup and to introduce broader options for submenus.
• Expanded specification of the expanded JSON generated by MATAnnotationInfoToJSON to enable forwarding of language information from the XML task.
• Added enhancements to allow for more extensive customization of annotation panels for specific tasks.
• Enabled the ability to include annotation tables in the standalone UI.
• Modified the getDocument() method of the standalone UI to be compatible with other annotation panels.

3.1.1

• Fixed a bug in XML input handling where self-closing tags were being incorrectly processed for XML overlays.
• Fixed a bug in XML output handling where XML tag exclusions were being ignored if they were specified as a Unicode string.
• Fixed a bug where modifying elements during reconciliation wasn't properly checking to advance the reconciliation stratum.
• Fixed a bug where guided navigation advancement wasn't carefully enough handled when guided navigation is available but not active.
• Fixed a bug in the MAT UI where setting annotation attribute values to undefined was causing an error (rather than removing the value).
• Fixed a couple small bugs in the join_attrs document conversion XML instruction.
• Fixed a typo in an error handling routine in the MAT UI.
• Added a mistakenly omitted class of custom attribute editor in the MAT UI.
• Fixed two obscure bugs in the MAT UI that arose when the user attempted to modify the extent of an annotation when the swipe covered multiple non-overlapping annotations.
• Fixed several bugs relating to how tasks inherit labels from their parent tasks.
• Fixed a bug in command-line processing for MATExportReconciliationDocument.
  
• Improved the documentation for the UI settings.
• Fixed a bug in generating comparison documents when not all the annotation-valued attributes are being compared, and improved the error handling for the corresponding reconciliation case.
• Added the similarity profile to the information panel in the UI for comparison and reconciliation documents.
• Added a warning in the documentation about two unfixable UI bugs in 3.1.1 related to the interaction between keyboard accelerators and hierarchical menus. These bugs will be fixed in the upcoming 3.2 release.
• Fixed a bug in workspaces where workflows that had no initial automated steps would break when documents are imported.
• Fixed a bug arising from a typo in the code supporting the new form of defining effective labels.
• Fixed a lurking bug in a seldom-used portion of the tokenizer wrapper.
• Fixed a bug in the jCarafe wrapper where engines that support multiple languages failed to create servers for any of the languages.
• Fixed two bugs relating to the generation of comparison and reconciliation documents.
• Fixed a subtle bug in the UI where redisplaying the spanless sidebar while tooltips were displayed would sometimes strand a tooltip.
• Fixed an omission in the installer which was incompatible with version strings for OpenJDK 8.
• Cleaned up some stale documentation.
  

3.1

    No changes.

3.1RC3

• Fixed a couple of tiny bugs related to the new inferred task capability.
• Added --content_annotations_all to MATScore.
• Fixed a bug which was blocking export of reconciliation documents to a file.
  
• Fixed a typo in the documentation.
• Updated the documentation about the meaning of auto-advance in reconciliation.
• Fixed a bug where CherryPy 3.2 was presenting Unicode text to the server when byte sequences were expected.
• Fixed a bug where exporting reconciled documents to a buffer was failing.
• Improved packaging scripts.
  

3.1RC2

• Added warnings about loading tasks to the documentation.
• Fixed UI warning about not finding any tasks.
  

3.1RC1

• Introduced guided navigation mode.
• Introduced more compact annotation declaration alternative.
• Introduced crossvalidation in the experiment engine.
• Introduced explicit overlap management, including new "Bring to front" operation for overlapping annotations.
  
• Reorganized the reference and use cases in the documentation for annotations and their display properties.
• Updated sample tasks to use the new annotation declaration method, and modified MATReport to generate this format.
• Introduce the ability to infer tasks in the UI, edit them, and save them.
  
• Extended the internal XML templating mechanism to support more concise declaration and processing of new annotation declaration alternative.
• Added the ability to store UI settings as cookies, and as XML files, and to reload the settings from an XML file.
• Added a setting to control whether annotations are stacked in the UI when annotation isn't available, and changed the default from non-stacked to stacked.
• Annotation mouseovers in the comparison view now show, for non-reference annotations, whether the annotation matches the reference or not.
  
• Fixed a subtle bug on the Mac where not all the UI elements had font fallbacks defined.
• Fixed a bug where the showSegments UI setting was not being respected appropriately.
• Fixed a bug where the repeatAnnotationAccelerator UI setting could get out of sync with the annotation popup menu.
• Added a missing UI setting to the XML template which governs them.
• Fixed a bug where unnamed similarity profiles weren't being communicated properly from the UI to the backend for comparison and reconciliation.
• Fixed a bug where moving from one stratum to the next in reconciliation documents in the UI was failing because the backend was leaving unserializable objects in the document metadata.
• Fixed a bug where annotation editors weren't smart enough to know when annotation-valued attributes weren't editable due to the step the document is in.
  

3.0.8

• Fixed a typo in generating IDs for radio buttons in the annotation editors in the UI.
• Fixed a bug in formatting descriptions of complex restrictions on annotation attributes in the UI.
• Fixed a bug where changing a choice attribute in the UI annotation editor fails to maintain the correct value when the requested value is forbidden by the annotation to which the current annotation is linked.
• Fixed a bug where if an encoding was unknown, MATTransducer and MATEngine would create empty files in the course of reporting the encoding error.
• Fixed a bug in describing write errors in MATTransducer.
  

3.0.7

• Fixed a bug where, in the UI, if a step can add multiple annotation sets and elements of only some of those sets had actually been added, undo was failing in undoable workflows.
• Fixed a bug where, in the workflow engine, if a step can add multiple annotation sets and elements of only some of those sets had actually been added, undo was ignored in undoable workflows.
• Added the <force_id> document conversion instruction.
• Ensured that document conversion preserves annotation IDs when possible.
• Fixed some documentation errors.
• Fixed a bug in task installation where the state of the module inventory was being preserved in an inappropriate way.
  

3.0.6

• Fixed a bug where keyboard accelerators for unavailable labels were still available when creating arguments for relations.
• Enabled context-sensitivity for keyboard accelerators. If the accelerator is used in multiple places, and one of them is visible, that's the one that will be used.
• Fixed a small bug in the XML inline reader.
• Fixed a small bug in the standalone viewer where the legend was being populated whether or not a div for it was defined.
• Small improvements to error messages for locked documents in workspaces.
• Added UI setting to automatically generate numbered accelerators for linking actions to an active annotation editor in the annotation popup.
• Repaired some omissions in the UI settings XML configuration defaults.
  
• Fixed a documentation link.
  

3.0.5

• Added the _remainder dimension to similarity profiles.
  

3.0.4

• Modified more Javascript APIs to better support extensive task customization.
• Fixed a subtle UI bug that in some circumstances was interfering with draggability of popup panels.
• Fixed a UI bug where choosing an annotation in choose mode via the annotation tables would bypass the label restriction checking.
• Fixed a UI bug where popup menu coloring to reinforce slot fill and choose mode elements was not being displayed correctly.
  

3.0.3

• Fixed a bug in the similarity profile selection in the UI for comparison and reconciliation.
• Forwarded task settings to the UI to use in customizations.
• Added tokenless_autotag_respects_delimiters to the <language> element in task.xml to support autotag matching in languages which don't use whitespace as a delimiter.
  
• Added utility in Javascript API to retrieve an attribute object.
• Fixed a bug which made it impossible to provide custom gestures to effective labels by attaching them to the true label in the annotation display in the task.xml file.
• Fixed a bug where choice attributes marked read-only in the annotation display in the task.xml file were still being made available for modification in the annotation popup menu.
• Modified some internal Javascript APIs to better support extensive task customization.
  

3.0.2

• Fixed a small bug in the XML reader.
• Fixed a small bug in the code which digests annotation steps from task.xml.
• Added the --half_verbose option to the align workflow step.
• Fixed a subtle bug in the align workflow step which failed to do the right thing when several content entities were perversely subsumed by the same token.
  

3.0.1

• Addressed an issue where Firefox no longer permits alert() in the hook that runs before the Web page is closed. There's no longer any way in Firefox to provide useful information when there's data to be saved; Safari still presents the informative message.
• Fixed a subtle bug in "Autotag matches".
• Fixed a subtle bug in how annotation tables are initially displayed when table viewing is enabled and a file is loaded.
• Ensured that the order of declarations in <annotation_display> in task.xml is the order in which the CSS rules are defined, so you can reliably have paragraph-extent elements appear "behind" mention-extent elements in non-stacked annotation viewing.
  

3.0

• Fixed a debugging setting in subprocess management.
  

3.0RC16

• Enhanced the experiment engine to support application of multiple models in experiment runs.
• Set up annotation pairer to automatically construct complex default stratum lists for similarity profiles (previously, defaults attempted to use a single stratum)
  
• Fixed an subtle bug in the UI which caused an error when displaying spanned annotations with annotation-valued attributes in the standalone viewer.
• Improved an error message relating to clashing attribute datatypes.
• Fixed a bug in MATManagePluginDirs which permitted multiple tasks with the same directory basename to be registered, even though this would cause an error when the remainder of the system is run.
• The MAT UI now ensures that strings with HTML-significant characters are displayed properly throughout the UI.
• The experiment engine no longer requires a model for each run.
• The UI splash screen now appears before tasks are retrieved from the backend, and reports task retrieval status, for those cases where task load takes a long time.
• The UI for set-valued string and int choice attributes has been improved. By default, it's now a multi-select of size no more than 5; above this size, the multi-select will scroll. The size of the multi-select is controllable in task.xml via the max_choice_size attribute of the <attribute> element in <annotation_display>.
  

3.0RC15

• Fixed a subtle bug with determining the appropriate annotations to insert into strata for pairing when annotation-valued attributes are defined in a separate annotation set.
• Removed the test, introduced in RC14, which ensures that a similarity profile must mention a label in its strata if it's mentioned in the tag profile (tag profiles are intended to be applicable in these cases, if, e.g., the step being scored only scores one of the labels).
  

3.0RC14

• Ensured that MATManagePluginDirs does the right thing when the task has settings.
• Fixed a bug in the experiment engine where the chosen trainable step was not being recorded appropriately for the models.
• Fixed a bug where trainable steps with spaces in their names weren't capturing their command line arguments properly in MATEngine.
• Fixed a bug where details spreadsheets in the scorer weren't respecting the restrictions on labels to be presented.
• Extended new --fresh_task command line option to the build settings in the experiment harness.
• Ensured that requesting unknown similarity or score profile will generate an error.
• Fixed a bug where the experiment engine was reporting scores for too many labels on its initial run.
• Fixed a long-standing bug where the experiment engine did not report corpus training statistics during rescoring.
• Fixed a bug where the experiment engine reported the wrong training statistics for effective labels whose attribute wasn't available in the step for which the model was trained.
  
• Forced the scorer to ignore annotation labels marked as not trainable; changed the designation to "processable" instead of "trainable".
• Ensured that the scorer would fail if a similarity profile mentions a label in its tag profile which is not mentioned in its strata.
  

3.0RC13

• Fixed a subtle UI annotation bug where in some cases, the UI bookkeeping failed when the annotator attempted to change the extent of a span annotation in tokenless annotation mode when the right endpoint of the annotation wasn't intended to change.
• Fixed a similar conceptual bug which affected autotagging.
  
• Optimized annotation palette to omit bookkeeping information for annotations which have no display information at all.
  

3.0RC12

• Compensated for a subtle bug in jCarafe training where attributes in addition to the effective label attribute being trained for was causing the trainer to use the wrong feature values.
• Forced writing documents to stdout to write encoded byte streams.
• Added the ability to mark annotation labels as not trainable.
• Added the --fresh_task option to MATTransducer and MATModelBuilder.
  
• Introduced a patch for CherryPy 3.1.2 which fixes a bug with proxying CherryPy behind Apache.
• Improved error handling for MATModelBuilder.
  
• Documentation updates.
  

3.0RC11

• Further refinements to issues with label groups.
• Fixed a small bug which prevented legend menus from working with effective labels.
• Added "validate" action to MATManagePluginDirs.
• Changed the default encoding for raw documents to UTF-8.
• Modified the UI file loader to try to infer the file type from the file extension.
• Fixed a subtle bug where annotations with attributes were still able to be replaced in the UI steps that weren't supposed to permit it.
  

3.0RC10

• Fixed a bug where label groups were not shown in the proper location in annotation popup menus when alphabetize_labels is "no".
• Corrected an error in the documentation where the element that determines the scope of annotation order was misidentified.
• Fixed a bug where the presence of label groups caused "Selected" to behave incorrectly in the annotation label control pane.
• Fixed a bug where known annotations were mistakenly treated as label groups in situations where label groups were defined but the annotations weren't relevant to the current situation (e.g., spanless annotations in the span annotation pane, or spanned annotation in the spanless sidebar).
• Fixed a bug where empty label groups weren't filtered out in the annotation popup menus.
  

3.0RC9

• Fixed a couple of very obscure bugs with reading and writing unknown annotations to XML.
• Added a UI setting to control the default handling of unknown annotations in XML inline input.
  
• Tiny bug fixes.
  

3.0RC8

• Fixed a bug where default values for float and boolean attributes were not being copied when the attribute properties were copied.
• Fixed a bug in the experiment engine where the tag subset column was not being copied from run spreadsheets to experiment summary spreadsheets.
  
• Fixed a small bug in computing the annotation attribute remainder in the scoring algorithm.
• Made it possible to use the reserved name <matdefault> to force access to the MAT default score and similarity profiles. Added this capability to the UI as the "(system default)" option.
  
• Made a subtle change to the pairing algorithm; annotations which are paired but have no similarity previously were reported as missing and spurious, and now they're reported as clashing.
  
• Added <attrs_alone> option to score profiles to enable viewing scores for individual attributes.
• Fixed a subtle bug relating to the interaction between comparison documents and the UI procedure that digests documents from the backend server.
• Fixed a couple interrelated bugs relating to how similarity profiles are reported to the UI at load time.
  
• Updated the standalone viewer to handle modified format for comparison documents.
  
• Documentation updates.
  

3.0RC7

• Fixed a small bug where document start and end weren't being recorded properly in the UI redisplay.
• Fixed a small bug where deleting an annotation at the end of the document would fail in the UI.
• Fixed a typo in the sample relation tagger code.
• Fixed a small bug in MATModelBuilder where omitting the task argument blocked the possibility of displaying the help string.
• Fixed an error where undoing steps that add annotation attribute values wasn't restoring the original defaults which were set when the annotation was created.
• Fixed an accompanying issue where defaults were not being ignored when the engine determined whether a step was clearly underway or not.
  
• Fixed a bug where changing the value of an effective attribute in the UI to an invisible label and then changing it back failed to redisplay the annotation.
• Fixed a number of bugs in the scorer relating to attributes defined in an annotation set other than the set of the label which bears them.
  
• Updated XML reader/writer docs to emphasize that annotation and attribute names are not modified when XML is created, even if the result will be ill-formed.
• Added the --debug option to MATScore.
• Minor enhancements to detail decomposition of scoring clashes in score output.
  
• Various documentation updates.
  

3.0RC6

• Removed a few Javascript syntax errors which arose only in Internet Explorer (it still doesn't work in IE, but it's closer).
  
• Improved error reporting for type collisions for annotation-valued attributes.
• Fixed a bug where label-specific spanless annotation counters were incremented incorrectly in the MAT UI.
• Fixed a tiny display bug in the UI triggered when display elements in a task provide an empty CSS attribute.
• Fixed infinite recursion in describing annotations for MATReport.
• Fixed a bug in triggering rescoring in the experiment engine.
  
• Small improvements to the redistribute script.
  

3.0RC5

• Fixed a small bug in creating set and list attribute values in taskless contexts in JavaScript and Python.
• Fixed a bug in presenting the scoring results of equivalence classes.
• Other minor bug fixes.
  

3.0RC4

• Fixed a small bug in the scorer involving what happens when confusabilty can't be generated.
• Fixed a bug in the UI where the <gesture> element wasn't encoding URL link components properly.
• Changed MATReport so it defaults to non-administrative annotation sets, rather than hand-annotatable annotation sets.
• Added command-line options to the align step.
  

3.0RC3

• Minor bug fixes.
  

3.0RC2

• Modified task updater to write UTF-8 XML files.
• Added "Mark gold, save & close" operation to workspace UI.
• Added command-line arguments to MATCreateComparisonDocument.
  
• Minor enhancements to Java library.
• Generalized choose mode access in annotation popup menus to include filling a slot in addition to attaching to a parent. Introduced UI setting to optionally suppress those actions.
• Expanded the use of color feedback in annotation linking.
• Ensured that the order of annotations listed in the dropdown menu in the annotation table tab is the same as the order in the legend.
• Ensured that the spanless sidebar will not appear if the task has no visible spanless annotations.
  
• Other tiny bug fixes.
  
• Documentation updates.
  

3.0RC1

• Fixed a bug in the UI where documents which were previously annotated and then undone would not be registered appropriately as being partiallly done when reannotated.
• Fixed a bug in the UI which was blocking annotations with attributes from being eligible for UI autotagging.
• Implemented faster localized redraw for hand span annotation.
  
• Refactored the implementation of UI autotagging to render it more compatible with faster redraw.
• Fixed a UI bug where annotations in the document which were unknown by the task were causing an error in rendering the annotation tables.
• Added legend menu controls, including selection, active/inactive, and visible/invisible.
  
• Fixed a bug which arose after localized redraw was implemented, where the wrong method was being called to trigger the update of file status in workspaces.
• Fixed a bug in reconciliation and comparison windows where selecting an annotation simultaneously presented an error and a popup menu.
• Fixed a couple small bugs in the standalone editor.
• Fixed a subtle series of bugs in the UI workflow processing relating to switching workflow, multiple sets, and set modification.
• Fixed a subtle bug where gold documents in the workspaces were skipping the following step when they were superfluously marked gold on import.
• Fixed several small bugs in the exporting of reconciliation documents in the UI.
  
• Updated logging calls.
  
• More documentation updates.
  

3.0pre5

• Fixed a number of bugs related to models in the tagger server: server now restarts when the model is newer than the last request, server now delays checking for models until a request is made, server now makes all specified models potentially available, regardless of whether the model exists at the time of server startup.
• Fixed a bug in the MAT server where default models for tasks with multiple languages enabled was not handled correctly.
  
• More documentation updates.
  

3.0pre4

• Added ability to modify choice attributes from annotation popup.
• Removed deprecated distinguishing_attributes_for_equality from task.xml file.
• Fixed more bugs which allowed runtime override of settings configuration.
• Fixed set modification in the UI.
• Fixed a bug in the workspace and experiment engine where workspace corpora remainders were not handled properly.
• Improved error reporting for computing model builder information.
• Fixed a bug in the experiment engine where trainable steps were not being passed appropriately to the model builder.
• Fixed a bug in the experiment engine where the scoring wasn't limiting itself to annotation sets associated with the trainable step.
• Fixed a bug in computing corpus statistics where spanless annotations were mistakenly consulted for token-level statistics.
• Added the option of marking gold without advancement in the workspace UI.
• Extended Java API to retrieve annotations in a given interval.
  
• Fixed a bug in creating SEGMENTs when there are no zones.
• Fixed a bug in workspaces where adding assigned documents to a basename during import didn't ensure that the signals matched.
• Added ability to customize crossvalidation minimum threshold in workspace configurations.
• Customized behavior of transform steps in the UI.
• Fixed a bug in UI retreat into a previous hand annotatable step where the document description was not being recomputed.
• Fixed a bug in the UI where content categories were not being identified appropriately.
• Fixed a bug in the UI where untaggable annotations weren't being added in the right circumstances.
• Fixed a bug in UI workspaces where hand annotation was not changing the status information.
• Expanded test suite.
  
• More documentation updates.
  

3.0pre3

• Fixed a host of tiny bugs related to workflows which contain both set addition and set modification steps.
• Eliminated the ability to move to document reconciliation from document alignment in the UI.
• Fixed a bug in the settings configuration which permitted PYTHONBIN to be overridden at runtime.
  
• Initial documentation updates.
  

3.0pre2

• Enhancements to subprocess invocation.
• Fixed a bug where the "Open File" dialog wasn't activating its menus when only a single task was installed.
  

3.0pre1

• Massive overhaul of tasks and steps. Introduced explicit engines, multiple languages. Added multi-step mixed initiative.
• Added new multi-step sample task, plus crude relation tagger to illustrate capabilities.
  
• Enabled multiple workspace configurations in tasks, and the ability to build a workspace out of any hand-annotatable workflow.
  
• Incorporated reconciliation and review into workspaces.
  
• Removed the Web demo.
  

2.1pre10

• Refactored annotation popup menus so that actions for selected annotations appear in more locations.
• Added the <gesture> element for labels for annotation displays, to define arbitrary functions for selected annotations of particular label types.
• Fixed a bug where annotations which point to, or are pointed to by, the same annotation in multiple ways were not being highlighted correctly in the UI.
  

2.1pre9

• Fixed a UI bug where annotations which pointed to multiple annotations couldn't be deleted cleanly.
  

2.1pre8

• Minor bug fixes.
  

2.1pre7

• Improved placement of spanless anchors in sidebar.
• Added customizations for installer.
• Added support for CherryPy 3.2.
• Fixed a bug where a temp directory wasn't being managed correctly in subprocess invocation.
• Added the TMPDIR_ROOT configuration setting.
  

2.1pre5

• Fixed a scoring bug where identical relations, each with missing arguments, were not matching.
  

2.1pre4

• Significant improvements to redraw speed for densely annotated documents.
  

2.1pre3

• Minor bug fixes in the standalone viewer.
• Minor documentation updates.
• Improvements to reconciliation tool.

2.1pre2

• Added a setting for controlling the default annotation editor popup width.
• Added a setting for controlling the default font size percentage in the body of the annotation editor.
• Added a setting for controlling the default font size percentage in the annotation tables.
• Added a setting for controlling the the initial width of the right-hand pane in the workbench.
• Added a setting for controlling the initial height of the right-hand pane in the workbench.
• Added a setting for extra line spacing in the text pane.
• Added the notion of an active editor.
  
• Added a setting to suppress the usual menu actions (e.g., create/replace/delete annotations) while annotation editors are open.
• Added menu items in the annotator menu popup to fill the annotation-valued attributes of an open annotation editor.
• Added ability to add URL and function gestures to the annotation popup menu.
• Fixed some bugs in the implementation of reconciliation advancement from stratum to stratum.
  

2.1pre1

• Fixed some bugs in the initial implementation of single-stratum reconciliation.
• Implemented general export/load for reconciliation documents.
• Added new capability for maintaining UI settings.
• Modified choose mode to require explicit exit for set- or list-valued annotation attributes.
• Added a setting for less verbose annotation editors.
• Added a setting to control where the annotation editor popups appear by default, and changed the default position to upper right.
  
• Added settings for modifying the default keyboard accelerators for repeating and deleting annotations.
• Added a setting for controlling whether link highlighting happens during annotation hover.
• Added UI setting management to the standalone viewer.
• Fixed a couple small bugs in the UI.
  

2.1alpha1

• Initial implementation of single-stratum reconciliation.

2.0.6

• Fixed a small documentation error.
• Fixed a bug which blocked reconciled documents from being exported to a buffer.
• Fixed a bug which blocked reconciled documents from being exported to a file.
• Fixed a bug where auto-advance couldn't be turned on in reconciliation if "Export to file" was set.
• Fixed a bug where enabling auto-advance in reconciliation would cause an infinite loop if there were no segments to reconcile.

2.0.5

• Fixed a small bug in the experiment engine where scores were not being collected or saved in some cases.

2.0.4

• Fixed an ordering bug in MAT-JSON deserialization in Java, Javascript and Python where annotations which both have annotation-valued attributes and are values of other annotation-valued attributes might not have their non-annotation-valued attributes decoded in time to satisfy the restrictions of the attribute they're a value of.
• Updated build script to update version number in JAR manifest.
• Fixed a couple small bugs in MATAnnotationInfoToJSON.
• Fixed a bug where the incorrect content type was being reported to the browser on UI document save, causing an incorrect file extension to be added in Safari.
• Fixed an suboptimal behavior where XML inline output was producing extraneous (but harmless) toplevel XML tags to ensure XML well-formedness.
• Expanded the options available in the UI for the xml-inline reader, and improved the associated documentation.
  

2.0.3

• Fixed a bug in the Java core library where annotation IDs were being processed incorrectly.
  

2.0.2

• Fixed a bug where right-to-left languages were incorrectly justified.
• Compensated for a bug in YUI which caused buttons to be attached at unreliable times, making it impossible to center the editor popups vertically.
• Fixed a small bug in the pairer where the similarity comparison was returning incorrect values in one specific case.
• Removed duplicate definitions of inChooseMode() in the standalone viewer.
• Compensated for a z-index bug in YUI which causes annotation menu popups to appear behind annotation editor popups in some cases, and for newer popups to appear behind older ones.
• Compensated for a series of YUI bugs which caused menu scrolling to behave strangely in annotation menu popups which exceeded the height of the viewport.
• Fixed a bug where menu scrolling in long annotation menu popups was too slow.
• Disabled the the "Unset" button in annotation editors for attributes in choose mode, to avoid mischief.
• Fixed a bug where pressing the close button on a tab while the user is being asked whether to reload or close would cause an error.
• Fixed a bug where it was possible to close tabs while waiting for a step result from the Web server.
  
• Made all question dialogs modal.
• Fixed bug in keypress handling where modern browsers were no longer sending what we previously expected for <Enter>/<Return>. Also changed <Tab> to "-" for deleting annotations because of cross-browser incompatibility issues.
• Fixed a bug in the UI where annotations without any display entry could be selected by clicking.
• Fixed a bug where some of the gestures in annotation context menus in annotation editors were not bound correctly.
• Fixed a number of bugs relating to how assigned files are processed from the UI.
• Fixed a bug where setting MAT_PKG_HOME as an environment variable possibly caused MAT to use the wrong value internally.
• It is no longer possible to open the same file more than once in the workspace UI. Previously, it was possible, but the older buffers couldn't be saved or operated on.
• Enabled busy wait image for workspace folder refresh in UI.
• Minor additional improvements to workspace panel file information display in UI.
• Fixed a bug where the UI panel which displayed unexpected errors from the backend was not draggable.
• Fixed a bug in the standalone viewer where overlapping annotations weren't being stacked on initial display when hand annotation was enabled.
• Fixed a bug where tokenized documents containing annotations whose indices aren't aligned with the tokens caused a failure in collecting corpus statistics during model building.
• Added version information to Java JAR manifest.
• Fixed a small bug in UI autotagging involving matching a pattern which doesn't fall on token boundaries.
• Fixed a bug with error handling in toolchain execution in workspaces.
  
• Minor documentation updates.
  

2.0.1

• Fixed a bug in the pairer which was failing to deal with spanless annotations when there was no task.
• Removed an outdated reference to --mark_gold_standard_reference in the MATWorkspace documentation.
• Fixed a typo in the toolchain which would cause an error in the case where the task has exactly one workflow.
• Fixed a typo in a Windows command in the MATEngine documentation.
• Improved documentation about the --lexicon_dir option to jCarafe.
• Fixed a bug where edit_immediately was not being respected for effective labels.
• Fixed a bug where true labels could not be used as a shorthand for effective labels for adding annotations in choose mode.
• Documented the MAT runtime environment variables and added a common command-line option to report where they're read from.
• Fixed a bug where the display properties of annotations with effective labels were not consistently updated when the effective label attribute value changes.
• Fixed an error in the MAT UI in segmenting documents which contain lines at the end which are wrapped.
• Fixed a bug in the MAT UI where highlighting an annotation's parents or children was mispositioning the attribute name tooltip.
• Fixed a bug in the MAT UI where the attribute name tooltips were mistakenly permitted to scroll off the right side of the annotation pane.
• Fixed a bug in the MAT UI where the annotation editor popup max size was not being computed correctly.
• Fixed a bug in the MAT UI where annotation editor popups, if moved, would jump back to the center of the viewport when the viewport was resized.
  

2.0

• Undid Java 1.7 requirement.
• Fixed a small bug in the toolchain where jCarafe was being executed even if there were no documents to tag.
• Disabled copying annotation-valued attributes during autotagging in the UI.
• Removed some old code from the Java MAT engine client API.
• Final documentation tweaks.
• Fixed a series of Windows bugs relating to situations where the workspace DB was not appropriately closed.
• Added access to score and similarity profiles to experiment XML.
  

2.0RC14

• Upgraded to jCarafe 0.9.8.5b-06.
• Raised Java requirement to 1.7.
• Fixed a typo in MATReport.
• Made the MAT-JSON Python reader a bit more forgiving.
  

2.0RC13

• Fixed a bug in MATTransducer where the record of steps applied to the document was not retained when a convertor was provided.
• Fixed a comparable bug in the reader/writer subsystem.
• Fixed a bug in the document reader where compound steps weren't being consulted correctly when inferring the steps applied to a document.
• Fixed a bug where the tokenizer was not anticipating the unlikely case where a document had been tokenized but the record of steps applied did not mention the tokenization.
• Fixed a bug where the scorer was failing to use the new comparison mechanisms for tokens.
• Fixed a bug where comparison sets in the scorer created because of transitive overlaps could have resulted in pairing annotations which don't overlap at all.
  

2.0RC12

• Fixed a subtle bug in the experiment engine which arose when labels which are present in the training corpus are not present in the test corpus.
• Documentation formatting tweaks.
• Small enhancement of document mapping to preserve XML formatting.
• Partial updates for possible future support of Internet Explorer.

2.0RC11

• Ensured that annotation editor titles don't grow without limit.
• Ensured that annotations without CSS display instructions aren't rendered in any way.
• Fixed a bug where it was possible for documents in the standalone viewer to lack a global annotation type repository.

2.0RC10

• Added new functionality to the XML template digester to allow tokenless_autotag_delimiters to be Unicode.
• Fixed some tiny bugs in the Web UI templates.
• Inserted some missing documentation pages.
  

2.0RC9

• Fixed a subtle but dangerous bug in the document convertor which was causing the wrong annotations to be converted.
• Fixed a couple of tiny bugs in the scoring pairer.
• Fixed a bug in the UI where documents which were identified as reconciliation documents but were not were causing the UI to break.
  

2.0RC8

• Tightened the restrictions on int attributes: choice restrictions and range restrictions cannot cooccur.
  
• Documentation updates.
  

2.0RC7

• Fixed a bug where the function that aligns annotations with new tokens didn't work with overlapping annotations, and failed to forcibly detach annotations it deleted.
  

2.0RC6

• Fixed a bug where annotations without visible displays were taking up space in the UI.
  

2.0RC5

• Added a warning to notify users that MAT currently can't distinguish in UI presentation between annotation labels which differ only in case.
• Documentation updates.
  

2.0RC4

• Implemented a workaround for a limitation on numbers of query parameters in sqlite3.
• Enhanced the behavior of MATTransducer for reporting errors, capturing transduction history.
• Extended pair comparison to handle multi-attribute dimensions, attribute equivalences.
  

2.0RC3

• Fixed an insidious bug where the endpoints of swipes in the UI during annotation were being misinterpreted in rare circumstances.
• Fixed a bug where annotation popups and informative dialogs were not necessarily being removed when documents were deselected (via tabs, e.g.) in the UI.
• Added more operations to the document conversion.
• Modified the workspace_configuration operation in workspaces to print out the task.
• Demoted the support level of Windows XP, which we no longer have access to.
• Fixed a bug where documents which had hand annotation available when they were loaded into the UI were not being displayed in stacked mode.
• Fixed a bug where it was not possible to modify the extent an annotation in the UI if the swipe overlapped with multiple annotations on the same layer. 
• Fixed a bug where closing dialogs that ask a question left the UI in a strange state.
  

2.0RC2

• Fixed a rare bug where HTML characters in the annotation tables were not displayed correctly.
• Updated to jCarafe 0.9.8.4.RC14 to fix an off-by-one error.
• Documentation updates.
  

2.0RC1

• Minor bug fixes.
• Documentation updates, including Java docs.
  

2.0pre29

• Minor extension to document conversion.
• Added more customization options to attribute entries in UI tables and editor/viewers: custom editor label customization, URL links.
  
• Minor bug fix.
  

2.0pre28

• Minor updates to score configurability.
• Fixed bug in UI where workspace listing was not retaining its scroll location.
• Added the ability to designate an attribute as read-only for UI editing purposes.
• Ensured that sidebar redisplay happens appropriately for standalone viewers.
  
• Minor bug fixes.
  

2.0pre27

• Documentation updates.
  

2.0pre26

• Implemented case-insensitive autotagging UI option.
• Expanded the score profile options.
• Fixed a bug in workspace logging from the UI.
• Added highlighting and scrolling improvements to the comparison view.
• More documentation updates.
• Other minor bug fixes.
  

2.0pre25

• Updated jCarafe bindings for latest version.
• More documentation updates.
• Updated standalone viewer to support new comparison paradigm.
  

2.0pre24

• More documentation updates.
• Minor bug fixes.
  

2.0pre23

• More documentation updates.
• Added description columns to the detail spreadsheet output of the scorer.
• A variety of tiny bug fixes.
• Ensured compatibility with, and upgraded to, jCarafe 0.9.8.4.RC10.
  

2.0pre22

• More documentation updates.
• Updates to Java bindings to deal with annotation attribute value restrictions.
• Added an (undocumented) capability to add Web operations to tasks.
• Workspace dialog in the UI now focuses on the next input element when you tab.
• Fixed a number of small bugs.
  

2.0pre21

• Lots of documentation updates.
• Modified the way effective labels are declared.
  
• Fixed a number of small bugs.

2.0pre20

• Imported document mapping XML capability (not yet documented).
• Scrubbed the current state of the UI for logging updates.
• Fixed a bug where double-clicking on a word at the end of a line in non-tokenized hand annotation was selecting the wrong text.
• Extended MATManagePluginDirs to list the task names in the task directories.
• Enhanced MATTransducer with the document mapping capability, along with a reporting capability based on the same tools as those used in MATReport.
• Fixed a bug in the distribution code related to Mac configuration of Terminator.
• Added utility to create comparison documents.
• Extended hand annotation tool to cover complex label restrictions.
• Implemented workspace logging awareness in UI.
• Made all annotation capabilities work in standalone UI.
• Improved scoring algorithm to do a better job of pairing spanless annotations.
• Removed obsolete default_tag_window_size and default_tag_window_position.
• Improved tagging feedback in the UI.
  
• Initial documentation updates.

2.0pre19

• Small UI bug fix.
  

2.0pre18

• Small UI bug fixes.
  

2.0pre17

• Hand annotation now supports adding overlapping annotations.
• Reorganized annotation pane to force span breaks at line wrap boundaries, in order to have the display behave properly with annotations which are longer than a single line and to line up annotations properly with spanless annotation icons.
• Cleaned up implementation of resize in annotation panes.
  
• MATReport now supports per-label reports.
• Introduced new document comparison pane, using output from the pairing algorithm, including relation comparison.
• Implemented read-only appropriately across the UI for annotation editor popups.
  
• Fixed a variety of small bugs.
  

2.0pre16

• Added ability to report spanless annotations in MATReport.
• Added ability to register convertors for tasks in readers/writers.
• Added "scroll to" action in UI.
• Added annotation highlighting to annotation table mouse hover.
• Fixed a couple of small bugs.
  

2.0pre15

• Imported workspace logger and reconciliation workflows from custom task.
• Added ability to treat all annotations as content annotations in MATReport.
• Initial implementation of expanded annotation ontology in Java.
• Added stubs for new comparison window.
• Added --version option to common command-line options, and documented common options separately.
• Added "About MAT" option to UI, with version number.
• Implemented defaults in annotation task descriptors.
• Implemented new comparison documents based on similarity profiles.
• Added spanless sidebar and annotation highlighting in UI.
• Introduced improvements to feedback for hand annotation and choose mode in UI.
• Improved plumbing for presenting results of document and annotation modifications in UI.
  
• Fixed a variety of small errors.

2.0pre14

• Implemented similarity profiles for Kuhn-Munkres-based scoring. Enabled scoring of annotation-valued attributes and spanless annotations.
• Added similarity and scoring profiles to the scorer, including tag aggregations and tag decompositions in the scoring profiles.
  
• Enhanced MATTransducer to work on files one at a time.
• Implemented default annotation values in Python and Javascript libraries, including the ability to use the text span as the default value for spanned annotations.
  
• Fixed two small bugs in the MAT UI.
  

2.0pre13

• Added MATAnnotationInfoToJSON to support standalone viewers.
• Fixed a small bug in computing tag orders for the UI in the new annotation specification regime.
• Initial implementation of the capability of stratified scoring of annotations with annotation-valued attributes (not yet tested).
• Separated the annotation pairing capability from the scorer.
• Added the ability to create custom attribute editors in the UI.
  

2.0pre12

• Re-enabled popup annotation editors in standalone viewer.
• Fixed some minor bugs in how the string-valued attributes appear in the annotation editor.
• Incorporated Kuhn-Munkres alignment algorithm for scoring (initially disabled).
  

2.0pre11

• Fixed bug with deleting annotations in the UI.
  

2.0pre10

• Fixed a couple minor UI bugs.
• Re-introduced the overlay manager to handle popup backgrounding and foregrounding.
• Reworked the UI annotation manipulation plumbing to provide more general support for redisplay.
• Fixed a bug where the annotation table was always being shown, even when it wasn't needed.
• Disabled the "Show annotation tables" menu item when a document shows tables regardless (because it has spanless annotations).
• Added the ability to delete annotations from various context menus.
• Added the ability to choose an annotation in "choose mode" from the annotation editor action menu.
• Discovered and corrected inconsistencies in how the annotation editors were invoked in the standalone viewer.

2.0pre9

• Fixed a Safari bug relating to the menu update for the annotation tables.
• Fixed bugs in the standalone viewer relating to the upgrade to the new annotation set descriptors.
  

2.0pre8

• Numerous tiny UI bug fixes.
• Expanded UI support for annotation-valued attributes, including actions to add annotations to parents.
• Added annotation tables and the ability to create spanless annotations.
  
• Added support for a global "choose mode" in the UI.
• Expanded range of editable attribute types in annotation editors.
  

2.0pre7

• Removed old desktop UI.
• Migrated all tasks and plugin manager to new well-formedness conditions.
• Implemented global and local annotation type managers for well-formedness condition support.
• Added utility to update task.xml files.
• Improvements to initial well-formedness condition checking in Python and Javascript.
• Implemented initial UI support for annotation-valued attributes. 
• Enhanced presentation of annotations and synchronization between annotation views in UI.
• Improved global management of view settings in UI.
• Added ability to edit annotations in tabs.
  

2.0pre6

• Migrated to new workbench UI.
• Added Javascript support for new attribute types and well-formedness conditions.
• Added annotation capability to standalone document viewer.
  

2.0pre5

• Added support for lists and sets of attribute values in MAT-JSON and Python, as well as float, string, int, and boolean attribute values.
• Initial Python implementation of well-formedness conditions for annotation attribute values.
  

2.0pre4

• Fixed a tiny bug in the UI where unzoned documents couldn't do autotag because no content intervals were found.
  
• Updated the workspace documentation with troubleshooting tips.
• Other minor doc updates.
• Fixed bugs in Java bindings where annotation IDs were being incremented incorrectly, and spanless Asets were being created incorrectly.
• Fixed a couple small bugs in the build script related to missing executables on MacOS X.
• Fixed a bug in CherryPyService where images from task documentation had the wrong MIME type.
  

Version 2.0pre4 is the current frozen version for the Callisto-compatible version of TooCAAn.

2.0pre3

• Added confidence capturing to the jCarafe wrapper.
• Added MATTransducer.
• Added the --xml_translate_all option to the XML inline reader.
  

2.0pre2

• Transitioned to new workspace structure. Introduced conversion tool.
• Initial reorganization of extensible command-line arguments.
• Removed Cygwin support.
• Imposed requirement of Python 2.6 or later.
• Introduced reconciliation capability in UI.
• Removed the legacy "operate" keyword in workspace commands.
• Fixed bug 21049: XML inline rendering and reading should skip the untaggable tags, which typically cause crossing dependency problems. Fixed by removing untaggables except in the UI.
• Added overlap handling to the scorer.
• Fixed a bug in ordering annotations in the UI by making it impossible to reuse annotation names as attribute set names in task.xml.
• Made it possible to restrict scoring to gold segments in either the hypothesis or the reference.
• Made the confusability matrix in the scorer more well-behaved and well-conceived.
• Changed recall, precision, and f-measure in the scorer to return scores between 0 and 1, rather than 0 and 100.
• Fixed a set of interrelated bugs that was blocking jCarafe from acting as a partial tagger.
• Extended scorer to make use of all recorded distinguishing attributes in annotations.
• Added global management of debug flags, subprocess information, temporary directories.
• Modified scorer and experiment engine to generate multiple spreadsheet types in the same run.
• Enhanced fake XML reader to recognize zero-length tags, nested tags, recover properly from "misformed" tags.
• Removed mat_controller.sh and accoutrements. Refactored MATWeb to take responsibility for starting up the tabbed terminal.
• Updated to new version of jCarafe.
• Added support for running experiments against workspaces to both MATExperimentEngine and MATWorkspaceEngine.
• Added closing checks for the MAT UI to warn about unsaved document and log changes, and to free workspace locks.
• Added initial support for adding hooks to present annotation editors.
  
• Added indication of when a document is modified to the document pane title.
• Fixed a subtle bug involving figuring out whether a document is dirty or not.
  

2.0pre1

• Extended mat-json format.
• Provided new comparison window in UI.
  

1.3

• Provided some final documentation updates to stress the lack of security in the MATWeb server.
• Fixed a final bug in the demo UI.
• Backported standalone document viewer wrapper from MAT 2.0 development branch.

1.3RC10

• Fixed a tiny CSS bug.
• Updated MATScore documentation to clarify that only documents with nonoverlapping content annotations can be scored.
  

1.3RC9

• Fixed a conceptual bug in the detail spreadsheet in the scorer.
• Fixed a bug in MATWeb where the server wasn't enforcing that the workspace directory appear beneath the workspace container directory.
  

1.3RC8

• Fixed a bug in the confusability matrices in the scorer.
• Fixed a bug in handling forced recomputation in the experiment engine.

1.3RC7

• Fixed a bug in the UI where workspace document windows were not being tiled appropriately.
• Added a plugin hook for tasks to modify the UI metadata.
  

1.3RC6

• Fixed a small bug in handling autotagging in the UI.

1.3RC5

• Introduced further granularity into the scorer to describe various subtypes of span clashes. Introduced an option to enable this granularity (disabled by default).
• Fixed a small scorer bug which arose when reference documents contained no annotations.
  

1.3RC4

• Fixed a tiny but important bug in counting characters in XML inline input where the background signal is itself XML.
• Improved debug handling in the model trainer.
  

1.3RC3

• Added --preserve_tempfiles to preserve tempfiles during model building.
• Improved error checking for color interpretation for Java clients.

1.3RC2

• Added the ability to ignore labels to the scorer.
• Provided a workaround for managing zones to compensate for a jCarafe bug in tokenization.
  

1.3RC1

• Fixed small bug in scorer which raised an error when tokens were present in one, but not both, of the reference and hypothesis documents.
• Added equivalence classes and confusability matrix generation to the scorer.
• Expanded the core scorer columns to reflect the various subtypes of clashes directly.
  

1.3pre16

• Small documentation tweak.
• Internal API modifications to the document writer.
  

1.3pre15

• Improved the handling of command line help messages for document readers.
  

1.3pre14

• Added character and pseudo-token scoring to the MAT scorer.
  

1.3pre13

• Fixed a bug in managing the Web service headers which caused spurious errors to be generated when running the demos.

1.3pre12

• Modified remove operation in workspaces not to remove all basenames by default.
  

1.3pre11

• Added the fake-xml-inline reader.
  

1.3pre10

• Fixed two security issues in workspace access. First, if workspace container directories are specified to MATWeb, they must be used; no absolute pathnames are permitted. Second, workspace keys are no longer passed on the command line when MATWeb restarts.
• Fixed a tiny typo in the document reader.
  

1.3pre9

• Fixed a small bug in the MAT toolchain where it was not possible to access errors from the document reader.
  

1.3pre8

• Fixed a small bug in the MAT UI which arose while implementing autotagging.
  

1.3pre7

• Added the ability to specify character encodings to MATScore.
  

1.3pre6

• Fixed a bug where input-only or output-only document formats were not being treated appropriately in the MAT UI.
• Added the ability to insert document convertors for reading and writing in the document IO subsystem, to, e.g., convert from PERSON to ENAMEX TYPE=PERSON when reading or writing.
  

1.3pre5

• Added capability to autotag similar text strings in the MAT UI.
• Moved the text_right_to_left property from the workflows to the Web customization, since it's essentially task-global.
  

1.3pre4

• Fixed a bug in the new redistribute.py utility which prevented it from working with file paths with spaces in them.
• Added to the redistribute.py utility the ability to remove tasks from the target distribution.
• Enhanced MATWeb with the ability to supersede existing MAT Web servers, to capture random output and error, and to bundle a number of related features under the new --as_service option.
• Added more restrictive color format recognition in the Java bindings to support Callisto color interpretation.
• Improved the encapsulation of the Web frontend core pieces to facilitate reuse.
• Fixed incorrect and overly literal MIME type in Web server responses.
  

1.3pre3

• Removed all attempts to access annotation category information via the document. All such accesses now happen via the task.
• Added the redistribute.py utility.
• Fixed a bug in the MAT UI where annotation attributes and values were not being shown when the mouse hovers over an annotation.
• Fixed a bug in the experiment engine where the order of documents in training corpora was not preserved across restarts of the experiment engine (important when corpus size iterators are used).
  

1.3pre2

• Bug fixes for Java API improvements.
  

1.3pre1

• Introduced notion of iterators to experiment engine.
• Extended experiment score summary files to include more information about runs and models.
• Enhanced Java API slightly to support better introspection about task information.
  

1.2

• Fixed a bug which corrupted line endings in CSV files on Windows.
• Upgraded to Java Carafe 0.9.7RC4 to address a problem with serializing mat-json files in the tokenizer.
  

1.2RC23

• Introduced a check to ensure that MAT 1.2 fails intelligently when presented with future versions of the mat-json serialization.
  
• Minor documentation updates.

1.2RC22

• Upgraded to Java Carafe 0.9.7RC3. This version of jCarafe is not compatible with models built with previous versions of Java Carafe.

1.2RC21

• Fixed two small bugs where ENAMEX-style tags were not being ordered correctly when labels were not being alphabetized in the UI.
• Fixed a small bug in the scorer where documents with no zones (e.g., XML documents which contain only annotations) were not being scored correctly when a task was specified.

1.2RC20

• Fixed a small bug in setting up the tabbed terminal in Windows.

1.2RC19

• Fixed a small bug where Java Carafe model training might do the wrong thing with duplicate file names.

1.2RC18

• Reorganized documentation links to create a better initial experience when unpacking the distribution.
• Fixed a small bug in the initialization sequence when interacting with the Java Carafe tagger server.
• Fixed a small bug in CSS displays of annotations with labels whose name contain a dash.

1.2RC17

• Small documentation updates.
• Bumped Java requirement to 1.6 update 4 due to issues with Java XML-RPC.

1.2RC16

• Bumped Java requirement to 1.6.
• Extended the capabilities of the property caches in the experiment engine to handle a wider range of data types.
• Removed the option to make smaller zones for PSA training because it's no longer needed. Removed the no_random_segments configuration option as part of this change.
  
• Removed option for SGD training because it's being removed from Carafe.
• Insulated the MAT Web server against stray ill-formed cookies that the browser might deliver to the server by accident.
• Retokenized sample data and updated Java Carafe bindings for 0.9.6 version.
  

1.2RC15

• Fixed a bug where MATReport and MATScore spreadsheets were being created as ASCII rather than UTF-8.
• Expanded optional subprocess monitoring to include children and remote children of subprocesses.
• Fixed a bug with restart in MATExperimentEngine.
  
• Small documentation updates.
  

1.2RC14

• Updated to Java Carafe 0.9.5. All previous models will have to be rebuilt.
• Clarified a few error messages, updated some comments.
• Fixed an obscure bug in Windows 7 which was resulting in a Windows memory error.
• Added the optional ability to monitor subprocess memory image sizes via the open-source psutil package. Enabled MATEngine, MATModelBuilder, MATExperimentEngine and MATWorkspaceEngine with this capability.
  
• Fixed a bug in the test suite which limited its ability to run on Windows.
  

1.2RC13

• Bug 19392: Ensured that scorer doesn't include annotations outside zones in the reference document, if zones are known to be present.
• Extended MATReport to provide file-level statistics.
  

1.2RC12

• Improved error message when character encoding is wrong.
• Added the "default_tag_window_size" and "default_tag_window_position" attribute in task.xml to provide the ability to control the annotation windows in the MAT UI.
• Add the MATReport tool to generate concordance-style annotation reports in CSV and text formats.
• Fixed a small installer bug which arose when Java could not be found.
• Fixed a bug where deleting annotations in workspace mode in the UI wasn't enough by itself to mark a document as needing to be saved.
• Fixed a bug where the new keyboard accelerator for repeating the last annotation wasn't working on Windows.
• Extended the (undocumented) mechanism for task-specific Javascript customizations in the UI to include the ability to customize the presentation of untaggable regions.
• Added the ability to add a diff file against a previous distribution to a distribution tarball.
  
• Documentation updates.
  

1.2RC11

• Tiny bug fix in error reporting from Java Carafe.

1.2RC10

• Enhanced Java Carafe wrapper with control for Java stack size.
• Fixed some small bugs in error reporting from Java Carafe.
• Improved the description of corpora in CVS output files in the experiment engine.
• Added the error text description to the logging CVS files in the MAT UI.
• Enhanced task.xml files with defaults for the Java subprocess parameters.
• Improved documentation and record-keeping for MAT builds.
  
• Minor documentation improvements.
  

1.2RC9

• Added the "alphabetize_labels" attribute in task.xml to provide the ability to control how the annotations are ordered in the UI legend and popup.
  

1.2RC8

• Fixed a typo bug in the tagging service.
• Updated some documentation as a result of user feedback.
• Added the ability to add last tag in the UI, with a consistent keyboard accelerator.
  

1.2RC7

• Fixed two more small command-line argument processing bugs, one in MATModelBuilder and one in MATRetokenize.
  

1.2RC6

• Fixed a bug in the scorer, where the task was not being used as a fallback source of tag label metadata.
• Modified MATScore to allow the user to provide lists of content annotations and token annotations directly, in case neither task nor tag label metadata is available.
• Fixed a bug in the processing of command line arguments in MATModelBuilder.
• Fixed a bug in MATRetokenize that arose when only a single task was defined.
  
• Added internal APIs to make it easier to make use of taggers and tokenizers other than Carafe.

1.2RC5

• Fixed a lurking bug where CRLF was not being handled correctly when XML inline was being read.
  

1.2RC4

• Further cleanup of documentation for Windows port.
  

1.2RC3

• Improvements and simplifications to the distribution and plugin installation code.
  

1.2RC2

• Fixed minor bugs in constructing cascaded annotation menu hierarchy.
• Whitespace in tag labels is now supported correctly in the MAT UI.
• The annotation window in the MAT UI now displays the content annotations under the mouse.
  

1.2RC1

• The MAT UI now supports cascaded annotation menus, via the <tag_group> element in the task.xml file.
• The task.xml file now provides a means to control text direction for individual workflows, using the text_right_to_left attribute.
• The task.xml file syntax has been modified to clarify how attributes are handled by MAT. For advanced users, this necessitates a number of changes in the task.xml file (see the upgrade notes).

1.2pre2

• Fixed a bug where the scorer was breaking if files had no tags to score.
• It is now possible to use workspace operations directly on the command-line, without the "operate" operation.
• Replaced the OCaml tokenizer and Carafe tagger with the Java reimplementation. This necessitates a number of changes, including retokenizing your documents and rebuilding your models (see the upgrade notes).
• Command-line options for steps which are used in more than one step are now appropriately cross-referenced in the help string.
• Workspace operations which don't affect any pathnames now raise an error.
• The system now appears to work in Python 2.6 and in MacOS X 10.6.
• The experiment engine XML file now allows you to define bindings for commonly repeated values, and also supports explicitly referring to the experiment directory and pattern directory.
• The system now works in native Windows, without Cygwin.
• The system is now distributed in a single zip file for all platforms.
• The MAT UI has been improved to support hand annotation without tokens (although this is not recommended).
  
• MacOS X no longer requires a special installation of Python, even though select.poll is still missing on that platform.
• Bug 24205: due to a bug in the experiment engine, non-default model configurations weren't being handled correctly. Fixed.
• Bugs 15815, 19669: Make sure that the whole system does the right thing with spaces in filenames. Fixed.

1.2pre1

• Extended and modified how corpora are specified in experiment XML, introducing considerably greater flexibility. It's now possible to specify n-way corpus splits, and group them arbitrarily in training and test runs.

1.1RC2

• Fixed an obscure bug where child tasks of visible tasks weren't being reported appropriately to the UI.
• Fixed a minor bug in formatting XML output.
• Fixed a bug in workspace locking.
• Minor documentation updates.
  

1.1RC1

• Enabled confidence interval reporting. Added reporting of standard deviation.
  
• Added general workspace locking for operations, importing and removing basenames, opening a workspace file in the UI, and listing the contents of the workspace folders.
  
• Initial support for Python 2.6.
• Upgraded to CherryPy 3.1.2.
  
• Added XML reader to MATWorkspaceEngine import; expanded documentation on MATExperimentEngine to describe how to prepare corpora with XML documents.
  
• Added support for multiple model build settings in task.xml.
• Added PluginMgr.AlignStep to the core to support alignment of externally-generated content tags with token boundaries.
• Converted calls to MATEngine command-line tool in the experiment engine to invocations of the MATEngine object.
• Expanded Java service API to cover checking for the existence of a workspace, listing the contents of a workspace folder, and opening a workspace file.
  
• Bug 24020: due to a bug in the demo infrastructure, it was not possible to reprocess a document reliably. Fixed.
• Bug 23988: proxies for multiple steps in task.xml weren't doing the right thing when the proxy set didn't exactly match between proxies. You shouldn't do this, and the UI still exhibits some unusual behavior, but it's fixed in the engine.
• Bug 21112: the experiment engine insisted on converting test documents to raw form before processing for the test run phase. It's now possible to specify other default preprocessing (e.g., just undo tagging).
• Bug 20835: global maintenance of annotation type objects was not thread-safe. Fixed by making annotation type objects local to a document.
  

1.1pre1

• Added XML reader and writer to MATEngine, MATModelBuilder, MATScore, UI.
  
• Added ability to define arbitrary readers and writers.
• Added the ability to use your own training engine in your own task.
  
• Added feedback for workspace import.
• Added option to enable workspace access from remote clients via the Web server.
• Added splash screen to UI.
• Enhanced scorer to provide file-level count data to support computing confidence intervals (not enabled yet).
• Enhanced load and save in workspace mode in UI by adding the document basename to the Web service result.
• Added the "rich, incoming"  folder to the core workspace, for importing files which are in rich format but not yet prepared for hand tagging.
• Enabled rich document readers to infer the processing state of documents which lack the appropriate metadata.
• Added "remove" operation to MATWorkspaceEngine.
• Fixed a subtle bug where tagging engines which fail on startup were not notifying the UI client properly about the failure.
  

1.0

• Fixed an obscure bug where MAT JSON documents which share the same annotation label but have different attribute orders were being digested inconsistently.
• Fixed a small bug in MATWeb which prevented logs from rotating appropriately.
• Fixed a small bug in the Java library which didn't correspond to the (correct) documentation.
• Bug 18427: in some situations where the UI viewport is small, the menu bar would disappear. Fixed.
• Bug 20757: ENAMEX-style tags and attributes didn't work with Carafe training on JSON documents. Fixed.
• Bug 23672: the MIME type of log spreadsheets was incorrect when saved from the MAT UI, causing Excel to fail to digest the logs properly on Windows. Fixed.
• Bug 23668: "Update workspace key" in the UI was failing to percolate appropriately to "Open workspace...". Fixed.
• Minor documentation updates.
  

1.0RC4

• Fixed a bug where command-line arguments of MATEngine weren't overriding step attributes in the task.xml file. Added appropriate tests.
• Fixed a bug where step attribute defaults weren't being postprocessed appropriately.
• Fixed a bug in the Java bindings where it was possible to create an annotation type with null attributes.
• Fixed an infelicity in the initial README when users first unpack the distribution.
• Minor documentation updates.
  

1.0RC3

• Fixed minor omission in experiment infrastructure.
• Fixed a bug where extra task paths were not being canonicalized when task directories were being computed.
  
• Added a temporary fix, to be backed out in 1.1, to use the Carafe engine for the anonymization task in the core distribution.
• Fixed a bug in branding in the UI.
• Minor documentation updates.

1.0RC2

• Minor documentation updates.

1.0RC1

• Vast tracts of documentation updates.
• Minor modifications of the UI log action names to improve consistency.
• Bug 19970: because of a deep bug in the interaction between Firefox and the UI toolkit when the backend server was down, no feedback was being provided to the user about the failure. Fixed.
• Bug 17989: the experiment engine was saving its raw gold files as ASCII, rather than a Unicode-compatible encoding like UTF-8. Fixed.
  
• Bug 17867: HTML and HTTP escapes were not being inserted appropriately in the routines that generate task-specific documentation. Fixed.
  
• Bug 17778: various operations that could be performed in the UI weren't blocking out rerequests while the operation was being performed. Fixed.
  
• Bug 16039: left click (not swipe) on an annotation while hand annotating selected just that token, not the entire annotation. Fixed.
  
• Extended branding to documentation.
  

1.0pre14

• Yet more documentation updates, including Java client library documentation.
  
• Changed the "inherit_actions" attribute of workspace operations in task.xml to "inherit_operations", for consistency.
  
• Fixed a bug in the plugin manager which was making demo files inaccessible for tasks without Python customizations.
  

1.0pre13

• Even more documentation updates.
• Corrected terminology for the deidentification task.
• Fixed MATWeb so that it preserves its workspace key across restarts.
• Expanded the Java client library to support workspace operations.
• Added branding capability to UI and tasks.
  

1.0pre12

• Vast documentation updates.
• Added a Java client library (not documented yet).
• Bug 19772: install.bat was failing with links in Cygwin installation. Fixed.
  

1.0pre11

• The settings for the Carafe model builder are now specified in the task.xml file, and can be overridden in the experiment XML or in the workspace settings. As part of this change, the "task" attribute to <build_settings> in the experiment XML file is no longer recognized.
• Added the MATModelBuilder tool.
• Added the MATManagePluginDirs tool.
• Expanded the documentation.
  

1.0pre10

• The tabbed terminal is now optional. Build, install and runtime have been appropriately updated.
  

1.0pre9

• The visible name of the rich JSON format in the UI, command line, and all configuration and experiment files has been changed to "mat-json". This change is in support of future additions of other readers and writers for the MAT system.
• The URL with which MAT can be accessed in the Web UI has been changed to 'http://<host>:<port>/MAT/desktop".
• For any task directory ending with basename <name>, you can now access a desktop restricted to that task at "http://<host>:<port>/tasks/<name>/desktop".
• Initial support for user-friendly demos has been introduced. The configuration file for this capability will be documented when the capability matures a little more.
• All XML configuration file layouts are now defined by a simple, user-readable templating system.
• It is now possible to define user-visible workflow steps which consist of sequences of implementations of other steps (e.g., "prep" might implement "zone,tokenize").
• The Web UI is no longer tied as closely to the names of the steps in the MAT engine.
• Tasks now support the option of defining "attribute sets" for annotations, so that, e.g., an ENAMEX tag with different values for the "type" attribute can map to different CSS configurations, and be treated differently in the scorer.
• Bug 19240: if a core task was installed using MATInstallApplication after a dependent task, and both had documentation, the documentation rendering was broken. This has been fixed by moving to live, on-line generation of the customized documentation.
  
• Bugs 14466, 16090: The tarball distribution contained multiple copies of the MAT executables, only one of which was configured correctly. All duplications have now been eliminated. As a result, all MAT executables in the tarball distribution should be accessed from src/MAT instead of build/MAT.
• Bugs 14507, 19391: Previously, there was a separate step to install the Web documents in a separate location. That step has now been eliminated.
  

1.0pre8

• MATServer has been removed from the system, and replaced by a thread in the CherryPy Web application.
• The underlying version of Carafe has been upgraded, and old training models are not forward-compatible with the new engine. Models will have to be rebuilt.
  
• Bug 14508: The Web server could not be limited to localhost. Now it can be.
  

1.0pre7

• Apache has been removed from the system, and replaced by the Python-based CherryPy Web application infrastructure.
  

1.0pre6

• The MAT scoring engine now defaults to writing CSV files with spreadsheet-interpretable equations for computed values. This behavior can be controlled on the command line for both the scorer and the experiment engine.
  

1.0pre5

• Conversion of ad-hoc mechanisms for Carafe customization to consistent command-line updates. Note: due to this change, the way to add a permanent Carafe model to a task file has changed.
  
• Bug 16450: the -prior-adjst argument of Carafe was not available for customization in any useful way, even though it controls recall/precision bias. Fixed.
  
• Bug 16225: the random zoning option to optimize PSA training was available only to resynthesized documents, because it was part of the de-identification task. It is now a feature of the trainer itself, and the zone tags are temporarily inserted into the document immediately before training if PSA training is requested.

1.0pre4

• Major refactor of internal management of steps to support step rollback in the backend and address a large number of bugs.
• Bug 18960: switching between workflows wasn't undoing all the appropriate UI configuration steps. Fixed.
  
• Bug 18898: After cleaning up the way transformation and nomination should work in the de-identification task, changing the replacer between the two steps could screw up the application of the transformation. Fixed by cleaning up the way metadata is handled in the UI and transported to the backend.
• Bug 18849: experiment directory is created incorrectly if it doesn't exist when the experiment is started. Fixed.
  
• Bug 18848: Transforming a document in the de-identification task would strip all the tags if tokenization had been skipped. Fixed by doing a smarter job of figuring out how to postprocess transformed documents.
• Bug 18534: hand annotations were not showing up when hand annotated documents were reloaded, because of a logic error in the relationship between steps done and steps visible. Fixed.
• Bug 18430: hand annotation was erroneously available during resynthesis. No longer.
• Bug 18426: the clean step can't be undone, but nothing was preventing this. Now raises an error.
  
• Bug 18425: the clean step was not a core step. It is now.
• Bug 17680: the psaTransform step was not being rolled back appropriately in one implementation of the de-identification task, because it wasn't "really" a transform step. Refactoring the step management fixed this.
• Bug 17610: rollback was not supported in the backend engine. Fixed. Engine now accepts an --undo_through argument.
  
• Bug 16603, 16604: newlines are preserved during transformation in the de-identification task, but the extent of the tags needed to be adjusted. In non-clear replacement, the right thing was happening by accident, because those documents were being tokenized when they shouldn't have been. These documents are now (properly) not tokenized, and the extent adjustment is now done correctly.
• Bug 14512: if there's only a single task or a single workflow, they should not need to be provided. Fixed.
  

1.0pre2

• Major new capability to manage workspaces of documents, including support for iterative model creation.
  
• New "Save/Hide" menu to allow better UI desktop management, as well as a hide widget in the window panes.
• Bug 17687: the UI was showing steps which weren't in the current workflow, because they were in the document. Fixed.
  
• Bug 16747: New windows appear directly on top of each other in the new UI. This has been fixed.
  

0.9p4

• Bug 18457: a number of closely related bugs in the de-identification task conspired to cause document-level cacheing of names and name components to fail.
  

0.9p1

• Added Windows batch scripts to run the various MAT tools outside of the MAT controller.
  

0.9

• A tiny bug fix to address a problem managing absent Carafe models in tasks in delivered tarballs.
  

0.9pre13

• Made entire system Unicode-aware. Added guard to ensure that Carafe does not currently see non-ASCII-compatible files. Added option to pass character encodings to the engine in the UI and on the command line.
  
• Improved error reporting in MATEngine.
  

0.9pre12

• Upgraded to Yahoo UI Toolkit version 2.6.0.
• Added ability to customize scorer to some degree for various tasks.
• Added --debug flag to experiment engine for debugging support.
• Fixed an unreported bug where rerunning the experiment engine to regenerate scores when source_corpus_dir was present was failing.
  
• Bug 17863: errors in the command line tool weren't reporting the file that caused the error for tokenization. Fixing this in all cases isn't really possible at the moment, because of the way batch processing works, but it's possible in non-batch, and should be now fixed.
• Corrected a subtle error in rich document encoding in Python where the sequence "\/" wasn't getting decoded correctly by the python-cjson library. Migrated to simplejson.
• In the de-identification task, added a simple Java client example for the MAT CGI script.

0.9pre11

• Enabled chains of source corpora dependencies in experiment engine. This corrected an error where the engine would fail if a corpus pointed to a source corpus which itself pointed to a source corpus. The only visible consequence of this change is that <prep> instructions in these chains are layered; if a corpus has a <prep> instruction and a source_corpus_dir attribute, the input to the <prep> instruction will be the output of any <prep> instructions in the source corpus, rather than ignoring the <prep> instruction in the source corpus as the system previously did. We know of no system deployments which exploited this previous configuration.
  
• Bug 17681: the psaTransform step was being recorded by mistake in the transformed document in the de-identification task. The system now imposes a more general requirement that steps are only recorded on documents modified by side effect, rather than freshly produced documents.
• Added the ability to pass a lexicon to the training engine.
• Added the ability to pass an experiment directory to the experiment engine.
• Relaxed the requirement that the source_corpus_dir attribute in the experiment configuration be an absolute pathname. Now, like other paths in the experiment configuration, if it is not absolute, the experiment directory will be prepended.
• The experiment engine now copies the experiment XML file into the experiment directory, if a file by that name is not already present.
  

0.9pre10

• Bug 17613: in the de-identification task, a small bug in the training engine was causing the training to fail if annotations already contained a replacement nomination. Fixed.
  
• Bug 17612: a repeat of bug 16099, where trailing newlines aligned tokens with tags incorrectly after replacement. The previous fix was slightly incorrect. Fixed.
  
• Fixed an element of inflexibility in the Carafe trainer, which didn't permit the command line in the <prep> experiment XML tag to support input documents which were rich documents. This command line no longer supplies the --input_file_type argument automatically, and it must now be provided explicitly in the experiment XML file.
• Added the beginnings of an improved test suite (not yet documented).
  

0.9pre9

• Bug 16095: Carafe trainer couldn't handle empty lines in the file specifying the available tags. Fixed in the experiment engine by automatically generating the tag file. As a result, the --task arguments are no longer supported in the <prep> and <run_settings> command lines in the experiment XML; the toplevel <experiment> tag now requires a "task" attribute; and the <build_settings> tag no longer accepts the "tag_set" attribute.
• Bug 16749: deleting a document window while the document was being tagged caused a Javascript error. Fixed.
• Bug 17027: in the UI, it was possible to include trailing or leading whitespace in an annotation during hand tagging. Fixed.
• Bug 17028: in the UI, it was possible to swipe from a taggable point across an untaggable point to another taggable point. This now generates an error to the user.
• Bug 17029: bug 16096 was not completely fixed in 0.9pre6; trying to repeat some errorful calls caused the system to hang. Fixed.
  

0.9pre8

• Fixed minor score accumulation bugs in experiment engine.
  

0.9pre7

• Fixed bug where distributions with only a single task defined caused an error in the UI.
  

0.9pre6

• Cleaned up computation of file save paths so the application never adds more than one pathname suffix.
  
• Bug 14509: to simplify task descriptions, display configuration, Javascript, and CSS files should be automatically inherited from their parent tasks. Fixed.
• Bug 14393: modification check was not performed when the task changed. This bug was mostly applicable to the old interface, but in the new interface, all client changes are tracked, and rollback confirmation is presented when any change is undone, or when document is closed. Fixed.
• Bug 15856: modification check was not performed when steps were undone. Same issue as bug 14393 above. Fixed.
• Bug 16096: various Carafe errors were not being percolated back to the client appropriately. Fixed many, many tiny bugs.
  

0.9pre5

• Added functionality to support customization of core documentation to allow incorporation of application-specific documentation and application-specific branding.
• Bug 14650: mat_controller.sh was starting up even if the ports that its servers needed were taken. Fixed.
  

0.9pre4

• Experiment engine now does the right thing when corpora are reused and modified in the consuming experiment.
  

0.9pre3

• Bug 16099: bracket redaction spans were mistakenly expanded over subsequent tokens in some cases. Fixed.
• Bug 16478: zone step in interface was not rolled back correctly. Fixed.
• Bug 16098: drop-down menus were obscured by document windows. Partially fixed; menu bar can still be partially obscured.
  

0.9pre2

• Feature request 16097: Added token-level scoring, better alignment for error detail to the scoring engine.
  
• Bug 16255: the experiment engine should switch to noninterleaved mode when it detects that some models have already been built. Fixed.
• Bug 16256: documentation for training_increment in experiment XML documentation was incomplete. Fixed.
  

0.9pre1

• New user interface, based on the Yahoo! UI toolkit, featuring a desktop-style interaction with a top menubar and multiple moveable, resizeable document panes.
• Bug 15870: annotation popups in Firefox 3 were popping up in the lower left corner. Fixed in the new UI.
  

0.8RC6

• Expanded UI log facility to record seconds since log start, more details about UI annotation popup interaction.

0.8RC5

• Feature request 15803: Initial support for keyboard accelerators in UI annotation popup.
  
• Bug 15826: the UI was mistakenly recording some steps that it shouldn't be recording. Fixed.
• Bug 15810: steps in the UI were not changing when the user changed the workflow. Fixed.
• Bug 15809: hand annotation was available before zoning was applied. In the current system, this could result in undigestible data. Fixed.
• Bug 15808: the "Save raw" button for raw documents made no sense, and was misleading. Now disabled.
• Bug 15806: tag names were not alphabetized in the annotation popup. Fixed.
• Bug 15807: some annotation names should be white foreground text. Easily supported through task specification; tasks updated.
• Bug 15814: annotations weren't visible when reloading saved rich documents. This bug revealed a number of issues with handling the global order of steps across workflows, which have hopefully all been addressed.
• Bug 14394: steps update wasn't working when workflow changed. Fixed.
• Bug 15815: Unix installation path cannot contain spaces. Temporarily fixed by aborting installation, with warning, if spaces are in the path.