Alice was beginning to get very tired of sitting by her sister on the bank and of
having nothing to do: once or twice she had peeped into the book her sister was reading,
but it had no pictures or conversations in it, "and what is the use of a book," thought
Alice, "without pictures or conversations?"
The XStandoff instance of this example is shown in .
Since both, p and q element belong to the logical document
structure level, only one level element is included. The quotation is annotated
as a q element constructed by the disjoint segment with the
identifier seg4 which itself uses the segments seg2 and seg3.Disjoint segments in XStandoff<xsf:corpusData xmlns="http://www.xstandoff.net/2009/xstandoff/1.1"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsf="http://www.xstandoff.net/2009/xstandoff/1.1"
xsi:schemaLocation="http://www.xstandoff.net/2009/xstandoff/1.1 ../xsd/xsf.xsd"
xsfVersion="1.1" xml:id="alice">
<xsf:primaryData start="0" end="302">
<textualContent>Alice was beginning to get very tired of sitting by her
sister on the bank and of having nothing to do: once or twice she had
peeped into the book her sister was reading, but it had no pictures or
conversations in it, "and what is the use of a book," thought Alice,
"without pictures or conversations?"</textualContent>
</xsf:primaryData>
<xsf:segmentation>
<xsf:segment xml:id="seg1" type="char" start="0" end="302"/>
<xsf:segment xml:id="seg2" type="char" start="218" end="250"/>
<xsf:segment xml:id="seg3" type="char" start="266" end="302"/>
<xsf:segment xml:id="seg4" type="seg" segments="seg2 seg3" mode="disjoint"/>
</xsf:segmentation>
<xsf:annotation>
<xsf:level xml:id="alice-log">
<xsf:layer xmlns:log="http://www.xstandoff.net/alice/log"
xsi:schemaLocation="http://www.xstandoff.net/alice/log ../xsd/alice-log.xsd"
priority="0">
<log:text xsf:segment="seg1">
<log:p xsf:segment="seg1">
<log:q xsf:segment="seg4"/>
</log:p>
</log:text>
</xsf:layer>
</xsf:level>
</xsf:annotation>
</xsf:corpusData>Due to the differentiation between the annotation concept (the level) and its XML
representation (the layer), it is possible to sum up different XML serializations underneath
the very same level element. XStandoff's layer element serves as a wrapper for
the slightly converted representation of the former inline annotation – the only
changes that are made concern the deletion of text nodes and the addition of the
segment attribute that links to the corresponding segment
element. The conversion applies to both the annotation instance and its underlying XML
schema description (into which XStandoff's base layer is imported to add the
segment attribute to all element nodes as optional attribute) – the
latter can be used both for validating the original inline annotation and the converted
representation as part of the XStandoff instance. In contrast to other approaches the
hierarchical structure of and the attributes included in the imported annotation layers
remain unchanged. In the authors make an argument for
distinguishing more broadly between dominance and containment. The former should be regarded
as the transitive closure of the parent/child
relation while the latter is regarded as superset/subset relation on the leaf nodes reachable from a
node by following parent-child arcs. To clarify this distinction between both
concepts different XStandoff representations are possible as well (cf. and ). In these
alternative representations, dominance is encoded as hierarchical relationship between the
two nodes p and q while containment is encoded via the
corresponding segments' start and end positions.Disjoint segments in XStandoff – Excerpt of an alternative representation
in which <code>p</code> dominates the <code>q</code> fragments<!-- ... -->
<xsf:layer xmlns:log="http://www.xstandoff.net/alice/log"
xsi:schemaLocation="http://www.xstandoff.net/alice/log ../xsd/alice-log.xsd"
priority="0">
<log:text xsf:segment="seg1">
<log:p xsf:segment="seg1">
<log:q xsf:segment="seg2"/>
<log:q xsf:segment="seg3"/>
</log:p>
<log:q xsf:segment="seg4"/>
</log:text>
</xsf:layer>
<!-- ... -->Disjoint segments in XStandoff – Excerpt of an alternative representation
in which <code>p</code> does not dominate <code>q</code><!-- ... -->
<xsf:layer xmlns:log="http://www.xstandoff.net/alice/log"
xsi:schemaLocation="http://www.xstandoff.net/alice/log ../xsd/alice-log.xsd"
priority="0">
<log:text xsf:segment="seg1">
<log:p xsf:segment="seg1"/>
<log:q xsf:segment="seg4"/>
</log:text>
</xsf:layer>
<!-- ... -->Other possible XStandoff instances include the separation of the q element
from the p layer at all (i.e. using separate layer elements which
is permitted by XStandoff), however, as already stated above, when dealing with disjoints
units, the inherent methods should be preferred.Inline XStandoffIn contrast to SGF, XStandoff supports a newly introduced inline representation (cf.
Section ), containing the virtual root element
inline and the generic milestone element (modelled according to
the respective TEI element, cf. ). During the Sekimo project
Schiller's 'Die Bürgschaft' was annotated on the following annotation levels: words,
morphemes, syllables, verse, prose and phraseAll single annotation instances are available at http://coli.lili.uni-bielefeld.de/Texttechnologie/Forschergruppe/Phase1/sekimo/internet-praesentation/buergschaft.html.. We've converted these six levels into a single XStandoff instanceThe converted XStandoff instance is available at http://www.xstandoff.net/examples/buergschaft.html.. The inline representation shown in was
generated afterwards (cf. ).Part of the inline XStandoff instance of Schiller's 'Die Bürgschaft' annotated on six
levels.<xsf:inline xmlns:xsf="http://www.xstandoff.net/2009/xstandoff/1.1"
xmlns="http://www.xstandoff.net/2009/xstandoff/1.1"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:silbe="http://www.xstandoff.net/buergschaft/silbe"
xmlns:wort="http://www.xstandoff.net/buergschaft/wort"
xmlns:vers="http://www.xstandoff.net/buergschaft/vers"
xmlns:morpheme="http://www.xstandoff.net/buergschaft/morpheme"
xmlns:prosa="http://www.xstandoff.net/buergschaft/prosa"
xmlns:phrase="http://www.xstandoff.net/buergschaft/phrase">
<silbe:text xsf:segment="seg1">
<silbe:body xsf:segment="seg1">
<wort:text xsf:segment="seg1">
<wort:body xsf:segment="seg1">
<vers:text a="b" xsf:segment="seg1">
<vers:body xsf:segment="seg1">
<morpheme:text xsf:segment="seg1">
<morpheme:body xsf:segment="seg1">
<prosa:text xsf:segment="seg1">
<prosa:body xsf:segment="seg1">
<phrase:text xsf:segment="seg1">
<phrase:body xsf:segment="seg1">
<silbe:head xsf:segment="seg2">
<wort:head xsf:segment="seg2">
<vers:head xsf:segment="seg2">
<morpheme:head xsf:segment="seg2">
<prosa:head xsf:segment="seg2">
<phrase:head xsf:segment="seg2">Die Bürgschaft</phrase:head>
</prosa:head>
</morpheme:head>
</vers:head>
<!-- ... -->
</xsf:inline>Since each single annotation layer contains a text root element, a
body and a head element, these elements occur six times in the
resulting XStandoff instance. It should be noted, that in this case the inline XStandoff
instance is more than two-thirds bigger in size than the 'classic' (i.e. standoff) XStandoff
instance (763.9 KB vs. 452.5 KB) and is far from being easily readable.Introducing the 'all' namespaceFor both 'classic' XStandoff and especially for inline XStandoff instances, a new
namespace – http://www.xstandoff.net/2009/all – was
introduced. Elements belonging to this namespace do not only share the same range of
characters but the generic identifier and are present in all annotation layers (e.g. the
above mentioned text root element). shows
the same part of the inline XStandoff instance using the newly introduced namespace. Note
that the a attribute of the text element of the verse layer
remains intact. Cf. in for an
extended excerpt.Excerpt of the inline XStandoff instance of Schiller's 'Die Bürgschaft' using the
<emphasis role="ital">all</emphasis> namespace.<xsf:inline xmlns:sgf="http://www.text-technology.de/sekimo"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://www.xstandoff.net/2009/xstandoff/1.1"
xmlns:all="http://www.xstandoff.net/2009/all"
xmlns:silbe="http://www.xstandoff.net/buergschaft/silbe"
xmlns:wort="http://www.xstandoff.net/buergschaft/wort"
xmlns:vers="http://www.xstandoff.net/buergschaft/vers"
xmlns:morpheme="http://www.xstandoff.net/buergschaft/morphem"
xmlns:prosa="http://www.xstandoff.net/buergschaft/prosa"
xmlns:phrase="http://www.xstandoff.net/buergschaft/phrase">
<all:text xsf:segment="seg1" vers:a="b">
<all:body xsf:segment="seg1">
<all:head xsf:segment="seg2">Die Bürgschaft</all:head>
<!-- ... -->
</xsf:inline>This mechanism allows for a better readability of a multi-dimensional annotated inline
XStandoff instance. However, one should be aware that the all namespace should only be used when the respective elements not only bear
the same generic identifier and character range but also the same semantic value.Note that the inline XStandoff instance lacks the support for validation of
multi-dimensional annotations that is present in 'classic' XStandoff instances and should be
therefore considered for demonstration purposes only.The XStandoff toolkit In the context of XStandoff's development several XSLT 2.0 stylesheets (cf. , ) have been created which allow for the
convenient generation and editing of XStandoff filesThe stylesheets are available at http://www.xstandoff.net/files/xsf_stylesheets.zip. An additional Java
implemention with a restricted functionality can be obtained as well.. Since there are no product specific extensions, transformations should be able to
be executed by every XSLT processor supporting XSLT 2.0. The transformations during the test
phase of the stylesheets have been performed by the Saxon XSLT processor which is available
both as Open Source version Saxon-B and as optimized, schema-aware, commercial version Saxon-SADownload the Open Source version at http://saxon.sourceforge.net, for the
commercial Saxon-SA visit http://www.saxonica.com.. Currently the XStandoff toolkit consists of four stylesheets which perform basic
processing of XStandoff instances. They are responsible for the conversion of a single inline
annotation to XStandoff, the merging of XStandoff annotation layers corresponding to the same
primary data, the removing of single layers from an XStandoff file and, the conversion of an
XStandoff instance to an XStandoff inline annotation (cf. ).
These tasks might seem simple, but as you will see it would hardly be possible to perform them
manually. Building XStandoff instances: <emphasis role="ital">inline2XF.xsl</emphasis> At the moment the easiest way of building XStandoff files is to transform an inline
annotation into XStandoff by the stylesheet inline2XSF.xsl.
This excludes the coverage of overlapping markup at first sight because of the inline
annotation not including such structures. But we will show how to include more than one
inline annotation into a single XStandoff file by merging several XStandoff annotations (cf.
). By this means overlapping structures can be
covered.The transformation into XStandoff requires an input XML file ideally containing elements
bound by XML namespaces. Every single namespace evokes the output of a layer in XStandoff
which contains the elements of the namespace. Thereby the default (or empty) namespace is
treated like the named ones. Thus inline annotations without explicit namespace declarations
can also be processed (in this case a namespace will be generated). The process of converting an inline annotation to XStandoff is divided into two steps:
Firstly, segments are built on the basis of the occurring elements. There are two possible
ways of mapping the element boundaries to the textual content in the form of character
positions. The preferred way of reaching such a mapping is the use of a primary data file
which contains the bare text without annotations. The name of this file can be provided
during the transformation call by specifying the stylesheet parameter primary-data. Providing the location of a primary data file leads to a
comparison of the content of the primary data file and the textual content of the input file
of the transformation. This guarantees primary data identity. If no primary data location is
provided, the textual content of the input file is used to build up the primary data. This
has certain disadvantages such as the lack of line breaks since these cannot easily be
inferred by the textual content of an XML file. Furthermore, the automatic conversion of the
textual content of the input file to be used as primary data relies on heuristics which have
to detect white space characters. Because of the complexity of this task it is possible to
get undesirable results.After the building of segments, the second step of the transformation is to return
layers on the basis of namespaces. Thus for every namespace the corresponding elements are
released from the initial inline annotation and copied into the layer maintaining the
embedding relations. Meanwhile the elements in the XStandoff layer get connected to the
according segments by ID/IDREF binding. In this manner one segment can serve as a reference
for elements from different layers.There are several additional optional parameters which can control certain aspects of
the transformation from inline to XStandoff. The most important of them will be briefly
outlined below.virtual-root (data type: xs:string; default value:
'')Specifies a unique element (if no error occurs) which serves as the starting point
of the conversion of the input file. By default the whole file will be converted
starting at the document root. meta-root (data type: xs:string; default value:
'header')This parameter determines the location of metadata in the input XML document (if
any). By this means it can be copied into the XStandoff instance. Analogous to the
parameter virtual-root an element name is expected as a value of meta-root.local-xsd (data type: xs:boolean; default value:
0)Determines whether the corresponding XStandoff XML Schema files are stored locally
or shall be taken from the WWW (i.e. from
http://www.xstandoff.net/2009/xstandoff/1.1). By default the global XSDs
are used.include-ws-segments (data type: xs:boolean; default
value: 0)XStandoff uses segments for referencing units of the primary data annotated in one
or more annotation layers. Additional segments for non-character data (i.e. white space
characters, such as blanks, line breaks, etc.) can be computed and returned by the
stylesheet as well if the parameter include-ws-segments is set to '1'.all-layer (data type: xs:boolean; default value:
0)The output of an 'all-layer' (cf. ) containing
elements present in all annotation layers depends on the specification of this
parameter. By default its value is set to '0' which avoids the transfer of the
respective elements into the 'all-layer' (cf. ). Merging XStandoff instances: <emphasis role="ital">mergeXSF.xsl</emphasis>Due to the frequent use of the ID/IDREF mechanism in XStandoff for establishing
connections between segment elements (i.e. the limits of the respective text
span in the primary data's character stream) and the corresponding annotation, manually
merging XStandoff files seems quite unpromising. The XSLT stylesheet
mergeXSF.xsl transforms two XStandoff instances into a single one containing
the annotation levels (or layers) from both input files. The first XStandoff file is
provided as the input file of the transformation, the second file's name has to be included
via the stylesheet parameter merge-with. The main problem is to adapt the segments from the involved XStandoff files to each
other. On the one hand there are segments in the different files spanning over the same
string of the primary data, but having distinct IDs. In this case the two segments have to
be replaced by one. On the other hand there will be segments with the same ID, but spanning
over different character positions. These have to get new unique IDs. The merging of the
XStandoff files in general leads to a complete reorganization of the segment list making it
necessary to update the segment references of the elements in the XStandoff layers. After
fulfilling this duty, the XStandoff layers are included in the new XStandoff file. The reorganization of the segment list can be disabled by configuring the stylesheet
parameter keep-segments. Specifying the value '1' causes
the perpetuation of the segments of the input XStandoff file. However, the segments of the
file provided by the merge-with parameter are always
subject to a reorganization.In addition, the stylesheet handles the optional output of the 'all-layer'. Setting the
value of the all-layer parameter to '1' evokes the
inclusion of this special layer containing the elements that are present in every single
annotation layer. It is irrelevant if there was an 'all-layer' present in the input files or
not. Though it might happen that no such layer is returned, namely if there are no elements
in the several layers which share the required features. Currently the stylesheet only supports the merging of two single XStandoff files.
Naturally this allows for a successive merging of more than two files. However it would be
more straightforward to have the possibility of merging more than two XStandoff files during
a single transformation. A future version of the stylesheet supporting multi-file-merge is
in preparation. Deleting parts of XStandoff instances: <emphasis role="ital">removeXSFcontent.xsl</emphasis>For deletion of parts of an XStandoff instance the stylesheet removeXSFcontent.xsl can be applied. During the transformation call one has to
supply the ID of the element to be deleted (either level or layer)
via the value of the remove-ID parameter. The matching
element is removed from the XStandoff file and the list of segments is updated, i.e. the
segments which solely were referenced by descendant elements of the mentioned element are
excluded. This, admittedly, again leads to a reorganzation of the segments since these by
default get a continuous numbering. Similar to the usage of the mergeXSF.xsl stylesheet the
reorganization of the segments can be disabled by the stylesheet parameter keep-segments. The value '1' lets the stylesheet keep the old
segments, but without those only referenced by descendant elements of the deleted
element.Furthermore, the removed XStandoff content can be returned in a separate file.
Specifying the value '1' for the stylesheet parameter return-removed will encourage the stylesheet to return the removed content
into a new XStandoff instance. Accordingly, the new file will contain the segments
referenced by elements of the removed content and the removed content itself. Building inline XStandoff annotations: <emphasis role="ital">XSF2inline.xsl</emphasis>In addition to the inline2XSF.xsl stylesheet there is a
counterpart. XSF2inline.xsl creates an inline annotation on
the basis of an XStandoff instance. The approach covers the handling of overlapping markup
insofar as these structures are represented by milestone elements in the resulting inline
annotation (cf. in ).
Concerning this matter, the first task of the stylesheet is to detect segments whose start
and end position information constitute an overlap. These segments are split up into
segments representing milestones so that they can be used as an adequate basis to build an
inline annotation. The linear list of segments is processed recursively by taking the
currently outermost segments (those who are not included in other segments' spans which
respectively are determined by their start and end positions in the character range of the
primary data). The elements of the XStandoff layers which are referenced by the outermost
segments are copied into the inline annotation. The segments which are embedded in the
outermost segments are processed recursively. However, copying the elements from the XStandoff layers has to be controlled by a
mechanism regarding the possibility of elements from different layers referencing the same
segment. These elements share the same positions for start and end tags and therefore a
decision has to be made in which order they should be nested into one another. The optional
stylesheet parameter sort-by refers to this circumstance.
Its default value is 'measure' which means that a statistical analysis is performed by the
stylesheet that diagnoses the embedding relations of all occuring element types by
frequency. An expected result would be that elements representing sentence boundaries are
embedded into those for paragraphs because this embedding relation is more frequent than
vice versa. The only case this approach could be inadequate for are
elements of different layers for which no definite statistical result for embedding can be
achieved. For instance, there could be elements whose boundaries always share the same
character positions, but which can clearly semantically be assigned to a certain embedding.
This case cannot be covered by the statistical method. In addition, the embedding can be based on the priority attribute of the
level (in SGF version 1.0) or the layer (in XStandoff version
1.1, cf. ) element. This strategy can be accessed by
specifying the value 'priority' for the parameter sort-by.
Low values of the priority attribute in the XStandoff annotation are nested
deeper in the inline annotation than higher ones. By this means the user can specify the
embedding manually, but one has to be sure to set the values of the attribute correctly to
get the desired result. This method underlies the assumption that there can be a
semantically grounded, definite decision for the embedding. The most promising concept was a
mixture of the both approaches which has to be realized in future work. There is an additional optional stylesheet parameter return-segID which can be very helpful to retain a connection between the
XStandoff file and the resulting inline annotation. By default the value of this parameter
is set to '1' which means that the segment attribute is retained throughout the
conversion process. The parameter has been added mainly for control issues. Demonstration: Creating an inline XStandoff annotation In this section the conversion of six original annotations into one inline XStandoff
annotation will be outlined. This will demonstrate the function and execution of the several
XSLT stylesheets which are involved in this process. An extract of the final result of the
conversion process can be seen in where the inline
XStandoff annotation was introduced. There are three major steps in order to create one inline XStandoff annotation out of
the six input annotations: Applying the stylesheet inline2XSF.xsl to each input annotationSuccessively merging the resulting standoff XStandoff files into one with
mergeXSF.xslCreating an inline XStandoff instance via the stylesheet XSF2inline.xslApplying inline2XSF.xsl The six original annotation files are given by different annotations of Schiller's
'Die Bürgschaft' which cover several linguistic fields: words, morphemes, syllables,
verse, prose and phrase.As already stated in the single-layered
annotation instances are available at http://coli.lili.uni-bielefeld.de/Texttechnologie/Forschergruppe/Phase1/sekimo/internet-praesentation/buergschaft.html. shows an extract of one of the original annotations. In
case of missing namespaces these are created automatically by the stylesheet. For the sake
of readability the namespace prefix 'wort' was added to the elements derived from this
namespace.The original word level annotation of 'Die Bürgschaft'
(<code>buergschaft-wort.xml</code>)<wort:text xmlns:wort="http://www.xstandoff.net/buergschaft/wort">
<wort:body>
<wort:head>Die Bürgschaft</wort:head>
<wort:ws> </wort:ws>
<wort:bibl><wort:author>Friedrich Schiller</wort:author></wort:bibl>
<wort:ws> </wort:ws>
<wort:p>
<wort:s>
<wort:w>Zu</wort:w>
<wort:w>Dionys</wort:w>
<wort:w>dem</wort:w>
<wort:w>Tirannen</wort:w>
<wort:w>schlich</wort:w>
<wort:w>Damon</wort:w>,
<!--...-->
</wort:text>The corresponding primary data file buergschaft-pd.txt has the following plain text
content:Primary data of 'Die Bürgschaft' (<code>buergschaft-pd.txt</code>)Die Bürgschaft Friedrich Schiller
Zu Dionys dem Tirannen schlich Damon, den Dolch im Gewande, Ihn schlugen (...)The result of the invoked transformation (saxon buergschaft-wort.xml
inline2XSF.xsl primary-data=buergschaft-pd.txt) is an XStandoff instance
containing the converted annotation including references to the respective spans in the
primary data's character stream defined by the segments elements (cf. ). The converted word level annotation of 'Die Bürgschaft'
(<code>buergschaft-wort-xsf.xml</code>)<xsf:corpusData xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsf="http://www.xstandoff.net/2009/xstandoff/1.1"
xsi:schemaLocation="http://www.xstandoff.net/2009/xstandoff/1.1 ../xsd/xsf.xsd"
xsfVersion="1.1"
xml:id="wort-ns">
<xsf:primaryData start="0" end="5188">
<primaryDataRef xmlns="http://www.xstandoff.net/2009/xstandoff/1.1" uri="buergschaft-pd.txt"/>
</xsf:primaryData>
<xsf:segmentation>
<xsf:segment xml:id="seg1" type="char" start="0" end="5186"/>
<xsf:segment xml:id="seg2" type="char" start="0" end="14"/>
<xsf:segment xml:id="seg3" type="char" start="14" end="15"/>
<xsf:segment xml:id="seg4" type="char" start="15" end="33"/>
<xsf:segment xml:id="seg5" type="char" start="33" end="34"/>
<!--...-->
</xsf:segmentation>
<xsf:annotation>
<xsf:level xml:id="wort-ns-level1">
<xsf:layer xmlns:wort="http://www.xstandoff.net/buergschaft/wort" priority="0"
xsi:schemaLocation="http://www.xstandoff.net/buergschaft/wort ../xsd/buergschaft_wort.xsd">
<wort:text xsf:segment="seg1">
<wort:body xsf:segment="seg1">
<wort:head xsf:segment="seg2"/>
<wort:ws xsf:segment="seg3"/>
<wort:bibl xsf:segment="seg4">
<wort:author xsf:segment="seg4"/>
</wort:bibl>
<wort:ws xsf:segment="seg5"/>
<wort:p xsf:segment="seg6">
<wort:s xsf:segment="seg7">
<wort:w xsf:segment="seg8"/>
<wort:w xsf:segment="seg9"/>
<wort:w xsf:segment="seg10"/>
<!--...-->
</xsf:corpusData>The other five input annotations are transformed analogously, serving as
the basis for the next processing step: the application of the mergeXSF stylesheet.
Merging XStandoff instancesAs mentioned above, several different XStandoff instances relying on the same primary
data can be combined into a single XStandoff file. However, it is not yet possible to
integrate the respective annotation levels and layers within a single transformation call.
In order to get a single result instance, one has to use five calls in total. The first
call merges two of the instances (e.g. saxon -o <RESULT>.xml
buergschaft-wort-xsf.xml mergeXSF.xsl merge-with=buergschaft-silbe-xsf.xml
all-layer=1). The next four transformation calls merge the result
(<RESULT>.xml from the previous call) with the single
remaining XStandoff instances. Due to the stylesheet parameter all-layer which is set to 1 in the call, the result will
contain a layer which stores those elements present in all layers (cf. ). shows an extract of the
result of the five mergeXSF transformations. The merged XStandoff annotation of 'Die Bürgschaft'
(<code>buergschaft-wort-prosa-silbe-vers-phrase-morphem-xsf.xml</code>)<xsf:corpusData xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsf="http://www.xstandoff.net/2009/xstandoff/1.1"
xsfVersion="1.1"
xml:id="wort-ns-prosa-ns-silbe-ns-vers-ns-phrase-ns-morphem-ns"
xsi:schemaLocation="http://www.xstandoff.net/2009/xstandoff/1.1 ../xsd/xsf.xsd">
<!-- ... -->
<xsf:annotation>
<xsf:level xml:id="all-level">
<xsf:layer xmlns:all="http://www.xstandoff.net/2009/all" priority="0"
xsi:schemaLocation="http://www.xstandoff.net/2009/all ../xsd/xsf.xsd">
<all:text xmlns:vers="http://www.xstandoff.net/buergschaft/vers"
xsf:segment="seg1"
vers:a="b">
<all:body xsf:segment="seg1">
<all:head xsf:segment="seg2"/>
<!--...-->
</all:body>
</all:text>
</xsf:layer>
</xsf:level>
<xsf:level xml:id="wort-ns-level1">
<xsf:layer xmlns:wort="http://www.xstandoff.net/buergschaft/wort" priority="0"
xsi:schemaLocation="http://www.xstandoff.net/buergschaft/wort ../xsd/buergschaft_wort.xsd">
<wort:p xsf:segment="seg6">
<!--...-->
</wort:p>
</xsf:layer>
</xsf:level>
<xsf:level xml:id="prosa-ns-level1">
<xsf:layer xmlns:prosa="http://www.xstandoff.net/buergschaft/prosa" priority="0"
xsi:schemaLocation="http://www.xstandoff.net/buergschaft/prosa ../xsd/buergschaft_prosa.xsd">
<prosa:p xsf:segment="seg6">
<!--...-->
</xsf:corpusData>Converting standoff to inline The last step which has to be applied in order to get an inline XStandoff annotation
created from the six original input annotations is the transformation done by the
stylesheet XSF2inline.xsl (saxon -o <RESULT>.xml
buergschaft-wort-prosa-silbe-vers-phrase-morphem-xsf.xml XSF2inline.xsl).
Overlaps which may occur, are covered by insertion of the newly introduced
milestone element. The inline XStandoff annotation of 'Die Bürgschaft'
(<code>buergschaft-wort-prosa-silbe-vers-phrase-morphem-inline.xml</code>)<xsf:inline xmlns:xsf="http://www.xstandoff.net/2009/xstandoff/1.1"
xmlns:silbe="http://www.xstandoff.net/buergschaft/silbe"
xmlns:wort="http://www.xstandoff.net/buergschaft/wort"
xmlns:vers="http://www.xstandoff.net/buergschaft/vers"
xmlns:morphem="http://www.xstandoff.net/buergschaft/morphem"
xmlns:prosa="http://www.xstandoff.net/buergschaft/prosa"
xmlns:phrase="http://www.xstandoff.net/buergschaft/phrase"
xmlns:all="http://www.xstandoff.net/2009/all">
<all:text xsf:segment="seg1" vers:a="b">
<all:body xsf:segment="seg1">
<all:head xsf:segment="seg2">Die Bürgschaft</all:head>
<all:bibl xsf:segment="seg3">
<all:author xsf:segment="seg3">Friedrich Schiller</all:author>
</all:bibl>
<silbe:p xsf:segment="seg4">
<wort:p xsf:segment="seg4">
<vers:lg xsf:segment="seg4">
<morphem:p xsf:segment="seg4">
<prosa:p xsf:segment="seg4">
<phrase:p xsf:segment="seg4">
<!--...-->
<xsf:milestone xsf:segment="seg16~1" xsf:unit="morphem:m" xsf:n="1" xsf:charpos="49"
xsf:type="start"/>
<wort:w xsf:segment="seg15">
<silbe:syll xsf:segment="seg17">Ti</silbe:syll>
<silbe:syll xsf:segment="seg18">ran</silbe:syll>
<silbe:syll xsf:segment="seg19">n<xsf:milestone xsf:segment="seg16~2"
xsf:unit="morphem:m" xsf:n="2" xsf:charpos="55" xsf:type="end"/>
<morphem:m xsf:segment="seg20">en</morphem:m>
</silbe:syll>
</wort:w>
<!--...-->
</xsf:inline Note that the above listing contains the representation of an overlap. Since morpheme
and syllable level contain concurring annotations
(<morphem:m>Tiran<silbe:syll>n</morphem:m>en</silbe:syll>)
milestone elements have been added to mark the original boundaries of the morphem:m
element.Analyzing cross-level relations with XStandoff and XQuerySince XStandoff is a meta language based on standard XML, a variety of XML processing
tools may be used. In addition to the XStandoff toolkit an XQuery script is available as a
first starting point for analyzing relations between elements derived from different
annotation levels (and layers respectively). The query takes a valid XStandoff instance as
input and demands to provide a target element on an included layer which is used as starting
point for the finding of relations to other elements derived on different annotation layers.
For this example we have chosen the English translation of Brothers Grimm's 'The three lazy
ones' which was annotated with two different linguistic parsers: the freely available TreeTaggerDownload available at http://www.ims.uni-stuttgart.de/projekte/corplex/TreeTagger/. and the POS tagger that is part of the eHumanities DesktopThe eHumanities Desktop can be found at http://hudesktop.hucompute.org/. which can be used at no charge as an online resource (cf. ). The resulting XStandoff instance can be downloaded at http://www.xstandoff.net/examples/grimm.html. The output of the query contains the
target element and the corresponding elements on other layers, starting with elements that share
the same segment (and share therefore the identical text span in the primary data). The
relations identified by the XQuery are based on the work done by . An excerpt of the output can be seen in .Excerpt of the output of the XQuery applied to the 'The three lazy ones'<results>
<relations docID="grimm_the_three_lazy_ones">
<targetElement xmlns:xsf="http://www.xstandoff.net/2009/xstandoff/1.1" start="0" end="1"
name="eHD:w" xml:id="w1_wo1" type="XY" lemma="A" xsf:segment="seg3" part="N">
<identical start="0" end="1" name="tree:token" word="a" pos="DT" xsf:segment="seg3"/>
<startPointIdentical start="0" end="1091" name="eHD:text" xml:id="t0" xsf:segment="seg1"/>
<startPointIdentical start="0" end="1091" name="eHD:body" xml:id="b0" xsf:segment="seg1"/>
<startPointIdentical start="0" end="1091" name="eHD:div" xml:id="w1_di1" xsf:segment="seg1"
part="N" org="uniform" sample="complete"/>
<startPointIdentical start="0" end="1091" name="eHD:p" xml:id="w1_pa1" xsf:segment="seg1"/>
<startPointIdentical start="0" end="127" name="eHD:s" xml:id="w1_se1" xsf:segment="seg2"
part="N"/>
<startPointIdentical start="0" end="1091" name="tree:corpus" xsf:segment="seg1"/>
</targetElement>
<targetElement xmlns:xsf="http://www.xstandoff.net/2009/xstandoff/1.1" start="2" end="6"
name="eHD:w" xml:id="w1_wo2" type="NN" lemma="king" xsf:segment="seg4" part="N">
<identical start="2" end="6" name="tree:token" word="king" pos="NN" xsf:segment="seg4"/>
<inclusion start="0" end="1091" name="eHD:text" xml:id="t0" xsf:segment="seg1"/>
<inclusion start="0" end="1091" name="eHD:body" xml:id="b0" xsf:segment="seg1"/>
<inclusion start="0" end="1091" name="eHD:div" xml:id="w1_di1" xsf:segment="seg1" part="N"
org="uniform" sample="complete"/>
<inclusion start="0" end="1091" name="eHD:p" xml:id="w1_pa1" xsf:segment="seg1"/>
<inclusion start="0" end="127" name="eHD:s" xml:id="w1_se1" xsf:segment="seg2" part="N"/>
<inclusion start="0" end="1091" name="tree:corpus" xsf:segment="seg1"/>
</targetElement>
<!-- ... -->
</relations>
</results>In this example the eHD:w element (derived from eHumanities Desktop's
tagger) is used as target element. An element that shares the same segment of the primary data
is the tree:token element (marked as identical element) derived from
the TreeTagger parse. Since the target element is the first element other elements share the
same starting point, such as eHD:text, eHD:body,
eHD:div, eHD:p and eHD:s (on the same layer) and
tree:corpus (derived from the TreeTagger output). The second target element
(the second eHD:w element) is again identical to the tree:token element, and is
included (in the sense of containment, cf. ) by a variety of
other elements.In this scenario XStandoff can be used for different purposes: for comparing different
linguistic resources' output in terms of both quality and quantity of the annotation, for
refining and boosting overall annotation performance by combining several resources, and of
course for analyzing virtual parenthoods between elements derived from different annotation
layers (cf. for a real-world example).Conclusion and Future WorkWe have demonstrated both the further developments that have been made to the Sekimo
Generic Format, resulting in XStandoff, and the respective XSLT stylesheets that can be used
to generate and modify XStandoff instances. It is planned that XStandoff's current development
version will be finished as a stable version in time with XML Schema 1.1 which is in the
Working Draft status at the time of writing () and which would
allow us to dismiss the embedded Schematron assertions. Regarding the XSLT part of the
XStandoff toolkit, future enhancements are supposable in the support of the
containment/dominance feature and since the creation of XStandoff instances is a multi-step
process (cf. ), providing an XProc pipeline (cf. ) is another future option. Further perspectives include the building
of a corpus of multi-dimensional annotated (and possibly overlapping) markup for further
developments regarding both XStandoff's specification and the respective XSLT stylesheets (a
starting point has been made at http://www.xstandoff.net/examples/index.html). This corpus should be gathered
together with other interested parties, e.g. the Markup Languages for Complex Documents (MLCD)
project, the XCONCUR developers and the LMNL community. Alternative XQuery realizations of the
discussed XSLT stylesheets should give clues about further optimizations regarding the
processing times (cf. for a small test example). In
addition, it would be promising to share our developments with the framework for format
translations described by .Although XStandoff's formal model is considered as multi-rooted tree – at least
if one restricts oneself to not using discontinous segments – additional future work
has to be made regarding its expressiveness compared to other approaches, including GODDAG
structures (cf. , , and ) and LMNL. This holds
especially for an examination of the containment and dominance relations described in but also to other formal aspects of concurrent
markup.ReferencesAlink, W., Bhoedjang, R., de
Vries, A. P., and Boncz, P. A. Efficient XQuery Support for Stand-Off
Annotation. In: Proceedings of the 3rd International Workshop on XQuery
Implementation, Experience and Perspectives, in cooperation with ACM SIGMOD, Chicago, USA,
2006Alink, W., Jijkoun, V., Ahn,
D., and de Rijke, M. Representing and Querying Multi-dimensional Markup
for Question Answering. In: Proceedings of the 5th EACL Workshop on NLP and XML
(NLPXML-2006): Multi-Dimensional Markup in Natural Language Processing, Trento,
2006Bird, S. and Liberman, M.
Annotation graphs as a framework for multidimensional linguistic data
analysis. In: Proceedings of the Workshop "Towards Standards and Tools for
Discourse Tagging", pages 1–10. Association for Computational Linguistics, 1999Bird, S. and Liberman, M.
A formal framework for linguistic annotation. Speech
Communication, 33(1–2): pages 23–60, 2001. Doi: 10.1016/S0167-6393(00)00068-6Bird, S., Chen, Y., Davidson, S.,
Lee, H. and Zheng,Y. Designing and Evaluating an XPath Dialect for
Linguistic Queries. In: Proceedings of the 22nd International Conference on Data
Engineering (ICDE), Atlanta, USA, 2006. Doi: 10.1109/ICDE.2006.48Burnard, L., and Bauman,
S. (eds.). TEI P5: Guidelines for Electronic Text Encoding and
Interchange. published for the TEI Consortium by Humanities Computing Unit,
University of Oxford, Oxford, Providence, Charlottesville, Nancy, 2008Carletta, J., Kilgour, J.,
O’Donnel, T. J., Evert, S. and Voormann, H. The NITE Object Model
Library for Handling Structured Linguistic Annotation on Multimodal Data Sets.
In: Proceedings of the EACL Workshop on Language Technology and the Semantic Web (3rd Workshop
on NLP and XML (NLPXML-2003)), Budapest, Ungarn, 2003Carletta, J.; Evert, S.;
Heid, U. and Kilgour, J. The NITE XML Toolkit: data model and query
language. In: Language Resources and Evaluation, Springer, Dordrecht, 2005, 39,
pages 313-334. Doi: 10.1007/s10579-006-9001-9Cowan, J., Tennison J., and Piez,
W. LMNL update. In: Proceedings of Extreme Markup Languages,
Montréal, Québec, 2006DeRose, S. J. Markup Overlap: A Review and a Horse. In: Proceedings of Extreme Markup
Languages, Montréal, Québec, 2004Dipper, S. XML-based stand-off representation and exploitation of multi-level linguistic
annotation. In: Proceedings of Berliner XML Tage 2005 (BXML 2005), pages 39–50,
Berlin, Germany, 2005Dipper, S., Götze, M., Küssner,
U. and Stede, M. Representing and Querying Standoff XML. In:
Rehm, G., Witt, A. and Lemnitzer, L. (eds.), Datenstrukturen für linguistische Ressourcen und
ihre Anwendungen. Data Structures for Linguistic Resources and Applications. Proceedings of
the Biennial GLDV Conference 2007, pages 337–346, Tübingen, 2007. Gunter Narr
VerlagDurusau, P. and
O'Donnell, M.B.. Concurrent Markup for XML Documents. In:
Proceedings of the XML Europe conference 2002.Durusau, P. &
O'Donnel, M. B. Tabling the Overlap Discussion. In:
Proceedings of Extreme Markup Languages, Montréal, Québec, 2004Goecke, D., Lüngen, H.,
Metzing, D., Stührenberg, M. and Witt, A. Different Views on Markup.
Distinguishing levels and layers. In: Linguistic modeling of information and
Markup Languages. Contributions to language technology. Springer, 2009. To
appearGleim, R., Waltinger, U., Ernst,
A., Mehler, A., Esch, D., and Feith, T. The eHumanities Desktop
– An Online System for Corpus Management and Analysis in Support of Computing in
the Humanities. In: Proceedings of the Demonstrations Session of the 12th
Conference of the European Chapter of the Association for Computational Linguistics EACL 2009,
30 March – 3 April, Athens, 2009Huitfeldt,
C. and Sperberg-McQueen, C. M. TexMECS: An experimental markup
meta-language for complex documents. Markup Languages and Complex Documents
(MLCD) Project, February 2001Iacob, I. E. and Dekhtyar,
A. Processing XML documents with overlapping hierarchies In:
JCDL '05: Proceedings of the 5th ACM/IEEE-CS joint conference on Digital libraries, ACM Press,
2005, pages 409-409. Doi: 10.1145/1065385.1065513Iacob, I. E. and
Dekhtyar, A. Towards a Query Language for Multihierarchical XML:
Revisiting XPath. In: Proceedings of the 8th International Workshop on the Web
& Databases (WebDB 2005), 2005, pages 49-54Ide, N. and Romary, L. Towards International Standards for Language Resources. In: Dybkjaer,
L., Hemsen, H., and Minker, W., (eds.), Evaluation of Text and Speech Systems, pages 263-284.
SpringerIde, N. and Suderman, K.
GrAF: A Graph-based Format for Linguistic Annotations. In:
Proceedings of the Linguistic Annotation Workshop, pages 1-8, Prague, Czech Republic.
Association for Computational Linguistics, 2007ISO/IEC 19757-2:2003. Information technology – Document Schema Definition Language (DSDL) –
Part 2: Regular-grammar-based validation – RELAX NG (ISO/IEC 19757-2).
International Standard, International Organization for Standardization, Geneva,
2003ISO/IEC 19757-3:2006.
Information technology – Document Schema Definition Language (DSDL)
– Part 3: Rule-based validation – Schematron. International standard,
International Organization for Standardization, Geneva, 2006Jagadish, H. V.,
Lakshmanany, L. V. S., Scannapieco, M., Srivastava, D. and Wiwatwattana, N. Colorful XML: One hierarchy isn’t enough. In: Proceedings of ACM
SIGMOD International Conference on Management of Data (SIGMOD 2004), pages 251–262, Paris,
June 13-18 2004. ACM Press New York, NY, USA. Doi: 10.1145/1007568.1007598Kay, M. XSL
Transformations (XSLT) Version 2.0. World Wide Web Consortium. 2007. –
W3C RecommendationKay, M. XSLT 2.0 and
XPath 2.0 Programmer’s Reference. Wiley Publishing, Indianapolis, 4th edition,
2008Marinelli, P., Vitali,
F., and Zacchiroli, S. Towards the unification of formats for
overlapping markup. In: New Review of Hypermedia and Multimedia, 14(1): pages
57-94, 2008. Doi: 10.1080/13614560802316145Marcoux, Y. Graph characterization of overlap-only texmecs and other overlapping markup
formalisms. In: Proceedings of Extreme Markup Languages, Montréal, Québec,
2008Marcoux, Y. Variants of GODDAGs and suitable first-layer semantics. Presentation given at the
GODDAG workshop, Amsterdam, 1-5 December 2008Pianta, E. and
Bentivogli., L. Annotating Discontinuous Structures in XML: the
Multiword Case. In: Proceedings of LREC 2004 Workshop on ”XML-based richly
annotated corpora”, pages 30–37, Lisbon, Portugal. Poesio, M., Diewald, N.,
Stührenberg, M., Chamberlain, J., Jettka, D., Goecke, D. and Kruschwitz, U. Markup Infrastructure for the Anaphoric Bank, Part I: Supporting Web
Collaboration. In: Mehler, A., Kühnberger, K.-U., Lobin, H., Lüngen, H., Storrer,
A. and Witt, A. (eds.), Modelling, Learning and Processing of Text Technological Data
Structures, Dordrecht: Springer, Berlin, New York. To appearSchonefeld, O. XCONCUR and XCONCUR-CL: A constraint-based approach for the validation of
concurrent markup. In: Rehm, G., Witt, A., Lemnitzer, L. (eds.), Datenstrukturen
für linguistische Ressourcen und ihre Anwendungen. Data Structures for Linguistic Resources
and Applications. Proceedings of the Biennial GLDV Conference 2007, Tübingen, Germany, 2007.
Gunter Narr VerlagSperberg-McQueen, C. M., Huitfeldt, C. and Renear, A.. Meaning and
Interpretation of markup. Markup Languages – Theory & Practice,
2, pages 215-234, 2000. Doi: 10.1162/109966200750363599Sperberg-McQueen, C. M., Dubin, D., Huitfeldt, C. and Renear, A. Drawing inferences on the basis of markup. In: Proceedings of Extreme Markup
Languages, 2002Sperberg-McQueen, C. M. and Huitfeldt, C. GODDAG: A Data Structure for
Overlapping Hierarchies. In: King, P. and Munson, E. V. (eds.), Proceedings of
the 5th International Workshop on the Principles of Digital Document Processing (PODDP 2000),
volume 2023 of Lecture Notes in Computer Science, pages 139–160. Springer, 2004Sperberg-McQueen,
C. M. Rabbit/Duck grammars: a validation method for overlapping
structures. In: Proceedings of Extreme Markup Languages, Montréal, Québec,
2006Sperberg-McQueen,
C. M. Representation of overlapping structures. In:
Proceedings of Extreme Markup Languages, Montréal, Québec, 2007Sperberg-McQueen, C. M. and Huitfeldt, C. Markup Discontinued
Discontinuity in TexMecs, Goddag structures, and rabbit/duck grammars. Presented at Balisage: The Markup Conference 2008, Montréal, Canada, August 12 - 15, 2008. In: Proceedings of Balisage: The Markup Conference 2008. Balisage Series on Markup Technologies, vol. 1 (2008). Doi: 10.4242/BalisageVol1.Sperberg-McQueen01Sperberg-McQueen, C. M. and Huitfeldt, C. GODDAG. Presented at the Goddag workshop, Amsterdam, 1-5 December 2008Stührenberg, M.,
Goecke, D, Diewald, N., Cramer, I. and Mehler, A. Web-based annotation
of anaphoric relations and lexical chains. In: Proceedings of the Linguistic
Annotation Workshop (LAW), pages 140–147, Prague. Association for Computational Linguistics,
2007Stührenberg, M.
and Goecke, D.SGF – An integrated model for multiple
annotations and its application in a linguistic domain. Presented at Balisage: The Markup Conference 2008, Montréal, Canada, August 12 - 15, 2008. In: Proceedings of Balisage: The Markup Conference 2008. Balisage Series on Markup Technologies, vol. 1 (2008). Doi: 10.4242/BalisageVol1.Stuehrenberg01Tennison, J. Layered Markup and Annotation Language (LMNL). In: Proceedings of Extreme Markup
Languages, Montréal, Québec, 2002Tennison, J. Creole: Validating Overlapping Markup.In: Proceedings of XTech 2007: The
Ubiquitous Web Conference, 2007 Thompson, H. S. and
D. McKelvie. Hyperlink semantics for standoff markup of read-only
documents. In: Proceedings of SGML Europe ’97: The next decade – Pushing the
Envelope, pages 227–229, Barcelona, 1997Waltinger, U., Mehler, A.
Mehler, and Stührenberg, M. An Integrated Model of Lexical Chaining:
Application, Resources and its Format. Proceedings of the 9th Conference on
Natural Language Processing (KONVENS 2008)Walsh, N., Milowski, A., and Thompson, H.
S. (2009). XProc: An XML Pipeline Language. W3C Candidate Recommendation 28 May 2009, World
Wide Web Consortium. Witt, A. Meaning
and interpretation of concurrent markup. In: Proceedings of ALLC-ACH2002, Joint
Conference of the ALLC and ACH, 2002Witt, A. Multiple
hierarchies: New Aspects of an Old Solution. In: Proceedings of Extreme Markup
Languages, 2004Witt, A., Rehm, G., Hinrichs, E.,
Lehmberg, T. and Stegmann, J. SusTEInability of Linguistic Resources through Feature
Structures. In: Literary and Linguistic Computing, 24(3): pages 363-372, 2009. Doi: 10.1093/llc/fqp024Witt, A., Stührenberg, M.,
Goecke, D. and Metzing, D. Integrated Linguistic Annotation Models and
their Application in the Domain of Antecedent Detection. In: Mehler, A.,
Kühnberger, K.-U., Lobin, H., Lüngen, H., Storrer, A. and Witt, A. (eds.), Modelling, Learning
and Processing of Text Technological Data Structures, Dordrecht: Springer, Berlin, New York.
To appearW3C XML Schema
Definition Language (XSD) 1.1 Part 1: Structures. W3C Working Draft, World Wide Web
Consortium, W3C Candidate Recommendation 30 April 2009. Online: http://www.w3.org/TR/2009/CR-xmlschema11-1-20090430/