Hi all,

For the impatient: Below the longish text is a fully self-contained Python
example that illustrates my problem.

I'm struggling to understand xml.etree's handling of namespaces. I'm trying to
parse an Inkscape document which uses several namespaces. From etree's
documentation:

If the XML input has namespaces, tags and attributes with prefixes in the
form prefix:sometag get expanded to {uri}sometag where the prefix is
replaced by the full URI.

Which means that given an Element e, I cannot directly access its attributes
using e.get() because in order to do that I need to know the URI of the
namespace. So rather than doing this (see example below):

label = e.get('inkscape:label')

I need to do this:

label = e.get('{' + uri_inkscape_namespace + '}label')

....which is the method mentioned in etree's docs:

One way to search and explore this XML example is to manually add the URI
to every tag or attribute in the xpath of a find() or findall().
[...]
A better way to search the namespaced XML example is to create a
dictionary with your own prefixes and use those in the search functions.

Good idea! Better yet, that dictionary or rather, its reverse, already exists,
because etree has used it to unnecessarily mangle the namespaces in the first
place. The documentation doesn't mention where it can be found, but we can
just use the 'xmlns:' attributes of the <svg> root element to rebuild it. Or
so I thought, until I found out that etree deletes exactly these attributes
before handing the <svg> element to the user.

I'm really stumped here. Apart from the fact that I think XML is bloated shit
anyway and has no place outside HTML, I just don't get the purpose of etree's
way of working:

1) Evaluate 'xmlns:' attributes of the <svg> element
2) Use that info to replace the existing prefixes by {uri}
3) Realizing that using {uri} prefixes is cumbersome, suggest to
the user to build their own prefix -> uri dictionary
to undo the effort of doing 1) and 2)
4) ...but witholding exactly the information that existed in the original
document by deleting the 'xmlns:' attributes from the <svg> tag

Why didn't they leave the whole damn thing alone? Keep <svg> intact and keep
the attribute 'prefix:key' literally as they are. For anyone wanting to use
the {uri} prefixes (why would they) they could have thrown in a helper
function for the prefix->URI translation.

I'm assuming that etree's designers knew what they were doing in order to make
my life easier when dealing with XML. Maybe I'm missing the forest for the
trees. Can anybody enlighten me? Thanks!

#### self-contained example
import xml.etree.ElementTree as ET

def test_svg(xml):
root = ET.fromstring(xml)
for e in root.iter():
print(e.tag) # tags are shown prefixed with {URI}
if e.tag.endswith('svg'):
# Since namespaces are defined inside the <svg> tag, let's use the info
# from the 'xmlns:' attributes to undo etree's URI prefixing
print('Element <svg>:')
for k, v in e.items():
print(' %s: %s' % (k, v))
# ...but alas: the 'xmlns:' attributes have been deleted by the parser

xml = '''<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- Created with Inkscape (http://www.inkscape.org/) -->

<svg
width="210mm"
height="297mm"
viewBox="0 0 210 297"
version="1.1"
id="svg285"
inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
sodipodi:docname="test.svg"
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
xmlns="http://www.w3.org/2000/svg"
xmlns:svg="http://www.w3.org/2000/svg">
<sodipodi:namedview
id="namedview287"
pagecolor="#ffffff"
bordercolor="#000000"
borderopacity="0.25"
inkscape:showpageshadow="2"
inkscape:pageopacity="0.0"
inkscape:pagecheckerboard="0"
inkscape:deskcolor="#d1d1d1"
inkscape:document-units="mm"
showgrid="false"
inkscape:zoom="0.2102413"
inkscape:cx="394.78447"
inkscape:cy="561.25984"
inkscape:window-width="1827"
inkscape:window-height="1177"
inkscape:window-x="85"
inkscape:window-y="-8"
inkscape:window-maximized="1"
inkscape:current-layer="layer1" />
<defs
id="defs282" />
<g
inkscape:label="Ebene 1"
inkscape:groupmode="layer"
id="layer1">
<rect
style="fill:#aaccff;stroke-width:0.264583"
id="rect289"
width="61.665253"
height="54.114403"
x="33.978813"
y="94.38559" />
</g>
</svg>
'''

if __name__ == '__main__':
test_svg(xml)

I mean, it's worth to look at BeautifulSoup source how do they do that.
With BS I work with attributes exactly as you want, and I explicitly
tell BS to use lxml parser.

Axy.

On 19/10/2022 14:25, Robert Latest via Python-list wrote:
> Hi all,
>
> For the impatient: Below the longish text is a fully self-contained Python
> example that illustrates my problem.
>
> I'm struggling to understand xml.etree's handling of namespaces. I'm trying to
> parse an Inkscape document which uses several namespaces. From etree's
> documentation:
>
> If the XML input has namespaces, tags and attributes with prefixes in the
> form prefix:sometag get expanded to {uri}sometag where the prefix is
> replaced by the full URI.
>
> Which means that given an Element e, I cannot directly access its attributes
> using e.get() because in order to do that I need to know the URI of the
> namespace. So rather than doing this (see example below):
>
> label = e.get('inkscape:label')
>
> I need to do this:
>
> label = e.get('{' + uri_inkscape_namespace + '}label')
>
> ...which is the method mentioned in etree's docs:
>
> One way to search and explore this XML example is to manually add the URI
> to every tag or attribute in the xpath of a find() or findall().
> [...]
> A better way to search the namespaced XML example is to create a
> dictionary with your own prefixes and use those in the search functions.
>
> Good idea! Better yet, that dictionary or rather, its reverse, already exists,
> because etree has used it to unnecessarily mangle the namespaces in the first
> place. The documentation doesn't mention where it can be found, but we can
> just use the 'xmlns:' attributes of the <svg> root element to rebuild it. Or
> so I thought, until I found out that etree deletes exactly these attributes
> before handing the <svg> element to the user.
>
> I'm really stumped here. Apart from the fact that I think XML is bloated shit
> anyway and has no place outside HTML, I just don't get the purpose of etree's
> way of working:
>
> 1) Evaluate 'xmlns:' attributes of the <svg> element
> 2) Use that info to replace the existing prefixes by {uri}
> 3) Realizing that using {uri} prefixes is cumbersome, suggest to
> the user to build their own prefix -> uri dictionary
> to undo the effort of doing 1) and 2)
> 4) ...but witholding exactly the information that existed in the original
> document by deleting the 'xmlns:' attributes from the <svg> tag
>
> Why didn't they leave the whole damn thing alone? Keep <svg> intact and keep
> the attribute 'prefix:key' literally as they are. For anyone wanting to use
> the {uri} prefixes (why would they) they could have thrown in a helper
> function for the prefix->URI translation.
>
> I'm assuming that etree's designers knew what they were doing in order to make
> my life easier when dealing with XML. Maybe I'm missing the forest for the
> trees. Can anybody enlighten me? Thanks!
>
>
> #### self-contained example
> import xml.etree.ElementTree as ET
>
> def test_svg(xml):
> root = ET.fromstring(xml)
> for e in root.iter():
> print(e.tag) # tags are shown prefixed with {URI}
> if e.tag.endswith('svg'):
> # Since namespaces are defined inside the <svg> tag, let's use the info
> # from the 'xmlns:' attributes to undo etree's URI prefixing
> print('Element <svg>:')
> for k, v in e.items():
> print(' %s: %s' % (k, v))
> # ...but alas: the 'xmlns:' attributes have been deleted by the parser
>
> xml = '''<?xml version="1.0" encoding="UTF-8" standalone="no"?>
> <!-- Created with Inkscape (http://www.inkscape.org/) -->
>
> <svg
> width="210mm"
> height="297mm"
> viewBox="0 0 210 297"
> version="1.1"
> id="svg285"
> inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
> sodipodi:docname="test.svg"
> xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
> xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
> xmlns="http://www.w3.org/2000/svg"
> xmlns:svg="http://www.w3.org/2000/svg">
> <sodipodi:namedview
> id="namedview287"
> pagecolor="#ffffff"
> bordercolor="#000000"
> borderopacity="0.25"
> inkscape:showpageshadow="2"
> inkscape:pageopacity="0.0"
> inkscape:pagecheckerboard="0"
> inkscape:deskcolor="#d1d1d1"
> inkscape:document-units="mm"
> showgrid="false"
> inkscape:zoom="0.2102413"
> inkscape:cx="394.78447"
> inkscape:cy="561.25984"
> inkscape:window-width="1827"
> inkscape:window-height="1177"
> inkscape:window-x="85"
> inkscape:window-y="-8"
> inkscape:window-maximized="1"
> inkscape:current-layer="layer1" />
> <defs
> id="defs282" />
> <g
> inkscape:label="Ebene 1"
> inkscape:groupmode="layer"
> id="layer1">
> <rect
> style="fill:#aaccff;stroke-width:0.264583"
> id="rect289"
> width="61.665253"
> height="54.114403"
> x="33.978813"
> y="94.38559" />
> </g>
> </svg>
> '''
>
> if __name__ == '__main__':
> test_svg(xml)

On 2022-10-19, Robert Latest <boblatest@yahoo.com> wrote:
> If the XML input has namespaces, tags and attributes with prefixes
> in the form prefix:sometag get expanded to {uri}sometag where the
> prefix is replaced by the full URI.
>
> Which means that given an Element e, I cannot directly access its attributes
> using e.get() because in order to do that I need to know the URI of the
> namespace.

That's because you *always* need to know the URI of the namespace,
because that's its only meaningful identifier. If you assume that a
particular namespace always uses the same prefix then your code will be
completely broken. The following two pieces of XML should be understood
identically:

<svg xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape">
<g inkscape:label="Ebene 1" inkscape:groupmode="layer" id="layer1">

and:

<svg xmlns:epacskni="http://www.inkscape.org/namespaces/inkscape">
<g epacskni:label="Ebene 1" epacskni:groupmode="layer" id="layer1">

So you can see why e.get('inkscape:label') cannot possibly work, and why
e.get('{http://www.inkscape.org/namespaces/inkscape}label') makes sense.

The xml.etree author obviously knew that this was cumbersome, and
hence you can do something like:

namespaces = {'inkspace': 'http://www.inkscape.org/namespaces/inkscape'}
element = root.find('inkspace:foo', namespaces)

which will work for both of the above pieces of XML.

But unfortunately as far as I can see nobody's thought about doing the
same for attributes rather than tags.

I have no idea why, I used to remove namespaces, following the advice
from stackoverflow:

https://stackoverflow.com/questions/4255277/lxml-etree-xmlparser-remove-unwanted-namespace

_ns_removal_xslt_transform = etree.XSLT(etree.fromstring('''
    <xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
    <xsl:output method="xml" indent="no"/>

    <xsl:template match="/|comment()|processing-instruction()">
        <xsl:copy>
          <xsl:apply-templates/>
        </xsl:copy>
    </xsl:template>

    <xsl:template match="*">
        <xsl:element name="{local-name()}">
          <xsl:apply-templates select="@*|node()"/>
        </xsl:element>
    </xsl:template>

    <xsl:template match="@*">
        <xsl:attribute name="{local-name()}">
          <xsl:value-of select="."/>
        </xsl:attribute>
    </xsl:template>
    </xsl:stylesheet>
'''))

xml_doc = _ns_removal_xslt_transform(

    etree.fromstring(my_xml_data)

)

Later on, when I worked with SVG, I used BeautifulSoup.

Axy.

Jon Ribbens wrote:
> That's because you *always* need to know the URI of the namespace,
> because that's its only meaningful identifier. If you assume that a
> particular namespace always uses the same prefix then your code will be
> completely broken. The following two pieces of XML should be understood
> identically:
>
> <svg xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" " rel="nofollow" target="_blank">http://www.inkscape.org/namespaces/inkscape">
> <g inkscape:label="Ebene 1" inkscape:groupmode="layer" id="layer1">
>
> and:
>
> <svg xmlns:epacskni="http://www.inkscape.org/namespaces/inkscape">
> <g epacskni:label="Ebene 1" epacskni:groupmode="layer" id="layer1">
>
> So you can see why e.get('inkscape:label') cannot possibly work, and why
> e.get('{http://www.inkscape.org/namespaces/inkscape}label') makes sense.

I get it. It does.

> The xml.etree author obviously knew that this was cumbersome, and
> hence you can do something like:
>
> namespaces = {'inkspace': 'http://www.inkscape.org/namespaces/inkscape'}
> element = root.find('inkspace:foo', namespaces)
>
> which will work for both of the above pieces of XML.

Makes sense. It forces me to make up my own prefixes which I can then safely
use in my code rather than relying on the xml's generator to not change their
prefixes.

BTW, I only now thought to look at what actually is at Inkscape's namespace
URI, and it turns out to be quite a nice explanation of what a namespace is and
why it looks like a URL.