On the prolog thread, somebody posted a link to:
<https://dirtsimple.org/2004/12/python-is-not-java.html>

One thing that it tangentially says is "XML is not the answer."

I read this page right when I was about to write an XML parser
to get data into the code for a research project I'm working on.
It seems to me that XML is the right approach for this sort of
thing, especially since the data is hierarchical in nature.

Does the advice on that page mean that I should find some other
way to get data into my programs, or does it refer to some kind
of misuse/abuse of XML for something that it wasn't designed
for?

If XML is not the way to package data, what is the recommended
approach?
--
Michael F. Stemper
Life's too important to take seriously.

On 2021-09-21, Michael F. Stemper <michael.stemper@gmail.com> wrote:
> On the prolog thread, somebody posted a link to:
><https://dirtsimple.org/2004/12/python-is-not-java.html>
>
> One thing that it tangentially says is "XML is not the answer."
>
> I read this page right when I was about to write an XML parser
> to get data into the code for a research project I'm working on.
> It seems to me that XML is the right approach for this sort of
> thing, especially since the data is hierarchical in nature.
>
> Does the advice on that page mean that I should find some other
> way to get data into my programs, or does it refer to some kind
> of misuse/abuse of XML for something that it wasn't designed
> for?
>
> If XML is not the way to package data, what is the recommended
> approach?

I'd agree that you should not use XML unless the data is being supplied
already in XML format or perhaps if there is already a schema defined in
XML for exactly your purpose.

If there is nothing pre-existing to build upon then I'd suggest JSON.

If anyone suggests YAML, then you should just back slowly away while
speaking in a low calm voice until you have reached sufficient safe
distance, then turn and run.

On Tue, 21 Sep 2021 13:12:10 -0500, Michael F. Stemper wrote:

> On the prolog thread, somebody posted a link to:
> <https://dirtsimple.org/2004/12/python-is-not-java.html>
>
> One thing that it tangentially says is "XML is not the answer."
>
> I read this page right when I was about to write an XML parser to get
> data into the code for a research project I'm working on.
> It seems to me that XML is the right approach for this sort of thing,
> especially since the data is hierarchical in nature.
>
> Does the advice on that page mean that I should find some other way to
> get data into my programs, or does it refer to some kind of misuse/abuse
> of XML for something that it wasn't designed for?
>
> If XML is not the way to package data, what is the recommended approach?

1'st can I say don't write your own XML parser, there are already a
number of existing parsers that should do everything you will need. This
is a wheel that does not need re-inventing.

2nd if you are not generating the data then you have to use whatever data
format you are supplied

as far as I can see the main issue with XML is bloat, it tries to do too
many things & is a very verbose format, often the quantity of mark-up can
easily exceed the data contained within it.

other formats such a JSON & csv have far less overhead, although again
not always suitable.

As in all such cases it is a matter of choosing the most apropriate tool
for the job in hand.

--
Antonym, n.:
The opposite of the word you're trying to think of.

"Michael F. Stemper" <michael.stemper@gmail.com> writes:
>If XML is not the way to package data, what is the recommended
>approach?

I think no general answer can be given. You need to judge
for yourself what is suited best for your application.

I would not use the verb "package" in this context.
For me, XML is a notation for structured data.
So, one uses it to /write/ data.

I have not looked at the page you mentioned, but some say
XML is too verbose because of the end tags are repeating
the element type name. Others say this makes large elements
more readable for humans.

If your data is not too complicated and you do not foresee
problems, XML might be good enough and you should not invest
too much time into looking at alternatives.

>I was about to write an XML parser

One of the most important reasons to use XML is the
availability of prefabricated libraries and tools.
Writing one's own parser undermines the very thing that
is one of the most important benefits of using XML!

If you prefer a more lightweight document notation you can
look at:

- ini files
- YAML
- JSON
- S expressions (i.e., LISP notation)
- Python source files (which can also represent some type of data,
and instead of parsing them, one just can load them as a module)

And just for completenes, let me also mention SGML (which,
however, is not "lightweight"!). Erik Naggum, who was
writing about SGML, wanted to create his own notation,
which he called "Enamel", but this never got very far.

And, if readability for humans is less important, there are
ASN.1 and IFF (Python still has the chunk module to read IFF
chunked data).

. I am sometimes using my own notation, which I call
"Unotal". It addresses some of my own XML criticism. For
example, I want an element to possibly have more or less
than one type, to have several attributes with the same
name, and I want types and attributes to be structured.
(My Unotal parser for Python is not yet finished.)

I published this web page with some of my criticism of XML
(one can also use "http:" to access it):

https://www.purl.org/stefan_ram/pub/xml_data_representation_en

. About Unotal:

https://www.purl.org/stefan_ram/pub/unotal_en

And another page about a problem with XML, in German:

https://www.purl.org/stefan_ram/pub/xmldesign_element_oder_attribut_de

On 21/09/2021 13.49, alister wrote:
> On Tue, 21 Sep 2021 13:12:10 -0500, Michael F. Stemper wrote:
>
>> On the prolog thread, somebody posted a link to:
>> <https://dirtsimple.org/2004/12/python-is-not-java.html>
>>
>> One thing that it tangentially says is "XML is not the answer."
>>
>> I read this page right when I was about to write an XML parser to get
>> data into the code for a research project I'm working on.
>> It seems to me that XML is the right approach for this sort of thing,
>> especially since the data is hierarchical in nature.
>>
>> Does the advice on that page mean that I should find some other way to
>> get data into my programs, or does it refer to some kind of misuse/abuse
>> of XML for something that it wasn't designed for?
>>
>> If XML is not the way to package data, what is the recommended approach?
>
> 1'st can I say don't write your own XML parser, there are already a
> number of existing parsers that should do everything you will need. This
> is a wheel that does not need re-inventing.

I was going to build it on top of xml.etree.ElementTree

> 2nd if you are not generating the data then you have to use whatever data
> format you are supplied

It's my own research, so I can give myself the data in any format that I
like.

> as far as I can see the main issue with XML is bloat, it tries to do too
> many things & is a very verbose format, often the quantity of mark-up can
> easily exceed the data contained within it.
>
> other formats such a JSON & csv have far less overhead, although again
> not always suitable.

I've heard of JSON, but never done anything with it.

How does CSV handle hierarchical data? For instance, I have
generators[1], each of which has a name, a fuel and one or more
incremental heat rate curves. Each fuel has a name, UOM, heat content,
and price. Each incremental cost curve has a name, and a series of
ordered pairs (representing a piecewise linear curve).

Can CSV files model this sort of situation?

> As in all such cases it is a matter of choosing the most apropriate tool
> for the job in hand.

Naturally. That's what I'm exploring.

[1] The kind made of tons of iron and copper, filled with oil, and
rotating at 1800 rpm.

--
Michael F. Stemper
This sentence no verb.

"Michael F. Stemper" <michael.stemper@gmail.com> writes:

> On 21/09/2021 13.49, alister wrote:
>> On Tue, 21 Sep 2021 13:12:10 -0500, Michael F. Stemper wrote:
> It's my own research, so I can give myself the data in any format that I
> like.
>
>> as far as I can see the main issue with XML is bloat, it tries to do
>> too many things & is a very verbose format, often the quantity of
>> mark-up can easily exceed the data contained within it. other formats
>> such a JSON & csv have far less overhead, although again not always
>> suitable.
>
> I've heard of JSON, but never done anything with it.

Then you should certainly try to get a basic understanding of it. One
thing JSON shares with XML is that it is best left to machines to
produce and consume. Because both can be viewed in a text editor there
is a common misconception that they are easy to edit. Not so, commas are
a common bugbear in JSON and non-trivial edits in (XML unaware) text
editors are tricky.

Consider what overhead you should worry about. If you are concerned
about file sizes then XML, JSON and CSV should all compress to a similar
size.

> How does CSV handle hierarchical data? For instance, I have
> generators[1], each of which has a name, a fuel and one or more
> incremental heat rate curves. Each fuel has a name, UOM, heat content,
> and price. Each incremental cost curve has a name, and a series of
> ordered pairs (representing a piecewise linear curve).
>
> Can CSV files model this sort of situation?

The short answer is no. CSV files represent spreadsheet row-column
values with nothing fancier such as formulas or other redirections.

CSV is quite good as a lowest common denominator exchange format. I say
quite because I would characterize it by 8 attributes and you need to
pick a dialect such as MS Excel which sets out what those are. XML and
JSON are controlled much better. You can easily verify that you conform
to those and guarantee that *any* conformant parser can read your
content. XML is more powerful in that repect than JSON in that you can
define and enforce schemas. In your case the fuel name, UOM, etc. can be
validated with standard tools. In JSON all that checking is entirely
handled by the consuming program(s).

>> As in all such cases it is a matter of choosing the most apropriate tool
>> for the job in hand.
>
> Naturally. That's what I'm exploring.

You might also like to consider HDF5. It is targeted at large volumes of
scientific data and its capabilities are well above what you need.
MATLAB, Octave and Scilab use it as their native format. PyTables and
h2py provide Python/NumPy bindings to it.

--
Pete Forman

On Tue, 21 Sep 2021 14:22:52 -0500, Michael F. Stemper wrote:

> On 21/09/2021 13.49, alister wrote:
>> On Tue, 21 Sep 2021 13:12:10 -0500, Michael F. Stemper wrote:
>>
>>> On the prolog thread, somebody posted a link to:
>>> <https://dirtsimple.org/2004/12/python-is-not-java.html>
>>>
>>> One thing that it tangentially says is "XML is not the answer."
>>>
>>> I read this page right when I was about to write an XML parser to get
>>> data into the code for a research project I'm working on.
>>> It seems to me that XML is the right approach for this sort of thing,
>>> especially since the data is hierarchical in nature.
>>>
>>> Does the advice on that page mean that I should find some other way to
>>> get data into my programs, or does it refer to some kind of
>>> misuse/abuse of XML for something that it wasn't designed for?
>>>
>>> If XML is not the way to package data, what is the recommended
>>> approach?
>>
>> 1'st can I say don't write your own XML parser, there are already a
>> number of existing parsers that should do everything you will need.
>> This is a wheel that does not need re-inventing.
>
> I was going to build it on top of xml.etree.ElementTree
>
so not writing a parser, using one, that's ok

>> 2nd if you are not generating the data then you have to use whatever
>> data format you are supplied
>
> It's my own research, so I can give myself the data in any format that I
> like.
>
>> as far as I can see the main issue with XML is bloat, it tries to do
>> too many things & is a very verbose format, often the quantity of
>> mark-up can easily exceed the data contained within it.
>>
>> other formats such a JSON & csv have far less overhead, although again
>> not always suitable.
>
> I've heard of JSON, but never done anything with it.
the python json library makes it simple.
it was originally invented for javascript, it looks very much like the
repl for a list/dictionary but if you are using std libraries you don't
really need to know except for academic interst
>
> How does CSV handle hierarchical data?
It dosn't, if you have heirachiacl data it is not a suitable format
> For instance, I have
> generators[1], each of which has a name, a fuel and one or more
> incremental heat rate curves. Each fuel has a name, UOM, heat content,
> and price. Each incremental cost curve has a name, and a series of
> ordered pairs (representing a piecewise linear curve).
>
> Can CSV files model this sort of situation?
>
>> As in all such cases it is a matter of choosing the most apropriate
>> tool for the job in hand.
>
> Naturally. That's what I'm exploring.
>
>
> [1] The kind made of tons of iron and copper, filled with oil, and
> rotating at 1800 rpm.

--
Riches cover a multitude of woes.
-- Menander

ram@zedat.fu-berlin.de (Stefan Ram) writes:
<snip>
> - S expressions (i.e., LISP notation)

If you're looking at hierarchical data and you don't have some good
reason to use something else, this is very likely to be your simplest
option.

On 2021-09-21, Pete Forman <petef4+usenet@gmail.com> wrote:
> CSV is quite good as a lowest common denominator exchange format. I say
> quite because I would characterize it by 8 attributes and you need to
> pick a dialect such as MS Excel which sets out what those are. XML and
> JSON are controlled much better. You can easily verify that you conform
> to those and guarantee that *any* conformant parser can read your
> content. XML is more powerful in that repect than JSON in that you can
> define and enforce schemas. In your case the fuel name, UOM, etc. can be
> validated with standard tools. In JSON all that checking is entirely
> handled by the consuming program(s).

That's not true. You can use "JSON Schema" to create a schema
for validating JSON files, and there appear to be at least four
implementations in Python.

In comp.lang.python, Michael F. Stemper <michael.stemper@gmail.com> wrote:
> I've heard of JSON, but never done anything with it.

You probably have used it inadvertantly on a regular basis over the
past few years. Websites live on it.

> How does CSV handle hierarchical data? For instance, I have
> generators[1], each of which has a name, a fuel and one or more
> incremental heat rate curves. Each fuel has a name, UOM, heat content,
> and price. Each incremental cost curve has a name, and a series of
> ordered pairs (representing a piecewise linear curve).
>
> Can CSV files model this sort of situation?

Can a string of ones and zeros encode the sounds of Bach, the images
of his sheet music, the details to reproduce his bust in melted plastic
extruded from nozzle under the control of machines?

Yes, CSV files can model that. But it would not be my first choice of
data format. (Neither would JSON.) I'd probably use XML.

I rather suspect that all (many) of those genomes that end up in
Microsoft Excel files get there via a CSV export from a command line
tool. Once you can model life in CSV, everything seems possible.

> [1] The kind made of tons of iron and copper, filled with oil, and
> rotating at 1800 rpm.

Those are rather hard to model in CSV, too, but I'm sure it could be
done.

Elijah
------
for bonus round, use punched holes in paper to encode the ones and zeros

Eli the Bearded <*@eli.users.panix.com> writes:

> In comp.lang.python, Michael F. Stemper <michael.stemper@gmail.com> wrote:
>> I've heard of JSON, but never done anything with it.
>
> You probably have used it inadvertantly on a regular basis over the
> past few years. Websites live on it.

If the user has any interaction whatever with the formats being used to
transfer data then something is very, very wrong. Someone using a
website built on JSON isn't using JSON in any meaningful sense of the
term.

>> How does CSV handle hierarchical data? For instance, I have
>> generators[1], each of which has a name, a fuel and one or more
>> incremental heat rate curves. Each fuel has a name, UOM, heat content,
>> and price. Each incremental cost curve has a name, and a series of
>> ordered pairs (representing a piecewise linear curve).
>>
>> Can CSV files model this sort of situation?
>
> Can a string of ones and zeros encode the sounds of Bach, the images
> of his sheet music, the details to reproduce his bust in melted plastic
> extruded from nozzle under the control of machines?
>
> Yes, CSV files can model that. But it would not be my first choice of
> data format. (Neither would JSON.) I'd probably use XML.
>
> I rather suspect that all (many) of those genomes that end up in
> Microsoft Excel files get there via a CSV export from a command line
> tool. Once you can model life in CSV, everything seems possible.

Whenever someone asks "can this be done?" in any sort of computer
related question, the real question is "is this practical?" I have hazy
memories of seeing a Turing Machine implemented in an Excel spreadsheet,
so *anything* can, with sufficiently ridiculous amounts of work. That's
not really helpful here.

>> [1] The kind made of tons of iron and copper, filled with oil, and
>> rotating at 1800 rpm.
>
> Those are rather hard to model in CSV, too, but I'm sure it could be
> done.

So let's try to point him at representations that are easy.

On 9/21/21 11:12 AM, Michael F. Stemper wrote:

> It seems to me that XML is the right approach for this sort of
> thing, especially since the data is hierarchical in nature.

If you're looking for a format that you can read (as a human) and possibly hand-edit,
check out NestedText:

https://nestedtext.org/en/stable/

--
~Ethan~

On Tue, Sep 21, 2021 at 7:26 PM Michael F. Stemper <
michael.stemper@gmail.com> wrote:

> If XML is not the way to package data, what is the recommended
> approach?
>

I prefer both JSON and YAML over XML.

XML has both elements and tags, but it didn't really need both. This
results in more complexity than necessary. Also, XSLT and XPath are not
really all that simple.

But there's hope. If you're stuck with XML, you can use xmltodict, which
makes XML almost as easy as JSON.

HTH.

Jon Ribbens <jon+usenet@unequivocal.eu> writes:

> On 2021-09-21, Pete Forman <petef4+usenet@gmail.com> wrote:
>> CSV is quite good as a lowest common denominator exchange format. I
>> say quite because I would characterize it by 8 attributes and you
>> need to pick a dialect such as MS Excel which sets out what those
>> are. XML and JSON are controlled much better. You can easily verify
>> that you conform to those and guarantee that *any* conformant parser
>> can read your content. XML is more powerful in that repect than JSON
>> in that you can define and enforce schemas. In your case the fuel
>> name, UOM, etc. can be validated with standard tools. In JSON all
>> that checking is entirely handled by the consuming program(s).
>
> That's not true. You can use "JSON Schema" to create a schema for
> validating JSON files, and there appear to be at least four
> implementations in Python.

Fair point. It has been a while since I looked at JSON schemas and they
were rather less mature then.

--
Pete Forman

On 21/09/2021 16.21, Pete Forman wrote:
> "Michael F. Stemper" <michael.stemper@gmail.com> writes:
>> On 21/09/2021 13.49, alister wrote:
>>> On Tue, 21 Sep 2021 13:12:10 -0500, Michael F. Stemper wrote:
>> It's my own research, so I can give myself the data in any format that I
>> like.
>>
>>> as far as I can see the main issue with XML is bloat, it tries to do
>>> too many things & is a very verbose format, often the quantity of
>>> mark-up can easily exceed the data contained within it. other formats
>>> such a JSON & csv have far less overhead, although again not always
>>> suitable.
>>
>> I've heard of JSON, but never done anything with it.
>
> Then you should certainly try to get a basic understanding of it. One
> thing JSON shares with XML is that it is best left to machines to
> produce and consume. Because both can be viewed in a text editor there
> is a common misconception that they are easy to edit. Not so, commas are
> a common bugbear in JSON and non-trivial edits in (XML unaware) text
> editors are tricky.

Okay, after playing around with the example in Lubanovic's book[1]
I've managed to create a dict of dicts of dicts and write it to a
json file. It seems to me that this is how json handles hierarchical
data. Is that understanding correct?

Is this then the process that I would use to create a *.json file
to provide data to my various programs? Copy and paste the current
hard-coded assignment statements into REPL, use json.dump(dict,fp)
to write it to a file, and then read the file into each program
with json.load(fp)? (Actually, I'd write a function to do that,
just as I would with XML.)

> Consider what overhead you should worry about. If you are concerned
> about file sizes then XML, JSON and CSV should all compress to a similar
> size.

Not a concern at all for my current application.

>> How does CSV handle hierarchical data? For instance, I have
>> generators[1], each of which has a name, a fuel and one or more
>> incremental heat rate curves. Each fuel has a name, UOM, heat content,
>> and price. Each incremental cost curve has a name, and a series of
>> ordered pairs (representing a piecewise linear curve).
>>
>> Can CSV files model this sort of situation?
>
> The short answer is no. CSV files represent spreadsheet row-column
> values with nothing fancier such as formulas or other redirections.

Okay, that was what I suspected.

> CSV is quite good as a lowest common denominator exchange format. I say
> quite because I would characterize it by 8 attributes and you need to
> pick a dialect such as MS Excel which sets out what those are. XML and
> JSON are controlled much better. You can easily verify that you conform
> to those and guarantee that *any* conformant parser can read your
> content. XML is more powerful in that repect than JSON in that you can
> define and enforce schemas. In your case the fuel name, UOM, etc. can be
> validated with standard tools.

Yeah, validating against a DTD is pretty easy, since lxml.etree does all
of the work.

> In JSON all that checking is entirely
> handled by the consuming program(s).
Well, the consumer's (almost) always going to need to do *some*
validation. For instance, as far as I can tell, a DTD can't specify
that there must be at least two of a particular item.

The designers of DTD seem to have taken the advice of MacLennan[2]:
"The only reasonable numbers are zero, one, or infinity."

Which is great until you need to make sure that you have enough
points to define at least one line segment.

>>> As in all such cases it is a matter of choosing the most apropriate tool
>>> for the job in hand.
>>
>> Naturally. That's what I'm exploring.
>
> You might also like to consider HDF5. It is targeted at large volumes of
> scientific data and its capabilities are well above what you need.

Yeah, I won't be looking at more than five or ten generators at most. A
small number is enough to confirm or refute the behavior that I'm
testing.

[1] _Introducing Python: Modern Computing in Simple Packages_,
Second Release, (c) 2015, Bill Lubanovic, O'Reilly Media, Inc.
[2] _Principles of Programming Languages: Design, Evaluation,
and Implementation_, Second Edition, (c) 1987, Bruce J. MacLennan,
Holt, Rinehart, & Winston
--
Michael F. Stemper
No animals were harmed in the composition of this message.

On 21/09/2021 19.30, Eli the Bearded wrote:
> In comp.lang.python, Michael F. Stemper <michael.stemper@gmail.com> wrote:
>> I've heard of JSON, but never done anything with it.
>
> You probably have used it inadvertantly on a regular basis over the
> past few years. Websites live on it.

I used to use javascript when I was running Windows (up until 2009),
since it was the only programming language to which I had ready
access. Then I got a linux box and quickly discovered python. I
dropped javascript like a hot potato.

>> How does CSV handle hierarchical data? For instance, I have
>> generators[1], each of which has a name, a fuel and one or more
>> incremental heat rate curves. Each fuel has a name, UOM, heat content,
>> and price. Each incremental cost curve has a name, and a series of
>> ordered pairs (representing a piecewise linear curve).
>>
>> Can CSV files model this sort of situation?
>
> Can a string of ones and zeros encode the sounds of Bach, the images
> of his sheet music, the details to reproduce his bust in melted plastic
> extruded from nozzle under the control of machines?
>
> Yes, CSV files can model that. But it would not be my first choice of
> data format. (Neither would JSON.) I'd probably use XML.

Okay. 'Go not to the elves for counsel, for they will say both no
and yes.' (I'm not actually surprised to find differences of opinion.)

>> [1] The kind made of tons of iron and copper, filled with oil, and
>> rotating at 1800 rpm.
>
> Those are rather hard to model in CSV, too, but I'm sure it could be
> done.

> for bonus round, use punched holes in paper to encode the ones and zeros

I've done cardboard.

--
Michael F. Stemper
No animals were harmed in the composition of this message.

On Tue, 21 Sep 2021 13:12:10 -0500, "Michael F. Stemper"
<michael.stemper@gmail.com> declaimed the following:

>On the prolog thread, somebody posted a link to:
><https://dirtsimple.org/2004/12/python-is-not-java.html>
>
>One thing that it tangentially says is "XML is not the answer."
>
>I read this page right when I was about to write an XML parser
>to get data into the code for a research project I'm working on.
>It seems to me that XML is the right approach for this sort of
>thing, especially since the data is hierarchical in nature.
>
>Does the advice on that page mean that I should find some other
>way to get data into my programs, or does it refer to some kind
>of misuse/abuse of XML for something that it wasn't designed
>for?

There are some that try to use XML as a /live/ data /storage/ format
(such as http://www.drivehq.com/web/brana/pandora.htm which has to parse
XML files for all configuration data and filter definitions on start-up,
and update those files on any changes).

If you control both the data generation and the data consumption,
finding some format with less overhead than XML is probably to be
recommended. XML is more a self-documented (in theory) means of packaging
data for transport between widely disparate applications, which are likely
written by different teams, if not different companies, who only interface
via the definition of the data as seen by XML.

>
>If XML is not the way to package data, what is the recommended
>approach?

Again, if you control both generation and consumption... I'd probably
use an RDBM. SQLite tends to be packaged with Python [Windows] or, at the
least, the DB-API adapter [Linux tends to expect SQLite as a standard
installed item]. SQLite is a "file server" model (as is the JET engine used
by M$ Access) -- each application (instance) is directly accessing the
database file; there is no server process mediating access.

Hierarchical (since you mention that in later posts) would be
represented by relations (terminology from relational theory -- a "table"
to most) linked by foreign keys.

--
Wulfraed Dennis Lee Bieber AF6VN
wlfraed@ix.netcom.com http://wlfraed.microdiversity.freeddns.org/

On Tuesday, September 21, 2021 at 7:12:33 PM UTC+1, Michael F. Stemper wrote:
> On the prolog thread, somebody posted a link to:
> <https://dirtsimple.org/2004/12/python-is-not-java.html>

Me, but as the moderators on this group/mailing list have no objection to some people slamming Python where ever and whenever they like it, and when I object I get banned need I say more? Python doesn't discriminate against anybody unless you're on the autistic spectrum, in which case you can fuck off.

> --
> Michael F. Stemper
> Life's too important to take seriously.

On 22/09/2021 14.41, Mark Lawrence wrote:
> On Tuesday, September 21, 2021 at 7:12:33 PM UTC+1, Michael F. Stemper wrote:
>> On the prolog thread, somebody posted a link to:
>> <https://dirtsimple.org/2004/12/python-is-not-java.html>
>
> Me, but as the moderators on this group/mailing list have no objection to some people slamming Python where ever and whenever they like it, and when I object I get banned need I say more? Python doesn't discriminate against anybody unless you're on the autistic spectrum, in which case you can fuck off.

What on earth did I do to deserve this?

--
Michael F. Stemper
A preposition is something you should never end a sentence with.

On Wed, 22 Sep 2021 09:52:59 -0500, "Michael F. Stemper"
<michael.stemper@gmail.com> declaimed the following:

>On 21/09/2021 19.30, Eli the Bearded wrote:
>> In comp.lang.python, Michael F. Stemper <michael.stemper@gmail.com> wrote:
>>> How does CSV handle hierarchical data? For instance, I have
>>> generators[1], each of which has a name, a fuel and one or more
>>> incremental heat rate curves. Each fuel has a name, UOM, heat content,
>>> and price. Each incremental cost curve has a name, and a series of
>>> ordered pairs (representing a piecewise linear curve).
>>>
>>> Can CSV files model this sort of situation?
>>
<SNIP>
>> Yes, CSV files can model that. But it would not be my first choice of
>> data format. (Neither would JSON.) I'd probably use XML.
>
>Okay. 'Go not to the elves for counsel, for they will say both no
>and yes.' (I'm not actually surprised to find differences of opinion.)
>
You'd have to include a "level" (and/or data type if multiple objects
can be at the same level) field (as the first field) in CSV which
identifies how to parse the rest of the CSV data (well, technically, the
CSV module has "parsed" it -- in terms of splitting at commas, handling
quoted strings (which may contain commas which are not split points, etc.).

1-generator, name
2-fuel, name, UOM, heat-content, price
2-curve, name
3-point, X, Y
3-point, X, Y
....
2-curve, name
3-point, X, Y
3-point, X, Y
....

You extract objects at each level; if the level is the same or "lower"
(numerically -- higher in hierarchy) you attach the "previously" extracted
object to the parent object... Whether list or dictionary, or class
instance(s):

class Point():
#Point may be overkill, easier to just use a tuple (X, Y)
def __init__(self, X, Y):
self.X = X
self.Y = Y

class Curve():
def __init__(self, name):
self.name = name
self.points = []

#use as aCurve.points.append(currentPoint)

class Fuel():
def __init__(self, name, ..., price):
self.name = name
...
self.price = price

class Generator():
def __init__(self, name):
self.name = name
self.fuel = None
self.curves = []

#aGenerator.fuel = currentCurve
#aGenerator.curves.append(currentCurve)

--
Wulfraed Dennis Lee Bieber AF6VN
wlfraed@ix.netcom.com http://wlfraed.microdiversity.freeddns.org/

Am 22.09.21 um 16:52 schrieb Michael F. Stemper:
> On 21/09/2021 19.30, Eli the Bearded wrote:
>> Yes, CSV files can model that. But it would not be my first choice of
>> data format. (Neither would JSON.) I'd probably use XML.
>
> Okay. 'Go not to the elves for counsel, for they will say both no
> and yes.' (I'm not actually surprised to find differences of opinion.)

It is wrong, CSV has no model of hierarchical data. A CSV file is a 2d
table, just like a database table or an Excel sheet.

You can /layer/ high-dimensional data on top of a 2D table, there is the
relational algebra theory behind this, but it is wrong (or misleading at
best) to say that CSV can model hierarchical data.

It's the same as saying "CSV supports images". Of course it doesn't, its
a textfile, but you could encode a JPEG as base64 and then put this
string into the cell of a CSV table. That definitely isn't what a sane
person would understand as "support".

Christian

On 9/22/21 10:31, Dennis Lee Bieber wrote:

> If you control both the data generation and the data consumption,
> finding some format ...

This is really the key. I rant at people seeming to believe that csv is
THE data interchange format, and it's about as bad as it gets at that,
if you have a choice. xml is noisy but at least (potentially)
self-documenting, and ought to be able to recover from certain errors.
The problem with csv is that a substantial chunk of the world seems to
live inside Excel, and so data is commonly both generated in csv so it
can be imported into excel and generated in csv as a result of exporting
from excel, so the parts often are *not* in your control.

Sigh.

I didn't slam Python. In the end I found PyPy, and had good results.
But there are like two kind of lists, one moderated and one unmoderated.
For example my PyPy testing is not found here:

https://mail.python.org/pipermail/python-list/2021-September/thread.html#start

On the other and this thread is intact:

https://groups.google.com/g/comp.lang.python/c/JrZ-Zywmzwg/m/wEArUPblAwAJ

Also on pipermail there is a strange chronological reordering. My
last recorded post is from 16. Sept from then on I got censored,
whereas Schachner, Joseph post is from 14. Sept and follows

my post. So I guess the list is heavily manipulated. Which is a pitty,
since Python has nothing to hide. Its a good language.

Mark Lawrence schrieb am Mittwoch, 22. September 2021 um 21:41:14 UTC+2:
> On Tuesday, September 21, 2021 at 7:12:33 PM UTC+1, Michael F. Stemper wrote:
> > On the prolog thread, somebody posted a link to:
> > <https://dirtsimple.org/2004/12/python-is-not-java.html>
> Me, but as the moderators on this group/mailing list have no objection to some people slamming Python where ever and whenever they like it, and when I object I get banned need I say more? Python doesn't discriminate against anybody unless you're on the autistic spectrum, in which case you can fuck off.
> > --
> > Michael F. Stemper
> > Life's too important to take seriously.

On Thu, Sep 23, 2021 at 10:55 PM Mats Wichmann <mats@wichmann.us> wrote:
>
> On 9/22/21 10:31, Dennis Lee Bieber wrote:
>
> > If you control both the data generation and the data consumption,
> > finding some format ...
>
> This is really the key. I rant at people seeming to believe that csv is
> THE data interchange format, and it's about as bad as it gets at that,
> if you have a choice. xml is noisy but at least (potentially)
> self-documenting, and ought to be able to recover from certain errors.
> The problem with csv is that a substantial chunk of the world seems to
> live inside Excel, and so data is commonly both generated in csv so it
> can be imported into excel and generated in csv as a result of exporting
> from excel, so the parts often are *not* in your control.
>
> Sigh.

The only people who think that CSV is *the* format are people who
habitually live in spreadsheets. People who move data around the
internet, from program to program, are much more likely to assume that
JSON is the sole format. Of course, there is no single ultimate data
interchange format, but JSON is a lot closer to one than CSV is.

(Or to be more precise: any such thing as a "single ultimate data
interchange format" will be so generic that it isn't enough to define
everything. For instance, "a stream of bytes" is a universal data
interchange format, but that's not ultimately a very useful claim.)

ChrisA

Or its a problem with the thread view on pipermail.
If I use the date view I see again other stuff.

---- end excursion, wont interupt again ---

Mostowski Collapse schrieb am Donnerstag, 23. September 2021 um 15:27:19 UTC+2:
> I didn't slam Python. In the end I found PyPy, and had good results.
> But there are like two kind of lists, one moderated and one unmoderated.
> For example my PyPy testing is not found here:
>
> https://mail.python.org/pipermail/python-list/2021-September/thread.html#start
>
> On the other and this thread is intact:
>
> https://groups.google.com/g/comp.lang.python/c/JrZ-Zywmzwg/m/wEArUPblAwAJ
>
> Also on pipermail there is a strange chronological reordering. My
> last recorded post is from 16. Sept from then on I got censored,
> whereas Schachner, Joseph post is from 14. Sept and follows
>
> my post. So I guess the list is heavily manipulated. Which is a pitty,
> since Python has nothing to hide. Its a good language.
> Mark Lawrence schrieb am Mittwoch, 22. September 2021 um 21:41:14 UTC+2:
> > On Tuesday, September 21, 2021 at 7:12:33 PM UTC+1, Michael F. Stemper wrote:
> > > On the prolog thread, somebody posted a link to:
> > > <https://dirtsimple.org/2004/12/python-is-not-java.html>
> > Me, but as the moderators on this group/mailing list have no objection to some people slamming Python where ever and whenever they like it, and when I object I get banned need I say more? Python doesn't discriminate against anybody unless you're on the autistic spectrum, in which case you can fuck off.
> > > --
> > > Michael F. Stemper
> > > Life's too important to take seriously.