Rainer Bielefeld <rainerbielefeldng@bielefeldundbuss.de> wrote:

>Well, it IS fixed.

Adding that icon turned your Subject into RFC 2047 encoded-word, which
is not universally displayable. People with different newsreaders that
may be able to decode it won't have a way to do that if their terminal's
character set isn't set for the same one used in the encoding. The
result is Subject changes from one followup to the next to the next,
leaving the reader with an unreadable mess.

It doesn't affect trn4 which doesn't decode in followup, keeping
encoded-word intact, but I do have to remember to delete the extra "Re: ".

Subject: =?UTF-8?Q?Re:_news.eternal-september.org_Down_=e2=9c=85?=

My terminal can display it if I use the MIME decoder.

https://dogmamix.com/MimeHeadersDecoder/

I am pointing this out as a bad example of how aspects of MIME are being
used in ways that do not enhance communication. RFC 2047 was never a
News standard but a Mail-only standard that got inflicted upon News much
much later. It's not incorporated into older News clients, plenty of
which remain in common use. It's not backwards compatible.

To the extent that internationalization is desireable, use of MIME,
especially encoded-word on Subject, should be avoided in instances like
this that have nothing to do with internationalization.

"Adam H. Kerman" <ahk@chinet.com> wrote:

> https://dogmamix.com/MimeHeadersDecoder/

Thanks for that. I've often had to decode an encoded header, because
the poster uses a non-English language which meant non-ASCII characters
got used in the Subject.

Alas, most NNTP clients work their filters on the raw source of a
message, not on the rendered version the user sees. Defining a filter
on the rendered version you see doesn't work, because that string is not
present in the encoded string.

Because some posters use other languages, like with accented characters,
I don't filter out articles with encoded Subjects. However, in the case
of Bielefeld, he added a superfluous checkmark windding that added
nothing to his Subject (just a garbage character that forced encoding).

As for the RFC 2047, it is about Internet messages, like RFC 822, which
could be e-mail or any other transport to send messages. MIME crept
into newsgroups eons ago (RFC 1341, c.1992). It became necessary to
transport binary content, like images or attachments, in messages that
are required to be all ASCII characters. E-mail grew in its capability
of what it could transport, but by using encoding and MIME to extend
beyond the restrictive ASCII requirement, especially to support
non-English languages since Usenet and e-mail became a worldwide
phenomenon growing outside the USA where it originated. Usenet
similarly enlarged what content it could transport. E-mail had tires
that were solid rubber, but eventually changed to air-filled tires.
Many didn't like having to get stuck with solid tires in Usenet. The
decision to add MIME to Usenet was made decades ago. The requirement to
support only [non-encoded] ASCII was a detriment to worldwide
communication via Usenet.

However, you will get users that will encode or use MIME when not
appropriate, especally spammers that want to hide from filters. Adding
a superfluous glitzy checkmark character is what got Bielefeld's Subject
encoded.

VanguardLH <V@nguard.LH> wrote:
>"Adam H. Kerman" <ahk@chinet.com> wrote:

>>https://dogmamix.com/MimeHeadersDecoder/

>Thanks for that. I've often had to decode an encoded header, because
>the poster uses a non-English language which meant non-ASCII characters
>got used in the Subject.

>Alas, most NNTP clients work their filters on the raw source of a
>message, not on the rendered version the user sees. Defining a filter
>on the rendered version you see doesn't work, because that string is not
>present in the encoded string.

>Because some posters use other languages, like with accented characters,
>I don't filter out articles with encoded Subjects. However, in the case
>of Bielefeld, he added a superfluous checkmark windding that added
>nothing to his Subject (just a garbage character that forced encoding).

>As for the RFC 2047, it is about Internet messages, like RFC 822, which
>could be e-mail or any other transport to send messages. MIME crept
>into newsgroups eons ago (RFC 1341, c.1992).

That was never intended to be a Usenet RFC. In fact, weren't binaries on
Usenet long before MIME? Maybe I'm wrong. I'm not finding archived
newgroup messages for binaries groups that are that old.

>It became necessary to
>transport binary content, like images or attachments, in messages that
>are required to be all ASCII characters.

Yeah, well, this is why ghod invented FTP. Email was being abused with
attachments. It was never necessary.

>E-mail grew in its capability
>of what it could transport, but by using encoding and MIME to extend
>beyond the restrictive ASCII requirement, especially to support
>non-English languages since Usenet and e-mail became a worldwide
>phenomenon growing outside the USA where it originated.

It sort of worked without character set declarations. There were
conventions for which character set to use pre-MIME. It mostly worked.

The problem with MIME is that it's a be-all and end-all standard that
was NEVER intended for Usenet and most of it didn't solve Usenet's
problems. Generally, MIME added to Usenet's problems.

Character-set declaration was useful. It didn't believe in the same
standard as how attachments were defined. Each of MIME's constituent
parts should have been in separate standards. There was no reason that a
client couldn't be character-set declaration compliant without being
compliant with every other aspect of MIME. But to be MIME compliant, the
client has to do everything instead of the very specific things that are
useful for Usenet.

>Usenet similarly enlarged what content it could transport.

Uh, that's 'cuz with broadband pricing, the massive uploaders into
Usenet weren't being charged for the overwhelming burden they placed
upon the Internet. You don't recall the days that binary Usenet was the
number one portion of internetwork traffic?

Binary Usenet was a Tragedy of the Commons problem.

Binaries never belonged on Usenet without being correctly priced.

>E-mail had tires
>that were solid rubber, but eventually changed to air-filled tires.
>Many didn't like having to get stuck with solid tires in Usenet. The
>decision to add MIME to Usenet was made decades ago. The requirement to
>support only [non-encoded] ASCII was a detriment to worldwide
>communication via Usenet.

There was no "decision". People just started doing it. RFC 1036 hadn't
yet been replaced.

>. . .

VanguardLH <V@nguard.LH> wrote:
>"Adam H. Kerman" <ahk@chinet.com> wrote:

>>https://dogmamix.com/MimeHeadersDecoder/

>Thanks for that. I've often had to decode an encoded header, because
>the poster uses a non-English language which meant non-ASCII characters
>got used in the Subject. . . .

For me, it's a two-step process to de-MIME Subject. I take the output of
that and paste it into this:

https://dan.hersam.com/tools/smart-quotes.php

It does ASCII substitutions for open and close single and double quote,
points of ellipsis (which is its own Unicode character very often used
for points of suspension too), em dash, and a few others.

It does not do ASCII substitutions for accented letters or other
combining characters. Nor does it get rid of emojis and non-punctuation
or non-mathematical symbols.

"Adam H. Kerman" <ahk@chinet.com> wrote:

> VanguardLH <V@nguard.LH> wrote:
>>"Adam H. Kerman" <ahk@chinet.com> wrote:
>
>>>https://dogmamix.com/MimeHeadersDecoder/
>
>>Thanks for that. I've often had to decode an encoded header, because
>>the poster uses a non-English language which meant non-ASCII characters
>>got used in the Subject.
>
>>Alas, most NNTP clients work their filters on the raw source of a
>>message, not on the rendered version the user sees. Defining a filter
>>on the rendered version you see doesn't work, because that string is not
>>present in the encoded string.
>
>>Because some posters use other languages, like with accented characters,
>>I don't filter out articles with encoded Subjects. However, in the case
>>of Bielefeld, he added a superfluous checkmark windding that added
>>nothing to his Subject (just a garbage character that forced encoding).
>
>>As for the RFC 2047, it is about Internet messages, like RFC 822, which
>>could be e-mail or any other transport to send messages. MIME crept
>>into newsgroups eons ago (RFC 1341, c.1992).
>
> That was never intended to be a Usenet RFC.

It was intended for Internet messages. Transport method was not
mandated.

> In fact, weren't binaries on
> Usenet long before MIME? Maybe I'm wrong. I'm not finding archived
> newgroup messages for binaries groups that are that old.

Based only on my recollection, MIME came first in order to allow
attachments, like binaries (images, files). Later yEnc, a very poor
compression method, got popular in binaries. MIME bloats the size of
content by 1.37 times, or more, so not a favorite with Usenet posters,
especially with porn. yEnc compresses while keeping its string as ASCII
(https://en.wikipedia.org/wiki/YEnc), but you needed to use a client
that added yEnc support. I don't think Thunderbird ever added it. My
ancient client (Dialog) doesn't support yEnc, but then I don't care
since I don't visit binary newsgroups. I think Forte, Newsrover,
Newsbin, and other clients designed to rebuild binaries across split
posts have yEnc support. MIME started around 1992. yEnc started around
2001. Been a couple decades, or more, since I visited binary newsgroups
to see what they're using now.

>> It became necessary to transport binary content, like images or
>> attachments, in messages that are required to be all ASCII
>> characters.
>
> Yeah, well, this is why ghod invented FTP.

Not sure how FTP helps with multi-language support. Besides, haven't
you noticed the list of publicly accessible FTP servers has severely
waned over the last couple of decades? Like with e-mail, Usenet, and
other communication servers, most users don't run their own FTP servers,
plus that engenders a severely limited and restricted community.

> The problem with MIME is that it's a be-all and end-all standard that
> was NEVER intended for Usenet and most of it didn't solve Usenet's
> problems. Generally, MIME added to Usenet's problems.

My view, as as the RFCs describe, they were intended to apply to
Internet messaging regardless of the transport.

Since I don't do binary newsgroups, MIME is not applicable to text-only
newsgroups that I do visit. No HTML (which is all ASCII), no
attachments, and no encodings in headers. I also don't visit
non-English newsgroups, so no need to use header encodings there;
however, there are lots of newsgroups that I don't visit that are for
non-English speakers. Charsets don't apply to headers. All headers
must be in ASCII which means MIME encoding (which uses all ASCII chars)
is needed to account for non-English languages.

> You don't recall the days that binary Usenet was the
> number one portion of internetwork traffic?

Yeah, back when Usenet was the primary transport of porn. Now it's the
Web.

> Binaries never belonged on Usenet without being correctly priced.

Users use whatever is cheapest. Back then it was the Usenet.

> There was no "decision". People just started doing it. RFC 1036 hadn't
> yet been replaced.

Considering the process to propose, vote, and ratify an RFC, no, there
was a lot of decisions to define MIME for use with Internet messaging.

VanguardLH <V@nguard.LH> wrote:
>Adam H. Kerman <ahk@chinet.com> wrote:
>>VanguardLH <V@nguard.LH> wrote:
>>>Adam H. Kerman <ahk@chinet.com> wrote:

>>>>https://dogmamix.com/MimeHeadersDecoder/

>>>Thanks for that. I've often had to decode an encoded header, because
>>>the poster uses a non-English language which meant non-ASCII characters
>>>got used in the Subject.

>>>Alas, most NNTP clients work their filters on the raw source of a
>>>message, not on the rendered version the user sees. Defining a filter
>>>on the rendered version you see doesn't work, because that string is not
>>>present in the encoded string.

>>>Because some posters use other languages, like with accented characters,
>>>I don't filter out articles with encoded Subjects. However, in the case
>>>of Bielefeld, he added a superfluous checkmark windding that added
>>>nothing to his Subject (just a garbage character that forced encoding).
>
>>>As for the RFC 2047, it is about Internet messages, like RFC 822, which
>>>could be e-mail or any other transport to send messages. MIME crept
>>>into newsgroups eons ago (RFC 1341, c.1992).

>>That was never intended to be a Usenet RFC.

>It was intended for Internet messages. Transport method was not
>mandated.

"Internet message" refers to an email message, not a Usenet article. You
know this. Furthermore, not all Usenet articles are transported over the
Internet via NNTP, which you also know.

At no point have mail extensions generally applied to Usenet, all of
which were written without consideration of the effect upon Usenet.

>>In fact, weren't binaries on
>>Usenet long before MIME? Maybe I'm wrong. I'm not finding archived
>>newgroup messages for binaries groups that are that old.

>Based only on my recollection, MIME came first in order to allow
>attachments, like binaries (images, files).

I may not recall at all, but I thought there were binary parts as the
entire article body without being an attachment pre-MIME.

>Later yEnc, a very poor
>compression method, got popular in binaries.

I though binaries were UUENCODED before yEnc, but I don't recall any
longer. I never used binary Usenet.

>. . .

>>>It became necessary to transport binary content, like images or
>>>attachments, in messages that are required to be all ASCII
>>>characters.
>>Yeah, well, this is why ghod invented FTP.

>Not sure how FTP helps with multi-language support.

FTP was the file-transfer protocol. File transport in email wasn't
desireable.

>Besides, haven't you noticed the list of publicly accessible FTP
>servers has severely waned over the last couple of decades?

The file server would have been in addition to the Mail server. It
shouldn't have been publicly accessible for the kind of communication we
are discussion.

>Like with e-mail, Usenet, and
>other communication servers, most users don't run their own FTP servers,
>plus that engenders a severely limited and restricted community.

>>The problem with MIME is that it's a be-all and end-all standard that
>>was NEVER intended for Usenet and most of it didn't solve Usenet's
>>problems. Generally, MIME added to Usenet's problems.

>My view, as as the RFCs describe, they were intended to apply to
>Internet messaging regardless of the transport.

>Since I don't do binary newsgroups, MIME is not applicable to text-only
>newsgroups that I do visit. No HTML (which is all ASCII), no
>attachments, and no encodings in headers. I also don't visit
>non-English newsgroups, so no need to use header encodings there;
>however, there are lots of newsgroups that I don't visit that are for
>non-English speakers. Charsets don't apply to headers. All headers
>must be in ASCII which means MIME encoding (which uses all ASCII chars)
>is needed to account for non-English languages.

Obviously we agree. Nevertheless, encoded-word is in RFC 2047, which is
indeed one of the MIME standards.

That's my point. MIME never should have been an all-encompassing
standard. There should have been separate standards for its constituent
parts, then clients wouldn't have been required to be MIME compliant,
just compliant with useful portions.

>>You don't recall the days that binary Usenet was the
>>number one portion of internetwork traffic?

>Yeah, back when Usenet was the primary transport of porn. Now it's the
>Web.

It wasn't mostly pr0n. Movies currently booked into theaters could be
downloaded from Usenet.

>>Binaries never belonged on Usenet without being correctly priced.

>Users use whatever is cheapest. Back then it was the Usenet.

It wasn't "cheapest". It was free. Massive content uploaders weren't
being charged for the content they were uploaded and its phenomenal
impact on internetwork traffic. If they had to pay, binary Usenet would
have been much smaller and far more manageable.

>>There was no "decision". People just started doing it. RFC 1036 hadn't
>>yet been replaced.

>Considering the process to propose, vote, and ratify an RFC, no, there
>was a lot of decisions to define MIME for use with Internet messaging.

Not with respect to Usenet. Users just began doing it.

Adam H. Kerman schrieb:
> Adding that icon turned your Subject into RFC 2047 encoded-word,

Well, may be.

But what has this to do with "news.eternal-september.org Down"?

CU

Rainer

....

Rainer Bielefeld <rainerbielefeldng@bielefeldundbuss.de> wrote:

>...

What do your three dots have to do with Subject?

"Adam H. Kerman" <ahk@chinet.com> wrote:

> VanguardLH <V@nguard.LH> wrote:
>
>> It was intended for Internet messages. Transport method was not
>> mandated.
>
> "Internet message" refers to an email message, not a Usenet article.

Nope. Internet message does not mandate the transport protocol. That's
you adding your skew to the RFC definitions.

> You know this. Furthermore, not all Usenet articles are transported
> over the Internet via NNTP, which you also know.

Yep, lots of mailing lists and NNTP-to-HTTP gateways exist. There's
some proof to you that not all Usenet messages use just NNTP, and that
different transport protocols can be used.

> I thought there were binary parts as the entire article body without
> being an attachment pre-MIME.

Base64 encoding. Since specifying base64 means non-text only, I filter
those out since I only visit text-only newsgroups. Base64 got added to
allow binary content, but I don't do binaries in Usenet. Others do. In
fact, Jeff Layman's client seems to devolve into using base64 despite it
is not needed for the content of his messages.

https://en.wikipedia.org/wiki/Base64

Yep, as you say, I think base64 showed up before MIME, but base64
encoded text strings to represent binary data bloats the size of a
message (no compression) just as does MIME encoding binary to text. I
have no idea, and never cared to investigate, why yEnc became popular
instead of Zip, or other archive compression schemes available back
then.

>> Later yEnc, a very poor compression method, got popular in binaries.
>
> I though binaries were UUENCODED before yEnc, but I don't recall any
> longer. I never used binary Usenet.

Been so long since I've seen that scheme that I completely forgot about
uuencoding.

Another scheme of binary-to-text translation that started about 1980. I
didn't get into Usenet until around 1992. UU came from Unix-to-Unix
copying, so maybe it was more used by posters on *NIX platforms. Files
that were uuencoded now use base64, or MIME. Maybe Windows NNTP clients
did uuencoding back then, but I don't recall using any on Windows.
Seemed more of a *NIX thing. It's another binary-to-text translation,
the the content gets bloated in size. Compression can be used, like
first compressing a file before uuencoding, but does not obviate the
resulting file size after uuencoding is bloated compared to the original
binary. Also, typically binary files don't compress well, anyway. Most
of the image formats are already compressed. Executable don't compress
much unless they have large data blocks inside the .exe file.

>>>Yeah, well, this is why ghod invented FTP.
>
>>Not sure how FTP helps with multi-language support.
>
> FTP was the file-transfer protocol. File transport in email wasn't
> desireable.

Alas, many users still [ab]use e-mail as a file transport. Some users
just gotta add pics of their kids, kittens, vacation spots, etc. Even
now you could upload the binaries to online file servers where they can
be accessed for free, and put a URL to them in your e-mail or newsgroups
message. That requires logic, and some extra effort. That does not
describe the typical e-mail or newsgroups user.

Even Microsoft came out with a feature in MS Office/365 where adding an
attachment had the option to add to a document, or upload the file to a
OneDrive account and put a URL to it in the document. Mozilla added the
Filelink feature to Thunderbird to upload large attachments to cloud
storage, and put a URL in the message. However, I think an add-on is
needed to complete the feature
(https://addons.thunderbird.net/en-US/thunderbird/tag/filelink).

No matter what feature gets added to reduce message size (e-mail or
newsgroups) by offloading the binary content to somewhere else, like
cloud storage, users take the easy and lazy route hence we suffer MIME
and bloated message size.

>>Besides, haven't you noticed the list of publicly accessible FTP
>>servers has severely waned over the last couple of decades?
>
> The file server would have been in addition to the Mail server. It
> shouldn't have been publicly accessible for the kind of communication we
> are discussion.

You have an account with your e-mail provider who, according to your
scenario, also provides cloud storage via their FTP server for use by
your account. How is the recipient of your message with an FTP link
going to get that binary content? They likely don't have an account at
the same e-mail provider as do you. Even if uploads required using the
same account as for your e-mail provider, and downloads were free and
publicly accessible, that engenders a huge abuse vector. Someone
uploads a file, and thousands may download it severely impacting the
bandwith of the FTP server, and maybe even DDOS'ing it.

In addition to the manpower and resources needed to setup and maintain
an e-mail or NNTP server, now you want the provider to also invest
manpower and resources into setting up and maintaining an FTP server.
Perhaps that's a possibility with a paid e-mail service, but *tons* of
users utilize free e-mail services (Hotmail/Outlook.com, Yahoo, Gmail,
and so on). Yes, those e-mail providers could add FTP or cloud storage
(which some already do), but likely would only be available in a paid
service tier. All freeloaders of e-mail providers, like me and millions
of others, won't have all those servers you think can be tossed into an
integrated solution, so they will continue using what is cheap (free)
and easy to use since the clients already have support, like MIME,
base64, and maybe even uuencoding is still used by some. The clients
handle the transport without involving more protocols and servers.

> Obviously we agree. Nevertheless, encoded-word is in RFC 2047, which is
> indeed one of the MIME standards.

Yep, I'll agree that header encoding is a MIME extension.
I don't agree that MIME was intended only for e-mail transport.

>>Yeah, back when Usenet was the primary transport of porn. Now it's the
>>Web.
>
> It wasn't mostly pr0n. Movies currently booked into theaters could be
> downloaded from Usenet.

Oh, guess you're looking at the more recent history of Usenet usage. I
was talking about back with porn consumed a huge portion of bandwidth
and disk space at the Usenet providers (and probably before "Usenet"
became a term instead of saying NNTP or newsgroups). Back when I
started Usenet, I was using dialup, and Usenet came from Unix-to-Unix
Copy on dial-up architecture. That was back when BBSes were popular,
and Usenet was known by few.

>>Users use whatever is cheapest. Back then it was the Usenet.
>
> It wasn't "cheapest". It was free. Massive content uploaders weren't
> being charged for the content they were uploaded and its phenomenal
> impact on internetwork traffic. If they had to pay, binary Usenet would
> have been much smaller and far more manageable.

Back when I started, you either paid a Usenet provider, or you used the
one from your ISP whom you were paying already (it was an add-on service
which some ISPs had, some didn't, and some eventually dropped since it
didn't engender revenue). I don't recall any free NNTP servers that
were publicly available back then. There were between colleges, but
those weren't public.

I'm not sure who were the earliest free Usenet providers that I used. I
remember Motzarella (deliberately misnamed from the cheese, and died)
and Albasani (became Eternal-September). For awhile, I also used the
Usenet service of my ISP who was something called "Central <something>"
back then who got acquired by another company who got acquired by
Comcast who eventually dropped Usenet (c.2008). So, likely I started
using free Usenet providers when I could no longer get it with my ISP.
I only have very faint recollection that before using my ISP's Usenet
service that I was using it at work at Sperry Rand where I could get it
for free, and was used to get help to problems at work in Dev and QA.

>>>There was no "decision". People just started doing it. RFC 1036 hadn't
>>>yet been replaced.
>
>>Considering the process to propose, vote, and ratify an RFC, no, there
>>was a lot of decisions to define MIME for use with Internet messaging.
>
> Not with respect to Usenet. Users just began doing it.

Actually it would've first have been "NNTP client authors began adopting
MIME". Until the clients added support, users didn't have it. Quite
often, and more so back then than now, users would make requests for
changes or feature adds. The authors felt pressured to comply to keep
their products more current with existing standards. If enough users
complain, sometimes products adapt. Even Microsoft sometimes do that
although most of the time it feels like you're pressing a disconnected
crosswalk button.

Rainer Bielefeld <rainerbielefeldng@bielefeldundbuss.de> wrote:

> Adam H. Kerman:
>
>> Adding that icon turned your Subject into RFC 2047 encoded-word,
>
> Well, may be.
>
> But what has this to do with "news.eternal-september.org Down"?
>
> CU
>
> Rainer

Side topics often start within a discussion. You used a superfluous
non-ASCII character (checkmark) that forced your client to use MIME
encoding in your Subject header.

If you want to stick with the main topic, and never track into side
topics, stay at the root level of the discussion. The further down in a
subthread you go, the more likely the discussion digresses. It's like
going to party: people start all sorts of other discussions, but they
start from somewhere.

By the way, we already know who you are in the From header. A non-sig
repeat of your nym is unnecessary. Also, I'm curious what "CU" means.
To me, I think of the element copper. Maybe a phonetic representation
of "See You" (also something that should be in a sigblock). Oh joy,
more smartphone texting abbreviations infiltrating into Usenet.

"Adam H. Kerman" <ahk@chinet.com> wrote:

> Rainer Bielefeld <rainerbielefeldng@bielefeldundbuss.de> wrote:
>
>>...
>
> What do your three dots have to do with Subject?

He was berating us (for not changing the Subject for a side topic) for
berating his use of MIME encoded headers. With subthreads, often it is
unknown if the subthread becomes substantial when started. Yes, when a
subthread goes off-topic, netiquette says to use:

Subject: new topic (was: old topic)

Since it was just Kerman and I discussing the use of MIME encoding in
Usenet messages, wasn't something we were about to get lost when delving
into a subthread.

On 2023-11-06 01:44, VanguardLH wrote:
> "Adam H. Kerman" <ahk@chinet.com> wrote:
>
>> VanguardLH <V@nguard.LH> wrote:

....

>>> Later yEnc, a very poor compression method, got popular in binaries.
>>
>> I though binaries were UUENCODED before yEnc, but I don't recall any
>> longer. I never used binary Usenet.
>
> Been so long since I've seen that scheme that I completely forgot about
> uuencoding.
>
> Another scheme of binary-to-text translation that started about 1980. I
> didn't get into Usenet until around 1992. UU came from Unix-to-Unix
> copying, so maybe it was more used by posters on *NIX platforms. Files
> that were uuencoded now use base64, or MIME. Maybe Windows NNTP clients
> did uuencoding back then, but I don't recall using any on Windows.
> Seemed more of a *NIX thing. It's another binary-to-text translation,
> the the content gets bloated in size. Compression can be used, like
> first compressing a file before uuencoding, but does not obviate the
> resulting file size after uuencoding is bloated compared to the original
> binary. Also, typically binary files don't compress well, anyway. Most
> of the image formats are already compressed. Executable don't compress
> much unless they have large data blocks inside the .exe file.

I seem to recall uuencoding in Fidonet.

--
Cheers,
Carlos E.R.

VanguardLH <V@nguard.LH> wrote:
>"Adam H. Kerman" <ahk@chinet.com> wrote:
>>VanguardLH <V@nguard.LH> wrote:

>>>It was intended for Internet messages. Transport method was not
>>>mandated.

>>"Internet message" refers to an email message, not a Usenet article.

>Nope. Internet message does not mandate the transport protocol. That's
>you adding your skew to the RFC definitions.

The title of the RFCs have been "ARPA Network Text Messages", "ARPA
Internet Text Message". RFC 2822 "Internet Message Format" says,

Abstract

This standard specifies a syntax for text messages that are sent
between computer users, within the framework of "electronic mail"
messages. This standard supersedes the one specified in Request For
Comments (RFC) 822, "Standard for the Format of ARPA Internet Text
Messages", updating it to reflect current practice and incorporating
incremental changes that were specified in other RFCs.

It's not possible to interpret it the way you did.

>>You know this. Furthermore, not all Usenet articles are transported
>>over the Internet via NNTP, which you also know.

>Yep, lots of mailing lists and NNTP-to-HTTP gateways exist.

Not thinking of gateways but UUCP.

>>. . .

>Yep, I'll agree that header encoding is a MIME extension.
>I don't agree that MIME was intended only for e-mail transport.

Multipurpose Internet Mail Extensions

>. . .

>>>Users use whatever is cheapest. Back then it was the Usenet.

>>It wasn't "cheapest". It was free. Massive content uploaders weren't
>>being charged for the content they were uploaded and its phenomenal
>>impact on internetwork traffic. If they had to pay, binary Usenet would
>>have been much smaller and far more manageable.

>Back when I started, you either paid a Usenet provider, or you used the
>one from your ISP whom you were paying already (it was an add-on service
>which some ISPs had, some didn't, and some eventually dropped since it
>didn't engender revenue). I don't recall any free NNTP servers that
>were publicly available back then. There were between colleges, but
>those weren't public.

The massive content uploaders weren't charged for the massively excessive
internetwork bandwidth they consumed. If they had been, binary Usenet
traffic wouldn't have become overwhelming.

>. . .

Yep, you are correct. The later RFCs do mention the phrase "e-mail".

RFC 822 (c.1982) obsoleted by RFC 2822 (c.2001) obsoleted by RFC 5322
(c.2008). Yes, they mention "mail" and later mention "e-mail" applied
to text messaging in UUCP and ARPANET. Back then mass communication
venues were sparse, and most were private.

Arpanet and UUCP were about transport of text messages later called
e-mail. While the standards started from that source, they were not
limited to just that transport method. Usenet started around 1980, too
close to when RFC 822 was introduced. Typical adoption of an RFC by
clients and servers takes around 6 years. UUCP started around 1979.
From where would the standards come for messages via Usenet?

Even the few RFCs specifically stated for UUCP say "Mail Interchange
Format Standard" (RFC 976, 1137). Where are those RFCs for Usenet that
specify messaging format for that communication venue that is wholly
separate and independent of e-mail standards? I would be interested in
reading the Usenet-only RFCs that dictate messaging format that do not
borrow from the e-mail standards.

https://www.rfc-editor.org/rfc/rfc1036
Yep, it borrows messaging standards from RFC 822. It does say:

The USENET News standard is more restrictive than the Internet
standard, placing additional requirements on each message and
forbidding use of certain Internet features.

Alas, the restrictions are not mentioned. It also says:

In any situation where this standard conflicts with the Internet
standard, RFC-822 should be considered correct and this standard in
error.

RFC 1036 on Usenet messaging defers to the RFC 822 on e-mail messaging
format. RFC 5536 (c.2009) obsoletes RFC 1036 which again defers to RFC
5322 (e-mail), but also adds mention of RFC 2045 regarding MIME. So,
here we have an RFC specifically on netnews message format that mentions
the inclusion of MIME. Appendix B of RFC 5536 states:

Appendix B: Differences from RFC 1036 and Its Derivatives
o MIME is recognized as an integral part of Netnews.

MIME can be misused and abused, but it was adopted into Netnews (aka
Usenet) around 13 years ago by those that define the messaging standards
-- which isn't you nor I. Netnews message format evolved to encompass
evolving changes to e-mail (Internet) message format.

I have not yet found an RFC specifically on Netnews (Usenet) that
prohibits the use of MIME, or its extensions. In contrast, I found an
RFC for Netnews that includes MIME as valid for Usenet messaging. RFC
5536 was ratified in 2009, so us old farts were using Usenet long before
MIME got added, but Usenet message format is not controlled nor defined
by us old farts unless you become part of IETF to define the standards.
Time for us old farts to catch up. MIME isn't new to Usenet, and,
despite what you want, a lot of message formats for Usenet is defined by
e-mail standards.

https://www.ietf.org/standards/process/informal/

Yuck, no thanks. I just want to use it, not define it.

You can rile against MIME in Usenet, but you won't find a large audience
of compatriots. The old way doesn't mean it's the best way now. (*)
Everyone else moved on. That's what happens with old stuff: the
young'uns don't care about what was used over a decade ago, or before
they were born, just how it is now.

(*) The old way with cars was the car survived a crash, but not the
passengers. The new way has the cars not survive a crash, but the
passengers survive. New doesn't mandate better, just different, but
sometimes different is better.

Had a bonfire Halloween party (and a maze, too). Later at the fire, we
played the game where you pick a movie, and then pick an actor in that
movie that you linked to another movie they were in, then pick a
different actor in that movie, and keep the chain going of actors across
movies. We hit a movie that had John Wayne, and the kids looked
puzzled, and said "Who's John Wayne".

VanguardLH <V@nguard.LH> wrote:

>Yep, you are correct. The later RFCs do mention the phrase "e-mail".

>RFC 822 (c.1982) obsoleted by RFC 2822 (c.2001) obsoleted by RFC 5322
>(c.2008). Yes, they mention "mail" and later mention "e-mail" applied
>to text messaging in UUCP and ARPANET. Back then mass communication
>venues were sparse, and most were private.

>Arpanet and UUCP were about transport of text messages later called
>e-mail. While the standards started from that source, they were not
>limited to just that transport method. Usenet started around 1980, too
>close to when RFC 822 was introduced. Typical adoption of an RFC by
>clients and servers takes around 6 years. UUCP started around 1979.
>From where would the standards come for messages via Usenet?

Mail message format and Usenet article format are similar but not
identical. Generally, Usenet article format is stricter than Mail
message format, the most obvious example being ":" is the required
separator between the Header name and its contents in Mail, but ": " in
News. But they have always been two different media of communication.

>Even the few RFCs specifically stated for UUCP say "Mail Interchange
>Format Standard" (RFC 976, 1137). Where are those RFCs for Usenet that
>specify messaging format for that communication venue that is wholly
>separate and independent of e-mail standards? I would be interested in
>reading the Usenet-only RFCs that dictate messaging format that do not
>borrow from the e-mail standards.

There aren't any.

Communications protocols are defined in a parallel track to format. It's
always been Mail is sent in its format via its protocol and News is sent
in its format via its protocol. That's why there must be gateways between
two communications networks because Mail messages aren't being sent over
News networks.

Gatewaying goes back almost to the beginning of Usenet. Because
communication was incredibly expensive, mailing lists from overseas were
gated into Usenet in lieu of having overseas subscribers. But spam from
the "other" communications medium has almost entirely killed off gated
mailing lists on Usenet, which is too bad.

>https://www.rfc-editor.org/rfc/rfc1036
>Yep, it borrows messaging standards from RFC 822. It does say:
>
> The USENET News standard is more restrictive than the Internet
> standard, placing additional requirements on each message and
> forbidding use of certain Internet features.

>Alas, the restrictions are not mentioned.

Sure they are. I explained above that the Header name and its contents
have different separators, and there are some common required headers
between the two but certain required headers are found in one and not
the other.

>It also says:

> In any situation where this standard conflicts with the Internet
> standard, RFC-822 should be considered correct and this standard in
> error.

RFC 1036 had errors in it. That's why Son-Of was written and became so
very influential.

>RFC 1036 on Usenet messaging defers to the RFC 822 on e-mail messaging
>format. RFC 5536 (c.2009) obsoletes RFC 1036 which again defers to RFC
>5322 (e-mail), but also adds mention of RFC 2045 regarding MIME.

I am well aware. The incorporation of MIME by reference was a much much
later development.

Since I've been on Usenet pre-MIME, it's always been my opinion that
MIME was nonstandard on Usenet. Nevertheless it was used on Usenet long
before 2009. I think MIME is a bad idea. I still that think.

Usenet is weird. It has standards that aren't well followed. There
wasn't a lot of 5536 that was evolutionary. Most of it described what
was already taking place.

Here's an example: 1036 forbid spaces on the Newsgroups header when
crossposting. Well, there were plenty of newsreaders that ignored
that. USEFOR decided that the spaces weren't breaking anything, so the
Newsgroups header now allows spaces.

MIME wasn't allowed because it was an evolutionary development on Usenet
in 2009. Certain newsreaders were already doing it. The problem with
MIME was that it broke other newsreaders because it's an extension and
not backwards compatible. And very little of MIME, except for the
character set declaration, is desireable on Usenet.

The character set declarations, to some extent, thwarted
internationalization till use of UTF-8 became more widespread. The long
tradition on Usenet was that, depending on the main language being used
in the "language" hierarchy, a specific character set was used by
convention. There were some exceptions for specific Asian languages
which had multiple character sets going way back and I have no idea how
anyone decided which to use.

Well, one guy began using UTF-8 in an environment in which UTF-8 wasn't
in widespread use on the systems used by others posting to the
hierarchy. But hey! It was standard! He declared it in his MIME header!

But others writing in the same language couldn't read what he wrote.

>. . .

>You can rile against MIME in Usenet, but you won't find a large audience
>of compatriots. The old way doesn't mean it's the best way now. (*)
>Everyone else moved on.

That's false. I keep pointing out the obvious: Too many people use old
newsreaders, especially in the Mac environment.

Worse still, certain newsreaders truly suck. They get format=flowed
wrong, which is too bad as that's one of the few standards we ever got
that was truly backwards compatible. They post in QP or BASE64. They
have no ability to parse the proto-article to determine which character
set is actually in use. The character set declaration is simply based
on the user's settings. But in followup, or when copying from the Web,
the quoted material isn't necessarily in the character set the user set.

There are newsreaders that decode encoded-word on Subject, then fail to
re-encode it for the followup. Or, they re-encode it in a different
character set than the author of the root article used, and include "?"
substitutions for now undisplayable characters. Again, it's based on the
user's settings and not parsing.

No, standardizing MIME for Usenet didn't get old clients rewritten, and
some clients that use MIME get it wrong anyway.

That's why I like trn4 so much, because it does so LITTLE. It actually
recognizes attachments, so it's vaguely aware of MIME. But if I write
non-ASCII, trn4 expects me to write my own MIME headers.

trn4 doesn't do well with UTF-8 multibyte characters 'cuz it eats what
it thinks is a control character when reading. But in followup, because
it expects the user to choose his own text editor (I use vim), and it
relies upon the terminal emulation to translate, I can use multibyte
characters or 8-bit characters as long as I remember to declare.

>. . .

"Adam H. Kerman" <ahk@chinet.com> wrote:

> VanguardLH <V@nguard.LH> wrote:
>
>>Yep, you are correct. The later RFCs do mention the phrase "e-mail".
>
>>RFC 822 (c.1982) obsoleted by RFC 2822 (c.2001) obsoleted by RFC 5322
>>(c.2008). Yes, they mention "mail" and later mention "e-mail" applied
>>to text messaging in UUCP and ARPANET. Back then mass communication
>>venues were sparse, and most were private.
>
>>Arpanet and UUCP were about transport of text messages later called
>>e-mail. While the standards started from that source, they were not
>>limited to just that transport method. Usenet started around 1980, too
>>close to when RFC 822 was introduced. Typical adoption of an RFC by
>>clients and servers takes around 6 years. UUCP started around 1979.
>>From where would the standards come for messages via Usenet?
>
> Mail message format and Usenet article format are similar but not
> identical. Generally, Usenet article format is stricter than Mail
> message format, the most obvious example being ":" is the required
> separator between the Header name and its contents in Mail, but ": " in
> News. But they have always been two different media of communication.

I always thought the syntax for e-mail or netnews was:

headername: headervalue

The space always had to be there. In the decades that I've been in
using e-mail, the space always had to be there.

https://www.rfc-editor.org/rfc/rfc733.html (c.1977)
Section B.1.2
Describes syntax as headname:headervalue. No space either side of
colon.

https://datatracker.ietf.org/doc/html/rfc822 (c.1982)
Section 3.2 Header Field Definitions
Syntax is a colon with no leading or trailing space.

https://www.rfc-editor.org/rfc/rfc2822.html (c.2001)
Doesn't specify actual syntax, but the examples show no spaces around
the colon.

https://www.rfc-editor.org/info/rfc5322 (c.2008)
Says headername:headervalue for syntax.

I was spam reporting over a decade ago, and don't recall ever seeing a
space between the colon and the header's value, as in:

headername: headervalue

In fact, their examples show a space after the colon. Wait, nope, their
examples show 2 spaces after the colon. Wait, nope, some examples show
2, or more, spaces after the colon. They'll show the syntax as:

field = field-name ":" [ field-body ] CRLF
field-name = 1*<any CHAR, excluding CTLs, SPACE, and ":">
field-body = field-body-contents
[CRLF LWSP-char field-body]
field-body-contents =
<the ASCII characters making up the field-body, as
defined in the following sections, and consisting
of combinations of atom, quoted-string, and
specials tokens, or else consisting of texts>

I know some, not all headers, can have continuation lines by the next
line starting with whitespace, but the field-body syntax above indicates
a CRLF sequence could prefix the field-body as as a LWSP. I think the
syntax accounts for continuation lines by having CRLF LWSP (linear
whitespace which can be a space or tab char). The spaces shown above
are to show parsing, not part of the token values. If there's
whitespace allowed, they show a token for whitespace, like LWSP. So,
their syntax doesn't have a space after the colon, but their examples
do. Guess it has to do with what is used for delimiters and what can be
before or after them. RFC 5322 is a bit more clear, and consistent with
the examples.

2.2. Header Fields

Header fields are lines beginning with a field name, followed by a
colon (":"), followed by a field body, and terminated by CRLF. A
field name MUST be composed of printable US-ASCII characters (i.e.,
characters that have values between 33 and 126, inclusive), except
colon. A field body may be composed of printable US-ASCII characters
as well as the space (SP, ASCII value 32) and horizontal tab (HTAB,
ASCII value 9) characters (together known as the white space
characters, WSP).

So, the field-body (what I called headervalue) can have spaces anywhere
which includes at the beginning. I suspect a space is commonly added
after the colon to facilitate parsing. The colon should be sufficient,
but human often view the headers, too.

I then looked at the Netnews RFC.

https://www.rfc-editor.org/rfc/rfc5536.html
2.2. Header Fields
All header fields in a Netnews article are compliant with [RFC5322];
this specification, however, is less permissive in what can be
generated and accepted by agents. The syntax allowed for Netnews
article headers is a strict subset of the Internet Message Format
headers, making all headers compliant with this specification
inherently compliant with [RFC5322].
o All agents MUST generate header fields so that at least one space
immediately follows the ':' separating the header field name and
the header field body (for compatibility with deployed software,
including NNTP [RFC3977] servers). News agents MAY accept header
fields that do not contain the required space.

The RFC says a space must follow the colon delimiter. The e-mail RFCs
do not bar the field-body from containing a leading space (and after the
colon delimiter). The RFC mentions the NNTP RFC.

https://www.rfc-editor.org/rfc/rfc3977

But I didn't see where there would be a problem parsing headers on ":"
versus requiring an added space, as in ": ". Seems an NNTP server
fuckup that was compensated by adding a requirement for a space that
e-mail did not, but could be used, still matched on RFC syntax for
e-mail, and has been used for so long that I don't remember ever seeing
the colon in an e-mail header without a space after it.

"The headers of an article consist of one or more header lines. Each
header line consists of a header name, a colon, a space, the header
content, and a CRLF, in that order."

That merely is a subset of the e-mail header syntax where a leading
space is allowed, but optional per spec.

The e-mail RFCs may show headername:headervalue for syntax, but
headervalue can contain leading, between, and trailing spaces, the
examples show space(s) after the colon delimiter, and it seems
headername: headervalue became the de facto standard for e-mail headers
which still complied with the RFC dictated syntax. The e-mail RFCs
didn't bar the leading space to the field-body token. The Usenet RFCs
mandate the space is required. The Usenet RFC is a subset of the email
RFCs regarding header syntax.

Since parsing is just as easy on ":" as for ": ", I suspect the problem
with parsing headers was with early NNTP server software. As 2.2
mentions, the space was needed for NNTP servers (probably older ones),
not for the news agents (clients). Why Netnews required the leading
space to field-body is not mentioned in RFC 3977. Probably some
behind-the-scenes problems with NNTP server software. Required for
servers. Not required by clients.

> Sure they are. I explained above that the Header name and its contents
> have different separators, and there are some common required headers
> between the two but certain required headers are found in one and not
> the other.

Huh? What I see in the e-mail and Usenet RFCs is the delimiter is the
colon for both.

Email: headername:headervalue (*)
Usenet: headername: headervalue
(*) No preclusion of leading space to headervalue.

Colon is used for both e-mail and Netnews for the separator.

As to which headers are appropriate to which venue, yep, some are not
required to the other venue. That's not a restriction, just an artifact
of each venue.

> Too many people use old
> newsreaders, especially in the Mac environment.
>
> Worse still, certain newsreaders truly suck. They get format=flowed
> wrong, ...
>
> There are newsreaders that decode encoded-word on Subject, then fail to
> re-encode it for the followup. Or ...
>
> No, standardizing MIME for Usenet didn't get old clients rewritten, and
> some clients that use MIME get it wrong anyway.
>
> That's why I like trn4 so much, because it does so LITTLE. It actually
> recognizes attachments, so it's vaguely aware of MIME. But if I write
> non-ASCII, trn4 expects me to write my own MIME headers.
>
> trn4 doesn't do well with UTF-8 multibyte characters 'cuz it eats what
> it thinks is a control character when reading. But in followup, because
> it expects the user to choose his own text editor (I use vim), and it
> relies upon the terminal emulation to translate, I can use multibyte
> characters or 8-bit characters as long as I remember to declare.
>
>>. . .

Should protocol progress get stymied by users of ancient and incapable
clients? The only constant in the universe is things change. There are
some folks still using Lynx (c.1982) despite Mosaic showed up (c.1993).

On 2023-11-06, VanguardLH <V@nguard.LH> wrote:

> In any situation where this standard conflicts with the Internet
> standard, RFC-822 should be considered correct and this standard in
> error.

I always thought of RFC-[2]822 as a general-purpose standard, usable
for any type of message. But I'm just lazy.

I really don't like quoted printable, and my newsreader (slrn) can't
decode those blocks of base64 with which some people encode plaintext.
I think we've reached the point were we can and should just switch
to UTF-8 and be done with it. No encoding headers, and (especially!)
no byte-order marks polluting text files. (But that's another topic.)

> Had a bonfire Halloween party (and a maze, too). Later at the fire,
> we played the game where you pick a movie, and then pick an actor in
> that movie that you linked to another movie they were in, then pick a
> different actor in that movie, and keep the chain going of actors
> across movies.

The theatre group I was once a member of played a variant called
"Six Degrees of Kevin Bacon". Starting with a random movie,
the goal was to link to Kevin Bacon in as few jumps as possible.

> We hit a movie that had John Wayne, and the kids looked puzzled,
> and said "Who's John Wayne".

It works both ways. Who is Taylor Swift?
(For that matter, who is John Galt?)

--
/~\ Charlie Gibbs | The Internet is like a big city:
\ / <cgibbs@kltpzyxm.invalid> | it has plenty of bright lights and
X I'm really at ac.dekanfrus | excitement, but also dark alleys
/ \ if you read it the right way. | down which the unwary get mugged.

VanguardLH <V@nguard.LH> wrote:
>"Adam H. Kerman" <ahk@chinet.com> wrote:
>>VanguardLH <V@nguard.LH> wrote:

[Massive cut of quotes of RFCs because Vanguard refused to believe a
statement that I had made.]

>. . .

>>Sure they are. I explained above that the Header name and its contents
>>have different separators, and there are some common required headers
>>between the two but certain required headers are found in one and not
>>the other.

>Huh? What I see in the e-mail and Usenet RFCs is the delimiter is the
>colon for both.

Was it necessary for you to deliberately misstate syntax in this sentence,
having gone through the trouble of quoting RFCs?

>Email: headername:headervalue (*)
>Usenet: headername: headervalue
>(*) No preclusion of leading space to headervalue.

None of the RFCs said that the space was precluded.

>Colon is used for both e-mail and Netnews for the separator.

Stop deliberately misstating syntax, Vanguard. Colon is NOT the
separator in Usenet. The correct separator is ": ".

>As to which headers are appropriate to which venue, yep, some are not
>required to the other venue. That's not a restriction, just an artifact
>of each venue.

Don't put words in my mouth. It's not a "restriction". It's certainly
not an "artifact". Obviously, the To or Cc aren't required for Usenet as
Usenet isn't Mail.

>>Too many people use old newsreaders, especially in the Mac environment.

>>Worse still, certain newsreaders truly suck. They get format=flowed
>>wrong, ...

>>There are newsreaders that decode encoded-word on Subject, then fail to
>>re-encode it for the followup. Or ...

>>No, standardizing MIME for Usenet didn't get old clients rewritten, and
>>some clients that use MIME get it wrong anyway.

>>That's why I like trn4 so much, because it does so LITTLE. It actually
>>recognizes attachments, so it's vaguely aware of MIME. But if I write
>>non-ASCII, trn4 expects me to write my own MIME headers.

>>trn4 doesn't do well with UTF-8 multibyte characters 'cuz it eats what
>>it thinks is a control character when reading. But in followup, because
>>it expects the user to choose his own text editor (I use vim), and it
>>relies upon the terminal emulation to translate, I can use multibyte
>>characters or 8-bit characters as long as I remember to declare.

>>>. . .

>Should protocol progress get stymied by users of ancient and incapable
>clients? The only constant in the universe is things change.

The only constant in the univrse of Usenet, which is what we are
discussing, is that the standard it used for years was less than ideal,
other people did things that were quite common but not standard, and
that old clients remain in use because they met long-time users needs,
and no one is writing new Usenet clients.

So-called MIME aware clients can truly suck because they implemented
MIME character-set declarations without a parsing mechanism and truly
have no idea which character set is in use.

>There are some folks still using Lynx (c.1982) despite Mosaic showed
>up (c.1993).

Your analogy sucks. For one thing, the Web isn't that old. Lynx was a
terminal client from the early '90s. Mosaic was the first popular graphical
client. They serve two different functions. Actually, my recollection is
that work on Mosaic began earlier than Lynx but I'm not looking to up.

The reason Lynx continues to exist is its usefulness for reading text and
demonstrating that text order is useful to the reader. If a Web page is
renderable in a terminal, then reading devices used by the blind and low
vision can interact with that Web page. The blind cannot use graphics, duh.

>. . .

>Personally I don't care for MIME in Usenet, or base64, or UUencoded, or
>any crap that steps outside of using just ASCII, but I'll grant encoding
>for language internationalization. I also don't like quoted-printable
>or format=flowed since they get fucked over too often. That I don't
>like them, and don't use them, doesn't mean I'll condemn others that do.

Any use of QP is an indicator of a failed implementation, like Google
Groups. Unlike in email, there is no 7bit/8bit negotiation. If QP is
being output, that's a bad client.

Format=flowed should have worked, sigh. That standard went out of its wa
to be backwards compatible.

But character set declaration, failing to parse for the character set in
use, is a bad implementation.

>The RFCs allow MIME into Usenet.

Stop it. RFC 5536 was not evolutionary. MIME was in use already. It was
more of an acknowledgement.

>What's the alternative? Not bothering with RFCs or complying with them?
>Anarchy makes a bad substitute for communication protocols.

In the long history of RFC 1036, it wasn't being complied with. That's
Useet.

No, I say pre-MiME, users found a way to communicate because everybody
knew which character set was being used by everybody else. Things were
worse during a long era in which SOME people used UTF-8 whilst most
other people continued to use the appropriate 8-bit character set for
the European language in use, or a common character set for the Asian
language in use.

You're getting it really really wrong. Declaring UTF-8 DOES NOT force
the reader to think, Hey! I better change the fonts I have that are
available to the terminal!

When a handful of people began posting in UTF-8, it was absolutely not
in common use. Character set declarations do not change reality.

>I might not like some of the rules, I didn't make them, I have no
>control over them, but I still want to play.

That's nice. If the programmer died or abandoned the Usenet client, how
does your desire force another programmer to take up development of the
client? How do you force anybody to write clients for the Mac, given
complete and utter apathy?

Character set declaration does not solve the problem that different
character sets are in use at different ends.

That's not how the world works. Old clients remain in use because Usenet
is almost exclusively used by us doddering old folks.

Charlie Gibbs <cgibbs@kltpzyxm.invalid> wrote:
>On 2023-11-06, VanguardLH <V@nguard.LH> wrote:

>> In any situation where this standard conflicts with the Internet
>> standard, RFC-822 should be considered correct and this standard in
>> error.

>I always thought of RFC-[2]822 as a general-purpose standard, usable
>for any type of message. But I'm just lazy.

It ain't. USEFOR must comply with MESFOR, but with a different set of
required headers. MESFOR doesn't comply with USEFOR. One is specific to
Mail, the other to News, and neither applies to any other medium of
communication.

>I really don't like quoted printable, and my newsreader (slrn) can't
>decode those blocks of base64 with which some people encode plaintext.

Right now, BASE64 encoding is one of the tests Ray Banana and others are
using for NoCeMs as the Google-originating spam countermeasure. If any
such articles get through to you, it's best to add an entry in your kill
file.

trn4 doesn't decode. An outside process decodes, then presents te
decoded characters on screen, if my character-set declaration in the
terminal matches the character set in use. Remember, that can be
different on Subject versus the body of the article. Also, the MIME
declaration can be wrong, or multiple character sets were used in the
body thanks to quoting.

>I think we've reached the point were we can and should just switch
>to UTF-8 and be done with it. No encoding headers, and (especially!)
>no byte-order marks polluting text files. (But that's another topic.)

You don't like BIG ENDIAN versus LITTLE ENDIAN? Heh

Unencoded UTF-8 on headers would break everything, so let's not go
there.

>. . .

On 2023-11-06 19:23, Adam H. Kerman wrote:
> Charlie Gibbs <cgibbs@kltpzyxm.invalid> wrote:
>> On 2023-11-06, VanguardLH <V@nguard.LH> wrote:
>
>>> In any situation where this standard conflicts with the Internet
>>> standard, RFC-822 should be considered correct and this standard in
>>> error.
>
>> I always thought of RFC-[2]822 as a general-purpose standard, usable
>> for any type of message. But I'm just lazy.
>
> It ain't. USEFOR must comply with MESFOR, but with a different set of
> required headers. MESFOR doesn't comply with USEFOR. One is specific to
> Mail, the other to News, and neither applies to any other medium of
> communication.
>
>> I really don't like quoted printable, and my newsreader (slrn) can't
>> decode those blocks of base64 with which some people encode plaintext.
>
> Right now, BASE64 encoding is one of the tests Ray Banana and others are
> using for NoCeMs as the Google-originating spam countermeasure. If any
> such articles get through to you, it's best to add an entry in your kill
> file.

How do you that in Thunderbird?

After all, this is a Thunderbird group ;-)

--
Cheers,
Carlos E.R.

"Adam H. Kerman" <ahk@chinet.com> wrote:

> VanguardLH <V@nguard.LH> wrote:
>
>>"Adam H. Kerman" <ahk@chinet.com> wrote:
>>
>>> I explained above that the Header name and its contents have
>>> different separators, and there are some common required headers
>>> between the two but certain required headers are found in one and
>>> not the other.
>
>> Huh? What I see in the e-mail and Usenet RFCs is the delimiter is
>> the colon for both.
>
> Was it necessary for you to deliberately misstate syntax in this sentence,
> having gone through the trouble of quoting RFCs?
>
>> Email: headername:headervalue (*)
>> Usenet: headername: headervalue
>> (*) No preclusion of leading space to headervalue.

Sorry, I do keep saying colon instead of colon-space for Usenet;
however, then I clarify with the synopsis which does show colon-space
for Usenet. I don't know why Usenet is so inept at parsing of headers
that 2 characters are needed as a separator. However, in the above
synopsis, was I really wrong? I show colon-space for Usenet, and colon
for e-mail.

Despite the RFCs, what I've seen for header syntax in both e-mail and
Usenet has been:

headername: headervalue

The space for e-mail headers has been there for as long as I can
remember. Have you ever seen e-mail headers without the space after the
colon? I don't keep e-mails over 5 years old, so I have no older
examples for me to check if the space was ever missing in e-mail
headers.

>>Colon is used for both e-mail and Netnews for the separator.
>
> Stop deliberately misstating syntax, Vanguard. Colon is NOT the
> separator in Usenet. The correct separator is ": ".

Something in Usenet is retarded on parsing. The space is not required
for parsing, and the RFCs not the clients do not require the space.
While the Usenet RFCs defer to the e-mail RFCs, then they declare the
separator is a minimum of colon-space without stating why it had to be
different than for e-mail. Any idea why Usenet mandated the space after
the colon?

>>There are some folks still using Lynx (c.1982) despite Mosaic showed
>>up (c.1993).
>
> Your analogy sucks.

The analogy was not on what they do. It was on users remaining on old
software while others moved to new software. Plus not all Usenetizens
are old enough to have started with the old NNTP clients. After birth,
users are more likely what was recently created, not what existed before
they were born.

> The reason Lynx continues to exist is its usefulness for reading text and
> demonstrating that text order is useful to the reader.

There are still text-only editors (e.g., Notepad). It is still used;
however, it falls over when attempting to write documents which are
chaptered, indexed, has a TOC, stresses some statement, or add
citations. Old and limited still works, but that doesn't preclude using
new and usable. The newer clients may not be perfect, but were the old
clients perfect?

> If a Web page is renderable in a terminal, then reading devices used
> by the blind and low vision can interact with that Web page. The
> blind cannot use graphics, duh.

That wasn't my analogy, and I certainly wasn't going to make statement
that then had to account for every variation, like 0.62% of the
population being blind.

>>The RFCs allow MIME into Usenet.
>
> Stop it. RFC 5536 was not evolutionary. MIME was in use already. It was
> more of an acknowledgement.

Most RFCs are evolutionary. They were being implemented or de facto
standards before an RFC was considered necessary to eludicate and define
a standard. RFCs are not written in a vacuum.

> No, I say pre-MiME, users found a way to communicate because everybody
> knew which character set was being used by everybody else.

Charsets are not applicable to headers which are mandated to be ASCII.
Hence encoding was needed to account to non-ASCII characters, like those
in other languages. No matter what headers are in the message
specifying charset, they don't affect the headers themselves.

>>I might not like some of the rules, I didn't make them, I have no
>>control over them, but I still want to play.
>
> That's nice. If the programmer died or abandoned the Usenet client, how
> does your desire force another programmer to take up development of the
> client?

Well, guess what, you just described the NNTP client that I use. Like
you yourself mention, some old stuff is still viable today. However,
there is no way I would expect anyone new to Usenet, or looking for a
replacement NNTP client to consider the one that I'm using. As a base
client, Dialog is very good, because, like you, it stuck with old Usenet
standards, but it did add MIME support (in the body, and header
encoding). Dev died back in 2005 (but that update was buggy, so users
are recommended to go back to the .41 c.2002 version).

When Microsoft dumped Outlook Express into Windows, did all those users
go investigating on which NNTP client best complied with RFCs, was most
table, didn't violate de facto standards (that are not RFCs)? Nah, they
used what was handed to them. The era of OE in Usenet was probably
worse than when AOL dumped their users into Usenet and the annual
Eternal-September effect.

Why aren't there lots more newer NNTP clients? Well, authors aren't
interested, and invest elsewhere. You can't force a programmer to
develop anything, unless you pay them, of course. I haven't heard of
anyone prodding programmers with a salary to develop new NNTP clients.
Despite Usenet is no [completely] dead, it is to a lot of other folks.

When I trialed Dialog as a candidate for a replacement NNTP client, it
failed 3 times in several trials of NNTP client replacement. Not until
I invested the effort and time to implement macros for custom scripts
(toolbar buttons), message handling scripts, and event scripts was
Dialog comparable to other choices, and exceeded most. However, rare
few users looking for a replacement NNTP client will even consider going
through all that. They want a ready-made solution, not something they
have to tweak and write code.

https://en.wikipedia.org/wiki/List_of_Usenet_newsreaders

No, the count of NNTP clients is not huge, but that's not necessary.
Different clients have different behaviors and features, so the user
finds one, or two, that best fits their wants. At one time, I looked at
Xnews (and many others), but its author wasn't interested in user
feedback, RFEs (Requests for Enhancements), bug fixes, or anything from
the users. He made the client for his personal use, and made it
available to others as-is. There was something defective or deficient
with Xnews that had me look elsewhere. I might've paid for Forte Agent,
but found other more desirable candidates. I tried many NNTP client
over several trials spanning several years. Some aren't listed in the
Wikipedia article. I'm not typical of a Usenet user. They won't go
through all that. They found something, want something better, get a
suggestion, and move to that. They have a more than large enough
selection from which to choose, so the lack of recent development on new
NNTP clients to further enlarge the candidate list is of no interest to
them.

> How do you force anybody to write clients for the Mac, given
> complete and utter apathy?

I've always stayed away from anything Apple. Even a *NIX distro is
better, but I'm still on Windows running on my own-built boxes.

> Character set declaration does not solve the problem that different
> character sets are in use at different ends.

For viewing, my client has several fallbacks trying to compensate. For
posting, it uses us-ascii; however, on replies, it will fallback through
other charsets trying to match the parent article (if I quote something
in the parent article that is outside us-ascii).

Just like with e-mail, you cannot force someone else to view a Usenet
message how you constructed it. Freedom of choice, lack of choice,
ignorance: take your pick. Whatever I write and however I see it may
not be what someone sees. I stopped being elaborate in my messages
since whatever write could get mangled at the other end.

> That's not how the world works. Old clients remain in use because Usenet
> is almost exclusively used by us doddering old folks.

Probably why we don't think "Usenet is dying", but us old farts are.

On 2023-11-06 23:48, VanguardLH wrote:
> "Adam H. Kerman" <ahk@chinet.com> wrote:
>
>> VanguardLH <V@nguard.LH> wrote:
>>
>>> "Adam H. Kerman" <ahk@chinet.com> wrote:
>>>
>>>> I explained above that the Header name and its contents have
>>>> different separators, and there are some common required headers
>>>> between the two but certain required headers are found in one and
>>>> not the other.
>>
>>> Huh? What I see in the e-mail and Usenet RFCs is the delimiter is
>>> the colon for both.
>>
>> Was it necessary for you to deliberately misstate syntax in this sentence,
>> having gone through the trouble of quoting RFCs?
>>
>>> Email: headername:headervalue (*)
>>> Usenet: headername: headervalue
>>> (*) No preclusion of leading space to headervalue.
>
> Sorry, I do keep saying colon instead of colon-space for Usenet;
> however, then I clarify with the synopsis which does show colon-space
> for Usenet. I don't know why Usenet is so inept at parsing of headers
> that 2 characters are needed as a separator. However, in the above
> synopsis, was I really wrong? I show colon-space for Usenet, and colon
> for e-mail.

I just had a look in my Thunderbird, and what I see is both mail and
news having "headername: headervalue".

> Despite the RFCs, what I've seen for header syntax in both e-mail and
> Usenet has been:
>
> headername: headervalue
>
> The space for e-mail headers has been there for as long as I can
> remember. Have you ever seen e-mail headers without the space after the
> colon? I don't keep e-mails over 5 years old, so I have no older
> examples for me to check if the space was ever missing in e-mail
> headers.

I have.

Looking now at email from 1999, ant it has the space:

X-Mailer: Microsoft Outlook Express 4.72.3110.5

X-Mozilla-Status: 8001

X-mailer: AspMail 2.55 (SMTPD1B712)

As you can see, samples from different mail programs.

Saved news samples, the older I have in this machine is from 2016. Colon
space, too.

X-Newsreader: MesNews/1.08.03.00-es

2010:

X-Newsreader: Claws Mail 3.7.2 (GTK+ 2.18.6; x86_64-suse-linux-gnu)

....

--
Cheers,
Carlos E.R.

"Carlos E. R." <robin_listas@es.invalid> wrote:

> I just had a look in my Thunderbird, and what I see is both mail and
> news having "headername: headervalue".

RFC 822 (c.1982) specifies:

field = field-name ":" [ field-body ] CRLF

for header syntax, or headername:headervalue for easier reading. No
spaces either side of colon, but field-body could have leading and
trailing spaces. Even the examples in the RFC show a space after the
colon. I don't remember anyone used the RFC syntax with no space after
the colon. Even RFC 733 (c.1977) that RFC 822 obsoleted shows a space,
or more than 1, after the colon for its examples.

Seems "Do what I do, not what I say" happened from the start.

Can't think of where I could find an e-mail exhibit that is 46+ years
old to check header syntax way back then, or earlier. I'm not sure
anyone can find an example of a really old e-mail where there was no
space after the colon to strictly match the syntax intention of the RFC
header syntax. Looks like a case where the de facto standard didn't
match the published standard. Not a huge surprise, though. For many,
standards just get in the way, or take way too long to ratify what is
already de facto standards.

The earliest I can remember doing e-mail was around 1982 when I bought
my first IBM AT with a whopping 640MB RAM (on a long daughtercard with
sockets I had to populate from tube-fulls of RAM chips) with a huge 10MB
HDD and something like a monsterous 14" monochrome green monitor all
costing around $25K that I paid off monthly to Sperry Rand for a year.
I think it was either Compuserve or AOL via super-fast 300 and then 1200
baud dial-up to do e-mail and BBSes. Later 56K was blistering fast, or
119K if you paired them. Might've existed, but I don't remember the Web
was around yet. CERN didn't make the Web protocol and code available
royalty-free until 1993 which is about when I remember Mosaic showing
up. Lynx was a year earlier, but I wasn't yet doing anything Web.

Weren't dinosaurs roaming the Earth back then? I know they roam the
Earth now, because we old farts became the dinosaurs.

"Adam H. Kerman" <ahk@chinet.com> wrote:

> Right now, BASE64 encoding is one of the tests Ray Banana and others
> are using for NoCeMs as the Google-originating spam countermeasure.
> If any such articles get through to you, it's best to add an entry in
> your kill file.

For reasons I don't know, Ed Cryer sometimes posts using Base64, but I
suspect he doesn't intend to. I think Thunderbird is making the
decision for him, but could be due to how he configured that client.
Luckily Jeff sumbits to ES, not GG, so he's safe from Ray -- but not
from my filters that flag any posts using Base64.

winston uses UTF-8 chars instead of ASCII in his From header that forces
MIME encoding *and* uses Base64 encoding. No idea why "winston" in all
ASCII isn't good enough for him. He also uses Thunderbird. I do have a
filter for "^=\?\S+\?\S+\?\S+\?=$" regex on the Subject header's value
(looking for "=?charset?encoding?encodedtext?="), but haven't yet added
one looking for encoding in the From header. For now, I don't delete,
but just colorize to alert me of header encoding.

So, there are good guys that are using Base 64 and MIME encoding of
headers without good reason. My guess it's a fuckup in the config of
Thunderbird, or with the logic in that client.