I am wondering whether it would be feasible to read Usenet text groups
using the IMAP protocol. Not only email clients tend to be better
maintained than News clients nowadays, IMAP would also support
server-side read status, and thus avoid the problem of syncing newsrc
between different machines.

Cyrus seems to have the capability of doing so:
<http://web.mit.edu/ghudson/dev/nokrb/third/cyrus-imapd/doc/
install-netnews.html>

> Cyrus has the ability to export Usenet via IMAP and/or export shared
> IMAP mailboxes via NNTP. This is made possible by a new NNTP daemon
> which is included with Cyrus.

But are there any publicly available Usenet providers with that kind of
service set up?

On 5/19/21 9:25 AM, Iiro Laiho wrote:
> I am wondering whether it would be feasible to read Usenet text
> groups using the IMAP protocol.

I believe that it's definitely /possible/ to do so. Though I don't know
if it's /feasible/ to do so.

> Not only email clients tend to be better maintained than News clients
> nowadays, IMAP would also support server-side read status, and thus
> avoid the problem of syncing newsrc between different machines.

I think the server-side read status that you're thinking of is provided
by the IMAP server and independent of the news server.

> Cyrus seems to have the capability of doing so:
> <http://web.mit.edu/ghudson/dev/nokrb/third/cyrus-imapd/doc/
> install-netnews.html>

I've read about that a few times. Though I don't run Cyrus so I've not
had the opportunity to test it.

Courier IMAP doesn't have the same type of news support built in.
Though Courier IMAP does support shared (Maildir) mailboxes. -- I have
dabbled with getting this shared maildir functionality to work to access
my (transit) news server running on the same host. It was very much a
kludge, but it did work.

In short, I created a shared Maildir and used a sym-link to make the
Maildir's cur/ folder be a sym-link to a newsgroup's directory in INN's
trad(itional)spool directory. Then I had my user's Maildir connect to
the shared news Maildir using standard Courier commands.

The read / unread / deleted status is actually maintained per-user. So
different users can have messages in different read / unread / deleted
sates. This is done through the usual file name semantics where the
files are themselves sym-links. -- Courier IMAP has some special code
to gracefully handle the existence of broken sym-links that point to
missing files as an indicator that the real backing file has been
removed and to update (remove) the broken sym-links.

It's been a while since I've done this, but I do remember it working.
The only annoying things was that I was going to need to create a shared
Maildir per newsgroup. I'm sure that it could be scripted.

> But are there any publicly available Usenet providers with that kind
> of service set up?

I don't really know what publicly available Usenet providers offer as I
run my own Usenet servers.

--
Grant. . . .
unix || die

Grant Taylor <gtaylor@tnetconsulting.net> wrote:
> I think the server-side read status that you're thinking of is provided
> by the IMAP server and independent of the news server.

Yes, exactly. The read status stuff is handled by the IMAP server.

If some provider like eternal-september started a news-by-imap service,
it would be their burden to keep track of which user has read which
messages. I have no idea whether or not it would place a significant
extra load on the servers.

Iiro Laiho <news@iirolaiho.net> wrote:
> Grant Taylor <gtaylor@tnetconsulting.net> wrote:
> > I think the server-side read status that you're thinking of is provided
> > by the IMAP server and independent of the news server.
>
> Yes, exactly. The read status stuff is handled by the IMAP server.
>
> If some provider like eternal-september started a news-by-imap service,
> it would be their burden to keep track of which user has read which
> messages. I have no idea whether or not it would place a significant
> extra load on the servers.

It seems like you have to configure the server with a list of groups you
want to read. Otherwise your IMAP client would have to present every
newsgroup in your list of IMAP mailboxes, which would be cumbersome to use.

It might be feasible to run a local Cyrus instance that fetches articles
from elsewhere. Then it's equivalent to a local NNTP server like Leafnode.

Theo

On 5/19/21 2:40 PM, Theo wrote:
> It seems like you have to configure the server with a list of groups
> you want to read. Otherwise your IMAP client would have to present
> every newsgroup in your list of IMAP mailboxes, which would be
> cumbersome to use.

I can't remember any IMAP client that doesn't support this. This is
where IMAP folder subscription comes into play. Quite similar to
newsgroup subscription.

--
Grant. . . .
unix || die

Grant Taylor wrote:

> Theo wrote:
>
>> It seems like you have to configure the server with a list of groups
>> you want to read.  Otherwise your IMAP client would have to present
>> every newsgroup in your list of IMAP mailboxes, which would be
>> cumbersome to use.
>
> I can't remember any IMAP client that doesn't support this.  This is
> where IMAP folder subscription comes into play.  Quite similar to
> newsgroup subscription.

I remember back in 16bit days, various newsreaders barfed at servers
with more than 65536 groups, ok so the limit will be at least 2 billion
nowadays, but I bet it'd get really inefficient well before then, who
has tens of thousands of mail folders?

In comp.misc, Andy Burns <usenet@andyburns.uk> wrote:
> I remember back in 16bit days, various newsreaders barfed at servers
> with more than 65536 groups, ok so the limit will be at least 2 billion
> nowadays, but I bet it'd get really inefficient well before then, who
> has tens of thousands of mail folders?

Lots of email servers have tens of thousands of mail folders. Few people
are reading tens of thousands of newsgroups. I've got about 150
subscribed groups in my .newsrc, which is probably more typical.

The IMAP model can easily support this for a single person. One IMAP
server per newsreader would probably work.

Where it falls down is the parallelism. News servers share the same
messages to lots of people and do not store per message state per
reader. IMAP servers share different messages to lots of people and do
store per message state per reader.

If ten people are subcribed to ten groups in common, NNTP doesn't
have ten copies of each message in each group. IMAP does.

You could probably hack the IMAP server to cope: a single base message
and then store state about it separately. It sounds like a lot of work,
but definitely technically feasible.

Easier to just have an IMAP server per reader.

Elijah
------
is not interested in reading news by IMAP

For your reference, records indicate that
"Iiro Laiho" <news@iirolaiho.net> wrote:

> Cyrus seems to have the capability of doing so:
> <http://web.mit.edu/ghudson/dev/nokrb/third/cyrus-imapd/doc/
> install-netnews.html>

That documentation is nearly 20 years old. The most recent page I could
find is still 5 years old:
<https://www.cyrusimap.org/imap/reference/admin/nntp.html>

> But are there any publicly available Usenet providers with that kind of
> service set up?

None that I know of, and I wouldn’t expect any large NNTP provider to
maintain such a gateway. I do like the idea, though, and I’ve considered
turning my Usenet reader into that sort of thing (though maybe just
working with a user’s local mail spool). Otherwise, I could see dropping
Cyrus on a Raspberry Pi or small cloud server and giving it a try. I’d
be interested in hearing of other NNTP-to-IMAP solutions, too.

--
"Also . . . I can kill you with my brain."
River Tam, Trash, Firefly

Hi all,

>> I am wondering whether it would be feasible to read Usenet text
>> groups using the IMAP protocol. Not only email clients tend to be
>> better maintained than News clients nowadays, IMAP would also
>> support server-side read status, and thus avoid the problem of
>> syncing newsrc between different machines.[...]
> I could see dropping
> Cyrus on a Raspberry Pi or small cloud server and giving it a try. I’d
> be interested in hearing of other NNTP-to-IMAP solutions, too.

In case it could help to achieve that, for those running the news server
INN, there is a feeding program called "imapfeed" that can serve as a
NNTP-to-IMAP gateway.

I have never had the chance to test it, and does not know whether it
still works fine with current IMAP servers, but of course I would be
glad to hear successful use of it; and if it does not work, try to have
a look at what is going wrong.

https://www.eyrie.org/~eagle/software/inn/docs/innfeed.html

"As an alternative to using NNTP, INN may also be fed to an IMAP server.
This is done by using an executable called imapfeed, which is identical
to innfeed except for the delivery process. The new version has two
types of connections: an LMTP connection to deliver regular messages and
an IMAP connection to handle control messages."

And from INSTALL:

"The feed entry for this is as follows:

imapfeed!\
:!*\
:Tc,Wnm*,S16384:<pathbin in inn.conf>/imapfeed

And set up entries for each remote site like:

remote.example.com/news.example.com\
:<newsgroups>\
:Tm:imapfeed!
"

--
Julien ÉLIE

« Je connais des sourds qui s'entendent très bien ! » (Philippe Geluck)

Le 30/05/2021 à 10:29, Julien ÉLIE a écrit :

> "As an alternative to using NNTP, INN may also be fed to an IMAP server.

is it the same as the mailman gateway? I used it some years ago

jdd
>

--
http://dodin.org

Hi jdd,

>> "As an alternative to using NNTP, INN may also be fed to an IMAP server.
>
> is it the same as the mailman gateway? I used it some years ago

I've not run a mailman gateway either, but I do not think it uses IMAP.

Just a standard news2mail call in INN to send articles to a mailing-list
address run by mailman
https://www.eyrie.org/~eagle/software/inn/docs/news2mail.html

or let mailman regularly connect to your news server to retrieve new
articles (gate_news):
https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/commands/docs/gatenews.html

Gérald will certainly tell us more as for how he once set up grisbi.*
gatewaying :-)

--
Julien ÉLIE

« Un dictionnaire est comme une montre ; il vaut mieux en avoir un que
pas du tout, mais le meilleur n'est jamais tout à fait exact. »
(Georges-Bernard Shaw)

Le 30/05/2021 à 11:26, Julien ÉLIE a écrit :
> Hi jdd,
>
>>> "As an alternative to using NNTP, INN may also be fed to an IMAP server.
>>
>> is it the same as the mailman gateway? I used it some years ago
>
> I've not run a mailman gateway either, but I do not think it uses IMAP.
>
> Just a standard news2mail call in INN to send articles to a mailing-list
> address run by mailman
> https://www.eyrie.org/~eagle/software/inn/docs/news2mail.html
>
> or let mailman regularly connect to your news server to retrieve new
> articles (gate_news):
>
> https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/commands/docs/gatenews.html
>
> Gérald will certainly tell us more as for how he once set up grisbi.*
> gatewaying :-)
>
yes, but mailma allow imap as any mail, but probably overkill :-(

jdd

--
http://dodin.org

On 5/30/21 2:43 AM, jdd wrote:
> is it the same as the mailman gateway? I used it some years ago

I know that news (NNTP) support that Mailman (as in the mailing list
manager) is specifically a single newsgroup.

I would hope that imapfeed can support multiple newsgroups as different
IMAP folders. But I don't know.

There's also the fact that it would remove Mailman from the mix.

--
Grant. . . .
unix || die

In news.software.nntp Grant Taylor <gtaylor@tnetconsulting.net> wrote:

> I would hope that imapfeed can support multiple newsgroups as different
> IMAP folders. But I don't know.

I was going to chime in earlier with this but decided I forgot more than I
remember but we (ripco.com) played around with this years ago (15~20?) and
decided it wasn't worth pursuing (news to imap).

I'm pretty sure after all the testing the conclusion was if you had 1 user
with 1 email account who wanted 1 news group fed into their mail, it was
fine. But rapidly fell apart for anything else.

If you wanted them to post articles, you would need some kind of mail2news
program (common in the early 2000's) and unless they cleaned up themselves
by deleting articles in the mail, there is no "expire".

I mean back then was a different world with 5-6 articles every second coming
in a full big-8 (used to be 7) plus alt verses today's one article every 5
or 6 seconds. Again disks were smaller, slower cpu's and all that factoring
in but it would of still ended up being quite messy.

I'm also sure you could of done multiple groups but it would require
something on the mail end to create individual folders via a filter to chop
things up. Even the basic webmail programs like Squirrelmail can do that but
if there are several hundred articles to process, since the filters only
come into play when the user logs in, it can create quite a delay when first
opened.

This is one of those things that look better on paper than actually putting
into practice. Too many drawbacks was the final conclusion from what I
remember. Seemed to be more of a novelty item to impress your freinds with
rather than being practical.

-bruce
bje@ripco.com

On 5/19/2021 10:25 AM, Iiro Laiho wrote:
> I am wondering whether it would be feasible to read Usenet text groups
> using the IMAP protocol. Not only email clients tend to be better
> maintained than News clients nowadays, IMAP would also support
> server-side read status, and thus avoid the problem of syncing newsrc
> between different machines.
>
> Cyrus seems to have the capability of doing so:
> <http://web.mit.edu/ghudson/dev/nokrb/third/cyrus-imapd/doc/
> install-netnews.html>
>
>> Cyrus has the ability to export Usenet via IMAP and/or export shared
>> IMAP mailboxes via NNTP. This is made possible by a new NNTP daemon
>> which is included with Cyrus.
>
> But are there any publicly available Usenet providers with that kind of
> service set up?
>
It's not exactly the same but thunderbird has usenet and it looks the
same as email.

On 6/28/21 12:58 PM, IckyPerkins wrote:
> It's not exactly the same but thunderbird has usenet and it looks the
> same as email.

I believe the OP was wanting to be able to leverage IMAP's ability to
have the read / unread status synchronized between multiple systems, but
for news.

Thunderbird's news read / unread status is very much per machine.

--
Grant. . . .
unix || die

On Wed, 19 May 2021 15:25:06 -0000 (UTC)
"Iiro Laiho" <news@iirolaiho.net> wrote:

> I am wondering whether it would be feasible to read Usenet text groups
> using the IMAP protocol.

I've had that thought as well but I reckoned it would be better just
for me to improve NNTP client support and share whatever I write with
the world. I'm mostly focused on CLI-only though (so my care for TUI
readers is out the window) and although slrnpull exists, I haven't
tried it yet and I don't know how well it will work with other
commandline tools.

> Not only email clients tend to be better
> maintained than News clients nowadays,

I currently use Claws Mail which has good NNTP support as well as
functioning primarily as a MUA. There's also Thunderbird which I
believe still supports NNTP, but I have no idea how well it works, nor
do I like Thunderbird for other reasons -- resource usage being my
primary complaint, but also the insistence of shoving a Web engine into
a mail client turns me off.

> IMAP would also support
> server-side read status, and thus avoid the problem of syncing newsrc
> between different machines.

This functionality would be nice to have, definitely. NNTP isn't very
fleshed out for multiple-client or mobile-device support.