Hi all,

I recently got my Usenet server up and running again the way that I like.
So far its been up for about 30 hours and I wanted to share some
statistics.

In the past 30 hours:

I have downloaded 112MB of new articles.
Total number of articles: 27041
The alt hierarchy is the largest with 7487 articles.

The top 15 newsgroups (that I carry) are:

2997 free.usenet
813 alt.atheism
742 talk.politics.guns
651 fr.soc.politique
597 rec.arts.tv
505 alt.test
502 rec.arts.drwho
419 rec.food.cooking
411 free.pt
402 alt.fan.rush-limbaugh
401 uk.legal
395 soc.culture.jewish
385 soc.culture.usa
385 alt.arts.poetry.comments
382 alt.checkmate

__
JE

Jason Evans <jsevans@mailfence.com> wrote:

>I recently got my Usenet server up and running again the way that I like.
>So far its been up for about 30 hours and I wanted to share some
>statistics.

>In the past 30 hours:

>I have downloaded 112MB of new articles.
>Total number of articles: 27041
>The alt hierarchy is the largest with 7487 articles.

Are you actively implementing spam countermeasures and telling us
the remaining article counts? Are you avoiding the double counting of
crossposts? Several of the higher volume newsgroups mentioned, sigh, there
isn't necessarily discussion but there are users who simply plagarize
articles found on the Web, reposting them to Usenet, crossposting among
a number of other so-called higher-volume newsgroups.

I don't count any of that as discussion but I have no idea how to
automatically identify articles plagarized from the Web to not count
them as Usenet traffic, to get some idea of how much discussion truly
takes place.

I've said before and I'll say again that reposting from the Web isn't
desireable traffic and while it increases someone's poast count, it
doesn't keep discussion going. That happens only when people write their
own articles for Usenet and not reposting someone else's words.

>The top 15 newsgroups (that I carry) are:

>2997 free.usenet
>813 alt.atheism
>742 talk.politics.guns
>651 fr.soc.politique
>597 rec.arts.tv
>505 alt.test
>502 rec.arts.drwho

Kill filing Yads and Tim and all the troll feeding of the two of them
leaves nearly no actual on-topic discussion. Another one of those
quality of traffic issues.

>419 rec.food.cooking

Finally! You are talking about a newsgroup with active on-topic
discussion.

>411 free.pt
>402 alt.fan.rush-limbaugh
>401 uk.legal
>395 soc.culture.jewish
>385 soc.culture.usa
>385 alt.arts.poetry.comments
>382 alt.checkmate

On Mon, 12 Sep 2022 18:48:34 -0000 (UTC), Adam H. Kerman wrote:

> Are you actively implementing spam countermeasures and telling us the
> remaining article counts?
Hi Adam,

No, I am not implementing spam countermeasures for 2 reasons. First of
all, I have a userbase of 1 user, me, and therefore have no need to
sensor my users. Secondly, I make my own Usenet archives. In a couple of
months, I will start making my own monthly archives and submitting them
to archive.org. The statistics that I am provided are raw data.

> Are you avoiding the double counting of
> crossposts? Several of the higher volume newsgroups mentioned, sigh,
> there isn't necessarily discussion but there are users who simply
> plagarize articles found on the Web, reposting them to Usenet,
> crossposting among a number of other so-called higher-volume newsgroups.

No, I'm not doing anything quite so complex. I am only counting raw the
actual number of articles that I receive. The reports that I run are from
fairly rudimentary BASH scripts. The scripts can be found here:
https://github.com/tgeek77/UsenetReport

>
> I don't count any of that as discussion but I have no idea how to
> automatically identify articles plagarized from the Web to not count
> them as Usenet traffic, to get some idea of how much discussion truly
> takes place.
>
> I've said before and I'll say again that reposting from the Web isn't
> desireable traffic and while it increases someone's poast count, it
> doesn't keep discussion going. That happens only when people write their
> own articles for Usenet and not reposting someone else's words.

I don't judge what people want to post. Though I also get annoyed that
some people ONLY post links with no discussion like what you'll see on
humanities.classics.

>>419 rec.food.cooking
>
> Finally! You are talking about a newsgroup with active on-topic
> discussion.
>

I too like rec.food.cooking. It's a very positive on-topic group. There
are are few trolls, but they are easily filtered.

__
JE

On Tue, 13 Sep 2022 06:24:54 -0000 (UTC), Jason Evans wrote:

> The scripts can be found here:
> https://github.com/tgeek77/UsenetReport

Sorry, wrong link. Here are the scripts:
https://github.com/tgeek77/Usenet_Scripts

Jason Evans <jsevans@mailfence.com> wrote:
>On Tue, 13 Sep 2022 06:24:54 -0000 (UTC), Jason Evans wrote:

>>The scripts can be found here:
>>https://github.com/tgeek77/UsenetReport

>Sorry, wrong link. Here are the scripts:
>https://github.com/tgeek77/Usenet_Scripts

Jason, I know too little about this to comment.

However, if an article is crossposted, on the News server, isn't the
article stored just once and referenced by other newsgroups? Doesn't
that make it possible to not count the crosspost redundantly? That won't
identify non-discussion articles (plagarized Web pages) and FAQ repostings
to the extent any cron jobs remain?

I suppose if your peers actively maintain spam countermeasures, there's
probably too little additional benefit to you to maintain your own, so
your statistics would be net of cancelable spam.

In news.groups Jason Evans <jsevans@mailfence.com> wrote:

> I have downloaded 112MB of new articles.
> Total number of articles: 27041
> The alt hierarchy is the largest with 7487 articles.

I think you are misguided.

Generally posts to usenet in actual discussion groups is around 10K a day.

Volume is around 40mb a day.

> 2997 free.usenet
> 411 free.pt

These are binary groups, not discussion groups.

Most of the free.* are like that, little discussion (or none) and the active
ones get around servers that don't carry alt.binaries.* and that don't use
limits on post sizes they accept.

https://nntp.ripco.com

Unless you want to count binaries as discussions, put in cleanfeed, limit
your articles to 30k or so in size and you'll see a major difference.

You are doing something wrong.

-bruce
bje@ripco.com

On Tue, 13 Sep 2022 15:57:58 -0000 (UTC), Adam H. Kerman wrote:

> However, if an article is crossposted, on the News server, isn't the
> article stored just once and referenced by other newsgroups? Doesn't
> that make it possible to not count the crosspost redundantly? That won't
> identify non-discussion articles (plagarized Web pages) and FAQ
> repostings to the extent any cron jobs remain?

That's not how INN works. You get one text file per article per newsgroup.
Users who crosspost to 5 newsgroups generate 5 news articles across the
network and those all appear seperately. I believe it's always been that
way even during the UUCP days.

Here are the different storage options for INN:
https://www.eyrie.org/~eagle/software/inn/docs-2.7/install.html#S6.
I use tradspool because it is simplest for troubleshooting. Nowadays, I
can get a VPS server for < $20 with 1TB of SSD storage and lots of
memory and CPU cores so I don't worry about the bottlenecks
hat admins had in the past with expensive and limited storage and
memory.

I can't speak for paid services that are running custom software that
focus on serving binary groups. Maybe they have optimized the process.

If you have the funds to get a VPS server and the Linux know-how, you
ought to set up your own server. It gives you a new appreciation for
what it means to use the Usenet.

Jason

Jason Evans <jsevans@mailfence.com> wrote:
>On Tue, 13 Sep 2022 15:57:58 -0000 (UTC), Adam H. Kerman wrote:

>>However, if an article is crossposted, on the News server, isn't the
>>article stored just once and referenced by other newsgroups? Doesn't
>>that make it possible to not count the crosspost redundantly? That won't
>>identify non-discussion articles (plagarized Web pages) and FAQ
>>repostings to the extent any cron jobs remain?

>That's not how INN works. . . .

My error; never mind

Jason Evans <jsevans@mailfence.com> wrote:
> On Tue, 13 Sep 2022 15:57:58 -0000 (UTC), Adam H. Kerman wrote:
>
> > However, if an article is crossposted, on the News server, isn't the
> > article stored just once and referenced by other newsgroups? Doesn't
> > that make it possible to not count the crosspost redundantly? That won't
> > identify non-discussion articles (plagarized Web pages) and FAQ
> > repostings to the extent any cron jobs remain?
>
> That's not how INN works. You get one text file per article per newsgroup.
> Users who crosspost to 5 newsgroups generate 5 news articles across the
> network and those all appear seperately. I believe it's always been that
> way even during the UUCP days.

Hmmm!? For me it's a long time ago, but I'm quite sure that there
is only one file per article and 5 links (with the names equal to
article numbers in each of the groups). At least that's what I remember
for tradspool (IIRC at the time there was only tradspool).

> Here are the different storage options for INN:
> https://www.eyrie.org/~eagle/software/inn/docs-2.7/install.html#S6.
> I use tradspool because it is simplest for troubleshooting.

Just have a look at those 'multiple files', I think you will find that
they're actually links (hard? soft?) to a single physical file.

In any case, "generate 5 news articles across the network and those
all appear seperately" is incorrect. Articles are sent or received by
message-id and the crossposted article has only one message-id. The
sender/receiver knows not to send/ask_for the same message-id more than
once.

Anyway, the *current* *real* newsadmins are over there in
news.software.nntp! :-)

[...]

On 13 Sep 2022 18:34:31 GMT, Frank Slootweg wrote:

> Just have a look at those 'multiple files', I think you will find that
> they're actually links (hard? soft?) to a single physical file.

I stand corrected. I checked for myself and they are hard links. I will look into updating my scripts to only show
the original files and not the linked file.

JE

Jason Evans <jsevans@mailfence.com> wrote:
>On 13 Sep 2022 18:34:31 GMT, Frank Slootweg wrote:

>> Just have a look at those 'multiple files', I think you will find that
>> they're actually links (hard? soft?) to a single physical file.

>I stand corrected. I checked for myself and they are hard links. I will
>look into updating my scripts to only show
>the original files and not the linked file.

Excellent!

In alt.fan.usenet Jason Evans <jsevans@mailfence.com> wrote:
> On 13 Sep 2022 18:34:31 GMT, Frank Slootweg wrote:
>
> > Just have a look at those 'multiple files', I think you will find that
> > they're actually links (hard? soft?) to a single physical file.
>
> I stand corrected. I checked for myself and they are hard links. I
> will look into updating my scripts to only show the original files and
> not the linked file.

Thanks for the feedback/confirmation. It's comforting to hear that
I apparently can still remember these things after two+ decades! :-)