Rocksolid Light

Welcome to RetroBBS

mail  files  register  newsreader  groups  login

Message-ID:  

< jaybonci> actually d-i stands for "divine intervention" ;) -- in #debian-devel


computers / news.software.nntp / Re: tradpoool v cnfs

SubjectAuthor
* tradpoool v cnfsNigel Reed
+- Re: tradpoool v cnfsnoel
+- Re: tradpoool v cnfsJulien ÉLIE
`- Re: tradpoool v cnfsIvo Gandolfo

1
tradpoool v cnfs

<20240127222552.1e214c1d@wibble.sysadmininc.com>

  copy mid

https://www.rocksolidbbs.com/computers/article-flat.php?id=2866&group=news.software.nntp#2866

  copy link   Newsgroups: news.software.nntp
Path: i2pn2.org!i2pn.org!usenet.goja.nl.eu.org!newsfeed.endofthelinebbs.com!.POSTED.47.186.47.220!not-for-mail
From: sysop@endofthelinebbs.com (Nigel Reed)
Newsgroups: news.software.nntp
Subject: tradpoool v cnfs
Date: Sat, 27 Jan 2024 22:25:52 -0600
Organization: End Of The Line BBS
Message-ID: <20240127222552.1e214c1d@wibble.sysadmininc.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Injection-Info: www.sysadmininc.com; posting-host="47.186.47.220";
logging-data="1932908"; mail-complaints-to="abuse@endofthelinebbs.com"
X-Newsreader: Claws Mail 4.2.0git6 (GTK 3.24.33; x86_64-pc-linux-gnu)
 by: Nigel Reed - Sun, 28 Jan 2024 04:25 UTC

The time has come for me to make a decision keep tradspool or move to
cnfs. I know there's a lot of pros and cons with both.

tradspool creates an issues when a newsgroup name contains number a
numeric component that clashes with an article number. This came
to light and I posted about it the other day.

I'm using zfs so there are no inode issues.

I also like the fact I can directly look at articles, grep for words
and things like that.

It's easy to see exactly how much disk space I have left and there's no
issues with deleting articles.

My system never expires articles except for test groups which are
expired after 7 days.

With cnfs there's no need to worry about those newsgroup name issues,
but there's no easy direct method of viewing the articles.
I also fear that I will lose articles if, for some how, the buffers get
full and start to overwrite without me noticing it.

Also, will space from expired or canceled articles be reused or does
that usually get reclaimed when the buffer recycles? I could end up
losing a lot of space.

Also, if using cnfs, would would be the best way to create the buffers?
I have a 1.9TB block of space that I can keep as a single partition or
divvy up into multiple.

After advise from those more in the knowledge of cnfs than me.

Thanks,
Nigel

--
End Of The Line BBS - Plano, TX
telnet endofthelinebbs.com 23

Re: tradpoool v cnfs

<65b5fb63$1@news.ausics.net>

  copy mid

https://www.rocksolidbbs.com/computers/article-flat.php?id=2867&group=news.software.nntp#2867

  copy link   Newsgroups: news.software.nntp
From: deletethis@invalid.lan (noel)
Subject: Re: tradpoool v cnfs
Newsgroups: news.software.nntp
References: <20240127222552.1e214c1d@wibble.sysadmininc.com>
X-No-Archive: Yes
User-Agent: Pan/0.141 (Tarzan's Death; 168b179 git.gnome.org/pan2)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
NNTP-Posting-Host: news.ausics.net
Message-ID: <65b5fb63$1@news.ausics.net>
Date: 28 Jan 2024 16:59:47 +1000
Organization: Ausics - https://newsgroups.ausics.net
Lines: 18
X-Complaints: abuse@ausics.net
Path: i2pn2.org!i2pn.org!news.nntp4.net!news.bbs.nz!news.ausics.net!not-for-mail
 by: noel - Sun, 28 Jan 2024 06:59 UTC

On Sat, 27 Jan 2024 22:25:52 -0600, Nigel Reed wrote:

>
> I also fear that I will lose articles if, for some how, the buffers get
> full and start to overwrite without me noticing it.

This !

apart from taking months to import news from DNews into INN, the other
key concern stopping us is this one, I'm told CNFS buffers is about as
close to DNews spool files (IIRC - has been a while since I discussed
that with Julien) out of all the choices, and if we don't create enough,
the over writing will be a problem, I think CNFS needs an option to
create more buffer files automatically on-the-fly when existing buffer
count hits a pre determined or user configurable, value of the total
available space.

Re: tradpoool v cnfs

<up53a2$kd4r$1@news.trigofacile.com>

  copy mid

https://www.rocksolidbbs.com/computers/article-flat.php?id=2868&group=news.software.nntp#2868

  copy link   Newsgroups: news.software.nntp
Path: i2pn2.org!i2pn.org!usenet.goja.nl.eu.org!weretis.net!feeder8.news.weretis.net!news.trigofacile.com!.POSTED.2a01cb080adc1100e52ad4afc5481596.ipv6.abo.wanadoo.fr!not-for-mail
From: iulius@nom-de-mon-site.com.invalid (Julien ÉLIE)
Newsgroups: news.software.nntp
Subject: Re: tradpoool v cnfs
Date: Sun, 28 Jan 2024 09:29:54 +0100
Organization: Groupes francophones par TrigoFACILE
Message-ID: <up53a2$kd4r$1@news.trigofacile.com>
References: <20240127222552.1e214c1d@wibble.sysadmininc.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Sun, 28 Jan 2024 08:29:54 -0000 (UTC)
Injection-Info: news.trigofacile.com; posting-account="julien"; posting-host="2a01cb080adc1100e52ad4afc5481596.ipv6.abo.wanadoo.fr:2a01:cb08:adc:1100:e52a:d4af:c548:1596";
logging-data="668827"; mail-complaints-to="abuse@trigofacile.com"
User-Agent: Mozilla Thunderbird
Cancel-Lock: sha1:UPdcx1PB+0PI9H6m2bZZhLSFSUo= sha256:Lr3svRuf32JQGzLGYdTZ2wQj+OajRmUtRhiGpWlLeT0=
sha1:3jaa6JGlPSoP+Alk0lAP1myVxc8= sha256:dlcRpLpol3uM8dMp2PHOO5K3YfxVuKCvsZMFmH06C9o=
In-Reply-To: <20240127222552.1e214c1d@wibble.sysadmininc.com>
 by: Julien ÉLIE - Sun, 28 Jan 2024 08:29 UTC

Hi Nigel,

> tradspool creates an issues when a newsgroup name contains number a
> numeric component that clashes with an article number. This came
> to light and I posted about it the other day.

The issue appears only once per newsgroup with a numeric component and
for which its direct parent exists as a newsgroup.
That is to say if you have a newsgroup named a.b.3, and no newsgroup
named a.b, there won't be any problem. It is only when you have both
a.b and a.b.3 as newsgroups.

It would be worthwhile checking how many newsgroups like that you have.
It may not be a big deal.

You'll then only lose an article per newsgroup when deleting it (in the
example, article number 3 for a.b cannot be kept).

Compared to the other pros/cons of CNFS, it's up to you to decide what
you prefer.

--
Julien ÉLIE

« Le cercle est le plus long chemin d'un point au même point. » (Tom
Stoppard, _Every Good Boy Deserves Favour_)

Re: tradpoool v cnfs

<up57j1$31e32$1@paganini.bofh.team>

  copy mid

https://www.rocksolidbbs.com/computers/article-flat.php?id=2870&group=news.software.nntp#2870

  copy link   Newsgroups: news.software.nntp
Path: i2pn2.org!i2pn.org!news.nntp4.net!paganini.bofh.team!not-for-mail
From: usenet@bofh.team (Ivo Gandolfo)
Newsgroups: news.software.nntp
Subject: Re: tradpoool v cnfs
Date: Sun, 28 Jan 2024 10:42:57 +0100
Organization: To protect and to server
Message-ID: <up57j1$31e32$1@paganini.bofh.team>
References: <20240127222552.1e214c1d@wibble.sysadmininc.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Date: Sun, 28 Jan 2024 09:42:57 -0000 (UTC)
Injection-Info: paganini.bofh.team; logging-data="3192930"; posting-host="UVa2lCJi2YFEtTlJ34XukQ.user.paganini.bofh.team"; mail-complaints-to="usenet@bofh.team"; posting-account="NnMoDEm3qkIJGKBtBYRUeg";
User-Agent: Mozilla Thunderbird
Cancel-Lock: sha256:n5c4mXLqTzIMEmBhPtR8UXr9+oPyb2aDloT62ArUDXs=
Content-Language: en-US
X-Notice: Filtered by postfilter v. 0.9.3
 by: Ivo Gandolfo - Sun, 28 Jan 2024 09:42 UTC

On 28/01/2024 05:25, Nigel Reed wrote:
>
> With cnfs there's no need to worry about those newsgroup name issues,
> but there's no easy direct method of viewing the articles.

You need to know the token, but it's easy with m-id. You can grab the
tokenname with grephistory, and then read the article with sm. It's just
a step + 1 to read, and not pointing directly the article into the
filesystem.

> I also fear that I will lose articles if, for some how, the buffers get
> full and start to overwrite without me noticing it.
>

Actually I'm setup 2 new newsserver, and grabbing news from various
source with cnfs and buffindexer without big trouble. The only defect
it's the harddisk space it's areally taken.

> Also, will space from expired or canceled articles be reused or does
> that usually get reclaimed when the buffer recycles? I could end up
> losing a lot of space.
>
> Also, if using cnfs, would would be the best way to create the buffers?
> I have a 1.9TB block of space that I can keep as a single partition or
> divvy up into multiple.
>
> After advise from those more in the knowledge of cnfs than me
>
> Thanks,
> Nigel
>

See INSTALL file from inn2 tarball, section "Creating the Article Spool
(CNFS only)" or mkbuf.in file in CONTRIB section. In my case I have do a
little rewrite them for daily use, and it's the "easy way" to create a
new one cyc buff and adding them to storage.conf file. I use a file into
directory /var/spool/news/cycbuff (same filesystem ad /, but you can do
a new partition for this with different filesystem).

For warning when a cyc buff it's "full" I have created a script, added
in crontab, running cnfsstat -a, when the value of cycles reach *.80
send me an email and my system create in auto a new cyc buffer with
predefined space, add it into storage.conf (using the pattern in Class
readed with cnfsstat), and reload inn2. But in my case I have 24TB of
diskspace, and create a new cyc it's not hard for me.
In your case with only 2TB of space I recommend you create some small
cycs for those groups (or hierarchies) that are not important to you and
which can also expire quickly and suppose 50/500MB it's enought for
control.* or *.test, and some "large" cycs for the groups (or
hierarchies) that instead you want to maintain, in that case 1/10GB. If
you set as sequential, not big trouble, and easy to reuse (or backup)
and you can set a good retention for everything's.

The stats of my newsserver (text only) it's a retention from '90/2000
take 4/5TB of space. A lot of group have millions of post (one I have
see have 20 millions, impossible to grep all with normal newsreader).

Sincerely

--
Ivo Gandolfo

1
server_pubkey.txt

rocksolid light 0.9.8
clearnet tor