I’m taking on partial binary feeds and am in the process of migrating to
Diablo, but am having an issue with the IFILTER/InternalFilter. I had used
Diablo with a similar setup many years ago, and thought this filter worked
well back then, but am finding it is detecting many binary articles posted in
binary groups and rejecting them.

Looking at the articles it is rejecting, I cannot determine why. Diablo isn’t
rejecting all binary articles, but a fair amount. The log lines look like:

2023-02-01 22:01:42.411 chomp-out -
<IdZuKgIdGdSnNsJzQaCpWqIt-1675310467589@nyuu> 42789 050200 IncomingFilter

When I look at the headers and beginning of the body, I don’t see any reason
why this wouldn’t pass through the InternalFilter:

> Message-Id: <IdZuKgIdGdSnNsJzQaCpWqIt-1675310467589@nyuu>
> Subject: [022/290] - "15de68c2ff06204845b63c707ea123e7.part03.rar.par2" yEnc
> (1/1) 40452
> From: 55c6e45471b4aef7@.com
> Newsgroups: alt.binaries.games.encrypted, alt.binaries.boneless
> Date: Thu, 02 Feb 2023 04:01:07 GMT
> Path:
> chomp-out!feed1.usenet.blueworldhosting.com!npeer.as286.net!npeer-ng0.as286.n
> et!proxad.net!feeder1-1.proxad.net!94.232.112.246.MISMATCH!feed.abavia.com!ab
> e006.abavia.com!abe003.abavia.com!feeder1.feed.usenet.farm!feed.usenet.farm!t
> r1.eu1.usenetexpress.com!feeder.usenetexpress.com!tr1.iad1.usenetexpress.com!
> news.newsgroupdirect.com!not-for-mail
> User-Agent: Nyuu/0.4.0
> Lines: 324
> Nntp-Posting-Date: Thu, 02 Feb 2023 04:01:07 +0000
> X-Received-Bytes: 42305
> Organization: NewsgroupDirect
> X-Complaints-To: abuse@newsgroupdirect.com
> Xref: feed1.usenet.blueworldhosting.com junk:25332712
>
> =ybegin part=1 total=1 line=128 size=40452
> name=15de68c2ff06204845b63c707ea123e7.part03.rar.par2
> =ypart begin=1 end=40452

The article matches the group filter *.bina* and the arttype should be
detected as binary, but seems not to be. Any ideas?

Regards,

Jesse Rehmer

On Feb 1, 2023 at 10:43:12 PM CST, "Jesse Rehmer"
<jesse.rehmer@blueworldhosting.com> wrote:

> I’m taking on partial binary feeds and am in the process of migrating to
> Diablo, but am having an issue with the IFILTER/InternalFilter. I had used
> Diablo with a similar setup many years ago, and thought this filter worked
> well back then, but am finding it is detecting many binary articles posted in
> binary groups and rejecting them.
>
> Looking at the articles it is rejecting, I cannot determine why. Diablo isn’t
> rejecting all binary articles, but a fair amount. The log lines look like:
>
> 2023-02-01 22:01:42.411 chomp-out -
> <IdZuKgIdGdSnNsJzQaCpWqIt-1675310467589@nyuu> 42789 050200 IncomingFilter
>
> When I look at the headers and beginning of the body, I don’t see any reason
> why this wouldn’t pass through the InternalFilter:
>
>> Message-Id: <IdZuKgIdGdSnNsJzQaCpWqIt-1675310467589@nyuu>
>> Subject: [022/290] - "15de68c2ff06204845b63c707ea123e7.part03.rar.par2" yEnc
>> (1/1) 40452
>> From: 55c6e45471b4aef7@.com
>> Newsgroups: alt.binaries.games.encrypted, alt.binaries.boneless
>> Date: Thu, 02 Feb 2023 04:01:07 GMT
>> Path:
>> chomp-out!feed1.usenet.blueworldhosting.com!npeer.as286.net!npeer-ng0.as286.n
>> et!proxad.net!feeder1-1.proxad.net!94.232.112.246.MISMATCH!feed.abavia.com!ab
>> e006.abavia.com!abe003.abavia.com!feeder1.feed.usenet.farm!feed.usenet.farm!t
>> r1.eu1.usenetexpress.com!feeder.usenetexpress.com!tr1.iad1.usenetexpress.com!
>> news.newsgroupdirect.com!not-for-mail
>> User-Agent: Nyuu/0.4.0
>> Lines: 324
>> Nntp-Posting-Date: Thu, 02 Feb 2023 04:01:07 +0000
>> X-Received-Bytes: 42305
>> Organization: NewsgroupDirect
>> X-Complaints-To: abuse@newsgroupdirect.com
>> Xref: feed1.usenet.blueworldhosting.com junk:25332712
>>
>> =ybegin part=1 total=1 line=128 size=40452
>> name=15de68c2ff06204845b63c707ea123e7.part03.rar.par2
>> =ypart begin=1 end=40452
>
> The article matches the group filter *.bina* and the arttype should be
> detected as binary, but seems not to be. Any ideas?
>
> Regards,
>
> Jesse Rehmer

I think I figured out why, but looking at the code am not sure how to fix it.
After looking at a larger sampling of articles, the one characteristic that
stands out is the filtered articles contain a Newsgroup: header with multiple
newsgroups, but contains a space after the comma separating the group names.

(Corrected my subject line, after staring at the code, I think I'm officially
toast for the night.)