Recently, some newsgroups that I follow are being flooded by
bot-generated spam The subjects are in a foreign language
(Indonesia?), and are fairly consistent, so one would guess that the
Bayesian filter in Agent would be effective. But when I try to move
examples of the spam to Junk, nothing happens. I'm running Agent
8.00/32.1272.

Here are some subject lines:

BANDAR TOGEL > Situs BANDAR TOGEL Mudah Menang
?TRIADMACAU? : SITUS GAME ONLINE TERPERCAYA DI INDONESIA ??
MIKI Gaming > Situs MIKI Gaming Mudah Menang
SBO SPORT > Situs SBO SPORT Mudah Menan
?MAMEN123, ?DAFTARMAMEN123, ?SITUS MAMEN123, ?LINK MAMEN123?LOGIN
MAMEN123
SLOT THAILAND ?? Rekomendasi Situs Slot Server Luar Negeri Paling
Gacor
Essebet ?? Situs Slot Dana Paling Gacor Mudah Menang ?
Mainaku ?? Situs Slot Dana Paling Gacor Mudah Menang ?
?HARTA138, ?DAFTARHARTA138, ?SITUS HARTA138, ?LINK HARTA138?LOGIN
HARTA138
Stationplay ?? Situs Slot Dana Paling Gacor Mudah Menang ?
Agen138 Slot: Why It's a Top Choice in Indonesia's Gambling Scene
Mitosplay ?? Situs Slot Dana Paling Gacor Mudah Menang
Dadu13 - Situs Slot Dana Paling Gacor Mudah Menan

Any ideas?

Joe Gwinn

On Wed, 29 Nov 2023 12:01:14 +1300 (GMT+13:00), Ralph Fox
<-rf-nz-@-.invalid> wrote:

>Joe Gwinn <joegwinn@comcast.net> wrote:
>
>> Recently, some newsgroups that I follow are being flooded by
>> bot-generated spam  The subjects are in a foreign language
>> (Indonesia?), and are fairly consistent, so one would guess that the
>> Bayesian filter in Agent would be effective.  But when I try to move
>> examples of the spam to Junk, nothing happens.  I'm running Agent
>> 8.00/32.1272.
>...
>> Any ideas?Joe Gwinn
>
>The Bayesian filter in Agent works only with email, not with
> newsgroup posts.

Well, that's too bad. Where is that documented? The only clue I had
was that the command was grayed out; no reason given.

Thanks,

Joe

On 11/28/2023 6:13 PM, Joe Gwinn wrote:
> On Wed, 29 Nov 2023 12:01:14 +1300 (GMT+13:00), Ralph Fox
> <-rf-nz-@-.invalid> wrote:
>
>> Joe Gwinn <joegwinn@comcast.net> wrote:
>>
>>> Recently, some newsgroups that I follow are being flooded by
>>> bot-generated spam  The subjects are in a foreign language
>>> (Indonesia?), and are fairly consistent, so one would guess that the
>>> Bayesian filter in Agent would be effective.  But when I try to move
>>> examples of the spam to Junk, nothing happens.  I'm running Agent
>>> 8.00/32.1272.
>> ...
>>> Any ideas?Joe Gwinn
>>
>> The Bayesian filter in Agent works only with email, not with
>> newsgroup posts.
>
> Well, that's too bad. Where is that documented? The only clue I had
> was that the command was grayed out; no reason given.
>
> Thanks,
>
> Joe

Joe.

Sadly!! I have reverted to Tools/Filters and then either used a
character in the subject line or the senders name and email address.

And then used the "Delete" function. Also, in the lower right "Run Now"

It seems to clear out a lot of CRAP!! I notice that when I click on
a newsgroup there is a number that flashes. Some in excess of 100 and
then they are gone gone gone.

Hope this helps!!

Les

On Tue, 28 Nov 2023 17:12:55 -0500, Joe Gwinn wrote:

> Recently, some newsgroups that I follow are being flooded by
> bot-generated spam The subjects are in a foreign language
> (Indonesia?), and are fairly consistent
....
> Any ideas?
>
> Joe Gwinn

A normal Usenet filter can kill much of it.

Some options

1. That spam is posted through Google Groups. (Google deliberately
makes it difficult for bots to use Google Groups, so I would
presume it is actually posted by humans.)

Some people block spam by killing _everything_ posted through
Google Groups. In Agent, use this Usenet filter expression:
message-id: googlegroups.com

Note that message-id filter expressions do not work in
Find Global, so you cannot use Find Global to test them.

2. You could use a filter for specific (Indonesian, non-English)
words in the subject. For example
subject: ( bandar | terpercaya | mudah )
A subject filter might need maintenance from time to time as
new subjects arrive.

3. I would suggest using a boolean filter combining tests on
the subject, the spammer's addy being a Gmail addy, and
spam being posted though Google Groups. For example:
subject: ( bandar | terpercaya | mudah ) & from: gmail.com & message-id: googlegroups.com

With this combination, you may be able to add other words
like 'slot' to the subject part of the filter without too
much chance of killing legitimate posts.

--
Kind regards
Ralph Fox

‪“Aut viam inveniam aut faciam.” —— Hannibal Barca, 218 B.C.

On Tue, 28 Nov 2023 18:13:57 -0500, Joe Gwinn wrote:
> On Wed, 29 Nov 2023 12:01:14 +1300 (GMT+13:00), Ralph Fox
> <-rf-nz-@-.invalid> wrote:
>
>> The Bayesian filter in Agent works only with email, not with
>> newsgroup posts.
>
> Well, that's too bad. Where is that documented?

See these places in the help file:

1. The release notes for the Bayesian junk filter (introduced in Agent
version 4.0) are under 'Email Features'.
Help >> Index >> Release Notes >> Agent 4.0 Release Notes >> Agent 4.0 Email Features >> Junk Detection
There is no equivalent under 'Usenet Features'.

2. The 'Agent Features' page
Help >> Index >> Agent features >> Agent Features
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ QUOTE ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Handle all your email needs
* Filter out junk mail (spam) using Agent's Bayesian filtering
junk detection. Never be bother with unwanted email again.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ QUOTE ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

3. The 'Junk Folder' page
Help >> Index >> Junk >> Junk Folder
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ QUOTE ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Junk Folder
A special folder created by Agent where you will route your
unwanted email (junk).
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ QUOTE ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Also note that Agent's 'Junk Detection' settings are under 'Inbound Email',
at: Tools >> Options >> Inbound Email >> Junk and Routing

--
Kind Regards
Ralph Fox

"Ne sorga, snotor guma! Selre bið æghwæm þæt he his freond wrece, þonne he fela murne."
—— Beowulf to Hroðgar, son of Ecgþeow.

On Wed, 29 Nov 2023 16:44:42 +1300, Ralph Fox <-rf-nz-@-.invalid>
wrote:

>On Tue, 28 Nov 2023 17:12:55 -0500, Joe Gwinn wrote:
>
>> Recently, some newsgroups that I follow are being flooded by
>> bot-generated spam The subjects are in a foreign language
>> (Indonesia?), and are fairly consistent
>...
>> Any ideas?
>>
>> Joe Gwinn
>
>
>A normal Usenet filter can kill much of it.
>
>Some options
>
> 1. That spam is posted through Google Groups. (Google deliberately
> makes it difficult for bots to use Google Groups, so I would
> presume it is actually posted by humans.)
>
> Some people block spam by killing _everything_ posted through
> Google Groups. In Agent, use this Usenet filter expression:
> message-id: googlegroups.com
>
> Note that message-id filter expressions do not work in
> Find Global, so you cannot use Find Global to test them.
>
> 2. You could use a filter for specific (Indonesian, non-English)
> words in the subject. For example
> subject: ( bandar | terpercaya | mudah )
> A subject filter might need maintenance from time to time as
> new subjects arrive.
>
> 3. I would suggest using a boolean filter combining tests on
> the subject, the spammer's addy being a Gmail addy, and
> spam being posted though Google Groups. For example:
> subject: ( bandar | terpercaya | mudah ) & from: gmail.com & message-id: googlegroups.com

I suspect that this spam is via Google; I'll check.

> With this combination, you may be able to add other words
> like 'slot' to the subject part of the filter without too
> much chance of killing legitimate posts.

I will try these tricks, which can likely be tweaked to handle posts
with subject containing long strings of "?" symbols (probably what
remains after some unsupported kind of script is encountered
upstream).

Hmm. One thing that marks these examples is a very large fraction of
the characters being question marks. Like:

"??????? | ???? ??????? | ????? ??????? | ??????? ?????? | ???????
????? | ??????? ????? | ??????? ??????? ??????"

"? 10???100?????????? ? ?????? 20 ??? 100 wallet ?"

"pg slot ??????? ?????? pg slot ???????? ???????pg soft"

Is there any way to count these ? symbols and trigger if too many are
found? Looking for a fixed length string of ? alone would suffer
from false alarms if too short, and wrong accepts if too long.

Thanks,

Joe

On Wed, 29 Nov 2023 14:00:18 -0500, Joe Gwinn wrote:

> I will try these tricks, which can likely be tweaked to handle posts
> with subject containing long strings of "?" symbols (probably what
> remains after some unsupported kind of script is encountered
> upstream).
>
> Hmm. One thing that marks these examples is a very large fraction of
> the characters being question marks. Like:
>
> "??????? | ???? ??????? | ????? ??????? | ??????? ?????? | ???????
> ????? | ??????? ????? | ??????? ??????? ??????"
>
> "? 10???100?????????? ? ?????? 20 ??? 100 wallet ?"
>
> "pg slot ??????? ?????? pg slot ???????? ???????pg soft"
>
> Is there any way to count these ? symbols and trigger if too many are
> found? Looking for a fixed length string of ? alone would suffer
> from false alarms if too short, and wrong accepts if too long.

For creating a filter to handle "too many" question marks in the
subject, you will need to make two decisions:

Q.1 How many is "too many"? In other words, what minimum number of
question marks will be the trigger?

Q.2 Which of these do you want to catch?
a) Too many _consecutive_ question marks?
b) Too many _total_ question marks?

For case 2a (consecutive question marks)

* See the very recent thread with subject "another filtering question,
question marks", which was started around 4-5 days ago. The filter
in that thread (when corrected) will catch subjects with 4 or more
consecutive question marks. You can tweak this for any other
minimum number.

* On Jim Bradley's web page "{regular expression} filter examples" at
<http://web.archive.org/web/20030122175504/http://jlbradley.home.att.net/REGULAR.HTM>.
there is an example which will catch subjects with 3 or more
consecutive question marks. Look for the example titled "Hit three
consecutive question marks".

* In Agent there is no way to enter a _number_ (like "20") to say
how many. You must just add more 'escaped' question marks to the
regular expression.

* Use Agent's 'Find Global' to test your subject pattern on real
messages to help adjust the number of question marks to avoid
false positives and false negatives.

For case 2b (total question marks [whether consecutive or not])

* This example will catch messages with a total of 4 or more
question marks in the subject.
subject: {\?[^?]*\?[^?]*\?[^?]*\?}

* You can tweak this for any other number. In Agent, to increase
the number the regular expression must become longer.

--
Kind regards
Ralph Fox
🦊

He that goes a borrowing goes a sorrowing.

In Message-ID:<p7ofmiph0337fvc0gc01k4d0jc11t4l3q2@4ax.com>,
Ralph Fox <-rf-nz-@-.invalid> wrote:

> * This example will catch messages with a total of 4 or more
> question marks in the subject.
> subject: {\?[^?]*\?[^?]*\?[^?]*\?}

That works, but it took me a while to figure out why the question
marks inside the brackets didn't need to be preceded by backslashes.
Thank you for the lesson.

--
Arthur T. - ar23hur "at" pobox "dot" com

On Thu, 30 Nov 2023 00:01:13 -0500, Arthur T. wrote:
> In Message-ID:<p7ofmiph0337fvc0gc01k4d0jc11t4l3q2@4ax.com>,
> Ralph Fox <-rf-nz-@-.invalid> wrote:
>
>> * This example will catch messages with a total of 4 or more
>> question marks in the subject.
>> subject: {\?[^?]*\?[^?]*\?[^?]*\?}
>
> That works, but it took me a while to figure out why the question
> marks inside the brackets didn't need to be preceded by backslashes.
> Thank you for the lesson.

I first wrote it with backslashes before the question marks inside the
brackets. But then I thought I should check the help file to see what
it said:

Help >> Index >> Regular expressions >> Regular Expressions Reference
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ QUOTE ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Rules for Writing Regular Expressions
...
* A "range" of characters, enclosed in [...] square brackets. A
range is defined according to the following rules:
...
e. To include the characters \, ^, [, ], {, } , or - inside a
range as literals, prefix them with a \ backslash.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ QUOTE ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I tested it carefully without the backslashes, just to be sure that this
was not a mistaken omission in the help file.

--
Kind regards
Ralph Fox
 
The mouse that hath but one hole is easily taken.

On Thu, 30 Nov 2023 14:11:51 +1300, Ralph Fox <-rf-nz-@-.invalid>
wrote:

>
>For case 2b (total question marks [whether consecutive or not])
>
> * This example will catch messages with a total of 4 or more
> question marks in the subject.
> subject: {\?[^?]*\?[^?]*\?[^?]*\?}
>
> * You can tweak this for any other number. In Agent, to increase
> the number the regular expression must become longer.

For mine (the post a few days ago), consecutive ?s has captured
them all. Thanks again. I mark them Read (which don't show
when I look at posts) so I can change View and see if they are
still there.

There were only thirteen of them on Nov. 29. Most of them
have WY88 in the title and text. Or THE88. Googling WY88,
the first hit looked like Indonesian or some such squiggles.

Every post has a new title. Every poster (among dozens I
checked) had a gmail name. Many different names for most
posts. My Newsgroup has been not ENTIRELY dead, just
MOSTLY. If the deluge continues, I would be willing to try
to interrupt it, but is there any effective approach?

--
Rich Ulrich

On Thu, 30 Nov 2023 14:11:51 +1300, Ralph Fox <-rf-nz-@-.invalid>
wrote:

>On Wed, 29 Nov 2023 14:00:18 -0500, Joe Gwinn wrote:
>
>> I will try these tricks, which can likely be tweaked to handle posts
>> with subject containing long strings of "?" symbols (probably what
>> remains after some unsupported kind of script is encountered
>> upstream).
>>
>> Hmm. One thing that marks these examples is a very large fraction of
>> the characters being question marks. Like:
>>
>> "??????? | ???? ??????? | ????? ??????? | ??????? ?????? | ???????
>> ????? | ??????? ????? | ??????? ??????? ??????"
>>
>> "? 10???100?????????? ? ?????? 20 ??? 100 wallet ?"
>>
>> "pg slot ??????? ?????? pg slot ???????? ???????pg soft"
>>
>> Is there any way to count these ? symbols and trigger if too many are
>> found? Looking for a fixed length string of ? alone would suffer
>> from false alarms if too short, and wrong accepts if too long.
>
>
>For creating a filter to handle "too many" question marks in the
>subject, you will need to make two decisions:
>
> Q.1 How many is "too many"? In other words, what minimum number of
> question marks will be the trigger?
>
> Q.2 Which of these do you want to catch?
> a) Too many _consecutive_ question marks?
> b) Too many _total_ question marks?
>
>For case 2a (consecutive question marks)
>
> * See the very recent thread with subject "another filtering question,
> question marks", which was started around 4-5 days ago. The filter
> in that thread (when corrected) will catch subjects with 4 or more
> consecutive question marks. You can tweak this for any other
> minimum number.
>
> * On Jim Bradley's web page "{regular expression} filter examples" at
> <http://web.archive.org/web/20030122175504/http://jlbradley.home.att.net/REGULAR.HTM>.
> there is an example which will catch subjects with 3 or more
> consecutive question marks. Look for the example titled "Hit three
> consecutive question marks".
>
> * In Agent there is no way to enter a _number_ (like "20") to say
> how many. You must just add more 'escaped' question marks to the
> regular expression.
>
> * Use Agent's 'Find Global' to test your subject pattern on real
> messages to help adjust the number of question marks to avoid
> false positives and false negatives.
>
>
>For case 2b (total question marks [whether consecutive or not])
>
> * This example will catch messages with a total of 4 or more
> question marks in the subject.
> subject: {\?[^?]*\?[^?]*\?[^?]*\?}
>
> * You can tweak this for any other number. In Agent, to increase
> the number the regular expression must become longer.

Something like this ought to suffice. I'll try it.

Hmm. Yet another follow-up question: The above expressions count the
absolute number of question marks, which may have problems with short
subject fields. Is there any way to approximate triggering if at
least say 80% (four of five) of the characters are question marks?

Thanks,

Joe

On Wed, 29 Nov 2023 16:44:42 +1300, Ralph Fox <-rf-nz-@-.invalid>
wrote:

>On Tue, 28 Nov 2023 17:12:55 -0500, Joe Gwinn wrote:
>
>> Recently, some newsgroups that I follow are being flooded by
>> bot-generated spam The subjects are in a foreign language
>> (Indonesia?), and are fairly consistent
>...
>> Any ideas?
>>
>> Joe Gwinn
>
>
>A normal Usenet filter can kill much of it.
>
>Some options
>
> 1. That spam is posted through Google Groups. (Google deliberately
> makes it difficult for bots to use Google Groups, so I would
> presume it is actually posted by humans.)
>
> Some people block spam by killing _everything_ posted through
> Google Groups. In Agent, use this Usenet filter expression:
> message-id: googlegroups.com
>
> Note that message-id filter expressions do not work in
> Find Global, so you cannot use Find Global to test them.
>
> 2. You could use a filter for specific (Indonesian, non-English)
> words in the subject. For example
> subject: ( bandar | terpercaya | mudah )
> A subject filter might need maintenance from time to time as
> new subjects arrive.
>
> 3. I would suggest using a boolean filter combining tests on
> the subject, the spammer's addy being a Gmail addy, and
> spam being posted though Google Groups. For example:
> subject: ( bandar | terpercaya | mudah ) & from: gmail.com & message-id: googlegroups.com
>
> With this combination, you may be able to add other words
> like 'slot' to the subject part of the filter without too
> much chance of killing legitimate posts.

Thanks for all the information in this thread.

The infestation of these things disappeared from the newsgroup I was
seeing them in when the thread started, so I haven't had occasion to
test any of the ideas presented.

Tempting as it is to think that the thread's very existence suppressed
them, I suspect it has more to do with Eternal September improving
their filtering.
--
"Here lies the Tuscan poet Aretino,
Who evil spoke of everyone but God,
Giving as his excuse, 'I never knew him.'"

On Thu, 30 Nov 2023 09:06:19 -0800, Paul S Person wrote:

> The infestation of these things disappeared from the newsgroup I was
> seeing them in when the thread started, so I haven't had occasion to
> test any of the ideas presented.
>
> Tempting as it is to think that the thread's very existence suppressed
> them, I suspect it has more to do with Eternal September improving
> their filtering.

A. While I am not an E-S user, I do gather that E-S is working on
improving its filtering. See for example Ray Banana's post
<http://al.howardknight.net/?ID=170136932600>

B. Google also appears to have done at least something. Newsgroup
alt.comp.freeware was absolutely riddled with spam from Google
Groups until the beginning of November 2023. Then the GG spam
flood stopped, even on Google Groups itself. (Sadly though,
the spammers have just moved to spam other groups.)
See <http://al.howardknight.net/?ID=170137043500>
and <http://al.howardknight.net/?ID=170137072000>

--
Kind regards
Ralph Fox
🦊

Better come at the latter end of a feast, then the beginning of a fray.

On Thu, 30 Nov 2023 10:41:22 -0500, Joe Gwinn wrote:

> Hmm. Yet another follow-up question: The above expressions count the
> absolute number of question marks, which may have problems with short
> subject fields. Is there any way to approximate triggering if at
> least say 80% (four of five) of the characters are question marks?

I think you would need to do each short length case by case.
For example, to catch subjects of length 4 with at least 3 question
marks, use one of these

subject: ( ={....} & {\?[^?]*\?[^?]*\?} )

subject: ( {^....$} & {\?[^?]*\?[^?]*\?} )

The first looks for subjects of length 4 _excluding_ any "Re: " reply
prefix. This means it will also catch replies to subjects of length 4
with at least 3 question marks, like these:
???p
Re: ???p

The second looks for subjects of length 4 _including_ any "Re: " reply
prefix. This means it will not catch replies to subjects of length 4
with at least 3 question marks (the reply subject has length 8 including
the "Re: ").

--
Kind regards
Ralph Fox
🦊

Fire and water are good servants, but bad ma∣sters.

On Fri, 01 Dec 2023 08:23:09 +1300, Ralph Fox <-rf-nz-@-.invalid>
wrote:

>On Thu, 30 Nov 2023 10:41:22 -0500, Joe Gwinn wrote:
>
>> Hmm. Yet another follow-up question: The above expressions count the
>> absolute number of question marks, which may have problems with short
>> subject fields. Is there any way to approximate triggering if at
>> least say 80% (four of five) of the characters are question marks?
>
>
>I think you would need to do each short length case by case.
>For example, to catch subjects of length 4 with at least 3 question
>marks, use one of these
>
> subject: ( ={....} & {\?[^?]*\?[^?]*\?} )
>
> subject: ( {^....$} & {\?[^?]*\?[^?]*\?} )
>
>
>The first looks for subjects of length 4 _excluding_ any "Re: " reply
>prefix. This means it will also catch replies to subjects of length 4
>with at least 3 question marks, like these:
> ???p
> Re: ???p
>
>The second looks for subjects of length 4 _including_ any "Re: " reply
>prefix. This means it will not catch replies to subjects of length 4
>with at least 3 question marks (the reply subject has length 8 including
>the "Re: ").

This does seem a bit clumsy for all but very short strings. Some
regex flavors can count, but most cannot. I think I'll just go with
the prior answers, and see what leaks through.

And pray that Google gets a handle on the problem. Maybe nuke the
offending ISPs.

Thanks,

Joe