Hallo, all,

Yes, I've set a filter to delete emails from a certain sender, but it
doesn't always work. I keep seeing it my Junk/Spam folder.

I feel kinda ignorant asking this, but here goes....

My question is, does it work to place the offending URL (in this case
"pcmiracles.com") in my HOSTs file to keep it from showing up at all?

--
Revvie Quar

Gryphons ~RAWK!~

On 2023-11-30 23:23, Revvie Quar wrote:
>
> Hallo, all,
>
> Yes, I've set a filter to delete emails from a certain sender, but it
> doesn't always work.  I keep seeing it my Junk/Spam folder.
>
> I feel kinda ignorant asking this, but here goes....
>
> My question is, does it work to place the offending URL (in this case
> "pcmiracles.com") in my HOSTs file to keep it from showing up at all?

Not in email, no.

You can avoid the URL from opening, that's all.

--
Cheers,
Carlos E.R.

On 11/30/2023 16:38, Carlos E. R. took the time to scribe:
> On 2023-11-30 23:23, Revvie Quar wrote:
>>
>> Hallo, all,
>>
>> Yes, I've set a filter to delete emails from a certain sender, but it
>> doesn't always work.  I keep seeing it my Junk/Spam folder.
>>
>> I feel kinda ignorant asking this, but here goes....
>>
>> My question is, does it work to place the offending URL (in this case
>> "pcmiracles.com") in my HOSTs file to keep it from showing up at all?
>
> Not in email, no.
>
> You can avoid the URL from opening, that's all.

Alrighty then, kinda thought so.

Thanks for the quick reply.

--
Revvie Quar

Gryphons ~RAWK!~

Revvie Quar <RevvieQuar@Yahoo.com> wrote:

> Yes, I've set a filter to delete emails from a certain sender, but it
> doesn't always work. I keep seeing it my Junk/Spam folder.
>
> I feel kinda ignorant asking this, but here goes....
>
> My question is, does it work to place the offending URL (in this case
> "pcmiracles.com") in my HOSTs file to keep it from showing up at all?

No, adding hosts to the hosts file (no extension) prevents you from
visiting there, not from getting messages with the URL in them. URLs
are just text strings. The hosts file would block you, or resource
links within a web page, from visiting the target site. Nothing to do
with e-mail at all.

Unless their nameserver is configured to accept any hostname to reach
their web server host, you cannot use just domain names in the hosts
file. It is, after all, a file listing hosts. You would need to
prepend "www.", or whatever hostname they specify, to block that host at
that domain. Adding just the domain name will not block every hostname
at the domain.

Junk filtering by your e-mail provider occurs first despite your rules
define either server- or client-side. Their junk filtering could be
moving the unwanted sender to your Junk folder. Junk filtering in Tbird
occurs before your rules inside of Tbird are exercised (*). You would
have to disable junk filtering in Tbird, and then your rule would get
applied first, like to mark-as-read and delete (to Trash folder) without
interference by Tbird's junk filtering.

(*) I have not use Tbird for many years. As I recall, junk filtering,
if enabled, was performed first, and then rules got applied. If it's
the other way around now, and your rules override Tbird's junk filtering
feature, then you should be able to always fire your rule before Tbird's
junk filtering gets applied. That won't help if the server is moving
the message to the Junk folder before your client ever connects to your
account (but you'll only see the server-side Junk folder when using
IMAP, Exchange, or Gmail Mail API, not when using POP).

If your e-mail provider is not flagging the unwanted message as junk to
move it into the server-side Junk folder, you'll need to define a
server-side rule to delete the unwanted message. By filtering up on the
server, you don't need to bother defining the rule in your local e-mail
client. Have the server do most of the filtering before your client
ever connects to your account.

On 11/30/2023 2:23 PM, Revvie Quar wrote:
>
> Hallo, all,
>
> Yes, I've set a filter to delete emails from a certain sender, but it
> doesn't always work. I keep seeing it my Junk/Spam folder.
>
> I feel kinda ignorant asking this, but here goes....
>
> My question is, does it work to place the offending URL (in this case
> "pcmiracles.com") in my HOSTs file to keep it from showing up at all?
>

Do you receive E-mail via POP3 or IMAP?

--
David E. Ross
<http://www.rosde.com/>

Paris mayor quits X platform, calling it a 'gigantic global sewer'.
Others characterize X (previously known as Twitter) as the place
where truth goes to die.

On 11/30/2023 21:29, David E. Ross took the time to scribe:
> On 11/30/2023 2:23 PM, Revvie Quar wrote:
>>
>> Hallo, all,
>>
>> Yes, I've set a filter to delete emails from a certain sender, but it
>> doesn't always work. I keep seeing it my Junk/Spam folder.
>>
>> I feel kinda ignorant asking this, but here goes....
>>
>> My question is, does it work to place the offending URL (in this case
>> "pcmiracles.com") in my HOSTs file to keep it from showing up at all?
>>
>
> Do you receive E-mail via POP3 or IMAP?

The "offending" server is Yahoo email, and I have set up as IMAP into my
Betterbird client.

Yahoo filters it as junk, which in turn, makes it show up in my
Betterbird Junk folder.

I have a filter in BBird set to have the offending email Mark As
Read/Delete. Doesn't always work.

I went straight to Yahoo Mail and created a specific filter for it there.

We shall now see if "PCMiracles.com" even makes it into BBird now.

That's where I stand as of now. I'm in let's-wait-and-see mode at present.

--
Revvie Quar

Gryphons ~RAWK!~

On 2023-12-01 23:27, Revvie Quar wrote:
> On 11/30/2023 21:29, David E. Ross took the time to scribe:
>> On 11/30/2023 2:23 PM, Revvie Quar wrote:
>>>
>>> Hallo, all,
>>>
>>> Yes, I've set a filter to delete emails from a certain sender, but it
>>> doesn't always work.  I keep seeing it my Junk/Spam folder.
>>>
>>> I feel kinda ignorant asking this, but here goes....
>>>
>>> My question is, does it work to place the offending URL (in this case
>>> "pcmiracles.com") in my HOSTs file to keep it from showing up at all?
>>>
>>
>> Do you receive E-mail via POP3 or IMAP?
>
> The "offending" server is Yahoo email, and I have set up as IMAP into my
> Betterbird client.
>
> Yahoo filters it as junk, which in turn, makes it show up in my
> Betterbird Junk folder.
>
> I have a filter in BBird set to have the offending email Mark As
> Read/Delete.  Doesn't always work.
>
> I went straight to Yahoo Mail and created a specific filter for it there.
>
> We shall now see if "PCMiracles.com" even makes it into BBird now.
>
> That's where I stand as of now.  I'm in let's-wait-and-see mode at present.

VanguardLH is probably right.

--
Cheers,
Carlos E.R.

On 11/30/2023 17:11, VanguardLH took the time to scribe:
> Revvie Quar <RevvieQuar@Yahoo.com> wrote:
>
>> Yes, I've set a filter to delete emails from a certain sender, but it
>> doesn't always work. I keep seeing it my Junk/Spam folder.
>>
>> I feel kinda ignorant asking this, but here goes....
>>
>> My question is, does it work to place the offending URL (in this case
>> "pcmiracles.com") in my HOSTs file to keep it from showing up at all?
>
> No, adding hosts to the hosts file (no extension) prevents you from
> visiting there, not from getting messages with the URL in them. URLs
> are just text strings. The hosts file would block you, or resource
> links within a web page, from visiting the target site. Nothing to do
> with e-mail at all.
>
> Unless their nameserver is configured to accept any hostname to reach
> their web server host, you cannot use just domain names in the hosts
> file. It is, after all, a file listing hosts. You would need to
> prepend "www.", or whatever hostname they specify, to block that host at
> that domain. Adding just the domain name will not block every hostname
> at the domain.
>
> Junk filtering by your e-mail provider occurs first despite your rules
> define either server- or client-side. Their junk filtering could be
> moving the unwanted sender to your Junk folder. Junk filtering in Tbird
> occurs before your rules inside of Tbird are exercised (*). You would
> have to disable junk filtering in Tbird, and then your rule would get
> applied first, like to mark-as-read and delete (to Trash folder) without
> interference by Tbird's junk filtering.
>
> (*) I have not use Tbird for many years. As I recall, junk filtering,
> if enabled, was performed first, and then rules got applied. If it's
> the other way around now, and your rules override Tbird's junk filtering
> feature, then you should be able to always fire your rule before Tbird's
> junk filtering gets applied. That won't help if the server is moving
> the message to the Junk folder before your client ever connects to your
> account (but you'll only see the server-side Junk folder when using
> IMAP, Exchange, or Gmail Mail API, not when using POP).
>
> If your e-mail provider is not flagging the unwanted message as junk to
> move it into the server-side Junk folder, you'll need to define a
> server-side rule to delete the unwanted message. By filtering up on the
> server, you don't need to bother defining the rule in your local e-mail
> client. Have the server do most of the filtering before your client
> ever connects to your account.

That, as I wrote in reply to David, is pretty much what I did; created
a filter on the Yahoo email site. Yahoo *did* mark it as Junk which in
turn put in BBird's Junk folder, but the filter I created to Mark As
Read/Delete didn't seem to work. Maybe since I created the filter
server-side, I will no longer see "pcmiracles" in my email at all.

Here's hoping, and thanks for your detailed reply.

--
Revvie Quar

Gryphons ~RAWK!~

Revvie Quar <RevvieQuar@Yahoo.com> wrote:

> On 11/30/2023 17:11, VanguardLH took the time to scribe:
>> Revvie Quar <RevvieQuar@Yahoo.com> wrote:
>>
>>> Yes, I've set a filter to delete emails from a certain sender, but it
>>> doesn't always work. I keep seeing it my Junk/Spam folder.
>>>
>>> I feel kinda ignorant asking this, but here goes....
>>>
>>> My question is, does it work to place the offending URL (in this case
>>> "pcmiracles.com") in my HOSTs file to keep it from showing up at all?
>>
>> No, adding hosts to the hosts file (no extension) prevents you from
>> visiting there, not from getting messages with the URL in them. URLs
>> are just text strings. The hosts file would block you, or resource
>> links within a web page, from visiting the target site. Nothing to do
>> with e-mail at all.
>>
>> Unless their nameserver is configured to accept any hostname to reach
>> their web server host, you cannot use just domain names in the hosts
>> file. It is, after all, a file listing hosts. You would need to
>> prepend "www.", or whatever hostname they specify, to block that host at
>> that domain. Adding just the domain name will not block every hostname
>> at the domain.
>>
>> Junk filtering by your e-mail provider occurs first despite your rules
>> define either server- or client-side. Their junk filtering could be
>> moving the unwanted sender to your Junk folder. Junk filtering in Tbird
>> occurs before your rules inside of Tbird are exercised (*). You would
>> have to disable junk filtering in Tbird, and then your rule would get
>> applied first, like to mark-as-read and delete (to Trash folder) without
>> interference by Tbird's junk filtering.
>>
>> (*) I have not use Tbird for many years. As I recall, junk filtering,
>> if enabled, was performed first, and then rules got applied. If it's
>> the other way around now, and your rules override Tbird's junk filtering
>> feature, then you should be able to always fire your rule before Tbird's
>> junk filtering gets applied. That won't help if the server is moving
>> the message to the Junk folder before your client ever connects to your
>> account (but you'll only see the server-side Junk folder when using
>> IMAP, Exchange, or Gmail Mail API, not when using POP).
>>
>> If your e-mail provider is not flagging the unwanted message as junk to
>> move it into the server-side Junk folder, you'll need to define a
>> server-side rule to delete the unwanted message. By filtering up on the
>> server, you don't need to bother defining the rule in your local e-mail
>> client. Have the server do most of the filtering before your client
>> ever connects to your account.
>
> That, as I wrote in reply to David, is pretty much what I did; created
> a filter on the Yahoo email site. Yahoo *did* mark it as Junk which in
> turn put in BBird's Junk folder, but the filter I created to Mark As
> Read/Delete didn't seem to work. Maybe since I created the filter
> server-side, I will no longer see "pcmiracles" in my email at all.
>
> Here's hoping, and thanks for your detailed reply.

If Yahoo's spam filtering is exercised before user-defined rules, even
server-side rules, Yahoo will first put the message into the Junk
folder, but your rules work on the Inbox folder. The only way to
override their junk filtering exercised first is to disable it, and then
rely solely on your rules.

If Yahoo's spam filtering detects a message a message, Yahoo move it to
your Junk filter. When your server-side rules get applied next, the
message is no longer in the Inbox folder where the rules get applied.
If Yahoo doesn't detect the spam, it stays in the Inbox folder where
your rules can then handle the spam. Think of the mailman who also does
junk filtering: if he finds spam mails and tosses them in the trash, you
looking inside the mailbox won't find them. If the mailman misses what
you consider spam but he doesn't, you'll still find it in the mailbox
for you to then toss them in the trash.

Instead of relying on server-side spam filtering and your server-side
rules, does Yahoo have the option of defining a blacklist of senders?
If so, perhaps they apply the blacklist before exercising their spam
filtering and before applying your rules. However, typically when that
is available, the blacklisted senders go into the Junk folder, not
marked-as-read into the Trash folder. No spam filtering is perfect
meaning there can be false positives: ham flagged as spam.

To prevent an e-mail provider from being overly aggressive, and you
losing important e-mails, they dump into the Junk folder giving you a
chance to recover the false positives. It's your responsibility to
check, or not check, the Junk folder before the server expires the
Junked messages to move into the Trash folder. I believe Microsoft and
Gmail expire messages in the Junk folder after they are 30 days old
although some e-mail providers let you decide how long to wait. Comcast
used to let me set 3 days, 7 days, or a month for retention, but that
option disappeared when they contracted for a new webmail client, so now
it's 7 days.

Just because your client notes there are new messages in the Junk folder
does not force the client to go look in the Junk folder nor that you
have to look there. If you believe spam filtering, blacklists, and
rules are perfect, just never bother looking in the Junk folder. For
me, I do look because there have been false positives, especially from
senders whose messages are very important.

Rather than show you the preview pane for the Junk folder which would
show the suspect message, some e-mail clients let you turn off the
preview pane on a per-folder basis. They also will let you enable an
option to view the first n-lines as plain text, so you have an idea of
what is in the message besides the Subject header. MS Outlook lets you
disable the preview pane (actually lets you select which view to use)
and show up to 3 lines of plain text in the body for the Junk folder (or
any other folder, like Trash). My current e-mail client, eM Client,
does not have that option. Been too long since I used Tbird to remember
if per-folder view mode was user selected. That way when you inspect
the Junk folder, you don't see the suspect messages in a preview pane as
you select between them, but just the header pane showing Subject, or
the header pane along with 1 to 3 lines of plain text from the body.
However, defining and selecting a view on the properties of a per-folder
basis was not a super easy process, but you only had to do it once.

If you want to ensure your server-side rules always get exercised
against every new message that shows up in the Inbox, you'll have to
disable the server-side spam filtering, and rely only on your rules to
get rid of spam. You can add client-side spam filtering (Tbird has its
Bayesian filtering), but typically those are worse than the server-side
spam filtering (Bayes is a guessing scheme using keyword weighting).
You can install add-ons to the client that will let you use a better
Bayes scheme (Tbird doesn't even have a floor function to expire old and
outdated keywords), but they can also use DNSBLs (DNS blacklists, like
Spamhaus and SpamCop), DCC (Distributed Checksum Clearinghous) to detect
bulk e-mails sents to many others also participating in DCC (not all
bulk e-mails contain the "Precedence: bulk" header, but that's something
your rules could look for), or other anti-spam schemes. If Bayesian is
included in the mix of anti-spam measures, it should be the *last*
method employed.

Some e-mail clients, and some e-mail services, let you blacklist every
sender except those in your Contacts. That is, everything gets Junked
unless in a whitelist (your Contacts). That will generate a lot of
false positives, like your dentist, doctor, clinic, coworker,
non-immediate family, old or infrequent friends, and anyone you didn't
add as a Contact. Comcast calls the option "E-mail safe list": you add
senders to the whitelist, and those are the only senders not subject to
server-side spam filtering. Comcast's safe list will discard non-safe
senders: those messages won't show up in any folder in your account.
You have no ability to check if a good sender got discarded because they
weren't in your safe list. Other e-mail providers might move all those
non-contact messages go into the Junk folder where it is still your
responsibility to check for false positives, or you'll have to force
yourself to never look in the Junk folder despite your client showing
there are new messages there. E-mail services are averse to deleting
suspect messages, because there is no such thing as perfect spam
filtering, and they don't want to be viewed as despots regarding your
e-mail traffic. I don't have a Yahoo account to look at what spam and
safe-sender settings they may offer, or if you can even disable their
server-side spam filtering. I know with Gmail you cannot disable their
spam filtering.

VanguardLH wrote:

> Junk filtering by your e-mail provider occurs first despite your rules
> define either server- or client-side. Their junk filtering could be
> moving the unwanted sender to your Junk folder. Junk filtering in Tbird
> occurs before your rules inside of Tbird are exercised (*). You would
> have to disable junk filtering in Tbird, and then your rule would get
> applied first, like to mark-as-read and delete (to Trash folder) without
> interference by Tbird's junk filtering.

Server-side filtering is the first eschelon of spam-filtering.
Depending on how a server is set up, the processes of marking messages
as Junk and Not Junk are generally sufficient to pretty much eliminate
both false negatives (junk that gets through) and false positives
(legitimate stuff marked as junk).

If the server filters are adequately tuned, then client filtering
(Thunderbird or any other client) is rarely necessary, as what reaches
the client shows only what has gotten past the server filters. In that
case, the thing to do is not to try to do more client-level filtering,
but go to the server's web client, and mark the offending content as junk.

That said, it's also important to periodically check (every couple of
weeks or so) the web client to make sure there aren't any false
positives (and correct those).

It's also important to note that user-tuned filtering normally uses
what's called Bayesian filters. With Bayesian, if a message is marked
as Junk (or Not Junk), then subsequent messages that are sufficiently
similar will also be considered in the same way. And "similar" means the
entire content of the message, including all headers, body and
attachments, not just the obvious headers of From: or Subject: . One of
the effects of of that is that if you receive a message with a forged
From: line (usually, it will be just the display name, and not a true
address), it's safe to mark as Junk, because the sender address has only
a small amount of weighting in the filter. Thus, subsequent messages
with similar content (especially HTML coding of the body) will also be
treated as Junk, regardless of whether the sender shows the same From:
line or not.

This really isn't a fault of Thunderbird, so much difference in capacity
between client and server. The server is interacting with mail at the
time it's received from the sender, and as a result, has a lot more
ability to interact while the SMTP session is open (and before accepting
mail), as well as what is done with an individual message after it's
been accepted. The client only has the ability to interact after
acceptance, and the incoming message has been written to a folder.

In my view, client-based spam filtering mostly stopped being especially
useful when the world moved away from dial-up POP connections to
realtime IMAP connections.

For the Original Poster, the thing to do is to use the web client --
mark the offending message as Junk, and be done with it. Occasionally,
the sender might vary content a little bit, where it might take a second
marking as Junk, but after that, it's unlikely to see any further
traffic from that sender, unless you go exploring of the Junk folder on
the server.

Smith

NFN Smith <worldoff9908@gmail.com> wrote:

> VanguardLH wrote:
>
>> Junk filtering by your e-mail provider occurs first despite your
>> rules define either server- or client-side. Their junk filtering
>> could be moving the unwanted sender to your Junk folder. Junk
>> filtering in Tbird occurs before your rules inside of Tbird are
>> exercised (*). You would have to disable junk filtering in Tbird,
>> and then your rule would get applied first, like to mark-as-read and
>> delete (to Trash folder) without interference by Tbird's junk
>> filtering.
>
> Server-side filtering is the first eschelon of spam-filtering.
> Depending on how a server is set up, the processes of marking messages
> as Junk and Not Junk are generally sufficient to pretty much eliminate
> both false negatives (junk that gets through) and false positives
> (legitimate stuff marked as junk).

It's taken 10 years, or more, before server-side spam filtering became
adequate. A lot of server- and client-side filters that I had became
unnecessary. How effective is the server-side spam filtering depends on
which e-mail provider you use.

> That said, it's also important to periodically check (every couple of
> weeks or so) the web client to make sure there aren't any false
> positives (and correct those).

Not required if you use IMAP, Exchange (ActiveSync), or Google Mail API
in the client to sync to your account. You only need the webmail client
if you don't subscribe to a folder in your client that is in your
account.

One reason to use the webmail client is to vote on spam/ham to get their
filters updated. At one time, Gmail would update their filter (with
lots of similar voting by MANY users) if you moved a message from the
Inbox folder to the Junk folder in your client, but only when using
IMAP. The move in the client effected a move on the server, and Gmail
would record your spam vote. That was many years ago, so I don't know
if they are still doing it. Most e-mail providers don't care how you
move messages around either at the server or client regarding what you
moved in or out of the Junk folder. The only way to get their weighting
updated is to use the Spam or Not Spam buttons in their webmail client.
Too often, however, those button are just move operations, and have no
effect to update their filtering. You might get an important e-mail
from someone that the server-side filtering keeps junking, and no matter
how many hundreds of times of clicking Not Spam or moving back to the
Inbox do they stop flagging that sender or their messages as spam.

If you cannot get the server-side filtering to stop flagging a good
sender as spam, check if there is a safe sender whitelist option in your
account. The expectation is the whitelist is exercised before their
spam filtering, but you have to test since that's not always the case.
For some accounts, senders in your Contacts are whitelisted. For some
accounts, you have to maintain a separate safe sender whitelist. Or,
for some accounts, you have the option of both: Contacts are never
flagged as spam, and safe senders (that you don't want to pollute into
your Contacts) are also never flagged as spam. If the providers has no
safe sender whitelist, you have to try using Contacts to keep a good
sender from getting flagged by the server-side spam filtering.

> It's also important to note that user-tuned filtering normally uses
> what's called Bayesian filters. With Bayesian, if a message is marked
> as Junk (or Not Junk), then subsequent messages that are sufficiently
> similar will also be considered in the same way.

A major defect with Bayes is the requirement the user MUST vote on spam
versus ham. Many users figure Bayes will just magically do the
filtering, and they never vote on spam/ham. They just move from Junk to
Inbox for ham flagged as spam, or just hit Delete on spam in the Inbox
folder. Bayes requires user involvement to update keyword weighting.

The volume of e-mail traffic is also important to Bayes scoring.
Getting 1 e-mail per week is insufficient volume to maintain a Bayes
database that is effective. You need dozens of e-mails per week, or
more, to effect decent weighting of keywords. The more e-mail volume
you get, and the more you click the Spam button to vote, the more
effective the Bayes filtering. Else, Bayes would be far too coarse to
be effective. Weighting is not red-green (bad-good). With each
additional score on a keyword, its weighting increases. Heavy weighted
keywords are useful for identifying spam. Lightly weighted keywords are
useless. Weighting is incremental, not catastophic, for granularity.

Bayes is always a statisical guessing game. If employed, it should be
the last method for spam filtering. DNSBLs are far more accurate. DCC
will help if you want to flag bulk mailings sent to a threshold in count
to other recipients.

> This really isn't a fault of Thunderbird, so much difference in capacity
> between client and server. The server is interacting with mail at the
> time it's received from the sender, and as a result, has a lot more
> ability to interact while the SMTP session is open (and before accepting
> mail), as well as what is done with an individual message after it's
> been accepted. The client only has the ability to interact after
> acceptance, and the incoming message has been written to a folder.

That the MTAs (SMTP servers) have an open session during spam filtering
is why that is the ONLY time a reject (bounce) should be sent to the
sending MTA by the receiving MTA. That is the only time the envelope
for the session is available. At that time, the receiving end actually
knows who is the sending end. After the session, the sender is no
longer known, and only the headers can be used to send a reject;
however, the headers can be forged or spoofed. Anyone sending fake
bounces on receiving e-mails is doing it too late, and are themself a
cause of spam. As a spam reporter, I will report anyone sending me fake
bounces sent by their client. Spammers never get those fake bounces.

I will also report as spam any challenges sent to me for e-mails never
sent by me. There are boobs that use Challenge-Response schemes: when
they get an e-mail, its delivery is pending, and a challenge is sent to
what the recipient /thinks/ was the sender. The From, Sender, and other
headers can be forged, so the actual sender never gets the challenge.
Either the server has to waste bandwidth and time to send challenges
that cannot be delivered, or the challenges get delivered to innocents.
It is not the job of senders to clean up the Inbox of recipients.
Challenges sent to innocents (not to the actual senders) is itself spam,
and will get reported as such.

VanguardLH wrote:
>
> If Yahoo's spam filtering is exercised before user-defined rules, even
> server-side rules, Yahoo will first put the message into the Junk
> folder, but your rules work on the Inbox folder. The only way to
> override their junk filtering exercised first is to disable it, and then
> rely solely on your rules.

I believe that's the case. I do have a Yahoo account, but it's been a
long time since I've looked at the procedures for spam handling. But I
think it's reasonably safe to assume that Yahoo's procedures take
precedence, and where user-defined rules only run after. And using the
Junk/Not Junk designations are included in the server handling, where
they're active before user-defined rules are run.

>
> If Yahoo's spam filtering detects a message a message, Yahoo move it to
> your Junk filter. When your server-side rules get applied next, the
> message is no longer in the Inbox folder where the rules get applied.
> If Yahoo doesn't detect the spam, it stays in the Inbox folder where
> your rules can then handle the spam.

Yep. Basically, user-defined rules only apply to stuff that gets past
the server filters.

>
> Instead of relying on server-side spam filtering and your server-side
> rules, does Yahoo have the option of defining a blacklist of senders?

I actually went and logged into my Yahoo account. The Help section is
useful. https://help.yahoo.com/kb/new-mail-for-desktop#/

> If so, perhaps they apply the blacklist before exercising their spam
> filtering and before applying your rules. However, typically when that
> is available, the blacklisted senders go into the Junk folder, not
> marked-as-read into the Trash folder. No spam filtering is perfect
> meaning there can be false positives: ham flagged as spam.

According to this, there is a capacity for blacklisting, although the
article doesn't note handling, whether mail from a specified address is
rejected or merely shunted to a Junk folder. There's also capacity for
blocking by domain, not just address. It's not clearly stated, but the
implication is that blacklisting is active before other filtering.

> To prevent an e-mail provider from being overly aggressive, and you
> losing important e-mails, they dump into the Junk folder giving you a
> chance to recover the false positives. It's your responsibility to
> check, or not check, the Junk folder before the server expires the
> Junked messages to move into the Trash folder.

And it is important to watch for false positives. Even if all the
user-trained filters are working well, there will always be something
that is sufficiently unique that requires review and tuning.

One of the places that is important to keep an eye on is for message
traffic that originates from applications that use outsourced mail
servers, where the mail server isn't part of the sender's domain, but
sending on behalf of that domain. One subset of that is mailing list
traffic. I won't belabor explanation of anti-forgery technologies such
as SPF, DKIM and DMARC, but if a sender's domain hasn't configured those
properly, then those messages are more likely to get into junk folders,
until they've been marked as Not Junk. I haven't seen a lot of that
kind of false positive with Yahoo (but I use my account only for
secondary traffic), but I know that both Gmail and Microsoft are more
aggressive about throwing stuff in Junk folders. Gmail seems to be
especially aggressive about treating traffic from mailing lists (even
coming through big providers such as Constant Contact, MailChimp, etc.)
if domain configs aren't done correctly.

A relevant aside... I've seen a lot of traffic that comes via mailing
list providers that includes footer text that appeals to the recipient
to "please add us to your address book", and seems to be something
included in a template. 15 or 20 years ago, that may have been useful,
when it was common for server configs to allow users to specify allowing
only inbound mail from known senders (and reject everything else).
However, now that use of SPF, DKIM and DMARC are now commonly used,
they're a lot more effective (and generally don't carry the risks of
accepting mail coming from forged addresses), and I'm not aware of any
provider (at least none of the majors, especially Gmail that offer that
capacity, much less encourage users to do).

> I believe Microsoft and
> Gmail expire messages in the Junk folder after they are 30 days old
> although some e-mail providers let you decide how long to wait. Comcast
> used to let me set 3 days, 7 days, or a month for retention, but that
> option disappeared when they contracted for a new webmail client, so now
> it's 7 days.

I believe that Yahoo also uses 30 days.

>
> Just because your client notes there are new messages in the Junk folder
> does not force the client to go look in the Junk folder nor that you
> have to look there. If you believe spam filtering, blacklists, and
> rules are perfect, just never bother looking in the Junk folder. For
> me, I do look because there have been false positives, especially from
> senders whose messages are very important.

See above.

There is also a question of how you have your client set to synchronize
against server folders. I believe that the normal handling of an IMAP
client is that in default settings, the server's Junk folder is not
synchronized with the local client. It's easy enough to add that (in
Thunderbird, right click on the account, select "Subscribe" and then
enable synchronization of the server's Junk folder). If you do that,
then you see what the server is throwing in Junk. That's useful for
recognizing false positives without having to go through the web client,
but if signal to noise ratio of occasional false positives to
comparatively large quantities of true junk, then it may not be worth
the effort (or the bandwidth to download stuff that you simply want to
discard.)

> Rather than show you the preview pane for the Junk folder which would
> show the suspect message, some e-mail clients let you turn off the
> preview pane on a per-folder basis. They also will let you enable an
> option to view the first n-lines as plain text, so you have an idea of
> what is in the message besides the Subject header.

I don't believe Thunderbird allows for that. Basically, a global
setting for enabling preview for all folders, and that's it. The only
other display is whether or not you want to enable display of all
headers (which is normally turned off).

>
> If you want to ensure your server-side rules always get exercised
> against every new message that shows up in the Inbox, you'll have to
> disable the server-side spam filtering, and rely only on your rules to
> get rid of spam.

And I don't see evidence that Yahoo allows that, or of whitelisting only
senders listed in the contacts list.

> You can add client-side spam filtering (Tbird has its
> Bayesian filtering), but typically those are worse than the server-side
> spam filtering (Bayes is a guessing scheme using keyword weighting).
> You can install add-ons to the client that will let you use a better
> Bayes scheme (Tbird doesn't even have a floor function to expire old and
> outdated keywords), but they can also use DNSBLs (DNS blacklists, like
> Spamhaus and SpamCop), DCC (Distributed Checksum Clearinghous) to detect
> bulk e-mails sents to many others also participating in DCC (not all
> bulk e-mails contain the "Precedence: bulk" header, but that's something
> your rules could look for), or other anti-spam schemes. If Bayesian is
> included in the mix of anti-spam measures, it should be the *last*
> method employed.

That kind of thing is doable, but questionable in effectiveness of
cost/benefit. I believe that Thunderbird's capacity for tagging as
Junk/Not Junk is Bayesian, although I don't think it's as robust as
server-based tools such as SpamAssassin. And even then, the scope is
limited, of what is being downloaded from the server's Inbox. It's a
lot faster/easier to do that on the server, and closer to the scene of
the action.

The thing about tools such as the DNSBLs, DCC, SpamAssassin, etc. is
that they're server tools that are used in the SMTP interaction, and
users don't have access to. Depending on how the server is configured,
they may leave tracks in message headers where it is possible to get to
them with client-created rules, but that takes work of sorting through
the headers. And a check of message traffic received in Yahoo shows
that they don't make any of the content available.

>
> Some e-mail clients, and some e-mail services, let you blacklist every
> sender except those in your Contacts. That is, everything gets Junked
> unless in a whitelist (your Contacts). That will generate a lot of
> false positives, like your dentist, doctor, clinic, coworker,
> non-immediate family, old or infrequent friends, and anyone you didn't
> add as a Contact.

NFN Smith <worldoff9908@gmail.com> wrote:

> One of the places that is important to keep an eye on is for message
> traffic that originates from applications that use outsourced mail
> servers, where the mail server isn't part of the sender's domain, but
> sending on behalf of that domain.

Another problem with bulk mail filtering at the server. This doesn't
just check for the presence of the "Precedence: bulk" header. The same,
or nearly the same, e-mail sent to many recipients at the same e-mail
provider (or across many e-mail providers by using DCC) can get flagged
as spam. For example, a site that uses 2FA to send you a code to
complete a login will be from the same sender and have identical content
except for the few digits comprising the login code. Since 2FA codes
are sent out by the thousands, or more, every day from many sources,
they can easily be seen as bulk mail. I often find these got dumped in
the Junk folder by the server-side filtering.

I've seen church groups sending out the same newsletter to their
members, and these hundreds of same-content messages get flagged as bulk
mail which are often handled as though spam. The church admin sending
those newsletters could be added to a whitelist, but that only works if
the whitelist is exercised before server-side spam filtering.

> There is also a question of how you have your client set to synchronize
> against server folders. I believe that the normal handling of an IMAP
> client is that in default settings, the server's Junk folder is not
> synchronized with the local client.

That would defeat the purpose of IMAP which is to show the current state
of server-side folders for those that are subscribed. Else, IMAP would
degenerate into POP if not all subscribed folders were synchronized.
Inbox, Trash, Sent, and Junk are normally subscribed by default when
adding an IMAP account into an IMAP e-mail client. Usually Outbox and
Drafts are local folders in the local e-mail client. You do want the
Junk/Spam folder to be subscribed to check on false positives instead of
having to use a webmail client to check that folder.

> Although there may be historical reasons to use client-defined rules,
> very few people use POP and dial-up connections. And with widespread use
> of SPF, DMARC and DKIM a place where servers are actually pretty good at
> filtering a lot of the junk, and where the remaining effort is best
> applied to user-level tuning, rather than spending time building and
> maintaining hand-crafted tools.

There's SPF. There's DKIM which is better using encryption instead of
IP addresses. There's DMARC which builds on SPF and DKIM and adds
"linkage to the author (“From:”) domain name, published policies for
recipient handling of authentication failures, and reporting from
receivers to senders, to improve and monitor protection of the domain
from fraudulent email." I forget the anti-spam scheme name, but there
is also the sending MTA checking the receiving domain's MX (mail
exchange) record as to which boundary hosts are granted permission to
accept e-mails. See https://en.wikipedia.org/wiki/MX_record. Users
trying to run SMTP servers on their own desktop PCs (often in violation
of the personal-use TOS regarding operating public-facing servers) will
fail the MX lookup unless they operate their own nameserver.

Ever get e-mails with a large block of garbage at the end, like a bunch
of words that look like they may be somewhat correct grammatically, but
make no sense? That's to generate a different hash on every spew of a
spam. The garbage block gets changed on every sent message, or on a
group of sent messages, to thwart using hashes to identify bulk
mailings. This was originally to thwart the use of DCC to report hashes
of user-report spam, but DCC adapted to use fuzzy logic which will
eliminate the inclusion of those garbage word blocks.

The spammers are fighting back as they always adapt to evolving
anti-spam tactics. Seems AI is getting involved in generating spam
e-mails. See:

https://fortune.com/2023/04/20/ai-chatbots-spam-changing-more-sophisticated/

The same AI tactics can be used by both parties in the spam wars.

None of the anti-spam tactics will succeed well with users that dole out
their true e-mail address to anyone that asks for it. I use an aliasing
service. Not a forwarding service since replies to a forwarded e-mail
can divulge your true e-mail address or, at least, from where you sent
it. An aliasing service does forwarding, but it also hides your account
when you reply to the aliased e-mails. Replies go back to the aliasing
service to have headers striped, and then added that make it appear the
reply originated at the aliasing service, not from your own e-mail
account. I used to use Spamgourmet, but its own had an illness that
killed him, his son tried to take over but didn't have the appropriate
skills, would update the software but was mute to reports about problems
with the new software, didn't know how to revert back to the prior
version, and generally the son was deaf to what was happening with the
aliasing service. I tried others, like SpamEx, but settled on AnonAddy.
The free service allows unlimited forwarding, but only 1 reply to let
you see how that works. Requires a subscription ($12/yr) to enable
aliased replies. No matter how well-known or trusted may be someone
that asks for my e-mail, I give them an alias. Only after 6 months, or
more, of them not betraying me (by giving my alias to someone else who
then spams me) do I update my account to reveal my true e-mail address.
Despite that may become trusted (never betrayed on an alias), often I
never give them my true e-mail address, and all they have is my aliased
e-mail address. If I get betrayed, or spam shows up on an alias, I can
kill it without ever touching my true e-mail address.

At one time, spammers used to generate random strings to use as the
username token in an e-mail address at a common domain (hotmail.com,
outlook.com, gmail.com, yahoo.com, etc). They'd spew their turds to all
those random e-mail address hoping some would hit valid usernames at
those domain. Providers struck back by suspending accounts that
exceeded anti-abuse quota, like how many bounces (rejects) got sent back
by receiving MTAs. Say, if an account sent out a hundred bulk e-mails
which resulted in 20 bounces per day, or more, the sender's account got
frozen. This can also happen for mailing lists: recipients leave, or
close their accounts, and the church ends up getting too many bounces
for all those recipients that can no longer be contacted.

Then spammers found another trick: use abbreviated SMTP sessions to cull
usernames at a domain. They start an SMTP session, and only send the
RCPT-TO command which specifies the username of the recipient. If
status returned was OK, that account existed, and the spambot would add
the username to their hit list. They could run through a slew of random
usernames to collect on a few that were found to exist. The providers
retaliated by always sending OK status on the RCPT-TO command. No
matter what username the sending MTA specified, all of them looked like
they existed at the receiving MTA. Not until the DATA command was used
to send a message did the receiving MTA return valid status of delivery
to an existing account. However, spammers don't want to send any
message when they're attempt to cull usernames. That would expose them
to every account they managed to hit, and quickly get them into spam
filters.

The spammers keep trying. The e-mail providers keep adapting. And the
war continues with more battles.

https://www.pcmag.com/news/google-upgrades-gmails-spam-filter-with-new-retvec-system

Too bad Google doesn't apply spam filtering on outbound e-mails from
their service instead of just on inbound e-mails.

Spam filtering gets better. Detection avoidance will follow.

On 2023-12-02 20:16, VanguardLH wrote:
> NFN Smith <worldoff9908@gmail.com> wrote:

....

>> That said, it's also important to periodically check (every couple of
>> weeks or so) the web client to make sure there aren't any false
>> positives (and correct those).
>
> Not required if you use IMAP, Exchange (ActiveSync), or Google Mail API
> in the client to sync to your account. You only need the webmail client
> if you don't subscribe to a folder in your client that is in your
> account.
>
> One reason to use the webmail client is to vote on spam/ham to get their
> filters updated. At one time, Gmail would update their filter (with
> lots of similar voting by MANY users) if you moved a message from the
> Inbox folder to the Junk folder in your client, but only when using
> IMAP. The move in the client effected a move on the server, and Gmail
> would record your spam vote. That was many years ago, so I don't know
> if they are still doing it.

I am subscribed to many mail lists. At one such, received on a gmail
account, a particular chap sent many mails, a fair proportion of them
automated, because he is a developer and sends many patches
automatically. Well, google though he was spammer. No matter how many
of those posts I moved from the junk folder to the inbox (usually by
clicking "not junk"), he was still classified as spammer. This was going
on for years. In the end, I had to use web mail, put him in my address
book, and clicked somewhere to white list those in my address book.

It worked.

....

--
Cheers,
Carlos E.R.