Filter log is enabled and working for email accounts.

I have filter log enabled on usenet accounts too, yet log remains empty,
should it work there too, did it used to work there?

Am 10.11.23 um 09:42 schrieb Andy Burns:
> Filter log is enabled and working for email accounts.
>
> I have filter log enabled on usenet accounts too, yet log remains empty,
> should it work there too, did it used to work there?

Yes and I use it a lot more than with e-mail.

--
Gutta cavat lapidem (Ovid)

On 11/10/2023 3:42 AM, Andy Burns wrote:
> Filter log is enabled and working for email accounts.
>
> I have filter log enabled on usenet accounts too, yet log remains empty, should it work there too, did it used to work there?

Mine says File Not Found.

C:\...\ABCD1234.default\News\servername.com\comp.lang.c.htm not found

Which implies, if there is a log, it might be stored as a .htm file.

Once I enabled the filter log, now there is a "stub" .htm file
with a font declaration in it. It could use Consolas for example,
as a font.

I just removed all the Google Groups posts from comp.lang.c and
the filter log file "comp.lang.c.htm" is 24MB and growing :-)

You can imagine what a digestion problem that might cause when
you go to look at it.

That could be why TB is railed right now. RAM is over 7.6GB. Procmon says it
was writing the .msf (which is now tiny, because a lot of stuff was removed) .
This shows the filter I was running (on a "synced" offline store for CLC).

comp.lang.c.dat

version="9"
logging="yes" <===
name="GoogleGroups"
enabled="yes"
type="20"
action="Delete"
condition="AND (\"message-id\",contains,googlegroups.com)"

The message filter contents don't show in Thunderbird, but I
can open the 24MB comp.lang.c.htm file manually.

The filter log window *eventually* paints, when the .htm is
rendered presumably. This is an example of the ThaiSpam after
removal. The entire run probably took more than ten minutes,
with a lot of that, the re-writing of the .msf Mork header file.

[Picture]

https://i.postimg.cc/9FsJ2myT/filter-log-example.gif

It's not a speed demon. And it does not shrink the sync file either,
it just shrinks the .msf to represent what the top-right pane will show.

Paul

Jörg Lorenz wrote:

> schrieb Andy Burns:
> >> I have filter log enabled on usenet accounts too, yet log remains empty,
>> should it work there too, did it used to work there?
>
> Yes and I use it a lot more than with e-mail.

Thanks, I did think I'd been able to read it in the past, I've cleared
out the various filterlog/junklog files, and will see if they start
working ...

Am 10.11.23 um 11:29 schrieb Paul:
> On 11/10/2023 3:42 AM, Andy Burns wrote:
>> Filter log is enabled and working for email accounts.
>>
>> I have filter log enabled on usenet accounts too, yet log remains empty, should it work there too, did it used to work there?
>
> Mine says File Not Found.

Hä?
My goodness, are you complicated.

--
Gutta cavat lapidem (Ovid)

On 11/10/2023 5:34 AM, Jörg Lorenz wrote:
> Am 10.11.23 um 11:29 schrieb Paul:
>> On 11/10/2023 3:42 AM, Andy Burns wrote:
>>> Filter log is enabled and working for email accounts.
>>>
>>> I have filter log enabled on usenet accounts too, yet log remains empty, should it work there too, did it used to work there?
>>
>> Mine says File Not Found.
>
> Hä?
> My goodness, are you complicated.

It said that *right in the Filter Log window* of Thunderbird.

Obviously, the logic in the Filter Log window needs work.
If you're defining a filter for the first time (which is
what I was doing in the test article, not in my own copy
of TB), then this should happen to you as well.

I work through the moving parts, so others
can review what their moving parts are doing.

Just waving your hands in the air and begging
for mercy, doesn't achieve an objective.

Paul

Paul wrote:

> Mine says File Not Found.
>
> C:\...\ABCD1234.default\News\servername.com\comp.lang.c.htm not found
>
> Which implies, if there is a log, it might be stored as a .htm file.
>
> Once I enabled the filter log, now there is a "stub" .htm file
> with a font declaration in it. It could use Consolas for example,
> as a font.

All the files in my profile (onr per server, rather than one per
subscribed group) are called filterlog.html

But the logs only seem to populate when I manually run filters, not
after retrieving new articles.

Andy Burns wrote:

> the logs only seem to populate when I manually run filters, not
> after retrieving new articles.

I have three usenet accounts in thunderbird, I only have filters defined
for one of them, with most rules ticked to apply when "run manually" and
"after getting mail" (sic)

Still nothing appearing in the log files, unless I do manually run the
filters.

Andy Burns wrote:
> Filter log is enabled and working for email accounts.
>
> I have filter log enabled on usenet accounts too, yet log remains empty,
> should it work there too, did it used to work there?

If the log is empty, it means that your filters haven't identified any
messages. That could mean simply that there hasn't yet been any traffic
to identify, but if there is traffic and no log entries, then it means
that your filters have been done in a way that they're not getting any hits.

The latter can happen if your filters are too complicated and where
you're trying to account for too many conditions in a single filter. In
particular, getting the difference between "match all" and "match any".
If your filter is set to "match all" and you have many things that
you're searching for, the lack of any of the conditions will be enough
to return a negative condition (and corresponding lack of a log entry).

When you're doing filters, the best approach is to start with something
simple, usually a single condition, and then add things incrementally.
In my experience, it's also better to do more simple rules than it is to
try to get everything with a single rule. Remember that part of
filtering is that if you have multiple filters defined, all of the
filters are active every time they run, and a hit on one filter won't
cause successive filters to be ignored, unless the list of tasks
assigned to a filter includes a "stop filter execution" direction.

With this in mind, this is why the filtering UI includes controls for
specifying the order of rules within a filter, because order can be
significant. To me, that indicates the intent of having more simple
conditions and fewer complex conditions.

I've never been fond of the structuring of how filtering works, as I
would really like to have the full capacities of Boolean logic, of being
able to do AND, OR and NOT with nesting, but with some trial and error,
it is possible to get filters to do most of the functionality that
Boolean could do.

Smith

NFN Smith wrote:

> Andy Burns wrote:
>
>> I have filter log enabled on usenet accounts too, yet log remains
>> empty
>
> If the log is empty, it means that your filters haven't identified any
> messages.  That could mean simply that there hasn't yet been any traffic
> to identify, but if there is traffic and no log entries, then it means
> that your filters have been done in a way that they're not getting any
> hits.

Sounds logical ... except that when run manually the filters do produce
a log, and when left to run automatically the filters do run [e.g. I can
tell that posts I don't want to see get marked as read] just they don't
produce any logs.

Andy Burns wrote:

> NFN Smith wrote:
>
>> Andy Burns wrote:
>>
>>> I have filter log enabled on usenet accounts too, yet log remains empty
>>
>> If the log is empty, it means that your filters haven't identified any
>> messages.  That could mean simply that there hasn't yet been any
>> traffic to identify, but if there is traffic and no log entries, then
>> it means that your filters have been done in a way that they're not
>> getting any hits.
>
> Sounds logical ... except that when run manually the filters do produce
> a log, and when left to run automatically the filters do run [e.g. I can
> tell that posts I don't want to see get marked as read] just they don't
> produce any logs.

Sorry, meant to add that I've looked in about:config for anything that
matches "filter" which appears it might disable autorun of filters ...

On 11/10/2023 10:58 AM, Andy Burns wrote:
> Andy Burns wrote:
>
>> NFN Smith wrote:
>>
>>> Andy Burns wrote:
>>>
>>>> I have filter log enabled on usenet accounts too, yet log remains empty
>>>
>>> If the log is empty, it means that your filters haven't identified any messages.  That could mean simply that there hasn't yet been any traffic to identify, but if there is traffic and no log entries, then it means that your filters have been done in a way that they're not getting any hits.
>>
>> Sounds logical ... except that when run manually the filters do produce a log, and when left to run automatically the filters do run [e.g. I can tell that posts I don't want to see get marked as read] just they don't produce any logs.
>
> Sorry, meant to add that I've looked in about:config for anything that matches "filter" which appears it might disable autorun of filters ...
>
>

If you're on Windows, you can try Sysinternals Procmon and select on Thunderbird.exe
and WriteFile and that's when I could see writing in the log file that
USENET News uses. Which was comp.lang.c.htm in my case. As the filter was
defined for comp.lang.c on a server that has every bit of the ThaiSpam
for test.

I popped the profile over to the other machine, for test. 115.3.1

Paul

Paul wrote:

> I could see writing in the log file that USENET News uses. Which was
> comp.lang.c.htm in my case. As the filter was defined for comp.lang.c
> on a server that has every bit of the ThaiSpam for test.

Most of my filters are defined at a server level, with just a couple at
the group level

Doesn't seem much point breaking-out procmon to look for write activity,
when the end result is that no file has been written?

On 11/10/2023 12:21 PM, Andy Burns wrote:
>
> Paul wrote:
>
>> I could see writing in the log file that USENET News uses. Which was
>> comp.lang.c.htm in my case. As the filter was defined for comp.lang.c
>> on a server that has every bit of the ThaiSpam for test.
>
> Most of my filters are defined at a server level, with just a couple at the group level
>
> Doesn't seem much point breaking-out procmon to look for write activity, when the end result is that no file has been written?

You can check for activity, and see what it is up to.

It should re-write a Mork file, after logging everything, and
the writing of the Mork file can take a while. My copy of TB
was using a lot of RAM while it was doing that, good thing
it was a 64-bit TB.

Since the Procmon trace captures everything, you can adjust the Filter
post-trace and pull out what you want of it. There is CreateFile
(which doesn't necessarily create anything), there is ReadFile
and WriteFile. That's the set I use for monitoring file I/O.

Paul

On 2023-11-10 16:40, NFN Smith wrote:
> Andy Burns wrote:

> I've never been fond of the structuring of how filtering works, as I
> would really like to have the full capacities of Boolean logic, of being
> able to do AND, OR and NOT with nesting, but with some trial and error,
> it is possible to get filters to do most of the functionality that
> Boolean could do.

I miss and, or, not, and parenthesis. A lot.

The other day I made a filter to match some mail list mail:

List-ID contains "some string"
and
age is greater than 2 days
then
move to another folder.

I have a bunch of strings on which to match, so this would be wonderful:

((List-ID contains "some string") or (List-ID contains "some string 2")
or (...) ...)
and
age is greater than 2 days
then
move to another folder.

Instead, I had to repeat the first rule a dozen times, one per ID.

Ok, I might do:

age is greater than 2 days
then do nothing and pass to the next filter, otherwise exit filtering.

If list id contains .... several or conditions
then move mail.

Two filters.

But I do not like this method of filtering, I might forget that the
ordering is important 5 years from now.

--
Cheers,
Carlos E.R.