I use a program that connects to a server online. Any changes that
occur on the server show up in the program as one or more lines of text.
However, the program lacks something like an audible alert. Is it
possible to highlight some area of my screen (say within the program)
and when changes occur, another program (or script?) produces a short
beep to notify me of changes? I know in Firefox, I used to had an add
on that would monitor changes on a website and then alert me. I could
select a specific area of the website for changes, or the entire
website. It would be nice to do the same thing within the text box of
the monitoring program I am using. Thanks in advance.

On 12/31/2023 3:02 PM, Jake M wrote:
> I use a program that connects to a server online. 
> Any changes that occur on the server show up in the program
> as one or more lines of text.  However, the program lacks
> something like an audible alert.  Is it possible to highlight
> some area of my screen (say within the program) and when
> changes occur, another program (or script?) produces a short beep
> to notify me of changes?  I know in Firefox, I used to had
> an add on that would monitor changes on a website and then
> alert me.  I could select a specific area of the website for
> changes, or the entire website. It would be nice to do the same
> thing within the text box of the monitoring program I am using.
>
>   Thanks in advance.

Normally, you would intercept information using the protocol
these things are already using.

However, if you want to do it at the screen level,
then consider there exists a screen capture program
already. In fact a few programs (FFMPEG using X11Grab).

To not miss a thing, you need to record the screen, synchronous
to VSYNC in a sense, so that the rate the screen updates at,
is the rate the program records at. Most screen recorders tend
to be async to the screen. And when you are async to the screen,
there can be "sampling jitter".

SimpleScreenRecorder # apt search simplescreenrecorder

https://www.maartenbaert.be/simplescreenrecorder/

To give an example, say I start recording the screen at 240Hz.
Your first reaction would be "that's not possible", and you'd be
correct. But what the software does, is produce frames like
this:

A A A A B B B B C C C C # Assume screen refresh is 60FPS, sampling is 240

Each frame that actually came from the screen, would be recorded
four times.

But because the schemes are asynchronous, at 240Hz it looks like
this.

A A A B B B B B

The precise instant in time a sample was taken, is only done
approximately by the software. Any process with a higher priority
might upset the sampling. For example, on some OSes, the usage
of "compositing" causes some of the jitter. Well, every OS
does some amount of compositing, in the year 2024.

A person looking at the individual video frames, would know that

A B

was what was truly sent. The collections of identical frames have
"the same checksum", and that's how we hope to tell them apart.
We see three frames with one checksum value, then five frames with
a different checksum value. You can use checksums on computer screens
that do not have natural content (like a Youtube video playing while
you are recording, which I've done).

OK, to experiment, set up a digital clock on the screen, where the time updates
once a second, and nothing on the page changes except the clock. Then
you should get frames, where the checksum differs only every 60 frame times.

cd ~/Downloads
mkdir framestore
cd framestore

ffmpeg -framerate 60 -f x11grab -i desktop -f image2 -q:v 1 -c:v mjpeg a%06d.jpg

Let that run for say, one minute (3600 frames).

Then

md5sum *.jpg # 3600 lines of output

and look at the pattern of checksums. I hope to see groups of sixty identical frames.

SimpleScreenRecorder does a similar thing to FFMPEG, except it has
a sampler which works a bit better than FFMPEG has in the past.
I don't know if it is screen synchronous though.

FFMPEG has "x11grab" for Linux and "gdigrab" for Windows. And the FFMPEG
people would write grabbers as the need arose, such as a Wayland grabber
when the time comes.

Anyway, that's just the basics of an idea. With FFMPEG you can narrow
down the capture area, and so can SimpleScreenRecorder. You should be able
to select a rectangular section, reducing the size of images collected.
A good capture program, offers mouse access to creating the rectangle
of recording.

Our idea can use a lot of CPU (to compress the content a bit) plus
use a lot of disk space in the long run. If, say, the capture program
itself had a difference detector, that would make the result still
CPU intensive, but use no disk space to speak of.

If you couple ffmpeg output to ffprobe input, you can get scene analysis
in terms of byte counts. A change to the screen, requires a tiny bit more
video bandwidth to capture the change.

-------?\-------- # Blip in bandwidth causes by a scene change

So it may be possible to take a datastream from ffprobe and "threshold it"
to detect that blip.

We're still CPU intensive though, and that costs money on the monthly
power bill. Even if we pass some of the tasks to the video card (CUDA say),
it still costs money.

*******

To capture the protocol coming into the computer, there is Wireshark.
For a "foreign" information stream, it'll be protected with
Secure Socket Layer (SSL/TLS) to prevent snooping. It's only when the
stream is decoded, we can read it. This takes some of the fun out of it,
as an easy means to "fork a stream". Twenty five years ago, this would
have been easy.

*******

It's better if a program is custom-made for recording the information
coming back. Instead of using a browser interface to the server,
if the protocol or function has a name, you'd check whether a text
recorder of the information was available. This reduces CPU overhead
to a tiny amount. Maybe it only costs 10 watts of electricity
over and above the idle power.

Normally, a status recorder, would keep a logfile somewhere. That's
what you really want.

I think you would get a better answer, if you included some keywords
as to what the status method is. SNMP or something. The people here
would much rather give you the "exact tool", than do some crazy stuff
like the above :-)

Paul

Paul wrote:

>SimpleScreenRecorder

Now, since this is the Ubuntu NG one may assume that Jake M actually
uses Ubuntu, which uses Wayland. SimpleScreenRecorder does not work with
Wayland, I tried it just a couple of weeks ago.

-jw-

--

And now for something completely different...

On 1/1/2024 5:37 AM, Joerg Walther wrote:
> Paul wrote:
>
>> SimpleScreenRecorder
>
> Now, since this is the Ubuntu NG one may assume that Jake M actually
> uses Ubuntu, which uses Wayland. SimpleScreenRecorder does not work with
> Wayland, I tried it just a couple of weeks ago.
>
> -jw-
>

But you have login options.

Login: Bullwinkle <=== click to get DE options on right side
Password:
Ubuntu \___ This option is vanilla Xorg and works
Ubuntu on Wayland / I tried a bunch of stuff, does not work. 22.04.3 NVidia

This is the first option.

[Picture]

https://i.postimg.cc/fLhWCvh3/simplescreenrecorder.gif

But really, it's going poorly so far.
We can't have X11 forever, and the FFMPEG solution so far is, well, forget it.
Why can't the Weasel At Wayland fix this :-)
Oh yeah, I forget, move fast and break things. Gotcha.

If XWayland can't do this, what good is it as a "workaround".
I managed to avoid the error 5 with my solution, but all
the screen captures are black. Which might as well be error 5 for
all the good it does.

These two do not work, black screen recorded.

ffmpeg -framerate 60 -f x11grab -i :1 output.mp4 # Ubuntu in Wayland black screen
ffmpeg -framerate 60 -f x11grab -i :1 -f image2 -c:v mjpeg a%06d.jpg # Ubuntu in Wayland black screen

Paul

On 1/1/24 11:53 AM, Paul wrote:
> On 1/1/2024 5:37 AM, Joerg Walther wrote:
>> Paul wrote:
>>
>>> SimpleScreenRecorder
>>
>> Now, since this is the Ubuntu NG one may assume that Jake M actually
>> uses Ubuntu, which uses Wayland. SimpleScreenRecorder does not work with
>> Wayland, I tried it just a couple of weeks ago.
>>
>> -jw-
>>
>
> But you have login options.
>
> Login: Bullwinkle <=== click to get DE options on right side
> Password:
> Ubuntu \___ This option is vanilla Xorg and works
> Ubuntu on Wayland / I tried a bunch of stuff, does not work. 22.04.3 NVidia
>
> This is the first option.
>
> [Picture]
>
> https://i.postimg.cc/fLhWCvh3/simplescreenrecorder.gif
>
> But really, it's going poorly so far.
> We can't have X11 forever, and the FFMPEG solution so far is, well, forget it.
> Why can't the Weasel At Wayland fix this :-)
> Oh yeah, I forget, move fast and break things. Gotcha.
>
> If XWayland can't do this, what good is it as a "workaround".
> I managed to avoid the error 5 with my solution, but all
> the screen captures are black. Which might as well be error 5 for
> all the good it does.
>
> These two do not work, black screen recorded.
>
> ffmpeg -framerate 60 -f x11grab -i :1 output.mp4 # Ubuntu in Wayland black screen
> ffmpeg -framerate 60 -f x11grab -i :1 -f image2 -c:v mjpeg a%06d.jpg # Ubuntu in Wayland black screen
>
> Paul

Sorry for the delay. Although using 22.04, I did keep X11 so
SimpleScreenRecorder does work.

I didn't think it would be this complicated. I view a text box that
updates with lines of text as updates occur. Instead of me having to
watch the small updated text all of the time, I was just looking for an
audio alert whenever a new text line showed up. The program that
displays the updates, unfortunately, does not have the capability for an
audio alert as well.

I think I mentioned that I used to have a Firefox add on that was called
"watch for changes" or at least something like that. I could place a
box around certain text on a website, save the selection and website and
the add on would audibly notify me when the website would change for the
area selected. I didn't have to watch it all the time. In fact, I
actually could have the triggered updated feeds sent to a website
instead of the program I'm using, and then use a "watch for changes" or
similar through Firefox like I used to. However, I don't know if I can
create a simple, free website without registering (any suggestions
welcome). If I could, then the issue here would be solved.