I'm writing a complex program at the moment and I want it to be absolutely bulletproof, so I'm using tools to find all memory leaks, all data races, etc. Mostly I'm using "-fsanitize" with g++.

My program has the main GUI thread, the Listener thread for reading from the network card, and the Sender thread for sending packets out on the network card. So that's three threads.

Sometimes my program shows as many as 8 threads. I don't know where these other 5 threads are being spawned nor what they're doing.

How would you go about figuring this out? I'm using the latest Ubuntu Linux on x86_64.

On 6/11/2023 2:16 PM, Frederick Virchanza Gotham wrote:
>
> I'm writing a complex program at the moment and I want it to be absolutely bulletproof, so I'm using tools to find all memory leaks, all data races, etc. Mostly I'm using "-fsanitize" with g++.
>
> My program has the main GUI thread, the Listener thread for reading from the network card, and the Sender thread for sending packets out on the network card. So that's three threads.
>
> Sometimes my program shows as many as 8 threads. I don't know where these other 5 threads are being spawned nor what they're doing.
>
> How would you go about figuring this out? I'm using the latest Ubuntu Linux on x86_64.

Not sure. Perhaps the sanitizer creates some extra threads? Create your
program in a release build with all sanitize stuff turned off.

The main thread runs your GUI loop, right?

Frederick Virchanza Gotham wrote:
>
> I'm writing a complex program at the moment and I want it to be absolutely bulletproof, so I'm using tools to find all memory leaks, all data races, etc. Mostly I'm using "-fsanitize" with g++.
>
> My program has the main GUI thread, the Listener thread for reading from the network card, and the Sender thread for sending packets out on the network card. So that's three threads.
>
> Sometimes my program shows as many as 8 threads. I don't know where these other 5 threads are being spawned nor what they're doing.
>
> How would you go about figuring this out? I'm using the latest Ubuntu Linux on x86_64.
>
You might want to try running your program under "strace".

Then you could try to correlate strace output for the system calls
creating the threas with log output and other system calls shown by
strace to figure out at what points the mysterious threads are created.

HTH
-Pavel

On Sunday, June 11, 2023 at 10:21:36 PM UTC+1, Chris M. Thomasson wrote:
>>
> > How would you go about figuring this out? I'm using the latest Ubuntu Linux on x86_64.
> Not sure. Perhaps the sanitizer creates some extra threads? Create your
> program in a release build with all sanitize stuff turned off.
>
> The main thread runs your GUI loop, right?

Just now I made a build with "-O3 -DNDEBUG", without all the address sanitizer stuff. It has 7 threads.

The main thread is managed by the wxWidgets library, it's the GUI thread. Two threads are spawned by my own code. I don't know where the other 4 have come from.

Ideally I'd like to be able to run it in a debugger and issue a command something like "find thread starts", and to have it come back with something like:

Thread 1 started in GUI_Dialog_Main.cpp on Line 57
Thread 2 started in common_callers.c on Line 89
Thread 3 started in main.cpp on Line 12
Thread 4 started inside shared library libWhatever.so

Actually just now inside the GDB debugger, I issued the command "info threads" and I got back:

(gdb) info threads
Id Target Id Frame
* 1 Thread 0x7ffff5e1ba80 (LWP 115888) "dynamo" 0x00007ffff691112f in __GI___poll (fds=0x555556432170, nfds=3, timeout=1000) at ../sysdeps/unix/sysv/linux/poll.c:29
2 Thread 0x7ffff57ff6c0 (LWP 115891) "gmain" 0x00007ffff691112f in __GI___poll (fds=0x555555d253a0, nfds=1, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
3 Thread 0x7ffff4ffe6c0 (LWP 115892) "pool-dynamo" syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
4 Thread 0x7fffeffff6c0 (LWP 115893) "gdbus" 0x00007ffff691112f in __GI___poll (fds=0x7fffe800ffa0, nfds=2, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
5 Thread 0x7fffef7fe6c0 (LWP 115894) "dconf worker" 0x00007ffff691112f in __GI___poll (fds=0x555555d3e750, nfds=1, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
6 Thread 0x7fffee86e6c0 (LWP 115895) "dynamo" syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
7 Thread 0x7fffee06d6c0 (LWP 115896) "dynamo" syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38

I tried issuing the command "thread apply all bt" but I'm not sure about what it comes back with -- it seems like some of the threads might be spawned inside libglib.

On 6/11/2023 2:35 PM, Pavel wrote:
> Frederick Virchanza Gotham wrote:
>>
>> I'm writing a complex program at the moment and I want it to be
>> absolutely bulletproof, so I'm using tools to find all memory leaks,
>> all data races, etc. Mostly I'm using "-fsanitize" with g++.
>>
>> My program has the main GUI thread, the Listener thread for reading
>> from the network card, and the Sender thread for sending packets out
>> on the network card. So that's three threads.
>>
>> Sometimes my program shows as many as 8 threads. I don't know where
>> these other 5 threads are being spawned nor what they're doing.
>>
>> How would you go about figuring this out? I'm using the latest Ubuntu
>> Linux on x86_64.
>>
> You might want to try running your program under "strace".
>
> Then you could try to correlate strace output for the system calls
> creating the threas with log output and other system calls shown by
> strace to figure out at what points the mysterious threads are created.

Yeah. Wondering if he is using a third party lib? Sometimes they can
create some threads...

On 11/06/2023 22:16, Frederick Virchanza Gotham wrote:
>
> I'm writing a complex program at the moment and I want it to be absolutely bulletproof, so I'm using tools to find all memory leaks, all data races, etc. Mostly I'm using "-fsanitize" with g++.
>
> My program has the main GUI thread, the Listener thread for reading from the network card, and the Sender thread for sending packets out on the network card. So that's three threads.
>
> Sometimes my program shows as many as 8 threads. I don't know where these other 5 threads are being spawned nor what they're doing.
>
> How would you go about figuring this out? I'm using the latest Ubuntu Linux on x86_64.
>

On a Windows system, I would:

a) run under a debugger and look at stack traces for the extra threads after breaking at some point
after the threads have appeared. This should reveal what the thread was created to achieve, i.e.
looking at the deepest entries before you get to the base OS thread startup routines. Also, simply
what the threads are currently doing might give some clues... (this is too late to reveal exactly
where the threads were created)

b) if curiosity is not satisfied, I'd set a breakpoint at the create thread OS API at the start of
the program, and see where the breakpoint is hit.

c) Umm, for Windows there are logging tools that would trace various OS calls including the
cretaion/termination of threads. Such a tool might be useful if Linux has one. (Perhaps inquire on
a Linux newsgroup.)

I assume there are equivalent debugging procedures for a Linux system. External libraries could be
creating and managing their own internal threads within your application, or maybe even the C++
library can create extra threads (or thread pools) for parallelising data related operations like
sorts or whatever?? [I can't say I've observed that myself, but I'm not too adventurous in that
respect.] Most likely it's something in a library you're using, and most developers aren't going to
worry too much over that, as they choose to concentrate on the correctness of their own code, which
is hard enough!

Mike.

Am 11.06.23 um 23:38 schrieb Frederick Virchanza Gotham:
>
> Actually just now inside the GDB debugger, I issued the command "info threads" and I got back:
>
> (gdb) info threads
> Id Target Id Frame
> * 1 Thread 0x7ffff5e1ba80 (LWP 115888) "dynamo" 0x00007ffff691112f in __GI___poll (fds=0x555556432170, nfds=3, timeout=1000) at ../sysdeps/unix/sysv/linux/poll.c:29
> 2 Thread 0x7ffff57ff6c0 (LWP 115891) "gmain" 0x00007ffff691112f in __GI___poll (fds=0x555555d253a0, nfds=1, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
> 3 Thread 0x7ffff4ffe6c0 (LWP 115892) "pool-dynamo" syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
> 4 Thread 0x7fffeffff6c0 (LWP 115893) "gdbus" 0x00007ffff691112f in __GI___poll (fds=0x7fffe800ffa0, nfds=2, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
> 5 Thread 0x7fffef7fe6c0 (LWP 115894) "dconf worker" 0x00007ffff691112f in __GI___poll (fds=0x555555d3e750, nfds=1, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
> 6 Thread 0x7fffee86e6c0 (LWP 115895) "dynamo" syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
> 7 Thread 0x7fffee06d6c0 (LWP 115896) "dynamo" syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
>
> I tried issuing the command "thread apply all bt" but I'm not sure about what it comes back with -- it seems like some of the threads might be spawned inside libglib.

I suspect that the "extra" threads are spawned by wxWidgets or GTK in
order to wait for specific events. It is not uncommon to do that in
order to integrate an event source into the main event loop. "gdbus",
e.g., is a library for interaction with the DBus process
intercommunication system.

Christian

Frederick Virchanza Gotham <cauldwell.thomas@gmail.com> writes:
>
>I'm writing a complex program at the moment and I want it to be absolutely bulletproof, so I'm using tools to find all memory leaks, all data races, etc. Mostly I'm using "-fsanitize" with g++.
>
>My program has the main GUI thread, the Listener thread for reading from the network card, and the Sender thread for sending packets out on the network card. So that's three threads.
>
>Sometimes my program shows as many as 8 threads. I don't know where these other 5 threads are
being spawned nor what they're doing?

What do you mean by "shows"? The ps(1) or top(1) commands or some internal
logic in your application?

>
>How would you go about figuring this out? I'm using the latest Ubuntu Linux on x86_64.

run the program under gdb and

(gdb) thread apply all bt

On Mon, 12 Jun 2023 14:55:36 GMT
scott@slp53.sl.home (Scott Lurndal) wrote:
>Frederick Virchanza Gotham <cauldwell.thomas@gmail.com> writes:
>>
>>I'm writing a complex program at the moment and I want it to be absolutely
>bulletproof, so I'm using tools to find all memory leaks, all data races, etc.
>Mostly I'm using "-fsanitize" with g++.
>>
>>My program has the main GUI thread, the Listener thread for reading from the
>network card, and the Sender thread for sending packets out on the network
>card. So that's three threads.
>>
>>Sometimes my program shows as many as 8 threads. I don't know where these
>other 5 threads are
>being spawned nor what they're doing?
>
>What do you mean by "shows"? The ps(1) or top(1) commands or some internal
>logic in your application?
>
>>
>>How would you go about figuring this out? I'm using the latest Ubuntu Linux
>on x86_64.
>
>run the program under gdb and
>
>(gdb) thread apply all bt

Not come across that, useful.