About 15 - 20 years ago, I started writing a desktop PC program with a graphical user interface to analyse Ethernet networks. Over the years I've added functionality here and there in dribs and drabs, and so now today I have about 11 thousand lines of code in about 80 source and header files.

In the past few weeks though I've become serious about releasing a new version of my software with many new features. But the thing is though, I want it to be rock solid. I'm going to torture test it on a network with dozens of hosts and leave it running for days on end.

But first I need to fix something. When building my program for Linux, it links with the GTK3 port of wxWidgets, which in turn links with libpango and libfontconfig. When my program ends, AddressSanitiser reports memory leaks for allocations that took place inside libfontconfig. I have identified three such leaks. The first one is actually the fault of libpango, and there's a solution to it in an issue raised on GitLab:

https://gitlab.gnome.org/GNOME/pango/-/issues/748

I haven't figured out the other two yet, and so in order to troubleshoot, I want to put together a little subsystem for managing deallocations. libpango and libfontconfig are both written in C, and so inside these C libraries, I would put C code like the following:

#include <stdlib.h> // malloc, free

extern void ReleaseAtExit(void (*const funcptr)(void*), void *const handle);

void Some_Library_Function(void)
{
void *p = malloc(64);
ReleaseAtExit(free, p);
}

And then inside my main program, I would put:

#include <cassert> // assert
#include <cstdlib> // atexit
#include <vector> // vector
#include <utility> // pair
#include <mutex> // mutex, lock_guard

namespace {

std::vector< std::pair<void (*)(void*), void*> > to_be_released;

void Releaser(void)
{
for ( auto &e : to_be_released )
{
assert( nullptr != e.first );
assert( nullptr != e.second );
e.first(e.second);
}
}
}

extern "C" void ReleaseAtExit(void (*const funcptr)(void*), void *const handle)
{
static int const retval_from_atexit = std::atexit(Releaser);

static std::mutex mtx;
std::lock_guard mylock(mtx);
to_be_released.emplace_back(funcptr,handle);
}

So using this little subsystem, I can then for example edit C source files inside libfontconfig, and where I see it allocate a new list, I can do:

FcValueListPtr p = FcValueListCreate();
ReleaseAtExit( FcValueListDestroy, p );

So then I can run my program and see if it 'fixes' the memory leak. Of course this isn't a real solution, because if it's an accumulative memory leak then it will just get worse and worse until the program ends, but it will at least let me know what tree to bark up.

Anyone got any pointers for me here?

On 5/28/2023 2:49 PM, Frederick Virchanza Gotham wrote:

>
> #include <stdlib.h> // malloc, free
>
> extern void ReleaseAtExit(void (*const funcptr)(void*), void *const handle);
>
> void Some_Library_Function(void)
> {
> void *p = malloc(64);
> ReleaseAtExit(free, p);
> }
>
> And then inside my main program, I would put:
>
> #include <cassert> // assert
> #include <cstdlib> // atexit
> #include <vector> // vector
> #include <utility> // pair
> #include <mutex> // mutex, lock_guard
>
> namespace {
>
> std::vector< std::pair<void (*)(void*), void*> > to_be_released;
>
> void Releaser(void)
> {
> for ( auto &e : to_be_released )
> {
> assert( nullptr != e.first );
> assert( nullptr != e.second );
> e.first(e.second);
> }
> }
> }
>
> extern "C" void ReleaseAtExit(void (*const funcptr)(void*), void *const handle)
> {
> static int const retval_from_atexit = std::atexit(Releaser);
>
> static std::mutex mtx;
> std::lock_guard mylock(mtx);
> to_be_released.emplace_back(funcptr,handle);
> }
>
> So using this little subsystem, I can then for example edit C source files inside libfontconfig, and where I see it allocate a new list, I can do:
>
> FcValueListPtr p = FcValueListCreate();
> ReleaseAtExit( FcValueListDestroy, p );
>
> So then I can run my program and see if it 'fixes' the memory leak. Of course this isn't a real solution, because if it's an accumulative memory leak then it will just get worse and worse until the program ends, but it will at least let me know what tree to bark up.
>
> Anyone got any pointers for me here?

First of all, your Releaser() function won't work as you think.

Your Releaser function will attempt to free the memory, even if
its already been freed, as you are NOT storing a NULL pointer in
ReleaseAtExit, instead it will attempt to free the pointer that was
malloc'ed even if it was already freed elsewhere in the code.

Second, when malloc'ed memory is released, the pointer doesn't get
nulled out, but this is mooted by the first issue above.

You're better off running under valgrind to find the memory leak.

29.05.2023 00:49 Frederick Virchanza Gotham kirjutas:
>
> But first I need to fix something. When building my program for Linux, it links with the GTK3 port of wxWidgets, which in turn links with libpango and libfontconfig. When my program ends, AddressSanitiser reports memory leaks for allocations that took place inside libfontconfig. I have identified three such leaks.

The first thing you should be concerned about is if these leaks are
one-time or cumulative. There is not much point to free one-time
allocations at the end of the program because the OS will do the same,
only faster and more reliably (unless your program needs to run without
proper OS, which I doubt).

For finding out if the leak is cumulative, you can just run the program
in a tight loop and monitor the process size over time. If the program
is massively multithreaded, one can expect some increase over time
because of fragmentation, but it should stabilize asymptotically.

If the leaks are cumulative, then these should be fixed. Note this means
fixing, not developing too-clever schemas to work around them. If you
discover a cumulative leak in fontconfig, the community will be all
thankful if you identify it and submit patches to fix it.

On Monday, May 29, 2023 at 1:41:26 AM UTC+1, red floyd wrote:
>
> First of all, your Releaser() function won't work as you think.
>
> Your Releaser function will attempt to free the memory, even if
> its already been freed, as you are NOT storing a NULL pointer in
> ReleaseAtExit, instead it will attempt to free the pointer that was
> malloc'ed even if it was already freed elsewhere in the code.

Yeah that can happen, but the AddressSanitiser will also be able to tell me about 'double freed' memory. If I can manipulate the C code so that it:
(a) No longer has a memory leak
(b) Doesn't double-deallocate any resources
then I will at least have a better understanding of the leak. Then I can try fix it.

> You're better off running under valgrind to find the memory leak.

I already have the exact line numbers from AddressSanitiser. I've had got a quick fix though because I don't know exactly when the memory in question should be deallocated.

On Monday, May 29, 2023 at 6:05:57 AM UTC+1, Paavo Helde wrote:
> There is not much point to free one-time
> allocations at the end of the program because the OS will do the same,
> only faster and more reliably (unless your program needs to run without
> proper OS, which I doubt).

For the release build of a program, I just let the OS clean things up.

For the debug build though, I keep track of the one-time allocations so
that I can de-allocate them all at the end of the program -- in order to get
a less-cluttered report from AddressSanitiser.

> For finding out if the leak is cumulative, you can just run the program
> in a tight loop and monitor the process size over time. If the program
> is massively multithreaded, one can expect some increase over time
> because of fragmentation, but it should stabilize asymptotically.
>
> If the leaks are cumulative, then these should be fixed. Note this means
> fixing, not developing too-clever schemas to work around them. If you
> discover a cumulative leak in fontconfig, the community will be all
> thankful if you identify it and submit patches to fix it.

You'd be surprised. I raised an issue on the GitLab website belonging to Gnome (who write libpango), and I copy-pasted the report from AddressSanitiser. They deleted my issue and blocked my account. I've emailed them to try rectify things but nothing back yet.

On 29/05/2023 06:03, Paavo Helde wrote:
> 29.05.2023 00:49 Frederick Virchanza Gotham kirjutas:
>>
>> But first I need to fix something. When building my program for Linux,
>> it links with the GTK3 port of wxWidgets, which in turn links with
>> libpango and libfontconfig. When my program ends, AddressSanitiser
>> reports memory leaks for allocations that took place inside
>> libfontconfig. I have identified three such leaks.
>
> The first thing you should be concerned about is if these leaks are
> one-time or cumulative. There is not much point to free one-time
> allocations at the end of the program because the OS will do the same,
> only faster and more reliably (unless your program needs to run without
> proper OS, which I doubt).
>
> For finding out if the leak is cumulative, you can just run the program
> in a tight loop and monitor the process size over time. If the program
> is massively multithreaded, one can expect some increase over time
> because of fragmentation, but it should stabilize asymptotically.
>
> If the leaks are cumulative, then these should be fixed. Note this means
> fixing, not developing too-clever schemas to work around them. If you
> discover a cumulative leak in fontconfig, the community will be all
> thankful if you identify it and submit patches to fix it.
>
>
>

I agree 110%. At least.

It doesn't matter if your program leaks a page or two thousand if it
only does it once. It'll keep running forever, which is what you need.

Checking the ASAN reports on a short run and a long run should give you
a clue on this.

I don't know libfontconfig, but I wouldn't be surprised if something is
loading some data for a font it never frees. But only once.

Andy

Frederick Virchanza Gotham <cauldwell.thomas@gmail.com> writes:

>But first I need to fix something. When building my program for Linux, it l=
>inks with the GTK3 port of wxWidgets, which in turn links with libpango and=
> libfontconfig. When my program ends, AddressSanitiser reports memory leaks=
> for allocations that took place inside libfontconfig. I have identified th=
>ree such leaks. The first one is actually the fault of libpango, and there'=
>s a solution to it in an issue raised on GitLab:

Address sanitizer tells you that _at exit_, there was allocated memory
that hadn't been released. That's not necessarily a leak, just code
that never deallocates data with a process lifetime. Process exit
will deallocate everything anyway.

If the real memory use grows constantly during the application
runtime, then you have a leak.

Yesterday I wrote:
> When my program ends, AddressSanitiser reports memory leaks
> for allocations that took place inside libfontconfig

I've created a patch for libpango that eradicates all of the memory leaks reported by AddressSanitizer:

https://github.com/healytpk/libpango/commit/629b67918d7d2de6060e3bfc6246c3d6e76c15a6

This isn't intended as a final solution, but rather just as a stepping stone toward solving the problem properly.

30.05.2023 02:04 Frederick Virchanza Gotham kirjutas:
> Yesterday I wrote:
>> When my program ends, AddressSanitiser reports memory leaks
>> for allocations that took place inside libfontconfig
>
> I've created a patch for libpango that eradicates all of the memory leaks reported by AddressSanitizer:
>
> https://github.com/healytpk/libpango/commit/629b67918d7d2de6060e3bfc6246c3d6e76c15a6
>
> This isn't intended as a final solution, but rather just as a stepping stone toward solving the problem properly.

This patch meant for helping to find the leaks does not diagnose the
leaks, but instead hides them by releasing the leaked memory just before
the program exit.

Why don't you record the allocation sequence number in your elaborate
leak registry? Then you could print out the leaked fonts at the program
exit to stderr, e.g.

LEAK: font allocation 17 is leaked.

If the program logic is single-threaded or at least somewhat repeatable,
you could then rerun the program, put a breakpoint on the 17-th
allocation, figure out why it gets leaked, and fix it properly.

On Tuesday, May 30, 2023 at 6:48:22 AM UTC+1, Paavo Helde wrote:
>
> This patch meant for helping to find the leaks does not diagnose the
> leaks, but instead hides them by releasing the leaked memory just before
> the program exit.

But at least I have a correct, finite list of the leaks. If I hadn't coded the patch properly, then AddressSanitizer might have complained about me 'double-freeing' memory. Now that I have the patch working and everything gets deallocated, I have a better understanding of the entire situation.

> Why don't you record the allocation sequence number in your elaborate
> leak registry? Then you could print out the leaked fonts at the program
> exit to stderr, e.g.
>
> LEAK: font allocation 17 is leaked.

Yeah I can do that. Also I can keep a realtime count of how many leaks there are in the registry, and update the figure on screen twice a second. So then if I run my program for 3 days, I can just glance at the GUI to check whether that number is increasing.

> If the program logic is single-threaded or at least somewhat repeatable,
> you could then rerun the program, put a breakpoint on the 17-th
> allocation, figure out why it gets leaked, and fix it properly.

It's not entirely that simple. I've looked through the functions where the memory gets allocated, but I'm not sure how long the font (or 'font map', or 'font class') is intended to live. I'll do more testing today but so far I think that they're supposed to live until the program ends. To give an example, let's say you have a wxTextBox, and then at runtime you set the font to Algerian, but then later in the program you set the font to Calibri. I don't know if the former font is just supposed to stay loaded until the program ends.