On 02/02/2024 22:13, bart wrote:
[Bitching about "make" snipped]

Try "cake", Zoltan wrote it many decades ago, when we were at
$GOSHWHATAUNIVERSITY, because he thought "make" was too prolix.

Cheers,
Gary B-)

On 02/02/2024 11:13, bart wrote:

> You're the one who needs to first write a pile of garbage within a
makefile in order for you to do:
>
> make prog
>
> Below is the makefile needed to build lua 5.4, which is a project of
only 35 C modules. Simple, isn't it?

> ---------------------------------
> # Makefile for building Lua
> # See ../doc/readme.html for installation and customization instructions.
>
> # == CHANGE THE SETTINGS BELOW TO SUIT YOUR ENVIRONMENT

Now this is an interesting comment. The makefile is set up for gcc. For
another compiler it won't work.

If I try to switch to 'tcc', there are a number of problems. First,
unless you do 'make clean', the .o files lying about (I guess a
consequence of being to do incremental builds), are incompatible.

At this point I discovered a bug in the makefile for Lua (you might say
it's not bug, it's one of the settings that need changing, but I've no
idea how or where):

Although this makefile works with gcc on Windows, it thinks the
executable is called 'lua', not 'lua.exe'. It will produce 'lua.exe'
with gcc, but it checks for the existence of 'lua'.

That is never present, so it always links; it never says 'is up-to-date'.

With tcc however, there's another issue: tcc requires the .exe extension
in the -o option, otherwise it writes the executable as 'lua'. Now, at
last, make sees 'lua' and deems it up-to-date. Unfortunately that won't
run under Windows.

Either not at all, or it will use the lua.exe left over from gcc. I can
bodge this by using '-o $@.exe', producing lua.exe from tcc, but make is
still checking 'lua'.

There are some minor things: tcc doesn't like the -lm option for example.

But what it comes down to is that it seems I need a separate makefile
for each compiler. As supplied, it didn't even work 100% for gcc on Windows.

That means duplicating all that file info.

This is a solution I used before, using this @ file:

------------------------------
-O2 -s -o lua.exe
lua.c lapi.c lcode.c lctype.c ldebug.c ldo.c ldump.c lfunc.c lgc.c
llex.c lmem.c lobject.c lopcodes.c lparser.c lstate.c lstring.c
ltable.c ltm.c lundump.c lvm.c lzio.c lauxlib.c lbaselib.c lcorolib.c
ldblib.c liolib.c lmathlib.c loadlib.c loslib.c lstrlib.c ltablib.c
lutf8lib.c linit.c
------------------------------

If I run it like this:

gcc @luafiles

it produces a 260KB executable. Which is another interesting thing:
using 'make lua' set up for gcc produces a 360KB executable.

But I can also run it like this:

tcc @luafiles

The same file works for both gcc and tcc.

It won't work for mcc unless I split it into two, as that first line of
options doesn't work there. However with mcc I can now just do this:

mcc lua

So two solutions for this project that (1) don't involve a makefile; (2)
work better than the makefile.

It's true that it involved recompiling every module. But tcc still
builds this project in 0.3 seconds.

This project contains 34 C files, or which 33 are needed (not 35 as I
said). That means that using *.c is not possible, unless that extra file
(I believe used when building a shared library) is renamed.

If that is done, then all compilers just need "*.c" plus whatever other
options are needed.

On Fri, 2 Feb 2024 10:47:12 +0100
David Brown <david.brown@hesbynett.no> wrote:

> On 01/02/2024 23:29, bart wrote:
> > On 01/02/2024 21:34, David Brown wrote:
> >> On 01/02/2024 19:34, bart wrote:
> >
> >>> You don't see that the language taking over task (1) of the
> >>> things that makefiles do, and possibly (2) (of the list I posted;
> >>> repeated below), can streamline makefiles to make them shorter,
> >>> simpler, easier to write and to read, and with fewer
> >>> opportunities to get stuff wrong?
> >>>
> >>> That was a rhetorical question. Obviously not.
> >>
> >> I've nothing against shorter or simpler makefiles.  But as far as
> >> I can see, you are just moving the same information from a
> >> makefile into the C files.
> >>
> >> Indeed, you are duplicating things - now your C files have to have
> >> "#pragma module this, #pragma module that" in addition to having
> >> "#include this.h, #include that.h".  With my makefiles, all the
> >> "this" and "that" is found automatically - writing the includes in
> >> the C code is sufficient.
> >
> > I don't think so. Seeing:
> >
> >     #include "file.h"
> >
> > doesn't necessarily mean there is a matching "file.c". It might not
> > exist, or the header might be for some external library, or maybe
> > it does exist but in a different location.
>
> As I said, you are duplicating things.
>
> For my builds, I do not have anywhere that I need to specify "file.c".
>
> >
> > Or maybe some code may use a file "fred.c", which needs to be
> > submitted to the compiler, but for which there is either no header
> > used, or uses a header file with a different name.
> >
> > As I said, C's uses of .h and .c files are chaotic.
>
> My uses of .h and .c files are not chaotic.
>
> Maybe you can't write well-structured C programs. Certainly not
> everyone can. (And /please/ do not give another list of open source
> programs that you don't like. I didn't write them. I can tell you
> how and why /I/ organise my projects and makefiles - I don't speak
> for others.)
>
> >
> > Did you have in mind using gcc's -MM option? For my 'cipher.c'
> > demo, that only gives a set of header names.  Missing are hmac.c
> > and sha2.c.
>
> I use makefiles where gcc's "-M" options are part of the solution -
> not the whole solution.
>
> > If I try it on lua.c, it gives me only 5 header files; the project
> > comprises 33 .c files and 27 .h files.
> >
>
> I don't care. I did not write lua.
>
> But I /have/ integrated lua with one of my projects, long ago. It
> fit into my makefile format without trouble - I added the lua
> directory as a subdirectory of my source directory, and that was all
> that was needed.
>
> >>>
> >>>
> >>>> Perhaps I would find your tools worked for a "Hello, world"
> >>>> project. Maybe they were still okay as it got slightly bigger.
> >>>> Then I'd have something that they could not handle, and I'd
> >>>> reach for make.  What would be the point of using "make" to
> >>>> automate - for example - post-processing of a binary to add a
> >>>> CRC check, but using your tools to handle the build?  It's much
> >>>> easier just to use "make" for the whole thing.
> >>>
> >>>
> >>> Because building one binary is a process should be the job of a
> >>> compiler, not some random external tool that knows nothing of the
> >>> language or compiler.
> >>
> >> No, it is the job of the linker.
> >
> > There is where you're still stuck in the past.
> >
> > I first got rid of a formal 'linker' about 40 years ago. I got rid
> > of the notion of combining independently compiled modules into an
> > executable a decade ago.
>
> No, you built a monolithic tool that /included/ the linker. That's
> fine for niche tools that are not intended to work with anything
> else. Most people work with many tools - that's why we have
> standards, defined file formats, and flexible tools with wide support.
>
> Other people got rid of monolithic tools forty years ago when they
> realised it was a terrible way to organise things.
>

Actually, nowadays monolithic tools are solid majority in programming.
I mean, programming in general, not C/C++/Fortran programming which by
itself is a [sizable] minority.
Even in C++, a majority uses non-monolithic tools well-hidden behind
front end (IDE) that makes them indistinguishable from monolithic.

>
> >
> > But I suspect you don't understand what a 'whole-program compiler'
> > does:
>
> I know exactly what it does. I am entirely without doubt that I know
> the point and advantages of them better than you do - the /real/
> points and advantages, not some pathetic "it means I don't have to
> use that horrible nasty make program" reason.
>
> > * It means that for each binary, all sources are recompiled at the
> > same time to create it
>
> No, it does not.
>
> >
> > * It doesn't mean that an application can only comprise one binary
>
> Correct.
>
> >
> > * It moves the compilation unit granularity from a module to a
> > single EXE or DLL file
>
> No, it does not.
>
> >
> > * Interfaces (in the case of a lower level language), are moved
> > inter- module to inter-program. The boundaries are between one
> > program or library and another, not between modules.
>
> Correct.
>
> >
> > A language which claims to have a module system, but still compiles
> > a module at a time, will probably still have discrete inter-module
> > interfaces, although they may be handled automatically.
> >
>
> Correct.
>
>
> In real-world whole program compilation systems, the focus is on
> inter-module optimisations. Total build times are expected to go
> /up/. Build complexity can be much higher, especially for large
> programs. It is more often used for C++ than C.
>
> The main point of a lot of whole-program compilation is to allow
> cross-module optimisation. It means you can have "access" functions
> hidden away in implementation files so that you avoid global
> variables or inter-dependencies between modules, but now they can be
> inline across modules so that you have no overhead or costs for this.
> It means you can write code that is more structured and modular,
> with different teams handling different parts, and with layers of
> abstractions, but when you pull it all together into one
> whole-program build, the run-time costs and overhead for this all
> disappear. And it means lots of checks and static analysis can be
> done across the whole program.
>
>
> For such programs, each translation unit is still compiled
> separately, but the "object" files contain internal data structures
> and analysis information, rather than generated code. Lots of the
> work is done by this point, with inter-procedural optimisations done
> within the unit. These compilations will be done as needed, in
> parallel, under the control of a build system. Then they are
> combined for the linking and link-time optimisation which fits the
> parts together. Doing this in a scalable way is hard, and the
> subject of a lot of research, as you need to partition it into chunks
> that can be handled in parallel on multiple cpu cores (or even
> distributed amongst servers). Once you have parts of code that are
> ready, they are handed on to backend compilers that do more
> optimisation and generate the object code, and this in turn is linked
> (sometimes incrementally in parts, again aiming at improving parallel
> building and scalability.
>
>
> You go to all this effort because you are building software that is
> used by millions of people, and your build effort is minor compared
> to the total improvements for all users combined. Or you do it
> because you are building speed-critical software. Or you want the
> best static analysis you can get, and want that done across modules.
> Or you are building embedded systems that need to be as efficient as
> possible.
>
> You don't do it because you find "make" ugly.
>
>
> It is also very useful on old-fashioned microcontrollers with
> multiple banks for data ram and code memory, and no good data stack
> access - the compiler can do large-scale lifetime analysis and
> optimise placement and the re-use of the very limited ram.
>
>
> >
> >> /Nobody/ has makefiles forced on them.  People use "make" because
> >> it is convenient, and it works.
> >
> > BUT IT DOESN'T.
>
> IT DOES WORK.
>
> People use it all the time.
>
> > It fails a lot of the time on Windows, but they are too
> > complicated to figure out why.
>
> People use it all the time on Windows.
>
> Even Microsoft ships its own version of make, "nmake.exe", and has
> done for decades.
>
> /You/ can't work it, but you excel at failing to get things working.
> You have a special gift - you just have to look at a computer with
> tools that you didn't write yourself, and it collapses.
>
> >
> >> But I have no interest in changing to something vastly more
> >> limited and which adds nothing at all.
> >
> > That's right; it adds nothing, but it takes a lot away! Like a lot
> > of failure points.
>
> Like pretty much everything I need.
>
>

On 02/02/2024 08:02, David Brown wrote:
> On 01/02/2024 23:55, Michael S wrote:
>> On Thu, 1 Feb 2024 22:38:13 +0100
>> David Brown <david.brown@hesbynett.no> wrote:
>>
>>> On 01/02/2024 21:23, Michael S wrote:
>>>> On Thu, 1 Feb 2024 18:34:08 +0000
>>>
>
>>>>
>>>> You proposal and needs of David Brown are not necessarily
>>>> contradictory.
>>>> All you need to do to satisfy him is to add to your compiler an
>>>> option for export of dependencies in make-compatible format, i.e.
>>>> something very similar to -MD option of gcc.
>>>>
>>>> Then David could write in his makefile:
>>>>
>>>> out/foo.elf : main_foo.c
>>>>     mcc -MD $< -o $@
>>>>
>>>> -include out/foo.d
>>>>
>>>> And then to proceed with automatiion of his pre and post-processing
>>>> needs.
>>>
>>> But then I'd still be using "make", and Bart would not be happy.
>>>
>>> And "gcc -MD" does not need any extra #pragmas, so presumably neither
>>> would an implementation of that feature in bcc (or mcc or whatever).
>>> So Bart's new system would disappear entirely.
>>>
>>>
>>>
>>
>> Bart spares you from managing list(s) of objects in your makefile and
>> from writing arcan helper macros.
>> Yes, I know, you copy&past arcan macros from project to project, but
>> you had to write them n years ago and that surely was not easy.
>>
>
> Google "makefile automatic dependencies", then adapt to suit your own
> needs.  Re-use the same makefile time and again.
>
> Yes, some of the functions I have in my makefiles are a bit hairy, and
> some of the command line options for gcc are a bit complicated.  They
> are done now.
>
> If there had been an easier way than this, which still let me do what I
> need (Bart's system does not), which is popular enough that you can
> easily google for examples, blogs, and tutorials, then I'd have been
> happy to use that at the time.  I won't change to something else unless
> it gives me significant additional benefits.
>
> People smarter and more experienced than Bart have been trying to invent
> better replacements for "make" for many decades.  None have succeeded.
> Some build systems are better in some ways, but nothing has come close
> to covering the wide range of features and uses of make, or gaining hold
> outside a particular niche.  Everyone who has ever made serious use of
> "make" knows it has many flaws, unnecessarily complications, limitations
> and inefficiencies.  Despite that, it is the best we have.
>
> With Bart's limited knowledge and experience,

That's true: only 47 years in computing, and 42 years of evolving,
implementing and running my systems language.

What can I possibly know about compiling sources files of a lower-level
language into binaries?

How many assemblers, compilers, linkers, and interpreters have /you/
written?

> and deeply ingrained
> prejudices and misunderstandings, the best we can hope for is something
> that works well enough for some simple cases of C programs.

With the proposal outlined in my OP, any of MY C programs, if I was to
write or port multi-module projects in that language, could be trivially
built by giving only the name of the compiler, and the name of one module.

>  More
> realistically, it will work for Bart's use alone.

It certainly won't for your stuff, or SL's, or JP's, or TR's, as you
all seem to delight in wheeling out the most complex scenarios you can find.

That is another aspect you might do well to learn how to do: KISS. (Yes
I can be a patronising fuck too.)

> And that, of course, is absolutely fine.  No one is paying Bart to write
> a generic build system, or something of use to anyone else.  He is free
> to write exactly what he wants, in the way he wants, and if ends up with
> a tool that he finds useful himself, that is great.  If he ends up with
> something that at least some other people find useful, that is even
> better, and I wish him luck with his work.
>
> But don't hold your breath waiting for something that will replace make,
> or attract users of any other build system.

Jesus. And you seem to determined to ignore everything I write, or have
a short memory.

I'm not suggesting replacing make, only to reduce its involvement.

Twice I posted a list of 3 things that make takes care of; I'm looking
at replacing just 1 of those things, the I which for me is more critical.

Kaz Kylheku wrote:

> On 2024-02-01, vallor <vallor@cultnix.org> wrote:
>> Oh, and sidenote: I "researched" (asked ChatGPT) about
>> make(1) being POSIX, and it says it isn't. However,
>> make(1) IS part of the SUS.

ChatGPT's answer was wrong. (Which is often the case.)

> "POSIX" and "Single Unix Specification" are one entity.
> It's beeen that way for years now.

Assuming you are being lazy and saying "POSIX" when you really
mean "POSIX.1", that's true (since 2001 to be precise) for the
"base volumes" of SUS. (There is also an XCurses volume in SUS
that isn't in POSIX.1.)

But make was in "POSIX" before that too: in the POSIX.2 shell and
utilities standard, first published in 1992.

--
Geoff Clare <netnews@gclare.org.uk>

On 02/02/2024 09:47, David Brown wrote:
> On 01/02/2024 23:29, bart wrote:

>> As I said, C's uses of .h and .c files are chaotic.
>
> My uses of .h and .c files are not chaotic.

We can't write tools that only work for careful users. Any open-source
project I want to build WILL be chaotic.

We can however write languages where you are forced to be more
disciplined. Mine doesn't have the equivalent of .h files for example.

However this is about C.

>> I first got rid of a formal 'linker' about 40 years ago. I got rid of
>> the notion of combining independently compiled modules into an
>> executable a decade ago.
>
> No, you built a monolithic tool that /included/ the linker.

No, I ELIMINATED the linker.

And in the past, I wrote a program called a Loader, much simpler than a
linker, and very fast (it had to be as I worked with floppies).

>  That's fine
> for niche tools that are not intended to work with anything else.  Most
> people work with many tools - that's why we have standards, defined file
> formats, and flexible tools with wide support.
>
> Other people got rid of monolithic tools forty years ago when they
> realised it was a terrible way to organise things.

>
>>
>> But I suspect you don't understand what a 'whole-program compiler' does:
>>
>
> I know exactly what it does.  I am entirely without doubt that I know
> the point and advantages of them better than you do

You can't create a language devised for whole-program compilation, and
implement a full-stack compiler for it, without learning a lot about the
ins and outs.

So I suspect I know a bit more about it than you do.

Probably you're mixing this up with whole-program optimisation.

- the /real/ points
> and advantages, not some pathetic "it means I don't have to use that
> horrible nasty make program" reason.
>
>> * It means that for each binary, all sources are recompiled at the same
>>    time to create it
>
> No, it does not.

That's not a whole-program compiler then. Not if half the modules were
compiled last week!

>>
>> * It doesn't mean that an application can only comprise one binary
>
> Correct.
>
>>
>> * It moves the compilation unit granularity from a module to a single
>>    EXE or DLL file
>
> No, it does not.

Again, it can't be a whole-program compiler if it can compile modules
independently.

> In real-world whole program compilation systems, the focus is on
> inter-module optimisations.  Total build times are expected to go /up/.
> Build complexity can be much higher, especially for large programs.  It
> is more often used for C++ than C.
>
> The main point of a lot of whole-program compilation is to allow
> cross-module optimisation.  It means you can have "access" functions
> hidden away in implementation files so that you avoid global variables
> or inter-dependencies between modules, but now they can be inline across
> modules so that you have no overhead or costs for this.  It means you
> can write code that is more structured and modular, with different teams
> handling different parts, and with layers of abstractions, but when you
> pull it all together into one whole-program build, the run-time costs
> and overhead for this all disappear.  And it means lots of checks and
> static analysis can be done across the whole program.
>
>
> For such programs, each translation unit is still compiled separately,
> but the "object" files contain internal data structures and analysis
> information, rather than generated code.  Lots of the work is done by
> this point, with inter-procedural optimisations done within the unit.
> These compilations will be done as needed, in parallel, under the
> control of a build system.  Then they are combined for the linking and
> link-time optimisation which fits the parts together.  Doing this in a
> scalable way is hard, and the subject of a lot of research, as you need
> to partition it into chunks that can be handled in parallel on multiple
> cpu cores (or even distributed amongst servers).  Once you have parts of
> code that are ready, they are handed on to backend compilers that do
> more optimisation and generate the object code, and this in turn is
> linked (sometimes incrementally in parts, again aiming at improving
> parallel building and scalability.

You've just described a tremendously complex way to do whole-program
analysis.

There are easier ways. The C transpiler I use takes a project of dozens
of modules in my language, and produces a single C source file which
will form one EXE or one DLL file.

Now any ordinary optimising C compiler has a view of the entire program
and can do wider optimisations (but that view does not span multiple
EXE/DLL files.)

> /You/ can't work it, but you excel at failing to get things working. You
> have a special gift - you just have to look at a computer with tools
> that you didn't write yourself, and it collapses.

Yes, I do. I'm like that kid poking fun at the emperor's new clothes;
I'm just stating what I see. But in one way it is hilarious seeing you
lot defend programs like 'as' to the death.

Why not just admit that it is a POS that you've had to learn to live
with, instead of trying to make out it is somehow superior?

On 02/02/2024 10:13, David Brown wrote:
> On 02/02/2024 01:26, Malcolm McLean wrote:
>> On 01/02/2024 21:34, David Brown wrote:
>>>
>>>> It works for me, and I'm sure could work for others if they didn't
>>>> have makefiles forced down their throats and hardwired into their
>>>> brains.
>>>
>>> /Nobody/ has makefiles forced on them.  People use "make" because it
>>> is convenient, and it works.  If something better comes along, and it
>>> is better enough to overcome the familiarity momentum, people will
>>> use that.
>>>
>> What?
>> You have total control of your programming environment and never have
>> to consider anybody else? For hobby programming you do in a way. Not
>> if you want other people to use your stuff. But can always say that
>> fun of doing things exactly your way outweighs the fun of getting
>> downloads.
>>
>
> Okay, none of the people talking about "make" /here/ had it forced on
> them for the uses they are talking about /here/.
>
> Yes, I have a very large degree of control over my programming
> environment - because I work in a company where employees get to make
> the decisions that they are best qualified to make, and management's job
> is to support them.  One of the important factors I consider is
> interaction with colleagues and customers, for which "make" works well.
>
> And while people may be required to use make, or particular compilers,
> or OS's, no one is forced to /like/ a tool or find it useful.  I believe
> that when people here say they like make, or find it works well for
> them, or that it can handle lots of different needs, or that they know
> of nothing better for their requirements, they are being honest about
> that.  If they didn't like it, they would say.
>
> The only person here whom we can be absolutely sure does /not/ have
> "make" forced upon them for their development, is Bart.  And he is the
> one who complains about it.
> My job is to write the algorithms. Not to set uo the build system.
Someone else was given that job and set up the system I described
recently, with git and conan and Azure. He didn't do a bad job at all
and we can get a bug fix out to customers within hours on request. My
input is just to moan when occasionally things go wrong. Had I done it
I'm sure it would have been a lot worse, because basically my skills are
in algorithm development, not setting things up like that.

It's makefile free, thank goodness. My main peeve is boost. When things
go wrong, it always seems to be the boost. I've refused to include it in
my library, despite requests. Whilst it would be nice to have the
threads, I just think it would be a perpetual source of build failures
and grief. That's from experience of boost in other projects. I might
ultimately have to give in. I don't have total control, at the end of
the day, it's not my personal code, it's company code.

Because I'm constantly shifting between platforms, make isn't veey
useful to me. And because mainly I do algorithms, the guts of it are
simple source files with no dependencies other than the standard
libraries. So you don't need elaborates systems for pulling things in.
It's just submit a list of sources to the compiler. And you don't need
make to do that. So I don't use make much, for reasons other than that I
don't particularly like it. CMake is a sort of front end to make, which
says it all really. But CMake can also spin uo an IDE project file. And
if you are developing rather than just building, that's far more
convenient. So I distribute using CMake in preference to make. But if
someone won't accept CMake, then I'd have no hesitation, and drop down
to make.

--
Check out Basic Algorithms and my other books:
https://www.lulu.com/spotlight/bgy1mm

On Fri, 2 Feb 2024 14:14:31 +0000
bart <bc@freeuk.com> wrote:

> On 02/02/2024 09:47, David Brown wrote:
> > On 01/02/2024 23:29, bart wrote:
>
> >> As I said, C's uses of .h and .c files are chaotic.
> >
> > My uses of .h and .c files are not chaotic.
>
> We can't write tools that only work for careful users. Any
> open-source project I want to build WILL be chaotic.
>
> We can however write languages where you are forced to be more
> disciplined. Mine doesn't have the equivalent of .h files for example.
>
> However this is about C.
>
>
> >> I first got rid of a formal 'linker' about 40 years ago. I got rid
> >> of the notion of combining independently compiled modules into an
> >> executable a decade ago.
> >
> > No, you built a monolithic tool that /included/ the linker.
>
> No, I ELIMINATED the linker.
>
> And in the past, I wrote a program called a Loader, much simpler than
> a linker, and very fast (it had to be as I worked with floppies).
>
> >  That's fine
> > for niche tools that are not intended to work with anything else.
> > Most people work with many tools - that's why we have standards,
> > defined file formats, and flexible tools with wide support.
> >
> > Other people got rid of monolithic tools forty years ago when they
> > realised it was a terrible way to organise things.
>
>
>
> >
> >>
> >> But I suspect you don't understand what a 'whole-program compiler'
> >> does:
> >
> > I know exactly what it does.  I am entirely without doubt that I
> > know the point and advantages of them better than you do
>
> You can't create a language devised for whole-program compilation,
> and implement a full-stack compiler for it, without learning a lot
> about the ins and outs.
>
> So I suspect I know a bit more about it than you do.
>
> Probably you're mixing this up with whole-program optimisation.
>
> - the /real/ points
> > and advantages, not some pathetic "it means I don't have to use
> > that horrible nasty make program" reason.
> >
> >> * It means that for each binary, all sources are recompiled at the
> >> same time to create it
> >
> > No, it does not.
>
> That's not a whole-program compiler then. Not if half the modules
> were compiled last week!
>
> >>
> >> * It doesn't mean that an application can only comprise one binary
> >>
> >
> > Correct.
> >
> >>
> >> * It moves the compilation unit granularity from a module to a
> >> single EXE or DLL file
> >
> > No, it does not.
>
> Again, it can't be a whole-program compiler if it can compile modules
> independently.
>
> > In real-world whole program compilation systems, the focus is on
> > inter-module optimisations.  Total build times are expected to go
> > /up/. Build complexity can be much higher, especially for large
> > programs.  It is more often used for C++ than C.
> >
> > The main point of a lot of whole-program compilation is to allow
> > cross-module optimisation.  It means you can have "access"
> > functions hidden away in implementation files so that you avoid
> > global variables or inter-dependencies between modules, but now
> > they can be inline across modules so that you have no overhead or
> > costs for this.  It means you can write code that is more
> > structured and modular, with different teams handling different
> > parts, and with layers of abstractions, but when you pull it all
> > together into one whole-program build, the run-time costs and
> > overhead for this all disappear.  And it means lots of checks and
> > static analysis can be done across the whole program.
> >
> >
> > For such programs, each translation unit is still compiled
> > separately, but the "object" files contain internal data structures
> > and analysis information, rather than generated code.  Lots of the
> > work is done by this point, with inter-procedural optimisations
> > done within the unit. These compilations will be done as needed, in
> > parallel, under the control of a build system.  Then they are
> > combined for the linking and link-time optimisation which fits the
> > parts together.  Doing this in a scalable way is hard, and the
> > subject of a lot of research, as you need to partition it into
> > chunks that can be handled in parallel on multiple cpu cores (or
> > even distributed amongst servers).  Once you have parts of code
> > that are ready, they are handed on to backend compilers that do
> > more optimisation and generate the object code, and this in turn is
> > linked (sometimes incrementally in parts, again aiming at improving
> > parallel building and scalability.
>
> You've just described a tremendously complex way to do whole-program
> analysis.
>

But it proves that your statement above (it can't be a whole-program
compiler if it can compile modules independently) is false.

> There are easier ways. The C transpiler I use takes a project of
> dozens of modules in my language, and produces a single C source file
> which will form one EXE or one DLL file.
>
> Now any ordinary optimising C compiler has a view of the entire
> program and can do wider optimisations (but that view does not span
> multiple EXE/DLL files.)
>

If the program in question is really big then there is a good chance
that your method will expose internal limits of the back-end compiler.
I think, that's one of the reason (not the only one) why Mozilla didn't
re-write the whole Firefox in Rust. According to my understanding, Rust
does something similar to your approach, except that it outputs LLVM IR
instead of C and there are real concern that LLVM back end would have
troubles with input as big as the whole FF.

> > /You/ can't work it, but you excel at failing to get things
> > working. You have a special gift - you just have to look at a
> > computer with tools that you didn't write yourself, and it
> > collapses.
>
> Yes, I do. I'm like that kid poking fun at the emperor's new clothes;
> I'm just stating what I see. But in one way it is hilarious seeing
> you lot defend programs like 'as' to the death.
>
> Why not just admit that it is a POS that you've had to learn to live
> with, instead of trying to make out it is somehow superior?
>
>

I never run gnu as directly. Running it by means of driver program
(personally I prefer clang for that task, but gcc will do the job as
well) isolates me from all peculiarities.

On 02/02/2024 14:28, Michael S wrote:
> On Fri, 2 Feb 2024 09:02:15 +0100
> David Brown <david.brown@hesbynett.no> wrote:
>>
>> But don't hold your breath waiting for something that will replace
>> make, or attract users of any other build system.
>>
>>
>
> It seems, you already forgot the context of my post that started this
> short sub-thread.
>

That is absolutely possible. It was not intentional, but the number of
posts in recent times has been overwhelming. I apologise if I have
misinterpreted what you wrote.

> BTW, I would imagine that Stu Feldman, if he is still in good health,
> would fine talking with Bart more entertaining that talking with you.

I have no idea who that is, so I'll take your word for it.

> I think, you, English speakers, call it birds of feather.
>

On Fri, 2 Feb 2024 15:49:20 +0100
David Brown <david.brown@hesbynett.no> wrote:

> On 02/02/2024 14:28, Michael S wrote:
> > On Fri, 2 Feb 2024 09:02:15 +0100
> > David Brown <david.brown@hesbynett.no> wrote:
> >>
> >> But don't hold your breath waiting for something that will replace
> >> make, or attract users of any other build system.
> >>
> >>
> >
> > It seems, you already forgot the context of my post that started
> > this short sub-thread.
> >
>
> That is absolutely possible. It was not intentional, but the number
> of posts in recent times has been overwhelming. I apologise if I
> have misinterpreted what you wrote.
>
> > BTW, I would imagine that Stu Feldman, if he is still in good
> > health, would fine talking with Bart more entertaining that talking
> > with you.
>
> I have no idea who that is, so I'll take your word for it.
>

Inventor of make

> > I think, you, English speakers, call it birds of feather.
> >
>

On 02/02/2024 14:47, bart wrote:
> On 02/02/2024 08:02, David Brown wrote:
>> On 01/02/2024 23:55, Michael S wrote:
>>> On Thu, 1 Feb 2024 22:38:13 +0100
>>> David Brown <david.brown@hesbynett.no> wrote:
>>>
>>>> On 01/02/2024 21:23, Michael S wrote:
>>>>> On Thu, 1 Feb 2024 18:34:08 +0000
>>>>
>>
>>>>>
>>>>> You proposal and needs of David Brown are not necessarily
>>>>> contradictory.
>>>>> All you need to do to satisfy him is to add to your compiler an
>>>>> option for export of dependencies in make-compatible format, i.e.
>>>>> something very similar to -MD option of gcc.
>>>>>
>>>>> Then David could write in his makefile:
>>>>>
>>>>> out/foo.elf : main_foo.c
>>>>>     mcc -MD $< -o $@
>>>>>
>>>>> -include out/foo.d
>>>>>
>>>>> And then to proceed with automatiion of his pre and post-processing
>>>>> needs.
>>>>
>>>> But then I'd still be using "make", and Bart would not be happy.
>>>>
>>>> And "gcc -MD" does not need any extra #pragmas, so presumably neither
>>>> would an implementation of that feature in bcc (or mcc or whatever).
>>>> So Bart's new system would disappear entirely.
>>>>
>>>>
>>>>
>>>
>>> Bart spares you from managing list(s) of objects in your makefile and
>>> from writing arcan helper macros.
>>> Yes, I know, you copy&past arcan macros from project to project, but
>>> you had to write them n years ago and that surely was not easy.
>>>
>>
>> Google "makefile automatic dependencies", then adapt to suit your own
>> needs.  Re-use the same makefile time and again.
>>
>> Yes, some of the functions I have in my makefiles are a bit hairy, and
>> some of the command line options for gcc are a bit complicated.  They
>> are done now.
>>
>> If there had been an easier way than this, which still let me do what
>> I need (Bart's system does not), which is popular enough that you can
>> easily google for examples, blogs, and tutorials, then I'd have been
>> happy to use that at the time.  I won't change to something else
>> unless it gives me significant additional benefits.
>>
>> People smarter and more experienced than Bart have been trying to
>> invent better replacements for "make" for many decades.  None have
>> succeeded. Some build systems are better in some ways, but nothing has
>> come close to covering the wide range of features and uses of make, or
>> gaining hold outside a particular niche.  Everyone who has ever made
>> serious use of "make" knows it has many flaws, unnecessarily
>> complications, limitations and inefficiencies.  Despite that, it is
>> the best we have.
>>
>> With Bart's limited knowledge and experience,
>
> That's true: only 47 years in computing, and 42 years of evolving,
> implementing and running my systems language.

Yes. Most of it using your languages, your tools, your programs, and
determinedly refusing to learn or use anything else more than the barest
minimum, and so completely convinced of your own superiority and the
failings of everyone else and all other languages and software that you
are unable to learn things properly or consider anything from a
viewpoint other than your own.

You have experience - but it is limited by the walls you put up around
yourself.

>
> What can I possibly know about compiling sources files of a lower-level
> language into binaries?

You know how /you/ do it, and how /you/ want to do it. You know sod all
about anyone else.

>
> That is another aspect you might do well to learn how to do: KISS. (Yes
> I can be a patronising fuck too.)
>

KISS is great. It's what encourages people to use existing standard
tools like "make" and "C", instead of trying to re-invent their own
personal wheels all the time. /Sometimes/ it is useful to re-invent
something from scratch. Most of the time, it is not.

>
>> And that, of course, is absolutely fine.  No one is paying Bart to
>> write a generic build system, or something of use to anyone else.  He
>> is free to write exactly what he wants, in the way he wants, and if
>> ends up with a tool that he finds useful himself, that is great.  If
>> he ends up with something that at least some other people find useful,
>> that is even better, and I wish him luck with his work.
>>
>> But don't hold your breath waiting for something that will replace
>> make, or attract users of any other build system.
>
> Jesus. And you seem to determined to ignore everything I write, or have
> a short memory.
>
> I'm not suggesting replacing make, only to reduce its involvement.

I didn't say you were trying to replace make, or even thought you were.
I said you were not replacing make. There's a difference.

>
> Twice I posted a list of 3 things that make takes care of; I'm looking
> at replacing just 1 of those things, the I which for me is more critical.
>

And I have repeatedly said that if you are making a tool that is useful
for you, then great - make your tool and use it.

bart <bc@freeuk.com> writes:
>On 02/02/2024 08:02, David Brown wrote:

>> With Bart's limited knowledge and experience,
>
>That's true: only 47 years in computing, and 42 years of evolving,
>implementing and running my systems language.

It's pretty clear that you have very limited knowledge
and experience with unix, make and and pretty much
anything that isn't your soi disant compiler.

>
>What can I possibly know about compiling sources files of a lower-level
>language into binaries?

Very little, it appears, outside of your toy projects.

>
>How many assemblers, compilers, linkers, and interpreters have /you/
>written?

Can't speak for David, but in my case, at least one of each, and
you can add operating systems and hypervisors to that list.

>It certainly won't for your stuff, or SL's, or JP's, or TR's, as you
>all seem to delight in wheeling out the most complex scenarios you can find.

The "stuff" I write is for customers. Any so-called-bart-complexity is based on
customer requirements. The customers are quite happy with the solutions
they get.

>
>That is another aspect you might do well to learn how to do: KISS. (Yes
>I can be a patronising fuck too.)

KISS is a good principle to follow, and while I cannot again speak
for David, it's a principle followed by most programmers I've worked
with. That doesn't mean throwing away perfectly usable tools
(one can easily make KISS-compliant makefiles, for example).

>I'm not suggesting replacing make, only to reduce its involvement.

And to reduce it's involvement, something must replace make. ipso facto.

On 02/02/2024 14:43, Michael S wrote:
> On Fri, 2 Feb 2024 14:14:31 +0000
> bart <bc@freeuk.com> wrote:

>> You've just described a tremendously complex way to do whole-program
>> analysis.
>>
>
> But it proves that your statement above (it can't be a whole-program
> compiler if it can compile modules independently) is false.

Then /every/ compiler can be regarded as a whole-program one, since the
end result, even if the modules were randomly compiled over the last
month, will always be a whole program.

So it comes down to what is meant by a whole-program compiler.

My definition is where you build one program (eg. one EXE or DLL file on
Windows) with ONE invocaton of the compiler, which processes ALL source
and support files from scratch.

The output (from my compiler) is a single file, usually an EXE or DLL,
that may use external shared libraries. Or, rarely, it may generate a
single OBJ file for more exotic requirements, but it will need external
tools. Then it may end up as a component of a larger program.

Or sometimes the output is fixed up in memory and run immediately.

That describes the compiler I use for my systems language.

My C compiler is not a whole-program one. Although you can submit all
modules and it can produce one EXE/DLL file, so that the behaviour can
appear similar, internally they are compiled independently.

I have thought about using real whole-program techniques (so that all
modules share a global symbol table for example, and common headers are
processed only once), but I don't use C enough to make that interesting
to attempt.

>> There are easier ways. The C transpiler I use takes a project of
>> dozens of modules in my language, and produces a single C source file
>> which will form one EXE or one DLL file.
>>
>> Now any ordinary optimising C compiler has a view of the entire
>> program and can do wider optimisations (but that view does not span
>> multiple EXE/DLL files.)
>>
>
> If the program in question is really big then there is a good chance
> that your method will expose internal limits of the back-end compiler.

My personal whole-program projects impose some limitations.

One is the scale of the application being compiled. However they are
designed for use with a fast compiler. That puts an upper limit of about
0.5M lines per project, if you want to keep build time below, say, 1 second.

(Figures pertain to my slowish PC, running an unoptimised compiler, so
are conservative. An optimised compiler might be 40% faster.)

0.5M lines of code means about a 5MB executable, which is a pretty hefty
project. The vast majority of executables and libraries on my PC are
smaller than that.

Another is that whole-program compilation is harder to parallelise (the
above figures are for a single core). But you can of course compile
multiple programs at the same time.

The killer is that most professional compilers are hugely complex: they
are big, and they take considerable machine resources. These are ones
like gcc or any using LLVM.

So to get around that in order to do whole-program stuff, things get
very, very complicated.

I can't help that.

But I can't remember how we got here. The thread subject is the
far-simpler-to-realise topic of discovering the modules for a
non-whole-program C compiler, which seems to give the big boys a lot
more trouble!

On 02/02/2024 14:45, Michael S wrote:
>
> Actually, nowadays monolithic tools are solid majority in programming.
> I mean, programming in general, not C/C++/Fortran programming which by
> itself is a [sizable] minority.
> Even in C++, a majority uses non-monolithic tools well-hidden behind
> front end (IDE) that makes them indistinguishable from monolithic.
>

It can often be helpful to have a single point of interaction - a
front-end that combines tools. But usually these are made of parts.

For many of the microcontrollers I work with, the manufacturer's
standard development toolset is based around Eclipse and gcc. From the
user point of view, it looks a lot like one monolithic IDE that lets you
write your code, compile and link it, and download and debug it on the
microcontroller. Under the hood, it is far from a monolithic
application. Different bits come from many different places. This
means the microcontroller manufacturer is only making the bits that are
specific to /their/ needs - such as special views while debugging, or
"wizards" for configuring chip pins. The Eclipse folk are experts at
making an editor and IDE, the gcc folks are experts at the compiler, the
openocd folks know about jtag debugging, and so on. And to a fair
extent, advanced users can use the bits they want and leave out other
bits. I sometimes use other editors, but might still use the toolchain
provided with the manufacturer's tools. I might swap out the debugger
connection. I might use the IDE for something completely different. I
might install additional features in the IDE. I might use different
toolchains. Manufacturers, when putting things together, might change
where they get their toolchains, or what debugging connectors they use.
It's even been known for them to swap out the base IDE while keeping
most of the rest the same (VS Code has become a popular choice now, and
a few use NetBeans rather than Eclipse).

(Oh, and for those that don't believe "make" and "gcc" work on Windows,
these development tools invariably have "make" and almost invariably use
gcc as their toolchain, all working in almost exactly the same way on
Linux and Windows. The only difference is builds are faster on Linux.)

This is getting the best (or at least, trying to) from all worlds. It
gives people the ease-of-use advantages of monolithic tools without the
key disadvantages of real monolithic tools - half-arse editors,
half-arsed project managers, half-arsed compilers, and poor
extensibility because the suppliers are trying to do far too much
themselves.

I don't think it is common now to have /real/ monolithic development
tools. But it is common to have front-ends aimed at making the
underlying tools easier and more efficient to use, and to provide
all-in-one base packages.

On 02/02/2024 15:14, bart wrote:

> Yes, I do. I'm like that kid poking fun at the emperor's new clothes;
> I'm just stating what I see. But in one way it is hilarious seeing you
> lot defend programs like 'as' to the death.
>

No, /you/ are the emperor in this analogy. Well, you are actually the
kid - except you are the kid with no clothes who /thinks/ he's an emperor.

> Why not just admit that it is a POS that you've had to learn to live
> with, instead of trying to make out it is somehow superior?
>

The whole world is out of step, except Bart.

Has it never occurred to you that when you are in disagreement with
everyone, /you/ might be the one that is wrong? I think you suffer from
the "misunderstood genius" myth. It's surprisingly common amongst
people who have invested heavily in going their own way, against common
knowledge or common practice. It's a sort of psychological defence
mechanism against realising you've been wrong all this time.

On 2024-02-02, bart <bc@freeuk.com> wrote:
> disciplined. Mine doesn't have the equivalent of .h files for example.

My musical instrument has frets for easy intonation, you silly violin
people, in your silly violin newsgroup.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

In article <20240202165335.0000384a@yahoo.com>,
Michael S <already5chosen@yahoo.com> wrote:
....
>> I have no idea who that is, so I'll take your word for it.
>>
>
>Inventor of make

At Bell Labs, in 1976.

Currently, like quite a few ex-Bell Labs people, a big wheel at Google.

--
Marshall: 10/22/51
Jessica: 4/4/79

On 02/02/2024 15:31, David Brown wrote:
> On 02/02/2024 15:14, bart wrote:
>
>> Yes, I do. I'm like that kid poking fun at the emperor's new clothes;
>> I'm just stating what I see. But in one way it is hilarious seeing you
>> lot defend programs like 'as' to the death.
>>
>
> No, /you/ are the emperor in this analogy.  Well, you are actually the
> kid - except you are the kid with no clothes who /thinks/ he's an emperor.
>
>> Why not just admit that it is a POS that you've had to learn to live
>> with, instead of trying to make out it is somehow superior?
>>
>
> The whole world is out of step, except Bart.

Has it ever occurred to YOU that the world is more than Unix and make
and massive compilers like gcc and clang?

> Has it never occurred to you that when you are in disagreement with
> everyone, /you/ might be the one that is wrong?  I think you suffer from
> the "misunderstood genius" myth.  It's surprisingly common amongst
> people who have invested heavily in going their own way, against common
> knowledge or common practice.  It's a sort of psychological defence
> mechanism against realising you've been wrong all this time.

This is a newsgroup about C. That is a language that can be fairly
adequately implemented with a 180KB program, the size of Tiny C. Tiny C
itself can turn C source into binary at about 10MB per second.

So, a toy language, really, and a toy implementation that nevertheless
does the job: in most cases, a user of the resulting program will not be
able to tell how it was compiled.

And yet there is this massive collection of of huge, complex tools built
around a toy language, dwarfing it by 1000:1, that you insist is what
it's really all about, and you want to put down anyone who disagrees.

It's like saying that the only businesses worth having are huge
corporations, or the only form of transport must be a jetliner.

The way 'as' works IS rubbish. It is fascinating how you keep trying to
turn it round and make it about me. There can't possibly be anything
wrong with it, whoever says so must be deluded!

On 2024-02-02, Kenny McCormack <gazelle@shell.xmission.com> wrote:
> In article <20240202165335.0000384a@yahoo.com>,
> Michael S <already5chosen@yahoo.com> wrote:
> ...
>>> I have no idea who that is, so I'll take your word for it.
>>>
>>
>>Inventor of make
>
> At Bell Labs, in 1976.
>
> Currently, like quite a few ex-Bell Labs people, a big wheel at Google.

It's like a cargo cult. If we hire old Unix geezers and prop them up in
chairs, be they dead or alive, magic will happen.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

On 2024-02-02, bart <bc@freeuk.com> wrote:
> Has it ever occurred to YOU that the world is more than Unix and make
> and massive compilers like gcc and clang?

There is more, but from your perspective, it's just more stuff to
shake your fist at and avoid learning about.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

On 02/02/2024 15:18, Scott Lurndal wrote:
> bart <bc@freeuk.com> writes:
>> On 02/02/2024 08:02, David Brown wrote:
>
>>> With Bart's limited knowledge and experience,
>>
>> That's true: only 47 years in computing, and 42 years of evolving,
>> implementing and running my systems language.
>
> It's pretty clear that you have very limited knowledge
> and experience with unix, make and and pretty much
> anything that isn't your soi disant compiler.

Yes. And?

>
>>
>> What can I possibly know about compiling sources files of a lower-level
>> language into binaries?
>
> Very little, it appears, outside of your toy projects.

That's right, I only have experience of the stuff I've done. And?

Most stuff I want to build is on a similar scale, so you'd probably
consider all that as toys too.

You're saying that anyone not using Unix, not building 10Mloc projects,
and not a fan of make, should FOAD?

>
>>
>> How many assemblers, compilers, linkers, and interpreters have /you/
>> written?

OK. How do I know these aren't just toys, or is it only you who is
allowed to judge?

BTW what exactly is a toy project?

> Can't speak for David, but in my case, at least one of each, and
> you can add operating systems and hypervisors to that list.

I don't do OSes. If I did, you probably have a good idea of what mine
would look like!

>> It certainly won't for your stuff, or SL's, or JP's, or TR's, as you
>> all seem to delight in wheeling out the most complex scenarios you can find.
>
> The "stuff" I write is for customers. Any so-called-bart-complexity is based on
> customer requirements. The customers are quite happy with the solutions
> they get.
>
>>
>> That is another aspect you might do well to learn how to do: KISS. (Yes
>> I can be a patronising fuck too.)
>
> KISS is a good principle to follow, and while I cannot again speak
> for David, it's a principle followed by most programmers I've worked
> with. That doesn't mean throwing away perfectly usable tools
> (one can easily make KISS-compliant makefiles, for example).
>
>
>> I'm not suggesting replacing make, only to reduce its involvement.
>
> And to reduce it's involvement, something must replace make. ipso facto.

No. I'm saying make should be less involved in specifying which files to
be submitted to a compiler-toolchain.

Especially for a makefile specifying a production or distribution build,
such as one done at a remote site by someone is not the developer, but
just wants a working binary.

bart <bc@freeuk.com> writes:
>On 02/02/2024 15:18, Scott Lurndal wrote:

>
>You're saying that anyone not using Unix, not building 10Mloc projects,
>and not a fan of make, should FOAD?

No. I'm saying that your dislike of make is personal. If you
don't like it, don't use it. Make your own, nobody is stopping
you. Just don't brag about how "small", "easy" or "nice" it is
until it can handle the same job as make.

bart <bc@freeuk.com> writes:
[...]
> The way 'as' works IS rubbish. It is fascinating how you keep trying
> to turn it round and make it about me. There can't possibly be
> anything wrong with it, whoever says so must be deluded!

"as" works. It's not perfect, but it's good enough. Its job is to
translate assembly code to object code. It does that. There's is
nothing you could do with your preferred user interface (whatever that
might be) that can't be done with the existing one. "as" is rarely
invoked directly, so any slight clumsiness in its well defined user
interface hardly matters. Any changes to its interface could break
existing scripts.

Nobody is claiming that "there can't possibly be anything wrong with
it". You made that up.

Why does the way "as" works offend you?

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Medtronic
void Void(void) { Void(); } /* The recursive call of the void */

On 2024-02-02, bart <bc@freeuk.com> wrote:
> The way 'as' works IS rubbish.

Pretend a developer of "as" (say, the GNU one) is reading this thread.

What is it that is broken?

Do you have a minimal repro test case of your issue?

What is the proposed fix?

> turn it round and make it about me. There can't possibly be anything
> wrong with it, whoever says so must be deluded!

A vast amount of code is being compiled daily, passing through as,
without anyone noticing.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

On 2/2/24 16:18, bart wrote:
>
> My definition is where you build one program (eg. one EXE or DLL file on
> Windows) with ONE invocaton of the compiler, which processes ALL source
> and support files from scratch.
>
And can you disclose the magic trick who let your magic
compiler know exactly the list of "ALL source and support
files" needed for a scratchy build ?

--
+---------------------------------------------------------------------+
| https://tube.interhacker.space/a/tth/video-channels |
+---------------------------------------------------------------------+