//Hello all,//

Hi,

I have started to learn Assembler out of an old book.

It is ancient (2003) but I don't think 8086 programming has changed
much. But the tools have.

I took assembly language in school but dropped out. Now I want another
go at it.

Would someone be my Mentor and answer a ton of questions that would
dwindle out as time went on?

If it's OK, we could do it here. Or netmail

Books are from a bookstore.

Book 1
Assembly Language for the PC 3rd edition, John Socha and Peter Norton.

Book 2
Assembly Language (step by step) Jeff Duntemann. Too Chatty.

I cannot afford a modern book at this time.

Thats what I picked up from the thrift store.

These books are dated around the time I was taking machine code in
school and I find it interesting now.

I hope someone picks me up.

I am running linux and using DOSemu

Also a modern DEBUG and and a modern Vi

I am also a Ham Radio Operator (45 years)

1:229/426.36

Regards,
Alan Beck
VY2XU
[Please reply directly unless the response is relatted to compilers. -John]

On Tuesday, March 21, 2023 at 2:40:22 PM UTC-7, Alan Beck wrote:

> I have started to learn Assembler out of an old book.

(Hopefully enough related to compilers.)

Not so long after I started learning OS/360 Fortran and PL/I, I found
the compiler option for printing out the generated code in sort-of
assembly language. (Not actually assembleable, though.)

About that time, I also had source listings on microfilm of
the OS/360 Fortran library, and some other Fortran callable
assembly programs. And also, the IBM S/370 Principles
of Operation.

With those, and no actual book meant to teach assembly
programming, I figured it out, and started writing my own
programs, though mostly callable from Fortran or PL/I.

Compilers today don't write out the generated code in the same way,
and there aren't so many libraries around to read. And, personally,
8086 is my least favorite to write assembly code in.

Learning C, and thinking about pointers and addresses, is a good start
toward assembly programming.

In any case, I don't think I have any idea how others learn
programming for any language, and especially not for assembly
programming. I used to read IBM reference manuals, cover to cover.
That was mostly high school years. After that, I figured out how to
use them as reference manuals.

Most of my 80x86 assembly programming in the last
20 years is (re)writing this one program:

rdtsc: rdtsc
ret

When called from C, and returning a 64 bit integer, it return the Time
Stamp Counter. (Works for 32 bit code, returning in EDX:EAX. 64 bit is
different.)

C programming works so well, that there are only a few
things you can't do in C, and so need assembly programs.

gah4 <gah4@u.washington.edu> schrieb:
> On Tuesday, March 21, 2023 at 2:40:22 PM UTC-7, Alan Beck wrote:
>
>> I have started to learn Assembler out of an old book.

At the risk of stating the blindingly obvious: There is more
than one assembler language, each computer architecture has its
own (with extensions over time, too). There are also sometimes
different syntax variant, for example AT&T vs. Intel.

[...]

> Compilers today don't write out the generated code in the same way,

Quite the opposite.

The standard on UNIXy systems is to write out assemblly language to
a file, which is then further processed with the actual assembler.
Use "-S" to just generate the foo.s file from foo.c.

Plus, you can disassemble object files and programs with "objdump -d".

> and there aren't so many libraries around to read.

Not ones written in assembler. But it is possible to download
the source code to many libraries, for example glibc, and then
examine what it is compiled to.

Another possibility would be to use http://godbolt.org, which shows
you assembler generated for different systems with differnt options.
To really make sense of it for different architectures you are
not familiar with may be difficult, though). Or build clang/LLVM
yourself and set different options for the architecture.

>And, personally,
> 8086 is my least favorite to write assembly code in.

I like 6502 even less :-)

> Learning C, and thinking about pointers and addresses, is a good start
> toward assembly programming.

That, I agree with. And it helps a lot to also look at the
generated code.

[...]

> C programming works so well, that there are only a few
> things you can't do in C, and so need assembly programs.

Bringing it back a bit towards compilers: Reading assembler code is
a good way to see where they generate inefficient or (more rarely)
incorrect code. In some special cases, writing in assembler can
bring benefits of a factor of 2 or even 4 over compiler-generated
code, usually when SIMD is involved.

Assembler is a bit like Latin: For most people, there is no need
to speak or write, but one should be able to read it.

gah4 <gah4@u.washington.edu> writes:
>Not so long after I started learning OS/360 Fortran and PL/I, I found
>the compiler option for printing out the generated code in sort-of
>assembly language. (Not actually assembleable, though.)
....
>Compilers today don't write out the generated code in the same way,

Unix (Linux) compilers like gcc usually write assembly-language code
if you use the option -S. This code can be assembled, because AFAIK
that's the way these compilers produce object code.

- anton
--
M. Anton Ertl
anton@mips.complang.tuwien.ac.at
http://www.complang.tuwien.ac.at/anton/

On 21/03/2023 22:40, Alan Beck wrote:
> //Hello all,//
>
> Hi,
>
> I have started to learn Assembler out of an old book.
>
> It is ancient (2003) but I don't think 8086 programming has changed
> much. But the tools have.
>
> I took assembly language in school but dropped out. Now I want another
> go at it.
>
> Would someone be my Mentor and answer a ton of questions that would
> dwindle out as time went on?
>
> If it's OK, we could do it here. Or netmail
>
> Books are from a bookstore.

I have both these books on my bookshelf - but it was a /long/ time ago
that I read them.

The big question here is /why/ you are doing this. The 8086 is ancient
history - totally irrelevant for a couple of decades at least. Modern
PC's use x86-64, which is a very different thing. You don't learn
modern Spanish by reading an old Latin grammar book, even though
Spanish is a Latin language.

There are, perhaps, four main reasons for being interested in learning
to write assembly:

1. You need some very niche parts of a program or library to run as
fast as feasible. Then you want to study the details of your target
processor (it won't be an 8086) and its instruction set - typically
focusing on SIMD and caching. Done well, this can lead to an order of
magnitude improvement for very specific tasks - done badly, your
results will be a lot worse than you'd get from a good compiler with
the right options. The "comp.arch" newsgroup is your first point of
call on Usenet for this.

2. You need some very low-level code for things that can't be
expressed in a compiled language, such as task switching in an OS.
Again, you need to focus on the right target. "comp.arch" could be a
good starting point here too.

3. You are working on a compiler. This requires a deep understanding of
the target processor, but you've come to the right newsgroup.

4. You are doing this for fun (the best reason for doing anything) and
learning. You can come a long way with getting familiar with
understanding (but not writing) assembly from looking at the output of
your favourite compilers for your favourite targets and favourite
programming languages on <https://godbolt.org>. Here I would pick an
assembly that is simple and pleasant - 8086 is neither.

I would recommend starting small, such as the AVR microcontroller
family. The instruction set is limited, but fairly consistent and easy
to understand. There is vast amounts of learning resources in the
Arduino community (though most Arduino development is in C or C++), and
you can buy an Arduino kit cheaply. Here you can write assembly code
that actually does something, and the processor ISA is small enough that
you can learn it /all/.

If none of that covers your motivation, then give some more details of
what you want to achieve, and you can probably get better help.
(comp.arch might be better than comp.compilers if you are not interested
in compilers.)

On Tue, 21 Mar 2023 17:40:18 -0400 (EDT), Alan.Beck@darkrealms.ca
(Alan Beck) wrote:

>... I don't think 8086 programming has changed
>much. But the tools have. ...
>Would someone be my Mentor and answer a ton of questions that would
>dwindle out as time went on?

Assembler mostly is off-topic here in comp.compilers, but
comp.lang.asm.x86 will be open to pretty much any question regarding
80x86 assembler.

>[Please reply directly unless the response is related to compilers. -John]

On Wednesday, March 22, 2023 at 12:06:05 PM UTC-7, Thomas Koenig wrote:
> gah4 <ga...@u.washington.edu> schrieb:

(snip)

> > Compilers today don't write out the generated code in the same way,

> Quite the opposite.

> The standard on UNIXy systems is to write out assemblly language to
> a file, which is then further processed with the actual assembler.

Yes, not the same way.

Well, to be sure that this is about compilers, my favorite complaint
is the lost art of small memory compilers. That is, ones that can
run in kilobytes instead of megabytes.

In any case, the OS/360 compilers don't write out assembly code
that an assembler would recognize. It is meant for people.

Some write it out in two columns to save paper.
Labels don't have to agree with what assemblers recognize.
They don't have to be in the same order that they would be for
an assembler, though OS/360 object programs don't have to be
in order, either.

Having not thought about this for a while, I believe they put
in some comments that help human readers, though likely not
what an assembly programmer would say.

Unixy systems might put in some comments, but mostly don't.

gah4 <gah4@u.washington.edu> schrieb:

[...]

> Well, to be sure that this is about compilers, my favorite complaint
> is the lost art of small memory compilers. That is, ones that can
> run in kilobytes instead of megabytes.

On the Internet, there is a project for almost everything - in this
case Tiny C, which still seems to be under active development. Or
at least there are sill commits at https://repo.or.cz/w/tinycc.git .

However, there is a reason why compilers got so big - there is
always a balance to be struck between comilation speed, compiler
size and optimization.

An extreme example: According to "Abstracting Away the Machine", the
very first FORTRAN compiler was so slow that the size of programs
it could compile was limited by the MTBF of the IBM 704 of around
eight hours.

The balance has shifted over time, because of increasing computing
power and available memory that can be applied to compilation,
and because relatively more people use programs than use compilers
than ever before. So, in today's environment, there is little
incentive for writing small compilers.

Also, languages have become bigger, more expressive, more powerful,
more bloated (take your pick), which also increases the size
of compilers.

In article <23-03-003@comp.compilers>,
Thomas Koenig <tkoenig@netcologne.de> wrote:
>Not ones written in assembler. But it is possible to download
>the source code to many libraries, for example glibc, and then
>examine what it is compiled to.

Getting more and more off topic, but I can't let this go.

Glibc is a S W A M P. A newbie who wanders in will drown and never
come out. Even if you are a very experienced C programmer, you don't
want to go there.

Learning assembler in order to understand how machines work is valuable.
Long ago I learned PDP-11 assembler, which is still one of the cleanest
architectures ever designed. I was taking a data structures course at
the same time, and recursion didn't click with me until I saw how it
was done in assembler.

My two cents,

Arnold
--
Aharon (Arnold) Robbins arnold AT skeeve DOT com
[I must admit that when I write C code I still imagine there's a
PDP-11 underneath. -John]

On Friday, March 24, 2023 at 7:10:00 AM UTC-7, Thomas Koenig wrote:

(snip about the lost art of small memory compilers.)

> On the Internet, there is a project for almost everything - in this
> case Tiny C, which still seems to be under active development. Or
> at least there are sill commits at https://repo.or.cz/w/tinycc.git .

> However, there is a reason why compilers got so big - there is
> always a balance to be struck between comilation speed, compiler
> size and optimization.

When I was writing the above, I was looking at the Program Logic
Manual for the OS/360 Fortran G compiler.
(G means it is supposed to run in 128K.)

Fortran G was not written by IBM, but contracted out. And is not
(mostly) in assembler, but in something called POP. That is, it
is interpreted by the POP interpreter, with POPcode written using
assembler macros. Doing that, for one, allows reusing the code
for other machines, though you still need to rewrite the code
generator. But also, at least likely, it decreases the size of
the compiler. POP instructions are optimized for things that
compiler need to do.

I also had the source to that so many years ago, but not the
manual describing it.

> An extreme example: According to "Abstracting Away the Machine", the
> very first FORTRAN compiler was so slow that the size of programs
> it could compile was limited by the MTBF of the IBM 704 of around
> eight hours.

I remember stories about how well its optimizer worked, when
it was believed that they had to compete in code speed with
experienced assembly programmers. I don't remember anything
about how fast it was.

> The balance has shifted over time, because of increasing computing
> power and available memory that can be applied to compilation,
> and because relatively more people use programs than use compilers
> than ever before. So, in today's environment, there is little
> incentive for writing small compilers.

I first thought about this, when reading about the Hercules project
of an IBM S/370 emulator, and couldn't run gcc in 16MB.
(Well, subtract some for the OS, but it still wouldn't fit.)

> Also, languages have become bigger, more expressive, more powerful,
> more bloated (take your pick), which also increases the size
> of compilers.

OK, the IBM PL/I (F) compiler, for what many consider a bloated
language, is designed to run (maybe not well) in 64K.
At the end of every compilation it tells how much memory was
used, how much available, and how much to keep the symbol table
in memory.

> OK, the IBM PL/I (F) compiler, for what many consider a bloated
> language, is designed to run (maybe not well) in 64K.
> At the end of every compilation it tells how much memory was
> used, how much available, and how much to keep the symbol table
> in memory.

It's... 30-some passes, iirc?

De
[Well, phases or overlays but yes, IBM was really good at slicing compilers
into pieces they could overlay. -John]

On Friday, March 24, 2023 at 9:13:05 PM UTC-7, Dennis Boone wrote:

(after I wrote)
> > OK, the IBM PL/I (F) compiler, for what many consider a bloated
> > language, is designed to run (maybe not well) in 64K.
> > At the end of every compilation it tells how much memory was
> > used, how much available, and how much to keep the symbol table
> > in memory.

> It's... 30-some passes, iirc?

> [Well, phases or overlays but yes, IBM was really good at slicing compilers
> into pieces they could overlay. -John]

It is what IBM calls, I believe, dynamic overlay. Each module specifically
requests others to be loaded into memory. If there is enough memory,
they can stay, otherwise they are removed.

And there are a few disk files to be used, when it is actually
a separate pass. The only one I actually know, is if the preprocessor
is used, it writes a disk file with the preprocessor output.

And as noted, if it is really short on memory, the symbol table
goes out to disk.

Fortran H, on the other hand, uses the overlay system generated
by the linkage editor. When running on virtual storage system, it is
usual to run the compiler through the linkage editor to remove
the overlay structure. (One of the few linkers that knows how
to read its own output.) Normally it is about 300K, without
overlay closer to 450K.
[Never heard of dynamic overlays on S/360. -John]

On Saturday, March 25, 2023 at 12:09:30 AM UTC-7, gah4 wrote:

(snip)

> It is what IBM calls, I believe, dynamic overlay. Each module specifically
> requests others to be loaded into memory. If there is enough memory,
> they can stay, otherwise they are removed.

Traditional overlays are generated by the linkage editor, and have
static offsets determined at link time.

PL/I (F) uses OS/360 LINK, LOAD, and DELETE macros to dynamically
load and unload modules. The addresses are not static. IBM says:

"The compiler consists of a number of phases
under the supervision of compiler control
routines. The compiler communicates with
the control program of the operating
system, for input/output and other
services, through the control routines."

All described in:

http://bitsavers.trailing-edge.com/pdf/ibm/360/pli/GY28-6800-5_PL1_F_Program_Logic_Manual_197112.pdf

They do seem to be called phases, but there are both physical and
logical phases, where physical phases are what are more commonly
called phases. There are way more than 100 modules, but I stopped
counting.

(snip)
> [Never heard of dynamic overlays on S/360. -John]

It seems not to actually have a name.

On 3/24/23 10:17 PM, gah4 wrote:

> Fortran G was not written by IBM, but contracted out. And is not
> (mostly) in assembler, but in something called POP. That is, it
> is interpreted by the POP interpreter, with POPcode written using
> assembler macros. Doing that, for one, allows reusing the code
> for other machines, though you still need to rewrite the code
> generator. But also, at least likely, it decreases the size of
> the compiler. POP instructions are optimized for things that
> compiler need to do.

After a look at "open software" I was astonished by the number of
languages and steps involved in writing portable C code. Also updates of
popular programs (Firefox...) are delayed by months on some platforms,
IMO due to missing manpower on the target systems for checks and the
adaptation of "configure". Now I understand why many people prefer
interpreted languages (Java, JavaScript, Python, .NET...) for a
simplification of their software products and spreading.

What's the actual ranking of programming languages? A JetBrains study
does not list any compiled language in their first 7 ranks in 2022. C++
follows on rank 8.

What does that trend mean to a compiler group? Interpreted languages
still need a front-end (parser) and back-end (interpreter), but don't
these tasks differ between languages compiled to hardware or interpretation?

DoDi

On Sat, 25 Mar 2023 13:07:57 +0100, Hans-Peter Diettrich
<DrDiettrich1@netscape.net> wrote:

>After a look at "open software" I was astonished by the number of
>languages and steps involved in writing portable C code. Also updates of
>popular programs (Firefox...) are delayed by months on some platforms,
>IMO due to missing manpower on the target systems for checks and the
>adaptation of "configure". Now I understand why many people prefer
>interpreted languages (Java, JavaScript, Python, .NET...) for a
>simplification of their software products and spreading.

Actually Python is the /only/ one of those that normally is
interpreted. And the interpreter is so slow the language would be
unusable were it not for the fact that all of its standard library
functions and most of its useful extensions are written in C.

In practice Java and Javascript almost always are JIT compiled to
native code rather than interpreted. There also exist offline (AOT)
compilers for both.

Many JIT runtimes do let you choose to have programs interpreted
rather than compiled, but running interpreted reduces performance so
much that it is rarely done unless memory is very tight.

..NET is not a language itself but rather a runtime system like the
Jave Platform. .NET consists of a virtual machine: the Common
Language Runtime (CLR); and a set of standard libraries. Similarly
the Java Platform consists of a virtual machine: the Java Virtual
Machine (JVM); and a set of standard libraries. Compilers target
these runtime systems.

The .NET CLR does not include an interpreter ... I'm not aware that
there even is one for .NET. There is an offline (AOT) compiler that
can be used instead of the JIT.

>What's the actual ranking of programming languages? A JetBrains study
>does not list any compiled language in their first 7 ranks in 2022. C++
>follows on rank 8.
>
>What does that trend mean to a compiler group? Interpreted languages
>still need a front-end (parser) and back-end (interpreter), but don't
>these tasks differ between languages compiled to hardware or interpretation?

The trend is toward "managed" environments which offer niceties like
GC, objects with automagic serialized access, etc., all to help
protect average programmers from themselves ... err, um, from being
unable to produce working software.

>DoDi
George

On 3/26/23 1:54 AM, George Neuner wrote:
> On Sat, 25 Mar 2023 13:07:57 +0100, Hans-Peter Diettrich
> <DrDiettrich1@netscape.net> wrote:
>
>> After a look at "open software" I was astonished by the number of
>> languages and steps involved in writing portable C code. Also updates of
>> popular programs (Firefox...) are delayed by months on some platforms,
>> IMO due to missing manpower on the target systems for checks and the
>> adaptation of "configure". Now I understand why many people prefer
>> interpreted languages (Java, JavaScript, Python, .NET...) for a
>> simplification of their software products and spreading.
>
> Actually Python is the /only/ one of those that normally is
> interpreted. And the interpreter is so slow the language would be
> unusable were it not for the fact that all of its standard library
> functions and most of its useful extensions are written in C.

My impression of "interpretation" was aimed at the back-end, where
tokenized (virtual machine...) code has to be brought to a physical
machine, with a specific firmware (OS). Then the real back-end has to
reside on the target machine and OS, fully detached from the preceding
compiler stages.

Then, from the compiler writer viewpoint, it's not sufficient to define
a new language and a compiler for it, instead it must placed on top of
some popular "firmware" like Java VM, CLR or C/C++ standard libraries,
or else a dedicated back-end and libraries have to be implemented on
each supported platform.

My impression was that the FSF favors C and ./configure for "portable"
code. That's why I understand that any other way is easier for the
implementation of really portable software, that deserves no extra
tweaks for each supported target platform, for every single program. Can
somebody shed some light on the current practice of writing portable
C/C++ software, or any other compiled language, that (hopefully) does
not require additional human work before or after compilation for a
specific target platform?

DoDi

In article <23-03-029@comp.compilers>,
Hans-Peter Diettrich <DrDiettrich1@netscape.net> wrote:
>My impression was that the FSF favors C and ./configure for "portable"
>code.

Like many things, this is the result of evolution. Autoconf is well
over 20 years old, and when it was created the ISO C and POSIX standards
had not yet spread throughout the Unix/Windows/macOS world. It and the
rest of the autotools solved a real problem.

Today, the C and C++ worlds are easier to program in, but it's still
not perfect and I don't think I'd want to do without the autotools.
Particularly for the less POSIX-y systems, like MinGW and OpenVMS.

>Can somebody shed some light on the current practice of writing portable
>C/C++ software, or any other compiled language, that (hopefully) does
>not require additional human work before or after compilation for a
>specific target platform?

Well, take a look at Go. The trend there (as in the Python, Java and
C# worlds) is to significantly beef up the standard libraries. Go
has regular expressions, networking, file system, process and all kinds
of other stuff in its libraries, all things that regular old C and C++ code
often has to (or had to) hand-roll. That makes it a lot easier for
someone to just write the code to get their job done, as well as
providing for uniformity across both operating systems and applications
written in Go.

Go goes one step further, even. Following the Plan 9 example, the
golang.org Go compilers are also cross compilers. I can build a Linux
x86_64 executable on my macOS system just by setting some environment
variables when running 'go build'. Really nice.

The "go" tool itself also takes over a lot of the manual labor, such
as downloading libraries from the internet, managing build dependencies
(no need for "make") and much more. I suspect that that is also a
trend.

Does that answer your question?

Arnold

On Tuesday, March 28, 2023 at 1:14:29 AM UTC-7, Hans-Peter Diettrich wrote:

(snip)
> Then, from the compiler writer viewpoint, it's not sufficient to define
> a new language and a compiler for it, instead it must placed on top of
> some popular "firmware" like Java VM, CLR or C/C++ standard libraries,
> or else a dedicated back-end and libraries have to be implemented on
> each supported platform.

From an announcement today here on an ACM organized conference:

"We encourage authors to prepare their artifacts for submission
and make them more portable, reusable and customizable using
open-source frameworks including Docker, OCCAM, reprozip,
CodeOcean and CK."

I hadn't heard about those until I read that one, but it does sound
interesting.

On Tue, 28 Mar 2023 09:21:50 +0200, Hans-Peter Diettrich
<DrDiettrich1@netscape.net> wrote:

>On 3/26/23 1:54 AM, George Neuner wrote:
>> On Sat, 25 Mar 2023 13:07:57 +0100, Hans-Peter Diettrich
>> <DrDiettrich1@netscape.net> wrote:
>>
>>> After a look at "open software" I was astonished by the number of
>>> languages and steps involved in writing portable C code. Also updates of
>>> popular programs (Firefox...) are delayed by months on some platforms,
>>> IMO due to missing manpower on the target systems for checks and the
>>> adaptation of "configure". Now I understand why many people prefer
>>> interpreted languages (Java, JavaScript, Python, .NET...) for a
>>> simplification of their software products and spreading.
>>
>> Actually Python is the /only/ one of those that normally is
>> interpreted. And the interpreter is so slow the language would be
>> unusable were it not for the fact that all of its standard library
>> functions and most of its useful extensions are written in C.
>
>My impression of "interpretation" was aimed at the back-end, where
>tokenized (virtual machine...) code has to be brought to a physical
>machine, with a specific firmware (OS). Then the real back-end has to
>reside on the target machine and OS, fully detached from the preceding
>compiler stages.

That is exactly as I meant it.

Python and Java both initially are compiled to bytecode. But at
runtime Python bytecode is interpreted: the Python VM examines each
bytecode instruction, one by one, and executes an associated native
code subroutine that implements that operation.

In contrast, at runtime Java bytecode is JIT compiled to equivalent
native code - which include calls to native subroutines to implement
complex operations like "new", etc. The JVM JIT compiles function by
function as the program executes ... so it takes some time before the
whole program exists as native code ... but once a whole load module
has been JIT compiled, the JVM can completely ignore and even unload
the bytecode from memory.

>Then, from the compiler writer viewpoint, it's not sufficient to define
>a new language and a compiler for it, instead it must placed on top of
>some popular "firmware" like Java VM, CLR or C/C++ standard libraries,
>or else a dedicated back-end and libraries have to be implemented on
>each supported platform.

Actually it simplifies the compiler writer's job because the
instruction set for the platform VM tends not to change much over
time. A compiler targeting the VM doesn't have to scramble to support
features of every new CPU - in many cases that can be left to the
platform's JIT compiler.

>My impression was that the FSF favors C and ./configure for "portable"
>code. That's why I understand that any other way is easier for the
>implementation of really portable software, that deserves no extra
>tweaks for each supported target platform, for every single program. Can
>somebody shed some light on the current practice of writing portable
>C/C++ software, or any other compiled language, that (hopefully) does
>not require additional human work before or after compilation for a
>specific target platform?

Right. When you work on a popular "managed" platform (e.g., JVM or
CLR), then its JIT compiler and CPU specific libraries gain you any
CPU specific optimizations that may be available, essentially for
free.

OTOH, when you work in C (or other independent language), to gain CPU
specific optimizations you have to write model specific code and/or
obtain model specific libraries, you have to maintain different
versions of your compiled executables (and maybe also your sources),
and you need to be able to identify the CPU so as to install or use
model specific code.

For most developers, targeting a managed platform tends to reduce the
effort needed to achieve an equivalent result.

>DoDi
George
[The usual python implementation interprets bytecodes, but there are
also versions for .NET, the Java VM, and a JIT compiler. -John]

>[The usual python implementation interprets bytecodes, but there are
>also versions for .NET, the Java VM, and a JIT compiler. -John]

Thanks John. I knew about the reference implementation, but I was not
aware of the others.
George

On Wednesday, March 29, 2023 at 1:52:41 AM UTC-7, George Neuner wrote:

> Right. When you work on a popular "managed" platform (e.g., JVM or
> CLR), then its JIT compiler and CPU specific libraries gain you any
> CPU specific optimizations that may be available, essentially for
> free.

For system like Matlab and Octave, and I believe also for Python,
or one of many higher math languages, programs should spend
most of the time in the internal compiled library routines.

You could write a whole matrix inversion algorithm in Matlab
or Python, but no reason to do that. That is the convenience
of matrix operations, and gets better as they get bigger.

In earlier days, there were Linpack and Eispack, and other
Fortran callable math libraries. And one could write a
small Fortran program to call them.

But now we have so many different (more or less) interpreted
math oriented languages, that it is hard to keep track of them,
and hard to know which one to use.

On 2023-03-28, Aharon Robbins <arnold@freefriends.org> wrote:
>
> In article <23-03-029@comp.compilers>,
> Hans-Peter Diettrich <DrDiettrich1@netscape.net> wrote:
>>My impression was that the FSF favors C and ./configure for "portable"
>>code.
>
> Like many things, this is the result of evolution. Autoconf is well
> over 20 years old, and when it was created the ISO C and POSIX standards
> had not yet spread throughout the Unix/Windows/macOS world. It and the
> rest of the autotools solved a real problem.
>
> Today, the C and C++ worlds are easier to program in, but it's still
> not perfect and I don't think I'd want to do without the autotools.
> Particularly for the less POSIX-y systems, like MinGW and OpenVMS.

Counterpoint: Autotools are a real detriment to GNU project programs.

When a release is cut of a typical GNU program, special steps
are execute to prepare a tarball which has a compiled configure
script.

You cannot just do a "git clone" of a GNU program, and then run
../configure and build. You must run some "make boostrap" nonsense, and
that requires you to have various Autotools installed, and in specific
versions!

In the past what I have done to build a GNU program from version
control, as a quick and dirty shortcut, done was to find the tarball
which is the closest match to the baseline that I'm trying to build
(e.g. of GNU Make or GNU Awk or whatever). Unpack the tarball over the
repository and run ./configure. Then "git reset --hard" the changes and
rebuild.

Most Autotools programs will not cleanly cross-compile. Autotools is tha
main reason why distro build systems use QEMU to create a virtual target
environment with native tools and libraries, and then build the
"cross-compiled" program as if it were native.

Among the problems are in Autoconf itself. If it knows the program
is being cross-compiled, any test which requires a test program to be
compiled and executed is disabled. Since the output of that configure
test is needed, bad defaults are substituted.
For instance, about a decade and a half ago I helped a company
replace Windriver cruft with an in-house distribution. Windriver's
cross-compiled Bash didn't have job control! Ctrl-Z, fg, bg stuff no
workie. The reason was that it was just cross-compiled straight, on an
x86 build box. It couldn't run the test to detect job control support,
and so it defaulted it off, even though the target machine had
"gnu-linux" in its string. In the in-house distro, my build steps for
bash exported numerous ac_cv_... internal variables to override the bad
defaults.

My TXR language project has a hand-written, not generated, ./configure
script. What you get in a txr-285.tar.gz tarball is exactly what you
get if you do a "git clone" and "git checkout txr-285", modulo
the presence of a .git directory and differing timestamps.

You just ./configure and make.

I have a "./configure --maintainer" mode which will require flex and bison
instead of using the shipped parser stuff, and that's about it.
You don't have to use that to do development work.

There is no incomprehensible nonsense in the build system at all.

None of my configure-time tests require the execution of a program;
For some situations, I have developed clever tricks to avoid it. For
instance, if you want to know the size of a data type:. Here
is a fragment:

printf "Checking what C integer type can hold a pointer ... "

if [ -z "$intptr" ] ; then
cat > conftest.c <<!
#include <stddef.h>
#include <limits.h>
#include "config.h"

#define D(N, Z) ((N) ? (N) + '0' : Z)
#define UD(S) D((S) / 10, ' ')
#define LD(S) D((S) % 10, '0')
#define DEC(S) { UD(S), LD(S) }

struct sizes {
char h_BYTE[32], s_BYTE[2];
#if HAVE_SUPERLONG_T
char h_SUPERLONG[32], s_SUPERLONG[2];
#endif
#if HAVE_LONGLONG_T
char h_LONGLONG[32], s_LONGLONG[2];
#endif
char h_PTR[32], s_PTR[2];
char h_LONG[32], s_LONG[2];
char h_INT[32], s_INT[2];
char h_SHORT[32], s_SHORT[2];
char h_WCHAR[32], s_WCHAR[2];
char nl[2];
} foo = {
"\nSIZEOF_BYTE=", DEC(CHAR_BIT),
#if HAVE_SUPERLONG_T
"\nSIZEOF_SUPERLONG_T=", DEC(sizeof (superlong_t)),
#endif
#if HAVE_LONGLONG_T
"\nSIZEOF_LONGLONG_T=", DEC(sizeof (longlong_t)),
#endif
"\nSIZEOF_PTR=", DEC(sizeof (char *)),
"\nSIZEOF_LONG=", DEC(sizeof (long)),
"\nSIZEOF_INT=", DEC(sizeof (int)),
"\nSIZEOF_SHORT=", DEC(sizeof (short)),
"\nSIZEOF_WCHAR_T=", DEC(sizeof (wchar_t)),
"\n"
};
!

In this generated program the sizes are encoded as two-digit decimal
strings, at compile time. So the compiled object file will contain
something like "SIZEOF_PTR= 8" surrounded by newlines. The configure
script can look for these strings and get the values out:

if ! conftest_o ; then # conftest_o is a function to build the .o
printf "failed\n\n"

printf "Errors from compilation: \n\n"
cat conftest.err
exit 1
fi

The script gets the SIZEOF lines out and evals them as shell
assignments. That's why we avoided SIZEOF_PTR=08; that would become
octal in the shell:

eval $(tr '\0' ' ' < conftest.o | grep SIZEOF | sed -e 's/ *//')

It also massages these SIZEOFs into header file material:

tr '\0' ' ' < conftest.o | grep SIZEOF | sed -e 's/= */ /' -e 's/^/#define /' >> config.h

if [ $SIZEOF_PTR -eq 0 -o $SIZEOF_BYTE -eq 0 ] ; then
printf "failed\n"
exit 1
fi

Here is how it then looks like in config.h:

#define SIZEOF_BYTE 8
#define SIZEOF_LONGLONG_T 8
#define SIZEOF_PTR 4
#define SIZEOF_LONG 4
#define SIZEOF_INT 4
#define SIZEOF_SHORT 2
#define SIZEOF_WCHAR_T 4

There is a minor cross-compiling complication in txr in that you need
txr to compile the standard library. So you must build a native txr
first and then specify TXR=/path/to/native/txr to use that one for
building the standard lib. Downstream distro people have figured this
out on their own.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

On 2023-03-28, gah4 <gah4@u.washington.edu> wrote:
> On Tuesday, March 28, 2023 at 1:14:29 AM UTC-7, Hans-Peter Diettrich wrote:
>
> (snip)
>> Then, from the compiler writer viewpoint, it's not sufficient to define
>> a new language and a compiler for it, instead it must placed on top of
>> some popular "firmware" like Java VM, CLR or C/C++ standard libraries,
>> or else a dedicated back-end and libraries have to be implemented on
>> each supported platform.
>
> From an announcement today here on an ACM organized conference:
>
>
> "We encourage authors to prepare their artifacts for submission
> and make them more portable, reusable and customizable using
> open-source frameworks including Docker, OCCAM, reprozip,
> CodeOcean and CK."

"We encourage authors to lock their software to third party boat
anchors, such as ..."

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
[If you are telling people not to use Docker, that whale sailed a long time ago. -John]

gah4 <gah4@u.washington.edu> schrieb:

> For system like Matlab and Octave, and I believe also for Python,
> or one of many higher math languages, programs should spend
> most of the time in the internal compiled library routines.

They should, but sometimes they don't.

If you run into things not covered by compiled libraries, but which
are compute-intensive, then Matlab and (interpreted) Python run
as slow as molasses, orders of magnitude slower than compiled code.

As far as the projects to create compiled versions with Python
go, one of the problems is that Python is a constantly evolving
target, which can lead to real problems, especially in long-term
program maintenance. As Konrad Hinsen reported, results in
published science papers have changed due to changes in the Python
infrastructure:

http://blog.khinsen.net/posts/2017/11/16/a-plea-for-stability-in-the-scipy-ecosystem/

At the company I work for, I'm told each Python project will only
use a certain specified version of Python will never be changed for
fear of incompatibilities - they treat each version as a new
programming language :-|

To bring this back a bit towards compilers - a language definition
is an integral part of compiler writing. If

- the specification to o be implemented is unclear or "whatever
the reference implementation does"

- the compiler writers always reserve the right for a better,
incompatible idea

- the compiler writers do not pay careful attention to
existing specifications

then the resuling compiler will be of poor quality, regardless of
the cool parsing or code generation that go into it.

And I know very well that reading and understanding language
standards is no fun, but I'm told that writing them is even
less fun.

On 3/28/23 4:42 PM, Aharon Robbins wrote:
> In article <23-03-029@comp.compilers>,
> Hans-Peter Diettrich <DrDiettrich1@netscape.net> wrote:
>> My impression was that the FSF favors C and ./configure for "portable"
>> code.
>
> Like many things, this is the result of evolution. Autoconf is well
> over 20 years old, and when it was created the ISO C and POSIX standards
> had not yet spread throughout the Unix/Windows/macOS world. It and the
> rest of the autotools solved a real problem.

About 20 years ago I could not build any open source program on Windows.
Messages like "Compiler can not build executables" popped up when using
MinGW or Cygwin. I ended up in ./configure in a Linux VM and fixing the
resulting compiler errors manually on Windows. Without that trick I had
no chance to load the "portable" source code into any development
environment for inspection in readable (compilable) form. Often I had
the impression that the author wanted the program not for use on Windows
machines. Kind of "source open for specific OS only" :-(

DoDi