On 2022-11-16 12:44, James Harris wrote:
> On 15/11/2022 10:06, Dmitry A. Kazakov wrote:

>> There is no problem with Unicode string literals whatsoever. You just
>> place characters as they are. The only escape is "" for ". That is all.
>
> Two problems with that, AISI:
>
> 1. some of the characters look like others

For the reader, not for the compiler. If you want Unicode you get the
whole package, homoglyphs included.

--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

On 16/11/2022 00:30, Bart wrote:
> On 15/11/2022 21:40, David Brown wrote:
>
>> If someone wants to write code that involves a lot of squaring, then
>> let them define operators so they can write "x = squareof y", or "x =
>> y²".  They'd be able to write more of a mess, but also be able to
>> write some things very nicely.
>
> I have such an operator, called `sqr`. And also briefly allowed the
> superscript version (as a postfix op), until Unicode came along and
> spoilt it all.
>

Why would Unicode spoil it?

> One reason I had sqr was because it was in Pascal (iirc). But it
> genuinely comes in useful. Sure, I could also use x**2, but ** used to
> be only defined for floats, while `sqr` has been used for much longer.
>
> You could also ask why some languages have a dedicated `sqrt` function
> when they could just as easily do x**0.5 or pow(x**0.5).
>

It's more common to see such functions in a language's library than part
of the core language. You'd have "sqrt" as a function because it is far
and away the most common use of raising something to a fractional power,
much more familiar from school mathematics, and much more efficient to
implement than a general power function.

On 15/11/2022 20:22, Bart wrote:
> On 15/11/2022 18:05, David Brown wrote:
>> On 15/11/2022 16:26, Bart wrote:
>
>>> I group them all together: ML, OCaml, Haskell, F#, sometimes even Lisp.
>>>
>>> Anything that makes a big deal out of closures, continuations,
>>> currying, lambdas and higher order functions. I have little use for
>>> such things otherwise.
>>>
>>
>> So because /you/ don't understand these things or how they are used,
>> you assume that people who /do/ understand them can't write programs
>> in functional programming languages?
>
> No. It's nothing I've ever used, and unlikely ever to use. I like my
> functions plain, and static, just like you prefer your expressions to
> use only simple operators with no side-effects.
>

Of course you won't use something when you won't even consider trying to
learn about it.

> I still can't comprehend why YOU think this stuff is simple and obvious,
> yet you are stumped by an increment of a pointer followed by a dereference.
>

I haven't written anything to suggest that I am "stumped" by this. My
point was to say it is unnecessary to support such expressions in a
programming language, and a language may be better in some ways if it
does not allow increment operators or even pointers.

Now, it is undeniable /fact/ that programming languages do not need
operators such as increment, or other operators that cause side-effects.
It is undeniable fact that programming languages do not need pointers.
There are countless languages that have neither of these things, and
they have been used successfully for all sorts of purposes. You don't
even have to look at functional programming languages or other "it's all
too complicated for me" languages - neither Python nor Pascal has
increment operators, Python has no pointers. BASIC has been a
phenomenally successful language, especially for people who like simple
languages, and has neither.

It is also objective fact that not having these features has advantages
and disadvantages (I've gone through these enough already).

Whether a given language is /better overall/ with or without these
features - that is an entirely different question. It is very
subjective, and depends highly on the kind of language (it's "ethos", as
James puts it), the kind of use it will see, and the kind of people who
will use it. That's why I raise suggestions here, rather than giving
recommendations. (On other topics, such as James' - forgive the strong
language - insane ideas about character encodings, I have given
recommendations.)

> On a list of must-haves for a programming language, not only would they
> not be at the top of my list, they wouldn't even be at the bottom!
>

Yes, but for you, a "must-have" list for a programming language would be
mainly "must be roughly like ancient style C in functionality, but with
enough change in syntax and appearance so that no one will think it is
C". If that's what you like, and what pays for your daily bread, then
that's absolutely fine.

And there's no doubt that a large proportion of programmers go through
their career without ever considering higher order functions (functions
that operate on or return functions).

But equally there's no doubt that they /are/ useful for many people in
many types of coding. Sometimes higher order functions are used without
people knowing about them - Python decorators are a fine example.

Actually, Python declarators are such a good example that I recommend
this link <https://realpython.com/primer-on-python-decorators/> that
gives a number of useful examples.

Think of this example. You have some code with functions "foo", "bar"
and "foobar". Mostly you call them as they are in your code.

But sometimes you want to "decorate" the calls for ease of debugging -
you want to print out "Starting foo with parameters..." at the start,
and "Returning from foo with result..." at the end. You don't want to
change "foo" itself, because you only want this tracing sometimes.

So you write :

int debug_foo(int x, double y, char * z) {
printf("Starting foo with parameters %i %f %p\n",
x, y, z);
int r = foo(x, y, z);
printf("Returning from foo with result %i\n", r);
return r;
}

That's fine - now you call "debug_foo" instead of "foo" in the cases you
want. You can even use "#define foo debug_foo" to make it active
throughout the rest of the file.

Then you need to do it again for "bar".

Then you need to do it again for "foobar".

Then you decide to add timings to the traces, and have to re-do all
three debug functions.

Then you need to copy it again for "barfoo", and debug the typo you made
in "debug_foobar".

You start thinking, "I know macros are evil, but maybe they can be used
to automate this somehow?". You get an ugly but workable solution
letting you write:

MAKE_DEBUG_FUNC(debug_foo, foo);
MAKE_DEBUG_FUNC(debug_bar, bar);

Then you realise that it doesn't work for "foobar", because that only
takes two parameters. And now you have two macros "MAKE_DEBUG_FUNC_idp"
and "MAKE_DEBUG_FUNC_id".

Then you start wondering if you can make a macro that makes the macros,
or if you should thrown the computer out the window.

Alternatively, you could make a higher order function in a language that
supports it. This is in C++20 (it's slightly neater than early C++
versions can do). I'm not claiming that this is at all clear or obvious
to people with experience only in imperative languages - understanding
how to make something like this takes effort and practice. And the
syntax is C++ style, which will be unfamiliar to people used to
functional programming languages. But I wanted to write it out, as a
working function.

#include <iostream>

auto debug(auto const& f) {
return [&f](auto... args) {
std::cout << "Calling ";
((std::cout << " " << args), ...);
std::cout << "\n";
auto r = f(args...);
std::cout << "Returning " << r << "\n";
return r;
};
}

Suppose your real functions are :

int foo(int x);
int bar(int x, double y);
double foobar(int x, double y, const char * p);

Your original code was:

int a = foo(10);
int b = bar(20, 3.14);
double c = foobar(30, 2.71828, "Hello");

Your debug version is :

int a = debug(foo)(10);
int b = debug(bar)(20, 3.14);
double c = debug(foobar)(30, 2.71828, "Hello");

Note that the "debug" function is applied to "foo", and returns a
function that is then called with "10" as the parameter.

You can also write (at file scope, or inside a function) :

auto debug_foo = debug(foo);

and use :
int a = debug_foo(10);

None of this gives you things you could not do by hand. But if you find
yourself doing the same thing by hand many times, then it is natural to
ask if it can be automated - if you can write a function to do that.
You can, if you have higher order functions.

(C++ is not perfect here by any means, and more could be added to the
language. For example, Python lets you make functions that take classes
as parameters or return classes. C++ does not (yet) have that level of
metaprogramming.)

>
>>> Haskell is great for elegantly defining certain kinds of types and
>>> algorithms, not so good for reams of boilerplate code or UI stuff
>>> which is what much of programming is.
>>
>> As I mentioned in another posts, there are opinions, and there are
>> /qualified/ opinions.
>
> My opinion comes from 20 years of writing code to /get things done/ in a
> working environment. Which includes developing the languages and
> choosing the features that that best made that possible. Never once did
> I think that 'currying' was going to dramatically transform how I coded;
> never did I spend days working around the omissions of closures.
>

That's like saying you have 20 years of experience as a taxi driver, and
never once had to use "flaps" or "ailerons", or even think about the
concept. You therefore can't understand why pilots want to use them all
the time. You can give a qualified opinion on driving round roundabouts
and may be an expert on gearing, but you have no basis for a qualified
opinion on flying.

So again - mocking and dismissing concepts that you know nothing about,
makes you look foolish. (Your ignorance of the topic is not the issue -
we are all ignorant of almost everything.)

>>> Such features have some very subtle behaviours which I find
>>> incredibly hard to get my head around.
>>>
>>
>> You are a smart guy.  You could get your head around it quite easily,
>> if only you were willing.
>
> No, it is hard, obscure, subtle. Take my word for it.
>

On 16/11/2022 16:50, David Brown wrote:
> On 15/11/2022 20:22, Bart wrote:

> Of course you won't use something when you won't even consider trying to
> learn about it.

I've thought about learning Chinese. Then I decided there was no point.

>> I still can't comprehend why YOU think this stuff is simple and
>> obvious, yet you are stumped by an increment of a pointer followed by
>> a dereference.
>>
>
> I haven't written anything to suggest that I am "stumped" by this.  My
> point was to say it is unnecessary to support such expressions in a
> programming language, and a language may be better in some ways if it
> does not allow increment operators or even pointers.

It is something I value, but you don't. And higher order functions are
something you value, but I don't.

That's all it is.

> Now, it is undeniable /fact/ that programming languages do not need
> operators such as increment, or other operators that cause side-effects.

Just `a := b` causes a side effect. Possibly quite a big one if 'b' is a
substantial data structure and ':=' does a deep copy.

There is usually a task to be done. In `A[++i] := 0`, I want two things
to change, which is going to happen whether write it like that, or as `i
:= i+1; A[i] := 0`. So why write `i` 3 times?

It is not a big deal. Maybe in functional programming it might be, but
here *I* am specifying the paradigm and I say it's OK.

I'm not asking you or anyone else to use my language.

> Yes, but for you, a "must-have" list for a programming language would be
> mainly "must be roughly like ancient style C in functionality, but with
> enough change in syntax and appearance so that no one will think it is
> C".  If that's what you like, and what pays for your daily bread, then
> that's absolutely fine.

Yes, I don't need a higher level language for what I use it for. But
there are still dozens of things which make the experience superior to
just using C. Ones either you genuinely don't appreciate, or are just
pissing on for the sake of it.

* Case-insensitive
* 1-based and N-based
* Algol-style syntax, line-oriented and largely semicolon-free, and sane
type syntax
* Module scheme (define everything in exactly one place)
* Namespaces
* Encapsulation (define functions inside records etc)
* Out-of-order definitions including easy mutual record cross-references
* Regular i8-i64 and u8-u64 type denotations, include 'byte' (u8)
* Default 64-bit 'int', 'word' types, and 64-bit integer constants
* Built-in print and read statements WITH NO STUPID FORMAT CODES
* Keyword and default function parameters
* Fewer, more intuitive operator precedences
* Does not conflate arrays and pointers
* 'Proper' for loops; for-in loops
* Separate 'switch' and 'case' selection; the latter has no restrictions
(and no stupid fallthrough on switch)
* Proper named constants
* Break out of nested loops
* Embed strings and binary files
* 'Tabledata' and 'enumdata' features (compare with X-macros)
* Function reflection
* Built-in, overloaded ops like abs, min, max
* 'Properties' such as .len and .lwb
* Built-in 'swap'
* Bit/field extracion/insertion syntax
* Multiple function return values
* Multiple assignment
* Slices (including slices of char arrays to give counted strings)
* Doc strings
* Whole-program compiler that does not need a separate build system
* Pass-by-reference
* Value arrays

Yeah, just like C! If you think this lot is just C with a paint-job,
then you're in denial.

Of course, I fully expect you to be completely dismissive of all of
this. I wouldn't swap any of these for higher-order functions.

> And there's no doubt that a large proportion of programmers go through
> their career without ever considering higher order functions (functions
> that operate on or return functions).

Too right. To be able to use such things, they MUST be 100% intuitive
and be usable with 100% confidence. But that's just the author; you need
to consider other readers of your code too, and those who have to
maintain it.

To me they are a very long way from being 100% intuitive. So what do you
think I should do: strive to be a 10th-rate programmer in a functional
language I've no clue about; give up programming and tend to my garden;
or carry on coding in a style that *I* understand 100% (and most others
will too)?

The stuff I do simply doesn't require a sophisticated language with
advanced types and curried functions invented on-the-fly. Here is an
actual example from an old app, a small function to keep it short:

proc displaypoletotal =
if not poleenabled then return fi
print @poledev, chr(31), chr(14) ! clear display
print @poledev, "Total:", rightstr(strcash(total, paymentunit), 14)
end

(This is part of a POS and displays running totals, on an LED display
mounted on a pole, driven from a serial port. It ran in a duty-free area
and worked with multiple currencies.)

What can higher-order-functions do for me here? Absolutely sod-all.

> But equally there's no doubt that they /are/ useful for many people in
> many types of coding.  Sometimes higher order functions are used without
> people knowing about them - Python decorators are a fine example.
>
> Actually, Python declarators are such a good example that I recommend

Decorators?

> this link <https://realpython.com/primer-on-python-decorators/> that
> gives a number of useful examples.

Decorators are a /very/ good example of a Python feature that I could
never get my head around. 5 minutes later, I'd have to look them up again.

> Think of this example.  You have some code with functions "foo", "bar"
> and "foobar".  Mostly you call them as they are in your code.

> auto debug(auto const& f) {
>     return [&f](auto... args) {
>         std::cout << "Calling ";
>         ((std::cout << " " << args), ...);
>         std::cout << "\n";
>         auto r = f(args...);
>         std::cout << "Returning " << r << "\n";
>         return r;
>     };
> }
>
> Suppose your real functions are :
>
>     int foo(int x);
>     int bar(int x, double y);
>     double foobar(int x, double y, const char * p);
>
> Your original code was:
>
>     int a = foo(10);
>     int b = bar(20, 3.14);
>     double c = foobar(30, 2.71828, "Hello");

>
> None of this gives you things you could not do by hand.  But if you find
> yourself doing the same thing by hand many times, then it is natural to
> ask if it can be automated - if you can write a function to do that. You
> can, if you have higher order functions.

I can't follow the C++ debug function at all. But I notice the user code
changes from 'foo()' to 'debug()()'; I thought this could be done while
leaving the foo() call unchanged.

But no, my language doesn't deal with parameter lists as a first class
entity at all. (At best it can access them as a list object, but it
doesn't help here.)

The best I can do here is to have a dedicated function for each number
of arguments, and to use dynamic code to allow the same function for any
types:

func debug3(f, a,b,c)=
println "Calling",f,"with",a,b,c
f(a,b,c)
end

func foobar(a,b,c)=
println "FooBar",a,b,c
return a+b+c
end

x:=debug3(foobar, 5,6,7) # in place of foobar(5, 6, 7)

println x

This displays:

Calling <procid:"foobar"> with 5 6 7
FooBar 5 6 7
18

However this loses to ability to use any keyword or default arguments
for FooBar, since they are only available for direct calls (it's done at
compile-time).

So I can see that that C++ debug does some very hairy stuff, to make it
work with static types and for any function, but I just can't understand it.

However, given the requirement you outlined, I could probably come up
with a custom feature to do just that. Although it might be in the form
of a compiler option which injects the debug code at the start of the
relevant functions. Then the user code does not need updating.

See, when you have control of the language and implementation, there are
more and better possibilities.

> That's like saying you have 20 years of experience as a taxi driver, and
> never once had to use "flaps" or "ailerons", or even think about the
> concept.  You therefore can't understand why pilots want to use them all
> the time.  You can give a qualified opinion on driving round roundabouts
> and may be an expert on gearing, but you have no basis for a qualified
> opinion on flying.

On 16/11/2022 12:02, Dmitry A. Kazakov wrote:
> On 2022-11-16 12:44, James Harris wrote:
>> On 15/11/2022 10:06, Dmitry A. Kazakov wrote:
>
>>> There is no problem with Unicode string literals whatsoever. You just
>>> place characters as they are. The only escape is "" for ". That is all.
>>
>> Two problems with that, AISI:
>>
>> 1. some of the characters look like others
>
> For the reader, not for the compiler. If you want Unicode you get the
> whole package, homoglyphs included.
>

Yes, Unicode has all kinds of problems for humans. Something else is
needed for programming but I don't think it's been created yet.

--
James Harris

On 16/11/2022 16:02, David Brown wrote:
> On 16/11/2022 00:30, Bart wrote:
>> On 15/11/2022 21:40, David Brown wrote:
>>
>>> If someone wants to write code that involves a lot of squaring, then
>>> let them define operators so they can write "x = squareof y", or "x =
>>> y²".  They'd be able to write more of a mess, but also be able to
>>> write some things very nicely.
>>
>> I have such an operator, called `sqr`. And also briefly allowed the
>> superscript version (as a postfix op), until Unicode came along and
>> spoilt it all.
>>
>
> Why would Unicode spoil it?

I was using 8-bit code pages for western European alphabets since
probably from the end of the 80s. It was simple, I supported it and it
worked well. (At that time, I was also responsible for vector fonts
within my apps.)

But Unicode makes everything harder, with characters taking up multiple
bytes, and a lot of the time it just doesn't work. (I've seen Unicode
errors on everything from TV subtitles to supermarket receipts, and that
was a few weeks ago.)

Then you have the choices of UCS2, UCS4, UTF8, with patchy support for
UTF8 within Windows. Even if I get it working on my machine, how do I
know that someone else running my program will have their machine set up
properly?

For me it's just not worth it.

On 14/11/2022 18:41, Dmitry A. Kazakov wrote:
> On 2022-11-14 19:26, James Harris wrote:
>> On 14/11/2022 11:29, Dmitry A. Kazakov wrote:
>>> On 2022-11-14 12:03, James Harris wrote:

....

>>>>    if is_name_first(b[j])
>>>>      a[i++] = b[j++]
>>>>      rep while is_name_follow(b[j])
>>>>        a[i++] = b[j++]
>>>>      end rep
>>>>      a[i] = 0
>>>>      return TOK_NAME
>>>>    end if
>>>>
>>>> Now, what don't you like about the ++ operators in that? How would
>>>> you prefer to write it?
>>>
>>>  From parser production code:
>>>
>>> procedure Get_Identifier
>>>            (  Code     : in out Source'Class;
>>>               Line     : String;
>>>               Pointer  : Integer;
>>>               Argument : out Tokens.Argument_Token
>>>            )  is
>>>     Index     : Integer := Pointer + 1;
>>>     Malformed : Boolean := False;
>>>     Underline : Boolean := False;
>>>     Symbol    : Character;
>>> begin
>>>     while Index <= Line'Last loop
>>>        Symbol := Line (Index);
>>>        if Is_Alphanumeric (Symbol) then
>>>           Underline := False;
>>>        elsif '_' = Symbol then
>>>           Malformed := Malformed or Underline;
>>>           Underline := True;
>>>        else
>>>           exit;
>>>        end if;
>>>        Index := Index + 1;
>>>     end loop;
>>>     Malformed := Malformed or Underline;
>>>     Set_Pointer (Code, Index);
>>>     Argument.Location := Link (Code);
>>>     Argument.Value := new Identifier (Index - Pointer);
>>>     declare
>>>        This : Identifier renames Identifier (Argument.Value.all);
>>>     begin
>>>        This.Location  := Argument.Location;
>>>        This.Malformed := Malformed;
>>>        This.Value     := Line (Pointer..Index - 1);
>>>     end;
>>> end Get_Identifier;
>>
>> Well, that's an astonishingly long piece of code, Dmitry,
>
> Because it is a production code.

So was the code which preceded it.

> It must deal with different types of
> sources, with error handling and syntax tree generation.

In fairness, detecting double and trailing underscores adds work to the
Ada code so I've been thinking how I might write the Ada version. The
following is my attempt. It is untested and in a somewhat experimental
form but I think it's quite readable. The main part of the code would be

errors = 0
last_char = line(pointer)
rep for i = pointer + 1, while i le line_last, ++i
ch = line(i)
if ch eq '_'
if last_char eq '_' so ++errors ;Consecutive underscores
on not is_alphanum(ch)
break rep ;If neither underscore nor alphanum we are done
end if
last_char = ch
end rep
if last_char eq '_' so ++errors ;Trailing underscore
....
this.Malformed = bool(errors)

Some notes on the code. I found the Ada program's

Malformed := Malformed or Underline;

to be clever but it took a bit of thinking about to work out what it was
intended to do in the context. So I changed it to an error count which
is incremented with

++errors

and used bool(errors) at the end.

Rather than having a boolean called Underline I just kept a copy of the
last character.

Since I didn't need the Underline boolean I found I could also get rid
of a branch of the if statement so the code is a little shorter. But
more important, I think the code is clearer. YMMV but I bet you can
understand it!

In the context of this discussion, the code uses ++i to increment the
index and ++errors to increment the error count. (There's no danger of
either of them overflowing, given the line length.) Neither is embedded
in an expression.

On the clarity of the "++" operator note that any of these would do the
same:

i = i + 1
i += 1
++i

I assert that the last one is the most readable. It makes the
programmer's intent clear at a glance.

....

>> But I am not sure I do understand it. Even allowing for what I believe
>> is meant to be double underscore detection (except at the start and
>> end?) it takes significantly more study than the simple name-first,
>> name-follow code which preceded it.
>
> That's how the language defines it. This example is from an Ada 95
> parser. Ada 95 RM 2.3:
>
>    https://www.adahome.com/rm95/rm9x-02-03.html
>

--
James Harris

On 16/11/2022 23:01, Bart wrote:
> On 16/11/2022 16:02, David Brown wrote:
>> On 16/11/2022 00:30, Bart wrote:
>>> On 15/11/2022 21:40, David Brown wrote:
>>>
>>>> If someone wants to write code that involves a lot of squaring, then
>>>> let them define operators so they can write "x = squareof y", or "x
>>>> = y²".  They'd be able to write more of a mess, but also be able to
>>>> write some things very nicely.
>>>
>>> I have such an operator, called `sqr`. And also briefly allowed the
>>> superscript version (as a postfix op), until Unicode came along and
>>> spoilt it all.
>>>
>>
>> Why would Unicode spoil it?
>
> I was using 8-bit code pages for western European alphabets since
> probably from the end of the 80s. It was simple, I supported it and it
> worked well. (At that time, I was also responsible for vector fonts
> within my apps.)
>

Such code pages did work, but were very limited. In the UK, code pages
typically meant nothing worse than mixups between # and £. Go beyond
the English speaking world, and code pages were a nightmare. If one
non-English Western European language was enough, they were often not
/too/ bad - but supporting multiple languages was often hugely
complicated and fraught with errors.

Unicode made some things more complex, but other things far easier - it
is not a surprise to me that it has supplanted pretty much every usage
where plain old 7-bit ASCII is insufficient. I understand how Unicode
can be difficult, but it is solving a difficult problem.

But back to your superscript square operator - does that mean you used
an extended ASCII code in a specific code page for superscript 2 (I
think it is 0xfb in Latin-9), but when Unicode came out you stopped
using anything beyond 7-bit ASCII?

> But Unicode makes everything harder, with characters taking up multiple
> bytes, and a lot of the time it just doesn't work. (I've seen Unicode
> errors on everything from TV subtitles to supermarket receipts, and that
> was a few weeks ago.)

That's not a Unicode problem - that's a software bug.

>
> Then you have the choices of UCS2, UCS4, UTF8, with patchy support for
> UTF8 within Windows. Even if I get it working on my machine, how do I
> know that someone else running my program will have their machine set up
> properly?
>
> For me it's just not worth it.
>
>

In the early days of Unicode, there were different encodings. For the
last couple of decades it's been clear that there is /one/ sensible
encoding - UTF-8. Everything else exists only as long as it is needed
for backwards compatibility and until software, OS's and API's are
changed - you only need them on the boundaries of code (file I/O,
calling external API's). And yes, I know that is an extra hassle.

I do understand that it is /much/ easier to stick to 7-bit ASCII if that
is all you need. But if 7-bit ASCII is not sufficient, the UTF-8 is
much easier than anything else.

On 17/11/2022 10:34, David Brown wrote:
> On 16/11/2022 23:01, Bart wrote:
>> On 16/11/2022 16:02, David Brown wrote:
>>> On 16/11/2022 00:30, Bart wrote:
>>>> On 15/11/2022 21:40, David Brown wrote:
>>>>
>>>>> If someone wants to write code that involves a lot of squaring,
>>>>> then let them define operators so they can write "x = squareof y",
>>>>> or "x = y²".  They'd be able to write more of a mess, but also be
>>>>> able to write some things very nicely.
>>>>
>>>> I have such an operator, called `sqr`. And also briefly allowed the
>>>> superscript version (as a postfix op), until Unicode came along and
>>>> spoilt it all.
>>>>
>>>
>>> Why would Unicode spoil it?
>>
>> I was using 8-bit code pages for western European alphabets since
>> probably from the end of the 80s. It was simple, I supported it and it
>> worked well. (At that time, I was also responsible for vector fonts
>> within my apps.)
>>
>
> Such code pages did work, but were very limited.  In the UK, code pages
> typically meant nothing worse than mixups between # and £.  Go beyond
> the English speaking world, and code pages were a nightmare.  If one
> non-English Western European language was enough, they were often not
> /too/ bad - but supporting multiple languages was often hugely
> complicated and fraught with errors.
>
> Unicode made some things more complex, but other things far easier - it
> is not a surprise to me that it has supplanted pretty much every usage
> where plain old 7-bit ASCII is insufficient.  I understand how Unicode
> can be difficult, but it is solving a difficult problem.
>
>
> But back to your superscript square operator - does that mean you used
> an extended ASCII code in a specific code page for superscript 2 (I
> think it is 0xfb in Latin-9), but when Unicode came out you stopped
> using anything beyond 7-bit ASCII?

I used one that I called 'ANSI' but is Windows-1252
(https://en.wikipedia.org/wiki/Windows-1252). There, superscript 2 was
code 0xB2.

Before that I used an older one associated with MSDOS, I think code page
850, where superscript 2 was code 0xFD.

(IMO the best set of extended character codes was used by the Amstrad
PCW from the 1980s; a Z80-based word processing machine. Very elegantly
set out.)

I provided support for French and German (mainly Swiss variations) and
Dutch. This included providing special keyboard layouts used on
digitising tablets, and supplying the vector fonts necessary for
pen-plotters. (These were largely nicked from AutoCAD but I added
support for accents, 'hats', cedillas etc, plus some special symbols.)

>> But Unicode makes everything harder, with characters taking up
>> multiple bytes, and a lot of the time it just doesn't work. (I've seen
>> Unicode errors on everything from TV subtitles to supermarket
>> receipts, and that was a few weeks ago.)
>
> That's not a Unicode problem - that's a software bug.

It means even the big boys have issues with it.

>>
>> Then you have the choices of UCS2, UCS4, UTF8, with patchy support for
>> UTF8 within Windows. Even if I get it working on my machine, how do I
>> know that someone else running my program will have their machine set
>> up properly?
>>
>> For me it's just not worth it.
>>
>>
>
> In the early days of Unicode, there were different encodings.  For the
> last couple of decades it's been clear that there is /one/ sensible
> encoding - UTF-8.

If I wanted to display UTF8 right now on Windows, say from a C program
even, I would have to fight it. If I write this (created with Notepad):

#include <stdio.h>
int main(void) {
printf("€°£");
}

and compile with gcc, it shows:

€°£

I'm not sure what code page it's on, but if I switch to 65001 which is
supposed to be UTF8, then it shows:

�������

(or equivalent in the terminal font). If I dump the C source, it does
indeed contain the E2 82 AC sequence which is the UTF8 for the 20AC code
for the Euro sign.

I'm sure that on Linux it works perfectly within a terminal window. But
I'm on Windows and I can't be bothered to do battle. Even if /I/ get it
to work, I can't guarantee it for anyone else.

Hmm, I just compiled it with both bcc and tcc, and they both correctly
show €°£ when using code page 65001. So that's something, but what's up
with gcc?

On 17/11/2022 11:24, Bart wrote:

> Hmm, I just compiled it with both bcc and tcc, and they both correctly
> show €°£ when using code page 65001. So that's something, but what's up
> with gcc?

BTW ° (degree symbol in case it doesn't display properly) was something
I did use more extensively both in my scripting language and my apps' CLI.

So sin(30°) would evaluate to 0.5, instead of having to do sin(pi/6
(typical of ordinary languages) or sin(30 deg) which was the fall-back
version.

Both ° and 'deg' applied a scaling factor to the number.

On 2022-11-17 12:24, Bart wrote:

> If I wanted to display UTF8 right now on Windows, say from a C program
> even, I would have to fight it. If I write this (created with Notepad):
>
>   #include <stdio.h>
>   int main(void) {
>       printf("€°£");
>   }

If you want to display UTF-8, you must obviously use UTF-8, no?

#include <stdio.h>
int main(void) {
printf("\xE2\x82\xAC\xC2\xB0\xC2\xA3");
}

In CMD:

>CHCP 65001
Active code page: 65001

>main.exe
€°£

Of course, you could use the code you wrote under the condition that
both the editor and the compiler use UTF-8.

Which is why every programming guideline must require ASCII-7 source
like I provided.

--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

On 17/11/2022 12:12, Dmitry A. Kazakov wrote:
> On 2022-11-17 12:24, Bart wrote:
>
>> If I wanted to display UTF8 right now on Windows, say from a C program
>> even, I would have to fight it. If I write this (created with Notepad):
>>
>>    #include <stdio.h>
>>    int main(void) {
>>        printf("€°£");
>>    }
>
> If you want to display UTF-8, you must obviously use UTF-8, no?
>
>     #include <stdio.h>
>     int main(void) {
>         printf("\xE2\x82\xAC\xC2\xB0\xC2\xA3");
>     }

This wasn't the problem. I verified that the text file contained the
correct UTF8 sequences, and the two other compilers worked. This was a
problem with gcc, which also fails your version.

>
> In CMD:
>
> >CHCP 65001
> Active code page: 65001
>
> >main.exe
> €°£
>
> Of course, you could use the code you wrote under the condition that
> both the editor and the compiler use UTF-8.

The point about UTF8 is that it doesn't matter. So the string contains
'character' E2; in C, this is just a byte array, it should just pass it
as it is to the printf function.

> Which is why every programming guideline must require ASCII-7 source
> like I provided.

That would work, but is also completely impractical for large amounts of
non-ASCII content. Or even small amounts. You /need/ editor support. I
don't have it and don't do enough with Unicode to make it worth the trouble.

On 2022-11-17 13:35, Bart wrote:
> On 17/11/2022 12:12, Dmitry A. Kazakov wrote:
>> On 2022-11-17 12:24, Bart wrote:
>>
>>> If I wanted to display UTF8 right now on Windows, say from a C
>>> program even, I would have to fight it. If I write this (created with
>>> Notepad):
>>>
>>>    #include <stdio.h>
>>>    int main(void) {
>>>        printf("€°£");
>>>    }
>>
>> If you want to display UTF-8, you must obviously use UTF-8, no?
>>
>>      #include <stdio.h>
>>      int main(void) {
>>          printf("\xE2\x82\xAC\xC2\xB0\xC2\xA3");
>>      }
>
>
> This wasn't the problem. I verified that the text file contained the
> correct UTF8 sequences, and the two other compilers worked. This was a
> problem with gcc, which also fails your version.

The above was compiled with gcc version 10.3.1 20210520.

>> In CMD:
>>
>>  >CHCP 65001
>> Active code page: 65001
>>
>>  >main.exe
>> €°£
>>
>> Of course, you could use the code you wrote under the condition that
>> both the editor and the compiler use UTF-8.
>
> The point about UTF8 is that it doesn't matter. So the string contains
> 'character' E2; in C, this is just a byte array, it should just pass it
> as it is to the printf function.

It does, but the terminal driver interprets octets you output there. You
can verify the actual output by redirecting the standard output.

> That would work, but is also completely impractical for large amounts of
> non-ASCII content. Or even small amounts. You /need/ editor support. I
> don't have it and don't do enough with Unicode to make it worth the
> trouble.

That's is another guideline topic: you never ever place localization
stuff in the source code.

--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

On 17/11/2022 13:20, Dmitry A. Kazakov wrote:
> On 2022-11-17 13:35, Bart wrote:
>> On 17/11/2022 12:12, Dmitry A. Kazakov wrote:
>>> On 2022-11-17 12:24, Bart wrote:
>>>
>>>> If I wanted to display UTF8 right now on Windows, say from a C
>>>> program even, I would have to fight it. If I write this (created
>>>> with Notepad):
>>>>
>>>>    #include <stdio.h>
>>>>    int main(void) {
>>>>        printf("€°£");
>>>>    }
>>>
>>> If you want to display UTF-8, you must obviously use UTF-8, no?
>>>
>>>      #include <stdio.h>
>>>      int main(void) {
>>>          printf("\xE2\x82\xAC\xC2\xB0\xC2\xA3");
>>>      }
>>
>>
>> This wasn't the problem. I verified that the text file contained the
>> correct UTF8 sequences, and the two other compilers worked. This was a
>> problem with gcc, which also fails your version.
>
> The above was compiled with gcc version 10.3.1 20210520.

And run on Windows?

Further tests show that it works in every case, including using gcc with
puts, and gcc+printf under WSL. It only fails with gcc + printf + Windows.

Odd. But then my point is you can't rely on it. You still need the UTF8
code page set.

That's not all, because console display is different from graphical display.

#include <stdio.h>
#include <windows.h>
int main(void) {
MessageBox(0,"\xE2\x82\xAC\xC2\xB0\xC2\xA3",
"\xE2\x82\xAC\xC2\xB0\xC2\xA3",0);
}

This displays only gobbledygook. Of course, this is set to use
MessageBoxA, which expects an ASCII string; but why won't it take UTF8
and show something sensible?

Presumably it needs the correct code page set for the WinAPI, not the
one I set for the console. But I can't find a way to do it; MS docs
suggest setting this in a resource file or XML manifest file; WTF?

The alternative is to use MessageBoxW, but that means switching
everything to UCS2; it is a massive upheaval.

Any wonder that I'm just not interested? Here I have to say that Linux
seems to get it right.

On 2022-11-17 16:06, Bart wrote:
> On 17/11/2022 13:20, Dmitry A. Kazakov wrote:
>> On 2022-11-17 13:35, Bart wrote:
>>> On 17/11/2022 12:12, Dmitry A. Kazakov wrote:
>>>> On 2022-11-17 12:24, Bart wrote:
>>>>
>>>>> If I wanted to display UTF8 right now on Windows, say from a C
>>>>> program even, I would have to fight it. If I write this (created
>>>>> with Notepad):
>>>>>
>>>>>    #include <stdio.h>
>>>>>    int main(void) {
>>>>>        printf("€°£");
>>>>>    }
>>>>
>>>> If you want to display UTF-8, you must obviously use UTF-8, no?
>>>>
>>>>      #include <stdio.h>
>>>>      int main(void) {
>>>>          printf("\xE2\x82\xAC\xC2\xB0\xC2\xA3");
>>>>      }
>>>
>>>
>>> This wasn't the problem. I verified that the text file contained the
>>> correct UTF8 sequences, and the two other compilers worked. This was
>>> a problem with gcc, which also fails your version.
>>
>> The above was compiled with gcc version 10.3.1 20210520.
>
> And run on Windows?

Of course.

> Further tests show that it works in every case, including using gcc with
> puts, and gcc+printf under WSL. It only fails with gcc + printf + Windows.
>
> Odd. But then my point is you can't rely on it. You still need the UTF8
> code page set.
>
> That's not all, because console display is different from graphical
> display.
>
>    #include <stdio.h>
>    #include <windows.h>
>    int main(void) {
>        MessageBox(0,"\xE2\x82\xAC\xC2\xB0\xC2\xA3",
>                     "\xE2\x82\xAC\xC2\xB0\xC2\xA3",0);
>    }

> This displays only gobbledygook. Of course, this is set to use
> MessageBoxA, which expects an ASCII string; but why won't it take UTF8
> and show something sensible?

Because Windows GDI is ASCII (MessageBoxA) or else UTF-16 (MessageBoxW).

> Presumably it needs the correct code page set for the WinAPI, not the
> one I set for the console. But I can't find a way to do it; MS docs
> suggest setting this in a resource file or XML manifest file; WTF?

There is no any pages in Windows GDI.

> Any wonder that I'm just not interested? Here I have to say that Linux
> seems to get it right.

Linux used code pages as well. It adopted Unicode very late and took
UTF-8 straight away. On its part Windows started early and took UCS-2 by
splitting all calls into xxxA and xxxW. Later on, when Unicode grew
larger Microsoft silently replaced UCS-2 with UTF-16. All xxxW calls are
UTF-16 now.

--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

On 16/11/2022 21:04, Bart wrote:
> On 16/11/2022 16:50, David Brown wrote:
>> On 15/11/2022 20:22, Bart wrote:
>
>> Of course you won't use something when you won't even consider trying
>> to learn about it.
>
> I've thought about learning Chinese. Then I decided there was no point.
>

That's fine. But you wouldn't enter a discussion with a linguist who
has some experience with Chinese, and try to tell them that Chinese
grammar is beyond human comprehension. You could say that /you/ think
it looks like Chinese writing would be hard to learn - but you could
/not/ say anything about how hard it is for Chinese speakers to learn.
You could not even say that it really would be difficult for you to
learn, because you haven't tried or investigated enough.

>>> I still can't comprehend why YOU think this stuff is simple and
>>> obvious, yet you are stumped by an increment of a pointer followed by
>>> a dereference.
>>>
>>
>> I haven't written anything to suggest that I am "stumped" by this.  My
>> point was to say it is unnecessary to support such expressions in a
>> programming language, and a language may be better in some ways if it
>> does not allow increment operators or even pointers.
>
> It is something I value, but you don't.

Again, I have written nothing to indicate that. You read so much
between the lines that you miss the words I actually write. When I
write "You could consider doing A in your /new/ language - it would give
these advantages...", that is /exactly/ what I mean. It does not mean
"I don't like B", or "I am stumped by C", or "You should not do D".

> And higher order functions are something you value, but I don't.
>

That bit is true.

> That's all it is.
>

I can fully respect your personal preferences - that's not the issue for
me. I find it sad and disappointing that someone can have such strong
opinions about something they have never really considered or tried to
learn about, and consequently don't understand, but I guess that is
human nature. It's ancient survival tactics, honed by evolution - when
something is new, different and unknown, human instinct is to fear it
and run away, rather than investigate and study it.

The bit that really bugs me is how you (and James) can hold such strong
opinions about how /other people/ might like and use these features and
languages that support them. Is it so hard to accept that some people
like using higher order functions? Or that some people write code in
functional programming languages, because they find it a better choice
for their needs? Is it so hard to accept that other people can write
code for the same task in widely different languages, and /your/ code in
/your/ language is not the "perfect" solution or the only "non-clunky" code?

>
>> Now, it is undeniable /fact/ that programming languages do not need
>> operators such as increment, or other operators that cause side-effects.
>
> Just `a := b` causes a side effect. Possibly quite a big one if 'b' is a
> substantial data structure and ':=' does a deep copy.

It is not an expression in many languages, but a statement. Indeed, the
symbol ":=" is not an "operator" in many languages - it's just part of
the syntax of an assignment statement.

>
> There is usually a task to be done. In `A[++i] := 0`, I want two things
> to change, which is going to happen whether write it like that, or as `i
> := i+1; A[i] := 0`. So why write `i` 3 times?
>

If I want two things to change, why try to squeeze it into /one/
expression or statement? Why not write two statements, each one doing a
single clear and simple task?

(As to writing "i" three times - again, these things are often found in
loops, where a good syntax can mean "i" is never written at all.)

> It is not a big deal. Maybe in functional programming it might be, but
> here *I* am specifying the paradigm and I say it's OK.
>
> I'm not asking you or anyone else to use my language.
>

I am not asking you to use functional programming either - I am merely
asking you to appreciate that /others/ do so, and they will find your
language as clunky, repetitive, ugly and inexpressive in comparison.

>
>> Yes, but for you, a "must-have" list for a programming language would
>> be mainly "must be roughly like ancient style C in functionality, but
>> with enough change in syntax and appearance so that no one will think
>> it is C".  If that's what you like, and what pays for your daily
>> bread, then that's absolutely fine.
>
> Yes, I don't need a higher level language for what I use it for. But
> there are still dozens of things which make the experience superior to
> just using C. Ones either you genuinely don't appreciate, or are just
> pissing on for the sake of it.
>

Again, I have written nothing to indicate that. You read so much
between the lines that you miss the words I actually write. When I
write "You could consider doing A in your /new/ language - it would give
these advantages...", that is /exactly/ what I mean. It does not mean
"I don't like B", or "I am stumped by C", or "You should not do D".

> * Case-insensitive

Subjective, and I disagree.

> * 1-based and N-based

(I assume you mean array indexing, or possibly loops?) Subjective, and
I disagree if the starting 1 is implicit. If it is explicit - "array 1
to 10 of int" - then I definitely like it.

> * Algol-style syntax, line-oriented and largely semicolon-free, and sane
> type syntax

Subjective on all points. But these are mostly syntactic details - C
and Pascal are both considered "ALGOL family" languages despite
syntactic differences.

> * Module scheme (define everything in exactly one place)

Objective - a good module system is always a good idea. /Defining/
things in one place is good, but being able to /declare/ them elsewhere
is useful as it lets you separate interfaces from implementations.

> * Namespaces

Objective - good.

> * Encapsulation (define functions inside records etc)

Objective - good. However, it is not so important that functions are
defined /inside/ a record/struct/etc. The important part is that you
can make a user-defined type containing data of other types, and that
you can restrict access to the internals to be via specific functions or
operations.

> * Out-of-order definitions including easy mutual record cross-references

Subjective - convenient in some ways, less convenient in others. I
personally don't mind for functions and variables. But it is definitely
useful for defining recursive structure types.

> * Regular i8-i64 and u8-u64 type denotations, include 'byte' (u8)

Objective - very good for low-level languages, unnecessary for higher
level languages (except for FFI and other interfacing). It's a lot
nicer if you can just use "integer" and let the compiler worry about
sizes, but that's harder to implement efficiently.

> * Default 64-bit 'int', 'word' types, and 64-bit integer constants

Subjective - it makes little difference in practice if default integer
sizes are 32-bit or 64-bit, assuming big target systems. Each is big
enough for "almost everything", and neither is big enough for
"absolutely everything".

> * Built-in print and read statements WITH NO STUPID FORMAT CODES

Subjective - though the modern trend is strongly towards avoiding
special statement types and preferring standard library functions for
this sort of thing. What constitutes "stupid" format codes is obviously
highly subjective - but it is objectively better if they can be deduced
automatically by the compiler.

> * Keyword and default function parameters

Subjective. I like keyword or named parameters. Others feel they
encourage having too complicated interfaces with too many parameters.

> * Fewer, more intuitive operator precedences

Subjective. Some people think it is best to give a total order for
operator precedence, which means a lot of levels. And "intuitive" is
completely subjective after basic mathematical arithmetic operators. I
am inclined to agree with you in general, however.

> * Does not conflate arrays and pointers

Objective, and good.

> * 'Proper' for loops; for-in loops

"Proper" is subjective. Good loop constructs are vital for an
imperative language, however.

> * Separate 'switch' and 'case' selection; the latter has no restrictions
> (and no stupid fallthrough on switch)

Subjective, I think. There are many ways to handle multiple choices or
pattern matching, and I don't think there is any justification for
claiming one particular way is "right" or "the best". One can do a lot
better than C's "switch", and I agree about "fallthrough".

On 17/11/2022 12:24, Bart wrote:
> On 17/11/2022 10:34, David Brown wrote:
>> On 16/11/2022 23:01, Bart wrote:

>>> But Unicode makes everything harder, with characters taking up
>>> multiple bytes, and a lot of the time it just doesn't work. (I've
>>> seen Unicode errors on everything from TV subtitles to supermarket
>>> receipts, and that was a few weeks ago.)
>>
>> That's not a Unicode problem - that's a software bug.
>
> It means even the big boys have issues with it.
>
What makes you think software for that kind of thing is made by "big
boys"? It's as likely to be small companies or single developers, as
anything else. /I/ have made software that printed out receipts in
machines in supermarkets, and that was in assembly with no Unicode in
sight. (It was a long time ago.)

And the "big boys" make mistakes as often as the small folks, both in
terms of understanding the problem, knowing about the solutions, and
implementing the code.

On 2022-11-18 11:03, David Brown wrote:

> And the "big boys" make mistakes as often as the small folks, both in
> terms of understanding the problem, knowing about the solutions, and
> implementing the code.

Big boys make more mistakes due to mismanagement typical for big
organizations.

--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

On 16/11/2022 00:11, Bart wrote:
> On 15/11/2022 18:05, David Brown wrote:
>> On 15/11/2022 16:26, Bart wrote:
>
>>>>  >>> import struct        # Standard module
>>>>  >>> bs = open("potato_c.cof").read()
>>>>
>>>>  >>> machine, nsections, timestamp, symtaboffset, nsymbols,
>>>> optheadersize, characteristics = struct.unpack_from("<HHIIIHH")
>>>>
>>>> That's it.  Three lines.  I would not think of C for this kind of
>>>> thing - Python is /much/ better suited.
>
> I don't believe you.

OK, I suppose. But I've done this in Python, I've done it in C, and I
would never choose C for any kind of serious file handling. Reading a
simple structure in C is okay, and of course if you already have the
struct defined in a C header, using it will save effort.

>>
>>> (BTW you might be missing an argument in that struct.unpack_from call.)
>>
>> No, I am not.  There is an optional third argument, but it is optional.
>
> What about the second argument? I don't understand how the function call
> knows to get the data from 'bs'.
>

My apologies - I must have had a copy-and-paste screwup. I didn't see
it, and assumed you were talking about the optional "offset" parameter,
or perhaps the odd "/" character that appears in the documentation.
Yes, I was missing "bs" as the second parameter.

<https://docs.python.org/3/library/struct.html>

>>>
>>> Using that approach for the nested structs and unions of my other
>>> example is not so straightforward. You basically have to fight for
>>> every field.
>>
>> You have to define every field in every language, or define the ones
>> you want along with offsets to skip uninteresting data.
>
> When properly supported, you can define the fields of a struct just as
> you would in any static language (see above example), and you can write
> handling code just as conveniently.
>
> You don't have to manually write strings of anonymous letter codes and
> have to remember their ordering everywhere they are used. That is just
> crass.

You don't have to do that in Python either. But it is really convenient
for small and simple cases.

If you have something bigger or more complicated, you can do what you
can always do in programming - divide and conquer. When I wrote network
code for Modbus, I used one pack/unpack set for the common packet
header, another for the Modbus function specific fields, and another for
the data (using "H" * n for the format string of multiple 16-bit
unsigned types, since that's what Modbus likes).

Or you can use ctypes:
<https://docs.python.org/3/library/ctypes.html#structures-and-unions>

I've found struct pack/unpack to be very convenient and simple. It is
particularly handy in combination with Python's array slicing, and easy
combination of arrays or strings by just "adding" them.

>
> I went out of my way to add such facilities in my scripting language,
> because I felt it was important. So you can code just as you would in a
> static language but with the convenience of informal scripting.
>
> Clearly you don't care for such things and prefer a hack.
>

You say "hack" as though there exists "right" and "wrong" ways to handle
particular tasks. I use a language and techniques that are quick,
simple, and do the job. Other people can understand, modify and use the
code as they need. (In that respect, pretty much everything beats your
solution - except maybe Perl :-) ) It's a solution, not a "hack".

>>>
>>> The result is a tuple of unnamed fields. You really want a proper
>>> record, which is yet another add-on, with a choice of several modules
>>> depending on which set of characterics you need.
>>
>> You can do that in Python.
>
> Yeah, I know, you can do anything in Python, since there is an army of
> people who will create the necessary add-on modules to create ugly and
> cumbersome bolted-on solutions.

Do you actually know much about Python? Maybe I've been assuming too much.

>
> I can list dozens of things that my scripting language does better than
> Python. (Here, such a list exists:
> https://github.com/sal55/langs/blob/master/QLang/QBasics.md.)
>

I'm not falling for that again. Suffice to say that you can assume any
other programmer (not just me) will throw out at least half the list
because they disagree with your opinions. (Which half will, of course,
vary enormously.) And anyone with experience with Python and who is
insanely bored could list hundreds or thousands of things that are
better in Python - assuming they could find solid documentation for your
language.

>
>
>> In short, you are making up shit in an attempt to make your own
>> language look better than other languages, because you'd rather say
>> something silly than admit that any other language could be better in
>> any way for any task.
>
> Not at all. Python is better for lots of things, mainly because there
> are a million libraries that people have written for it, armies of
> volunteers who have written suitable, bindings or written all sorts of
> shit. And there is huge community and lots of resources to help out.
>
> It is also full of as many advanced, esoteric features that you could
> wish for.
>
> But it is short of the more basic and primitive features of the kind I
> use and find invaluable.
>
>>> 'struct' is also not a true Python module; it's a front end for an
>>> internal one called `_struct`, likely implemented in C, and almost
>>> certainly using pointers.
>>
>> Please re-think what you wrote there.  I hope you can realise how
>> ridiculous you are being.
>
> Tell me. Maybe struct.py could be written in pure Python; I don't know.

Your last point is quite telling - you /don't know/, yet you feel
qualified to make claims about it. But you are missing the /real/ point
- the implementation is /irrelevant/. It doesn't matter if "struct" is
written in pure Python, or as a module written in C, or anything else.
It is a standard Python library module (thus not an "add-on" or "some
weird extra module"). When people write code in the Python language
using the standard Python library to do tasks like the one you gave
above, they do not use pointers.

(In case you are curious, some implementations of Python - such as PyPy
- have pure Python implementations of modules like "struct". Others,
such as CPython, have C modules for that kind of thing.)

> I'm saying I guarantee mine would have the necessary features to do so.
>
> But this started off being about pointers. Here's another challenge:
> this program is for Windows, and displays the first 2 bytes of the
> executable image of the interpreter, as loaded in memory:
>
>     println peek(0x400000, u16):"m"
>
>     fun peek(addr, t=byte) = makeref(addr, t)^
>
> This displays 'MZ' (the signature of PE files on Windows). But of
> interest is how Python would implement that peek() function.
>

You think it is a good think that programs have direct access to the
memory their interpreter's executable? Really?

import struct
bs = open("/usr/bin/python").read()
print(struct.unpack_from("H", bs, 0x40000)[0])

That prints the 16-bit value at offset 0x40000 from the start of the
"python" file. Was that what you wanted?

On 15/11/2022 18:32, James Harris wrote:
> On 14/11/2022 15:23, David Brown wrote:
>> On 14/11/2022 11:47, Bart wrote:
>
> ...
>
>>> In-place, value-returning increment ops written as ++ and -- are
>>> common in languages.
>>>
>>
>> Yes.  And bugs are common in programs.  Being common does not
>> necessarily mean it's a good idea.
>>
>> (It doesn't necessarily mean it's a bad idea either - I am not
>> implying that increment and decrement are themselves a major cause of
>> bugs!  But mixing side-effects inside expressions /is/ a cause of bugs.)
>
> The side effects of even something awkward such as
>
>   *(++p) = *(q++);
>
> are little different from those of the longer version
>
>   p = p + 1;
>   *p = *q;
>   q = q + 1;
>
> The former is clearer, however. That makes it easier to see the intent..

Really? I have no idea what the programmer's intent was. "*p++ =
*q++;" is common enough that the intent is clear there, but from your
first code I can't see /why/ the programmer wanted to /pre/increment
"p". Maybe he/she made a mistake? Maybe he/she doesn't really
understand the difference between pre-increment and post-increment?
It's a common beginners misunderstanding.

On the other hand, it is quite clear from the separate lines exactly
what order the programmer intended.

What would you say are the differences in side-effects of these two code
snippets? (I'm assuming we are talking about C here.)

>
> Just blaming operators you don't like is unsound - especially since, as
> you seem to suggest below, you use them in your own code!!!
>

All I am saying is that it's worth considering the advantages and
disadvantages of making a decision about such operators. I'm not
denying that the operators can be useful - I am questioning whether
those uses are enough in comparison to the advantages of /not/ having them.

When I program in C, I use the features of C as best I can in the
clearest way. I don't change the language (though I limit myself to a
subset of the possibilities, of course, as everyone does in every language).

> ...
>
> [discussion of ++ and -- operators]
>
>>> Is your point that you shouldn't have either of those operators?
>>
>> Yes!  What gave it away - the first three or four times I said as much?
>
> ...
>
>> ... (Of course I use increment operator, especially in loops, because
>> that's how C is written.  But a new language can do better than that.)
>
> If you think ++ and -- shouldn't exist then why not ban them from your
> own programming for a while before you try to get them banned from a new
> language?
>

Banning them from C would have no benefits because C has side-effects in
expressions, and avoiding the operators won't change that. Besides, the
real problem is not careful and considered use, it is the /abuse/ that
some people call "clever coding". It is not the "char c = *p++;" that
is the problem, it is the "++E*++".

On 18/11/2022 10:48, David Brown wrote:
> On 16/11/2022 00:11, Bart wrote:

>> I can list dozens of things that my scripting language does better
>> than Python. (Here, such a list exists:
>> https://github.com/sal55/langs/blob/master/QLang/QBasics.md.)
>>
>
> I'm not falling for that again.  Suffice to say that you can assume any
> other programmer (not just me) will throw out at least half the list
> because they disagree with your opinions.  (Which half will, of course,
> vary enormously.)  And anyone with experience with Python and who is
> insanely bored could list hundreds or thousands of things that are
> better in Python - assuming they could find solid documentation for your
> language.

The point of that list was to show the basics that are either missing or
that have clunky bolted-on implementations, or that have to be achieved
by abusing advanced features (I'm thinking of people who have added even
'goto' to Python).

As one example, if I want the ASCII code for "A", I write 'A' as one
might do in C.

In Python it's ord("A") (which used to involve a global lookup then a
function call; perhaps it still does, since 'ord' might have been
reassigned).

In Lua it's string.byte("A"). Just what you need in interpreted code:
extra overheads!

What I'm saying is that there are too many advanced features with less
support for fundamental ones.

>> This displays 'MZ' (the signature of PE files on Windows). But of
>> interest is how Python would implement that peek() function.
>>
>
> You think it is a good think that programs have direct access to the
> memory their interpreter's executable?  Really?

I don't claim my language is safe. I made a decision that still allowing
raw pointers in scripting languages can be useful in the right hands.
(However these have some extra protections compared with their C
equivalents.)

But Python has so many possibilities with Cython or Ctypes or C
extension modules, that I'm sure just as much mischief can be done if
somebody wants, but the whole thing is so complicated that you can't
audit that easily.

>
>     import struct
>     bs = open("/usr/bin/python").read()
>     print(struct.unpack_from("H", bs, 0x40000)[0])
>
> That prints the 16-bit value at offset 0x40000 from the start of the
> "python" file.  Was that what you wanted?

Not quite. Your code just reads 2 bytes from a file (at a 256K offset
from the start of the file; the MZ signature is at offset 0).

My code accesses my task's virtual memory space, where the executable
image is placed at offset 0x400000 (4MB) from address 0x000000. Think of
it as a form of reflection.

On 18/11/2022 07:12, David Brown wrote:
> If I want two things to change, why try to squeeze it into /one/
> expression or statement? Why not write two statements, each one
> doing a single clear and simple task?

I think there is a danger of over-egging this sort of case, partly
caused by writing simple examples. It depends on context, but one reason
to write one expression rather than two statements is to keep code short
and structured:

a[ i +:= 1 ] := x

vs

begin
i +:= 1;
a[i] := x
end

[on the assumption that this is the controlled clause of a conditional or
loop, so needs to be turned into one statement]. Repeated several times
in a few lines of code, it turns a half-page procedure you can comprehend
"at a glance" into a page-and-a-half for which you have to keep scrolling
up and down, the "begins" and "ends" [or moral equivalents] start to
obtrude, and the whole structure of the code is less clear. You've
replaced one task ["assign to the next element of the array"] by two,
that may or may not be linked.

Clearly this can be overdone, but there is a balance to be struck,
and I would argue that Pascal and, judging by Dmitry's tokeniser example,
Ada are too far one way and C[++] too far the other.

--
Andy Walker, Nottingham.
Andy's music pages: www.cuboid.me.uk/andy/Music
Composer of the day: www.cuboid.me.uk/andy/Music/Composers/Couperin

On 18/11/2022 07:12, David Brown wrote:
> On 16/11/2022 21:04, Bart wrote:

> That's fine.  But you wouldn't enter a discussion with a linguist who
> has some experience with Chinese, and try to tell them that Chinese
> grammar is beyond human comprehension.  You could say that /you/ think
> it looks like Chinese writing would be hard to learn - but you could
> /not/ say anything about how hard it is for Chinese speakers to learn.
> You could not even say that it really would be difficult for you to
> learn, because you haven't tried or investigated enough.

I mentioned Chinese because I and a friend did attend a short (and free)
taster course.

But I also mentioned it because it's not that useful in the UK, even if
I'd mastered some it.

It's also interesting because if you look at the English Wikipedia entry
on China, the web page source is something like 99% ASCII, with the rest
Unicode. But even if you look at the /Chinese/ page, the page source is
still 80% ASCII (mainly due to tags etc)

This ties in with the discussion on Unicode, and also my point about
giving weight to more basic language features.

Unicode understands the importance of ASCII, since ASCII occupies the
first 128 Unicode codepoints.

(Imagine if ASCII had been relegated to some obscure alphabet.)

> I can fully respect your personal preferences - that's not the issue for
> me.  I find it sad and disappointing that someone can have such strong
> opinions about something they have never really considered or tried to
> learn about,

I've tried lots of times. I've even tried implementing some of it.

> and consequently don't understand, but I guess that is
> human nature.  It's ancient survival tactics, honed by evolution - when
> something is new, different and unknown, human instinct is to fear it
> and run away, rather than investigate and study it.

There is a VAST amount of complexity about now, most of which is not
justified. And that applies across everything - processors, computers,
operating systems, languages, tools and applications.

It all barely manages to work, mainly thanks to just throwing more
resources at any problems instead of tackling it at root, by simplifying.

The big thing in new languges now is not really advanced kinds of
functions, it is even more advanced and esoteric type systems.

I'm not interested in advanced functions and even less in types. You can
do a huge amount with the basics; what you do can be understood by more
people; and it is accessible to more people. It's probably also easier
to make things efficient.

Tell me what the latest thing is in English now; what's new? There are
lots of new words about, but most people communicate using a much more
basic vocabulary.

> The bit that really bugs me is how you (and James) can hold such strong
> opinions about how /other people/ might like and use these features and
> languages that support them.  Is it so hard to accept that some people
> like using higher order functions?

Is it so hard to accept that the vast majority of functions in any
application are VERY ordinary? So why make so much of the tiny minority
that need those special features?

Look at any library: OpenGL exports 500 ordinary functions; GTK and
Windows perhaps the best part of 10,000 functions each.

> Or that some people write code in
> functional programming languages, because they find it a better choice
> for their needs?

I said I don't care what other people use.

I care when functional style is inflicted on me, but if I reject it,
then I'm just being deliberately ignorant; a diehard; a Luddite; a
stick-in-the-mud.

All I see is something that makes ordinary coding much harder to do and
harder to understand.

>  Is it so hard to accept that other people can write
> code for the same task in widely different languages, and /your/ code in
> /your/ language is not the "perfect" solution or the only "non-clunky"
> code?

I accept my 1980s-style code is clunky. But it still works extraordinary
well. Code in my scripting language is clear enough and conservative
enough to act as pseudo-code.

(There was a period in clc when Ben Bacarisse used to post short 10-line
solitions to some task in Haskell. Except that they were rather
difficult to understand.

I had a habit then where I posted solutions in my scripting language
which were far simpler for /anyone/ to understand. But mine might have
been 15 lines instead of 10.)

> If I want two things to change, why try to squeeze it into /one/
> expression or statement?  Why not write two statements, each one doing a
> single clear and simple task?

What about:

swap(a, b) # from my stuff
(a, b) = (b, a) # from Python
(a, b) = f() # multiple function return values
x := (a, b, c, d) # record/struct constructor assigns 4 fields
A[3..6] := (1,2,3,4) # change 4 elements of A

This stuff happens everywhere.

> (As to writing "i" three times - again, these things are often found in
> loops, where a good syntax can mean "i" is never written at all.)

If you're talking about C-style for-loops, that remains one of the great
mysteries of the world: why so many languages would copy such a crappy
feature:

for (i=a; i<=b; ++i)

instead of:

for i in a..b # or countless variations

But it's amusing that people who defend that C-style loop dismiss that
superior version and say, Ah, but you shouldn't be using such iteration
at all! Let's jump a language level or two and go straight into advanced
array manipulation features, or straight to functional code.

Here, show me this in any language:

to n do
println random()
od

Just repeat-N-times. It terms of low-hanging fruit of simple-to-add and
simple-to-understand and convenient features, it is one of the lowest.
But as a feature it is rare.

>> Yeah, just like C! If you think this lot is just C with a paint-job,
>> then you're in denial.
>
> Yes, it is a lot like C.

Yes, it is that /level/ of language. But done properly (IMO).

>  It has a number of changes, some that I think
> are good, some that I think are bad, but basically it is mostly like C.

There are some massive changes. Even C++ has only just acquired modules.

> In particular, it's quite clear to me that when you developed your
> language, you had the assembly level implementation heavily in mind when
> doing so.  Why are your numbers 64-bit integers by default?

What other size makes sense these days? C has to use i32 for
compatibility reasons, although the language itself would allow i64.

>  It is not
> because it is a particularly useful size for integers, but because it
> fits the cpu's you are targeting.  Why do you want integer types that
> are powers of [2]?

For the same reason that 10 other languages I can't be bothered to list,
that have fixed-width integer types, do so too.

Yes you can have the luxury of ignoring the CPU, but you're going to
have a slower and/or more complex language and/or a more massive and
slower compiler or JIT to get it up to speed.

> It's a low-level language.

I've never said anything else. My two languages are marked as M and Q
here, on a scale of language level:

C--M-----Q---------------Python

Plus other kinds of languages exist outside of that line. It's a
particular niche I find useful for the things I want to do.

>> Of course, I fully expect you to be completely dismissive of all of
>> this. I wouldn't swap any of these for higher-order functions.
>>
>
> I can't imagine why you would think adding higher-order functions would
> mean dropping any of it.

Languages having such functions ahead of nearly everything else annoy
me. One new language proposal over on Reddit described 3 kinds of functions.

The first two were advanced ones; the last were ordinary 'named'
functions, as are used for 99% of the functions in real applications.

They were almost an afterthought!

> "Intuitive" means you've used it often enough to use the feature without
> thinking about it.  Nothing more.  Stop imagining that everything you
> learned along your programming career is somehow easier than other
> methods seen by other people.  It's so long since you learned to program
> that you've forgotten how it goes.  When you have a long history of
> programming in ALGOL, assembly, and perhaps a spot of FORTRAN or BASIC,
> the step to C or your own language is minor.  That makes it /seem/
> intuitive, but it is not - it's just what you are used to.

It was also what I could do within 16KB of memory on an 8-bit processor,
written in an assembler written in hex machine code.

Then I realised how much potential even that simple language had. With
that language (and with an extra 16KB data memory and 8KB video memory),
I worked on two kinds of programs I had an interest in, apart from the
language stuff:

On 16/11/2022 23:02, James Harris wrote:
> On 14/11/2022 18:41, Dmitry A. Kazakov wrote:
>> On 2022-11-14 19:26, James Harris wrote:
>>> On 14/11/2022 11:29, Dmitry A. Kazakov wrote:

....

>>>>     Index     : Integer := Pointer + 1;
>>>>     Malformed : Boolean := False;
>>>>     Underline : Boolean := False;
>>>>     Symbol    : Character;
>>>> begin
>>>>     while Index <= Line'Last loop
>>>>        Symbol := Line (Index);
>>>>        if Is_Alphanumeric (Symbol) then
>>>>           Underline := False;
>>>>        elsif '_' = Symbol then
>>>>           Malformed := Malformed or Underline;
>>>>           Underline := True;
>>>>        else
>>>>           exit;
>>>>        end if;
>>>>        Index := Index + 1;
>>>>     end loop;
>>>>     Malformed := Malformed or Underline;

....

>     errors = 0
>     last_char = line(pointer)
>     rep for i = pointer + 1, while i le line_last, ++i
>       ch = line(i)
>       if ch eq '_'
>         if last_char eq '_' so ++errors ;Consecutive underscores
>       on not is_alphanum(ch)
>         break rep ;If neither underscore nor alphanum we are done
>       end if
>       last_char = ch
>     end rep
>     if last_char eq '_' so ++errors ;Trailing underscore

....

It occurred to me that the code could be made yet shorter, leading to

errors = 0
rep for i = pointer, while ++i le line_last
if line(i) eq '_'
if line(i - 1) eq '_' so ++errors
on not alphanum(line(i))
exit rep ;Neither underscore nor alphanum so we're done
end if
end rep
if line(i - 1) eq '_' so ++errors ;Trailing underscore

Overall, this bit of the code has gone from 17 lines to 12 to 9. That's
an appreciable reduction. And no tricks were involved. This is basic
refactored code.

Don't get me wrong. I'm not saying that short code is in and of itself a
benefit. But if the 'complexity density' remains the same then less code
does help: it gives a programmer less to read, less to understand, less
to keep in his head, and less to debug.

As well as having fewer lines, some of the lines in the latter version
are also shorter and simpler than those in the version which preceded it.

The most recent changes:

I stripped out the variables ch and last_char. Referring, instead, to
line(i) and line(i - 1). IMO that makes clearer what's being referred
to: the char at the current index and the one before.

Another change is relevant to this thread: I made better use of one of
the "++" operators. The prior code began its loop with

i = pointer + 1, while i ...., ++i

That code was an obvious candidate for changing to the simpler

i = pointer, while ++i ....

I return to my earlier assertion: allowing programmers to use the nudge
operators ("++" etc) allows certain algorithms to be expressed more
elegantly, more clearly, and more simply.

--
James Harris

On 15/11/2022 21:40, David Brown wrote:
> On 15/11/2022 20:09, James Harris wrote:
>> On 15/11/2022 17:31, David Brown wrote:
>>> On 15/11/2022 17:58, James Harris wrote:

....

>> The question is not whether prevention would be possible but whether
>> you (i.e. DB) would consider it /advisable/. If you prevented it then
>> a lot of familiar programming patterns and a number of existing APIs
>> would become unavailable to you so choose wisely...! :-)
>>
>
> I am not the language designer here

Uh, huh.

> - and I still don't really grok what
> kind of language /you/ want, what you understand from before, what uses
> it should have, or what you think is wrong with existing languages.  (Or
> maybe this is all for fun and interest, which is always the best reason
> for doing anything.)  That makes it hard to give recommendations.

....

>>> You assume /so/ many limitations on what you can do as a language
>>> designer.  You can do /anything/.  If you want to allow something,
>>> allow it.  If you want to prohibit it, prohibit it.
>>
>> Sorry, but it doesn't work like that.
>
> Yes, it does.

No, it does not. Your view of language design is far too simplistic.
Note, also, that in a few paragraphs you say that you are not the
language designer whereas I am, but then you go on to try to tell me how
it works and how it doesn't and, previously, that anything can be done.
You'd gain by /trying/ it yourself. They you might see that it's not as
straightforward as you suggest.

>
>> A language cannot be built on ad-hoc choices such as you have suggested.
>
> I haven't suggested ad-hoc choices.  I have tried to make reasoned
> suggestions.  Being different from languages you have used before, or
> how you envision your new language, does not make them ad-hoc.

Saying you'd like selected combinations of operators to be banned looks
like an ad-hoc approach to me.

....

>>>> BTW, any time one thinks of 'treating X separately' it's good to be
>>>> wary. Step-outs tend to make a language hard to learn and awkward to
>>>> use.
>>>>
>>>
>>> So do over-generalisations.
>>
>> Not really.
>
> Yes, really.

You simply repeating phrases back to me but in the negative does not
make your assertions correct or mine wrong.

>
> Imagine if you were to stop treating "letters", "digits" and
> "punctuation" separately, and say "They are all just characters.  Let's
> treat them the same".  Now people can name a function "123", or "2+2".
> It's conceivable that you'd work out a grammar and parsing rules that
> allow that (Forth, for example, has no problem with functions that are
> named by digits.  You can redefine "2" to mean "1" if you like).  Do you
> think that would make the language easier to learn and less awkward to use?

Certainly not. Why do you ask?

>
>> It's ad-hoc rules which become burdensome.
>
> Agreed.

Phew!

....

>> Seriously, try designing a language, yourself. You don't have to
>> implement it. Just try coming up with a cohesive design of something
>> you would like to program in.
>>
>
> If I had the time...  :-)
>
> I fully appreciate that this is not an easy task.

I'm sure you do but your view of the details is superficial. You have
ideas which are interesting in themselves but you don't appear to
appreciate how decisions bear on each other when you have to bring
hundreds of them together.

....

>> Bart came up with an example something like
>>
>>    +(+(+(+ x)))
>>
>> That's not at all sensible. You want that banned, too?
>>
>
> Yes :-)  Seriously, I appreciate that there will always be compromises -
> trying to ban everything silly while allowing everything sensible would
> mean countless ad-hoc rules, and you are right to reject that.  I am
> advocating drawing a line, just like you - the difference is merely a
> matter of where to draw that line.  I'd draw the line so that it throws
> out the increment and decrement operators entirely.  But if you really
> wanted to keep them, I'd make them postfix only and as statements, not
> in expressions - let "x++" mean "x += 1" which means "x = 1" which
> should, IMHO, be a statement and not allowed inside an expression.

This does, indeed, in a sense, come down to where the designer decides
to draw the line. Unfortunately there is no simple line.

For example, you spoke about banning side effects in expressions. For
sure, you could do that. But then you thought side effects in function
calls in expressions should possibly be treated differently and be left
in! Making such rules is not as simple as it may appear. All the
decisions a language designer makes have a tendency to bear on each
other, even if only in the ethos of the final language: whether it's
simple and cohesive or ad hoc and confusing.

Further, remember that the decisions the language designer makes have to
be communicated to the programmer. If a designer says "these side
effects are allowed but these other ones are not" then that just gives
the programmer more to learn and remember.

As I say, you could try designing a language. You are a smart guy. You
could work on a design in your head while walking to the shops, while
waiting for a train, etc. As one of my books on language design says,
"design repeatedly: it will make you a better designer".

--
James Harris