On Tue, 2 Jan 2024 17:45:58 -0000 (UTC), Dan Cross wrote:

> A significant amount of work is done maintaining drivers, but inside
> MSFT and for OEMs. Linux has become the most important OS in the world,
> and writing drivers for Windows is notoriously tedious.

In the words of the infamous “Halloween documents”
<http://www.catb.org/~esr/halloween/halloween2.html>:

An important attribute to note which has led to volume drivers is
the ease with which you can write drivers for linux, and the
relatively powerful debugging infrastructure that linux has.
Finding and installing the DDK, and trying to hook up the kernel
debugger and do any sort of interaction with user-mode without
tearing the NT system to bits is much more challenging than
writing the simple device-drivers for linux. Any idiot could write
a driver in 2 days with a book like "Linux Device Drivers" — there
is no such thing as a 2-day device-driver for NT

On Tue, 2 Jan 2024 17:41:20 -0000 (UTC), Dan Cross wrote:

> Note that the Windows API is, by design, opaque to application
> programs ...

Worse than that, it is poorly specified. Compare POSIX, where it is not
just the functionality of each call that is documented, but also the
possible error conditions and what they mean. This is missing from the
Win32 API docs.

This is why the WINE project, for example, took 15 years to get to version
1.0: because they had to determine all the possible error paths in Win32
by exhaustive search.

Jeremy Allison goes into detail about this here
<https://www.samba.org/samba/news/articles/low_point/
tale_two_stds_os2.html>.

Lawrence D'Oliveiro wrote:

> Andy Burns wrote:
>
>> ... there is a certain technical elegance to the
>> WSL1 mechanism.
>
> When Windows NT was first created, it was supposed to support the idea of
> “personalities” to allow different kinds of userland apps to run on top of
> the same core kernel. For example, Win32 was just one kind of
> “personality”, while the POSIX layer was implemented as another kind.
>
> You would think the WSL1 would have been implemented as just another
> “personality”. But no. It appears the whole extensibility of the
> “personality” architecture has kind of bitrotted away in the years since
> the introduction of NT,

They've tipped the nomenclature on its head now "windows subsystem for
linux" rather than "posix subsystem for windows"

I've read that the posix subsystem did the absolute bare minimum to pass
tests, without being in any way in any way useful, just so that WinNT
could tick the "posix" box and be allowed into govt contracts.

search for "ncommander posix"

> so WSL1 was implemented in its own unique kind of
> way (don’t ask me how).
>
> And, in spite of the fact that it was supposed to be implementing a well-
> documented API, with plenty of example source code to refer to, they still
> couldn’t get WSL1 to work right. So they had to chuck it away and bring in
> a proper Linux kernel instead.

I didn't do anything other than "play" with WSL1, but I thought
performance was the issue.

I think I did more with MKS Toolkit and Cygwin than WSL.

On Tue, 2 Jan 2024 13:39:03 -0000 (UTC), Simon Clubley wrote:

> You may wish to make it clear that RMS is a part of VMS in that it is
> within VMS itself and not just some user-mode library linked in to each
> user program.

The symbols have to come from some file that is input to the linker
(.OLB, .OBJ, .STB). All the system entry points (RMS and the kernel
proper) reside at absolute locations; this used to be at 80000168 hex
(note: system space), then later (in either VMS V3 or V4) that was moved
to 7FFEDF68 (note: P1 space). Not sure why: to allow per-process
interception of system calls? Of course the old location continued to be
valid forever for existing services, but all new services added thereafter
were only available at the new place.

RMS was one of those things that was quite difficult to explain to new
users. I was the micro support guy, whom people came to if they had
trouble moving files between these newfangled PC things and the VAX
machines. And of course VMS software would typically refuse to work
properly with files unless they had the right record format attributes
etc. And files transferred from other systems tended to end up as “fixed-
length 512-byte records”, which was just about the least useful format
imaginable. So fixing the RMS attributes was a very common need. Assuming
the users could understand the right settings to use.

Linus “Mr Linux” Torvalds did apparently use VMS for a while, and he hated
it. The reason had to do with RMS.

On 1/2/24 7:24 AM, Arne Vajhøj wrote:
> On 1/2/2024 7:46 AM, Andy Burns wrote:
>> Single Stage to Orbit wrote:
>>> Arne Vajhøj wrote:
>>>> You sort of get both with WSL.
>>>
>>> No, WSL is terrible! Use WSL2 instead.
>>
>> Performance-wise sure, but there is a certain technical elegance to
>> the WSL1 mechanism.
>
> Was the problem with WSL1 performance?
>
> I always thought that the problem was compatibility
> (that 99.9% compatibility was not god enough).

The docs say that WSL2 performance is worse than WSL1 for operations
involving file system integration. Details here:

https://github.com/microsoft/WSL/issues/4197#issuecomment-604592340

In article <un1n5c$2q36g$4@dont-email.me>,
Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
>On Tue, 2 Jan 2024 12:46:14 +0000, Andy Burns wrote:
>> ... there is a certain technical elegance to the
>> WSL1 mechanism.
>
>When Windows NT was first created, it was supposed to support the idea of
>"personalities" to allow different kinds of userland apps to run on top of
>the same core kernel. For example, Win32 was just one kind of
>"personality", while the POSIX layer was implemented as another kind.
>
>You would think the WSL1 would have been implemented as just another
>"personality". But no. It appears the whole extensibility of the
>"personality" architecture has kind of bitrotted away in the years since
>the introduction of NT, so WSL1 was implemented in its own unique kind of
>way (don't ask me how).

With the caveat that I have no first-hand experience with the
WSL implementation in the Windows kernel, the mechanism is
likely to be fairly straight-forward. When the system goes to
invoke an executable program, it examines the image file; for
both PECOFF (e.g., Windows .EXE format) and Linux-styled ELF
binaries, the image formats are pretty well defined and the
kernel can easily distinguish between them. Depending on which
it is attempting to run, it can select the appropriate loader
and it can install a per-process system call handler. Traps
into the OS for system calls will then consult the local system
call table for that process and decide how to handle the call.

The interesting bit is inside the kernel, where the different
interfaces are handled using (presumably) a common set of
functionality.

>And, in spite of the fact that it was supposed to be implementing a well-
>documented API, with plenty of example source code to refer to, they still
>couldn't get WSL1 to work right. So they had to chuck it away and bring in
>a proper Linux kernel instead.

This is a well-understood problem in the Linux space; gVisor
suffers from a similar issue.

The problem is not that Linux's behavior is well-documented, it
is also what is not documented. It's not enough to be
consistent with the documented interfaces, but one must also be
"bug-compatible" with the system. Further, Linux has a
long-standing policy of preserving userspace behaviors, even
when doing so means that the implementation contradicts a
documented interface in some way or another.

Furthermore, the rate of change in Linux is high; following
along from outside is fraught. And we haven't even gotten
to things like:
https://docs.kernel.org/process/stable-api-nonsense.html

- Dan C.

In article <kvj3t3Fbk87U1@mid.individual.net>,
bill <bill.gunshannon@gmail.com> wrote:
>On 1/2/2024 12:41 PM, Dan Cross wrote:
>> In article <umvlnj$2cmas$1@dont-email.me>,
>> Arne Vajhøj <arne@vajhoej.dk> wrote:s.
>>
>>> And they have known about both concepts for a long
>>> time. WSL1 concept goes back to 90's - Win32 API on
>>> top of NT kernel and Win9x kernel.
>>
>> Not really, but the basic concept of emulating a system
>> interface on another system goes much further back than
>> that; e.g., PA1050 on TOPS-20, or the DTSS environment on
>> Multics.
>
>And lets not forget The Software Tools Virtual Operating System
>in 1980 that gave you a bunch of the Unix API on everything from
>CP/M to Exec-8 on the Univac 1100 including all of the DEC systems
>of the time. :-)

Indeed! Though that was more about a suite of useful
functionality for source-level compatibility, more than running
unmodified binaries on compatible machines.

- Dan C.

On Tue, 2 Jan 2024 19:25:02 +0000, Andy Burns wrote:

> I've read that the posix subsystem did the absolute bare minimum to pass
> tests, without being in any way in any way useful, just so that WinNT
> could tick the "posix" box and be allowed into govt contracts.

The true horror of it has to be seen to be believed
<https://www.youtube.com/watch?v=BOeku3hDzrM>.

> I think I did more with MKS Toolkit and Cygwin than WSL.

Interestingly, Cygwin seems to do a better job in some ways than Windows
can manage natively, like get select(2)/poll(2) to work with pipes. Maybe
that was the sort of thing that sank WSL1?

In article <un1nm8$2q36g$6@dont-email.me>,
Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
>On Tue, 2 Jan 2024 17:41:20 -0000 (UTC), Dan Cross wrote:
>
>> Note that the Windows API is, by design, opaque to application
>> programs ...
>
>Worse than that, it is poorly specified. Compare POSIX, where it is not
>just the functionality of each call that is documented, but also the
>possible error conditions and what they mean. This is missing from the
>Win32 API docs.
>
>This is why the WINE project, for example, took 15 years to get to version
>1.0: because they had to determine all the possible error paths in Win32
>by exhaustive search.
>
>Jeremy Allison goes into detail about this here
><https://www.samba.org/samba/news/articles/low_point/
>tale_two_stds_os2.html>.

That's a separate issue. That is more about underspecification
of the documented programmer's interface, but I'm talking about
the opacity of the kernel/user interface.

- Dan C.

Lawrence D'Oliveiro wrote:

> Compare POSIX, where it is not
> just the functionality of each call that is documented, but also the
> possible error conditions and what they mean

Yet NT *can* pass the NIST/FIPS/Posix tests, so is Posix that good a test?

In article <un1nc8$2q36g$5@dont-email.me>,
Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
>On Tue, 2 Jan 2024 17:45:58 -0000 (UTC), Dan Cross wrote:
>
>> A significant amount of work is done maintaining drivers, but inside
>> MSFT and for OEMs. Linux has become the most important OS in the world,
>> and writing drivers for Windows is notoriously tedious.
>
>In the words of the infamous “Halloween documents”
><http://www.catb.org/~esr/halloween/halloween2.html>:
>
> An important attribute to note which has led to volume drivers is
> the ease with which you can write drivers for linux, and the
> relatively powerful debugging infrastructure that linux has.
> Finding and installing the DDK, and trying to hook up the kernel
> debugger and do any sort of interaction with user-mode without
> tearing the NT system to bits is much more challenging than
> writing the simple device-drivers for linux. Any idiot could write
> a driver in 2 days with a book like "Linux Device Drivers" — there
> is no such thing as a 2-day device-driver for NT

Yeah, NatBro's comment here is a bit dated. WinDbg is actually
amazing and trounces anything in the Linux/GNU world. The only
thing comparable might be `mdb` in the Solaris/illumos world.

- Dan C.

In article <kvj7djF8sf8U8@mid.individual.net>,
Andy Burns <usenet@andyburns.uk> wrote:
>Lawrence D'Oliveiro wrote:
>
>> Compare POSIX, where it is not
>> just the functionality of each call that is documented, but also the
>> possible error conditions and what they mean
>
>Yet NT *can* pass the NIST/FIPS/Posix tests, so is Posix that good a test?

Not really. POSIX is sort of a least-common-denominator thing,
and has become increasingly less relevant over time. z/OS is
POSIX compliant, IIRC; make of that what you will. :-)

- Dan C.

Lawrence D'Oliveiro wrote:

> Andy Burns wrote:
>
>> I've read that the posix subsystem did the absolute bare minimum to pass
>> tests, without being in any way in any way useful, just so that WinNT
>> could tick the "posix" box and be allowed into govt contracts.
>
> The true horror of it has to be seen to be believed
> <https://www.youtube.com/watch?v=BOeku3hDzrM>.

Yes, that's the "ncommander posix" search that I referred to

On 1/2/2024 2:42 PM, Dan Cross wrote:
> In article <kvj3t3Fbk87U1@mid.individual.net>,
> bill <bill.gunshannon@gmail.com> wrote:
>> On 1/2/2024 12:41 PM, Dan Cross wrote:
>>> In article <umvlnj$2cmas$1@dont-email.me>,
>>> Arne Vajhøj <arne@vajhoej.dk> wrote:s.
>>>
>>>> And they have known about both concepts for a long
>>>> time. WSL1 concept goes back to 90's - Win32 API on
>>>> top of NT kernel and Win9x kernel.
>>>
>>> Not really, but the basic concept of emulating a system
>>> interface on another system goes much further back than
>>> that; e.g., PA1050 on TOPS-20, or the DTSS environment on
>>> Multics.
>>
>> And lets not forget The Software Tools Virtual Operating System
>> in 1980 that gave you a bunch of the Unix API on everything from
>> CP/M to Exec-8 on the Univac 1100 including all of the DEC systems
>> of the time. :-)
>
> Indeed! Though that was more about a suite of useful
> functionality for source-level compatibility, more than running
> unmodified binaries on compatible machines.
>

One of the big "selling" points for Unix was always source
compatibility. In the early Linux days there were compatibility
compatibility libraries that were supposed to make Linux stuff
run on BSD. I never had much luck with it and usually found it
easier to just recompile under BSD.

bill

On Tue, 2 Jan 2024 19:41:21 -0000 (UTC), Dan Cross wrote:

> Furthermore, the rate of change in Linux is high; following along from
> outside is fraught.

Strange, isn’t it: Microsoft must have access to at least a couple of
orders of magnitude greater development resources than that available to
the Linux kernel project, yet they cannot keep up with what the Linux
developers have achieved.

In article <un1s48$2r0sq$1@dont-email.me>,
Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
>On Tue, 2 Jan 2024 19:41:21 -0000 (UTC), Dan Cross wrote:
>> Furthermore, the rate of change in Linux is high; following along from
>> outside is fraught.
>
>Strange, isn't it: Microsoft must have access to at least a couple of
>orders of magnitude greater development resources than that available to
>the Linux kernel project,

Almost certainly the inverse is true. Indeed, this is _why_, if
I were MSFT, I would be looking at how I could leverage the rest
of the world's investment in Linux while minimizing my own cost
for maintaining Windows.

>yet they cannot keep up with what the Linux developers have achieved.

Don't sell Microsoft short: their kernel engineers are
first-rate and incredibly sharp. But they only have so many,
and regardless of how many they have, they have fewer than
Linux.

On the other hand, don't look at Linux with rose-colored
glasses. The idea of the lone Linux kernel hacker independently
building something that rivals commercial vendors with nothing
more than a bag of dorittos and bottle of Jolt is a distant
memory. The reality is that _most_ Linux development is heavily
subsidized by commercial interests these days; getting something
non-trivial into the kernel requires lots of time and resources.
In theory an individual could do it, but in practice that's
pretty rare.

- Dan C.

In article <kvjaabF2supU1@mid.individual.net>,
bill <bill.gunshannon@gmail.com> wrote:
>On 1/2/2024 2:42 PM, Dan Cross wrote:
>> In article <kvj3t3Fbk87U1@mid.individual.net>,
>> bill <bill.gunshannon@gmail.com> wrote:
>>> On 1/2/2024 12:41 PM, Dan Cross wrote:
>>>> In article <umvlnj$2cmas$1@dont-email.me>,
>>>> Arne Vajhøj <arne@vajhoej.dk> wrote:s.
>>>>
>>>>> And they have known about both concepts for a long
>>>>> time. WSL1 concept goes back to 90's - Win32 API on
>>>>> top of NT kernel and Win9x kernel.
>>>>
>>>> Not really, but the basic concept of emulating a system
>>>> interface on another system goes much further back than
>>>> that; e.g., PA1050 on TOPS-20, or the DTSS environment on
>>>> Multics.
>>>
>>> And lets not forget The Software Tools Virtual Operating System
>>> in 1980 that gave you a bunch of the Unix API on everything from
>>> CP/M to Exec-8 on the Univac 1100 including all of the DEC systems
>>> of the time. :-)
>>
>> Indeed! Though that was more about a suite of useful
>> functionality for source-level compatibility, more than running
>> unmodified binaries on compatible machines.
>
>One of the big "selling" points for Unix was always source
>compatibility. In the early Linux days there were compatibility
>compatibility libraries that were supposed to make Linux stuff
>run on BSD. I never had much luck with it and usually found it
>easier to just recompile under BSD.

Yeah. It's much easier to take source and get it to run
somewhere else, though increasingly often I find Linux
assumptions baked into a lot of code. But Linux is the big boy
on the block, so they can get away with it.

I similarly found the Linux binary compatibility stuff in BSD
did a mediocre job. Similarly with other compability things
over the years: I brought up Torek's port of 4.4BSD-Lite on a
SPARCstation 1 by backporting some bits from -Encumbered, and
while it _could_ run SunOS 4.x binaries, they dumped core
disturbingly often.

- Dan C.

On Tue, 2 Jan 2024 20:56:56 -0000 (UTC), Dan Cross wrote:

> In article <un1s48$2r0sq$1@dont-email.me>,
> Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
>
>>On Tue, 2 Jan 2024 19:41:21 -0000 (UTC), Dan Cross wrote:
>>
>>> Furthermore, the rate of change in Linux is high; following along from
>>> outside is fraught.
>>
>>Strange, isn't it: Microsoft must have access to at least a couple of
>>orders of magnitude greater development resources than that available to
>>the Linux kernel project,
>
> Almost certainly the inverse is true.

Yet you yourself have already admitted the opposite.

> Don't sell Microsoft short: their kernel engineers are first-rate and
> incredibly sharp.

Maybe not quite so sharp; else why can’t they figure out how to get
select(2)/poll(2) working with pipes <https://docs.python.org/3/library/
select.html>? That is the one irritating reason why users of Python
asyncio on Windows have to make a choice between two different, partially-
overlapping event-loop implementations, neither of which is really a
general solution <https://docs.python.org/3/library/asyncio-
eventloop.html#event-loop-implementations>.

> On the other hand, don't look at Linux with rose-colored glasses. The
> idea of the lone Linux kernel hacker independently building something
> that rivals commercial vendors with nothing more than a bag of dorittos
> and bottle of Jolt is a distant memory.

It still happens. Look at who created WireGuard, just for a recent
example.

In article <un1s48$2r0sq$1@dont-email.me>,
Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
>On Tue, 2 Jan 2024 19:41:21 -0000 (UTC), Dan Cross wrote:
>
>> Furthermore, the rate of change in Linux is high; following along from
>> outside is fraught.
>
>Strange, isn’t it: Microsoft must have access to at least a couple of
>orders of magnitude greater development resources than that available to
>the Linux kernel project, yet they cannot keep up with what the Linux
>developers have achieved.

This presumes that the high rate of change of Linux is actually good. I
rather suspect it is not.
--scott
--
"C'est un Nagra. C'est suisse, et tres, tres precis."

On 2 Jan 2024 22:05:31 -0000, Scott Dorsey wrote:

> In article <un1s48$2r0sq$1@dont-email.me>,
> Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
>
>>On Tue, 2 Jan 2024 19:41:21 -0000 (UTC), Dan Cross wrote:
>>
>>> Furthermore, the rate of change in Linux is high; following along from
>>> outside is fraught.
>>
>>Strange, isn’t it: Microsoft must have access to at least a couple of
>>orders of magnitude greater development resources than that available to
>>the Linux kernel project, yet they cannot keep up with what the Linux
>>developers have achieved.
>
> This presumes that the high rate of change of Linux is actually good. I
> rather suspect it is not.

You can think in terms of passive users of a project, versus active
contributors to the project.

In the open-source world, those passive users don’t matter so much: the
ones that matter are the active contributors.

The passive users also include the type who like to complain about things
and bemoan the fact that the project doesn’t cater to their needs.

In article <un1uj4$2raer$1@dont-email.me>,
Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
>On Tue, 2 Jan 2024 20:56:56 -0000 (UTC), Dan Cross wrote:
>
>> In article <un1s48$2r0sq$1@dont-email.me>,
>> Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
>>
>>>On Tue, 2 Jan 2024 19:41:21 -0000 (UTC), Dan Cross wrote:
>>>
>>>> Furthermore, the rate of change in Linux is high; following along from
>>>> outside is fraught.
>>>
>>>Strange, isn't it: Microsoft must have access to at least a couple of
>>>orders of magnitude greater development resources than that available to
>>>the Linux kernel project,
>>
>> Almost certainly the inverse is true.
>
>Yet you yourself have already admitted the opposite.

I don't believe I have. Perhaps you could tell me where you
think I did?

>> Don't sell Microsoft short: their kernel engineers are first-rate and
>> incredibly sharp.
>
>Maybe not quite so sharp;

I've worked closely with at least six engineers who worked on
the Windows kernel. I can assure you that they are excellent,
and based on what they told me, they were representative of
most Windows kernel people.

I recongize that's anecdotal, but I see no disconfirming
evidence.

>else why can’t they figure out how to get
>select(2)/poll(2) working with pipes <https://docs.python.org/3/library/
>select.html>?

Probably because they started out with a very different system
architecture and din't care about implementing Unix interfaces
at the time, and now have to retrofit it onto a very large
existing codebase and that's kind of a niche use case.

I'm curious: have you ever actually had a serious technical
conversation with someone who's worked on Windows? Did you ask
them this question, and if so, what was their answer?

>That is the one irritating reason why users of Python
>asyncio on Windows have to make a choice between two different, partially-
>overlapping event-loop implementations, neither of which is really a
>general solution <https://docs.python.org/3/library/asyncio-
>eventloop.html#event-loop-implementations>.

When you write that about select/poll and pipes and that being
an "irritating reason" for a more general issue, that seems
specious. Different systems do things differently; sadly Python
exposes that to the programmer. Oh well, but that's about par
for the course.

>> On the other hand, don't look at Linux with rose-colored glasses. The
>> idea of the lone Linux kernel hacker independently building something
>> that rivals commercial vendors with nothing more than a bag of dorittos
>> and bottle of Jolt is a distant memory.
>
>It still happens. Look at who created WireGuard, just for a recent
>example.

What about WireGuard? The implementation was funded.

Anyway, anecdotal evidence, at best, I'm afraid. I'm using
conversations with primary sources for my assertion (Ts'o et
al).

- Dan C.

In article <un21bb$7ea$1@panix2.panix.com>,
Scott Dorsey <kludge@panix.com> wrote:
>In article <un1s48$2r0sq$1@dont-email.me>,
>Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
>>On Tue, 2 Jan 2024 19:41:21 -0000 (UTC), Dan Cross wrote:
>>
>>> Furthermore, the rate of change in Linux is high; following along from
>>> outside is fraught.
>>
>>Strange, isn’t it: Microsoft must have access to at least a couple of
>>orders of magnitude greater development resources than that available to
>>the Linux kernel project, yet they cannot keep up with what the Linux
>>developers have achieved.
>
>This presumes that the high rate of change of Linux is actually good. I
>rather suspect it is not.

I think there's a lot of churn. What's missing is a lot of
principled engineering; Linux has evolved, and it's impressive
what it has evolved into (honestly the world's most important
operating system) but that doesn't necessarily mean that it is
_good_ in all respects (it is objectively not).

Take, for instance, the lack of internal kernel interfaces and
gregkh's insistence that, not only are they unnecessary, they
are _bad_: this can make things very hard to work with, with
multiple generations of partial interfaces all coresident at the
same time, and little incentive to clean up the resulting
complexity or mess. Honestly, I'm often surprised that it works
as well as it does. Compare to Solaris, on the other hand, that
has documented stable interfaces all over. Sure, there are some
cases where they're not used (or used well...) but this means
that binary modules compiled out-of-tree can work across
multiple versions of the operating system, which is pretty cool.
And neigh impossible for non-trivial things in the Linux world.

- Dan C.

On Tue, 2 Jan 2024 23:16:45 -0000 (UTC), Dan Cross wrote:

> In article <un1uj4$2raer$1@dont-email.me>,
> Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
>
>>On Tue, 2 Jan 2024 20:56:56 -0000 (UTC), Dan Cross wrote:
>>
>>> In article <un1s48$2r0sq$1@dont-email.me>,
>>> Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
>>>
>>>>On Tue, 2 Jan 2024 19:41:21 -0000 (UTC), Dan Cross wrote:
>>>>
>>>>> Furthermore, the rate of change in Linux is high; following along
from
>>>>> outside is fraught.
>>>>
>>>>Strange, isn't it: Microsoft must have access to at least a couple of
>>>>orders of magnitude greater development resources than that available
to
>>>>the Linux kernel project,
>>>
>>> Almost certainly the inverse is true.
>>
>>Yet you yourself have already admitted the opposite.
>
> I don't believe I have. Perhaps you could tell me where you
> think I did?

Where you used the word “fraught”.

>>else why can’t they figure out how to get
>>select(2)/poll(2) working with pipes <https://docs.python.org/3/library/
>>select.html>?
>
> Probably because they started out with a very different system
> architecture and din't care about implementing Unix interfaces
> at the time, and now have to retrofit it onto a very large
> existing codebase and that's kind of a niche use case.

That “niche” is the reason why they have had to resort to WSL2, to bring
Linux-type APIs to Windows. And why do they need Linux-type APIs on
Windows, anyway? Because that’s what the developers are increasingly
relying on. Why didn’t WSL1 work? Because the Windows kernel wasn’t up to
it.

> When you write that about select/poll and pipes and that being
> an "irritating reason" for a more general issue, that seems
> specious.

Maybe you don’t realize how important Python has become in the computing
world lately, with its applications in data science and other areas.
Microsoft is even planning to offer Python access to Excel users (at least
cloud-based ones).

And it’s not just Python, of course: Windows limitations affect cross-
platform development across the board. To the point where Microsoft has to
do something to stem the tide of developers simply giving up on the
Windows platform. Hence WSL. Which is essentially a bag duct-taped on the
side of Windows.

Remember why Microsoft needs WSL, clunky as it is: it’s not something it
bestowed as a special favour on the Linux or open-source world or anything
like that: it created it because it had to, for sheer business survival.

In article <un26bd$2sllq$2@dont-email.me>,
Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
>On Tue, 2 Jan 2024 23:16:45 -0000 (UTC), Dan Cross wrote:
>
>> In article <un1uj4$2raer$1@dont-email.me>,
>> Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
>>
>>>On Tue, 2 Jan 2024 20:56:56 -0000 (UTC), Dan Cross wrote:
>>>
>>>> In article <un1s48$2r0sq$1@dont-email.me>,
>>>> Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
>>>>
>>>>>On Tue, 2 Jan 2024 19:41:21 -0000 (UTC), Dan Cross wrote:
>>>>>
>>>>>> Furthermore, the rate of change in Linux is high; following along
>from
>>>>>> outside is fraught.
>>>>>
>>>>>Strange, isn't it: Microsoft must have access to at least a couple of
>>>>>orders of magnitude greater development resources than that available
>to
>>>>>the Linux kernel project,
>>>>
>>>> Almost certainly the inverse is true.
>>>
>>>Yet you yourself have already admitted the opposite.
>>
>> I don't believe I have. Perhaps you could tell me where you
>> think I did?
>
>Where you used the word "fraught".

I don't see it.

What I said is that it is a fraught proposition to try to keep
up with the rate of change in the Linux kernel from outside of
the Linux kernel.

On the other hand, you said,

|Microsoft must have access to at least a couple
|of orders of magnitude greater development resources than that
|available to the Linux kernel project

To which I replied that almost certainly the inverse is true;
i.e., that the Linux kernel folks probably have access to
several orders of magnitude more resources than Microsoft can
put to Windows.

The two are unrelated.

>>>else why can’t they figure out how to get
>>>select(2)/poll(2) working with pipes <https://docs.python.org/3/library/
>>>select.html>?
>>
>> Probably because they started out with a very different system
>> architecture and din't care about implementing Unix interfaces
>> at the time, and now have to retrofit it onto a very large
>> existing codebase and that's kind of a niche use case.
>
>That "niche" is the reason why they have had to resort to WSL2, to bring
>Linux-type APIs to Windows. And why do they need Linux-type APIs on
>Windows, anyway? Because that's what the developers are increasingly
>relying on. Why didn't WSL1 work? Because the Windows kernel wasn't up to
>it.

That seems like speculation on your part. Do you have any
special expertise in this area?

They surely moved to WSL2 because the best, cheapest way to be
"bug compatible" with Linux is to just run Linux. That really
doesn't say much about what the Windows kernel is, or is not,
capable of. It might say something about their development
priorities, though.

>> When you write that about select/poll and pipes and that being
>> an "irritating reason" for a more general issue, that seems
>> specious.
>
>Maybe you don't realize how important Python has become in the computing
>world lately, with its applications in data science and other areas.

Non sequitur. Python is important; this has does not imply that
Unix-style select/poll on a pipe under Windows in Python is
important. For more general types of asynchronous IO (storage,
networking) it says nothing at all, and these are certainly more
important than pipes which are just one kind of IPC mechanism
(and not super relevant to Python specifically).

>Microsoft is even planning to offer Python access to Excel users (at least
>cloud-based ones).

Again, this is conflating "python" (which I understand runs on
Windows just fine; I wouldn't know, as I don't do Windows) with
pipes+select and python.

>And it's not just Python, of course: Windows limitations affect cross-
>platform development across the board. To the point where Microsoft has to
>do something to stem the tide of developers simply giving up on the
>Windows platform. Hence WSL. Which is essentially a bag duct-taped on the
>side of Windows.

This seems like pure speculation.

>Remember why Microsoft needs WSL, clunky as it is: it's not something it
>bestowed as a special favour on the Linux or open-source world or anything
>like that: it created it because it had to, for sheer business survival.

I actually agree with this, but I think your argument here is
poor. MSFT needs Linux compatability because the world is
trending towards Linux and they can't keep up, yes. It does not
follow that their engineers are bad, or even that their kernel
is bad. It does mean that their kernel was designed with
different goals than Linux (which is a Unix knock-off).

- Dan C.

On 1/2/2024 2:25 PM, Andy Burns wrote:
> Lawrence D'Oliveiro wrote:
>> And, in spite of the fact that it was supposed to be implementing a well-
>> documented API, with plenty of example source code to refer to, they
>> still
>> couldn’t get WSL1 to work right. So they had to chuck it away and
>> bring in
>> a proper Linux kernel instead.
>
> I didn't do anything other than "play" with WSL1, but I thought
> performance was the issue.
>
> I think I did more with MKS Toolkit and Cygwin than WSL.

Note that Cygwin and WSL does completely different things.

Cygwin:

*nix and/or Windows source--Cygwin toolchain-->Windows executable

WSL:

*nix source--Linux toolchain-->Linux executable

Arne