That example I posted elsewhere (a function returning a function) relies
on lexical binding in order to work properly.

Is lexical binding accepted as a standard part of Common Lisp yet? I know
it was built in from the start in Scheme. But some in the older Lisp
community still seem to think dynamic binding is useful.

On 1/14/2024 2:04 PM, Lawrence D'Oliveiro wrote:
> That example I posted elsewhere (a function returning a function) relies
> on lexical binding in order to work properly.
>
> Is lexical binding accepted as a standard part of Common Lisp yet? I know
> it was built in from the start in Scheme. But some in the older Lisp
> community still seem to think dynamic binding is useful.

If by "Common Lisp" you mean the language that we all think you are
talking about then lexical binding is very much apart of it and the
default. That does not change the fact that dynamic (or special)
bindings are very useful. That is why Common Lisp offers both. Having
both is one of the really really good features of the language.
--
Jeff Barnett

Lawrence D'Oliveiro <ldo@nz.invalid> writes:

Is lexical binding accepted as a standard part of Common Lisp yet?

Lexical binding has been the default behavior of Common Lisp from its
very beginning.

In the early history of LISP there was some experimentation with other
behaviors. But by the late 70s, well before Common Lisp was even
conceived, everybody was convinced that lexical binding was the way to
go. So your use of the word "yet" is hilarious.

I will never understand now these myths perpetuate. There are still
people who think that Lisp is always interpreted, despite the fact that
we've been compiling to machine code since the days of LISP 1.5 in
around 1960!

- Alan

On 2024-01-14, Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
> That example I posted elsewhere (a function returning a function) relies
> on lexical binding in order to work properly.
>
> Is lexical binding accepted as a standard part of Common Lisp yet?

Common Lisp standardization concluded in 1994 and has not reconvened.

Lexical binding for variables was standardized as the default mode.

Dynamic binding is requested in several ways:

- if the symbol is marked for dynamic binding, that is used.

- declare can be used to request dynamic binding for symbols
not marked for dynamic binding.

Symbols that are the target of a defvar or defparameter form become
marked for dynamic binding. Most of the time, that is the preferred
way to introduce a dynamic variable: to define it globally via a
top-level defvar or defparameter form and make sure that is processed
before any code that locally bind it. Make sure your compiler
sees your (defvar *foo*) before it sees any (let ((*foo* ...)) ...)
expression, otherwise it will wrongly compile that as lexical
binding.

> I know
> it was built in from the start in Scheme. But some in the older Lisp
> community still seem to think dynamic binding is useful.

It is seen as useful in the Scheme community, where it gets reinvented
as fluid-let (SRFI 15) or the really bizarre concoction of "parameter
objects" (SRFI 39).

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.

Alan Bawden <alan@csail.mit.edu> writes:
> I will never understand now these myths perpetuate.

Maybe people confuse Common Lisp with Emacs Lisp, which was historically
purely dynamically scoped. I don't know if it is still that way, not
counting Guile Emacs. I never used Maclisp or Lisp 1.5 so I don't know
how their scoping worked. But dynamic scope is convenient for
simple-minded implementations.

Now you have me wondering about the scoping rules of Church's lambda
calculus, or if the concept even applies. At first impression it seems
lexical.

On Sun, 14 Jan 2024 20:22:15 -0800, Paul Rubin wrote:

> Now you have me wondering about the scoping rules of Church's lambda
> calculus, or if the concept even applies. At first impression it seems
> lexical.

It wouldn’t have worked otherwise.

Paul Rubin <no.email@nospam.invalid> writes:

... I never used Maclisp or Lisp 1.5 so I don't know how their
scoping worked.

MacLisp was weird. You typically debugged your program using the
MacLisp interpreter because it made the debugging cycle faster, and the
interpreter was purely dynamicly scoped. But when the MacLisp compiler
compiled your code, all the variables you hadn't declared SPECIAL became
lexical! I know that sounds crazy, but because MacLisp didn't really
support closures, it wasn't too hard to write code in a style such that
it didn't really matter whether variables were dynamic or lexical.

Assuming that the interpreter in the appendix of the Lisp 1.5 manual, I
would say that Lisp 1.5 was lexical. But I'm not 100% certain that that
interpreter really reflects the system's true semantics. (Does the
source code for the Lisp 1.5 system still exist anywhere?)

Now you have me wondering about the scoping rules of Church's lambda
calculus, or if the concept even applies. At first impression it seems
lexical.

It's definitely lexical, but you are right to wonder if the distinction
even applies. Our notion of "dynamic" scoping seems pretty closely tied
to the workings of the typical applicative order Lisp evaluator. I
imagine Church himself would be pretty puzzled by our notion of dynamic
scoping...

- Alan

On Mon, 15 Jan 2024 02:29:28 -0500, Alan Bawden wrote:

> Assuming that the interpreter in the appendix of the Lisp 1.5 manual, I
> would say that Lisp 1.5 was lexical.

I recall in McCarthy’s memoir of the origins of Lisp, he thought the
scoping issue (the “FUNARG problem”, I think he called it) was a simple
coding bug, and got one of his grad students to fix it.

Lawrence D'Oliveiro <ldo@nz.invalid> writes:
>I recall in McCarthy’s memoir of the origins of Lisp, he thought the
>scoping issue (the “FUNARG problem”, I think he called it) was a simple
>coding bug, and got one of his grad students to fix it.

Stoyan wrote:

|McCarthy admits that he underestimated the problem. "I must
|confess that I regarded this difficulty as just a bug and
|expressed confidence that Steve Russell would soon fix it..."

. Stoyan commented:

|The funarg event proves that McCarthy was now far from his
|LISP. He won the theoretical battle und searched for the next
|field. The mathematical theory of computation, time sharing,
|and proof checking were his new interests. LISP lost first
|priority.

.

On 2024-01-15, Alan Bawden <alan@csail.mit.edu> wrote:
> Paul Rubin <no.email@nospam.invalid> writes:
>
> ... I never used Maclisp or Lisp 1.5 so I don't know how their
> scoping worked.
>
> MacLisp was weird. You typically debugged your program using the
> MacLisp interpreter because it made the debugging cycle faster, and the
> interpreter was purely dynamicly scoped. But when the MacLisp compiler
> compiled your code, all the variables you hadn't declared SPECIAL became
> lexical! I know that sounds crazy, but because MacLisp didn't really
> support closures, it wasn't too hard to write code in a style such that
> it didn't really matter whether variables were dynamic or lexical.
>
> Assuming that the interpreter in the appendix of the Lisp 1.5 manual, I
> would say that Lisp 1.5 was lexical. But I'm not 100% certain that that
> interpreter really reflects the system's true semantics. (Does the
> source code for the Lisp 1.5 system still exist anywhere?)
>
> Now you have me wondering about the scoping rules of Church's lambda
> calculus, or if the concept even applies. At first impression it seems
> lexical.
>
> It's definitely lexical, but you are right to wonder if the distinction
> even applies. Our notion of "dynamic" scoping seems pretty closely tied
> to the workings of the typical applicative order Lisp evaluator. I
> imagine Church himself would be pretty puzzled by our notion of dynamic
> scoping...

Even though LC is lexical, I suspect that examples of LC that break
under dynamic scope have to be either incorrect, or convoluted.

What I'm referring to by "incorrect" is the presence of unbound
references. Some function (lambda (x) y) occurs where y is not
bound to anything, and that is then passed and called somewhere where y
is dynamically bound. This "breaks" in the sense that it's ill
formed under normal LC, but dynamic LC accepts it.

The other cases deal with references to the wrong parameter.

Wrong references happen in two ways:

- the same identifier is reused for two different purposes, which
"cross wires" due to the scoping.

- the same variable is activated multiple times, which are expected
to be different instances but aren't.

The first instance is easier to first show with bound functions,
like:

(define fun (x) (x x))

(lambda (x)
(fun (lambda (y) x)))

Where inside fun, the x references are to its own parameter,
so the (lambda (y) x) gets confused. We can get rid of the define:

(lambda (x)
((lambda (x) (x x)) (lambda (y) x)))

OK, but when we do this, this becomes confusing regardless of
the scoping discipline, due to the reuse of the parameter name!
All the variables in the example are singly instantiated, so
the only problem is the naming; if we make it:

(lambda (z)
((lambda (x) (x x)) (lambda (y) z)))

the problem goes away and it now means the same thing under
dynamic scope.

The remaining category is problems where the LC breaks because a the
same variable is instantiated multiple times, and those instantiations
are not separately being captured by a lambda, since there is
actually only one variable.

In Lisp, it's easy to come up with examples: such as the difference
between a loop construct which steps the dummy variable versus one
which freshly binds it.

People are running into this even in Blub languages. E.g. Go language
recently fixed it foreach-type looping construct to bind a fresh
variable, under the belief that this is what programmers expect.

Under dynamic scope, it won't make a difference. You can pretend to bind
a fresh variable, but it's really equivalent to stepping, since there is
only one variable. In Common Lisp, if our implementation has a DOLIST
loop that freshly binds, but we use a special variable as the dummy,
then that is reduced to being equivalent to mutating DOLIST.

It's not obvious to me how to make a simple example of this in lambda
calculus. You don't have loops that can step a variable versus bind.
You have to use recursion in order to evaluate the same functions
multiple times, with different values of free variables that have
to be demonstrably captured.

I can see why it might not have been obvious to MacCarthy that saving
and restoring variables isn't enough for lambda-calculus-like
computation.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.

Kaz Kylheku <433-929-6894@kylheku.com> writes:

Even though LC is lexical, I suspect that examples of LC that break
under dynamic scope have to be either incorrect, or convoluted.

I'm having trouble understanding what you wrote here because I can't
figure out for the life of me what you mean by "LC". At first I thought
you meant LC as short for Lambda Calculus, but later you start talking
about "normal LC" vs. "dynamic LC", and since I have no idea what
"dynamic lambda calculus" could be, I'm stumped.

- Alan

ram@zedat.fu-berlin.de (Stefan Ram) writes:

Stoyan wrote:

|McCarthy admits that he underestimated the problem. "I must
|confess that I regarded this difficulty as just a bug and
|expressed confidence that Steve Russell would soon fix it..."

Stoyan commented:

|The funarg event proves that McCarthy was now far from his
|LISP. He won the theoretical battle und searched for the next
|field. The mathematical theory of computation, time sharing,
|and proof checking were his new interests. LISP lost first
|priority.

It's always important to keep in mind that McCarthy didn't add LAMBDA to
Lisp because he was trying to build a programming language based on
Lambda Calculus. What he _was_ trying to do was build a programming
language based on first order predicate calculus. He introduced LAMBDA
because he needed it in order to define recursive functions. I can't
find it right now, but someplace in his writings he apologized for the
fact that LAMBDA made it possible to write code that was definitely
_not_ first order, which is not something that he intended!

McCarthy was a really smart guy. So smart that some of his mistakes are
more interesting than many other smart people's best ideas.

- Alan

Alan Bawden <alan@csail.mit.edu> writes:
>Kaz Kylheku <433-929-6894@kylheku.com> writes:
> Even though LC is lexical, I suspect that examples of LC that break
> under dynamic scope have to be either incorrect, or convoluted.

In Usenet, I prefer to use ">" to mark quotations, not indentation.

"LC" could also be "Lisp code".

On 1/14/2024 9:22 PM, Paul Rubin wrote:
> Alan Bawden <alan@csail.mit.edu> writes:
>> I will never understand now these myths perpetuate.
>
> Maybe people confuse Common Lisp with Emacs Lisp, which was historically
> purely dynamically scoped. I don't know if it is still that way, not
> counting Guile Emacs. I never used Maclisp or Lisp 1.5 so I don't know
> how their scoping worked. But dynamic scope is convenient for
> simple-minded implementations.
>
> Now you have me wondering about the scoping rules of Church's lambda
> calculus, or if the concept even applies. At first impression it seems
> lexical.

In the old days, Lisps were LOCALLY scoped and included special
variables (dynamic scoped) too. Closures were available in some
implementations for a specified set of special variables. N.B. that by
LEXICAL scope we normally mean that nested function definitions see
local bindings where ever they are executed - not so for LOCAL binding.

Lexical bindings were desired for most of us but we neither had the
address space or memory to make effectively fast, usable
implementations. ALGOL had enough extra structure rules and no specials
so could do LEXICAL as shown by the Boroughs (sp?) Algol Machines that
existed decades before the various Lisp machines and were primary used
in the Banking business.
--
Jeff Barnett

On Mon, 15 Jan 2024 17:25:08 -0700, Jeff Barnett wrote:

> Lexical bindings were desired for most of us but we neither had the
> address space or memory to make effectively fast, usable
> implementations.

Basically, call frames (or parts of them) go on the heap. I think Python
manages some efficiency gains by only including referenced variables.

On 2024-01-15, Alan Bawden <alan@csail.mit.edu> wrote:
> Kaz Kylheku <433-929-6894@kylheku.com> writes:
>
> Even though LC is lexical, I suspect that examples of LC that break
> under dynamic scope have to be either incorrect, or convoluted.
>
> I'm having trouble understanding what you wrote here because I can't
> figure out for the life of me what you mean by "LC". At first I thought
> you meant LC as short for Lambda Calculus, but later you start talking
> about "normal LC" vs. "dynamic LC", and since I have no idea what
> "dynamic lambda calculus" could be, I'm stumped.

It has been observed that a lot of Lisp code works fine if we subsitute
dynamic binding for lexical.

So we can translate that into the lambda calculus domain. We need to
imagine a lambda calculus that has dynamic binding.

Some regular lambda calculus examples would still work. Some that don't
work could be repaired just by renaming clashing variables.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.

On Tue, 16 Jan 2024 01:29:06 -0000 (UTC), Kaz Kylheku wrote:

> It has been observed that a lot of Lisp code works fine if we subsitute
> dynamic binding for lexical.

I tend to use function factories a lot. And Python allows class factories
as well. None of that would work.

On 1/15/2024 5:37 PM, Lawrence D'Oliveiro wrote:
> On Mon, 15 Jan 2024 17:25:08 -0700, Jeff Barnett wrote:
>
>> Lexical bindings were desired for most of us but we neither had the
>> address space or memory to make effectively fast, usable
>> implementations.
>
> Basically, call frames (or parts of them) go on the heap. I think Python
> manages some efficiency gains by only including referenced variables.

The optimization code itself needed to fit in small computer
environments. The early Lisp 1.5s, for the most part, were developed on
computers with a few hundred thousand bytes (in today's memory units)
with no virtually addressing and the later ones more or less followed
with compatible languages on newer hardware.

To see the effect of such memory limitations, consider the GC:
structures were folded or compacted and under the table erasure lists
were used. There wasn't enough address space or memory to do copy
collects and flips.

The technology was understood but the hardware wasn't up to it.
--
Jeff Barnett

Kaz Kylheku [2024-01-15 01:16:10] wrote:
> It is seen as useful in the Scheme community, where it gets reinvented
> as fluid-let (SRFI 15) or the really bizarre concoction of "parameter
> objects" (SRFI 39).

Most of SRFI-39 strikes me as intuitive rather than bizarre (it's
basically plain-old dynamic scoping just using objects instead of
symbols to "name" the "variables"), but I do find the `converter` part
rather odd.

Does anyone know why that `converter` was added to SRFI 39?

I also can't see how to define something equivalent to `progv` on top of
SRFI-39.

Stefan

> In the old days, Lisps were LOCALLY scoped and included special variables
> (dynamic scoped) too. Closures were available in some implementations for
> a specified set of special variables. N.B. that by LEXICAL scope we normally
> mean that nested function definitions see local bindings where ever they are
> executed - not so for LOCAL binding.

See for example the definition of "closures" in the context of
Lisp-Machine-Lisp at https://hanshuebner.github.io/lmman/fd-clo.xml#closure

Basically, they're functions that remember the value of dynbound
variables at the time that the closure was created and then rebind those
dynbound vars to those values around the evaluation of their body.

In ELisp you could implement it as follows:

(oclosure-define (lml-closure
(:predicate lml-closurep))
bindings function)

(defun lml-closure (varlist function)
"Create a \"closure\" over the dynamic variables in VARLIST."
(oclosure-lambda
(lml-closure (bindings (mapcar (lambda (v) (cons v (symbol-value v)))
varlist))
(function function))
(&rest args)
(cl-progv (mapcar #'car bindings) (mapcar #'cdr bindings)
(apply function args))))

-- Stefan

On 1/16/2024 4:16 PM, Stefan Monnier wrote:
>> In the old days, Lisps were LOCALLY scoped and included special variables
>> (dynamic scoped) too. Closures were available in some implementations for
>> a specified set of special variables. N.B. that by LEXICAL scope we normally
>> mean that nested function definitions see local bindings where ever they are
>> executed - not so for LOCAL binding.
>
> See for example the definition of "closures" in the context of
> Lisp-Machine-Lisp at https://hanshuebner.github.io/lmman/fd-clo.xml#closure

Note that above I was talking about Lisps that predated the Lisp Machines.

> Basically, they're functions that remember the value of dynbound
> variables at the time that the closure was created and then rebind those
> dynbound vars to those values around the evaluation of their body.

There was no "rebinding". Rather the original binding was used by all
who could see it vis a vis the scoping rules. Since multiple closures
(including the one where that binding occurred) could reference the same
lexical binding, creating multiple rebinding would screw the semantics.

> In ELisp you could implement it as follows:
>
> (oclosure-define (lml-closure
> (:predicate lml-closurep))
> bindings function)
>
> (defun lml-closure (varlist function)
> "Create a \"closure\" over the dynamic variables in VARLIST."
> (oclosure-lambda
> (lml-closure (bindings (mapcar (lambda (v) (cons v (symbol-value v)))
> varlist))
> (function function))
> (&rest args)
> (cl-progv (mapcar #'car bindings) (mapcar #'cdr bindings)
> (apply function args))))--
Jeff Barnett

On 2024-01-17, Jeff Barnett <jbb@notatt.com> wrote:
> On 1/16/2024 4:16 PM, Stefan Monnier wrote:
>>> In the old days, Lisps were LOCALLY scoped and included special variables
>>> (dynamic scoped) too. Closures were available in some implementations for
>>> a specified set of special variables. N.B. that by LEXICAL scope we normally
>>> mean that nested function definitions see local bindings where ever they are
>>> executed - not so for LOCAL binding.
>>
>> See for example the definition of "closures" in the context of
>> Lisp-Machine-Lisp at https://hanshuebner.github.io/lmman/fd-clo.xml#closure
>
> Note that above I was talking about Lisps that predated the Lisp Machines.
>
>> Basically, they're functions that remember the value of dynbound
>> variables at the time that the closure was created and then rebind those
>> dynbound vars to those values around the evaluation of their body.
>
> There was no "rebinding". Rather the original binding was used by all
> who could see it vis a vis the scoping rules. Since multiple closures
> (including the one where that binding occurred) could reference the same
> lexical binding, creating multiple rebinding would screw the semantics.

By the way, I once made a (proprietary, closed source) Lisp dialect
which had dynamic scope, but with proper closures!

The dialect used deep binding for the dynamic scope. Basically it's
the same as lexical scope in a recursive interpreter, except that the
environment chain crosses the function call boundary.

Closures worked by actually walking the environment chain and capturing
it.

To make the closures lexical, the function activation frames had a
special marker in them to identify them. Thus the dynamic closure
capture mechanism was able to stop at that marker and not capture
beyond.

When calling a closure, the captured environment got spliced in, so that
the search for a dynamic variable would look at the function parameters
first, then the captured environment, then the calling environment.

This dialect also had threads.

I remember it was quite nice to be able to override special variables
around lambdas that executed in separate thread.

(let* ((*some-param* (whatever))
(thr (create-thread (lambda ...))))
...)

I was using that for testing API's on an embedded target, where the
some of the test functions could be controlled by various special
variable parameters.

I /think/ I may also have had dlambda construct that captured the
full dynamic scope across the function boundary, so this was possible:

(defun foo ()
(dlambda () ...))

The returned lambda would see bindings of dynamic variables as they
existed at the time foo was called.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.

On Tue, 16 Jan 2024 18:16:02 -0500, Stefan Monnier wrote:

> In ELisp you could implement it as follows ...

Luckily, you can now specify it for the contents of a .el file with a
header comment:

;;; -*- lexical-binding: t; -*-

On Wed, 17 Jan 2024 20:37:43 -0000 (UTC), Kaz Kylheku wrote:

> By the way, I once made a (proprietary, closed source) Lisp dialect
> which had dynamic scope, but with proper closures!

I’m just a humble Python programmer, but it seems to me there are easier
ways of doing such things: create and instantiate a class.

Then you can define methods that access and update the internal state,
coupled with a method that says “do the actual work”. You can even manage
that internal state via assignable “properties”, instead of explicit
getter/setter method calls.

And for added flavour, if you define a “__call__()” method that does the
actual work, you can call the class instance as though it were a function.

On 2024-01-17, Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
> On Wed, 17 Jan 2024 20:37:43 -0000 (UTC), Kaz Kylheku wrote:
>
>> By the way, I once made a (proprietary, closed source) Lisp dialect
>> which had dynamic scope, but with proper closures!
>
> I’m just a humble Python programmer, but it seems to me there are easier
> ways of doing such things: create and instantiate a class.

Whether or not that is true, that dialect didn't evolve to the point
it had objects. I quit that company and trained for a marathon for 3
months instead of working anywhere. 5 days a week of weights, 6 days
of running. I hit a time of 3 hours and 19 minutes, with no natural
talent or anything, and well over thirty.

I caught word from the manager that he also bailed two weeks later
and the project folded.

While I was training for the marathon, the resumes I had sent out at the
beginning of the three month period started to gain traction. I got hired at a
new place, and my starting day was exactly the Monday after the Sunday marathon
race.

Good times!

> Then you can define methods that access and update the internal state,
> coupled with a method that says “do the actual work”. You can even manage
> that internal state via assignable “properties”, instead of explicit
> getter/setter method calls.

I think all that would have been verbose compared to the dynamic closures,
but of course it provides tighter encapsulation.

> And for added flavour, if you define a “__call__()” method that does the
> actual work, you can call the class instance as though it were a function.

In TXR Lisp that is called lambda. No ugly underscores, because the
lambda symbol is properly packaged: you can have my:lambda in your own
package called "my" or whatever which doesn't clash with usr:lambda.

1> (defstruct functor ()
(:method lambda (me arg) (put-line `@me: @arg!`)))
#<struct-type functor>
2> (new functor)
#S(functor)
3> [*2 42]
#S(functor): 42!
t

There is also a lambda-set which will let the call expression
denote a place that can take assignments:

4> (defmeth functor lambda-set (me arg new-val)
(put-line `@me: @arg set to @{new-val}!`))
(meth functor
lambda-set)
5> (set [*2 42] :blah)
#S(functor): 42 set to :blah!
:blah

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.