Does someone know of an article or rationale explaining why `lambda` is
a macro, i.e. why the fully-expanded code needs to be (function (lambda
....)) rather than just (lambda ...)?

I can guess its origin as an accident of history, but I think there's
been enough years between the introduction of #' to fix the accident and
the design of Common Lisp to have it fixed, so I'm wondering if there
were other reasons to keep `lambda` as something that's relegated to
`function` as opposed to being a special-form.

Stefan

> * Stefan Monnier <zbaavre@veb.hzbagerny.pn> [2022-02-15 09:35:18 -0500]:
>
> Does someone know of an article or rationale explaining why `lambda` is
> a macro, i.e. why the fully-expanded code needs to be (function (lambda
> ...)) rather than just (lambda ...)?

Speaking as an implementor, a macro is "cheaper" than a special
operator, so _requiring_ something to be a special operator is a weird
thing to do unless there is a compelling reason for that.

I can implement a macro as a SO if I want to but I am not allowed to
implement a SO as a macro (IIRC).

--
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.2113
http://childpsy.net http://calmchildstories.com http://steingoldpsychology.com
http://think-israel.org https://ij.org/ https://iris.org.il https://camera.org
Let us remember that ours is a nation of lawyers and order.

Sam Steingold [2022-02-15 10:51:33] wrote:
>> * Stefan Monnier <zbaavre@veb.hzbagerny.pn> [2022-02-15 09:35:18 -0500]:
>> Does someone know of an article or rationale explaining why `lambda` is
>> a macro, i.e. why the fully-expanded code needs to be (function (lambda
>> ...)) rather than just (lambda ...)?
> Speaking as an implementor, a macro is "cheaper" than a special
> operator, so _requiring_ something to be a special operator is a weird
> thing to do unless there is a compelling reason for that.

But any Common Lisp implementation will need to have special handling of
(function (lambda ...)) and I fail to see how it would be cheaper than
moving that work to a (lambda ...) special form.

Stefan

On 2022-02-15, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> Does someone know of an article or rationale explaining why `lambda` is
> a macro, i.e. why the fully-expanded code needs to be (function (lambda
> ...)) rather than just (lambda ...)?

I've always understood this macro to be a compatibility device.

In Common Lisp, lambda is #'(lambda () ...); the macro removes the #'
dirt that doesn't appear in some other dialects, for better
compatibility.

If Common Lisp made lambda a special form, it would still need two
flavors of lambda.

If we have lambda as a special form here:

(mapcar (lambda (...) ...) ...)

where it is a form, we still need a story which explains what lambda is in

((lambda (...) ...) ...)

where it cannot be a special form, since it isn't a form.

The "unified theory of Common Lisp lambda" is that lambda is an expression,
which serves in place of a function name. Whereas a name like (setf foo)
or xyzzy denotes a function binding that exists elsewhere, the lambda
expression, in a name-like way, literally denotes the function described
by itself.

It's very tidy, except for the bit of #' dirt that appears on lambda; so
then the lambda macro is thrown in to appease those who don't want that.

And of course, (function (lambda () ...)) *is* a special form.
It has to do all the work of a (lambda () ...) special form in another
dialect; it's just dispatched under the locus of the function operator.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal

On 2022-02-15, Sam Steingold <sds@gnu.org> wrote:
>> * Stefan Monnier <zbaavre@veb.hzbagerny.pn> [2022-02-15 09:35:18 -0500]:
>>
>> Does someone know of an article or rationale explaining why `lambda` is
>> a macro, i.e. why the fully-expanded code needs to be (function (lambda
>> ...)) rather than just (lambda ...)?
>
> Speaking as an implementor, a macro is "cheaper" than a special
> operator, so _requiring_ something to be a special operator is a weird
> thing to do unless there is a compelling reason for that.

But you're already paying the cost anyway, because

(function (lambda (..) ...))

is a special form. And that has to exist; function is necessary because
of Lisp-2, and for consistency it is essential that lambda be
substitutable for name in (function name).

In light of that, you might not want two special forms; i.e.
that both (function (lambda (...) ...)) and (lambda (...) ...)
exist as special forms.

On the other hand, they can share most of their implementation.

When we are compiling (function ...) and see that the argument is
a lambda expression, we can just recurse into the lambda compiler
with that argument.

Still, there is a pressure to keep the special form count down, and it's
probably highly desirable in a standard.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal

> where it is a form, we still need a story which explains what lambda is in
>
> ((lambda (...) ...) ...)

Was it considered important to support this syntax? Why?

Stefan

On 2022-02-15, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>> where it is a form, we still need a story which explains what lambda is in
>>
>> ((lambda (...) ...) ...)
>
> Was it considered important to support this syntax? Why?

It's important for consistency. If a macro inserts a name into a form
like this:

`(,name ...)

it's nice if that doesn't break when name happens to be a lambda
expression. Ideally, lambda expression should be able to appear wherever
a function name can appear (in a way that makes sense; i.e. not when the
name is a binding being established or mutated.)

Immediately-called lambda is historically significant also, because it
provides a way of binding variables. The let construct came later, and
can be explained in terms of an immediately-applied anonymous function.
Almost needless to mention, such lambdas occur in lambda calculus.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal

Kaz Kylheku [2022-02-15 19:36:24] wrote:
> On 2022-02-15, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>>> where it is a form, we still need a story which explains what lambda is in
>>> ((lambda (...) ...) ...)
>> Was it considered important to support this syntax? Why?
> It's important for consistency. If a macro inserts a name into a form
> like this:
>
> `(,name ...)
>
> it's nice if that doesn't break when name happens to be a lambda
> expression.

I wonder why/when such a thing would happen. If `lambda` is a special
form, then `name` would either hold a function value or a form (whose
evaluation will return a function value), neither of which would
naturally include function names. And those cases using `(funcall
',name ...) resp. `(funcall ,name ...) would make more sense.

IOW I think the above expectation is a result of the use of #' for
lambda, rather than the other way around.

> Ideally, lambda expression should be able to appear wherever
> a function name can appear

I wonder if such a desire was part of the motivation of this design
choice, tho.

> Immediately-called lambda is historically significant also, because it
> provides a way of binding variables. The let construct came later, and
> can be explained in terms of an immediately-applied anonymous function.
> Almost needless to mention, such lambdas occur in lambda calculus.

Yes, as I said, I can see good reasons why this appeared in Lisp's
history, but I'm interested to know why Common Lisp *kept* this design.
I haven't found any hint that it was a conscious decision, so maybe it's
just been kept because noone considered changing it.

Stefan

Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> Does someone know of an article or rationale explaining why `lambda` is
> a macro, i.e. why the fully-expanded code needs to be (function (lambda
> ...)) rather than just (lambda ...)?

Making it one increases the number of special operators and hence means
that every code walker needs to be modified to know about it. Not making
it one has no cost at all to anyone: any implementation which wants to
treat it as a special operator internally is entirely free to do so.

This is also why defining it as a macro was safe: code walkers already know
both how to expand macros and how to deal with FUNCTION forms.

--
the small snake

Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> Was it considered important to support this syntax? Why?
>

Because CL aimed to be compatible with earlier implementations in which
this syntax was legal because they wanted people not to have to rewrite
code for no possible useful purpose. Because people write macros which
expand to this. Because (x ...) in the default case expects x to be a
function name and (lambda () ...) is a function name.

--
the small snake

On 2022-02-15, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> Kaz Kylheku [2022-02-15 19:36:24] wrote:
>> On 2022-02-15, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>>>> where it is a form, we still need a story which explains what lambda is in
>>>> ((lambda (...) ...) ...)
>>> Was it considered important to support this syntax? Why?
>> It's important for consistency. If a macro inserts a name into a form
>> like this:
>>
>> `(,name ...)
>>
>> it's nice if that doesn't break when name happens to be a lambda
>> expression.
>
> I wonder why/when such a thing would happen. If `lambda` is a special
> form, then `name` would either hold a function value or a form (whose
> evaluation will return a function value), neither of which would
> naturally include function names. And those cases using `(funcall
> ',name ...) resp. `(funcall ,name ...) would make more sense.

Well you need `(funcall (function ,name) ...)`. So now you're
superfluously including both pieces of Lisp-2 vebiage: funcall and
function.

If you know that the argument is a name, which it *must* be in order
to serve as an argument of the function operator, then you can just
generate `(,name ...); and if name can be lambda, then that is
a correct reduction of `(funcall (function ,name) ...)`.

>
> IOW I think the above expectation is a result of the use of #' for
> lambda, rather than the other way around.
>
>> Ideally, lambda expression should be able to appear wherever
>> a function name can appear
>
> I wonder if such a desire was part of the motivation of this design
> choice, tho.

It must have been; the Glossary even documents that a lambda expression
is: "n. a list which can be used in place of a function name in certain
contexts to denote a function by directly describing its behavior rather
than indirectly by referring to the name of an established function; its
name derives from the fact that its first element is the symbol lambda."

The main place where function names are used are in function calls,
so it would be an odd exception not to have lambda work in exactly
that "popular" spot.

The situation of being able to use a lambda instead of a function name
anywhere the function definition isn't being (re)defined is simpler
than such a situation, *plus* an arbitrary exception.

>> Immediately-called lambda is historically significant also, because it
>> provides a way of binding variables. The let construct came later, and
>> can be explained in terms of an immediately-applied anonymous function.
>> Almost needless to mention, such lambdas occur in lambda calculus.
>
> Yes, as I said, I can see good reasons why this appeared in Lisp's
> history, but I'm interested to know why Common Lisp *kept* this design.
> I haven't found any hint that it was a conscious decision, so maybe it's
> just been kept because noone considered changing it.

Old examples from old historic texts and books wouldn't work. If you
were teaching newcomers about how let related to lambda, you'd have to
always add, "Well, that's how LET originally related to lambda, but
Common Lisp got rid of that: you can't apply arguments directly to a
LAMBDA any more". Then you'd nee a time-wasting explanation for those
students who wanted to know why.

It would be critized as a quirk by the users of other languages.
How come you can write an anonymous function, but not just apply
arguments to it directly?

Speaking of other languages, in the abombination known as Javascript,
statements cannot return values. Only expressions, and function bodies,
via the return statement. If you want, say a switch to return a value,
you must write an immediately applied function:

val colorName = (function () {
switch (colorNum) {
// Of course, a dictionary object would
// actually be used for this specific situation.
case 0 : return "red";
case 1 : return "green";
case 2 : return "blue";
case 3 : return "unknown color";
}
})();

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal

Zyni Moë [2022-02-15 20:25:47] wrote:
> Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>> Does someone know of an article or rationale explaining why `lambda` is
>> a macro, i.e. why the fully-expanded code needs to be (function (lambda
>> ...)) rather than just (lambda ...)?
> Making it one increases the number of special operators and hence means
> that every code walker needs to be modified to know about it.

I don't see the benefit: all code walkers need to know about (function
(lambda ...)) instead.

> This is also why defining it as a macro was safe: code walkers already know
> both how to expand macros and how to deal with FUNCTION forms.

If you presume that (function (lambda ...)) exists, of course it doesn't
make sense to define `lambda` as a special form, but my question is
rather about the choice between

(lambda ...)

and

(function (lambda ...))

-- Stefan

Zyni Moë [2022-02-15 20:30:06] wrote:
> Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>> Was it considered important to support this syntax? Why?
>
> Because CL aimed to be compatible with earlier implementations in which
> this syntax was legal because they wanted people not to have to rewrite
> code for no possible useful purpose. Because people write macros which
> expand to this. Because (x ...) in the default case expects x to be a
> function name and (lambda () ...) is a function name.

So the main motivation was backward compatibility?
That does sound very likely, indeed. Does someone have a pointer to
some document describing this decision?

Stefan

>> I wonder why/when such a thing would happen. If `lambda` is a special
>> form, then `name` would either hold a function value or a form (whose
>> evaluation will return a function value), neither of which would
>> naturally include function names. And those cases using `(funcall
>> ',name ...) resp. `(funcall ,name ...) would make more sense.
>
> Well you need `(funcall (function ,name) ...)`.

That's only if your argument is neither a function value nor a form
returning function value, but is instead a function name that should be
lexically resolved.

If it's a function value (or a function name that should be globally
resolved) you need `(funcall ',name ...) and if it's a form you need
`(funcall ,name ...).

I don't see why (other than force of habit and backward compatibility)
"a function name that should be lexically resolved" should include
lambda expressions.

> If you know that the argument is a name, which it *must* be in order
> to serve as an argument of the function operator,

That seems to presume you want lambda expression in your "namespace",
whereas what I'm asking is why should we presume that.

>>> Ideally, lambda expression should be able to appear wherever
>>> a function name can appear
>> I wonder if such a desire was part of the motivation of this design
>> choice, tho.
> It must have been; the Glossary even documents that a lambda expression
> is: "n. a list which can be used in place of a function name in certain
> contexts to denote a function by directly describing its behavior rather
> than indirectly by referring to the name of an established function; its
> name derives from the fact that its first element is the symbol lambda."

This documents the result of the design, not necessarily the rationale
for it, tho.

> The main place where function names are used are in function calls,
> so it would be an odd exception not to have lambda work in exactly
> that "popular" spot.

Would you really describe ((lambda ...) ...) as popular in Common Lisp?
[ In ELisp, it's very rare. ]

> The situation of being able to use a lambda instead of a function name
> anywhere the function definition isn't being (re)defined is simpler
> than such a situation, *plus* an arbitrary exception.

I'm not sure it's simpler overall. Instead the rule would be that all
forms are of the share (SYMBOL . ARGS) and that to refer to the function
value of a function name you use #' and that to call a function value
you use `funcall`. I don't see any arbitrary exception.

> Old examples from old historic texts and books wouldn't work. If you
> were teaching newcomers about how let related to lambda, you'd have to
> always add, "Well, that's how LET originally related to lambda, but
> Common Lisp got rid of that: you can't apply arguments directly to a
> LAMBDA any more".

You could still write

(funcall (lambda ...) ...)

which just makes the justification for introducing `let` more obvious.
[ Actually I do exactly that in my courses ;-) ]
And you don't need to bother telling your students that some earlier
versions of the language allowed a different syntax.

> Speaking of other languages, in the abombination known as Javascript,
> statements cannot return values. Only expressions, and function bodies,
> via the return statement. If you want, say a switch to return a value,
> you must write an immediately applied function:
>
> val colorName = (function () {
> switch (colorNum) {
> // Of course, a dictionary object would
> // actually be used for this specific situation.
> case 0 : return "red";
> case 1 : return "green";
> case 2 : return "blue";
> case 3 : return "unknown color";
> }
> })();

Luckily in Lisp you don't need such silly immediately-applied lambdas
because the rest of the language is more sane.

BTW, part of what prompted this question is that you can trivially
provide an alternative (my-lambda ...) via a macro but you can't so easily
provide an alternative #'(my-lambda ...).

So this #'(lambda ...) syntax looks to me like a bit of
.... an arbitrary exception?

Stefan

* Stefan Monnier <jwv35kjaiju.fsf-monnier+comp.lang.lisp @gnu.org> :
Wrote on Tue, 15 Feb 2022 17:28:20 -0500:

> I'm not sure it's simpler overall. Instead the rule would be that all
> forms are of the share (SYMBOL . ARGS) and that to refer to the function
> value of a function name you use #' and that to call a function value
> you use `funcall`. I don't see any arbitrary exception.

I really detest all the source code changes that you check in to elisp
sources that change perfectly normal uses of (lambda () ...) to
#'(lambda () ) just to satisfy your perverted aesthetic.

Stefan Monnier <monnier@iro.umontreal.ca> wrote:

>
> I don't see the benefit: all code walkers need to know about (function
> (lambda ...)) instead.

Try to think, Already code walkers must know about this and they will
continue to need to know about it for ever because (function x) will always
continue to be valid code and x can be (lambda ...) because otherwise would
clearly be insane. Changing that would cause uncountable lossage because
it is 1975; there is much existing code. Adding lambda as a special
operator would mean code walkers would either need to be modified to know
about it, or would silently fail. Adding lambda as a macro means no user
code needs to change.

And the CL designers made the right choice because they were aware that a
world outside their heads existed.

--
the small snake

Madhu <enometh@meer.net> wrote:

> I really detest all the source code changes that you check in to elisp
> sources that change perfectly normal uses of (lambda () ...) to
> #'(lambda () ) just to satisfy your perverted aesthetic.
>

Oh no, he isn't is he? That is a really impressive level of braindamage.

--
the small snake

Stefan Monnier <monnier@iro.umontreal.ca> wrote:

>
> So the main motivation was backward compatibility?
> That does sound very likely, indeed. Does someone have a pointer to
> some document describing this decision?

Clue is in the name: *Common* Lisp: CL was famously intended to unify
various existing dialects, mostly flavours of MACLISP, although also at
least in theory Interlisp (in fact of course not so much Interlisp). There
was lots of MACLISP and other code, that code needed to not be changed
gratuitously. I am surprised you were unaware of this.

Indeed some of the designers of CL also were involved in the design of
another Lisp-family language, and that language does rather famously not
have FUNCTION or need FUNCALL, and in that language LAMBDA is a special
operator. Because that language did not require to be compatible with
MACLISP &c (or even, often, with earlier versions of itself).

But there is another reason: (funcall x ...) is a shit syntax. People read
code left to right (or, perhaps, right to left if they are Hebrew or Arabic
speakers etc): the left edge of the code is the thing you see first when
browsing through code, and you want the things at the left edge to be the
most important. This is why some old message-passing object systems in
Lisp were horrid: if you casually read the code you see

(send ...)
....
(send ...)
(send ...)
...

And this conveys approximately zero information. And similarly

(funcall ...)
....
(funcall ...)
(funcall ...)
...

is shit. Put the interesting thing -- usually the function -- as far to
the left as you can. And this is what ((lambda (...) ...) ...) does.

--
the small snake

> I really detest all the source code changes that you check in to elisp
> sources that change perfectly normal uses of (lambda () ...) to
> #'(lambda () ) just to satisfy your perverted aesthetic.

Care to point to an example of that?
I do a lot of gratuitous cosmetic changes to satisfy my perverted
aesthetic in ELisp, indeed, but I prefer (lambda ...) over #'(lambda ...)
so if I were to make such a change it would be by mistake.

Stefan

Zyni Moë [2022-02-16 10:19:31] wrote:
> Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>> I don't see the benefit: all code walkers need to know about (function
>> (lambda ...)) instead.
> Try to think, Already code walkers must know about this and they will
> continue to need to know about it for ever because (function x) will always
> continue to be valid code and x can be (lambda ...) because otherwise would
> clearly be insane.

Insane, definitely not. But, if I read between the lines you're arguing
that the motivation to not consider such a change was
backward compatibility.

I guess that makes sense, thanks,

Stefan

> And this conveys approximately zero information. And similarly
>
> (funcall ...)
> ...
> (funcall ...)
> (funcall ...)
> ...
>
> is shit.

Can't argue with that. I do also prefer Lisp-1 ;-)

> Put the interesting thing -- usually the function -- as far to
> the left as you can. And this is what ((lambda (...) ...) ...) does.

Hmm... two problem here:
- this just replaces `funcall` with `lambda` as the first thing on the
left: hardly better.
- In my experience, ((lambda ...) ...) is *very* rarely used.

Stefan

> * Stefan Monnier <zbaavre@veb.hzbagerny.pn> [2022-02-15 11:08:23 -0500]:
>
> Sam Steingold [2022-02-15 10:51:33] wrote:
>>> * Stefan Monnier <zbaavre@veb.hzbagerny.pn> [2022-02-15 09:35:18 -0500]:
>>> Does someone know of an article or rationale explaining why `lambda` is
>>> a macro, i.e. why the fully-expanded code needs to be (function (lambda
>>> ...)) rather than just (lambda ...)?
>> Speaking as an implementor, a macro is "cheaper" than a special
>> operator, so _requiring_ something to be a special operator is a weird
>> thing to do unless there is a compelling reason for that.
>
> But any Common Lisp implementation will need to have special handling of
> (function (lambda ...)) and I fail to see how it would be cheaper than
> moving that work to a (lambda ...) special form.

One has to handle (function (lambda ...)) anyway; why do the work twice?

--
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.2113
http://childpsy.net http://calmchildstories.com http://steingoldpsychology.com
https://mideasttruth.com https://www.peaceandtolerance.org/
I have been called worse. By better people.

* Stefan Monnier <jwvbkz76ia2.fsf-monnier+comp.lang.lisp @gnu.org> :
Wrote on Wed, 16 Feb 2022 08:42:56 -0500:

>> I really detest all the source code changes that you check in to elisp
>> sources that change perfectly normal uses of (lambda () ...) to
>> #'(lambda () ) just to satisfy your perverted aesthetic.
>
> Care to point to an example of that?
> I do a lot of gratuitous cosmetic changes to satisfy my perverted
> aesthetic in ELisp, indeed, but I prefer (lambda ...) over #'(lambda ...)
> so if I were to make such a change it would be by mistake.

I sincerely apologize for the slander and false witness, I should've
checked before posting. Perhaps I was remembering something from long
ago, but there is really no excuse.

The changes in the code base in recent have indeed been towards removing
redundant #' from lambda. Mea culpa. and sorry for misleading zyni.

On 2022-02-15, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>>> I wonder why/when such a thing would happen. If `lambda` is a special
>>> form, then `name` would either hold a function value or a form (whose
>>> evaluation will return a function value), neither of which would
>>> naturally include function names. And those cases using `(funcall
>>> ',name ...) resp. `(funcall ,name ...) would make more sense.
>>
>> Well you need `(funcall (function ,name) ...)`.
>
> That's only if your argument is neither a function value nor a form
> returning function value, but is instead a function name that should be
> lexically resolved.
>
> If it's a function value (or a function name that should be globally
> resolved) you need `(funcall ',name ...) and if it's a form you need
> `(funcall ,name ...).
>
> I don't see why (other than force of habit and backward compatibility)
> "a function name that should be lexically resolved" should include
> lambda expressions.
>
>> If you know that the argument is a name, which it *must* be in order
>> to serve as an argument of the function operator,
>
> That seems to presume you want lambda expression in your "namespace",
> whereas what I'm asking is why should we presume that.

Because I'm used to the idea that, in mathematics, a name like x can denote,
say, an integer. Likewise, a literal like 42 can denote an integer.

Both x and 42 are entities of the same kind, and in all situations where we
just reference the value, we can substitute one for the other.

(There are situations where x isn't referenced as a value, like

"let x be an an integer greater than 100"

here, we cannot say:

"let 42 be an integer greater than 100"

but in a formula where x is referenced, like x + y, we can be
justified in making a substitution to 42 + y.)

Anyway, likewise, a lambda is a function literal. So likewise, just
like in any situation where the integer x is refernced, I should be
able to specify a concrete integer like 42, wherever some function f
is referenced, I should be able to specify the lambda literal.

>>>> Ideally, lambda expression should be able to appear wherever
>>>> a function name can appear
>>> I wonder if such a desire was part of the motivation of this design
>>> choice, tho.
>> It must have been; the Glossary even documents that a lambda expression
>> is: "n. a list which can be used in place of a function name in certain
>> contexts to denote a function by directly describing its behavior rather
>> than indirectly by referring to the name of an established function; its
>> name derives from the fact that its first element is the symbol lambda."
>
> This documents the result of the design, not necessarily the rationale
> for it, tho.

That is true; rationales tend to be separately given. Like for ANSI C
1989, a separate Rationale was published.

>> The main place where function names are used are in function calls,
>> so it would be an odd exception not to have lambda work in exactly
>> that "popular" spot.
>
> Would you really describe ((lambda ...) ...) as popular in Common Lisp?
> [ In ELisp, it's very rare. ]

Well, the spot is popular not necessarily the lambda being there.

The lambda is just denotes the right kind of thing that fits into
that spot.

In TXR Lisp, a Lisp-2, (lambda ...) is a special form. For quite a long time,
((lambda ...) ...) did not work. But: consistently, neither did
(fun (lambda ...) ...)

>> The situation of being able to use a lambda instead of a function name
>> anywhere the function definition isn't being (re)defined is simpler
>> than such a situation, *plus* an arbitrary exception.
>
> I'm not sure it's simpler overall. Instead the rule would be that all
> forms are of the share (SYMBOL . ARGS) and that to refer to the function
> value of a function name you use #' and that to call a function value
> you use `funcall`. I don't see any arbitrary exception.

These definitions, by themselves, do not talk about anonymous functions
at all. So when those are introduced, it should be in some internally
consistent way.

>> Old examples from old historic texts and books wouldn't work. If you
>> were teaching newcomers about how let related to lambda, you'd have to
>> always add, "Well, that's how LET originally related to lambda, but
>> Common Lisp got rid of that: you can't apply arguments directly to a
>> LAMBDA any more".
>
> You could still write
>
> (funcall (lambda ...) ...)

Yes; you could introduce troduce lambda into the dialect as being a kind
of expression that, only under evaluation, produces a function,
and not something substitutable for function names.

I believe that for consistency, you should then also banish
(function (lambda ...))).

So either have lambdas that can be used almost everywhere where function
names can be used, and so ((lambda ...) ...) is possible as well
as #'(lambda ...). Or else, say no: lambdas are not expressions that can be
used in place of a function name but strictly evaluated expressions that
produce a a closure. Then, have neither ((lambda ...) ...) nor #'(lambda ...).

Either situation is simpler than an ad-hoc mixture: one works but not the other.

Arguably, CL would therefore be simpler without the lambda macro;
it's already conceptually consistent without it.

The CL lambda macro causes lambda expressions to do something which function
names cannot: appear in an evaluated context without any kind of quoting,
and produce a functional value.

> which just makes the justification for introducing `let` more obvious.
> [ Actually I do exactly that in my courses ;-) ]
> And you don't need to bother telling your students that some earlier
> versions of the language allowed a different syntax.

That's undoubtedly how education works in the world of Javascript
or Python and whatnot. History is important though, and it's a good
thing we have strong connections to it in the Lisp world.

> BTW, part of what prompted this question is that you can trivially
> provide an alternative (my-lambda ...) via a macro but you can't so easily
> provide an alternative #'(my-lambda ...).

Or an alternative ((my-lambda ...) ...)!

We can think about a mechanism which allows this, yet clings to Lisp-2
tooth and nail.

Suppose we introduce the concept of "function name macros". A function name
macro invocation occurs when a compound expression appears as a function
name that is not a lambda or setf.

((my-lambda ...) ...)

#'(my-lambda ...)

This could be defined with some new definer macro, e.g.:

(define-function-name-macro kons ()
'cons)

Now we can have:

((kons) 1 2) -> (cons 1 2) -> (1 . 2)

Furthermore, there can be a rule that when an expression is being
expanded in the ordinary evaluation context, then function name
macros are visible (with a low precedence: if kons is both a macro
and a function name macro, the macro wins). When a function name
macro is expanded in ordinary context, it is wrapped with the function
operator, either before or after expansion:

Then we can do

(funcall (kons) 1 2) -> (funcall (function (kons)) 1 2)

-> (funcall (function cons) 1 2) -> (1 . 2)

Partial application macro:

(define-function-name-macro kons (car-expr)
(let ((cdr (gensym)))
`(lambda (,cdr) (cons ,car-expr ,cdr))))

((kons 1) 2) -> (1 . 2)

(mapcar (kons 1) '(a b c)) -> ((1 . a) (1 . b) (1 . c))

(mapcar #'(kons 1) '(a b c)) -> ((1 . a) (1 . b) (1 . c))

> So this #'(lambda ...) syntax looks to me like a bit of
> ... an arbitrary exception?

Only the existence of the lambda macro makes it look that way, but
#'(lambda ...) is in the same category as ((lambda ..) ...) which is
consistent. You can't target either position with a macro because of Lisp-2;
ordinary macros are evaluated expressions, and inactive in a function name
context.

But Lisps can have macros in any space; macros /per se/ don't require an
evaluation context. For instance, a pattern matcher can allow new patterns to
be defined in terms of existing pattern operators by providing a defpattern
macro for defining them. That is basically a macro mechanism in the pattern
space.

The CL lambda macro creates an inconsistency in that it looks exactly like a
lambda expression, which can be used in the above two contexts; but any other
macro you define by any other name cannot be. (Which we could fix with the
function name macros idea.)

Incidentally, under the above function name macros, lambda could be defined as
a trivial, no-op function macro:

(define-function-name-macro lambda (&rest args &whole lambda-expr)
lambda-expr)

> Anyway, likewise, a lambda is a function literal. So likewise, just
> like in any situation where the integer x is refernced, I should be
> able to specify a concrete integer like 42, wherever some function f
> is referenced, I should be able to specify the lambda literal.

That makes sense, indeed.
I wonder if that was part of the motivation in the decision back then.
Also, I'll note that AFAIK Common Lisp does not obey that reasoning when
it comes to macros :-(
And Scheme doesn't either, FWIW.

> I believe that for consistency, you should then also banish
> (function (lambda ...))).

Definitely, yes.

> So either have lambdas that can be used almost everywhere where function
> names can be used, and so ((lambda ...) ...) is possible as well
> as #'(lambda ...). Or else, say no: lambdas are not expressions that can be
> used in place of a function name but strictly evaluated expressions that
> produce a a closure. Then, have neither ((lambda ...) ...) nor #'(lambda ...).

100% agreed. I guess I thought it was obvious, so I didn't spell it out
in my original message :-(

>> BTW, part of what prompted this question is that you can trivially
>> provide an alternative (my-lambda ...) via a macro but you can't so easily
>> provide an alternative #'(my-lambda ...).
> Or an alternative ((my-lambda ...) ...)!

Right.

> Suppose we introduce the concept of "function name macros".

;-)

Stefan