In CL-WHO, expressions which give attribute values are evaluated. If
they produce text it is HTML-escaped and turns into an attribute.

(:table :border (+ 1 2)) -> <table border='3'></table>

But, expressions elsewhere in the template are only evaluated;
their values do not become HTML output.

(:table :border (+ 1 2)
(identity "abc")) -> <table border='3'></table>

Except for constant strings:

(:table :border (+ 1 2)
"abc") -> <table border='3'>abc</table>

You have to make explicit use of certain local macros like str:

(:table :border (+ 1 2)
(str (identity "abc"))) -> <table border='3'>abc</table>

It seems inconsistent/inconvenient. Does anyone know why Weitz
designed and kept it that way?

Is it too much of a footgun to interpolate all the expressions
into the output?

Or was it more important to be able to write in side-effects
with minimal fuss?

I'm leaning toward wanting it to just interpolate values
by default and having some operator for indicating that
forms are there for effect only:

(:div :class "greeting"
"Hello, " user ;; emitted
;; effect only:
(do (debug "emitted greeting for user ~a" user)))

->

Here, the HTML macro recognizes (do ...) and converts
it to (progn ...), but any other form becomes (str form),
and so outputs.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

On 2023-05-29, Kaz Kylheku <864-117-4973@kylheku.com> wrote:
> I'm leaning toward wanting it to just interpolate values
> by default and having some operator for indicating that
> forms are there for effect only:
>
> (:div :class "greeting"
> "Hello, " user ;; emitted
> ;; effect only:
> (do (debug "emitted greeting for user ~a" user)))

But, the thing is, most of the CL-WHO examples do some kind of loop:

(:table :cellpadding 0 :border 4
(loop for r in rows
do (htm (:tr ...))))

Here, the loop is purely effectful; its return value isn't being
considered for output. If you have a lot of code like this, you're
looking at explicitly annotating all those loops as being side-effect
only. If you forget, you get extra bits of garbage in the HTML output
you might not even notice.

If it sucked this way, people would ahve changed it years ago,
or added some mode flag.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

On 2023-05-29, Kaz Kylheku <864-117-4973@kylheku.com> wrote:
> On 2023-05-29, Kaz Kylheku <864-117-4973@kylheku.com> wrote:
>> I'm leaning toward wanting it to just interpolate values
>> by default and having some operator for indicating that
>> forms are there for effect only:
>>
>> (:div :class "greeting"
>> "Hello, " user ;; emitted
>> ;; effect only:
>> (do (debug "emitted greeting for user ~a" user)))
>
> But, the thing is, most of the CL-WHO examples do some kind of loop:
>
> (:table :cellpadding 0 :border 4
> (loop for r in rows
> do (htm (:tr ...))))
>
> Here, the loop is purely effectful; its return value isn't being
> considered for output. If you have a lot of code like this, you're
> looking at explicitly annotating all those loops as being side-effect
> only. If you forget, you get extra bits of garbage in the HTML output
> you might not even notice.
>
> If it sucked this way, people would ahve changed it years ago,
> or added some mode flag.

On the other hand, that handling of attributes is vulnerable to injection; it's
not doing any escaping!

[1]> (cl-who:with-html-output-to-string (out)
(:table :cellpadding "0'>hahaha</table>" "what???"))
"<table cellpadding='0'>hahaha</table>'>what???</table>"

Simply astonishing. If you happen to have any sensitive of data going into
attributes that a user could manipulate, you have to remember to your own
escaping, or you've enabled an injection attack.

The local macro (fmt ...) has the same problem; no escaping.

If you want escaping you use (esc ...), but that doesn't have any
format cruft you might want.

If you want formatting with (fmt ...), that doesn't escape; you have
a possible injection attack if you interpolate some string controlled
by a remote user.

Then there is the local macro (str ...) which combines the lack of
functionality of esc, with the insecurity of (fmt ...).

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

On Mon, 29 May 2023 22:39:05 -0000 (UTC)
Kaz Kylheku <864-117-4973@kylheku.com> wrote:
> On 2023-05-29, Kaz Kylheku <864-117-4973@kylheku.com> wrote:
> > If it sucked this way, people would ahve changed it years ago,
> > or added some mode flag.

Perhaps not many people use the library.

> On the other hand, that handling of attributes is vulnerable to injection; it's
> not doing any escaping!
>
> [1]> (cl-who:with-html-output-to-string (out)
> (:table :cellpadding "0'>hahaha</table>" "what???"))
> "<table cellpadding='0'>hahaha</table>'>what???</table>"
>
> Simply astonishing. If you happen to have any sensitive of data going into
> attributes that a user could manipulate, you have to remember to your own
> escaping, or you've enabled an injection attack.

I've never used the library so I don't know in which situations you'd want
to use it. Can you describe a realistic scenario where the possibilities
you mention would be relevant and what you would consider a good behaviour
in such situations ?

On 2023-05-30, Spiros Bousbouras <spibou@gmail.com> wrote:
> On Mon, 29 May 2023 22:39:05 -0000 (UTC)
> Kaz Kylheku <864-117-4973@kylheku.com> wrote:
>> On 2023-05-29, Kaz Kylheku <864-117-4973@kylheku.com> wrote:
>> > If it sucked this way, people would ahve changed it years ago,
>> > or added some mode flag.
>
> Perhaps not many people use the library.
>
>> On the other hand, that handling of attributes is vulnerable to injection; it's
>> not doing any escaping!
>>
>> [1]> (cl-who:with-html-output-to-string (out)
>> (:table :cellpadding "0'>hahaha</table>" "what???"))
>> "<table cellpadding='0'>hahaha</table>'>what???</table>"
>>
>> Simply astonishing. If you happen to have any sensitive of data going into
>> attributes that a user could manipulate, you have to remember to your own
>> escaping, or you've enabled an injection attack.
>
> I've never used the library so I don't know in which situations you'd want
> to use it.

Generating HTML; e.g. in the dynamic pages of a web application.

> Can you describe a realistic scenario where the possibilities
> you mention would be relevant and what you would consider a good behaviour
> in such situations ?

Say we are coding a forum site and we have a page where user
profiles can be viewed. Users have websites:

... "Website:" (:a :href user-website-url ...) ...

The user-website-url comes from the user's profile; it is user-editable.

If user-website-url is not subject to HTML escaping, we have a security
issue: a user can put malicious HTML fragments into their website
string, which become part of the profile page, so that the page then
perpetrates an attack on someone who merely loads it.

The default interpolation behavior in HTML generation utilities should
be secure. When a HTML utility is used in its most succinct, convenient
and canonical way, as recommended by its documentation and examples, its
behavior should be secure.

Opting out of security should be the inconvenient choice requiring some
extra code, not opting in.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

On Tue, 30 May 2023 14:20:15 -0000 (UTC)
Kaz Kylheku <864-117-4973@kylheku.com> wrote:
> On 2023-05-30, Spiros Bousbouras <spibou@gmail.com> wrote:
> > I've never used the library so I don't know in which situations you'd want
> > to use it.
>
> Generating HTML; e.g. in the dynamic pages of a web application.
>
> > Can you describe a realistic scenario where the possibilities
> > you mention would be relevant and what you would consider a good behaviour
> > in such situations ?
>
> Say we are coding a forum site and we have a page where user
> profiles can be viewed. Users have websites:
>
> ... "Website:" (:a :href user-website-url ...) ...
>
> The user-website-url comes from the user's profile; it is user-editable.
>
> If user-website-url is not subject to HTML escaping, we have a security
> issue: a user can put malicious HTML fragments into their website
> string, which become part of the profile page, so that the page then
> perpetrates an attack on someone who merely loads it.

The user will modify their profile by filling fields on some form ,
right ? How do you go from that to evaluating
(:a :href user-website-url ...) ? Wouldn't you sanitise what the user
entered at the time they entered it ?

Spiros Bousbouras wrote:

> On Tue, 30 May 2023 14:20:15 -0000 (UTC)
> Kaz Kylheku <864-117-4973@kylheku.com> wrote:
>
>> On 2023-05-30, Spiros Bousbouras <spibou@gmail.com> wrote:
>>
>>> Can you describe a realistic scenario where the possibilities you
>>> mention would be relevant and what you would consider a good behaviour
>>> in such situations ?
>>
>> Say we are coding a forum site and we have a page where user
>> profiles can be viewed. Users have websites:
>>
>> ... "Website:" (:a :href user-website-url ...) ...
>>
>> The user-website-url comes from the user's profile; it is user-editable.
>>
>> If user-website-url is not subject to HTML escaping, we have a security
>> issue: a user can put malicious HTML fragments into their website
>> string, which become part of the profile page, so that the page then
>> perpetrates an attack on someone who merely loads it.
>
> The user will modify their profile by filling fields on some form ,
> right ? How do you go from that to evaluating
> (:a :href user-website-url ...) ? Wouldn't you sanitise what the user
> entered at the time they entered it ?

In my experience it's much more common to do sanitizing on output, not on
input. For one thing, different output formats require different forms of
sanitization. If you're producing HTML, you probably want to change every
"&" to "&amp;". If you're producing JSON you don't want to do that, but
there are other forms of escaping you need to do. Keeping the data in its
original form in your data store means that you can defer the decision about
which form to use until you actually know what output format you're
targeting.

Another consideration is that new forms of injection are being created and
discovered all the time. If you sanitize the user's input as soon as they
enter it, how are you going to handle the possibility that it contains
malicious code that you weren't able to identify at the time, but learn
about later?

You could sanitize on the way in *and* on the way out, but you'd better hope
that your processing is idempotent. I guess you could periodically
re-sanitize your entire database, but that has the same problem and could be
quite resource intensive. Or--as above--you could defer the decision until
you're actually ready to do something with the user's input, at which point
you have the most up-to-date information about which entries might be
problematic.

Benjamin