When opening an HTML email using Control + L (launch in browser), the page always opens with what appears to be the wrong character encoding.

Instead of an inverted comma, I see a black diamond with a ? inside.

I am using Chrome as my default browser and I have a Chrome extension that allows me to set the character encoding back to Western (Windows-1252) and then all is well.

I just wondered if this was an Agent issue or a browser issue?

Thanks

On Fri, 18 Nov 2022 16:00:43 +0000, Me wrote:

> When opening an HTML email using Control + L (launch in browser), the
> page always opens with what appears to be the wrong character encoding.
>
> Instead of an inverted comma, I see a black diamond with a ? inside.
>
> I am using Chrome as my default browser and I have a Chrome extension
> that allows me to set the character encoding back to Western
> (Windows-1252) and then all is well.
>
> I just wondered if this was an Agent issue or a browser issue?

Neither. It is an issue with the message's HTML code itself.

The HTML code is missing one of these:

<meta charset="Windows-1252">
<meta http-equiv="content-type" content="text/html; charset=Windows-1252">

--
Kind regards
Ralph

🔠🔡🔢🔣🔤

On Sat, 19 Nov 2022 06:32:11 +1300, Ralph Fox <-rf-nz-@-.invalid> wrote:

>On Fri, 18 Nov 2022 16:00:43 +0000, Me wrote:
>
>> When opening an HTML email using Control + L (launch in browser), the
>> page always opens with what appears to be the wrong character encoding.
>>
>> Instead of an inverted comma, I see a black diamond with a ? inside.
>>
>> I am using Chrome as my default browser and I have a Chrome extension
>> that allows me to set the character encoding back to Western
>> (Windows-1252) and then all is well.
>>
>> I just wondered if this was an Agent issue or a browser issue?
>
>
>Neither. It is an issue with the message's HTML code itself.
>
>The HTML code is missing one of these:
>
> <meta charset="Windows-1252">
> <meta http-equiv="content-type" content="text/html; charset=Windows-1252">

Thank you Ralph, so it is out of my control then?

On Sat, 19 Nov 2022 10:10:42 +0000, Me wrote:

> Thank you Ralph, so it is out of my control then?

Indeed so.

--
Kind regards
Ralph

🎈🐦🥧🌥️

On Sun, 20 Nov 2022 08:39:25 +1300, Ralph Fox <-rf-nz-@-.invalid> wrote:

>On Sat, 19 Nov 2022 10:10:42 +0000, Me wrote:
>
>> Thank you Ralph, so it is out of my control then?
>
>Indeed so.

Thanks Ralph.

I do wonder why so many, if not all, of my HTML attachments are like that though when opened through Agent, which in turn opens the document in Chrome.

Within the article, there is often a link to open the same page "in a browser", and when I click that, the document displays correctly, hence my thought that Agent might be doing something strange.

On Tue, 22 Nov 2022 09:56:52 +0000, Me wrote:

> I do wonder why so many, if not all, of my HTML attachments are like that
> though when opened through Agent, which in turn opens the document in
> Chrome.
>
> Within the article, there is often a link to open the same page "in a
> browser", and when I click that, the document displays correctly, hence
> my thought that Agent might be doing something strange.

When you view an HTML email in an email program, the email also has
a 'Content-Type' header which tells the email program what the
character encoding is.

When viewing the HTML via an http(s) link, there are two ways in which
your web browser can be told what the character encoding is.

1) The web server's HTTP(s) response headers may contain a
'Content-Type' header which says what the character encoding is.

2) Or, the HTML code itself may contain a meta tag which says what
the character encoding is.

The 'Content-Type' header is not part of the HTML file. When you
launch the HTML as a file in your browser, the browser does not have
the 'Content-Type' header.

It seems your message is using a Content-Type header but not a meta
tag.

--
Kind regards
Ralph

Ὁ βίος βραχύς, ἡ δὲ τέχνη μακρή
Vita brevis, ars longa
The lyf so short, the craft so long to lerne

On Wed, 23 Nov 2022 07:10:42 +1300, Ralph Fox <-rf-nz-@-.invalid> wrote:

>On Tue, 22 Nov 2022 09:56:52 +0000, Me wrote:
>
>> I do wonder why so many, if not all, of my HTML attachments are like that
>> though when opened through Agent, which in turn opens the document in
>> Chrome.
>>
>> Within the article, there is often a link to open the same page "in a
>> browser", and when I click that, the document displays correctly, hence
>> my thought that Agent might be doing something strange.
>
>
>When you view an HTML email in an email program, the email also has
>a 'Content-Type' header which tells the email program what the
>character encoding is.
>
>When viewing the HTML via an http(s) link, there are two ways in which
>your web browser can be told what the character encoding is.
>
> 1) The web server's HTTP(s) response headers may contain a
> 'Content-Type' header which says what the character encoding is.
>
> 2) Or, the HTML code itself may contain a meta tag which says what
> the character encoding is.
>
>The 'Content-Type' header is not part of the HTML file. When you
>launch the HTML as a file in your browser, the browser does not have
>the 'Content-Type' header.
>
>It seems your message is using a Content-Type header but not a meta
>tag.

Thanks again Ralph and yet again it does appear that there is nothing that I can do to fix the issue. I have installed a Chrome extension where I can change the character encoding to Western (Windows-1252) but I guess that that is as
much as I can do.

I am surprised that more people using Agent for e-mail don't see, or appear to have, this issue.

On Wed, 23 Nov 2022 07:10:42 +1300, Ralph Fox <-rf-nz-@-.invalid> wrote:

>On Tue, 22 Nov 2022 09:56:52 +0000, Me wrote:
>
>> I do wonder why so many, if not all, of my HTML attachments are like that
>> though when opened through Agent, which in turn opens the document in
>> Chrome.
>>
>> Within the article, there is often a link to open the same page "in a
>> browser", and when I click that, the document displays correctly, hence
>> my thought that Agent might be doing something strange.
>
>
>When you view an HTML email in an email program, the email also has
>a 'Content-Type' header which tells the email program what the
>character encoding is.
>
>When viewing the HTML via an http(s) link, there are two ways in which
>your web browser can be told what the character encoding is.
>
> 1) The web server's HTTP(s) response headers may contain a
> 'Content-Type' header which says what the character encoding is.
>
> 2) Or, the HTML code itself may contain a meta tag which says what
> the character encoding is.
>
>The 'Content-Type' header is not part of the HTML file. When you
>launch the HTML as a file in your browser, the browser does not have
>the 'Content-Type' header.
>
>It seems your message is using a Content-Type header but not a meta
>tag.

Update:

Hi Ralph,

Actually, I have found a way of fixing it within Agent.

Under settings -> languages, I have English (International) selected.

In the settings for English (International), I had the Code Page set to windows-1252. When I changed it to Unicode FULL UTF-8 (Codepage 65001) the problem disappeared.

The Default Charset is currently set to Western Europe (windows-1252). Do you think I should leave that as is, or change that to UTF-8 too?

On Wed, 23 Nov 2022 11:00:02 +0000, Me wrote:

> Update:
>
> Hi Ralph,
>
> Actually, I have found a way of fixing it within Agent.
>
> Under settings -> languages, I have English (International) selected.
>
> In the settings for English (International), I had the Code Page set to
> windows-1252. When I changed it to Unicode FULL UTF-8 (Codepage 65001)
> the problem disappeared.
>
> The Default Charset is currently set to Western Europe (windows-1252). Do
> you think I should leave that as is, or change that to UTF-8 too?

I recommend leaving 'Default Charset' as 'Western Europe (windows-1252)'.

When an incoming message does not have a 'Content-Type' header with
the character encoding, Agent assumes the incoming message's character
encoding is what the 'Default Charset' setting says.

Messages in UTF-8 character encoding always have a 'Content-Type'
header with the character encoding. Messages in UTF-8 character
encoding never need the 'Default Charset' setting.

HTML messages always have a 'Content-Type' header which also says
the type is HTML (text/html).

Messages without a 'Content-Type' header to specify the character
encoding, are messages posted by old programs using the sender's
default ANSI code page. For English-language messages, that is
almost always 'Western Europe (windows-1252)'.

--
Kind regards
Ralph

Δος μοι πα στω και ταν γαν κινάσω
-- Archimedes [in Syracusan Greek (Doric)]

On Thu, 24 Nov 2022 06:47:45 +1300, Ralph Fox <-rf-nz-@-.invalid> wrote:

>On Wed, 23 Nov 2022 11:00:02 +0000, Me wrote:
>
>> Update:
>>
>> Hi Ralph,
>>
>> Actually, I have found a way of fixing it within Agent.
>>
>> Under settings -> languages, I have English (International) selected.
>>
>> In the settings for English (International), I had the Code Page set to
>> windows-1252. When I changed it to Unicode FULL UTF-8 (Codepage 65001)
>> the problem disappeared.
>>
>> The Default Charset is currently set to Western Europe (windows-1252). Do
>> you think I should leave that as is, or change that to UTF-8 too?
>
>
>I recommend leaving 'Default Charset' as 'Western Europe (windows-1252)'.
>
>
>When an incoming message does not have a 'Content-Type' header with
>the character encoding, Agent assumes the incoming message's character
>encoding is what the 'Default Charset' setting says.
>
>Messages in UTF-8 character encoding always have a 'Content-Type'
>header with the character encoding. Messages in UTF-8 character
>encoding never need the 'Default Charset' setting.
>
>HTML messages always have a 'Content-Type' header which also says
>the type is HTML (text/html).
>
>Messages without a 'Content-Type' header to specify the character
>encoding, are messages posted by old programs using the sender's
>default ANSI code page. For English-language messages, that is
>almost always 'Western Europe (windows-1252)'.

Thank you again Ralph.

I will leave everything alone now, knowing that having changed the Code Page to Unicode FULL UTF-8, it fixed the problem.

On Thu, 24 Nov 2022 10:42:57 +0000, Me <Me@me.com> wrote:

>On Thu, 24 Nov 2022 06:47:45 +1300, Ralph Fox <-rf-nz-@-.invalid> wrote:
>
>>On Wed, 23 Nov 2022 11:00:02 +0000, Me wrote:
>>
>>> Update:
>>>
>>> Hi Ralph,
>>>
>>> Actually, I have found a way of fixing it within Agent.
>>>
>>> Under settings -> languages, I have English (International) selected.
>>>
>>> In the settings for English (International), I had the Code Page set to
>>> windows-1252. When I changed it to Unicode FULL UTF-8 (Codepage 65001)
>>> the problem disappeared.
>>>
>>> The Default Charset is currently set to Western Europe (windows-1252). Do
>>> you think I should leave that as is, or change that to UTF-8 too?
>>
>>
>>I recommend leaving 'Default Charset' as 'Western Europe (windows-1252)'.
>>
>>
>>When an incoming message does not have a 'Content-Type' header with
>>the character encoding, Agent assumes the incoming message's character
>>encoding is what the 'Default Charset' setting says.
>>
>>Messages in UTF-8 character encoding always have a 'Content-Type'
>>header with the character encoding. Messages in UTF-8 character
>>encoding never need the 'Default Charset' setting.
>>
>>HTML messages always have a 'Content-Type' header which also says
>>the type is HTML (text/html).
>>
>>Messages without a 'Content-Type' header to specify the character
>>encoding, are messages posted by old programs using the sender's
>>default ANSI code page. For English-language messages, that is
>>almost always 'Western Europe (windows-1252)'.
>
>Thank you again Ralph.
>
>I will leave everything alone now, knowing that having changed the Code Page to Unicode FULL UTF-8, it fixed the problem.

Actually, a new problem has occurred ...

For example, having changed the code page to Unicode FULL UTF-8, in a normal text e-mail, the word "here's" now comes out as "here’s"

Is there any way I can have the best of both worlds - ie using Unicode FULL UTF-8 to sort out my browser issue, and also use parts of windows-1252 so that I see "here's" and not "here’s"?

On Fri, 25 Nov 2022 12:56:32 +0000, Me wrote:
> On Thu, 24 Nov 2022 10:42:57 +0000, Me <Me@me.com> wrote:
>
>> I will leave everything alone now, knowing that having changed the Code Page to Unicode FULL UTF-8, it fixed the problem.
>
> Actually, a new problem has occurred ...
>
> For example, having changed the code page to Unicode FULL UTF-8, in a normal text e-mail, the word "here's" now comes out as "here’s"
>
> Is there any way I can have the best of both worlds - ie using Unicode FULL UTF-8 to sort out my browser issue, and also use parts of windows-1252 so that I see "here's" and not "here’s"?

I need to update my answer to an earlier question of yours:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ QUOTE ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| When opening an HTML email using Control + L (launch in browser), the
| page always opens with what appears to be the wrong character encoding.
|
| Instead of an inverted comma, I see a black diamond with a ? inside.
|
| I am using Chrome as my default browser and I have a Chrome extension
| that allows me to set the character encoding back to Western
| (Windows-1252) and then all is well.
|
| I just wondered if this was an Agent issue or a browser issue?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ QUOTE ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

There is an Agent v6.00 issue which can cause this. That issue was
fixed in Agent v7.00. I am using Agent 8.00 where this issue does not
occur, but you are using Agent 6.00 where this issue does exist.

There are two different things you could do about this Agent 6.00 issue
a) Either, upgrade to Agent 7.00 or later.
b) Or, keep using Agent 6.00 but change the message's language
to one whose code page is the same as the email's charset (these
days, emails are often UTF-8).

====
However, I cannot explain how "here's" comes out as "here’s".
Even in Agent 6.00, I see this:

1. If the sender wrote "here's", I will always see "here's". Never
"here’s".

2. If the sender wrote "here’s", I will always see "here’s" or
"here�s". Never "here's".

--
Kind regards
Ralph

🔣🡪📧🡪🔣

On Sun, 27 Nov 2022 21:38:49 +1300, Ralph Fox <-rf-nz-@-.invalid> wrote:

>On Fri, 25 Nov 2022 12:56:32 +0000, Me wrote:
>> On Thu, 24 Nov 2022 10:42:57 +0000, Me <Me@me.com> wrote:
>>
>>> I will leave everything alone now, knowing that having changed the Code Page to Unicode FULL UTF-8, it fixed the problem.
>>
>> Actually, a new problem has occurred ...
>>
>> For example, having changed the code page to Unicode FULL UTF-8, in a normal text e-mail, the word "here's" now comes out as "here’s"
>>
>> Is there any way I can have the best of both worlds - ie using Unicode FULL UTF-8 to sort out my browser issue, and also use parts of windows-1252 so that I see "here's" and not "here’s"?
>
>
>I need to update my answer to an earlier question of yours:
>
>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ QUOTE ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>| When opening an HTML email using Control + L (launch in browser), the
>| page always opens with what appears to be the wrong character encoding.
>|
>| Instead of an inverted comma, I see a black diamond with a ? inside.
>|
>| I am using Chrome as my default browser and I have a Chrome extension
>| that allows me to set the character encoding back to Western
>| (Windows-1252) and then all is well.
>|
>| I just wondered if this was an Agent issue or a browser issue?
>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ QUOTE ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
>
>There is an Agent v6.00 issue which can cause this. That issue was
>fixed in Agent v7.00. I am using Agent 8.00 where this issue does not
>occur, but you are using Agent 6.00 where this issue does exist.
>
>There are two different things you could do about this Agent 6.00 issue
> a) Either, upgrade to Agent 7.00 or later.
> b) Or, keep using Agent 6.00 but change the message's language
> to one whose code page is the same as the email's charset (these
> days, emails are often UTF-8).
>
>====
>
>However, I cannot explain how "here's" comes out as "here’s".
>Even in Agent 6.00, I see this:
>
>1. If the sender wrote "here's", I will always see "here's". Never
> "here’s".
>
>2. If the sender wrote "here’s", I will always see "here’s" or
> "here�s". Never "here's".

Thank you Ralph

I will ponder my options.

In case it did not come out clearly, here is an example of what I see in some emails if code page is set to Unicode FULL UTF-8

https://ibb.co/0DXJ1JT

On Mon, 28 Nov 2022 10:43:20 +0000, Me wrote:
> On Sun, 27 Nov 2022 21:38:49 +1300, Ralph Fox <-rf-nz-@-.invalid> wrote:
>> On Fri, 25 Nov 2022 12:56:32 +0000, Me wrote:
>>> On Thu, 24 Nov 2022 10:42:57 +0000, Me <Me@me.com> wrote:
>>>
>>>> I will leave everything alone now, knowing that having changed the Code Page to Unicode FULL UTF-8, it fixed the problem.
>>>
>>> Actually, a new problem has occurred ...
>>>
>>> For example, having changed the code page to Unicode FULL UTF-8, in a normal text e-mail, the word "here's" now comes out as "here’s"
>>>
>>> Is there any way I can have the best of both worlds - ie using Unicode FULL UTF-8 to sort out my browser issue, and also use parts of windows-1252 so that I see "here's" and not "here’s"?
>>
>>
>> I need to update my answer to an earlier question of yours:
>>
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ QUOTE ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> | When opening an HTML email using Control + L (launch in browser), the
>> | page always opens with what appears to be the wrong character encoding.
>> |
>> | Instead of an inverted comma, I see a black diamond with a ? inside.
>> |
>> | I am using Chrome as my default browser and I have a Chrome extension
>> | that allows me to set the character encoding back to Western
>> | (Windows-1252) and then all is well.
>> |
>> | I just wondered if this was an Agent issue or a browser issue?
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ QUOTE ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>
>>
>> There is an Agent v6.00 issue which can cause this. That issue was
>> fixed in Agent v7.00. I am using Agent 8.00 where this issue does not
>> occur, but you are using Agent 6.00 where this issue does exist.
>>
>> There are two different things you could do about this Agent 6.00 issue
>> a) Either, upgrade to Agent 7.00 or later.
>> b) Or, keep using Agent 6.00 but change the message's language
>> to one whose code page is the same as the email's charset (these
>> days, emails are often UTF-8).
>>
>> ====
>>
>> However, I cannot explain how "here's" comes out as "here’s".
>> Even in Agent 6.00, I see this:
>>
>> 1. If the sender wrote "here's", I will always see "here's". Never
>> "here’s".
>>
>> 2. If the sender wrote "here’s", I will always see "here’s" or
>> "here�s". Never "here's".
>
> Thank you Ralph
>
> I will ponder my options.
>
> In case it did not come out clearly, here is an example of what I see in some emails if code page is set to Unicode FULL UTF-8
>
> https://ibb.co/0DXJ1JT

The sender writes "here’s" and it comes out in your browser as "here’s".
No plain ASCII apostrophe either side.

Three things you can do:
a) Either, upgrade to Agent 7.00 or later.
b) Or, keep using Agent 6.00, but in your browser set these
particular emails' character encoding to UTF-8.
c) Or, keep using Agent 6.00, but for these particular emails the
language in Agent should have its Code Page set to windows-1252.

If you want to do (c) then you will find it easier if you do it this
way:
1. Have two different languages in Agent
* English (International), Code Page set to windows-1252
* Unicode UTF-8 (English), Code Page set to UTF-8
2. Use the Edit >> Language menu to switch the email to one or
the other language.

--
Kind regards
Ralph

On ælcere ea swa wyrse fordes, swa betere fisces.

On Tue, 29 Nov 2022 20:47:04 +1300, Ralph Fox <-rf-nz-@-.invalid> wrote:

>On Mon, 28 Nov 2022 10:43:20 +0000, Me wrote:
>> On Sun, 27 Nov 2022 21:38:49 +1300, Ralph Fox <-rf-nz-@-.invalid> wrote:
>>> On Fri, 25 Nov 2022 12:56:32 +0000, Me wrote:
>>>> On Thu, 24 Nov 2022 10:42:57 +0000, Me <Me@me.com> wrote:
>>>>
>>>>> I will leave everything alone now, knowing that having changed the Code Page to Unicode FULL UTF-8, it fixed the problem.
>>>>
>>>> Actually, a new problem has occurred ...
>>>>
>>>> For example, having changed the code page to Unicode FULL UTF-8, in a normal text e-mail, the word "here's" now comes out as "here’s"
>>>>
>>>> Is there any way I can have the best of both worlds - ie using Unicode FULL UTF-8 to sort out my browser issue, and also use parts of windows-1252 so that I see "here's" and not "here’s"?
>>>
>>>
>>> I need to update my answer to an earlier question of yours:
>>>
>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ QUOTE ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>> | When opening an HTML email using Control + L (launch in browser), the
>>> | page always opens with what appears to be the wrong character encoding.
>>> |
>>> | Instead of an inverted comma, I see a black diamond with a ? inside.
>>> |
>>> | I am using Chrome as my default browser and I have a Chrome extension
>>> | that allows me to set the character encoding back to Western
>>> | (Windows-1252) and then all is well.
>>> |
>>> | I just wondered if this was an Agent issue or a browser issue?
>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ QUOTE ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>
>>>
>>> There is an Agent v6.00 issue which can cause this. That issue was
>>> fixed in Agent v7.00. I am using Agent 8.00 where this issue does not
>>> occur, but you are using Agent 6.00 where this issue does exist.
>>>
>>> There are two different things you could do about this Agent 6.00 issue
>>> a) Either, upgrade to Agent 7.00 or later.
>>> b) Or, keep using Agent 6.00 but change the message's language
>>> to one whose code page is the same as the email's charset (these
>>> days, emails are often UTF-8).
>>>
>>> ====
>>>
>>> However, I cannot explain how "here's" comes out as "here’s".
>>> Even in Agent 6.00, I see this:
>>>
>>> 1. If the sender wrote "here's", I will always see "here's". Never
>>> "here’s".
>>>
>>> 2. If the sender wrote "here’s", I will always see "here’s" or
>>> "here?s". Never "here's".
>>
>> Thank you Ralph
>>
>> I will ponder my options.
>>
>> In case it did not come out clearly, here is an example of what I see in some emails if code page is set to Unicode FULL UTF-8
>>
>> https://ibb.co/0DXJ1JT
>
>
>The sender writes "here’s" and it comes out in your browser as "here’s".
>No plain ASCII apostrophe either side.
>
>Three things you can do:
> a) Either, upgrade to Agent 7.00 or later.
> b) Or, keep using Agent 6.00, but in your browser set these
> particular emails' character encoding to UTF-8.
> c) Or, keep using Agent 6.00, but for these particular emails the
> language in Agent should have its Code Page set to windows-1252.
>
>
>If you want to do (c) then you will find it easier if you do it this
>way:
> 1. Have two different languages in Agent
> * English (International), Code Page set to windows-1252
> * Unicode UTF-8 (English), Code Page set to UTF-8
> 2. Use the Edit >> Language menu to switch the email to one or
> the other language.

Many thanks again Ralph for the very comprehensive explanation.