I have a very short Python program that runs on one of my Raspberry
Pis to collect temperatures from a 1-wire sensor and write them to a
database:-

#!/usr/bin/python3
#
#
# read temperature from 1-wire sensor and store in database with date and time
#
import sqlite3
import time

ftxt = str(open("/sys/bus/w1/devices/28-01204e1e64c3/w1_slave").read(100))
temp = (float(ftxt[ftxt.find("t=") +2:]))/1000
#
#
# insert date, time and temperature into the database
#
tdb = sqlite3.connect("/home/chris/.cfg/share/temperature/temperature.db")
cr = tdb.cursor()
dt = time.strftime("%Y-%m-%d %H:%M")
cr.execute("Insert INTO temperatures (DateTime, Temperature) VALUES(?, round(?, 2))", (dt, temp)
)
tdb.commit()
tdb.close()

It's run by cron every 10 minutes.

At 03:40 last night it suddenly started throwing the following error every
time it ran:-

Fatal Python error: initfsencoding: Unable to get the locale encoding
LookupError: unknown encoding: UTF-8

Current thread 0xb6f8db40 (most recent call first):
Aborted

Running the program from the command line produced the same error.
Restarting the Pi system has fixed the problem.

What could have caused this? I certainly wasn't around at 03:40! :-)
There aren't any automatic updates enabled on the system, the only
thing that might have been going on was a backup as that Pi is also
my 'NAS' with a big USB drive connected to it. The backups have been
running without problems for more than a year. Looking at the system
logs shows that a backup was started at 03:35 so I suppose that *could*
have provoked something but I fail to understand how.

--
Chris Green
·

On 07/12/2021 16:28, Chris Green wrote:
> I have a very short Python program that runs on one of my Raspberry
> Pis to collect temperatures from a 1-wire sensor and write them to a
> database:-
>
> #!/usr/bin/python3
> #
> #
> # read temperature from 1-wire sensor and store in database with date and time
> #
> import sqlite3
> import time
>
> ftxt = str(open("/sys/bus/w1/devices/28-01204e1e64c3/w1_slave").read(100))
> temp = (float(ftxt[ftxt.find("t=") +2:]))/1000
> #
> #
> # insert date, time and temperature into the database
> #
> tdb = sqlite3.connect("/home/chris/.cfg/share/temperature/temperature.db")
> cr = tdb.cursor()
> dt = time.strftime("%Y-%m-%d %H:%M")
> cr.execute("Insert INTO temperatures (DateTime, Temperature) VALUES(?, round(?, 2))", (dt, temp)
> )
> tdb.commit()
> tdb.close()
>
> It's run by cron every 10 minutes.
>
>
> At 03:40 last night it suddenly started throwing the following error every
> time it ran:-
>
> Fatal Python error: initfsencoding: Unable to get the locale encoding
> LookupError: unknown encoding: UTF-8
>
> Current thread 0xb6f8db40 (most recent call first):
> Aborted
>
> Running the program from the command line produced the same error.
> Restarting the Pi system has fixed the problem.
>
>
> What could have caused this? I certainly wasn't around at 03:40! :-)
> There aren't any automatic updates enabled on the system, the only
> thing that might have been going on was a backup as that Pi is also
> my 'NAS' with a big USB drive connected to it. The backups have been
> running without problems for more than a year. Looking at the system
> logs shows that a backup was started at 03:35 so I suppose that *could*
> have provoked something but I fail to understand how.

Since it's a one-off, doesn't sound like a system problem. The easiest
might be that you try-catch that call and retry when needed, and I'd
also check that 'ftxt' is what it should be: "external devices" may
fail, including when they do produce output...

Julio

Julio Di Egidio <julio@diegidio.name> wrote:
> On 07/12/2021 16:28, Chris Green wrote:
> > I have a very short Python program that runs on one of my Raspberry
> > Pis to collect temperatures from a 1-wire sensor and write them to a
> > database:-
> >
> > #!/usr/bin/python3
> > #
> > #
> > # read temperature from 1-wire sensor and store in database with date and time
> > #
> > import sqlite3
> > import time
> >
> > ftxt = str(open("/sys/bus/w1/devices/28-01204e1e64c3/w1_slave").read(100))
> > temp = (float(ftxt[ftxt.find("t=") +2:]))/1000
> > #
> > #
> > # insert date, time and temperature into the database
> > #
> > tdb = sqlite3.connect("/home/chris/.cfg/share/temperature/temperature.db")
> > cr = tdb.cursor()
> > dt = time.strftime("%Y-%m-%d %H:%M")
> > cr.execute("Insert INTO temperatures (DateTime, Temperature) VALUES(?,
> round(?, 2))", (dt, temp)
> > )
> > tdb.commit()
> > tdb.close()
> >
> > It's run by cron every 10 minutes.
> >
> >
> > At 03:40 last night it suddenly started throwing the following error every
> > time it ran:-
> >
> > Fatal Python error: initfsencoding: Unable to get the locale encoding
> > LookupError: unknown encoding: UTF-8
> >
> > Current thread 0xb6f8db40 (most recent call first):
> > Aborted
> >
> > Running the program from the command line produced the same error.
> > Restarting the Pi system has fixed the problem.
> >
> >
> > What could have caused this? I certainly wasn't around at 03:40! :-)
> > There aren't any automatic updates enabled on the system, the only
> > thing that might have been going on was a backup as that Pi is also
> > my 'NAS' with a big USB drive connected to it. The backups have been
> > running without problems for more than a year. Looking at the system
> > logs shows that a backup was started at 03:35 so I suppose that *could*
> > have provoked something but I fail to understand how.
>
> Since it's a one-off, doesn't sound like a system problem. The easiest
> might be that you try-catch that call and retry when needed, and I'd
> also check that 'ftxt' is what it should be: "external devices" may
> fail, including when they do produce output...
>
Well it repeated every ten minutes through the night until I rebooted
the system, so it wasn't really a "one off". I hasn't repeated since
though.

--
Chris Green
·

On 08/12/2021 10:50, Chris Green wrote:
> Julio Di Egidio <julio@diegidio.name> wrote:
>> On 07/12/2021 16:28, Chris Green wrote:
>>> I have a very short Python program that runs on one of my Raspberry
>>> Pis to collect temperatures from a 1-wire sensor and write them to a
>>> database:-
>>>
>>> #!/usr/bin/python3
>>> #
>>> #
>>> # read temperature from 1-wire sensor and store in database with date and time
>>> #
>>> import sqlite3
>>> import time
>>>
>>> ftxt = str(open("/sys/bus/w1/devices/28-01204e1e64c3/w1_slave").read(100))
>>> temp = (float(ftxt[ftxt.find("t=") +2:]))/1000
>>> #
>>> #
>>> # insert date, time and temperature into the database
>>> #
>>> tdb = sqlite3.connect("/home/chris/.cfg/share/temperature/temperature.db")
>>> cr = tdb.cursor()
>>> dt = time.strftime("%Y-%m-%d %H:%M")
>>> cr.execute("Insert INTO temperatures (DateTime, Temperature) VALUES(?,
>> round(?, 2))", (dt, temp)
>>> )
>>> tdb.commit()
>>> tdb.close()
>>>
>>> It's run by cron every 10 minutes.
>>>
>>>
>>> At 03:40 last night it suddenly started throwing the following error every
>>> time it ran:-
>>>
>>> Fatal Python error: initfsencoding: Unable to get the locale encoding
>>> LookupError: unknown encoding: UTF-8
>>>
>>> Current thread 0xb6f8db40 (most recent call first):
>>> Aborted
>>>
>>> Running the program from the command line produced the same error.
>>> Restarting the Pi system has fixed the problem.
>>>
>>>
>>> What could have caused this? I certainly wasn't around at 03:40! :-)
>>> There aren't any automatic updates enabled on the system, the only
>>> thing that might have been going on was a backup as that Pi is also
>>> my 'NAS' with a big USB drive connected to it. The backups have been
>>> running without problems for more than a year. Looking at the system
>>> logs shows that a backup was started at 03:35 so I suppose that *could*
>>> have provoked something but I fail to understand how.
>>
>> Since it's a one-off, doesn't sound like a system problem. The easiest
>> might be that you try-catch that call and retry when needed, and I'd
>> also check that 'ftxt' is what it should be: "external devices" may
>> fail, including when they do produce output...
>>
> Well it repeated every ten minutes through the night until I rebooted
> the system, so it wasn't really a "one off". I hasn't repeated since
> though.

Still your code wouldn't pass review: you do need some exception
handling there, that it's not failing right now is no excuse. Moreover,
next time it fails, you won't be any wiser since you keep not handling
it... Unless this is all just for fun, in which case of course never mind.

HTH,

Julio

Chris Green wrote at 2021-12-7 15:28 +0000:
>I have a very short Python program that runs on one of my Raspberry
>Pis to collect temperatures from a 1-wire sensor and write them to a
>database:-
> ...
>At 03:40 last night it suddenly started throwing the following error every
>time it ran:-
>
> Fatal Python error: initfsencoding: Unable to get the locale encoding
> LookupError: unknown encoding: UTF-8
>
> Current thread 0xb6f8db40 (most recent call first):
> Aborted
>
>Running the program from the command line produced the same error.
>Restarting the Pi system has fixed the problem.

Python has not its own locale database but uses that of the operating
system. From my point of view, the operating system state seems
to have got corrupted. A restart apparently has ensured a clean state again.

On Wed, Dec 8, 2021 at 2:52 AM Chris Green <cl@isbd.net> wrote:
>
>
> At 03:40 last night it suddenly started throwing the following error every
> time it ran:-
>
> Fatal Python error: initfsencoding: Unable to get the locale encoding
> LookupError: unknown encoding: UTF-8
>
> Current thread 0xb6f8db40 (most recent call first):
> Aborted
>
> Running the program from the command line produced the same error.
> Restarting the Pi system has fixed the problem.
>

This error means Python can not find its standard libraries. There are
some possibilities.

* You set the wrong PYTHONHOME
PYTHONHOME is very rarely useful. It shouldn't be used if you can
not solve this kind of problem.

* Your Python installation is broken.
Some files are deleted or overwritten. You need to *clean* install
Python again.

Bets,
--
Inada Naoki <songofacandy@gmail.com>

Julio Di Egidio <julio@diegidio.name> wrote:
> On 08/12/2021 10:50, Chris Green wrote:
> > Julio Di Egidio <julio@diegidio.name> wrote:
> >> On 07/12/2021 16:28, Chris Green wrote:
> >>> What could have caused this? I certainly wasn't around at 03:40! :-)
> >>> There aren't any automatic updates enabled on the system, the only
> >>> thing that might have been going on was a backup as that Pi is also
> >>> my 'NAS' with a big USB drive connected to it. The backups have been
> >>> running without problems for more than a year. Looking at the system
> >>> logs shows that a backup was started at 03:35 so I suppose that *could*
> >>> have provoked something but I fail to understand how.
> >>
> >> Since it's a one-off, doesn't sound like a system problem. The easiest
> >> might be that you try-catch that call and retry when needed, and I'd
> >> also check that 'ftxt' is what it should be: "external devices" may
> >> fail, including when they do produce output...
> >>
> > Well it repeated every ten minutes through the night until I rebooted
> > the system, so it wasn't really a "one off". I hasn't repeated since
> > though.
>
> Still your code wouldn't pass review: you do need some exception
> handling there, that it's not failing right now is no excuse. Moreover,
> next time it fails, you won't be any wiser since you keep not handling
> it... Unless this is all just for fun, in which case of course never mind.
>
It is basically 'just for fun' we wanted to see how warm/cold the
cats' home above our garage was now that winter has arrived.

However catching and re-trying isn't going to help at all. It happily
produced the same arror every 10 minutes throughout the night until I
rebooted the system. I suppose I *could* reboot after (say) three
failures but it seems a bit drastic! :-)

--
Chris Green
·

Inada Naoki <songofacandy@gmail.com> wrote:
> On Wed, Dec 8, 2021 at 2:52 AM Chris Green <cl@isbd.net> wrote:
> >
> >
> > At 03:40 last night it suddenly started throwing the following error every
> > time it ran:-
> >
> > Fatal Python error: initfsencoding: Unable to get the locale encoding
> > LookupError: unknown encoding: UTF-8
> >
> > Current thread 0xb6f8db40 (most recent call first):
> > Aborted
> >
> > Running the program from the command line produced the same error.
> > Restarting the Pi system has fixed the problem.
> >
>
> This error means Python can not find its standard libraries. There are
> some possibilities.
>
> * You set the wrong PYTHONHOME
> PYTHONHOME is very rarely useful. It shouldn't be used if you can
> not solve this kind of problem.
>
> * Your Python installation is broken.
> Some files are deleted or overwritten. You need to *clean* install
> Python again.
>
So how was it that rebooting the system fixed the error? It has been
working perfectly OK now for a few days since the above failure and
I've changed nothing.

I think it was probably a bit of power supply noise or something that
corrupted a memory read or write, or something of that sort anyway.

--
Chris Green
·

On 09/12/2021 11:38, Chris Green wrote:
> Julio Di Egidio <julio@diegidio.name> wrote:
>> On 08/12/2021 10:50, Chris Green wrote:
>>> Julio Di Egidio <julio@diegidio.name> wrote:
>>>> On 07/12/2021 16:28, Chris Green wrote:
>>>>> What could have caused this? I certainly wasn't around at 03:40! :-)
>>>>> There aren't any automatic updates enabled on the system, the only
>>>>> thing that might have been going on was a backup as that Pi is also
>>>>> my 'NAS' with a big USB drive connected to it. The backups have been
>>>>> running without problems for more than a year. Looking at the system
>>>>> logs shows that a backup was started at 03:35 so I suppose that *could*
>>>>> have provoked something but I fail to understand how.
>>>>
>>>> Since it's a one-off, doesn't sound like a system problem. The easiest
>>>> might be that you try-catch that call and retry when needed, and I'd
>>>> also check that 'ftxt' is what it should be: "external devices" may
>>>> fail, including when they do produce output...
>>>>
>>> Well it repeated every ten minutes through the night until I rebooted
>>> the system, so it wasn't really a "one off". I hasn't repeated since
>>> though.
>>
>> Still your code wouldn't pass review: you do need some exception
>> handling there, that it's not failing right now is no excuse. Moreover,
>> next time it fails, you won't be any wiser since you keep not handling
>> it... Unless this is all just for fun, in which case of course never mind.
>>
> It is basically 'just for fun' we wanted to see how warm/cold the
> cats' home above our garage was now that winter has arrived.
>
> However catching and re-trying isn't going to help at all. It happily
> produced the same arror every 10 minutes throughout the night until I
> rebooted the system.

OK, I see.

> I suppose I *could* reboot after (say) three
> failures but it seems a bit drastic! :-)

If all else fails, it'd still be better than waking up at night... :)

To me it seems to hint at a problem with Pi's file system itself, not
being able to access files, since it only happens when backups start
running: but then I wouldn't know (and I have zero experience with Pi),
maybe a driver needs updating, or an incompatibility of drivers, maybe
with the backup app, maybe someone/noone else is having the same
problem?, etc.

So, good luck and, should you find the root cause of the problem, it
would be interesting to know what it was.

Cheers,

Julio

On 2021-12-09 10:38:58 +0000, Chris Green wrote:
> Julio Di Egidio <julio@diegidio.name> wrote:
> > Still your code wouldn't pass review: you do need some exception
> > handling there
[...]
> However catching and re-trying isn't going to help at all. It happily
> produced the same arror every 10 minutes throughout the night until I
> rebooted the system.

I agree. Only catch exceptions which you can reasonably handle. For
those which you can't handle (and "some part of my Python installation
has just gone AWOL" is one of them) crashing with a stack trace is
almost certainly your best option.

> I suppose I *could* reboot after (say) three failures

Maybe, but I wouldn't do that in the python program itself, but in a
separate monitor process.

> but it seems a bit drastic! :-)

Yup. I'd do that only if I've run out of other options (which has
happened in the past and probably will happen in the future).

hp

--
_ | Peter J. Holzer | Story must make more sense than reality.
|_|_) | |
| | | hjp@hjp.at | -- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"