R.Wieser <address@not.available> wrote:
> Hello all,
>
> Question #1 :
>
> At some point I noticed that my RPi 3, used as a YouTube viewer, sometimes
> seemed to freeze and also responding very slowly (20-30 sec) when I, using
> the "Windows" key, call the system menu to shut down. After than it
> sometimes responded almost as slowly to present me with the shutdown
> confirmation dialog.
>
> I'm /assuming/ that it has something to do with the micro-SD cards
> performance degrading, perhaps because of having to swap blocks around to
> egalize wear-and-tear. Could someone confirm this or, if this is not the
> case, tell me what is going on ?

This describes how to check the condition using mmc-utils, if the
Micro SD card supports it (and you can understand what it says):
https://developer.toradex.com/knowledge-base/emmc-linux#Configuration_and_Monitoring

I ought to try it myself some time...

> Question #2:
> I attempted to make a DD copy of the micro-SD card to another (to check if
> it was the card or perhaps the installed software), but that failed because
> the newer, also 16 GB card, was 20 MB smaller than the other origional card
> (rather unexpected I might say). :-\
>
> Is there a way to tell a micro-SD card that its a bit smaller than it
> actually is, so that the installed image (NOOBS 3.2.1) will be smaller and a
> DD of it will always fit on a new card ?

I don't think that's the right approach. You'd probably be better
off by making the partitions stop some safe distance from the end,
then using the "count" option to dd (which is multiplied by the
"bs" setting) to limit the length of data read or written.

Or if you want to be able to downgrade SD card size more freely,
you could try using fsarchiver instead of dd. It can restore to
differently sized partitions, but you have to archive individual
partitions instead of the whole disk. That means you have to create
the partitions again before restoring the archive to a new SD card.

--
__ __
#_ < |\| |< _#

Kev,

> This describes how to check the condition using mmc-utils, if
> the Micro SD card supports it (and you can understand what
> it says):
> https://developer.toradex.com/knowledge-base/emmc-linux#Configuration_and_Monitoring

Thank you. I was hoping for something like that.

> I don't think that's the right approach. You'd probably be
> better off by making the partitions stop some safe distance
> from the end,

As mentioned, I'm using a NOOBS image. AFAIK I've got zero control over
which, and with which sizes it creates the partitions (it will just fill out
all available space).

Besides, DD doesn't care about partitions, so I don't think that that
solution will solve much of anything - unless I could somehow tell DD to
stop where the last partition ends ...

> Or if you want to be able to downgrade SD card size more
> freely, you could try using fsarchiver instead of dd.
.....
> That means you have to create the partitions again before
> restoring the archive to a new SD card.

Not what I expect to be doing on a simple restoring/duplication ...

> It can restore to differently sized partitions, but you have
> to archive individual partitions instead of the whole disk.

I take it that it only backups the actual used space in the partition. If
so, the actual backup would become smaller, which is a pre. On the other
hand the ammount and complexity of work to restore it would increase, which
is a con - especially having to recreate those partitions by hand (from
what information?).

Facing that I would probably go for the simpler method, and just grab myself
a handfull of identical SD cards. :-)

Thanks for the info.

Regards,
Rudy Wieser

On 30/09/2021 09:09, Computer Nerd Kev wrote:
> R.Wieser <address@not.available> wrote:
>> Is there a way to tell a micro-SD card that its a bit smaller than it
>> actually is, so that the installed image (NOOBS 3.2.1) will be smaller and a
>> DD of it will always fit on a new card ?
>
> I don't think that's the right approach. You'd probably be better
> off by making the partitions stop some safe distance from the end,
> then using the "count" option to dd (which is multiplied by the
> "bs" setting) to limit the length of data read or written.

dd is pretty safe as it will stop when it gets to the end of the source
or destination.

The key bit is to make the last partition a bit smaller so your image
will fit on any SD card claiming to be the same size, as they can vary
by up to 100s of MB.

That's easy with a normal single OS installation as it will have just
the boot (FAT) and root (ext4) partitions. Put the SD card in another
Linux machine and use gparted resize the last partition so there is
enough space after it to allow it fir on another card.

NOOBs is a bit more complicated as it has multiple OS's and puts their
boot and root partitions inside extended partitions. It is possible to
resize the last root partition and then shrink the extended partition,
but it's more involved.

As ever the recommendation is try out different OS's on a NOOBs card,
but once you've decided which one(s) you want to use, put them on a
dedicated SD card with a simple partition layout, and avoid a lot of grief.

---druck

Druck,

> dd is pretty safe as it will stop when it gets to the end of the source or
> destination.

:-) It would be rather safe if it would continue too. Its just that on
reading you get large image files, and on writing you waste a lot of time
with out anything to show for.

Also, it depends on your definition of "safe" : if it stops with writing
because the target (micro-SD card) is smaller than the source (image file)
than you stand a good chance of an unusable, or perhaps even crashing copy.
:-|

Though in my case my DD program just told me that the target micro-SD card
was to small to contain the source image file, and stopped. Rightly so.

> Put the SD card in another Linux machine and use gparted resize the last
> partition so there is enough space after it to allow it fir on another
> card.

Yikes! And than just run a DD in a "just do it, ignore that its too big"
mode ? That feels to me as if I'm just asking for trouble (see above) ...

On the other hand, I would not mind telling DD to include all partitions,
ignoring any (slack) space after the last one though.

> It is possible to resize the last root partition and then shrink the
> extended partition, but it's more involved.

Hence my idea to make the NOOBS installation think the micro-SD card is
actually smaller than it is ...

> As ever the recommendation is try out different OS's on a NOOBs card, but
> once you've decided which one(s) you want to use, put them on a dedicated
> SD card with a simple partition layout, and avoid a lot of grief.

Do you have a bit more info about that ? I would not mind having the NOOBS
pre-boot "press shift to reinstall" stuff removed.

Regards,
Rudy Wieser

Dana Thu, 30 Sep 2021 12:36:24 +0200, R.Wieser <address@not.available> napis'o:
> Kev,
>
>> This describes how to check the condition using mmc-utils, if
>> the Micro SD card supports it (and you can understand what
>> it says):
>> https://developer.toradex.com/knowledge-base/emmc-linux#Configuration_and_Monitoring
>
> Thank you. I was hoping for something like that.
>
>> I don't think that's the right approach. You'd probably be
>> better off by making the partitions stop some safe distance
>> from the end,
>
> As mentioned, I'm using a NOOBS image. AFAIK I've got zero control over
> which, and with which sizes it creates the partitions (it will just fill out
> all available space).
>
> Besides, DD doesn't care about partitions, so I don't think that that
> solution will solve much of anything - unless I could somehow tell DD to
> stop where the last partition ends ...

It knows what the size of partition is. It is written in the
partition table.

You could make partition smaller.
Look at resize2fs

After that you could also make partition smaller in the partition table.
That way you will be able to copy from one card to another.
In case you want to "reserve" the end of the card... you can make dummy
partition at the end. With some kind of "weird" type your pi would not
try to mount.

On Thu, 30 Sep 2021 13:54:46 +0200, "R.Wieser" <address@not.available>
declaimed the following:

>Hence my idea to make the NOOBS installation think the micro-SD card is
>actually smaller than it is ...
>

I don't think the Raspberry-Pi foundation even distributes NOOBS format
any more. The closest is the "Raspberry-Pi Imager" -- which runs on
Windows/Macs (and strangely, listed for Ubuntu even though R-Pi OS is based
on Debian).

As I understand it, this program does a network download of the
selected OS and writes just it to an SD card. One does not have the NOOBS
overhead of having a couple of different OS on the card (along with the
network install OS). {the imager installer is just 20MB for Windows
machines -- so it obviously can not contain a 3GB OS image itself}

[Trimmed]
C:\Users\Wulfraed>dir e:\MicroSupport\RPI3_4
Directory of e:\MicroSupport\RPI3_4

03/28/2021 06:28 PM 3,007,638,235
2021-03-04-raspios-buster-armhf-full.zip

06/23/2021 06:27 PM 3,005,999,855
2021-05-07-raspios-buster-armhf-full.zip

09/18/2021 12:29 PM 19,772,672 imager_1.6.2.exe

10/13/2020 03:09 PM 2,441,652,758 NOOBS_v3_5_0.zip

>Do you have a bit more info about that ? I would not mind having the NOOBS
>pre-boot "press shift to reinstall" stuff removed.

Not using an out-of-date NOOBS image might be an option...
https://www.raspberrypi.org/software/operating-systems/
The last NOOBS image was 3.5.0, and I show a download date a year ago. In
the meantime, there have been at least TWO RaspiOS images made available.
(See above listing)

Still won't do anything for the partition size filling the SD card --
when different brands of cards may reserve different amounts of blocks for
bad-block remapping.

--
Wulfraed Dennis Lee Bieber AF6VN
wlfraed@ix.netcom.com http://wlfraed.microdiversity.freeddns.org/

On Thu, 30 Sep 2021 10:26:23 -0400, Dennis Lee Bieber wrote:

> On Thu, 30 Sep 2021 13:54:46 +0200, "R.Wieser" <address@not.available>
> declaimed the following:
>
>
>>Hence my idea to make the NOOBS installation think the micro-SD card is
>>actually smaller than it is ...
>>
>>
> I don't think the Raspberry-Pi foundation even distributes NOOBS
format
> any more. The closest is the "Raspberry-Pi Imager" -- which runs on
> Windows/Macs (and strangely, listed for Ubuntu even though R-Pi OS is
> based on Debian).
>
> As I understand it, this program does a network download of the
> selected OS and writes just it to an SD card. One does not have the
> NOOBS overhead of having a couple of different OS on the card (along
> with the network install OS). {the imager installer is just 20MB for
> Windows machines -- so it obviously can not contain a 3GB OS image
> itself}
>
> [Trimmed]
> C:\Users\Wulfraed>dir e:\MicroSupport\RPI3_4
> Directory of e:\MicroSupport\RPI3_4
>
> 03/28/2021 06:28 PM 3,007,638,235
> 2021-03-04-raspios-buster-armhf-full.zip
>
> 06/23/2021 06:27 PM 3,005,999,855
> 2021-05-07-raspios-buster-armhf-full.zip
>
> 09/18/2021 12:29 PM 19,772,672 imager_1.6.2.exe
>
> 10/13/2020 03:09 PM 2,441,652,758 NOOBS_v3_5_0.zip
>
>>Do you have a bit more info about that ? I would not mind having the
>>NOOBS pre-boot "press shift to reinstall" stuff removed.
>
> Not using an out-of-date NOOBS image might be an option...
> https://www.raspberrypi.org/software/operating-systems/
> The last NOOBS image was 3.5.0, and I show a download date a year ago.
> In the meantime, there have been at least TWO RaspiOS images made
> available. (See above listing)
>
> Still won't do anything for the partition size filling the SD
card --
> when different brands of cards may reserve different amounts of blocks
> for bad-block remapping.

You'll need a USB connected SD card reader. Put a new SD card in it and
connect it to the RPi.

Use the 'parted' command-line to format the new SD card with two
partitions:
- partition no 1: a 1 GB fat32 partition with its boot flag set
- partition no 2: an ext4 partition occupying the rest of the card

Use the 'dd' utility to copy /dev/mmcblk0p1 into the first partition and
/dev/root into the second partition.

Now the RPi should boot from the new card.

Yes, I know that the RPi OS installation is currently only 50 MB, but
disk space is cheap these days and the process of upgrading the OS will
require the boot partition to be at least twice the size of the installed
operation system, because at one point in the upgrade process both the
old and new versions are on the partition at the same time.

If you're using a tiny SD card, you can make the boot partition somewhat
smaller, but don't make it less than 200GB or an upgrade may fail. Don't
ask how I know this...

Lastly, if you're not already using 'apt' to do regular system upgrades
over the 'net, start using it: its faster than messing round with Noobs
and easy once you've done it the first time. Just run:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade
sudo apt-get autoremove
sudo apt-get autoclean
sudo reboot

And, of course you can always save time and typing by putting that
command sequence in a shell script and running that.

I always make a system backup before doing an upgrade but ymmv.

--
--
Martin | martin at
Gregorie | gregorie dot org

On 30/09/2021 14:02, Nikolaj Lazic wrote:
> You could make partition smaller.
> Look at resize2fs
>
> After that you could also make partition smaller in the partition table.

This is what gparted does graphically, in a much more friendly way and
less disaster prone way than using resize2fs and parted manually.

> That way you will be able to copy from one card to another.

Once the partitions are correct, you can just copy with dd, and it will
stop automatically when it has filled the new card.

> In case you want to "reserve" the end of the card... you can make dummy
> partition at the end. With some kind of "weird" type your pi would not
> try to mount.

No don't do this, the system will object is there isn't room for this
dummy partition on the new card. Just leave the space empty, then the
system doesn't care if it is there or isn't there.

---druck

On 30/09/2021 12:54, R.Wieser wrote:
> Druck,
>
>> dd is pretty safe as it will stop when it gets to the end of the source or
>> destination.
>
> :-) It would be rather safe if it would continue too. Its just that on
> reading you get large image files, and on writing you waste a lot of time
> with out anything to show for.
>
> Also, it depends on your definition of "safe" : if it stops with writing
> because the target (micro-SD card) is smaller than the source (image file)
> than you stand a good chance of an unusable, or perhaps even crashing copy.
> :-|

Not if you have resized the partitions first with gparted as I advised.

> Though in my case my DD program just told me that the target micro-SD card
> was to small to contain the source image file, and stopped. Rightly so.
>
>> Put the SD card in another Linux machine and use gparted resize the last
>> partition so there is enough space after it to allow it fir on another
>> card.
>
> Yikes! And than just run a DD in a "just do it, ignore that its too big"
> mode ? That feels to me as if I'm just asking for trouble (see above) ...

Right this is turning in to another one of these troll threads where
you ask for advice, and then deliberately ignore / misinterpret it to
cause a pointless argument. Pack it in.

---druck

On 30/09/2021 17:08, Martin Gregorie wrote:
> Use the 'parted' command-line to format the new SD card with two
> partitions:
> - partition no 1: a 1 GB fat32 partition with its boot flag set
> - partition no 2: an ext4 partition occupying the rest of the card
>
> Use the 'dd' utility to copy /dev/mmcblk0p1 into the first partition and
> /dev/root into the second partition.
>
> Now the RPi should boot from the new card.

This wont work unless the two partitions are exactly the same size on
both cards, which they can't be if the new card is smaller.

If you are manually creating new partitions as above, don't use dd, but
copy all the files with:-

rsync -axP /mnt/source/boot/ /mnt/dest/boot
rsync -axVAXP /mnt/source/root /mnt/dest/root

---druck

On 01/10/2021 07:54, druck wrote:
> If you are manually creating new partitions as above, don't use dd, but
> copy all the files with:-
>
> rsync -axP /mnt/source/boot/ /mnt/dest/boot
> rsync -axVAXP /mnt/source/root /mnt/dest/root

This should be /mnt/source/root/ note the trailing slash or it will copy
to /mnt/dest/root/root

---druck

Dennis,

> I don't think the Raspberry-Pi foundation even distributes NOOBS
> format any more.

That I'm using a NOOBS installation is irrelevant. My question was aimed
at making *an* installation think the micro-SD card would be smaller than it
actually is. IOW, changing something about the micro-SD card /before/
throwing an OS at it.

> As I understand it, this program does a network download of
> the selected OS and writes just it to an SD card.

I only install from stuff I have already downloaded. That way I can
re-install and have exactly the same as I did before. IOW, I rather
dislike (understatement) the "just download it again and deal with whatever
changes they made" approach.

> Not using an out-of-date NOOBS image might be an option...

:-) Don't try to tell me that removing the "repair" part is something they
only thought of after NOOBS 3.5.0.

> Still won't do anything for the partition size filling the SD card --

Indeed. And as I've not seen you mention anything about that in regard to
whatever the latest available RPi OS version is (if-and-when it still
supports my aging RPi ofcourse) ...

Regards,
Rudy Wieser

Martin,

> Use the 'parted' command-line to format the new SD card with two
> partitions:
> - partition no 1: a 1 GB fat32 partition with its boot flag set
> - partition no 2: an ext4 partition occupying the rest of the card
>
> Use the 'dd' utility to copy /dev/mmcblk0p1 into the first partition and
> /dev/root into the second partition.

Thanks, but I'm not sure I follow. Should I understand from this that
RPis 'dd' utility will automatically resize the source partition onto the
target one ? Or is something else going on ?

Regards,
Rudy Wieser

druck

> Not if you have resized the partitions first with gparted as I advised.

You misunderstood what I was trying to say : You're suggesting to use a
hack, ignoring the "target is too small" warning I take your 'dd' gives
(mine on Windows won't even start the copy) and have not mentioning anything
in regard to checking if it actually went alright. Not cool dude, not cool.

> Right this is turning in to another one of these troll threads where you
> ask for advice, and then deliberately ignore / misinterpret it to cause a
> pointless argument.

I understood your advice well enough. In fact well enough to point out to
you that the result of your hack could not be trusted. And now *I'm* the
bad guy ?

> Pack it in.

To use another, similar phrase : "go take a hike".

Regards,
Rudy Wieser

On 01/10/2021 13:51, R.Wieser wrote:
> Martin,
>
>> Use the 'parted' command-line to format the new SD card with two
>> partitions:
>> - partition no 1: a 1 GB fat32 partition with its boot flag set
>> - partition no 2: an ext4 partition occupying the rest of the card
>>
>> Use the 'dd' utility to copy /dev/mmcblk0p1 into the first partition and
>> /dev/root into the second partition.
>
> Thanks, but I'm not sure I follow. Should I understand from this that
> RPis 'dd' utility will automatically resize the source partition onto the
> target one ? Or is something else going on ?

See drucks post. The problem is that it *wont*.

create partitions manually and then transfer files as per drucks rsync cribs

I see no merit whatsoever in using DD on partitions - once you are at
the file/directory level, use rsync to copy files

The merit of DD is that it copies the whole disk image. So can recreate
partitions.

--
“The urge to save humanity is almost always only a false face for the
urge to rule it.”
– H. L. Mencken

On Fri, 1 Oct 2021 14:40:56 +0100
The Natural Philosopher <tnp@invalid.invalid> wrote:

> I see no merit whatsoever in using DD on partitions - once you are at
> the file/directory level, use rsync to copy files

There's are reasons to use it, dd will copy any kind of partition
correctly even ones you can't mount while rsync can be troublesome in the
face of devices, sparse files, named pipes, mounts and the like. It may
also be faster if the filesystem is fairly full with a great many small
files.

There are reasons not to use it too, primarily that it will copy all
the unused space and can only copy to a target at least as large as the
partition.

--
Steve O'Hara-Smith
Odds and Ends at http://www.sohara.org/