(I've never had occasion to use git.)

After installing git,

1) What should I read to learn how to "check out" (obtain) the source
for some particular piece of software from git.freebsd.org?

2) When the code is part of a larger work, such as one particular X11
driver, will I ultimately have to build all of X11 just to build the
one driver, or can I build just the driver using the
already-installed X11 include files and libraries?

3) Do the source files obtained by git normally go in /usr/ports/, or
does location not matter?

TIA,
-WBE

Winston <wbe@UBEBLOCK.psr.com.invalid> writes:

> (I've never had occasion to use git.)

That's fine. You don't seem to be a developer, so I don't see why you
would have.

> After installing git,
>
> 1) What should I read to learn how to "check out" (obtain) the source
> for some particular piece of software from git.freebsd.org?

Check out the "Running Git" chapter of the appendices to the FreeBSD
Handbook.

> 2) When the code is part of a larger work, such as one particular X11
> driver, will I ultimately have to build all of X11 just to build the
> one driver, or can I build just the driver using the
> already-installed X11 include files and libraries?

So you want to build a particular port locally, and you have other ports
already installed by some other means, such as pkg. You absolutely do
not need to rebuild everything you already have installed; there are
some caveats (one such being that you will likely be building from a
later version of the ports tree than were the packages you already have
installed), but it will in general just work.

> 3) Do the source files obtained by git normally go in /usr/ports/, or
> does location not matter?

If you are checking out the ports tree, they will normally go in
/usr/ports, but you can put them anywhere you want. If you put it
elsewhere you may need to set the PORTSDIR variable (defined in
${PORTSDIR}/Mk/bsd.port.mk) when working with that tree.

If you already have something in /usr/ports, such as from portsnap, then
consider using the same method to do your upgrades. If not, you might as
well go ahead and put your new ports tree there.

My guess is that you want to compile a driver with options different
than the ones that the standard pkg repositories use. If that is the
case, you will be able to set those options when you build the port; but
bear in mind that your system may still need to build a lot of other
things first -- things that are not needed to run the driver, but are
needed to build it.

Good luck.
--
Lowell Gilbert, embedded/networking software engineer
http://be-well.ilk.org/~lowell/

I previously posted:
>> (I've never had occasion to use git.)

Lowell Gilbert <lgusenet@be-well.ilk.org> kindly replied:
> That's fine. You don't seem to be a developer, so I don't see why you
> would have.

I've used other source code managers, just not git.
Even created my own back in the mid-80s when I was tech. lead on a
project and each of us needed to work on our own, possibly slightly
overlapping, pieces.

> Check out the "Running Git" chapter of the appendices to the FreeBSD
> Handbook.

Ah, that's where it is. OK, thanks!

> So you want to build a particular port locally, and you have other
> ports already installed by some other means, such as pkg.

One's a port, one's a core program. For the port, yes, exactly.

> You absolutely do not need to rebuild everything you already have
> installed;

Yay! IIRC (from several years ago), portsnap seemed to want to pull in
every other dependency and rebuild it, too, which I sometimes was
willing to let it do, but I prefer not to mix pkg and ports things on a
system.

>> 3) Do the source files obtained by git normally go in /usr/ports/, or
>> does location not matter?

> If you are checking out the ports tree, they will normally go in
> /usr/ports, but you can put them anywhere you want.

OK, good. The ports thing I have in mind is one particular X11 driver,
which, I would think, is a relatively stand-alone piece.

> If you put it elsewhere you may need to set the PORTSDIR variable
> (defined in ${PORTSDIR}/Mk/bsd.port.mk) when working with that tree.

OK, easy enough.

> If you already have something in /usr/ports, such as from portsnap,

Exactly, but with 14.0, there's only git, and I didn't know how those
mix, if at all, since "make"s in my old portsnap tree expect to be able
to pull in the source files. I've been tempted to rm -r /usr/ports/*,
but haven't yet. It's portsnap content is from years ago.

> then consider using the same method to do your upgrades. If not, you
> might as well go ahead and put your new ports tree there.

So "rm -r /usr/ports/*" followed by obtaining just the X11 driver in
question should work -- it doesn't need the rest. Good.

> My guess is that you want to compile a driver with options different
> than the ones that the standard pkg repositories use.

A good guess, but no. :-) In comments to a bug report, the maintainer
offered a patch, and I think maybe he's waiting for me to test it, even
though the fix applies to all the related drivers, he's made it in some
of them, and we know the fix works in those.

If, like with portsnap, building a patched version required dragging in
and rebuilding all the rest of X11, (which, in theory, might include
some new change elsewhere that affects the driver, making installing
just the driver an inappropriate test), where I have to uninstall the
pkg version of all of X, etc., that's more than I feel like fighting.
But it sounds like I can build generic+patch of this one piece against
the pkg-installed header files and libraries, so that's fine.

Time to go read the "Running Git" section of the FreeBSD Handbook...

Thanks!
-WBE

On Tue, 26 Dec 2023 17:21:41 -0500, Lowell Gilbert wrote:

> but bear in
> mind that your system may still need to build a lot of other things first
> -- things that are not needed to run the driver, but are needed to build
> it.

If the above is true, I'd recommend using portsnap to make sure the ports
tree is up to date first, then use portinstall (alias of portupgrade) with
the -P (--use-packages) option so any of the build dependencies which
have binary packages as up to date as ports, things will go much more
quickly. Ideally, the port you are building will not be one of those
binary packages and will do a "make config" at the start where you can
make the required changes. Worst case, the required port can then be
rebuilt in whatever way is chosen, possibly even using
make config fetch extract and then patching or whatever before doing
a make deinstall reinstall

HTH :-)

--
ad astra tabernamque

Don't feed the trolls. You might catch something nasty.

On 2023-12-27, DaveG <nospam@nospam.net> wrote:

> If the above is true, I'd recommend using portsnap to make sure the ports
> tree is up to date first, then use portinstall (alias of portupgrade) with

From the 14.0 release notes:
The portsnap(8) utility has been removed. Users are encouraged
to fetch the ports tree by using pkg install git and then git
clone https://git.FreeBSD.org/ports.git /usr/ports.

--
Christian "naddy" Weisgerber naddy@mips.inka.de

In article <slrnuop661.12lk.naddy@lorvorc.mips.inka.de>
Christian Weisgerber <naddy@mips.inka.de> writes:
>On 2023-12-27, DaveG <nospam@nospam.net> wrote:
>
>> If the above is true, I'd recommend using portsnap to make sure the ports
>> tree is up to date first, then use portinstall (alias of portupgrade) with
>
>From the 14.0 release notes:
> The portsnap(8) utility has been removed. Users are encouraged
> to fetch the ports tree by using pkg install git and then git
> clone https://git.FreeBSD.org/ports.git /usr/ports.

Thanks for that. I'd heard there was a change, but wash't sure of
the practical details.

So is 'portsnap fetch update' now just 'cd /usr/ports ; git pull'?

(Still waiting for 14.1 to make the switch.)

--
Drew Lawson | Radioactive cats have
| 18 half-lives
|

On Wed, 27 Dec 2023 23:03:35 +0000, Drew Lawson wrote:

> In article <slrnuop661.12lk.naddy@lorvorc.mips.inka.de>
> Christian Weisgerber <naddy@mips.inka.de> writes:
>>On 2023-12-27, DaveG <nospam@nospam.net> wrote:
>>
>>> If the above is true, I'd recommend using portsnap to make sure the
>>> ports tree is up to date first, then use portinstall (alias of
>>> portupgrade) with
>>
>>From the 14.0 release notes:
>> The portsnap(8) utility has been removed. Users are encouraged to
>> fetch the ports tree by using pkg install git and then git clone
>> https://git.FreeBSD.org/ports.git /usr/ports.
>
> Thanks for that. I'd heard there was a change, but wash't sure of the
> practical details.
>
> So is 'portsnap fetch update' now just 'cd /usr/ports ; git pull'?

Not quite:
https://docs.freebsd.org/en/books/handbook/book/#ports-using-installation-
methods

There are a few more steps, no mountains to climb tho.

> (Still waiting for 14.1 to make the switch.)

--
To health and anarchy

On Dec 27, 2023 at 5:03:35 PM CST, "Drew Lawson" <Drew Lawson> wrote:

> In article <slrnuop661.12lk.naddy@lorvorc.mips.inka.de>
> Christian Weisgerber <naddy@mips.inka.de> writes:
>> On 2023-12-27, DaveG <nospam@nospam.net> wrote:
>>
>>> If the above is true, I'd recommend using portsnap to make sure the ports
>>> tree is up to date first, then use portinstall (alias of portupgrade) with
>>
>> From the 14.0 release notes:
>> The portsnap(8) utility has been removed. Users are encouraged
>> to fetch the ports tree by using pkg install git and then git
>> clone https://git.FreeBSD.org/ports.git /usr/ports.
>
> Thanks for that. I'd heard there was a change, but wash't sure of
> the practical details.
>
> So is 'portsnap fetch update' now just 'cd /usr/ports ; git pull'?
>
> (Still waiting for 14.1 to make the switch.)

Everything I've read claims the procedure is 'git pull' inside the /usr/src/
directory once you've checked it out, but even though I don't make any
changes, I always get this error:

error: Your local changes to the following files would be overwritten by merge
UPDATING
contrib/tzdata/CONTRIBUTING
contrib/tzdata/Makefile
...

Not sure if packages or OS updates touch /usr/src, or if things get touched
enough during certain builds, but every time I have tried to do a 'git pull' I
get that error. Not sure it is the 'right' way, but I've been doing 'git pull
--autostash' to deal with that.

On Dec 27, 2023 at 7:07:39 PM CST, "Jesse Rehmer"
<jesse.rehmer@blueworldhosting.com> wrote:

> On Dec 27, 2023 at 5:03:35 PM CST, "Drew Lawson" <Drew Lawson> wrote:
>
>> In article <slrnuop661.12lk.naddy@lorvorc.mips.inka.de>
>> Christian Weisgerber <naddy@mips.inka.de> writes:
>>> On 2023-12-27, DaveG <nospam@nospam.net> wrote:
>>>
>>>> If the above is true, I'd recommend using portsnap to make sure the ports
>>>> tree is up to date first, then use portinstall (alias of portupgrade) with
>>>
>>> From the 14.0 release notes:
>>> The portsnap(8) utility has been removed. Users are encouraged
>>> to fetch the ports tree by using pkg install git and then git
>>> clone https://git.FreeBSD.org/ports.git /usr/ports.
>>
>> Thanks for that. I'd heard there was a change, but wash't sure of
>> the practical details.
>>
>> So is 'portsnap fetch update' now just 'cd /usr/ports ; git pull'?
>>
>> (Still waiting for 14.1 to make the switch.)
>
> Everything I've read claims the procedure is 'git pull' inside the /usr/src/
> directory once you've checked it out, but even though I don't make any
> changes, I always get this error:
>
> error: Your local changes to the following files would be overwritten by merge
> UPDATING
> contrib/tzdata/CONTRIBUTING
> contrib/tzdata/Makefile
> ...
>
> Not sure if packages or OS updates touch /usr/src, or if things get touched
> enough during certain builds, but every time I have tried to do a 'git pull' I
> get that error. Not sure it is the 'right' way, but I've been doing 'git pull
> --autostash' to deal with that.

Derp, I mention /usr/src and pasted from output in that directory, but this is
about ports, though I have the same kind of issues with 'git pull' there too
since upgrading to 14.0.

On Wed, 27 Dec 2023 21:36:01 -0000 (UTC), Christian Weisgerber wrote:
> On 2023-12-27, DaveG <nospam@nospam.net> wrote:
>
>> If the above is true, I'd recommend using portsnap to make sure the ports
>> tree is up to date first, then use portinstall (alias of portupgrade) with
>
> From the 14.0 release notes:
> The portsnap(8) utility has been removed. Users are encouraged
> to fetch the ports tree by using pkg install git and then git
> clone https://git.FreeBSD.org/ports.git /usr/ports.
>

Instead I use:
/usr/ports/net/gitup

gitup ports updates /usr/ports
gitup release updates /usr/src

More in the gitup.conf

It's very fast.

Detlef
--
https://www.12schrittefrei.de/
https://www.noart.de/

Den 2023-12-28 skrev Jesse Rehmer <jesse.rehmer@blueworldhosting.com>:
> On Dec 27, 2023 at 5:03:35 PM CST, "Drew Lawson" <Drew Lawson> wrote:
>
>> So is 'portsnap fetch update' now just 'cd /usr/ports ; git pull'?
>>
>> (Still waiting for 14.1 to make the switch.)
>
> Everything I've read claims the procedure is 'git pull' inside the /usr/src/
> directory once you've checked it out, but even though I don't make any
> changes, I always get this error:
>
> error: Your local changes to the following files would be overwritten by merge
> UPDATING
> contrib/tzdata/CONTRIBUTING
> contrib/tzdata/Makefile
> ...
>
> Not sure if packages or OS updates touch /usr/src, or if things get touched
> enough during certain builds, but every time I have tried to do a 'git pull' I
> get that error. Not sure it is the 'right' way, but I've been doing 'git pull
> --autostash' to deal with that.

Something has caused non-commited changes in your repository. If you
run freebsd-update without excluding /usr/ports and /usr/src, that can
happen.

There are two ways I usually deal with that type of conflict. First is
to do

git reset --hard
git pull

to discard local uncommited changes and update. If I have local
commits I also want to get rid of, I do

git fetch
git reset --hard origin/$insert_branch_name

and any local commit I have added on my branch is also discarded. If I
have local commited changes that I want to keep, I usually do

git pull --rebase

with --rebase lifting my commits and trying to apply them on top of
the latest upstream commit. This can result in conflicts that need to
be worked out.

Den 2023-12-28 skrev Andreas Kempe <kempe@lysator.liu.se>:
> Den 2023-12-28 skrev Jesse Rehmer <jesse.rehmer@blueworldhosting.com>:
>> On Dec 27, 2023 at 5:03:35 PM CST, "Drew Lawson" <Drew Lawson> wrote:
>>
>>> So is 'portsnap fetch update' now just 'cd /usr/ports ; git pull'?
>>>
>>> (Still waiting for 14.1 to make the switch.)
>>
>> Everything I've read claims the procedure is 'git pull' inside the /usr/src/
>> directory once you've checked it out, but even though I don't make any
>> changes, I always get this error:
>>
>> error: Your local changes to the following files would be overwritten by merge
>> UPDATING
>> contrib/tzdata/CONTRIBUTING
>> contrib/tzdata/Makefile
>> ...
>>
>> Not sure if packages or OS updates touch /usr/src, or if things get touched
>> enough during certain builds, but every time I have tried to do a 'git pull' I
>> get that error. Not sure it is the 'right' way, but I've been doing 'git pull
>> --autostash' to deal with that.

I forgot to address the autostash. It works, but it will save the
conflicting changes every time in the Git repository, causing it to
grow in size over time. Using git reset --hard discards the changes.

>
> Something has caused non-commited changes in your repository. If you
> run freebsd-update without excluding /usr/ports and /usr/src, that can
> happen.
>
> There are two ways I usually deal with that type of conflict. First is
> to do
>
> git reset --hard
> git pull
>
> to discard local uncommited changes and update. If I have local
> commits I also want to get rid of, I do
>
> git fetch
> git reset --hard origin/$insert_branch_name
>
> and any local commit I have added on my branch is also discarded. If I
> have local commited changes that I want to keep, I usually do
>
> git pull --rebase
>
> with --rebase lifting my commits and trying to apply them on top of
> the latest upstream commit. This can result in conflicts that need to
> be worked out.

Re: observed result regarding git and /usr/ports

I asked:
>>> 3) Do the source files obtained by git normally go in /usr/ports/,
>>> or does location not matter?

to which Lowell Gilbert <lgusenet@be-well.ilk.org> kindly replied (in part):
>> If you already have something in /usr/ports, such as from portsnap,

> Exactly, [... What I currently have in /usr/ports is] portsnap content
> from years ago.

"git clone ..." complains when /usr/ports isn't empty, so any portsnap
content already there is apparently incompatible with "git clone"'s
results being put in /usr/ports. OK.

Re: using git to obtain and build specific pieces

>> You absolutely do not need to rebuild everything you already have
>> installed;

to which I replied:
> Yay! IIRC (from several years ago), portsnap seemed to want to pull in
> every other dependency and rebuild it, too, which I sometimes was
> willing to let it do, but I prefer not to mix pkg and ports things on a
> system.

Here's where I'd like advice from those of you who know more about git.

* Do I have to have a local clone of the remote repository to be able to
obtain the individual parts?

I'd prefer not have to clone gigabytes worth of the /usr/ports and
/usr/src repositories just to obtain and build one program or piece.

Doing:
git clone --sparse --depth 1 --single-branch \
https://git.freebsd.org/ports.git /usr/ports

is fine and fairly small. I did that (mistakenly) thinking that
git-sparse-checkout could then be used to expand the tree down to the
piece I wanted, but after actually reading the man page for
git-sparse-checkout, it appears that's not what it does.

ISTM that there ought to be a way to get a copy of a subset directly
from the remote repository. The man page for the most likely command,
"git checkout", however, talks about saving and restoring a work area
to/from multiple branches (which maybe doesn't apply if
--single-branch?) etc., which would seem (on casual reading) to require
a local repository.

* Is there some not-too-painful way to do what I'm trying to do: obtain
enough of just the parts I'm interested in to be able to make local
changes and build the result against the existing, installed header
files and libraries? I don't need to be able to upload the changes or
make commits to the remote repositories (which I'm not authorized to
do anyway), so part of what git does regarding branches and merging
isn't relevant. If git just reports which version it obtained (for
reference), I should be fine.

As an alternative to git, I tried visiting the FreeBSD repository files
with a browser. Clicking down to the pieces I want is easy, but that
doesn't give me C code for file foo.c: it comes with HTML wrapping that
formats and numbers every line of the code. Removing all that wrapping
to get something compilable would be a pain.

TVMIA,
-WBE

Winston <wbe@UBEBLOCK.psr.com.invalid> writes:

> Re: observed result regarding git and /usr/ports
>
> I asked:
>>>> 3) Do the source files obtained by git normally go in /usr/ports/,
>>>> or does location not matter?
>
> to which Lowell Gilbert <lgusenet@be-well.ilk.org> kindly replied (in part):
>>> If you already have something in /usr/ports, such as from portsnap,
>
>> Exactly, [... What I currently have in /usr/ports is] portsnap content
>> from years ago.

That's completely useless to you at this point. Might as well wipe it out.

> "git clone ..." complains when /usr/ports isn't empty, so any portsnap
> content already there is apparently incompatible with "git clone"'s
> results being put in /usr/ports. OK.

I'd recommend clearing that out and putting it there, which will
slightly simplify the procedure.

> Re: using git to obtain and build specific pieces
>
>>> You absolutely do not need to rebuild everything you already have
>>> installed;
>
> to which I replied:
>> Yay! IIRC (from several years ago), portsnap seemed to want to pull in
>> every other dependency and rebuild it, too, which I sometimes was
>> willing to let it do, but I prefer not to mix pkg and ports things on a
>> system.

Ports from git won't normally do that, but neither would portsnap. They
are different ways to download the tree, but how the tree works once you
have it will be the same as far as I can recall.

> Here's where I'd like advice from those of you who know more about git.
>
> * Do I have to have a local clone of the remote repository to be able to
> obtain the individual parts?

No, but it will be a lot less work. For one thing I can think of
offhand, you will have to go back to fill in missing dependencies one at
a time.

> I'd prefer not have to clone gigabytes worth of the /usr/ports and
> /usr/src repositories just to obtain and build one program or piece.
>
> Doing:
> git clone --sparse --depth 1 --single-branch \
> https://git.freebsd.org/ports.git /usr/ports
>
> is fine and fairly small. I did that (mistakenly) thinking that
> git-sparse-checkout could then be used to expand the tree down to the
> piece I wanted, but after actually reading the man page for
> git-sparse-checkout, it appears that's not what it does.

Hmm. It looks to me like it can do that, but it isn't
straightforward. You might try
git sparse-checkout set <desired subtree>

Perhaps more importantly to you, a sparse checkout downloads almost
exactly the same amount of data that it would without using --sparse;
the difference is what it puts into the working tree. For the ports
tree, that is slightly under one gigabyte, and it sounds to me as though
you have already done that.

> ISTM that there ought to be a way to get a copy of a subset directly
> from the remote repository. The man page for the most likely command,
> "git checkout", however, talks about saving and restoring a work area
> to/from multiple branches (which maybe doesn't apply if
> --single-branch?) etc., which would seem (on casual reading) to require
> a local repository.

git is a distributed RCS. You are probably accustomed to client/server
version control, where most operations go back to the remote server.

> * Is there some not-too-painful way to do what I'm trying to do: obtain
> enough of just the parts I'm interested in to be able to make local
> changes and build the result against the existing, installed header
> files and libraries? I don't need to be able to upload the changes or
> make commits to the remote repositories (which I'm not authorized to
> do anyway), so part of what git does regarding branches and merging
> isn't relevant. If git just reports which version it obtained (for
> reference), I should be fine.

Depends on your personal tolerance for different kinds of pain, I
suppose. Downloading the database of the whole default branch of the
ports tree with git takes about a minute at 100 megabits per second, and
checking the objects out once they're downloaded will take much longer
than that. In typical usage, It's all one operation from the command
line, of course.

For version numbers, remember (again) that the model is
distributed. There is no monotonically increasing version value you can
apply, because there's no single master location to assign it.

> As an alternative to git, I tried visiting the FreeBSD repository files
> with a browser. Clicking down to the pieces I want is easy, but that
> doesn't give me C code for file foo.c: it comes with HTML wrapping that
> formats and numbers every line of the code. Removing all that wrapping
> to get something compilable would be a pain.

AS best I recall, there is more than one web interface to the ports
collection, but most likely you're looking at cgit, which has a link on
each leaf page labelled "(plain)"; see it next to the blob hash at, for
example:
https://cgit.freebsd.org/ports/tree/x11-drivers/xf86-video-voodoo/Makefile

Good luck.
--
Lowell Gilbert, embedded/networking software engineer
http://be-well.ilk.org/~lowell/

(Partial reply)

Lowell Gilbert <lgusenet@be-well.ilk.org> kindly replied:
> ... most likely you're looking at cgit, which has a link on each leaf
> page labelled "(plain)"; see it next to the blob hash at, for example:
> https://cgit.freebsd.org/ports/tree/x11-drivers/xf86-video-voodoo/Makefile

Well, lookie there! :) That's also true for src tree code.
That should take care of the src mods I want to try.

Thanks!
-WBE

[I'll reply to the rest later.]

On 2023-12-28, Winston <wbe@UBEBLOCK.psr.com.invalid> wrote:

> Here's where I'd like advice from those of you who know more about git.
>
> * Do I have to have a local clone of the remote repository to be able to
> obtain the individual parts?

Yes.
The topic already came up for the switch from CVS to Subversion,
because the latter keeps an extra copy of the checkout on disk, and
then again for Subversion to Git, and each time the developer
consensus was "disk space is cheap".

> I'd prefer not have to clone gigabytes worth of the /usr/ports and
> /usr/src repositories just to obtain and build one program or piece.

Well, it's... 1.4G for ports, 1.7G for src. You can mess around
with...

> Doing:
> git clone --sparse --depth 1 --single-branch \
> https://git.freebsd.org/ports.git /usr/ports

.... well, I see you already did.

> ISTM that there ought to be a way to get a copy of a subset directly
> from the remote repository.

There is not. Git is a distributed version control system. A local
clone of the repository is an inherent part of that. Git is not
designed as a source distribution system.

> * Is there some not-too-painful way to do what I'm trying to do: obtain
> enough of just the parts I'm interested in to be able to make local
> changes and build the result against the existing, installed header
> files and libraries? I don't need to be able to upload the changes or
> make commits to the remote repositories (which I'm not authorized to
> do anyway), so part of what git does regarding branches and merging
> isn't relevant.

Actually, if you have local changes, then it makes perfect sense
to commit them to a local branch. Now that is what Git is designed
for. You have local code under version control that you synchronize
from time to time with a remote repository.

The not-too-painful way is to go along with how Git works, not to
fight it at every step. *shrug*

BTW, I highly recommend the Git Book. Chapters 1-3 provide an
excellent introduction to Git and its concepts.
https://git-scm.com/book/en/v2

> As an alternative to git, I tried visiting the FreeBSD repository files
> with a browser.

As somebody else already mentioned, net/gitup is probably what you
are looking for.

--
Christian "naddy" Weisgerber naddy@mips.inka.de