(all names and addresses have been altered, but the form is the same.)

I am trying to mount a directory from Serv to Client

On Cient I have in /etc/fstab
Serv:/local/ /local1 nfs4 rw,rsize=8192,wsize=8192 0 0
and /local1 exists as an empty directory.

On Serv, I have
/local 142.103.222.0/26(fsid=21,no_subtree_check,rw,no_root_squash)
in /etc/exports
where the address of Client is 142.103.222.11
and I ran exportfs -s
The directory /local exists on Serv.
ls -dl /local
drwxrwxr-x 22 root root 4096 Sep 18 2022 /local/

But when I type
mount /local1
on Client as root I get
mount.nfs4: mounting Serv:/local/ failed, reason given by server: No such file or directory

I cannot find anything in any of the /var/log files which gives me a clue
as to what could be going on here.

If on Client I change the entry to
Serv:/local/ /local1 nfs4 rw,rsize=8192,wsize=8192,vers=3 0 0
I then get an entry in journalctl of
Serv rpc.mountd[775]: authenticated mount request from 142.103.222.11:958 for /local/unruhhome (/local/unruhhome)
But now the error message on Clinet is
mount.nfs: access denied by server while mounting Serv:/local/

Ie, nfs seems to be all F up, but I cannot figure out why.

Note that this all happened after I rebooted Serv. nfs has been working
fine until recently. and all of the machines had their nfs mounts
working. The problem started when I rebooted Client and nothing was
mounted.
Note that all of the directories mounted from Serv onto Client are
affected. The case below is just an example.

All are on updated Mga8 systems.

On 2023-08-12, William Unruh <unruh@invalid.ca> wrote:
> (all names and addresses have been altered, but the form is the same.)
>
> I am trying to mount a directory from Serv to Client
>
> On Cient I have in /etc/fstab
> Serv:/local/ /local1 nfs4 rw,rsize=8192,wsize=8192 0 0
> and /local1 exists as an empty directory.
>
> On Serv, I have
> /local 142.103.222.0/26(fsid=21,no_subtree_check,rw,no_root_squash)
> in /etc/exports
> where the address of Client is 142.103.222.11
> and I ran exportfs -s
> The directory /local exists on Serv.
> ls -dl /local
> drwxrwxr-x 22 root root 4096 Sep 18 2022 /local/
>
> But when I type
> mount /local1
> on Client as root I get
> mount.nfs4: mounting Serv:/local/ failed, reason given by server: No such file or directory
>
> I cannot find anything in any of the /var/log files which gives me a clue
> as to what could be going on here.
>
> If on Client I change the entry to
> Serv:/local/ /local1 nfs4 rw,rsize=8192,wsize=8192,vers=3 0 0
> I then get an entry in journalctl of
> Serv rpc.mountd[775]: authenticated mount request from 142.103.222.11:958 for /local/unruhhome (/local/unruhhome)
> But now the error message on Clinet is
> mount.nfs: access denied by server while mounting Serv:/local/
>
> Ie, nfs seems to be all F up, but I cannot figure out why.

And of course just after I sent that out, the systems suddenly started
working again. Not at all clear what happened.

On 2023-08-12, William Unruh <unruh@invalid.ca> wrote:
>
> Note that this all happened after I rebooted Serv. nfs has been working
> fine until recently. and all of the machines had their nfs mounts
> working. The problem started when I rebooted Client and nothing was
> mounted.
> Note that all of the directories mounted from Serv onto Client are
> affected. The case below is just an example.
>
> All are on updated Mga8 systems.
>
>
> On 2023-08-12, William Unruh <unruh@invalid.ca> wrote:
>> (all names and addresses have been altered, but the form is the same.)
>>
>> I am trying to mount a directory from Serv to Client
>>
>> On Cient I have in /etc/fstab
>> Serv:/local/ /local1 nfs4 rw,rsize=8192,wsize=8192 0 0
>> and /local1 exists as an empty directory.
>>
>> On Serv, I have
>> /local 142.103.222.0/26(fsid=21,no_subtree_check,rw,no_root_squash)
>> in /etc/exports
>> where the address of Client is 142.103.222.11
>> and I ran exportfs -s
>> The directory /local exists on Serv.
>> ls -dl /local
>> drwxrwxr-x 22 root root 4096 Sep 18 2022 /local/
>>
>> But when I type
>> mount /local1
>> on Client as root I get
>> mount.nfs4: mounting Serv:/local/ failed, reason given by server: No such file or directory
>>
>> I cannot find anything in any of the /var/log files which gives me a clue
>> as to what could be going on here.
>>
>> If on Client I change the entry to
>> Serv:/local/ /local1 nfs4 rw,rsize=8192,wsize=8192,vers=3 0 0
>> I then get an entry in journalctl of
>> Serv rpc.mountd[775]: authenticated mount request from 142.103.222.11:958 for /local/unruhhome (/local/unruhhome)
>> But now the error message on Clinet is
>> mount.nfs: access denied by server while mounting Serv:/local/
>>
>> Ie, nfs seems to be all F up, but I cannot figure out why.

On Sat, 12 Aug 2023 18:46:02 -0400, William Unruh <unruh@invalid.ca> wrote:

> And of course just after I sent that out, the systems suddenly started
> working again. Not at all clear what happened.

Does "ip -s link" show any errors?

Regards, Dave Hodgins

On 2023-08-12, David W. Hodgins <dwhodgins@nomail.afraid.org> wrote:
> On Sat, 12 Aug 2023 18:46:02 -0400, William Unruh <unruh@invalid.ca> wrote:
>
>> And of course just after I sent that out, the systems suddenly started
>> working again. Not at all clear what happened.
>
> Does "ip -s link" show any errors?
>
> Regards, Dave Hodgins

On Serv:

2: enp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
link/ether f8:32:e4:70:14:5b brd ff:ff:ff:ff:ff:ff
RX: bytes packets errors dropped missed mcast
69357175201 558088800 0 37942 0 15654970
TX: bytes packets errors dropped carrier collsns
1395857318948 985313209 0 0 0 0

On Client:
2: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
link/ether 20:cf:30:90:5c:cb brd ff:ff:ff:ff:ff:ff
RX: bytes packets errors dropped missed mcast
715899055 1004543 0 1095 88 83900
TX: bytes packets errors dropped carrier collsns
379358269 614341 0 0 0 0

I have no idea what time span those results cover, and the mount
problems occured abot 5 days ago.

On Tue, 15 Aug 2023 13:46:06 -0400, William Unruh <unruh@invalid.ca> wrote:

> On 2023-08-12, David W. Hodgins <dwhodgins@nomail.afraid.org> wrote:
>> On Sat, 12 Aug 2023 18:46:02 -0400, William Unruh <unruh@invalid.ca> wrote:
>>
>>> And of course just after I sent that out, the systems suddenly started
>>> working again. Not at all clear what happened.
>>
>> Does "ip -s link" show any errors?
>>
>> Regards, Dave Hodgins
>
> On Serv:
>
> 2: enp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
> link/ether f8:32:e4:70:14:5b brd ff:ff:ff:ff:ff:ff
> RX: bytes packets errors dropped missed mcast
> 69357175201 558088800 0 37942 0 15654970
> TX: bytes packets errors dropped carrier collsns
> 1395857318948 985313209 0 0 0 0
>
> On Client:
> 2: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
> link/ether 20:cf:30:90:5c:cb brd ff:ff:ff:ff:ff:ff
> RX: bytes packets errors dropped missed mcast
> 715899055 1004543 0 1095 88 83900
> TX: bytes packets errors dropped carrier collsns
> 379358269 614341 0 0 0 0
>
> I have no idea what time span those results cover, and the mount
> problems occured abot 5 days ago.

The dropped packet count is very high for that number of packets.

On my system after 2 days since reboot ...
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
link/ether 10:fe:ed:03:8d:b3 brd ff:ff:ff:ff:ff:ff
RX: bytes packets errors dropped missed mcast
12995214891 24029134 0 0 0 7143
TX: bytes packets errors dropped carrier collsns
12474040445 27377647 0 0 0 0

All three of my computers, two of which use ethernet and the other wireless have
zero dropped packets.

The most likely causes of packet cause is one or more bad ethernet cables, bad
sockets, or just not quite being fully plugged into the socket.

Regards, Dave Hodgins

On 2023-08-15, David W. Hodgins <dwhodgins@nomail.afraid.org> wrote:
> On Tue, 15 Aug 2023 13:46:06 -0400, William Unruh <unruh@invalid.ca> wrote:
>
>> On 2023-08-12, David W. Hodgins <dwhodgins@nomail.afraid.org> wrote:
>>> On Sat, 12 Aug 2023 18:46:02 -0400, William Unruh <unruh@invalid.ca> wrote:
>>>
>>>> And of course just after I sent that out, the systems suddenly started
>>>> working again. Not at all clear what happened.
>>>
>>> Does "ip -s link" show any errors?
>>>
>>> Regards, Dave Hodgins
>>
>> On Serv:
>>
>> 2: enp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
>> link/ether f8:32:e4:70:14:5b brd ff:ff:ff:ff:ff:ff
>> RX: bytes packets errors dropped missed mcast
>> 69357175201 558088800 0 37942 0 15654970
>> TX: bytes packets errors dropped carrier collsns
>> 1395857318948 985313209 0 0 0 0
>>
>> On Client:
>> 2: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
>> link/ether 20:cf:30:90:5c:cb brd ff:ff:ff:ff:ff:ff
>> RX: bytes packets errors dropped missed mcast
>> 715899055 1004543 0 1095 88 83900
>> TX: bytes packets errors dropped carrier collsns
>> 379358269 614341 0 0 0 0
>>
>> I have no idea what time span those results cover, and the mount
>> problems occured abot 5 days ago.
>
> The dropped packet count is very high for that number of packets.
>
> On my system after 2 days since reboot ...
> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
> link/ether 10:fe:ed:03:8d:b3 brd ff:ff:ff:ff:ff:ff
> RX: bytes packets errors dropped missed mcast
> 12995214891 24029134 0 0 0 7143
> TX: bytes packets errors dropped carrier collsns
> 12474040445 27377647 0 0 0 0
>
> All three of my computers, two of which use ethernet and the other wireless have
> zero dropped packets.
>
> The most likely causes of packet cause is one or more bad ethernet cables, bad
> sockets, or just not quite being fully plugged into the socket.
>
Thanks for the hint. It wou.d be strange if such a cause could reliably
result in "directory name does not exist" as the message from the server
to the client, while ssh connections between the two seems to be fine.

> Regards, Dave Hodgins

On Wed, 16 Aug 2023 13:31:35 -0400, William Unruh <unruh@invalid.ca> wrote:

> On 2023-08-15, David W. Hodgins <dwhodgins@nomail.afraid.org> wrote:
>> On Tue, 15 Aug 2023 13:46:06 -0400, William Unruh <unruh@invalid.ca> wrote:
>>
>>> On 2023-08-12, David W. Hodgins <dwhodgins@nomail.afraid.org> wrote:
>>>> On Sat, 12 Aug 2023 18:46:02 -0400, William Unruh <unruh@invalid.ca> wrote:
>>>>
>>>>> And of course just after I sent that out, the systems suddenly started
>>>>> working again. Not at all clear what happened.
>>>>
>>>> Does "ip -s link" show any errors?
>>>>
>>>> Regards, Dave Hodgins
>>>
>>> On Serv:
>>>
>>> 2: enp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
>>> link/ether f8:32:e4:70:14:5b brd ff:ff:ff:ff:ff:ff
>>> RX: bytes packets errors dropped missed mcast
>>> 69357175201 558088800 0 37942 0 15654970
>>> TX: bytes packets errors dropped carrier collsns
>>> 1395857318948 985313209 0 0 0 0
>>>
>>> On Client:
>>> 2: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
>>> link/ether 20:cf:30:90:5c:cb brd ff:ff:ff:ff:ff:ff
>>> RX: bytes packets errors dropped missed mcast
>>> 715899055 1004543 0 1095 88 83900
>>> TX: bytes packets errors dropped carrier collsns
>>> 379358269 614341 0 0 0 0
>>>
>>> I have no idea what time span those results cover, and the mount
>>> problems occured abot 5 days ago.
>>
>> The dropped packet count is very high for that number of packets.
>>
>> On my system after 2 days since reboot ...
>> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
>> link/ether 10:fe:ed:03:8d:b3 brd ff:ff:ff:ff:ff:ff
>> RX: bytes packets errors dropped missed mcast
>> 12995214891 24029134 0 0 0 7143
>> TX: bytes packets errors dropped carrier collsns
>> 12474040445 27377647 0 0 0 0
>>
>> All three of my computers, two of which use ethernet and the other wireless have
>> zero dropped packets.
>>
>> The most likely causes of packet cause is one or more bad ethernet cables, bad
>> sockets, or just not quite being fully plugged into the socket.
>>
> Thanks for the hint. It wou.d be strange if such a cause could reliably
> result in "directory name does not exist" as the message from the server
> to the client, while ssh connections between the two seems to be fine.

I'm not sure if that is the cause or not. I haven't looked into how nfs/nfsd
and ssh/sshd handle some failed packets. It wouldn't surprise me if they work
differently, causing the problem only for nfs. The server may not be generating
that message. It may be coming from the client whenever the mount fails.

I'd try fixing that issue and then see if the problem still occurs.

Regards, Dave Hodgins

Thanks will try that if it occurs again. The reason I suspect the server
is that it occured to at least two clients. On the one client I guess it
had been rebooted and had mounted nothing from the server. On the second
client, it had not been rebooted, and nfs worked fine, until I unmounted
one of the partition, and then on remount, it had the same problem.

I finally and accidentally did something to the server, and everything
started working again ( the clients all remouted without problems). of
course I did not notice what I did.

On 2023-08-16, David W. Hodgins <dwhodgins@nomail.afraid.org> wrote:
> On Wed, 16 Aug 2023 13:31:35 -0400, William Unruh <unruh@invalid.ca> wrote:
>
>> On 2023-08-15, David W. Hodgins <dwhodgins@nomail.afraid.org> wrote:
>>> On Tue, 15 Aug 2023 13:46:06 -0400, William Unruh <unruh@invalid.ca> wrote:
>>>
>>>> On 2023-08-12, David W. Hodgins <dwhodgins@nomail.afraid.org> wrote:
>>>>> On Sat, 12 Aug 2023 18:46:02 -0400, William Unruh <unruh@invalid.ca> wrote:
>>>>>
>>>>>> And of course just after I sent that out, the systems suddenly started
>>>>>> working again. Not at all clear what happened.
>>>>>
>>>>> Does "ip -s link" show any errors?
>>>>>
>>>>> Regards, Dave Hodgins
>>>>
>>>> On Serv:
>>>>
>>>> 2: enp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
>>>> link/ether f8:32:e4:70:14:5b brd ff:ff:ff:ff:ff:ff
>>>> RX: bytes packets errors dropped missed mcast
>>>> 69357175201 558088800 0 37942 0 15654970
>>>> TX: bytes packets errors dropped carrier collsns
>>>> 1395857318948 985313209 0 0 0 0
>>>>
>>>> On Client:
>>>> 2: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
>>>> link/ether 20:cf:30:90:5c:cb brd ff:ff:ff:ff:ff:ff
>>>> RX: bytes packets errors dropped missed mcast
>>>> 715899055 1004543 0 1095 88 83900
>>>> TX: bytes packets errors dropped carrier collsns
>>>> 379358269 614341 0 0 0 0
>>>>
>>>> I have no idea what time span those results cover, and the mount
>>>> problems occured abot 5 days ago.
>>>
>>> The dropped packet count is very high for that number of packets.
>>>
>>> On my system after 2 days since reboot ...
>>> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
>>> link/ether 10:fe:ed:03:8d:b3 brd ff:ff:ff:ff:ff:ff
>>> RX: bytes packets errors dropped missed mcast
>>> 12995214891 24029134 0 0 0 7143
>>> TX: bytes packets errors dropped carrier collsns
>>> 12474040445 27377647 0 0 0 0
>>>
>>> All three of my computers, two of which use ethernet and the other wireless have
>>> zero dropped packets.
>>>
>>> The most likely causes of packet cause is one or more bad ethernet cables, bad
>>> sockets, or just not quite being fully plugged into the socket.
>>>
>> Thanks for the hint. It wou.d be strange if such a cause could reliably
>> result in "directory name does not exist" as the message from the server
>> to the client, while ssh connections between the two seems to be fine.
>
> I'm not sure if that is the cause or not. I haven't looked into how nfs/nfsd
> and ssh/sshd handle some failed packets. It wouldn't surprise me if they work
> differently, causing the problem only for nfs. The server may not be generating
> that message. It may be coming from the client whenever the mount fails.
>
> I'd try fixing that issue and then see if the problem still occurs.
>
> Regards, Dave Hodgins

On Tue, 22 Aug 2023 09:16:23 -0400, William Unruh <unruh@invalid.ca> wrote:

> Thanks will try that if it occurs again. The reason I suspect the server
> is that it occured to at least two clients. On the one client I guess it
> had been rebooted and had mounted nothing from the server. On the second
> client, it had not been rebooted, and nfs worked fine, until I unmounted
> one of the partition, and then on remount, it had the same problem.
>
> I finally and accidentally did something to the server, and everything
> started working again ( the clients all remouted without problems). of
> course I did not notice what I did.

I still recommend figuring out why so many packets are being dropped. Even
if the nfs mounts are working again now, the networking problems are likely
to cause additional problems later.

In addition to checking the cables and sockets, is the time being synced
properly on all of the systems?

Regards, Dave Hodgins