Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[202405] arp_evict_nocarrier doesn't work for non-port-channels #21075

Open
arista-nwolfe opened this issue Dec 6, 2024 · 3 comments
Open

[202405] arp_evict_nocarrier doesn't work for non-port-channels #21075

arista-nwolfe opened this issue Dec 6, 2024 · 3 comments
Assignees
@prsunny
Labels
Triaged this issue has been triaged

Comments

@arista-nwolfe
Copy link
Contributor

arista-nwolfe commented Dec 6, 2024 •
edited
Loading

I see this issue via the ipfwd/test_nhop_group.py::test_nhop_group_interface_flap sonic-mgmt test on 202405 on T2 devices.

The test does the following from a high level:

  1. set arp_evict_nocarrier=0 for our egress port
  2. Configure a direct route on the egress port
  3. Configure an ARP on the egress port
  4. Configure a via route to the egress port
  5. Flap the egress port's peer port
  6. Inject a packet which will confirm the if the ARP wasn't evicted

Here is the results if I run on an Ethernet port (ARP is evicted):

root@cmp227-4:~# sudo ip netns exec asic0 sh -c 'echo 0 > /proc/sys/net/ipv4/conf/Ethernet96/arp_evict_nocarrier'
root@cmp227-4:~# sudo ip netns exec asic0 ip address add 171.16.0.3/16 dev Ethernet96
root@cmp227-4:~# sudo ip netns exec asic0 arp -s 171.16.0.11 C0:FF:EF:00:00:0b
root@cmp227-4:~# sudo ip netns exec asic0 arp -s 171.16.0.12 C0:FF:EF:00:00:0c
root@cmp227-4:~# sudo ip netns exec asic0 ip route add 192.168.100.60/32 nexthop via 171.16.0.11 nexthop via 171.16.0.12

root@cmp227-4:~# sudo ip netns exec asic0 cat /proc/sys/net/ipv4/conf/Ethernet96/arp_evict_nocarrier
0

root@cmp227-4:~# sudo ip netns exec asic0 show ip route 171.16.0.3
Routing entry for 171.16.0.0/16
  Known via "connected", distance 0, metric 0, best
  Last update 00:01:08 ago
  * directly connected, Ethernet96

root@cmp227-4:~# sudo ip netns exec asic0 ip neighbor show | grep -i 171
171.16.0.12 dev Ethernet96 lladdr c0:ff:ef:00:00:0c PERMANENT
171.16.0.11 dev Ethernet96 lladdr c0:ff:ef:00:00:0b PERMANENT

root@cmp227-4:~# sudo ip netns exec asic0 show ip route 192.168.100.60
Routing entry for 192.168.100.60/32
  Known via "kernel", distance 0, metric 0, best
  Last update 00:01:02 ago
  * 171.16.0.12, via Ethernet96
  * 171.16.0.11, via Ethernet96


bko508(config-if-Et27/1)#shutdown
bko508(config-if-Et27/1)#no shutdown


root@cmp227-4:~# sudo ip netns exec asic0 cat /proc/sys/net/ipv4/conf/Ethernet96/arp_evict_nocarrier
0

root@cmp227-4:~# sudo ip netns exec asic0 show ip route 171.16.0.3
Routing entry for 171.16.0.0/16
  Known via "connected", distance 0, metric 0, best
  Last update 00:01:49 ago
  * directly connected, Ethernet96

root@cmp227-4:~# sudo ip netns exec asic0 ip neighbor show | grep -i 171

root@cmp227-4:~# sudo ip netns exec asic0 show ip route 192.168.100.60

Here is the results if I run on an PortChannel port (ARP isn't evicted):

root@cmp227-4:~# sudo ip netns exec asic0 sh -c 'echo 0 > /proc/sys/net/ipv4/conf/PortChannel102/arp_evict_nocarrier'
root@cmp227-4:~# sudo ip netns exec asic0 ip address add 170.16.0.3/16 dev PortChannel102
root@cmp227-4:~# sudo ip netns exec asic0 arp -s 170.16.0.11 C0:FF:FF:00:00:0b
root@cmp227-4:~# sudo ip netns exec asic0 arp -s 170.16.0.12 C0:FF:FF:00:00:0c
root@cmp227-4:~# sudo ip netns exec asic0 ip route add 192.168.100.70/32 nexthop via 170.16.0.11 nexthop via 170.16.0.12


root@cmp227-4:~# sudo ip netns exec asic0 cat /proc/sys/net/ipv4/conf/PortChannel102/arp_evict_nocarrier
0

root@cmp227-4:~# sudo ip netns exec asic0 show ip route 170.16.0.3
Routing entry for 170.16.0.0/16
  Known via "connected", distance 0, metric 0, best
  Last update 00:00:38 ago
  * directly connected, PortChannel102

root@cmp227-4:~# sudo ip netns exec asic0 ip neighbor show | grep -i 170
170.16.0.12 dev PortChannel102 lladdr c0:ff:ff:00:00:0c PERMANENT
170.16.0.11 dev PortChannel102 lladdr c0:ff:ff:00:00:0b PERMANENT

root@cmp227-4:~# sudo ip netns exec asic0 show ip route 192.168.100.70
Routing entry for 192.168.100.70/32
  Known via "kernel", distance 0, metric 0, best
  Last update 00:00:58 ago
  * 170.16.0.12, via PortChannel102
  * 170.16.0.11, via PortChannel102



bko508(config-if-Et15/1,15/3)#shutdown
bko508(config-if-Et15/1,15/3)#no shutdown



root@cmp227-4:~# sudo ip netns exec asic0 cat /proc/sys/net/ipv4/conf/PortChannel102/arp_evict_nocarrier
0

root@cmp227-4:~# sudo ip netns exec asic0 show ip route 170.16.0.3
Routing entry for 170.16.0.0/16
  Known via "connected", distance 0, metric 0, best
  Last update 00:00:26 ago
  * directly connected, PortChannel102

root@cmp227-4:~# sudo ip netns exec asic0 ip neighbor show | grep -i 170
170.16.0.12 dev PortChannel102 lladdr c0:ff:ff:00:00:0c PERMANENT
170.16.0.11 dev PortChannel102 lladdr c0:ff:ff:00:00:0b PERMANENT

root@cmp227-4:~# sudo ip netns exec asic0 show ip route 192.168.100.70
Routing entry for 192.168.100.70/32
  Known via "kernel", distance 0, metric 0, best
  Last update 00:03:10 ago
  * 170.16.0.12, via PortChannel102
  * 170.16.0.11, via PortChannel102
@arista-nwolfe
Copy link
Contributor Author

@arlakshm

@arista-nwolfe
Copy link
Contributor Author

I see the same behavior on CW2 (single-asic) LC.

Ethernet interfaces don't see the ARP preserved:

root@cmp227-6:~# echo 0 > /proc/sys/net/ipv4/conf/Ethernet76/arp_evict_nocarrier
root@cmp227-6:~# ip address add 173.16.0.3/16 dev Ethernet76
root@cmp227-6:~# arp -s 173.16.0.11 C0:FF:DF:00:00:0b
root@cmp227-6:~# arp -s 173.16.0.12 C0:FF:DF:00:00:0c
root@cmp227-6:~# ip route add 192.168.100.80/32 nexthop via 173.16.0.11 nexthop via 173.16.0.12

root@cmp227-6:~# cat /proc/sys/net/ipv4/conf/Ethernet76/arp_evict_nocarrier
0

root@cmp227-6:~# show ip route 173.16.0.3
Routing entry for 173.16.0.0/16
  Known via "connected", distance 0, metric 0, best
  Last update 00:01:10 ago
  * directly connected, Ethernet76

root@cmp227-6:~# ip neighbor show | grep -i 173
173.16.0.11 dev Ethernet76 lladdr c0:ff:df:00:00:0b PERMANENT
173.16.0.12 dev Ethernet76 lladdr c0:ff:df:00:00:0c PERMANENT

root@cmp227-6:~# show ip route 192.168.100.80
Routing entry for 192.168.100.80/32
  Known via "kernel", distance 0, metric 0, best
  Last update 00:01:31 ago
  * 173.16.0.11, via Ethernet76
  * 173.16.0.12, via Ethernet76



upp684(config-if-Et2/1)#shutdown
upp684(config-if-Et2/1)#no shutdown



root@cmp227-6:~# cat /proc/sys/net/ipv4/conf/Ethernet76/arp_evict_nocarrier
0
root@cmp227-6:~# show ip route 173.16.0.3
Routing entry for 173.16.0.0/16
  Known via "connected", distance 0, metric 0, best
  Last update 00:00:19 ago
  * directly connected, Ethernet76


root@cmp227-6:~# ip neighbor show | grep -i 173

root@cmp227-6:~# show ip route 192.168.100.80

While PortChannel interfaces do see the ARP preserved:

root@cmp227-6:~# echo 0 > /proc/sys/net/ipv4/conf/PortChannel1049/arp_evict_nocarrier
root@cmp227-6:~# ip address add 174.16.0.3/16 dev PortChannel1049
root@cmp227-6:~# arp -s 174.16.0.11 C0:FF:CF:00:00:0b
root@cmp227-6:~# arp -s 174.16.0.12 C0:FF:CF:00:00:0c
root@cmp227-6:~# ip route add 192.168.100.90/32 nexthop via 174.16.0.11 nexthop via 174.16.0.12

root@cmp227-6:~# cat /proc/sys/net/ipv4/conf/PortChannel1049/arp_evict_nocarrier
0

root@cmp227-6:~# show ip route 174.16.0.3
Routing entry for 174.16.0.0/16
  Known via "connected", distance 0, metric 0, best
  Last update 00:01:13 ago
  * directly connected, PortChannel1049

root@cmp227-6:~# ip neighbor show | grep -i 174
174.16.0.11 dev PortChannel1049 lladdr c0:ff:cf:00:00:0b PERMANENT
174.16.0.12 dev PortChannel1049 lladdr c0:ff:cf:00:00:0c PERMANENT

root@cmp227-6:~# show ip route 192.168.100.90
Routing entry for 192.168.100.90/32
  Known via "kernel", distance 0, metric 0, best
  Last update 00:01:06 ago
  * 174.16.0.11, via PortChannel1049
  * 174.16.0.12, via PortChannel1049



nv598(config-if-Et12/1,49/1)#shutdown
nv598(config-if-Et12/1,49/1)#no shutdown



root@cmp227-6:~# cat /proc/sys/net/ipv4/conf/PortChannel1049/arp_evict_nocarrier
0

root@cmp227-6:~# show ip route 174.16.0.3
Routing entry for 174.16.0.0/16
  Known via "connected", distance 0, metric 0, best
  Last update 00:00:13 ago
  * directly connected, PortChannel1049

root@cmp227-6:~# ip neighbor show | grep -i 174
174.16.0.11 dev PortChannel1049 lladdr c0:ff:cf:00:00:0b PERMANENT
174.16.0.12 dev PortChannel1049 lladdr c0:ff:cf:00:00:0c PERMANENT

root@cmp227-6:~# show ip route 192.168.100.90
Routing entry for 192.168.100.90/32
  Known via "kernel", distance 0, metric 0, best
  Last update 00:02:23 ago
  * 174.16.0.11, via PortChannel1049
  * 174.16.0.12, via PortChannel1049

@tjchadaga
Copy link
Contributor

@prsunny for viz

@tjchadaga tjchadaga added the Triaged this issue has been triaged label Dec 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@prsunny prsunny

Labels
Triaged this issue has been triaged
Projects
SONiC Chassis
Status: No status
Milestone
No milestone
Development

No branches or pull requests
3 participants
@prsunny @tjchadaga @arista-nwolfe