Dell S5248F-ON incorrect media_settings.json causing orchagent crash

[henrymao-zz]

build image based on 202405 branch, and load on DELL S5248F-ON. Orchagent crash is observed.

debug shows it is caused by media_settings.json for S5248F-ON does not match the ports on board.
2024 Aug 30 12:33:17.754499 sonic ERR syncd#syncd: [none] SAI_API_PORT:brcm_sai_create_port_serdes:9612 Port lane count 4 is different from supported lane count 1

2024 Aug 30 12:33:17.745170 sonic ERR swss#orchagent: :- create: create status: SAI_STATUS_INVALID_ATTRIBUTE_MAX
2024 Aug 30 12:33:17.745239 sonic ERR swss#orchagent: :- setPortSerdesAttribute: Failed to create port serdes for port 0x1000000000023
2024 Aug 30 12:33:17.745285 sonic ERR swss#orchagent: :- handleSaiCreateStatus: Encountered failure in create operation, exiting orchagent, SAI API: SAI_API_PORT, status: SAI_STATUS_INVALID_ATTRIBUTE_MAX
2024 Aug 30 12:33:17.745331 sonic NOTICE swss#orchagent: :- notifySyncd: sending syncd: SYNCD_INVOKE_DUMP
2024 Aug 30 12:33:17.753428 sonic INFO syncd#syncd: [none] SAI_API_PORT:_brcm_port_rx_squelch_set:18081 Port 82 rx squelch set: Not 1G/10G, skipping...
2024 Aug 30 12:33:17.754499 sonic INFO syncd#syncd: [none] SAI_API_PORT:_brcm_sai_link_event_cb:1533 Port 82 link down event cause: ADMIN_DOWN
2024 Aug 30 12:33:17.754499 sonic ERR syncd#syncd: [none] SAI_API_PORT:brcm_sai_create_port_serdes:9612 Port lane count 4 is different from supported lane count 1
2024 Aug 30 12:33:17.754499 sonic ERR syncd#syncd: :- sendApiResponse: api SAI_COMMON_API_CREATE failed in syncd mode: SAI_STATUS_INVALID_ATTRIBUTE_MAX
2024 Aug 30 12:33:17.754499 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_PORT_SERDES_ATTR_PORT_ID: oid:0x1000000000023
2024 Aug 30 12:33:17.754499 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_PORT_SERDES_ATTR_PREEMPHASIS: 4:1198600,1198600,1198600,1198600
2024 Aug 30 12:33:17.754499 sonic NOTICE syncd#syncd: :- processNotifySyncd: Invoking SAI failure dump 
2024 Aug 30 12:33:17.755875 sonic NOTICE swss#orchagent: :- sai_redis_notify_syncd: invoked DUMP succeeded
2024 Aug 30 12:33:17.772728 sonic WARNING pmon#xcvrd: $$$ Ethernet1 handle_port_update_event() : op=SET DB:STATE_DB Table:TRANS

Core file decode:

root@sonic:/debug# gzip -d orchagent.1725021197.55.core.gz 
root@sonic:/debug# cd ../
root@sonic:/# gdb /usr/bin/orchagent
GNU gdb (Ubuntu 15.0.50.20240403-0ubuntu1) 15.0.50.20240403-git
Copyright (C) 2024 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/orchagent...
Reading symbols from /usr/lib/debug/.build-id/d9/a9504a7d1f77dc9ae364097f6547044d4138eb.debug...
(gdb) core /debug/orchagent.1725021197.55.core
[New LWP 55]
[New LWP 76]
[New LWP 91]
[New LWP 77]
[New LWP 79]
[New LWP 74]
[New LWP 78]
[New LWP 181]
[New LWP 182]
[New LWP 226]
warning: could not find '.gnu_debugaltlink' file for /lib/x86_64-linux-gnu/libsaimeta.so.0
warning: could not find '.gnu_debugaltlink' file for /lib/x86_64-linux-gnu/libsaimetadata.so.0
warning: could not find '.gnu_debugaltlink' file for /lib/x86_64-linux-gnu/libhiredis.so.1.1.0
warning: could not find '.gnu_debugaltlink' file for /lib/x86_64-linux-gnu/libboost_serialization.so.1.83.0
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/bin/orchagent -d /var/log/swss -b 1024 -s -m 20:88:10:49:0f:80'.
Program terminated with signal SIGABRT, Aborted.
#0  0x00007168f5940b1c in pthread_kill () from /lib/x86_64-linux-gnu/libc.so.6
[Current thread is 1 (Thread 0x7168f51f8a40 (LWP 55))]
(gdb) bt
#0  0x00007168f5940b1c in pthread_kill () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007168f58e726e in raise () from /lib/x86_64-linux-gnu/libc.so.6

#2  0x00007168f58ca8ff in abort () from /lib/x86_64-linux-gnu/libc.so.6
#3  0x000062245ad769d9 in handleSaiFailure(bool) [clone .constprop.0] (abort_on_failure=true) at /usr/src/sonic-1.0.0/orchagent/saihelper.cpp:834
#4  0x000062245ad76af8 in handleSaiCreateStatus(_sai_api_t, int, void*) [clone .constprop.0] (api=<optimized out>, status=<optimized out>, context=0x0) at /usr/src/sonic-1.0.0/orchagent/saihelper.cpp:619
#5  0x000062245add3019 in PortsOrch::setPortSerdesAttribute(unsigned long, unsigned long, std::map<_sai_port_serdes_attr_t, std::vector<unsigned int, std::allocator<unsigned int> >, std::less<_sai_port_serdes_attr_t>, std::allocator<std::pair<_sai_port_serdes_attr_t const, std::vector<unsigned int, std::allocator<unsigned int> > > > >&) [clone .isra.0] (port_id=281474976710691, switch_id=9288674231451648, serdes_attr=std::map with 1 element = {...}, this=<optimized out>) at /usr/src/sonic-1.0.0/orchagent/portsorch.cpp:8489
#6  0x000062245aaffbd3 in PortsOrch::doPortTask (this=<optimized out>, consumer=...) at /usr/src/sonic-1.0.0/orchagent/portsorch.cpp:4591
#7  0x000062245ab0773f in PortsOrch::doTask (this=0x62245b4cb340, consumer=...) at /usr/src/sonic-1.0.0/orchagent/portsorch.cpp:5551
#8  0x000062245aa92f95 in Consumer::drain (this=0x62245b4cd080) at /usr/src/sonic-1.0.0/orchagent/orch.cpp:260
#9  Consumer::drain (this=0x62245b4cd080) at /usr/src/sonic-1.0.0/orchagent/orch.cpp:257
#10 Consumer::execute (this=0x62245b4cd080) at /usr/src/sonic-1.0.0/orchagent/orch.cpp:254
#11 0x000062245aa878e3 in OrchDaemon::start (this=0x62245b498cc0) at /usr/src/sonic-1.0.0/orchagent/orchdaemon.cpp:873
#12 0x000062245a9ee8d5 in main (argc=<optimized out>, argv=<optimized out>) at /usr/src/sonic-1.0.0/orchagent/main.cpp:800
(gdb) 
#0  0x00007168f5940b1c in pthread_kill () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007168f58e726e in raise () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x00007168f58ca8ff in abort () from /lib/x86_64-linux-gnu/libc.so.6

[tjchadaga]

@jeff-yin - Could you please help take a look?

[jeff-yin]

@arunlk-dell can you check on this, since you recently worked on this platform to support dynamic breakout?

The media settings do look like they need to be fixed.