Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

queuestat: Empty on 202211 #13

Open
bluecmd opened this issue Mar 1, 2024 · 5 comments
Open

queuestat: Empty on 202211 #13

bluecmd opened this issue Mar 1, 2024 · 5 comments

Comments

@bluecmd
Copy link
Member

bluecmd commented Mar 1, 2024

Seems to have worked on 202111 but on 202211 it is empty:

(vrf:mgmt)bluecmd@ixp-lab-sw1:~$ queuestat
  Port    TxQ    Counter/pkts    Counter/bytes    Drop/pkts    Drop/bytes
------  -----  --------------  ---------------  -----------  ------------

  Port    TxQ    Counter/pkts    Counter/bytes    Drop/pkts    Drop/bytes
------  -----  --------------  ---------------  -----------  ------------

  Port    TxQ    Counter/pkts    Counter/bytes    Drop/pkts    Drop/bytes
------  -----  --------------  ---------------  -----------  ------------
[..]
@bluecmd
Copy link
Member Author

bluecmd commented Mar 1, 2024

Seems the COUNTERS_QUEUE_NAME_MAP table is empty in COUNTERS_DB:

127.0.0.1:6379[2]> hgetall COUNTERS_QUEUE_NAME_MAP
(empty array)

In 202111 it is populated.

@bluecmd
Copy link
Member Author

bluecmd commented Mar 1, 2024

Seems to be handled by "Flex counter" in SONiC. It seems enabled:

(vrf:mgmt)bluecmd@ixp-lab-sw1:~$ sudo counterpoll show
Type                        Interval (in ms)    Status
--------------------------  ------------------  --------
QUEUE_STAT                  default (10000)     enable
PORT_STAT                   default (1000)      enable
PORT_BUFFER_DROP            default (60000)     enable
RIF_STAT                    default (1000)      enable
QUEUE_WATERMARK_STAT        default (60000)     enable
PG_WATERMARK_STAT           default (60000)     enable
PG_DROP_STAT                default (10000)     enable
BUFFER_POOL_WATERMARK_STAT  default (60000)     enable
ACL                         10000               enable

I do see the following message in the log:

Feb 28 13:45:07.272066 ixp-lab-sw1 WARNING python3: :- blockable: Key '{COUNTERS_PORT_NAME_MAP}' unavailable in database '{COUNTERS_DB}'
Feb 28 13:45:07.278638 ixp-lab-sw1 WARNING python3: :- blockable: Key '{COUNTERS_PORT_NAME_MAP}' unavailable in database '{COUNTERS_DB}'

@bluecmd
Copy link
Member Author

bluecmd commented Mar 1, 2024

Might be related to that we do not have a BUFFER_QUEUE table in the config?
Maybe related to this feature: https://github.com/sonic-net/SONiC/blob/417a096734f24b6d6ee8e852c74a164410568fe1/doc/SONiC_202205_Release_Notes.md#queue-statistics-based-on-queue-configurations-and-not-max

Although this shouldn't matter, as if the interface is not specified it should default to max number of buffer counters.

Ethernet60 seems to read a sensible default anyway:

Mar  1 15:35:03.837925 ixp-lab-sw1 INFO swss#orchagent: :- initializeQueues: Get 20 queues for port Ethernet60
Mar  1 15:35:03.879777 ixp-lab-sw1 INFO swss#orchagent: :- initializeQueues: Get queues for port Ethernet60

getQueueTypeAndIndex seems to be called 3 times, but I can find 4 call sites, so maybe the missing one is ours?

@bluecmd
Copy link
Member Author

bluecmd commented Mar 1, 2024

Loading the following configuration made queue stats appear for Ethernet68:

{
  "BUFFER_POOL": {
    "ingress_lossless_pool": {
      "size": "32744448",
      "type": "ingress",
      "mode": "static"
    },
    "egress_lossy_pool": {
      "size": "32744448",
      "type": "egress",
      "mode": "dynamic"
    }
  },
  "BUFFER_PROFILE": {
    "ingress_lossy_profile": {
      "pool": "ingress_lossless_pool",
      "size": "0",
      "static_th": "32744448"
    },
    "egress_lossy_profile": {
      "pool": "egress_lossy_pool",
      "size": "0",
      "dynamic_th": "3"
    }
  },
  "BUFFER_PG": {
    "Ethernet68|0": {
      "profile": "ingress_lossy_profile"
    }
  },
  "BUFFER_QUEUE": {
    "Ethernet68|0-6": {
      "profile": "egress_lossy_profile"
    }
  }
}

@bluecmd
Copy link
Member Author

bluecmd commented Mar 1, 2024

FWIW, this is the queue stats on one of the more active ports on 202111:

       Port    TxQ    Counter/pkts    Counter/bytes    Drop/pkts    Drop/bytes
-----------  -----  --------------  ---------------  -----------  ------------
Ethernet124    UC0    137424676816   80666063346143        10191      14384228
Ethernet124    UC1               0                0            0             0
Ethernet124    UC2               0                0            0             0
Ethernet124    UC3               0                0            0             0
Ethernet124    UC4               0                0            0             0
Ethernet124    UC5               0                0            0             0
Ethernet124    UC6               0                0            0             0
Ethernet124    UC7         6927623        634638331            0             0
Ethernet124    UC8               0                0            0             0
Ethernet124    UC9               0                0            0             0
Ethernet124   MC10           14848         17854334            0             0
Ethernet124   MC11               0                0            0             0
Ethernet124   MC12               0                0            0             0
Ethernet124   MC13               0                0            0             0
Ethernet124   MC14               0                0            0             0
Ethernet124   MC15               0                0            0             0
Ethernet124   MC16               0                0            0             0
Ethernet124   MC17         1613345        150992262            0             0
Ethernet124   MC18               0                0            0             0
Ethernet124   MC19               0                0            0             0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant