From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D2B9443E40; Thu, 11 Apr 2024 09:16:29 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B9FFF402ED; Thu, 11 Apr 2024 09:16:25 +0200 (CEST) Received: from mail-yb1-f174.google.com (mail-yb1-f174.google.com [209.85.219.174]) by mails.dpdk.org (Postfix) with ESMTP id 93B604029E for ; Wed, 10 Apr 2024 09:56:31 +0200 (CEST) Received: by mail-yb1-f174.google.com with SMTP id 3f1490d57ef6-dd045349d42so5897556276.2 for ; Wed, 10 Apr 2024 00:56:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1712735791; x=1713340591; darn=dpdk.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=WvKEWdekwr6fEXpxPOVWk1xFhDGbw+/mr/9jf8i671s=; b=F1K9ZH67VMkPy+1fljCjXYsVkvUBopzOXL5gnPOGOLdiTlXSl5W19oMrVSlVwDtgT6 Gwjxgkf7v2aUCKM44But2X0aAXeyr+YHXwyyY3hf0SN2+h6IHQEeHzEoodP9AlKjSSwF di1XKB0j5BQPgDsbIKQchU5nLX7MI8M3JOBnk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712735791; x=1713340591; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=WvKEWdekwr6fEXpxPOVWk1xFhDGbw+/mr/9jf8i671s=; b=MYCa+XD2k9ly6s3PK5AbG2XU4w64i53K7z+vD8hN9/6yOpQV1pajbcbOAjqUeYr/Qk JyncBwxLps88DBqptHMenXDMAz5nhz1IN12HoeGKejwEhKAr3xlNP6UNTgKoBkEWIyQh quCDrDe0otFKW+e9JLRgipiRUI5+NNY/cgwfoIohrA71srAUCNl1Q4t7cHNvOjcDJ6CQ XENcL0YfpeC2rSLU3XJM6icHRLPL3KvaujdBW90SBf/tDyMwasv4ixlac8aeHUQFHDMk h+uIKGku6tAN4Xz8+3nLe+JuDR5V3hsqobf4bguSvOCbpaUiHxu3u6weBtEb+bzHV1hh +saw== X-Gm-Message-State: AOJu0Yx08/B6DfrWjbgCuTlcDux96N3BBJ5zkCNiIazKJ5Ww7B7i+Uci 7xb5u37X/Dl+wgL+drl6DD7aLoOKxyEl3EEXWm71G+1hsOtabPtKn04P7XrXYCSJ+slBzQkp4SU cQk6zRdwS7VF0joNr0NO2QHjaUIidjsQ5KC7ej9Yu9v++o+slzjfiByAvhJKZCOo0duu7mfs1y6 XMpGNP X-Google-Smtp-Source: AGHT+IGnc9By9rvYbnB8zrfLb4Zx7cKBa3mtDiVEEFFVQD8oq4CSjN/I7UwygRoenbbtlj61JOmXGnWi4gSX/u7HWto= X-Received: by 2002:a25:684d:0:b0:dcf:313d:d4ab with SMTP id d74-20020a25684d000000b00dcf313dd4abmr2016775ybc.54.1712735790636; Wed, 10 Apr 2024 00:56:30 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Samar Yadav Date: Wed, 10 Apr 2024 13:26:18 +0530 Message-ID: Subject: Re: DPDK Secondary process not able to xmit packets with MLX5 VF To: Slava Ovsiienko Cc: "dev@dpdk.org" , "users@dpdk.org" , Matan Azrad , Ori Kam , Suanming Mou , Mukul Sinha , Tathagat Priyadarshi , Srinivasa Srikanth Srikanth Podila , Vipin PR Content-Type: multipart/alternative; boundary="00000000000065ce0e0615b9607b" X-Mailman-Approved-At: Thu, 11 Apr 2024 09:16:22 +0200 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --00000000000065ce0e0615b9607b Content-Type: text/plain; charset="UTF-8" Hi Slava, Thanks so much for replying, appreciate it. >> Did you start queues in the secondary process? Yes, tx_queue_state in secondary process is RTE_ETH_QUEUE_STATE_STARTED, *gdb output from secondary process for port1 (non-working port, managed by secondary process) * (gdb) p rte_eth_devices[1]->data[0] $3 = {name = "0000:00:08.0", '\000' , rx_queues = 0x600177c28980, tx_queues = 0x600177c26900, nb_rx_queues = 1, nb_tx_queues = 1, sriov = {active = 0 '\000', nb_q_per_pool = 0 '\000', def_vmdq_idx = 0, def_pool_q_idx = 0}, dev_private = 0x600177c6d7c0, dev_link = { link_speed = 40000, link_duplex = 1, link_autoneg = 1, link_status = 1}, dev_conf = {link_speeds = 0, rxmode = {mq_mode = RTE_ETH_MQ_RX_NONE, mtu = 9000, max_lro_pkt_size = 0, offloads = 8193, reserved_64s = {0, 0}, reserved_ptrs = {0x0, 0x0}}, txmode = {mq_mode = RTE_ETH_MQ_TX_NONE, offloads = 32815, pvid = 0, hw_vlan_reject_tagged = 0 '\000', hw_vlan_reject_untagged = 0 '\000', hw_vlan_insert_pvid = 0 '\000', reserved_64s = {0, 0}, reserved_ptrs = {0x0, 0x0}}, lpbk_mode = 0, rx_adv_conf = {rss_conf = {rss_key = 0x0, rss_key_len = 0 '\000', rss_hf = 0}, vmdq_dcb_conf = { nb_queue_pools = (unknown: 0), enable_default_pool = 0 '\000', default_pool = 0 '\000', nb_pool_maps = 0 '\000', pool_map = {{vlan_id = 0, pools = 0} < repeats 64 times>}, dcb_tc = "\000\000\000\000\000\000\000"}, dcb_rx_conf = {nb_tcs = (unknown: 0), dcb_tc = "\000\000\000\000\000\000\000"}, vmdq_rx_conf = {nb_queue_pools = ( unknown: 0), enable_default_pool = 0 '\000', default_pool = 0 '\000', enable_loop_back = 0 '\000', nb_pool_maps = 0 '\000', rx_mode = 0, pool_map = {{vlan_id = 0, pools = 0} }}}, tx_adv_conf = { vmdq_dcb_tx_conf = {nb_queue_pools = (unknown: 0), dcb_tc = " \000\000\000\000\000\000\000"}, dcb_tx_conf = {nb_tcs = (unknown: 0), dcb_tc = "\000\000\000\000\000\000\000"}, vmdq_tx_conf = {nb_queue_pools = (unknown: 0)}}, dcb_capability_en = 0, intr_conf = {lsc = 1, rxq = 0, rmv = 0}}, mtu = 1500, min_rx_buf_size = 4294967295, rx_mbuf_alloc_failed = 0, mac_addrs = 0x600177c6d7e0, mac_pool_sel = {0 }, hash_mac_addrs = 0x0, port_id = 1, promiscuous = 1 '\001', scattered_rx = 0 '\000', all_multicast = 1 '\001', dev_started = 1 '\001', lro = 0 '\000', dev_configured = 1 '\001', flow_configured = 0 '\000', rx_queue_state = " \001", '\000' , tx_queue_state = "\001", '\000' , dev_flags = 75, numa_node = -1, vlan_filter_conf = {ids = {0 < repeats 64 times>}}, owner = {id = 0, name = '\000' }, representor_id = 0, backer_port_id = 32, flow_ops_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' < repeats 39 times>, __align = 0}} (gdb) p rte_eth_devices[1] $4 = {rx_pkt_burst = 0x5591d17ddfb2 , tx_pkt_burst = 0x5591d197268c , tx_pkt_prepare = 0x0, rx_queue_count = 0x0, rx_descriptor_status = 0x5591d17dc903 , tx_descriptor_status = 0x5591d17f8dd0 , data = 0x600177cfb500, process_private = 0x600179b1b500, dev_ops = 0x5591d20ac180 , device = 0x5591d5a55be0, intr_handle = 0x5591d5a54dd0, link_intr_cbs = {tqh_first = 0x0, tqh_last = 0x5591d3c575d8 }, post_rx_burst_cbs = {0x600177c00180, 0x0 }, pre_tx_burst_cbs = {0x600179e52d80, 0x0 }, state = RTE_ETH_DEV_ATTACHED, security_ctx = 0x0} *gdb output from primary process for port1 (non-working port, managed by secondary process)* (gdb) p rte_eth_devices[1] $3 = {rx_pkt_burst = 0x5591d17ddfb2 , tx_pkt_burst = 0x5591d197268c , tx_pkt_prepare = 0x0, rx_queue_count = 0x5591d17dcd20 , rx_descriptor_status = 0x5591d17dc903 , tx_descriptor_status = 0x5591d17f8dd0 , data = 0x600177cfb500, process_private = 0x600177c68540, dev_ops = 0x5591d20abde0 , device = 0x5591d6076440, intr_handle = 0x5591d60d0350, link_intr_cbs = {tqh_first = 0x0, tqh_last = 0x5591d3c575d8 }, post_rx_burst_cbs = {0x0 }, pre_tx_burst_cbs = {0x0 }, state = RTE_ETH_DEV_ATTACHED, security_ctx = 0x0} (gdb) p rte_eth_devices[1]->data[0] $4 = {name = "0000:00:08.0", '\000' , rx_queues = 0x600177c28980, tx_queues = 0x600177c26900, nb_rx_queues = 1, nb_tx_queues = 1, sriov = {active = 0 '\000', nb_q_per_pool = 0 '\000', def_vmdq_idx = 0, def_pool_q_idx = 0}, dev_private = 0x600177c6d7c0, dev_link = { link_speed = 40000, link_duplex = 1, link_autoneg = 1, link_status = 1}, dev_conf = {link_speeds = 0, rxmode = {mq_mode = RTE_ETH_MQ_RX_NONE, mtu = 9000, max_lro_pkt_size = 0, offloads = 8193, reserved_64s = {0, 0}, reserved_ptrs = {0x0, 0x0}}, txmode = {mq_mode = RTE_ETH_MQ_TX_NONE, offloads = 32815, pvid = 0, hw_vlan_reject_tagged = 0 '\000', hw_vlan_reject_untagged = 0 '\000', hw_vlan_insert_pvid = 0 '\000', reserved_64s = {0, 0}, reserved_ptrs = {0x0, 0x0}}, lpbk_mode = 0, rx_adv_conf = {rss_conf = {rss_key = 0x0, rss_key_len = 0 '\000', rss_hf = 0}, vmdq_dcb_conf = { nb_queue_pools = (unknown: 0), enable_default_pool = 0 '\000', default_pool = 0 '\000', nb_pool_maps = 0 '\000', pool_map = {{vlan_id = 0, pools = 0} < repeats 64 times>}, dcb_tc = "\000\000\000\000\000\000\000"}, dcb_rx_conf = {nb_tcs = (unknown: 0), dcb_tc = "\000\000\000\000\000\000\000"}, vmdq_rx_conf = {nb_queue_pools = ( unknown: 0), enable_default_pool = 0 '\000', default_pool = 0 '\000', enable_loop_back = 0 '\000', nb_pool_maps = 0 '\000', rx_mode = 0, pool_map = {{vlan_id = 0, pools = 0} }}}, tx_adv_conf = { vmdq_dcb_tx_conf = {nb_queue_pools = (unknown: 0), dcb_tc = " \000\000\000\000\000\000\000"}, dcb_tx_conf = {nb_tcs = (unknown: 0), dcb_tc = "\000\000\000\000\000\000\000"}, vmdq_tx_conf = {nb_queue_pools = (unknown: 0)}}, dcb_capability_en = 0, intr_conf = {lsc = 1, rxq = 0, rmv = 0}}, mtu = 1500, min_rx_buf_size = 4294967295, rx_mbuf_alloc_failed = 0, mac_addrs = 0x600177c6d7e0, mac_pool_sel = {0 }, hash_mac_addrs = 0x0, port_id = 1, promiscuous = 1 '\001', scattered_rx = 0 '\000', all_multicast = 1 '\001', dev_started = 1 '\001', lro = 0 '\000', dev_configured = 1 '\001', flow_configured = 0 '\000', rx_queue_state = " \001", '\000' , tx_queue_state = "\001", '\000' , dev_flags = 75, numa_node = -1, vlan_filter_conf = {ids = {0 < repeats 64 times>}}, owner = {id = 0, name = '\000' }, representor_id = 0, backer_port_id = 32, flow_ops_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' < repeats 39 times>, __align = 0}} (gdb) *gdb output from secondary process for port0 (working port, managed by primary process) * (gdb) p rte_eth_devices[0] $1 = {rx_pkt_burst = 0x5591d17ddfb2 , tx_pkt_burst = 0x5591d197268c , tx_pkt_prepare = 0x0, rx_queue_count = 0x0, rx_descriptor_status = 0x5591d17dc903 , tx_descriptor_status = 0x5591d17f8dd0 , data = 0x600177cf9d00, process_private = 0x600179e49440, dev_ops = 0x5591d20ac180 , device = 0x5591d5a55fa0, intr_handle = 0x5591d5a4dd20, link_intr_cbs = {tqh_first = 0x0, tqh_last = 0x5591d3c53558 }, post_rx_burst_cbs = {0x0 }, pre_tx_burst_cbs = {0x0 }, state = RTE_ETH_DEV_ATTACHED, security_ctx = 0x0} (gdb) p rte_eth_devices[0]->data[0] $2 = {name = "0000:00:07.0", '\000' , rx_queues = 0x600176c9c980, tx_queues = 0x600176c9a900, nb_rx_queues = 1, nb_tx_queues = 1, sriov = {active = 0 '\000', nb_q_per_pool = 0 '\000', def_vmdq_idx = 0, def_pool_q_idx = 0}, dev_private = 0x600177d29d80, dev_link = { link_speed = 40000, link_duplex = 1, link_autoneg = 1, link_status = 1}, dev_conf = {link_speeds = 0, rxmode = {mq_mode = RTE_ETH_MQ_RX_NONE, mtu = 9000, max_lro_pkt_size = 0, offloads = 8193, reserved_64s = {0, 0}, reserved_ptrs = {0x0, 0x0}}, txmode = {mq_mode = RTE_ETH_MQ_TX_NONE, offloads = 32815, pvid = 0, hw_vlan_reject_tagged = 0 '\000', hw_vlan_reject_untagged = 0 '\000', hw_vlan_insert_pvid = 0 '\000', reserved_64s = {0, 0}, reserved_ptrs = {0x0, 0x0}}, lpbk_mode = 0, rx_adv_conf = {rss_conf = {rss_key = 0x0, rss_key_len = 0 '\000', rss_hf = 0}, vmdq_dcb_conf = { nb_queue_pools = (unknown: 0), enable_default_pool = 0 '\000', default_pool = 0 '\000', nb_pool_maps = 0 '\000', pool_map = {{vlan_id = 0, pools = 0} < repeats 64 times>}, dcb_tc = "\000\000\000\000\000\000\000"}, dcb_rx_conf = {nb_tcs = (unknown: 0), dcb_tc = "\000\000\000\000\000\000\000"}, vmdq_rx_conf = {nb_queue_pools = ( unknown: 0), enable_default_pool = 0 '\000', default_pool = 0 '\000', enable_loop_back = 0 '\000', nb_pool_maps = 0 '\000', rx_mode = 0, pool_map = {{vlan_id = 0, pools = 0} }}}, tx_adv_conf = { vmdq_dcb_tx_conf = {nb_queue_pools = (unknown: 0), dcb_tc = " \000\000\000\000\000\000\000"}, dcb_tx_conf = {nb_tcs = (unknown: 0), dcb_tc = "\000\000\000\000\000\000\000"}, vmdq_tx_conf = {nb_queue_pools = (unknown: 0)}}, dcb_capability_en = 0, intr_conf = {lsc = 1, rxq = 0, rmv = 0}}, mtu = 1500, min_rx_buf_size = 4294967295, rx_mbuf_alloc_failed = 0, mac_addrs = 0x600177d29da0, mac_pool_sel = {0 }, hash_mac_addrs = 0x0, port_id = 0, promiscuous = 1 '\001', scattered_rx = 0 '\000', all_multicast = 1 '\001', dev_started = 1 '\001', lro = 0 '\000', dev_configured = 1 '\001', flow_configured = 0 '\000', rx_queue_state = " \001", '\000' , tx_queue_state = "\001", '\000' , dev_flags = 75, numa_node = -1, vlan_filter_conf = {ids = {0 < repeats 64 times>}}, owner = {id = 0, name = '\000' }, representor_id = 0, backer_port_id = 32, flow_ops_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' < repeats 39 times>, __align = 0}} *gdb output from primary process for port0 (working port, managed by primary process)* (gdb) p rte_eth_devices[0] $1 = {rx_pkt_burst = 0x5591d17ddfb2 , tx_pkt_burst = 0x5591d197268c , tx_pkt_prepare = 0x0, rx_queue_count = 0x5591d17dcd20 , rx_descriptor_status = 0x5591d17dc903 , tx_descriptor_status = 0x5591d17f8dd0 , data = 0x600177cf9d00, process_private = 0x600176cdc600, dev_ops = 0x5591d20abde0 , device = 0x5591d6076800, intr_handle = 0x5591d606e580, link_intr_cbs = {tqh_first = 0x0, tqh_last = 0x5591d3c53558 }, post_rx_burst_cbs = { 0x600177c69100, 0x0 }, pre_tx_burst_cbs = { 0x600179e52cc0, 0x0 }, state = RTE_ETH_DEV_ATTACHED, security_ctx = 0x0} (gdb) p rte_eth_devices[0]->data[0] $2 = {name = "0000:00:07.0", '\000' , rx_queues = 0x600176c9c980, tx_queues = 0x600176c9a900, nb_rx_queues = 1, nb_tx_queues = 1, sriov = {active = 0 '\000', nb_q_per_pool = 0 '\000', def_vmdq_idx = 0, def_pool_q_idx = 0}, dev_private = 0x600177d29d80, dev_link = { link_speed = 40000, link_duplex = 1, link_autoneg = 1, link_status = 1}, dev_conf = {link_speeds = 0, rxmode = {mq_mode = RTE_ETH_MQ_RX_NONE, mtu = 9000, max_lro_pkt_size = 0, offloads = 8193, reserved_64s = {0, 0}, reserved_ptrs = {0x0, 0x0}}, txmode = {mq_mode = RTE_ETH_MQ_TX_NONE, offloads = 32815, pvid = 0, hw_vlan_reject_tagged = 0 '\000', hw_vlan_reject_untagged = 0 '\000', hw_vlan_insert_pvid = 0 '\000', reserved_64s = {0, 0}, reserved_ptrs = {0x0, 0x0}}, lpbk_mode = 0, rx_adv_conf = {rss_conf = {rss_key = 0x0, rss_key_len = 0 '\000', rss_hf = 0}, vmdq_dcb_conf = { nb_queue_pools = (unknown: 0), enable_default_pool = 0 '\000', default_pool = 0 '\000', nb_pool_maps = 0 '\000', pool_map = {{vlan_id = 0, pools = 0} < repeats 64 times>}, dcb_tc = "\000\000\000\000\000\000\000"}, dcb_rx_conf = {nb_tcs = (unknown: 0), dcb_tc = "\000\000\000\000\000\000\000"}, vmdq_rx_conf = {nb_queue_pools = ( unknown: 0), enable_default_pool = 0 '\000', default_pool = 0 '\000', enable_loop_back = 0 '\000', nb_pool_maps = 0 '\000', rx_mode = 0, pool_map = {{vlan_id = 0, pools = 0} }}}, tx_adv_conf = { vmdq_dcb_tx_conf = {nb_queue_pools = (unknown: 0), dcb_tc = " \000\000\000\000\000\000\000"}, dcb_tx_conf = {nb_tcs = (unknown: 0), dcb_tc = "\000\000\000\000\000\000\000"}, vmdq_tx_conf = {nb_queue_pools = (unknown: 0)}}, dcb_capability_en = 0, intr_conf = {lsc = 1, rxq = 0, rmv = 0}}, mtu = 1500, min_rx_buf_size = 4294967295, rx_mbuf_alloc_failed = 0, mac_addrs = 0x600177d29da0, mac_pool_sel = {0 }, hash_mac_addrs = 0x0, port_id = 0, promiscuous = 1 '\001', scattered_rx = 0 '\000', all_multicast = 1 '\001', dev_started = 1 '\001', lro = 0 '\000', dev_configured = 1 '\001', flow_configured = 0 '\000', rx_queue_state = " \001", '\000' , tx_queue_state = "\001", '\000' , dev_flags = 75, numa_node = -1, vlan_filter_conf = {ids = {0 < repeats 64 times>}}, owner = {id = 0, name = '\000' }, representor_id = 0, backer_port_id = 32, flow_ops_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' < repeats 39 times>, __align = 0}} >> Only one process in any moment of time manages the queue, no shared (between process) queue data sending is allowed. Yes, we are using proc1(secondary process) to manage the port 1 and proc0 (primary process) to manage port0 . There is no sharing of the queue between processes for sending data. >> From your description it looks like there is no completions seen (in CQE), I would recommend checking the UAR/Doorbell mapping in secondary process Below are the logs from mlx5_txq_devx_obj_new which is called by proc0(rte_proc_primary) for port 1 Core:0 04/10/24 06:45:47.626580 UTC mlx5_net: Core 0 mlx5_txq_devx_obj_new 1563 ppriv: 0x600177c68540 ,ppriv->uar_table: 0x600177c68548, txq_ctrl->uar_mmap_offset:0, ppriv->uar_table[txq_data->idx]:0x7f0411ae3800, txq_data->idx: 0, txq_data->db_nc:0 and logs from txq_uar_init_secondary which gets called by proc1(rte_proc_secondary) for port 1 Core:1 04/10/24 06:45:47.767512 UTC mlx5_net: priv: 0x600177c6d7c0, priv->sh: 0x600176c25a00, priv->sh->pppriv: 0x600177c68540 Core:1 04/10/24 06:45:47.767528 UTC mlx5_net: Core 1 txq_uar_init_secondary 535 txq_ctrl:0x600177c04980 priv:0x600177c6d7c0 Core:1 04/10/24 06:45:47.767553 UTC mlx5_net: Core 1 txq_uar_init_secondary 562 primary_ppriv->uar_table: 0x600177c68548 ,uar_va:7f0411ae3800 offset:800 addr:0x7f0425b18800 Core:1 04/10/24 06:45:47.767604 UTC mlx5_net: Core 1 txq_uar_init_secondary 566 port 1 of txq 0 ppriv:0x600179b1b500 ppriv->uar_table[txq->idx]:0x7f0425b18800, txq->idx:0 , ppriv->uar_table[0]:0x7f0425b18800,ppriv->uar_table[1]:(nil) @viacheslavo@nvidia.com I noticed that data->dev_private->sh->tx_uar.obj has mmap_offset as 0 in VF case but is non-zero in PF case, Is that okay? this is returned by glue in using uar = mlx5_glue->devx_alloc_uar(cdev->ctx, uar_mapping); VF: (gdb) p *(struct mlx5dv_devx_uar *) $7 $9 = {reg_addr = 0x7f041b02d800, base_addr = 0x7f041b02d000, page_id = 26, mmap_off = 0, comp_mask = 0} PF: txq_ctrl->uar_mmap_offset:*9441280*, ppriv->uar_table[txq_data->idx]:0x7ff88d2be800, txq_data->idx: 0, txq_data->db_nc:0 Also we are using MLNX_OFED_LINUX-5.8-2.0.3.0-ubuntu20.04-x86_64 Please note the MLX PF RSS works fine for us where we have 4 queues there q0 is managed by primary process(proc0) and q1,q2,q3 managed by secondary processes proc1,proc2,proc3 respectively. We are only seeing the issue in *MLX VF.* If it works better for you, we can get on Zoom call for the same. Please let us know a convenient time for the same. Thanks, Samar -- This electronic communication and the information and any files transmitted with it, or attached to it, are confidential and are intended solely for the use of the individual or entity to whom it is addressed and may contain information that is confidential, legally privileged, protected by privacy laws, or otherwise restricted from disclosure to anyone else. If you are not the intended recipient or the person responsible for delivering the e-mail to the intended recipient, you are hereby notified that any use, copying, distributing, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited. If you received this e-mail in error, please return the e-mail to the sender, delete it from your computer, and destroy any printed copy of it. --00000000000065ce0e0615b9607b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi=C2=A0Slava,Thanks so much for=C2=A0replying, appre= ciate it.=C2=A0

>>=C2=A0Did you start queues in the secondary process?
Yes, tx_queue_state in sec= ondary=C2=A0process is=C2=A0RTE_ETH_QUEUE_STATE_STARTED,
gdb output from secondary pr= ocess for port1 (non-working port, managed by secondary process)=C2=A0
(gdb) p rte_eth_devices[1]->data[0]
$3 =3D {nam= e =3D "0000:00:08.0", '\000= ' <re= peats 51 times>, rx_queues =3D 0x600177c28980, t= x_queues =3D 0x600177c26900, nb_rx_queues =3D 1, nb_tx_queues =3D 1, sriov =3D {active = =3D 0 '<= span style=3D"color:rgb(215,186,125)">\000', nb_q_per_= pool =3D 0 '\000', def_vmdq_idx =3D= 0, def_pool_q_idx = =3D 0}, dev_private =3D 0x600177c6d7c0, dev_link =3D {
link_speed = =3D 40000, link_duplex =3D 1, link_autoneg =3D 1, link_status =3D 1}, dev_conf =3D {link_speeds =3D 0, rxmode =3D {mq_mode =3D R= TE_ETH_MQ_RX_NONE, mtu= =3D 9000, max_lro_= pkt_size =3D 0, offloads =3D 8193, reserved_64s =3D {0, 0}, reser= ved_ptrs =3D {0x0, 0x0}}, txmode <= span style=3D"color:rgb(212,212,212)">=3D {mq_mode =3D RTE_ETH_MQ_TX_NONE,
=
offloads =3D 32815, pvid =3D 0, hw_vlan_reject_ta= gged =3D 0 '\000', hw_vlan_reject_untagged =3D 0 '= \000', hw_vlan_insert_pvid =3D 0 '\000', reserved_64s =3D {0, 0}, reserved_ptrs =3D {0x0, 0x0}}, lpbk_mode =3D 0, rx_adv_conf =3D {rss_conf = =3D {rss_key =3D 0x0, rss_key_len =3D 0 '\000', rss_hf =3D 0}, vmdq_dcb_conf =3D {
nb_queue_pools =3D (unknown: 0), enable_default_pool =3D= 0 '\000= ', default_pool =3D 0 '\000', nb_pool_maps =3D 0 '\000', pool_map =3D {{vlan_id =3D 0, pools =3D 0} <repeats <= span style=3D"color:rgb(181,206,168)">64 times>}= , dcb_tc =3D "= \000\000\000\000\000\000\000<= /span>"}, dcb_rx_conf =3D {nb_tcs =3D (unknown: 0= ),
dcb_tc <= span style=3D"color:rgb(212,212,212)">=3D "\000\000\0= 00\000\000\000\000"}, vmdq_rx_conf =3D {nb_queue_pools =3D= (unknown: 0), enabl= e_default_pool =3D 0 '\000', default_pool =3D= 0 '\000= ', enable_loop_back =3D 0 '\000', = nb_pool_maps =3D 0<= /span> '\000= 9;, rx_mode =3D 0, pool_map =3D {{vlan_id =3D = 0, pools =3D= 0} <repeats 64 times>}}}, tx_adv_conf =3D {
vmdq_dcb_tx_conf =3D {nb_queue_poo= ls =3D (unknown: 0), dcb_tc =3D "\000\000\000\000= \000\000\000"}, <= span style=3D"color:rgb(156,220,254)">dcb_tx_conf =3D {nb_= tcs =3D (unknown: 0), dcb_tc =3D "\000\000\000\= 000\000\000\000"}= , vmdq_tx_conf =3D {nb_queue_pools =3D (<= span style=3D"color:rgb(78,201,176)">unknown: 0)}}, dcb_ca= pability_en =3D 0, intr_conf =3D {= lsc =3D 1, <= span style=3D"color:rgb(156,220,254)">rxq =3D 0, rmv =3D 0}},
mtu =3D = 1500, min_rx_buf_size = =3D 4294967295, rx_= mbuf_alloc_failed =3D = 0, mac_addrs =3D 0x600177c6d7e0, mac_pool_sel =3D {0 = <repeats 128 times>}, has= h_mac_addrs =3D 0x0, port_id =3D <= span style=3D"color:rgb(181,206,168)">1, promiscuous =3D 1 '\001', scattered_rx =3D 0 '\000', all_multicast =3D 1 '\001', dev_started =3D 1 '\001', lro =3D 0 '\000',
dev_c= onfigured =3D 1 '\001', flow_configured =3D 0 '\000', rx_queue_state =3D "\001", '\000' <repeats 1022 tim= es>, tx_queue_state =3D "<= span style=3D"color:rgb(215,186,125)">\001", '\000' <<= /span>repeats 1022 = times>, dev_flags =3D 75, numa_node =3D -1, vlan_filter_conf = =3D {ids =3D {0 <repeats 64 times>}}, owner =3D {= id =3D 0, name =3D '= \000' <repeats 63 times<= /span>>},
representor_id =3D 0, backer_port_id =3D 32, flow_ops_mutex =3D {__data =3D<= /span> {__lock =3D = 0, __count =3D 0, __owner =3D 0, __nusers =3D 0, __kind =3D 0, __spins= =3D 0, __elision =3D 0, __list= =3D {__prev =3D 0x0, __next =3D 0x0}}, __size =3D '<= span style=3D"color:rgb(215,186,125)">\000' <repeats 39 times>, __align = =3D 0}}
(gdb) p rte_eth_device= s[1]
$4 =3D {rx_pkt_burst =3D 0x5591d17ddfb2 <mlx5_rx_burst>, tx_pk= t_burst =3D 0x5591d197268c <mlx5_tx_burst_full>, tx_pkt_prepare = =3D 0x0, rx_queue_count =3D 0x0,= rx_descriptor_status =3D 0x5591d17dc903 <mlx5= _rx_descriptor_status>, tx= _descriptor_status =3D= 0x5591d17f8dd0 <mlx5_tx_descriptor_status>,
<= div> data =3D 0x60= 0177cfb500, process_private =3D 0x600179b1b500, dev_ops =3D 0x5591d20ac180 <mlx5_dev_sec_ops>, device =3D<= /span> 0x5591d5a55be0, intr_handle =3D 0x5591d5a= 54dd0, link_intr_cbs <= span style=3D"color:rgb(212,212,212)">=3D {tqh_first =3D 0x0, tqh_last =3D 0x5591d3c575d8 <rte_eth_devices+<= /span>16600>}, p= ost_rx_burst_cbs =3D {= 0x600177c00180,
= 0x0 <repeats 1023 times>= }, pre_tx_burst_cbs =3D {0x600179e52d80, = 0x0 <repeats 1023 times>}, state =3D RTE_ETH_DEV_ATTACHED, security_ctx =3D 0x0}<= /div>

gdb output from primary process for port1 (non-working port, mana= ged by secondary process)
(gdb) p rte_eth_devices[1]
$3 =3D {rx_= pkt_burst =3D 0x5591d17ddfb2 <mlx5_rx_burst>, tx_pkt_burst =3D 0x5591d197268c <mlx5_tx_burst_full>, tx_pkt_prepare =3D 0x0,= rx_queue_count =3D 0x5591d17dcd20 <mlx5_rx_= queue_count>, rx_descripto= r_status =3D 0x5591d17dc903 <mlx5_rx_descriptor_status>,
tx_descriptor_status =3D = 0x5591d17f8dd0 <mlx5_tx_des= criptor_status>, data =3D 0x600177cfb500, process_private =3D 0x600177c68540, dev_ops =3D 0x5591d20abde0 <mlx5_dev_ops>, device =3D 0x5591d60764= 40, intr_handle =3D 0x5591d60d0350, link_= intr_cbs =3D {tqh_first =3D 0x0,
tqh_last =3D 0x5591d3c575d8 <rte_eth_devices+16600>}, post_rx_burst_cbs =3D {0x0 <repeats 1= 024 times>}, pre_tx_burst_cbs =3D {0x0 <repea= ts 1024 times>}, state =3D RTE_ETH_DEV_ATTACHED, s= ecurity_ctx =3D 0x0}
(gdb) p rte_eth_devices[1]->data[0]
$4 =3D = {name =3D "000= 0:00:08.0", '= \000' <repeats 51 times<= /span>>, rx_queues =3D 0x600177c28980,= tx_queues =3D 0x60= 0177c26900, nb_rx_queues =3D 1, nb_tx_que= ues =3D 1, s= riov =3D {active =3D 0 '\000', nb_q_per_pool =3D 0 '\000', def_vmdq_idx =3D 0, def_pool_q_idx = =3D 0}, dev_private= =3D 0x600177c6d7c0, dev_link =3D = {
link_speed =3D 40000, link_duple= x =3D 1, lin= k_autoneg =3D 1, link_status =3D <= span style=3D"color:rgb(181,206,168)">1}, dev_conf =3D {link_speeds =3D 0, rxmode =3D {mq_mode =3D RTE_ETH_MQ_RX_NONE, mtu =3D 9000, max_lro_pkt_size =3D 0, offloads =3D 819= 3, reserved_64s =3D {0, 0}, reserved_ptrs =3D {0x0, 0x0}}, txmode =3D {mq_mode =3D RTE_ETH_MQ_TX_NONE,
offloads =3D 32815, pvid =3D= 0, hw_vlan_reject_tagged =3D 0 '\000'= , hw_vlan_reject_untagged =3D 0 '\000', hw_vlan_inse= rt_pvid =3D 0 '\000', reserved_64s =3D= {0, 0}, reserved_p= trs =3D {0x0, 0x0}}, lpbk_mode <= span style=3D"color:rgb(212,212,212)">=3D 0, rx_adv_conf =3D {rss_conf =3D {rss_key =3D 0x0, rss_key_len= =3D 0 '= \000', rss_hf =3D 0}, vmdq_= dcb_conf =3D {
nb_queue_pools =3D (unknown: 0)= , enable_default_pool =3D 0 '\000', default_pool =3D 0 '\000', nb_po= ol_maps =3D 0 '\000', pool_map =3D {{<= span style=3D"color:rgb(156,220,254)">vlan_id =3D 0= , pools =3D 0} <repeats 64 times>}, d= cb_tc =3D "\000\000\000\000\000\000\000"}, dcb_rx_conf =3D {nb_tcs =3D<= /span> (unknown: 0),
dcb_tc = =3D "\000\000\000\000\000\000\000"}, vmdq_rx_conf =3D {nb_queue_pools =3D (unknown: 0), enable_default_pool =3D = 0 '\000&= #39;, default_pool =3D 0 '\000', enable_loop_= back =3D 0 '\000', nb_pool_maps =3D= 0 '\000= ', rx_mode =3D<= /span> 0, pool_map = =3D {{vlan_id =3D 0, pools =3D 0} <repeats 64 times>}}}, tx_adv_conf = =3D {
vmdq_dc= b_tx_conf =3D {nb_queue_pools =3D (unknown= : 0), dcb_tc =3D "\000\000\000\000\000\000\000"}, dcb_tx_conf =3D {nb_tcs =3D (unkno= wn: 0), dcb_tc =3D "\000\000\000\000\000\000\000"}, vmdq_tx_conf =3D {nb_queue_pools =3D (unknown: 0)}}, dcb_capability_en =3D 0, intr_conf =3D {lsc =3D 1, rxq =3D 0, rmv =3D 0}},
mtu =3D 1500, min_rx_buf_size =3D 4294967295, rx_mbuf_alloc_failed =3D 0, mac_addrs = =3D 0x600177c6d7e0, mac_pool_sel =3D {0 <repeats <= span style=3D"color:rgb(181,206,168)">128 times>= }, hash_mac_addrs =3D 0x0, port_id =3D 1, promiscuous= =3D 1 '= \001', scattere= d_rx =3D 0 '\000', all_multicast =3D 1 '\001', dev_started =3D 1 '= \001', lro =3D 0 '\000',
dev_configured =3D 1 '\001', flow_configured =3D 0 '\000', rx_q= ueue_state =3D "\001", <= span style=3D"color:rgb(206,145,120)">'\000' <repeats 1022<= /span> times>, t= x_queue_state =3D "\001"= , '\000' <repeats 10= 22 times>, dev_flags =3D 75, numa_node =3D = -1, vlan_filter_con= f =3D {ids = =3D {0 <r= epeats 64 times>}}, owner =3D {id =3D 0, name =3D '\000' <= span style=3D"color:rgb(212,212,212)"><repeats 63 times>},
representor_id =3D 0, backer_port_id =3D 32, flow_ops_mutex =3D {__data =3D {__lock =3D= 0, __count = =3D 0, __owner =3D 0, __nusers =3D 0, __kind =3D 0, __spins =3D 0, __elision =3D 0, __list =3D {__pr= ev =3D 0x0, = __next =3D 0x0}}, __size =3D '\000' <repeats 39= times>, __align= =3D 0}}
(gdb)

gd= b output from secondary process for port0 (working port, managed by primary= process)=C2=A0
(gdb) p rte_eth_= devices[0]
$= 1 =3D {rx_pkt_burst= =3D 0x5591d17ddfb2 <mlx5_rx_burst>, tx_pkt_burst =3D 0x5591d197268c <mlx5_tx_burst_full>, tx_pkt_prepare =3D 0x0, rx_queue_count =3D 0x0, rx_descriptor_status =3D 0x5591d17dc903 <m= lx5_rx_descriptor_status>, tx_descriptor_status =3D 0x5591d17f8dd0 <mlx5_tx_descriptor_status>,
=
data =3D 0x6= 00177cf9d00, process_private<= /span> =3D 0x600179e49440, dev_ops =3D 0x5591d20ac180 <mlx5_dev_sec_ops>, device =3D= 0x5591d5a55fa0, intr_handle =3D 0x5591d5= a4dd20, link_intr_cbs = =3D {tqh_first =3D<= /span> 0x0, tqh_last =3D 0x5591d3c53558 <= span style=3D"color:rgb(212,212,212)"><rte_eth_devices+= 88>}, pos= t_rx_burst_cbs =3D {0x0 <repeats 1024 times>},
pre_tx_burst_cbs<= /span> =3D {0x0 <= ;repeats 1024 times>}, state =3D RTE_ETH_DEV_ATT= ACHED, security_ctx =3D 0x0}
(g= db) p rte_eth_devices[0]->data[0]
$2 =3D {name =3D<= /span> "0000:00:07.0", '\000' = <repeats = 51 times>, rx_queues =3D 0x600176c9c980, tx_queues =3D 0x600176c9a900, <= span style=3D"color:rgb(156,220,254)">nb_rx_queues =3D 1, nb_tx_queues =3D 1, sriov =3D {active =3D 0 '\000', nb_q_per_pool =3D= 0 '\000= ', def_vmdq_idx =3D 0, def_pool_q_idx =3D 0}, dev_private =3D 0x600177d29d80, dev_link =3D {
link_speed =3D 40000, link_duplex =3D 1, link_autoneg =3D 1, link_status =3D 1}, dev_co= nf =3D {link_speeds =3D 0, rxmode =3D {mq_mode= =3D RTE_ETH_MQ_RX_NONE, mtu =3D 9000, max_lro_pkt_size =3D<= /span> 0, offloads = =3D 8193, reserved_64s =3D {0, 0}, reserved_ptrs =3D= {0x0, 0x0}}, txmode =3D {mq_mode =3D RTE_ETH_MQ_TX_N= ONE,
offloads= =3D 32815, = pvid =3D 0, hw_vlan_reject_tagged =3D 0 '\000', hw_vlan_reject_untagged =3D 0= '\000', hw_vlan_insert_pvid =3D 0 '\000', reserved_64s<= /span> =3D {0, 0}, reserved_ptrs =3D {0x0, 0x0}}, lpbk_mode =3D 0= , rx_adv_conf =3D {= rss_conf =3D {rss_key =3D 0x0, rss_key_len =3D 0 '\000', rss_hf =3D 0= }, vmdq_dcb_conf =3D {
nb_queue_pools =3D (unknown: 0), enable_default_pool =3D 0 '\000', <= span style=3D"color:rgb(156,220,254)">default_pool =3D 0 '\000'= ;, nb_pool_maps =3D 0 '\000', pool_map= =3D {{vlan_id =3D<= /span> 0, pools =3D= 0} <repe= ats 64 times= >}, dcb_tc =3D "\000\000\000\000\= 000\000\000"}, dcb_rx_conf =3D {nb_t= cs =3D (unknown: 0),
dcb= _tc =3D "\000\000\000\000\000\000\000"}, vmdq_rx_conf= =3D {nb_queue_pools =3D (unknown: 0), enable_default_pool = =3D 0 '\= 000', default_pool =3D 0 '\000', = enable_loop_back =3D 0 '\000', nb_pool_maps= =3D 0 '= \000', rx_mode<= /span> =3D 0, pool_= map =3D {{vlan_id =3D 0, pools =3D 0} <repeats 64 times>}}}, tx_adv_co= nf =3D {
= vmdq_dcb_tx_conf =3D {nb_queue_pools =3D (unknown: 0), dcb= _tc =3D "\000\000\000\000\000\000\000"}, dcb_tx_conf = =3D {nb_tcs =3D (unknown: 0), dc= b_tc =3D "\000\000\000\000\000\000\000"}, vmdq_tx_conf =3D {nb_queue_pools =3D (unknown: 0)}}, dcb_capability_en =3D 0, intr_conf =3D {lsc =3D 1, rxq =3D 0, rmv =3D 0}},
mtu =3D 1500, min_r= x_buf_size =3D 4294967295, rx_mbuf_alloc_failed =3D 0, mac_addrs =3D 0x600177d29da= 0, mac_pool_sel =3D {0 <repeats 128 times>}, hash_mac_addrs =3D 0x0, port_id =3D 0, promiscuous =3D 1 '\001'= , scattered_rx =3D = 0 '\000&= #39;, all_multicast =3D 1 '\001', dev_started= =3D 1 '= \001', l= ro =3D 0 = 9;\000',
dev_configured = =3D 1 '\= 001', flow_configured =3D 0 <= span style=3D"color:rgb(206,145,120)">'\000', rx_queue_state =3D "\001", '\000' <repeats 1022 times>, tx_queue_state =3D "\001", '\000<= span style=3D"color:rgb(206,145,120)">' <repeats 1022 times>= , dev_flags =3D 75, numa_node =3D -1, vlan_filter_conf =3D {ids =3D {0 <repeats 64 tim= es>}}, owner =3D {id =3D 0, name =3D '\000' <repeats 63 times>},=
representor_id <= span style=3D"color:rgb(212,212,212)">=3D 0, backer_port_i= d =3D 32, fl= ow_ops_mutex =3D {__data =3D {__lock =3D 0, __count =3D 0, __owne= r =3D 0, __n= users =3D 0, __kind =3D 0, __spins =3D 0, __elision =3D <= span style=3D"color:rgb(181,206,168)">0, __list =3D= {__prev =3D 0x0, __next =3D 0x= 0}}, __size =3D '\000' <repeats 39 times>, <= span style=3D"color:rgb(156,220,254)">__align =3D 0= }}

<= /font>
gdb output from pri= mary process for port0 (working port, managed by primary process)
(gdb) p rte_eth_devices[0]
$1 =3D {rx_pkt_burst =3D 0x5591d1= 7ddfb2 <mlx5_rx_burst>, tx_pkt_burst =3D 0x5591d197268c <mlx5_t= x_burst_full>, tx_pkt_prep= are =3D 0x0, rx_queue_count =3D 0x5591d17dcd20 <mlx5_rx_queue_count>, rx_descriptor_status =3D 0x5591d17dc= 903 <mlx5_rx_descriptor_sta= tus>,
tx_descr= iptor_status =3D 0x5591d17f8dd0 <mlx5_tx_descriptor_status>, data =3D 0x600177cf9d00, <= span style=3D"color:rgb(156,220,254)">process_private =3D = 0x600176cdc600, dev_ops =3D 0x5591d20abde0 <mlx5_dev_ops>, device= =3D 0x5591d6076800, intr_handle =3D 0x5591d606e580, link_intr_cbs =3D {tqh_first<= /span> =3D 0x0,
tqh_last =3D 0x5591d3c53558 <rte_eth_devices+88>}, post_rx_burst= _cbs =3D {0x600177c69100, 0x0 <repeats 1023 times= >}, pre_tx_burst_cbs =3D {0x600179e52c= c0, 0x0 <= repeats 1023 times>}, state =3D RTE_ETH_DEV_ATTACHED, security_ctx =3D <= span style=3D"color:rgb(181,206,168)">0x0}
(gdb) p rte_eth_devices[0]->data[0]
$2 =3D {name =3D &quo= t;0000:00:07.0", '\000' <= repeats 51 t= imes>, rx_queues =3D 0x600176c9c980, tx_queues =3D 0x600176c9a900, nb_rx_queu= es =3D 1, nb= _tx_queues =3D 1, sriov =3D {active =3D 0 '\000', = nb_q_per_pool =3D 0= '\000&#= 39;, def_vmdq_idx =3D 0, def_pool_q_idx =3D 0}, dev_p= rivate =3D 0x600177d29d80, dev_link =3D {
link_speed =3D 40000, link= _duplex =3D 1, link_autoneg =3D 1, link_status =3D 1}, dev_conf = =3D {link_speeds =3D 0, rxmode =3D {mq_mode =3D RTE_ETH_MQ_RX_NONE, mtu =3D 9000, max_lro_pkt_size =3D 0, offloads =3D 8193, reserved_64s = =3D {0, 0}, = reserved_ptrs =3D {= 0x0, 0x0}}, txmode =3D {mq_mode =3D RTE_ETH_MQ_TX_NONE,
offloads = =3D 32815, pvid =3D 0, hw_vlan_reject_tagged =3D 0 '\000'= , hw_vlan_reject_untagged =3D 0 '\000', hw_vl= an_insert_pvid =3D 0 '\000', reserved_64s =3D= {0, 0}, res= erved_ptrs =3D {0x0, 0x0}}, lpbk_mode =3D 0, rx_adv_c= onf =3D {rss_conf =3D {rss_key <= span style=3D"color:rgb(212,212,212)">=3D 0x0, rss_key_len= =3D 0 '= \000', r= ss_hf =3D 0}, vmdq_dcb_conf =3D {=
nb_queue_pools =3D (unknown: 0= ), enable_default_pool= =3D 0 '= \000', default_= pool =3D 0 '\000', nb_pool_maps =3D= 0 '\000= ', pool_map =3D= {{vlan_id =3D 0, pools =3D 0} <repeats 64 times>}, dcb_tc =3D <= span style=3D"color:rgb(206,145,120)">"\000\000\000\000\000\000\000"}, dcb_rx= _conf =3D {nb_tcs =3D (unknown: 0),
dcb_tc =3D "\000\000\000\000\000\000\000"}, vmdq_rx_conf =3D= {nb_queue_pools =3D (unknown: 0), = enable_default_pool =3D 0 '\000', default_pool =3D 0 '\000', enable_= loop_back =3D 0 '\000', nb_pool_maps =3D 0 '\000', rx_mode = =3D 0, pool_map =3D {{vlan_id =3D 0, pools =3D 0} <repeats 64 times= >}}}, tx_adv_conf =3D {
vmdq_= dcb_tx_conf =3D {nb_queue_pools =3D (unkno= wn: 0), dcb_tc =3D "\000\000\000\000\000\000\000"}, dcb_tx_conf =3D {nb_tcs =3D (unkn= own: 0), dcb_tc =3D "\000\000\000\000\000\000\000"}, vmdq_tx_conf =3D<= /span> {nb_queue_pools =3D (unknown: 0)}},= dcb_capability_en =3D 0, intr_conf =3D {lsc =3D 1, rxq =3D 0, rmv =3D 0}},
mtu =3D<= /span> 1500, min_rx_buf_size =3D 4294967295, rx_mbuf_alloc_failed =3D 0, mac_addrs= =3D 0x600177d29da0, mac_pool_sel =3D {0 <repeats = 128 times>}, hash_mac_addrs =3D 0x0, port_id =3D 0, promiscuous =3D 1 '\001', scatter= ed_rx =3D 0 '\000', all_multicast =3D 1 '\001', dev_started =3D 1 '= \001', lro =3D 0 '\000',
dev_configured =3D 1 '\001', flow_configured =3D 0 '\000', rx_q= ueue_state =3D "\001", <= span style=3D"color:rgb(206,145,120)">'\000' <repeats 1022<= /span> times>, t= x_queue_state =3D "\001"= , '\000' <repeats 10= 22 times>, dev_flags =3D 75, numa_node =3D = -1, vlan_filter_con= f =3D {ids = =3D {0 <r= epeats 64 times>}}, owner =3D {id =3D 0, name =3D '\000' <= span style=3D"color:rgb(212,212,212)"><repeats 63 times>},
representor_id =3D 0, backer_port_id =3D 32, flow_ops_mutex =3D {__data =3D {__lock =3D= 0, __count = =3D 0, __owner =3D 0, __nusers =3D 0, __kind =3D 0, __spins =3D 0, __elision =3D 0, __list =3D {__pr= ev =3D 0x0, = __next =3D 0x0}}, __size =3D '\000' <repeats 39= times>, __align= =3D 0}}


>>=C2=A0Only one process in any moment o= f time manages the queue, no shared (between process) queue data sending is= allowed.
Yes, we are using pr= oc1(secondary process) to manage the port 1 and proc0 (primary process) to = manage port0 . There is no sharing of the queue between processes for sendi= ng data.

>>=C2=A0From your descriptio= n it looks like there is no completions seen (in CQE), I would recommend ch= ecking the UAR/Doorbell mapping in secondary process
=

Below are the logs from mlx5_txq_devx_obj_new whi=
ch is called by proc0(rte_proc_primary) for port 1
Core:0 04/10/24 06:45:47.626580 UTC mlx5_net: Core 0=
 mlx5_txq_devx_obj_new 1563 ppriv: 0x600177c68540=
 ,ppriv->u=
ar_table: 0x600177c68548, txq_ctrl->uar=
_mmap_offset:0, ppriv->uar_table[txq_data->idx<=
/span>]:0x7f0411ae3800, txq_data->idx: =
0, txq_data->db_nc:0
and logs from txq_uar_init_secondary which gets called by =
proc1(rte_proc_secondary) for port 1
Cor=
e:1 04/10/24 06:45:47.767512 =
UTC mlx5_net:   priv<=
/span>: 0x600177c6d7c0, priv->sh: 0x600176c25a00, priv->=
;sh->pppri=
v: 0x600177c68540
Core:1 04/10/24 06:45:47.767528 UTC mlx5_net=
: Core 1 txq_uar_in=
it_secondary 535 txq_ctrl:0x600177c04980 pr=
iv:0x600177c6d7c0
Core:1 04/10/24 06:45:47.767553 UTC mlx5_net=
: Core 1 txq_uar_in=
it_secondary 562   primary_ppriv->uar_t=
able: 0x600177c68548 ,=
uar_va:7f0411ae3800 o=
ffset:800 addr:0x7f0425b18800
Co=
re:1 04/10/24 06:45:47.767604=
 UTC mlx5_net: Core<=
/span> 1 txq_uar_init_secondary 566 port =
1 of txq 0 ppriv:0x600179b1b500<=
/span> ppriv->ua=
r_table[txq->idx]:0x7f0425b18800, txq->idx:0
        , ppriv->uar_table[0]:0x7f0425b18800,ppriv->uar_table[1]:(nil)

@viacheslavo@nvidia.com=C2=A0I noticed that data->dev_private->=
;sh->tx_uar.obj has mmap_offset as 0 in VF case but is non-zero in PF ca=
se, Is that okay? this is returned by glue in using  uar =3D mlx5_glue->=
devx_alloc_uar(cdev->ctx, uar_mapping);
VF:
    (gdb)=
 p *(struct mlx5dv_devx_uar *) $7
    $9 =3D {reg_addr =3D 0x7f041b02d80=
0, base_addr =3D 0x7f041b02d000, page_id =3D 26, mmap_off =3D 0, comp_mask =
=3D 0}
PF:
        t=
xq_ctrl->uar_mmap_offset:9441280, ppriv->uar_table[txq_data-&g=
t;idx]:0x7ff88d2be800, txq_data->idx: 0, txq_data->db_nc:0


Also we are using MLNX_OFED_LINUX-5.8-2.0.3.0-ub=
untu20.04-x86_64 
Please note the MLX PF RSS works fine for us whe=
re we have 4 queues there q0 is managed by primary process(proc0) and q1,q2=
,q3 managed by secondary processes proc1,proc2,proc3 respectively. We are o=
nly seeing the issue in MLX VF.
If it works better for you, we can get on Zoom call for the =
same. Please let us know a convenient time for the same.
T=
hanks,
Samar

This ele= ctronic communication and the information and any files transmitted with it= , or attached to it, are confidential and are intended solely for the use o= f the individual or entity to whom it is addressed and may contain informat= ion that is confidential, legally privileged, protected by privacy laws, or= otherwise restricted from disclosure to anyone else. If you are not the in= tended recipient or the person responsible for delivering the e-mail to the= intended recipient, you are hereby notified that any use, copying, distrib= uting, dissemination, forwarding, printing, or copying of this e-mail is st= rictly prohibited. If you received this e-mail in error, please return the = e-mail to the sender, delete it from your computer, and destroy any printed= copy of it. --00000000000065ce0e0615b9607b--