From: Olivier Gournet <ogournet@corp.free.fr>
To: dev@dpdk.org
Subject: [dpdk-dev] mlx5 and secondary processes
Date: Tue, 14 Nov 2017 10:58:02 +0100 [thread overview]
Message-ID: <2377aa87-fc6c-ee1e-91b8-ae226cb79a87@corp.free.fr> (raw)
Hi,
I can't get TX on secondary process to works with the lastest dpdk, it was running fine with dpdk-16.11.
RX/TX is ok on primary processs, and I'm not interested in RX on secondary process. Each process has its
owns TX queues.
I upgraded everything to:
linux 4.14
dpdk 17.11-rc4
rdma-core from yesterday git-master
mlx-fw 12.21.1000
On seconday process, TX queue gets full and is never emptied.
It seems like bf_reg is correctly re-mapped from secondary process; here's dmesg from primary process:
[...]
[85361.895778] mlx5_1:uar_mmap:1722:(pid 9533): uar idx 0x6, pfn 0x00000000000f8020
[85361.895781] mlx5_1:uar_mmap:1735:(pid 9533): mapped best effort WC at 0x7ffff7faa000, PA 0x00000000f8020000
[85361.895784] mlx5_1:uar_mmap:1722:(pid 9533): uar idx 0x7, pfn 0x00000000000f8021
[85361.895787] mlx5_1:uar_mmap:1735:(pid 9533): mapped best effort WC at 0x7ffff7fa9000, PA 0x00000000f8021000
Then from secondary process:
[...]
[85408.038295] mlx5_1:mlx5_ib_mmap:1778:(pid 9551): mapped internal timer at 0x7ffff7f7f000, PA 0xf8001000
[85408.040229] mlx5_1:uar_mmap:1722:(pid 9551): uar idx 0x6, pfn 0x00000000000f8020
[85408.040233] mlx5_1:uar_mmap:1735:(pid 9551): mapped best effort WC at 0x7ffff7faa000, PA 0x00000000f8020000
[85408.040239] mlx5_1:uar_mmap:1722:(pid 9551): uar idx 0x7, pfn 0x00000000000f8021
[85408.040241] mlx5_1:uar_mmap:1735:(pid 9551): mapped best effort WC at 0x7ffff7fa9000, PA 0x00000000f8021000
But then from mlx5_rxtx.h:mlx5_tx_dbrec_cond_wmb() it doesn't seems like writing to txq->bf_reg has any effect.
I know it isn't supposed to be read, but when reading from primary process is always give *dst==0xe5ccdabae5ccdaba,
whereas from secondary process *dst is zero or the last written value.
I don't really have any other clues, and don't know where to search. Can anybody give me some hints ?
Regards,
=============
# ibv_devinfo -d mlx5_1
hca_id: mlx5_1
transport: InfiniBand (0)
fw_ver: 12.21.1000
node_guid: 248a:0703:008a:638e
sys_image_guid: 248a:0703:008a:638e
vendor_id: 0x02c9
vendor_part_id: 4115
hw_ver: 0x0
board_id: MT_2140110033
phys_port_cnt: 1
Device ports:
port: 1
state: PORT_ACTIVE (4)
max_mtu: 4096 (5)
active_mtu: 1024 (3)
sm_lid: 0
port_lid: 0
port_lmc: 0x00
link_layer: Ethernet
next reply other threads:[~2017-11-14 9:57 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-11-14 9:58 Olivier Gournet [this message]
2017-11-14 10:34 ` Nelio Laranjeiro
2017-11-14 13:47 ` Olivier Gournet
2018-02-01 7:28 ` Xueming(Steven) Li
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2377aa87-fc6c-ee1e-91b8-ae226cb79a87@corp.free.fr \
--to=ogournet@corp.free.fr \
--cc=dev@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).