From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4B60CA0C57; Thu, 4 Nov 2021 18:50:56 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2EFFB41223; Thu, 4 Nov 2021 18:50:56 +0100 (CET) Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by mails.dpdk.org (Postfix) with ESMTP id 6AD20411C9 for ; Thu, 4 Nov 2021 18:50:55 +0100 (CET) Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1A4Hfdxt029169; Thu, 4 Nov 2021 17:50:52 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=message-id : date : mime-version : subject : to : cc : references : from : in-reply-to : content-type : content-transfer-encoding; s=pp1; bh=K5uCn0renMr1lbPTyjPTZtZU+vkdZarXhvAQH9iSkgk=; b=GZr/yFF6xO+m9T/Cyc7XQ8SZUCV7eAseIE6CMeTYJ2ljhiZyf1xU1Nm+YyX/3k0vPKz/ aY74K5YClPml2hTXuwVUh34KN7MtSpMn8oFi83KSzA5qHRmdesrTLymnEX/PSi0SFAn/ il9UEjAlx1zzsPIIr+wcGU82TV3ugEbho9eOGuKG3nRYVneaeG8/rN9MwKTbJpg5rqYF AVG2iBO1sDsaEvpGD5tlJcv7/WjTJKrqQw4Qei2yFaSQ24+HfshtpVhj3OvdwG3H/cAJ as+ciQBadOi5qioMtx1v0zXmYnEkbDOZ4L2GepX89/ITsGd8OzogvxDFVe/boQewnyua qg== Received: from ppma01wdc.us.ibm.com (fd.55.37a9.ip4.static.sl-reverse.com [169.55.85.253]) by mx0b-001b2d01.pphosted.com with ESMTP id 3c4jfk2g6n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 04 Nov 2021 17:50:52 +0000 Received: from pps.filterd (ppma01wdc.us.ibm.com [127.0.0.1]) by ppma01wdc.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 1A4HlIDM009675; Thu, 4 Nov 2021 17:50:51 GMT Received: from b03cxnp08028.gho.boulder.ibm.com (b03cxnp08028.gho.boulder.ibm.com [9.17.130.20]) by ppma01wdc.us.ibm.com with ESMTP id 3c0wpc6pmg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 04 Nov 2021 17:50:51 +0000 Received: from b03ledav006.gho.boulder.ibm.com (b03ledav006.gho.boulder.ibm.com [9.17.130.237]) by b03cxnp08028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 1A4HoouJ34799908 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 4 Nov 2021 17:50:50 GMT Received: from b03ledav006.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B57FEC6059; Thu, 4 Nov 2021 17:50:50 +0000 (GMT) Received: from b03ledav006.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5887BC605B; Thu, 4 Nov 2021 17:50:50 +0000 (GMT) Received: from [9.211.89.93] (unknown [9.211.89.93]) by b03ledav006.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 4 Nov 2021 17:50:50 +0000 (GMT) Message-ID: <56eac959-bcc1-18fe-f41e-3152cc224c98@linux.vnet.ibm.com> Date: Thu, 4 Nov 2021 10:50:49 -0700 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0) Gecko/20100101 Thunderbird/91.2.1 Content-Language: en-US To: Xueming Li , dev@dpdk.org Cc: Viacheslav Ovsiienko , Lior Margalit , Matan Azrad , Ruifeng Wang , Bruce Richardson , Konstantin Ananyev References: <20210727034204.20649-1-xuemingl@nvidia.com> <20211104123320.1638915-1-xuemingl@nvidia.com> <20211104123320.1638915-15-xuemingl@nvidia.com> From: David Christensen In-Reply-To: <20211104123320.1638915-15-xuemingl@nvidia.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-TM-AS-GCONF: 00 X-Proofpoint-GUID: mVFcT5357cLTfj8JUscm_hWk7aGvq0DW X-Proofpoint-ORIG-GUID: mVFcT5357cLTfj8JUscm_hWk7aGvq0DW X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-11-04_05,2021-11-03_01,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1011 priorityscore=1501 impostorscore=0 mlxscore=0 malwarescore=0 spamscore=0 lowpriorityscore=0 bulkscore=0 adultscore=0 suspectscore=0 phishscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000 definitions=main-2111040067 Subject: Re: [dpdk-dev] [PATCH v4 14/14] net/mlx5: add shared Rx queue port datapath support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 11/4/21 5:33 AM, Xueming Li wrote: > From: Viacheslav Ovsiienko > > When receive packet, mlx5 PMD saves mbuf port number from > RxQ data. > > To support shared RxQ, save port number into RQ context as user index. > Received packet resolve port number from CQE user index which derived > from RQ context. > > Legacy Verbs API doesn't support RQ user index setting, still read from > RxQ port number. > > Signed-off-by: Xueming Li > Signed-off-by: Viacheslav Ovsiienko > Acked-by: Slava Ovsiienko > --- > drivers/net/mlx5/mlx5_devx.c | 1 + > drivers/net/mlx5/mlx5_rx.c | 1 + > drivers/net/mlx5/mlx5_rxq.c | 3 ++- > drivers/net/mlx5/mlx5_rxtx_vec_altivec.h | 6 ++++++ > drivers/net/mlx5/mlx5_rxtx_vec_neon.h | 12 +++++++++++- > drivers/net/mlx5/mlx5_rxtx_vec_sse.h | 8 +++++++- > 6 files changed, 28 insertions(+), 3 deletions(-) > > diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c > index d3d189ab7f2..a9f9f4af700 100644 > --- a/drivers/net/mlx5/mlx5_devx.c > +++ b/drivers/net/mlx5/mlx5_devx.c > @@ -277,6 +277,7 @@ mlx5_rxq_create_devx_rq_resources(struct mlx5_rxq_priv *rxq) > MLX5_WQ_END_PAD_MODE_NONE; > rq_attr.wq_attr.pd = cdev->pdn; > rq_attr.counter_set_id = priv->counter_set_id; > + rq_attr.user_index = rte_cpu_to_be_16(priv->dev_data->port_id); > if (rxq_data->shared) /* Create RMP based RQ. */ > rxq->devx_rq.rmp = &rxq_ctrl->obj->devx_rmp; > /* Create RQ using DevX API. */ > diff --git a/drivers/net/mlx5/mlx5_rx.c b/drivers/net/mlx5/mlx5_rx.c > index 1ffa1b95b88..4d85f64accd 100644 > --- a/drivers/net/mlx5/mlx5_rx.c > +++ b/drivers/net/mlx5/mlx5_rx.c > @@ -709,6 +709,7 @@ rxq_cq_to_mbuf(struct mlx5_rxq_data *rxq, struct rte_mbuf *pkt, > { > /* Update packet information. */ > pkt->packet_type = rxq_cq_to_pkt_type(rxq, cqe, mcqe); > + pkt->port = unlikely(rxq->shared) ? cqe->user_index_low : rxq->port_id; > > if (rxq->rss_hash) { > uint32_t rss_hash_res = 0; > diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c > index 8feb3e2c0fb..4515d531835 100644 > --- a/drivers/net/mlx5/mlx5_rxq.c > +++ b/drivers/net/mlx5/mlx5_rxq.c > @@ -186,7 +186,8 @@ rxq_alloc_elts_sprq(struct mlx5_rxq_ctrl *rxq_ctrl) > mbuf_init->data_off = RTE_PKTMBUF_HEADROOM; > rte_mbuf_refcnt_set(mbuf_init, 1); > mbuf_init->nb_segs = 1; > - mbuf_init->port = rxq->port_id; > + /* For shared queues port is provided in CQE */ > + mbuf_init->port = rxq->shared ? 0 : rxq->port_id; > if (priv->flags & RTE_PKTMBUF_POOL_F_PINNED_EXT_BUF) > mbuf_init->ol_flags = RTE_MBUF_F_EXTERNAL; > /* > diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_altivec.h b/drivers/net/mlx5/mlx5_rxtx_vec_altivec.h > index 1d00c1c43d1..423e229508c 100644 > --- a/drivers/net/mlx5/mlx5_rxtx_vec_altivec.h > +++ b/drivers/net/mlx5/mlx5_rxtx_vec_altivec.h > @@ -1189,6 +1189,12 @@ rxq_cq_process_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq, > > /* D.5 fill in mbuf - rearm_data and packet_type. */ > rxq_cq_to_ptype_oflags_v(rxq, cqes, opcode, &pkts[pos]); > + if (unlikely(rxq->shared)) { > + pkts[pos]->port = cq[pos].user_index_low; > + pkts[pos + p1]->port = cq[pos + p1].user_index_low; > + pkts[pos + p2]->port = cq[pos + p2].user_index_low; > + pkts[pos + p3]->port = cq[pos + p3].user_index_low; > + } > if (rxq->hw_timestamp) { > int offset = rxq->timestamp_offset; > if (rxq->rt_timestamp) { > diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_neon.h b/drivers/net/mlx5/mlx5_rxtx_vec_neon.h > index aa36df29a09..b1d16baa619 100644 > --- a/drivers/net/mlx5/mlx5_rxtx_vec_neon.h > +++ b/drivers/net/mlx5/mlx5_rxtx_vec_neon.h > @@ -787,7 +787,17 @@ rxq_cq_process_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq, > /* C.4 fill in mbuf - rearm_data and packet_type. */ > rxq_cq_to_ptype_oflags_v(rxq, ptype_info, flow_tag, > opcode, &elts[pos]); > - if (rxq->hw_timestamp) { > + if (unlikely(rxq->shared)) { > + elts[pos]->port = container_of(p0, struct mlx5_cqe, > + pkt_info)->user_index_low; > + elts[pos + 1]->port = container_of(p1, struct mlx5_cqe, > + pkt_info)->user_index_low; > + elts[pos + 2]->port = container_of(p2, struct mlx5_cqe, > + pkt_info)->user_index_low; > + elts[pos + 3]->port = container_of(p3, struct mlx5_cqe, > + pkt_info)->user_index_low; > + } > + if (unlikely(rxq->hw_timestamp)) { > int offset = rxq->timestamp_offset; > if (rxq->rt_timestamp) { > struct mlx5_dev_ctx_shared *sh = rxq->sh; > diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_sse.h b/drivers/net/mlx5/mlx5_rxtx_vec_sse.h > index b0fc29d7b9e..f3d838389e2 100644 > --- a/drivers/net/mlx5/mlx5_rxtx_vec_sse.h > +++ b/drivers/net/mlx5/mlx5_rxtx_vec_sse.h > @@ -736,7 +736,13 @@ rxq_cq_process_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq, > *err |= _mm_cvtsi128_si64(opcode); > /* D.5 fill in mbuf - rearm_data and packet_type. */ > rxq_cq_to_ptype_oflags_v(rxq, cqes, opcode, &pkts[pos]); > - if (rxq->hw_timestamp) { > + if (unlikely(rxq->shared)) { > + pkts[pos]->port = cq[pos].user_index_low; > + pkts[pos + p1]->port = cq[pos + p1].user_index_low; > + pkts[pos + p2]->port = cq[pos + p2].user_index_low; > + pkts[pos + p3]->port = cq[pos + p3].user_index_low; > + } > + if (unlikely(rxq->hw_timestamp)) { > int offset = rxq->timestamp_offset; > if (rxq->rt_timestamp) { > struct mlx5_dev_ctx_shared *sh = rxq->sh; > Reviewed-by: David Christensen