From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4841FA0C4E; Tue, 19 Oct 2021 08:28:37 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0B34640142; Tue, 19 Oct 2021 08:28:37 +0200 (CEST) Received: from shelob.oktetlabs.ru (shelob.oktetlabs.ru [91.220.146.113]) by mails.dpdk.org (Postfix) with ESMTP id CE9234003E for ; Tue, 19 Oct 2021 08:28:35 +0200 (CEST) Received: from [192.168.38.17] (aros.oktetlabs.ru [192.168.38.17]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by shelob.oktetlabs.ru (Postfix) with ESMTPSA id 23F1B7F530; Tue, 19 Oct 2021 09:28:35 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 shelob.oktetlabs.ru 23F1B7F530 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=oktetlabs.ru; s=default; t=1634624915; bh=NUiFP68DKUsfGXkMZKv4Zag/us2+XoXcUmjT7zmhPSQ=; h=Subject:To:Cc:References:From:Date:In-Reply-To; b=mJBX7aETmBrTXZqxMiV/kiWT4TcXfTsKzRqNC+r9fcE8zsNlnDghkrKl1dR0ZCmUs bseF3r0l4WxlU9FjxVjwMkNtj1HYs+mYlHopQcwdannwoeJ/0lAiYhtc9YX6AU3x13 pOCsbzp/lOIpbRYP/Qinz81Jcb+rJmblIJavHA9Y= To: Xueming Li , dev@dpdk.org Cc: Jerin Jacob , Ferruh Yigit , Viacheslav Ovsiienko , Thomas Monjalon , Lior Margalit , Ananyev Konstantin References: <20210727034204.20649-1-xuemingl@nvidia.com> <20211018125942.2067353-1-xuemingl@nvidia.com> <20211018125942.2067353-2-xuemingl@nvidia.com> From: Andrew Rybchenko Organization: OKTET Labs Message-ID: <2f4fbfcf-7ed0-7d18-563a-13bc560a3968@oktetlabs.ru> Date: Tue, 19 Oct 2021 09:28:34 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 MIME-Version: 1.0 In-Reply-To: <20211018125942.2067353-2-xuemingl@nvidia.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH v8 1/6] ethdev: introduce shared Rx queue X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 10/18/21 3:59 PM, Xueming Li wrote: > In current DPDK framework, each Rx queue is pre-loaded with mbufs to > save incoming packets. For some PMDs, when number of representors scale > out in a switch domain, the memory consumption became significant. > Polling all ports also leads to high cache miss, high latency and low > throughput. > > This patch introduce shared Rx queue. Ports in same Rx domain and > switch domain could share Rx queue set by specifying non-zero sharing > group in Rx queue configuration. > > Shared Rx queue is identified by share_rxq field of Rx queue > configuration. Port A RxQ X can share RxQ with Port B RxQ Y by using > same shared Rx queue ID. > > No special API is defined to receive packets from shared Rx queue. > Polling any member port of a shared Rx queue receives packets of that > queue for all member ports, port_id is identified by mbuf->port. PMD is > responsible to resolve shared Rx queue from device and queue data. > > Shared Rx queue must be polled in same thread or core, polling a queue > ID of any member port is essentially same. > > Multiple share groups are supported. Device should support mixed > configuration by allowing multiple share groups and non-shared Rx queue > on one port. > > Example grouping and polling model to reflect service priority: > Group1, 2 shared Rx queues per port: PF, rep0, rep1 > Group2, 1 shared Rx queue per port: rep2, rep3, ... rep127 > Core0: poll PF queue0 > Core1: poll PF queue1 > Core2: poll rep2 queue0 > > PMD advertise shared Rx queue capability via RTE_ETH_DEV_CAPA_RXQ_SHARE. > > PMD is responsible for shared Rx queue consistency checks to avoid > member port's configuration contradict to each other. > > Signed-off-by: Xueming Li with few nits below: Reviewed-by: Andrew Rybchenko [snip] > diff --git a/doc/guides/prog_guide/switch_representation.rst b/doc/guides/prog_guide/switch_representation.rst > index ff6aa91c806..fe89a7f5c33 100644 > --- a/doc/guides/prog_guide/switch_representation.rst > +++ b/doc/guides/prog_guide/switch_representation.rst > @@ -123,6 +123,17 @@ thought as a software "patch panel" front-end for applications. > .. [1] `Ethernet switch device driver model (switchdev) > `_ > > +- For some PMDs, memory usage of representors is huge when number of > + representor grows, mbufs are allocated for each descriptor of Rx queue. > + Polling large number of ports brings more CPU load, cache miss and > + latency. Shared Rx queue can be used to share Rx queue between PF and > + representors among same Rx domain. ``RTE_ETH_DEV_CAPA_RXQ_SHARE`` in > + device info is used to indicate the capability. Setting non-zero share > + group in Rx queue configuration to enable share, share_qid is used to > + identifiy the shared Rx queue in group. Polling any member port can identifiy -> identify > + receive packets of all member ports in the group, port ID is saved in > + ``mbuf.port``. > + > Basic SR-IOV > ------------ > > diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst > index d5435a64aa1..2143e38ff11 100644 > --- a/doc/guides/rel_notes/release_21_11.rst > +++ b/doc/guides/rel_notes/release_21_11.rst > @@ -75,6 +75,12 @@ New Features > operations. > * Added multi-process support. > > +* **Added ethdev shared Rx queue support.** > + > + * Added new device capability flag and rx domain field to switch info. rx -> Rx > + * Added share group and share queue ID to Rx queue configuration. > + * Added testpmd support and dedicate forwarding engine. > + > * **Added new RSS offload types for IPv4/L4 checksum in RSS flow.** > > Added macros ETH_RSS_IPV4_CHKSUM and ETH_RSS_L4_CHKSUM, now IPv4 and [snip]