From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C4F0742A0F; Thu, 27 Apr 2023 10:35:21 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B94BE42D47; Thu, 27 Apr 2023 10:35:21 +0200 (CEST) Received: from loongson.cn (mail.loongson.cn [114.242.206.163]) by mails.dpdk.org (Postfix) with ESMTP id 7F70842B8C for ; Thu, 27 Apr 2023 10:35:19 +0200 (CEST) Received: from loongson.cn (unknown [10.20.42.90]) by gateway (Coremail) with SMTP id _____8AxNPDDM0pkOYEBAA--.2550S3; Thu, 27 Apr 2023 16:35:15 +0800 (CST) Received: from [10.20.42.90] (unknown [10.20.42.90]) by localhost.localdomain (Coremail) with SMTP id AQAAf8DxfbPBM0pkwWU+AA--.19061S3; Thu, 27 Apr 2023 16:35:13 +0800 (CST) Subject: Re: [PATCH] net/ixgbe: consider DCB/VMDq conf when getting RSS conf To: qiming.yang@intel.com, wenjun1.wu@intel.com Cc: dev@dpdk.org, maobibo@loongson.cn References: <20230412100144.1713426-1-zhoumin@loongson.cn> From: zhoumin Message-ID: <392196bb-1682-fccf-80c3-6c6a657bb9dd@loongson.cn> Date: Thu, 27 Apr 2023 16:35:13 +0800 User-Agent: Mozilla/5.0 (X11; Linux loongarch64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: <20230412100144.1713426-1-zhoumin@loongson.cn> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-CM-TRANSID: AQAAf8DxfbPBM0pkwWU+AA--.19061S3 X-CM-SenderInfo: 52kr3ztlq6z05rqj20fqof0/ X-Coremail-Antispam: 1Uk129KBjvJXoW3Gr45Xry5XFy5Cr47ur4Durg_yoW7Kw4fpr 4DWF1SyF1UWFs29wn3X3y3urnIkr4Fqr4UX3yxKw15Z398C3y0qF1kKw18AF18JryDuwsx ZFWfWrs7KF1kZFDanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUj1kv1TuYvTs0mT0YCTnIWj qI5I8CrVACY4xI64kE6c02F40Ex7xfYxn0WfASr-VFAUDa7-sFnT9fnUUIcSsGvfJTRUUU bxkYFVCjjxCrM7AC8VAFwI0_Jr0_Gr1l1xkIjI8I6I8E6xAIw20EY4v20xvaj40_Wr0E3s 1l1IIY67AEw4v_Jr0_Jr4l8cAvFVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xv wVC0I7IYx2IY67AKxVWUCVW8JwA2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVWUJVW8JwA2z4 x0Y4vEx4A2jsIE14v26F4UJVW0owA2z4x0Y4vEx4A2jsIEc7CjxVAFwI0_Cr1j6rxdM2AI xVAIcxkEcVAq07x20xvEncxIr21l57IF6xkI12xvs2x26I8E6xACxx1l5I8CrVACY4xI64 kE6c02F40Ex7xfMcIj6xIIjxv20xvE14v26r1Y6r17McIj6I8E87Iv67AKxVWUJVW8JwAm 72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IY64vIr41lc7I2V7IY0VAS07AlzVAYIcxG8wCF04 k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v26r1j6r18 MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_JF0_Jw1lIxkGc2Ij64vIr4 1lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAFwI0_Jr0_Gr1l IxAIcVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r4UMIIF0xvEx4 A2jsIEc7CjxVAFwI0_Jr0_GrUvcSsGvfC2KfnxnUUI43ZEXa7IU1QVy3UUUUU== X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Kindly ping. Any comments or suggestions will be appreciated. Best regards Min On Wed, Apr 12, 2023 at 6:01PM, Min Zhou wrote: > The mrqe field of MRQC register is an enum. From the Intel 82599 datasheet, > we know that these values below for the mrqe field are all related to RSS > configuration: > 0000b = RSS disabled. > 0001b = RSS only -- Single set of RSS 16 queues. > 0010b = DCB enabled and RSS disabled -- 8 TCs, each allocated 1 queue. > 0011b = DCB enabled and RSS disabled -- 4 TCs, each allocated 1 queue. > 0100b = DCB and RSS -- 8 TCs, each allocated 16 RSS queues. > 0101b = DCB and RSS -- 4 TCs, each allocated 16 RSS queues. > 1000b = Virtualization only -- 64 pools, no RSS, each pool allocated > 2 queues. > 1010b = Virtualization and RSS -- 32 pools, each allocated 4 RSS queues. > 1011b = Virtualization and RSS -- 64 pools, each allocated 2 RSS queues. > > The ixgbe pmd will check whether the rss is enabled or not when getting > rss conf. So, beside comparing the value of mrqe field with xxx0b and > xxx1b, we also needto consider the other configurations, such as > DCB + RSS or VMDQ + RSS. Otherwise, we may not get the correct rss conf > in some cases, such as when we use DCB and RSS with 8 TCs which corresponds > to 0100b for the mrqe field. > > Signed-off-by: Min Zhou > --- > drivers/net/ixgbe/ixgbe_rxtx.c | 91 ++++++++++++++++++++++++++++++---- > 1 file changed, 80 insertions(+), 11 deletions(-) > > diff --git a/drivers/net/ixgbe/ixgbe_rxtx.c b/drivers/net/ixgbe/ixgbe_rxtx.c > index c9d6ca9efe..1eff0053ed 100644 > --- a/drivers/net/ixgbe/ixgbe_rxtx.c > +++ b/drivers/net/ixgbe/ixgbe_rxtx.c > @@ -3461,18 +3461,89 @@ static uint8_t rss_intel_key[40] = { > 0x6A, 0x42, 0xB7, 0x3B, 0xBE, 0xAC, 0x01, 0xFA, > }; > > +/* > + * This function removes the rss configuration in the mrqe field of MRQC > + * register and tries to maintain other configurations in the field, such > + * DCB and Virtualization. > + * > + * The MRQC register supplied in section 7.1.2.8.3 of the Intel 82599 datasheet. > + * From the datasheet, we know that the mrqe field is an enum. So, masking the > + * mrqe field with '~IXGBE_MRQC_RSSEN' may not completely disable rss > + * configuration. For example, the value of mrqe is equal to 0101b when DCB and > + * RSS with 4 TCs configured, however 'mrqe &= ~0x01' is equal to 0100b which > + * corresponds to DCB and RSS with 8 TCs. > + */ > +static void > +ixgbe_mrqc_rss_remove(struct ixgbe_hw *hw) > +{ > + uint32_t mrqc; > + uint32_t mrqc_reg; > + uint32_t mrqe_val; > + > + mrqc_reg = ixgbe_mrqc_reg_get(hw->mac.type); > + mrqc = IXGBE_READ_REG(hw, mrqc_reg); > + mrqe_val = mrqc & IXGBE_MRQC_MRQE_MASK; > + > + switch (mrqe_val) { > + case IXGBE_MRQC_RSSEN: > + /* Completely disable rss */ > + mrqe_val = 0; > + break; > + case IXGBE_MRQC_RTRSS8TCEN: > + mrqe_val = IXGBE_MRQC_RT8TCEN; > + break; > + case IXGBE_MRQC_RTRSS4TCEN: > + mrqe_val = IXGBE_MRQC_RT4TCEN; > + break; > + case IXGBE_MRQC_VMDQRSS64EN: > + /* FIXME. Can 32 pools with rss convert to 64 pools without rss? */ > + case IXGBE_MRQC_VMDQRSS32EN: > + mrqe_val = IXGBE_MRQC_VMDQEN; > + break; > + default: > + /* No rss configured, leave it as it is */ > + break; > + } > + mrqc = (mrqc & ~IXGBE_MRQC_MRQE_MASK) | mrqe_val; > + IXGBE_WRITE_REG(hw, mrqc_reg, mrqc); > +} > + > static void > ixgbe_rss_disable(struct rte_eth_dev *dev) > { > struct ixgbe_hw *hw; > + > + hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private); > + /* Remove the rss configuration and maintain the other configurations */ > + ixgbe_mrqc_rss_remove(hw); > +} > + > +/* > + * This function checks whether the rss is enabled or not by comparing the mrqe > + * field with some RSS related enums and also considers the configurations for > + * DCB + RSS and Virtualization + RSS. It is necessary for getting the correct > + * rss hash configurations from the RSS Field Enable field of MRQC register > + * when both RSS and DCB/VMDQ are used. > + */ > +static bool > +ixgbe_rss_enabled(struct ixgbe_hw *hw) > +{ > uint32_t mrqc; > uint32_t mrqc_reg; > + uint32_t mrqe_val; > > - hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private); > mrqc_reg = ixgbe_mrqc_reg_get(hw->mac.type); > mrqc = IXGBE_READ_REG(hw, mrqc_reg); > - mrqc &= ~IXGBE_MRQC_RSSEN; > - IXGBE_WRITE_REG(hw, mrqc_reg, mrqc); > + mrqe_val = mrqc & IXGBE_MRQC_MRQE_MASK; > + > + if (mrqe_val == IXGBE_MRQC_RSSEN || > + mrqe_val == IXGBE_MRQC_RTRSS8TCEN || > + mrqe_val == IXGBE_MRQC_RTRSS4TCEN || > + mrqe_val == IXGBE_MRQC_VMDQRSS64EN || > + mrqe_val == IXGBE_MRQC_VMDQRSS32EN) > + return true; > + > + return false; > } > > static void > @@ -3530,9 +3601,7 @@ ixgbe_dev_rss_hash_update(struct rte_eth_dev *dev, > struct rte_eth_rss_conf *rss_conf) > { > struct ixgbe_hw *hw; > - uint32_t mrqc; > uint64_t rss_hf; > - uint32_t mrqc_reg; > > hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private); > > @@ -3541,7 +3610,6 @@ ixgbe_dev_rss_hash_update(struct rte_eth_dev *dev, > "NIC."); > return -ENOTSUP; > } > - mrqc_reg = ixgbe_mrqc_reg_get(hw->mac.type); > > /* > * Excerpt from section 7.1.2.8 Receive-Side Scaling (RSS): > @@ -3553,8 +3621,7 @@ ixgbe_dev_rss_hash_update(struct rte_eth_dev *dev, > * disabled at initialization time. > */ > rss_hf = rss_conf->rss_hf & IXGBE_RSS_OFFLOAD_ALL; > - mrqc = IXGBE_READ_REG(hw, mrqc_reg); > - if (!(mrqc & IXGBE_MRQC_RSSEN)) { /* RSS disabled */ > + if (!ixgbe_rss_enabled(hw)) { /* RSS disabled */ > if (rss_hf != 0) /* Enable RSS */ > return -(EINVAL); > return 0; /* Nothing to do */ > @@ -3594,12 +3661,14 @@ ixgbe_dev_rss_hash_conf_get(struct rte_eth_dev *dev, > } > } > > - /* Get RSS functions configured in MRQC register */ > - mrqc = IXGBE_READ_REG(hw, mrqc_reg); > - if ((mrqc & IXGBE_MRQC_RSSEN) == 0) { /* RSS is disabled */ > + if (!ixgbe_rss_enabled(hw)) { /* RSS is disabled */ > rss_conf->rss_hf = 0; > return 0; > } > + > + /* Get RSS functions configured in MRQC register */ > + mrqc = IXGBE_READ_REG(hw, mrqc_reg); > + > rss_hf = 0; > if (mrqc & IXGBE_MRQC_RSS_FIELD_IPV4) > rss_hf |= RTE_ETH_RSS_IPV4;