From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wi0-f180.google.com (mail-wi0-f180.google.com [209.85.212.180]) by dpdk.org (Postfix) with ESMTP id 60B0919F5 for ; Thu, 8 Jan 2015 10:43:18 +0100 (CET) Received: by mail-wi0-f180.google.com with SMTP id n3so2061603wiv.1 for ; Thu, 08 Jan 2015 01:43:18 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=ztyXb3WLQMowUBLROa0UitdWt2QRvWLKC0glaoWwAxA=; b=c/MHJAvFoELNN4tCYK5hmJF+XxSP81sRZs4M6MQzR+/D02dIuQVGEsyi+pV8A7TD4i Hj9Vy0vtrz51ormzjZC4cVuT64sPk2bFv1VNQLtUxWUaR+pJnBCDNbXU+Q0JTLOGwvDn ashwtx8skjC2WqrHZZsCpEOE+G1YWG6durJ9afa3EmikHC+wcCfbn0OTYz5c7xWEzxgN Jy+cth73kjH1m/AOJkLF3fGfSFvcOSx8WjV9bPThFDME4e4uMTz0kpNLBnbYCZNMxsa8 voQTwStlv4eenKZBC1JDRxsdxxjIxrwnZsWVOtMKPDm935ypYLj6z2ht+qlTzFdTm/iG QXsg== X-Gm-Message-State: ALoCoQk1HHbiwz7nXDfGycxIchUaXLByU5uysop9JpeTxwPkC2ReJiY6mJQf1nzKwivDjJ/dwgQz X-Received: by 10.194.78.229 with SMTP id e5mr16852422wjx.11.1420710198149; Thu, 08 Jan 2015 01:43:18 -0800 (PST) Received: from [10.0.0.4] ([109.66.137.113]) by mx.google.com with ESMTPSA id kn7sm5376055wjc.45.2015.01.08.01.43.17 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 08 Jan 2015 01:43:17 -0800 (PST) Message-ID: <54AE5134.3070808@cloudius-systems.com> Date: Thu, 08 Jan 2015 11:43:16 +0200 From: Vlad Zolotarov User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: Ouyang Changchun , dev@dpdk.org References: <1420355937-18484-1-git-send-email-changchun.ouyang@intel.com> <1420612355-6666-1-git-send-email-changchun.ouyang@intel.com> <1420612355-6666-6-git-send-email-changchun.ouyang@intel.com> In-Reply-To: <1420612355-6666-6-git-send-email-changchun.ouyang@intel.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH v5 5/6] ixgbe: Config VF RSS X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jan 2015 09:43:18 -0000 On 01/07/15 08:32, Ouyang Changchun wrote: > It needs config RSS and IXGBE_MRQC and IXGBE_VFPSRTYPE to enable VF RSS. > > The psrtype will determine how many queues the received packets will distribute to, > and the value of psrtype should depends on both facet: max VF rxq number which > has been negotiated with PF, and the number of rxq specified in config on guest. > > Signed-off-by: Changchun Ouyang > > Changes in v4: > - the number of rxq from config should be power of 2 and should not bigger than > max VF rxq number(negotiated between guest and host). > > --- > lib/librte_pmd_ixgbe/ixgbe_pf.c | 15 ++++++ > lib/librte_pmd_ixgbe/ixgbe_rxtx.c | 103 +++++++++++++++++++++++++++++++++----- > 2 files changed, 106 insertions(+), 12 deletions(-) > > diff --git a/lib/librte_pmd_ixgbe/ixgbe_pf.c b/lib/librte_pmd_ixgbe/ixgbe_pf.c > index dbda9b5..93f6e43 100644 > --- a/lib/librte_pmd_ixgbe/ixgbe_pf.c > +++ b/lib/librte_pmd_ixgbe/ixgbe_pf.c > @@ -187,6 +187,21 @@ int ixgbe_pf_host_configure(struct rte_eth_dev *eth_dev) > IXGBE_WRITE_REG(hw, IXGBE_MPSAR_LO(hw->mac.num_rar_entries), 0); > IXGBE_WRITE_REG(hw, IXGBE_MPSAR_HI(hw->mac.num_rar_entries), 0); > > + /* > + * VF RSS can support at most 4 queues for each VF, even if > + * 8 queues are available for each VF, it need refine to 4 > + * queues here due to this limitation, otherwise no queue > + * will receive any packet even RSS is enabled. > + */ > + if (eth_dev->data->dev_conf.rxmode.mq_mode == ETH_MQ_RX_VMDQ_RSS) { > + if (RTE_ETH_DEV_SRIOV(eth_dev).nb_q_per_pool == 8) { > + RTE_ETH_DEV_SRIOV(eth_dev).active = ETH_32_POOLS; > + RTE_ETH_DEV_SRIOV(eth_dev).nb_q_per_pool = 4; > + RTE_ETH_DEV_SRIOV(eth_dev).def_pool_q_idx = > + dev_num_vf(eth_dev) * 4; > + } > + } > + > /* set VMDq map to default PF pool */ > hw->mac.ops.set_vmdq(hw, 0, RTE_ETH_DEV_SRIOV(eth_dev).def_vmdq_idx); > > diff --git a/lib/librte_pmd_ixgbe/ixgbe_rxtx.c b/lib/librte_pmd_ixgbe/ixgbe_rxtx.c > index f69abda..e83a9ab 100644 > --- a/lib/librte_pmd_ixgbe/ixgbe_rxtx.c > +++ b/lib/librte_pmd_ixgbe/ixgbe_rxtx.c > @@ -3327,6 +3327,68 @@ ixgbe_alloc_rx_queue_mbufs(struct igb_rx_queue *rxq) > } > > static int > +ixgbe_config_vf_rss(struct rte_eth_dev *dev) > +{ > + struct ixgbe_hw *hw; > + uint32_t mrqc; > + > + ixgbe_rss_configure(dev); > + > + hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private); > + > + /* MRQC: enable VF RSS */ > + mrqc = IXGBE_READ_REG(hw, IXGBE_MRQC); > + mrqc &= ~IXGBE_MRQC_MRQE_MASK; > + switch (RTE_ETH_DEV_SRIOV(dev).active) { > + case ETH_64_POOLS: > + mrqc |= IXGBE_MRQC_VMDQRSS64EN; > + break; > + > + case ETH_32_POOLS: > + case ETH_16_POOLS: Isn't ETH_16_POOLS mode is invalid for VF RSS? It's what both spec states and what u handle in this patch in ixgbe_pf_host_configure(). IMHO it would be better to treat this mode value as an error here since if u get it here it indicates of a SW bug. > + mrqc |= IXGBE_MRQC_VMDQRSS32EN; > + break; > + > + default: > + PMD_INIT_LOG(ERR, "Invalid pool number in IOV mode"); > + return -EINVAL; > + } > + > + IXGBE_WRITE_REG(hw, IXGBE_MRQC, mrqc); > + > + return 0; > +} > + > +static int > +ixgbe_config_vf_default(struct rte_eth_dev *dev) > +{ > + struct ixgbe_hw *hw = > + IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private); > + > + switch (RTE_ETH_DEV_SRIOV(dev).active) { > + case ETH_64_POOLS: > + IXGBE_WRITE_REG(hw, IXGBE_MRQC, > + IXGBE_MRQC_VMDQEN); > + break; > + > + case ETH_32_POOLS: > + IXGBE_WRITE_REG(hw, IXGBE_MRQC, > + IXGBE_MRQC_VMDQRT4TCEN); > + break; > + > + case ETH_16_POOLS: > + IXGBE_WRITE_REG(hw, IXGBE_MRQC, > + IXGBE_MRQC_VMDQRT8TCEN); > + break; > + default: > + PMD_INIT_LOG(ERR, > + "invalid pool number in IOV mode"); > + break; > + } > + return 0; > +} > + > +static int > ixgbe_dev_mq_rx_configure(struct rte_eth_dev *dev) > { > struct ixgbe_hw *hw = > @@ -3358,24 +3420,25 @@ ixgbe_dev_mq_rx_configure(struct rte_eth_dev *dev) > default: ixgbe_rss_disable(dev); > } > } else { > - switch (RTE_ETH_DEV_SRIOV(dev).active) { > /* > * SRIOV active scheme > - * FIXME if support DCB/RSS together with VMDq & SRIOV > + * Support RSS together with VMDq & SRIOV > */ > - case ETH_64_POOLS: > - IXGBE_WRITE_REG(hw, IXGBE_MRQC, IXGBE_MRQC_VMDQEN); > - break; > - > - case ETH_32_POOLS: > - IXGBE_WRITE_REG(hw, IXGBE_MRQC, IXGBE_MRQC_VMDQRT4TCEN); > + switch (dev->data->dev_conf.rxmode.mq_mode) { > + case ETH_MQ_RX_RSS: > + case ETH_MQ_RX_VMDQ_RSS: > + ixgbe_config_vf_rss(dev); > break; > > - case ETH_16_POOLS: > - IXGBE_WRITE_REG(hw, IXGBE_MRQC, IXGBE_MRQC_VMDQRT8TCEN); > - break; > + /* FIXME if support DCB/RSS together with VMDq & SRIOV */ > + case ETH_MQ_RX_VMDQ_DCB: > + case ETH_MQ_RX_VMDQ_DCB_RSS: > + PMD_INIT_LOG(ERR, > + "Could not support DCB with VMDq & SRIOV"); > + return -1; > default: > - PMD_INIT_LOG(ERR, "invalid pool number in IOV mode"); > + ixgbe_config_vf_default(dev); > + break; > } > } > > @@ -3993,6 +4056,19 @@ ixgbevf_dev_rx_init(struct rte_eth_dev *dev) > PMD_INIT_FUNC_TRACE(); > hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private); > > + if (rte_is_power_of_2(dev->data->nb_rx_queues) == 0) { > + PMD_INIT_LOG(ERR, "The number of Rx queue invalid, " > + "it should be power of 2"); > + return -1; > + } > + > + if (dev->data->nb_rx_queues > hw->mac.max_rx_queues) { > + PMD_INIT_LOG(ERR, "The number of Rx queue invalid, " > + "it should be equal to or less than %d", > + hw->mac.max_rx_queues); > + return -1; > + } > + > /* > * When the VF driver issues a IXGBE_VF_RESET request, the PF driver > * disables the VF receipt of packets if the PF MTU is > 1500. > @@ -4094,6 +4170,9 @@ ixgbevf_dev_rx_init(struct rte_eth_dev *dev) > IXGBE_PSRTYPE_IPV6HDR; > #endif > > + /* Set RQPL for VF RSS according to max Rx queue */ > + psrtype |= (dev->data->nb_rx_queues >> 1) << > + IXGBE_PSRTYPE_RQPL_SHIFT; > IXGBE_WRITE_REG(hw, IXGBE_VFPSRTYPE, psrtype); > > if (dev->data->dev_conf.rxmode.enable_scatter) {