From: Vlad Zolotarov <vladz@cloudius-systems.com>
To: "Ouyang, Changchun" <changchun.ouyang@intel.com>,
"dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] [PATCH v3 5/6] ixgbe: Config VF RSS
Date: Thu, 25 Dec 2014 15:20:16 +0200 [thread overview]
Message-ID: <549C0F10.8050402@cloudius-systems.com> (raw)
In-Reply-To: <F52918179C57134FAEC9EA62FA2F96251194A773@shsmsx102.ccr.corp.intel.com>
On 12/25/14 04:43, Ouyang, Changchun wrote:
> Hi,
> Sorry miss some comments, so continue my response below,
>
>> -----Original Message-----
>> From: Vlad Zolotarov [mailto:vladz@cloudius-systems.com]
>> Sent: Wednesday, December 24, 2014 6:40 PM
>> To: Ouyang, Changchun; dev@dpdk.org
>> Subject: Re: [dpdk-dev] [PATCH v3 5/6] ixgbe: Config VF RSS
>>
>>
>> On 12/24/14 07:23, Ouyang Changchun wrote:
>>> It needs config RSS and IXGBE_MRQC and IXGBE_VFPSRTYPE to enable VF
>> RSS.
>>> The psrtype will determine how many queues the received packets will
>>> distribute to, and the value of psrtype should depends on both facet:
>>> max VF rxq number which has been negotiated with PF, and the number of
>> rxq specified in config on guest.
>>> Signed-off-by: Changchun Ouyang <changchun.ouyang@intel.com>
>>> ---
>>> lib/librte_pmd_ixgbe/ixgbe_pf.c | 15 +++++++
>>> lib/librte_pmd_ixgbe/ixgbe_rxtx.c | 92
>> ++++++++++++++++++++++++++++++++++-----
>>> 2 files changed, 97 insertions(+), 10 deletions(-)
>>>
>>> diff --git a/lib/librte_pmd_ixgbe/ixgbe_pf.c
>>> b/lib/librte_pmd_ixgbe/ixgbe_pf.c index cbb0145..9c9dad8 100644
>>> --- a/lib/librte_pmd_ixgbe/ixgbe_pf.c
>>> +++ b/lib/librte_pmd_ixgbe/ixgbe_pf.c
>>> @@ -187,6 +187,21 @@ int ixgbe_pf_host_configure(struct rte_eth_dev
>> *eth_dev)
>>> IXGBE_WRITE_REG(hw, IXGBE_MPSAR_LO(hw-
>>> mac.num_rar_entries), 0);
>>> IXGBE_WRITE_REG(hw, IXGBE_MPSAR_HI(hw-
>>> mac.num_rar_entries), 0);
>>>
>>> + /*
>>> + * VF RSS can support at most 4 queues for each VF, even if
>>> + * 8 queues are available for each VF, it need refine to 4
>>> + * queues here due to this limitation, otherwise no queue
>>> + * will receive any packet even RSS is enabled.
>> According to Table 7-3 in the 82599 spec RSS is not available when port is
>> configured to have 8 queues per pool. This means that if u see this
>> configuration u may immediately disable RSS flow in your code.
>>
>>> + */
>>> + if (eth_dev->data->dev_conf.rxmode.mq_mode ==
>> ETH_MQ_RX_VMDQ_RSS) {
>>> + if (RTE_ETH_DEV_SRIOV(eth_dev).nb_q_per_pool == 8) {
>>> + RTE_ETH_DEV_SRIOV(eth_dev).active =
>> ETH_32_POOLS;
>>> + RTE_ETH_DEV_SRIOV(eth_dev).nb_q_per_pool = 4;
>>> + RTE_ETH_DEV_SRIOV(eth_dev).def_pool_q_idx =
>>> + dev_num_vf(eth_dev) * 4;
>> According to 82599 spec u can't do that since RSS is not allowed when port is
>> configured to have 8 function per-VF. Have u verified that this works? If yes,
>> then spec should be updated.
>>
>>> + }
>>> + }
>>> +
>>> /* set VMDq map to default PF pool */
>>> hw->mac.ops.set_vmdq(hw, 0,
>>> RTE_ETH_DEV_SRIOV(eth_dev).def_vmdq_idx);
>>>
>>> diff --git a/lib/librte_pmd_ixgbe/ixgbe_rxtx.c
>>> b/lib/librte_pmd_ixgbe/ixgbe_rxtx.c
>>> index f69abda..a7c17a4 100644
>>> --- a/lib/librte_pmd_ixgbe/ixgbe_rxtx.c
>>> +++ b/lib/librte_pmd_ixgbe/ixgbe_rxtx.c
>>> @@ -3327,6 +3327,39 @@ ixgbe_alloc_rx_queue_mbufs(struct
>> igb_rx_queue *rxq)
>>> }
>>>
>>> static int
>>> +ixgbe_config_vf_rss(struct rte_eth_dev *dev) {
>>> + struct ixgbe_hw *hw;
>>> + uint32_t mrqc;
>>> +
>>> + ixgbe_rss_configure(dev);
>>> +
>>> + hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
>>> +
>>> + /* MRQC: enable VF RSS */
>>> + mrqc = IXGBE_READ_REG(hw, IXGBE_MRQC);
>>> + mrqc &= ~IXGBE_MRQC_MRQE_MASK;
>>> + switch (RTE_ETH_DEV_SRIOV(dev).active) {
>>> + case ETH_64_POOLS:
>>> + mrqc |= IXGBE_MRQC_VMDQRSS64EN;
>>> + break;
>>> +
>>> + case ETH_32_POOLS:
>>> + case ETH_16_POOLS:
>>> + mrqc |= IXGBE_MRQC_VMDQRSS32EN;
>> Again, this contradicts with the spec.
> Yes, the spec say the hw can't support vf rss at all, but experiment find that could be done.
The spec explicitly say that VF RSS *is* supported in particular in the
table mentioned above.
What your code is doing is that in case of 16 VFs u setup a 32 pools
configuration and use only 16 out of them.
> We can focus on discussing the implementation firstly.
>
>>> + break;
>>> +
>>> + default:
>>> + PMD_INIT_LOG(ERR, "Invalid pool number in IOV mode");
>>> + return -EINVAL;
>>> + }
>>> +
>>> + IXGBE_WRITE_REG(hw, IXGBE_MRQC, mrqc);
>>> +
>>> + return 0;
>>> +}
>>> +
>>> +static int
>>> ixgbe_dev_mq_rx_configure(struct rte_eth_dev *dev)
>>> {
>>> struct ixgbe_hw *hw =
>>> @@ -3358,24 +3391,38 @@ ixgbe_dev_mq_rx_configure(struct
>> rte_eth_dev *dev)
>>> default: ixgbe_rss_disable(dev);
>>> }
>>> } else {
>>> - switch (RTE_ETH_DEV_SRIOV(dev).active) {
>>> /*
>>> * SRIOV active scheme
>>> * FIXME if support DCB/RSS together with VMDq & SRIOV
>>> */
>>> - case ETH_64_POOLS:
>>> - IXGBE_WRITE_REG(hw, IXGBE_MRQC,
>> IXGBE_MRQC_VMDQEN);
>>> + switch (dev->data->dev_conf.rxmode.mq_mode) {
>>> + case ETH_MQ_RX_RSS:
>>> + case ETH_MQ_RX_VMDQ_RSS:
>>> + ixgbe_config_vf_rss(dev);
>>> break;
>>>
>>> - case ETH_32_POOLS:
>>> - IXGBE_WRITE_REG(hw, IXGBE_MRQC,
>> IXGBE_MRQC_VMDQRT4TCEN);
>>> - break;
>>> + default:
>>> + switch (RTE_ETH_DEV_SRIOV(dev).active) {
>> Sorry for nitpicking but have u considered taking this encapsulated "switch-
>> case" block into a separate function? This could make the code look a lot
>> nicer. ;)
>>
>>> + case ETH_64_POOLS:
>>> + IXGBE_WRITE_REG(hw, IXGBE_MRQC,
>>> + IXGBE_MRQC_VMDQEN);
>>> + break;
>>>
>>> - case ETH_16_POOLS:
>>> - IXGBE_WRITE_REG(hw, IXGBE_MRQC,
>> IXGBE_MRQC_VMDQRT8TCEN);
>>> + case ETH_32_POOLS:
>>> + IXGBE_WRITE_REG(hw, IXGBE_MRQC,
>>> + IXGBE_MRQC_VMDQRT4TCEN);
>>> + break;
>>> +
>>> + case ETH_16_POOLS:
>>> + IXGBE_WRITE_REG(hw, IXGBE_MRQC,
>>> + IXGBE_MRQC_VMDQRT8TCEN);
>>> + break;
>>> + default:
>>> + PMD_INIT_LOG(ERR,
>>> + "invalid pool number in IOV mode");
>>> + break;
>>> + }
>>> break;
>>> - default:
>>> - PMD_INIT_LOG(ERR, "invalid pool number in IOV
>> mode");
>>> }
>>> }
>>>
>>> @@ -3989,10 +4036,32 @@ ixgbevf_dev_rx_init(struct rte_eth_dev *dev)
>>> uint16_t buf_size;
>>> uint16_t i;
>>> int ret;
>>> + uint16_t valid_rxq_num;
>>>
>>> PMD_INIT_FUNC_TRACE();
>>> hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
>>>
>>> + valid_rxq_num = RTE_MIN(dev->data->nb_rx_queues,
>>> +hw->mac.max_rx_queues);
>>> +
>>> + /*
>>> + * VMDq RSS can't support 3 queues, so config it into 4 queues,
>>> + * and give user a hint that some packets may loss if it doesn't
>>> + * poll the queue where those packets are distributed to.
>>> + */
>>> + if (valid_rxq_num == 3)
>>> + valid_rxq_num = 4;
>> Why to configure more queues that requested and not less (2)? Why to
>> configure anything at all and not return an error?
> Sorry, I don't agree this is "anything" you say, because I don't use 5,6,7, 8, ..., 16, 2014, 2015,... etc.
> By considering 2 or 4,
> I prefer 4, the reason is if user need more than 3 queues per vf to do something,
> And pf has also the capability to setup 4 queues per vf, confining to 2 queues is also not good thing,
> So here try to enable 4 queues, and give user hints here.
> Btw, change it into 2 is another way, depends on other guys' more insight here.
Like I said before, trying to guess what user wants is a way to making a
code that is very hard to use and to maintain. Pls., just return an
error and let the user code deal with it the way he/she really wants and
not the way u *think* he/she wants.
>
>>> +
>>> + if (dev->data->nb_rx_queues > valid_rxq_num) {
>>> + PMD_INIT_LOG(ERR, "The number of Rx queue invalid, "
>>> + "it should be equal to or less than %d",
>>> + valid_rxq_num);
>>> + return -1;
>>> + } else if (dev->data->nb_rx_queues < valid_rxq_num)
>>> + PMD_INIT_LOG(ERR, "The number of Rx queue is less "
>>> + "than the number of available Rx queues:%d, "
>>> + "packets in Rx queues(q_id >= %d) may loss.",
>>> + valid_rxq_num, dev->data->nb_rx_queues);
>> Who ever looks in the "INIT_LOG" if everything "work well" and u make it
>> look so by allowing this call to succeed. And then some packets will just
>> silently not arrive?! And what the used should somehow guess to do?
>> - Look in the "INIT_LOG"?! This is a nightmare!
>>
>>> +
>>> /*
>>> * When the VF driver issues a IXGBE_VF_RESET request, the PF
>> driver
>>> * disables the VF receipt of packets if the PF MTU is > 1500.
>>> @@ -4094,6 +4163,9 @@ ixgbevf_dev_rx_init(struct rte_eth_dev *dev)
>>> IXGBE_PSRTYPE_IPV6HDR;
>>> #endif
>>>
>>> + /* Set RQPL for VF RSS according to max Rx queue */
>>> + psrtype |= (valid_rxq_num >> 1) <<
>>> + IXGBE_PSRTYPE_RQPL_SHIFT;
>>> IXGBE_WRITE_REG(hw, IXGBE_VFPSRTYPE, psrtype);
>>>
>>> if (dev->data->dev_conf.rxmode.enable_scatter) {
next prev parent reply other threads:[~2014-12-25 13:20 UTC|newest]
Thread overview: 144+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-12-15 2:56 [dpdk-dev] [PATCH 0/6] Enable VF RSS for Niantic Ouyang Changchun
2014-12-15 2:57 ` [dpdk-dev] [PATCH 1/6] ixgbe: Code cleanup Ouyang Changchun
2014-12-15 2:57 ` [dpdk-dev] [PATCH 2/6] ixgbe: Negotiate VF API version Ouyang Changchun
2014-12-15 2:57 ` [dpdk-dev] [PATCH 3/6] ixgbe: Get VF queue number Ouyang Changchun
2014-12-15 2:57 ` [dpdk-dev] [PATCH 4/6] ether: Check VMDq RSS mode Ouyang Changchun
2014-12-15 2:57 ` [dpdk-dev] [PATCH 5/6] ixgbe: Config VF RSS Ouyang Changchun
2014-12-15 2:57 ` [dpdk-dev] [PATCH 6/6] testpmd: Set Rx VMDq RSS mode Ouyang Changchun
2014-12-15 10:55 ` [dpdk-dev] [PATCH 0/6] Enable VF RSS for Niantic Bruce Richardson
2014-12-16 0:58 ` Ouyang, Changchun
2014-12-24 2:56 ` [dpdk-dev] [PATCH v2 " Ouyang Changchun
2014-12-24 2:56 ` [dpdk-dev] [PATCH v2 1/6] ixgbe: Code cleanup Ouyang Changchun
2014-12-24 3:08 ` Zhang, Helin
2014-12-24 3:22 ` Ouyang, Changchun
2014-12-24 3:41 ` Zhang, Helin
2014-12-24 3:50 ` Ouyang, Changchun
2014-12-24 3:53 ` Zhang, Helin
2014-12-24 4:46 ` Ouyang, Changchun
2014-12-24 2:56 ` [dpdk-dev] [PATCH v2 2/6] ixgbe: Negotiate VF API version Ouyang Changchun
2014-12-24 2:56 ` [dpdk-dev] [PATCH v2 3/6] ixgbe: Get VF queue number Ouyang Changchun
2014-12-24 2:56 ` [dpdk-dev] [PATCH v2 4/6] ether: Check VMDq RSS mode Ouyang Changchun
2014-12-24 2:56 ` [dpdk-dev] [PATCH v2 5/6] ixgbe: Config VF RSS Ouyang Changchun
2014-12-24 2:56 ` [dpdk-dev] [PATCH v2 6/6] testpmd: Set Rx VMDq RSS mode Ouyang Changchun
2014-12-24 5:22 ` [dpdk-dev] [PATCH v3 0/6] Enable VF RSS for Niantic Ouyang Changchun
2014-12-24 5:22 ` [dpdk-dev] [PATCH v3 1/6] ixgbe: Code cleanup Ouyang Changchun
2014-12-24 5:23 ` [dpdk-dev] [PATCH v3 2/6] ixgbe: Negotiate VF API version Ouyang Changchun
2014-12-24 5:23 ` [dpdk-dev] [PATCH v3 3/6] ixgbe: Get VF queue number Ouyang Changchun
2014-12-24 5:23 ` [dpdk-dev] [PATCH v3 4/6] ether: Check VMDq RSS mode Ouyang Changchun
2014-12-24 5:23 ` [dpdk-dev] [PATCH v3 5/6] ixgbe: Config VF RSS Ouyang Changchun
2014-12-24 10:39 ` Vlad Zolotarov
2014-12-25 2:14 ` Ouyang, Changchun
2014-12-25 13:13 ` Vlad Zolotarov
2014-12-26 2:07 ` Ouyang, Changchun
2014-12-25 2:43 ` Ouyang, Changchun
2014-12-25 13:20 ` Vlad Zolotarov [this message]
2014-12-26 1:52 ` Ouyang, Changchun
2014-12-26 6:49 ` Vladislav Zolotarov
2014-12-26 7:26 ` Ouyang, Changchun
2014-12-26 7:37 ` Vladislav Zolotarov
2014-12-26 8:45 ` Ouyang, Changchun
2014-12-28 10:14 ` Vlad Zolotarov
2015-01-05 10:29 ` Bruce Richardson
2015-01-06 1:00 ` Ouyang, Changchun
2014-12-25 13:38 ` Vlad Zolotarov
2014-12-26 1:26 ` Ouyang, Changchun
2015-01-04 2:10 ` Liang, Cunming
2015-01-04 6:25 ` Ouyang, Changchun
2014-12-24 5:23 ` [dpdk-dev] [PATCH v3 6/6] testpmd: Set Rx VMDq RSS mode Ouyang Changchun
2014-12-24 9:59 ` [dpdk-dev] [PATCH v3 0/6] Enable VF RSS for Niantic Vlad Zolotarov
2014-12-25 1:46 ` Ouyang, Changchun
2015-01-05 10:38 ` Bruce Richardson
2015-01-05 13:02 ` Vlad Zolotarov
2015-01-06 1:11 ` Ouyang, Changchun
2015-01-06 11:18 ` Vlad Zolotarov
2015-01-06 11:18 ` Vlad Zolotarov
2015-01-06 1:04 ` Ouyang, Changchun
2014-12-24 10:49 ` Vlad Zolotarov
2014-12-25 2:26 ` Ouyang, Changchun
2014-12-25 12:46 ` Vlad Zolotarov
2014-12-26 2:37 ` Ouyang, Changchun
[not found] ` <CAOYyTHbrB-VinN5ZEd1tYTnS7_GhCT1jiHiZzNKkQUEJ1rG79w@mail.gmail.com>
2014-12-26 5:16 ` Vladislav Zolotarov
2014-12-26 5:25 ` Ouyang, Changchun
2015-01-04 7:18 ` [dpdk-dev] [PATCH v4 " Ouyang Changchun
2015-01-04 7:18 ` [dpdk-dev] [PATCH v4 1/6] ixgbe: Code cleanup Ouyang Changchun
2015-01-04 8:22 ` Vlad Zolotarov
2015-01-04 7:18 ` [dpdk-dev] [PATCH v4 2/6] ixgbe: Negotiate VF API version Ouyang Changchun
2015-01-04 8:26 ` Vlad Zolotarov
2015-01-04 8:30 ` Vlad Zolotarov
2015-01-04 8:37 ` Ouyang, Changchun
2015-01-04 8:40 ` Vlad Zolotarov
2015-01-04 8:51 ` Ouyang, Changchun
2015-01-04 9:37 ` Vlad Zolotarov
2015-01-04 7:18 ` [dpdk-dev] [PATCH v4 3/6] ixgbe: Get VF queue number Ouyang Changchun
2015-01-04 8:38 ` Vlad Zolotarov
2015-01-05 2:59 ` Ouyang, Changchun
2015-01-05 10:07 ` Vlad Zolotarov
2015-01-06 1:54 ` Ouyang, Changchun
2015-01-06 11:26 ` Vlad Zolotarov
2015-01-07 1:18 ` Ouyang, Changchun
2015-01-04 7:18 ` [dpdk-dev] [PATCH v4 4/6] ether: Check VMDq RSS mode Ouyang Changchun
2015-01-04 8:45 ` Vlad Zolotarov
2015-01-04 8:58 ` Ouyang, Changchun
2015-01-04 9:45 ` Vlad Zolotarov
2015-01-05 1:00 ` Ouyang, Changchun
2015-01-05 10:09 ` Vlad Zolotarov
2015-01-06 1:56 ` Ouyang, Changchun
2015-01-06 19:56 ` Vlad Zolotarov
2015-01-07 2:28 ` Ouyang, Changchun
2015-01-04 7:18 ` [dpdk-dev] [PATCH v4 5/6] ixgbe: Config VF RSS Ouyang Changchun
2015-01-04 7:18 ` [dpdk-dev] [PATCH v4 6/6] testpmd: Set Rx VMDq RSS mode Ouyang Changchun
2015-01-04 8:49 ` Vlad Zolotarov
2015-01-04 9:01 ` Ouyang, Changchun
2015-01-04 9:46 ` Vlad Zolotarov
2015-01-05 2:38 ` Ouyang, Changchun
2015-01-05 10:12 ` Vlad Zolotarov
2015-01-06 2:01 ` Ouyang, Changchun
2015-01-06 12:53 ` Vlad Zolotarov
2015-01-07 1:50 ` Ouyang, Changchun
2015-01-07 6:32 ` [dpdk-dev] [PATCH v5 0/6] Enable VF RSS for Niantic Ouyang Changchun
2015-01-07 6:32 ` [dpdk-dev] [PATCH v5 1/6] ixgbe: Code cleanup Ouyang Changchun
2015-01-07 6:32 ` [dpdk-dev] [PATCH v5 2/6] ixgbe: Negotiate VF API version Ouyang Changchun
2015-01-07 6:32 ` [dpdk-dev] [PATCH v5 3/6] ixgbe: Get VF queue number Ouyang Changchun
2015-01-08 9:01 ` Vlad Zolotarov
2015-01-07 6:32 ` [dpdk-dev] [PATCH v5 4/6] ether: Check VMDq RSS mode Ouyang Changchun
2015-01-08 9:19 ` Vlad Zolotarov
2015-01-08 18:48 ` Vlad Zolotarov
2015-01-09 5:54 ` Ouyang, Changchun
2015-01-09 13:49 ` Vlad Zolotarov
2015-01-12 3:41 ` Ouyang, Changchun
2015-01-12 13:58 ` Vlad Zolotarov
2015-01-13 1:50 ` Ouyang, Changchun
2015-01-13 9:00 ` Vlad Zolotarov
2015-01-14 0:44 ` Ouyang, Changchun
2015-01-07 6:32 ` [dpdk-dev] [PATCH v5 5/6] ixgbe: Config VF RSS Ouyang Changchun
2015-01-08 9:43 ` Vlad Zolotarov
2015-01-09 6:07 ` Ouyang, Changchun
2015-01-09 14:01 ` Vlad Zolotarov
2015-01-12 5:11 ` Ouyang, Changchun
2015-01-07 6:32 ` [dpdk-dev] [PATCH v5 6/6] testpmd: Set Rx VMDq RSS mode Ouyang Changchun
2015-01-08 9:46 ` Vlad Zolotarov
2015-01-08 9:56 ` [dpdk-dev] [PATCH v5 0/6] Enable VF RSS for Niantic Vlad Zolotarov
2015-01-18 21:58 ` Thomas Monjalon
2015-01-19 9:40 ` Vlad Zolotarov
2015-01-12 5:59 ` [dpdk-dev] [PATCH v6 " Ouyang Changchun
2015-01-12 5:59 ` [dpdk-dev] [PATCH v6 1/6] ixgbe: Code cleanup Ouyang Changchun
2015-01-12 5:59 ` [dpdk-dev] [PATCH v6 2/6] ixgbe: Negotiate VF API version Ouyang Changchun
2015-01-12 5:59 ` [dpdk-dev] [PATCH v6 3/6] ixgbe: Get VF queue number Ouyang Changchun
2015-01-19 9:13 ` Wodkowski, PawelX
2015-01-20 0:54 ` Ouyang, Changchun
2015-01-12 5:59 ` [dpdk-dev] [PATCH v6 4/6] ether: Check VMDq RSS mode Ouyang Changchun
2015-01-12 14:06 ` Vlad Zolotarov
2015-01-18 22:04 ` Thomas Monjalon
2015-01-19 10:31 ` Wodkowski, PawelX
2015-01-20 1:03 ` Ouyang, Changchun
2015-01-12 5:59 ` [dpdk-dev] [PATCH v6 5/6] ixgbe: Config VF RSS Ouyang Changchun
2015-01-12 14:04 ` Vlad Zolotarov
2015-01-20 9:35 ` Wodkowski, PawelX
2015-01-21 2:43 ` Ouyang, Changchun
2015-01-21 8:44 ` Wodkowski, PawelX
2015-01-22 12:59 ` Vlad Zolotarov
2015-01-22 13:19 ` Wodkowski, PawelX
2015-01-12 5:59 ` [dpdk-dev] [PATCH v6 6/6] testpmd: Set Rx VMDq RSS mode Ouyang Changchun
2015-01-12 14:05 ` Vlad Zolotarov
2015-01-18 22:24 ` [dpdk-dev] [PATCH v6 0/6] Enable VF RSS for Niantic Thomas Monjalon
2015-01-19 4:51 ` Ouyang, Changchun
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=549C0F10.8050402@cloudius-systems.com \
--to=vladz@cloudius-systems.com \
--cc=changchun.ouyang@intel.com \
--cc=dev@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).