Re: [dpdk-dev] [PATCH 2/2] ixgbe: don't override mbuf buffer length

[Jean-Mickael Guerin <jean-mickael.guerin@6wind.com>]

On 04/12/2014 16:32, Bruce Richardson wrote:
> On Thu, Dec 04, 2014 at 03:29:04PM +0000, Ananyev, Konstantin wrote:
>>
>>
>>> -----Original Message-----
>>> From: Richardson, Bruce
>>> Sent: Thursday, December 04, 2014 3:15 PM
>>> To: Ananyev, Konstantin
>>> Cc: Jean-Mickael Guerin; dev@dpdk.org
>>> Subject: Re: [dpdk-dev] [PATCH 2/2] ixgbe: don't override mbuf buffer length
>>>
>>> On Thu, Dec 04, 2014 at 02:50:11PM +0000, Ananyev, Konstantin wrote:
>>>> Hi,
>>>>
>>>>> -----Original Message-----
>>>>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Jean-Mickael Guerin
>>>>> Sent: Thursday, December 04, 2014 2:26 PM
>>>>> To: dev@dpdk.org
>>>>> Subject: [dpdk-dev] [PATCH 2/2] ixgbe: don't override mbuf buffer length
>>>>>
>>>>> The template mbuf_initializer is hard coded with a buflen which
>>>>> might have been set differently by the application at the time of
>>>>> mbuf pool creation.
>>>>>
>>>>> Switch to a mbuf allocation, to fetch the correct default values.
>>>>> There is no performance impact because this is not a data-plane API.
>>>>>
>>>>> Signed-off-by: Jean-Mickael Guerin <jean-mickael.guerin@6wind.com>
>>>>> Acked-by: David Marchand <david.marchand@6wind.com>
>>>>> Fixes: 0ff3324da2 ("ixgbe: rework vector pmd following mbuf changes")
>>>>> ---
>>>>>   lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c | 19 ++++++++++++-------
>>>>>   1 file changed, 12 insertions(+), 7 deletions(-)
>>>>>
>>>>> diff --git a/lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c b/lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c
>>>>> index c1b5a78..f7b02f5 100644
>>>>> --- a/lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c
>>>>> +++ b/lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c
>>>>> @@ -732,17 +732,22 @@ static struct ixgbe_txq_ops vec_txq_ops = {
>>>>>   int
>>>>>   ixgbe_rxq_vec_setup(struct igb_rx_queue *rxq)
>>>>>   {
>>>>> -	struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */
>>>>> +	struct rte_mbuf *mb_def;
>>>>>
>>>>> -	mb_def.nb_segs = 1;
>>>>> -	mb_def.data_off = RTE_PKTMBUF_HEADROOM;
>>>>> -	mb_def.buf_len = rxq->mb_pool->elt_size - sizeof(struct rte_mbuf);
>>>>> -	mb_def.port = rxq->port_id;
>>>>> -	rte_mbuf_refcnt_set(&mb_def, 1);
>>>>> +	mb_def = rte_pktmbuf_alloc(rxq->mb_pool);
>>>>
>>>> Could you explain to me, what is an advantage of using dynamic allocation vs local struct here?
>>>> I don't see any.
>>>
>>> It means that we get an mbuf that is initialized as done by the initialization
>>> function passed to the mempool_create call. The static variable method assumes
>>> that we configure the mbuf using default setting for buf_len etc.
>>>
>>
>> I understand that, but  why it can't be done in some other way?
>> Without allocating/freeing?
>> Let say, at mempool_create() store obj_init() and then add ability to call it again?

This is a good idea, useful in other places of mbuf API.

>> Anyway, it doesn't look to me like a critical problem, that requires an urgent patch for 1.8.
>>

This is about data corruption - a simple function like 
rte_pktmbuf_tailroom() returns an incorrect value...

Let me try obj_init() variant and we will see if it is acceptable in 1.8 
- it does not look a big change after all.

>>>> Plus if rte_pktmbuf_alloc() would fail, we'll leave our rx queue not configured properly.
>>>> As I can see ixgbe_dev_rx_queue_setup() doesn't check the return value of >  ixgbe_rxq_vec_setup()
>>>> (as it is just not supposed to fail).
>>>> So ixgbe_dev_rx_queue_setup() will return OK for unconfigured RX queue.
>>>
>>> Good catch, that's something that should perhaps be looked at in a V2 patch, I
>>> think.
>>>
>>>>
>>>>> +	if (mb_def == NULL) {
>>>>> +		PMD_INIT_LOG(ERR, "ixgbe_rxq_vec_setup: could not allocate one mbuf");
>>>>> +		return -1;
>>>>> +	}
>>>>> +	/* nb_segs, refcnt, data_off and buf_len are already set */
>>>>> +	mb_def->port = rxq->port_id;
>>>>>
>>>>>   	/* prevent compiler reordering: rearm_data covers previous fields */
>>>>>   	rte_compiler_barrier();
>>>>
>>>> I don't think we need it here.
>>>
>>> I think we might, as the compiler doesn't know that the rearm data overlaps
>>> with the previously set fields, so may reorder the reads and writes thinking
>>> they are independent.
>>
>> Why it doesn't?
>> I suppose compiler has all the knowledge of the mbuf structure layout at that point.
>> Or there is a some sort of bug in some version of the compiler?
>>
>
> No, we're just violating the layout here by dereferencing past the end of the array :-)
>
> /Bruce
>
>>>>
>>>>> -	rxq->mbuf_initializer = *((uint64_t *)&mb_def.rearm_data);
>>>>> +	rxq->mbuf_initializer = *((uint64_t *)&mb_def->rearm_data);
>>>>> +
>>>>> +	rte_pktmbuf_free(mb_def);
>>>>> +
>>>>>   	return 0;
>>>>>   }
>>>>>
>>>>> --
>>>>> 2.1.3
>>>>
>>>> Somy vote -  NACK for the whole series.
>>>> Konstantin
>>>>