Re: [dpdk-dev] [PATCH v2 2/3] net/af_xdp: support pinning of IRQs

[Ferruh Yigit <ferruh.yigit@intel.com>]

On 10/21/2019 4:54 PM, Bruce Richardson wrote:
> On Mon, Oct 21, 2019 at 04:24:26PM +0100, Ferruh Yigit wrote:
>> On 10/14/2019 3:43 PM, Bruce Richardson wrote:
>>> On Thu, Oct 03, 2019 at 02:23:07PM +0100, Loftus, Ciara wrote:
>>>>
>>>>
>>>>> -----Original Message----- From: Stephen Hemminger
>>>>> <stephen@networkplumber.org> Sent: Monday 30 September 2019 18:12 To:
>>>>> Loftus, Ciara <ciara.loftus@intel.com> Cc: dev@dpdk.org; Ye, Xiaolong
>>>>> <xiaolong.ye@intel.com>; Laatz, Kevin <kevin.laatz@intel.com>;
>>>>> Richardson, Bruce <bruce.richardson@intel.com> Subject: Re: [dpdk-dev]
>>>>> [PATCH v2 2/3] net/af_xdp: support pinning of IRQs
>>>>>
>>>>> On Mon, 30 Sep 2019 16:42:04 +0000 Ciara Loftus
>>>>> <ciara.loftus@intel.com> wrote:
>>>>>
>>>>>> +/* drivers supported for the queue_irq option */ +enum
>>>>>> supported_drivers { +	I40E_DRIVER, +	IXGBE_DRIVER, +
>>>>>> MLX5_DRIVER, +	NUM_DRIVERS +};
>>>>>
>>>>> Anything device specific like this raises a red flag to me.
>>>>>
>>>>> This regex etc, seems like a huge hack. Is there a better way  using
>>>>> irqbalance and smp_affinity in kernel drivers?
>>>>>
>>>>> NACK
>>>>
>>>> Hi Stephen,
>>>>  
>>>> Thanks for looking at the patch. I understand your concern however
>>>> unfortunately I haven't been able to identify a way to achieve the
>>>> desired outcome by using your suggestions of irqbalance and smp_affinity.
>>>> Did you have something specific in mind or are aware of any generic way
>>>> of retrieving interrupt numbers for NICs regardless of vendor or range?
>>>>  
>>>> I think this feature is really important for the usability of this PMD.
>>>> Without it, to configure the IRQs the user has to open up
>>>> /proc/interrupts, trawl through it and identify the correct IRQ number
>>>> for their given NIC and qid (the format for which is unlikely to be known
>>>> off-hand), and manually pin them by writing the appropriate values in the
>>>> appropriate format to the appropriate file - prone to error if not
>>>> automated IMO.  If the user fails to set the affinity it's probably fine
>>>> for a single pmd, however with multiple pmds all irqs will by default
>>>> land on core 0 and lead to terrible performance.
>>>>
>>>> It should be possible to rework the code to remove the regexes and use a
>>>> direct string compare. Would that make the solution more palatable?
>>>>  
>>>
>>> Hi Ciara, Stephen,
>>>
>>> is there any way forward on this patch?
>>>
>>> From my experience with using AF_XDP the pinning of interrupts is both
>>> necessary for performance and sadly rather awkward to implement in
>>> practice. If we can't find a better way to do this, I think merging this
>>> patch is the best thing to do. It may be a bit messy, but the overall user
>>> experience should be far improved over not having it.
>>>
>>
>> Can we have this external to the PMD, like a helper script that run after you
>> start the DPDK app?
> 
> It could be, but the main objection here seems to be with the method used
> to find the interrupts, which would not change based on a script version,
> not to mention the fact that the usability would be far less in that case.
> For ease of use, this needs to be part of the driver, and the current
> implementation is the best we can do right now, and nobody has suggested a
> concrete alternative to it.
> 

The method used to find interrupts can be same but it being embedded into a
specific PMD with the list of supported driver hardcoded versus putting this
information into a script differs I think.

I tend to agree that this looks hack for the PMD.

And not sure if the usability will be far less, instead of providing parameters
to the PMD, you will provide to the script. And script becomes something more
generic.