Re: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative

Re: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative

[Trahe, Fiona]

Hi Shally, Ahmed, and anyone else interested in compressdev,

I mentioned last week that we've been exploring using something other than mbufs to pass src/dst buffers to compressdev PMDs.

Reasons:
 - mbuf data is limited to 64k-1 in each segment of a chained mbuf. Data for compression
    can be greater and it would add cycles to have to break up into smaller segments.
 - data may originate in mbufs, but is more likely, particularly for storage use-cases,  to 
    originate in other data structures.
 - There's a 2 cache-line overhead for every segment in a chain, most of this data
    is network-related, not needed by compressdev
So moving to a custom structure would minimise memory overhead, remove restriction on 64k-1 size and give more flexibility if compressdev ever needs any comp-specific meta-data.

We've come up with a compressdev-specific structure using the struct iovec from sys/uio.h, which is commonly used by storage applications. This would replace the src and dest mbufs in the  op.
I'll not include the code here - Pablo will push that to github shortly and we'd appreciate review comments there.
https://github.com/pablodelara/dpdk-draft-compressdev
Just posting on the mailing list to give a heads-up and ensure this reaches a wider audience than may see it on github.  

Note : We also considered having no data structures in the op, instead the application
would supply a callback which the PMD would use to retrieve meta-data (virt address, iova, length) 
for each next segment as needed. While this is quite flexible and allow the application
to keep its data in its native structures, it's likely to cost more cycles.
So we're not proposing this at the moment, but hope to benchmark it later while the API is still experimental.

General feedback on direction is welcome here on the mailing list.
For feedback on the details of implementation we would appreciate comments on github.

Regards,
Fiona.

Re: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative

[Verma, Shally]

HI Fiona

So I understand we're moving away from mbufs because of its size limitation (64k) and cacheline overhead and their more suitability to n/w applications. Given that, I understand benefit of having another structure to input data but then what is proposal for ipcomp like application where mbuf usage may be a better option? Should we keep support for both (mbuf and this structure) so that apps can use appropriate data structure depending on their requirement.

Further comments, on github.

Thanks
Shally

>-----Original Message-----
>From: Trahe, Fiona [mailto:fiona.trahe@intel.com]
>Sent: 12 March 2018 21:31
>To: Ahmed Mansour <ahmed.mansour@nxp.com>; Verma, Shally <Shally.Verma@cavium.com>; dev@dpdk.org
>Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Athreya, Narayana Prasad <NarayanaPrasad.Athreya@cavium.com>;
>Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>; Challa, Mahipal
><Mahipal.Challa@cavium.com>; Jain, Deepak K <deepak.k.jain@intel.com>; Hemant Agrawal <hemant.agrawal@nxp.com>; Roy
>Pledge <roy.pledge@nxp.com>; Youri Querry <youri.querry_1@nxp.com>; fiona.trahe@gmail.com; Daly, Lee <lee.daly@intel.com>;
>Jozwiak, TomaszX <tomaszx.jozwiak@intel.com>
>Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
>
>Hi Shally, Ahmed, and anyone else interested in compressdev,
>
>I mentioned last week that we've been exploring using something other than mbufs to pass src/dst buffers to compressdev PMDs.
>
>Reasons:
> - mbuf data is limited to 64k-1 in each segment of a chained mbuf. Data for compression
>    can be greater and it would add cycles to have to break up into smaller segments.
> - data may originate in mbufs, but is more likely, particularly for storage use-cases,  to
>    originate in other data structures.
> - There's a 2 cache-line overhead for every segment in a chain, most of this data
>    is network-related, not needed by compressdev
>So moving to a custom structure would minimise memory overhead, remove restriction on 64k-1 size and give more flexibility if
>compressdev ever needs any comp-specific meta-data.
>
>We've come up with a compressdev-specific structure using the struct iovec from sys/uio.h, which is commonly used by storage
>applications. This would replace the src and dest mbufs in the  op.
>I'll not include the code here - Pablo will push that to github shortly and we'd appreciate review comments there.
>https://github.com/pablodelara/dpdk-draft-compressdev
>Just posting on the mailing list to give a heads-up and ensure this reaches a wider audience than may see it on github.
>
>Note : We also considered having no data structures in the op, instead the application
>would supply a callback which the PMD would use to retrieve meta-data (virt address, iova, length)
>for each next segment as needed. While this is quite flexible and allow the application
>to keep its data in its native structures, it's likely to cost more cycles.
>So we're not proposing this at the moment, but hope to benchmark it later while the API is still experimental.
>
>General feedback on direction is welcome here on the mailing list.
>For feedback on the details of implementation we would appreciate comments on github.
>
>Regards,
>Fiona.

Re: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative

[Trahe, Fiona]

Hi Shally,

> -----Original Message-----
> From: Verma, Shally [mailto:Shally.Verma@cavium.com]
> Sent: Tuesday, March 13, 2018 8:15 AM
> To: Trahe, Fiona <fiona.trahe@intel.com>; Ahmed Mansour <ahmed.mansour@nxp.com>; dev@dpdk.org
> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Athreya, Narayana Prasad
> <NarayanaPrasad.Athreya@cavium.com>; Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu, Sunila
> <Sunila.Sahu@cavium.com>; Challa, Mahipal <Mahipal.Challa@cavium.com>; Jain, Deepak K
> <deepak.k.jain@intel.com>; Hemant Agrawal <hemant.agrawal@nxp.com>; Roy Pledge
> <roy.pledge@nxp.com>; Youri Querry <youri.querry_1@nxp.com>; fiona.trahe@gmail.com; Daly, Lee
> <lee.daly@intel.com>; Jozwiak, TomaszX <tomaszx.jozwiak@intel.com>
> Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
> 
> HI Fiona
> 
> So I understand we're moving away from mbufs because of its size limitation (64k) and cacheline overhead
> and their more suitability to n/w applications. Given that, I understand benefit of having another structure
> to input data but then what is proposal for ipcomp like application where mbuf usage may be a better
> option? Should we keep support for both (mbuf and this structure) so that apps can use appropriate data
> structure depending on their requirement.
[Fiona] An application can use pass buffers from an mbuf or mbuf chain to compressdev by filling in the 
compressdev struct fields with the mbuf meta-data, using rte_pktmbuf_data_len(), 
rte_pktmbuf_mtod(), rte_pktmbuf_mtophys(), etc
For simplicity I'd prefer to offer only 1 rather than 2 data formats on the API.
We see storage applications rather than IPComp as the main use-case for compressdev, so would prefer
to optimise for that.
Do you think otherwise?

> 
> Further comments, on github.
> 
> Thanks
> Shally
> 
> >-----Original Message-----
> >From: Trahe, Fiona [mailto:fiona.trahe@intel.com]
> >Sent: 12 March 2018 21:31
> >To: Ahmed Mansour <ahmed.mansour@nxp.com>; Verma, Shally <Shally.Verma@cavium.com>;
> dev@dpdk.org
> >Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Athreya, Narayana Prasad
> <NarayanaPrasad.Athreya@cavium.com>;
> >Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>; Challa, Mahipal
> ><Mahipal.Challa@cavium.com>; Jain, Deepak K <deepak.k.jain@intel.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>; Roy
> >Pledge <roy.pledge@nxp.com>; Youri Querry <youri.querry_1@nxp.com>; fiona.trahe@gmail.com; Daly,
> Lee <lee.daly@intel.com>;
> >Jozwiak, TomaszX <tomaszx.jozwiak@intel.com>
> >Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
> >
> >Hi Shally, Ahmed, and anyone else interested in compressdev,
> >
> >I mentioned last week that we've been exploring using something other than mbufs to pass src/dst
> buffers to compressdev PMDs.
> >
> >Reasons:
> > - mbuf data is limited to 64k-1 in each segment of a chained mbuf. Data for compression
> >    can be greater and it would add cycles to have to break up into smaller segments.
> > - data may originate in mbufs, but is more likely, particularly for storage use-cases,  to
> >    originate in other data structures.
> > - There's a 2 cache-line overhead for every segment in a chain, most of this data
> >    is network-related, not needed by compressdev
> >So moving to a custom structure would minimise memory overhead, remove restriction on 64k-1 size and
> give more flexibility if
> >compressdev ever needs any comp-specific meta-data.
> >
> >We've come up with a compressdev-specific structure using the struct iovec from sys/uio.h, which is
> commonly used by storage
> >applications. This would replace the src and dest mbufs in the  op.
> >I'll not include the code here - Pablo will push that to github shortly and we'd appreciate review
> comments there.
> >https://github.com/pablodelara/dpdk-draft-compressdev
> >Just posting on the mailing list to give a heads-up and ensure this reaches a wider audience than may see
> it on github.
> >
> >Note : We also considered having no data structures in the op, instead the application
> >would supply a callback which the PMD would use to retrieve meta-data (virt address, iova, length)
> >for each next segment as needed. While this is quite flexible and allow the application
> >to keep its data in its native structures, it's likely to cost more cycles.
> >So we're not proposing this at the moment, but hope to benchmark it later while the API is still
> experimental.
> >
> >General feedback on direction is welcome here on the mailing list.
> >For feedback on the details of implementation we would appreciate comments on github.
> >
> >Regards,
> >Fiona.

Re: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative

[Verma, Shally]

>-----Original Message-----
>From: Trahe, Fiona [mailto:fiona.trahe@intel.com]
>Sent: 13 March 2018 21:22
>To: Verma, Shally <Shally.Verma@cavium.com>; Ahmed Mansour <ahmed.mansour@nxp.com>; dev@dpdk.org
>Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Athreya, Narayana Prasad <NarayanaPrasad.Athreya@cavium.com>;
>Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>; Challa, Mahipal
><Mahipal.Challa@cavium.com>; Jain, Deepak K <deepak.k.jain@intel.com>; Hemant Agrawal <hemant.agrawal@nxp.com>; Roy
>Pledge <roy.pledge@nxp.com>; Youri Querry <youri.querry_1@nxp.com>; Daly, Lee <lee.daly@intel.com>; Jozwiak, TomaszX
><tomaszx.jozwiak@intel.com>; Trahe, Fiona <fiona.trahe@intel.com>
>Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
>
>Hi Shally,
>
>> -----Original Message-----
>> From: Verma, Shally [mailto:Shally.Verma@cavium.com]
>> Sent: Tuesday, March 13, 2018 8:15 AM
>> To: Trahe, Fiona <fiona.trahe@intel.com>; Ahmed Mansour <ahmed.mansour@nxp.com>; dev@dpdk.org
>> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Athreya, Narayana Prasad
>> <NarayanaPrasad.Athreya@cavium.com>; Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu, Sunila
>> <Sunila.Sahu@cavium.com>; Challa, Mahipal <Mahipal.Challa@cavium.com>; Jain, Deepak K
>> <deepak.k.jain@intel.com>; Hemant Agrawal <hemant.agrawal@nxp.com>; Roy Pledge
>> <roy.pledge@nxp.com>; Youri Querry <youri.querry_1@nxp.com>; fiona.trahe@gmail.com; Daly, Lee
>> <lee.daly@intel.com>; Jozwiak, TomaszX <tomaszx.jozwiak@intel.com>
>> Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
>>
>> HI Fiona
>>
>> So I understand we're moving away from mbufs because of its size limitation (64k) and cacheline overhead
>> and their more suitability to n/w applications. Given that, I understand benefit of having another structure
>> to input data but then what is proposal for ipcomp like application where mbuf usage may be a better
>> option? Should we keep support for both (mbuf and this structure) so that apps can use appropriate data
>> structure depending on their requirement.
>[Fiona] An application can use pass buffers from an mbuf or mbuf chain to compressdev by filling in the
>compressdev struct fields with the mbuf meta-data, using rte_pktmbuf_data_len(),
>rte_pktmbuf_mtod(), rte_pktmbuf_mtophys(), etc
>For simplicity I'd prefer to offer only 1 rather than 2 data formats on the API.
>We see storage applications rather than IPComp as the main use-case for compressdev, so would prefer
>to optimise for that.
>Do you think otherwise?

[Shally] Yea. We plan to use it for ipcomp and other such possible n/w apps. So, we envision mbuf support as necessary. So, I think we can add two APIs one which process on rte_comp_op and other on rte_mbufs to make it simpler.

>
>>
>> Further comments, on github.
>>
>> Thanks
>> Shally
>>
>> >-----Original Message-----
>> >From: Trahe, Fiona [mailto:fiona.trahe@intel.com]
>> >Sent: 12 March 2018 21:31
>> >To: Ahmed Mansour <ahmed.mansour@nxp.com>; Verma, Shally <Shally.Verma@cavium.com>;
>> dev@dpdk.org
>> >Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Athreya, Narayana Prasad
>> <NarayanaPrasad.Athreya@cavium.com>;
>> >Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>; Challa, Mahipal
>> ><Mahipal.Challa@cavium.com>; Jain, Deepak K <deepak.k.jain@intel.com>; Hemant Agrawal
>> <hemant.agrawal@nxp.com>; Roy
>> >Pledge <roy.pledge@nxp.com>; Youri Querry <youri.querry_1@nxp.com>; fiona.trahe@gmail.com; Daly,
>> Lee <lee.daly@intel.com>;
>> >Jozwiak, TomaszX <tomaszx.jozwiak@intel.com>
>> >Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
>> >
>> >Hi Shally, Ahmed, and anyone else interested in compressdev,
>> >
>> >I mentioned last week that we've been exploring using something other than mbufs to pass src/dst
>> buffers to compressdev PMDs.
>> >
>> >Reasons:
>> > - mbuf data is limited to 64k-1 in each segment of a chained mbuf. Data for compression
>> >    can be greater and it would add cycles to have to break up into smaller segments.
>> > - data may originate in mbufs, but is more likely, particularly for storage use-cases,  to
>> >    originate in other data structures.
>> > - There's a 2 cache-line overhead for every segment in a chain, most of this data
>> >    is network-related, not needed by compressdev
>> >So moving to a custom structure would minimise memory overhead, remove restriction on 64k-1 size and
>> give more flexibility if
>> >compressdev ever needs any comp-specific meta-data.
>> >
>> >We've come up with a compressdev-specific structure using the struct iovec from sys/uio.h, which is
>> commonly used by storage
>> >applications. This would replace the src and dest mbufs in the  op.
>> >I'll not include the code here - Pablo will push that to github shortly and we'd appreciate review
>> comments there.
>> >https://github.com/pablodelara/dpdk-draft-compressdev
>> >Just posting on the mailing list to give a heads-up and ensure this reaches a wider audience than may see
>> it on github.
>> >
>> >Note : We also considered having no data structures in the op, instead the application
>> >would supply a callback which the PMD would use to retrieve meta-data (virt address, iova, length)
>> >for each next segment as needed. While this is quite flexible and allow the application
>> >to keep its data in its native structures, it's likely to cost more cycles.
>> >So we're not proposing this at the moment, but hope to benchmark it later while the API is still
>> experimental.
>> >
>> >General feedback on direction is welcome here on the mailing list.
>> >For feedback on the details of implementation we would appreciate comments on github.
>> >
>> >Regards,
>> >Fiona.

Re: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative

[Trahe, Fiona]

Hi Shally, Ahmed, et al,

Following internal and community feedback we've decided that there's still too much churn in this. 
We're proposing, in the interest of getting the API out in 18.05, to stick with mbufs - acknowledging
that they're not optimal for storage and we may propose changes in 18.08. 
Compressdev will start as an experimental API in 18.05 - we'll POC and benchmark alternatives 
or API extensions once we get time to do so.

Regards,
Fiona

> -----Original Message-----
> From: Verma, Shally [mailto:Shally.Verma@cavium.com]
> Sent: Wednesday, March 14, 2018 12:51 PM
> To: Trahe, Fiona <fiona.trahe@intel.com>; Ahmed Mansour <ahmed.mansour@nxp.com>; dev@dpdk.org
> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Athreya, Narayana Prasad
> <NarayanaPrasad.Athreya@cavium.com>; Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu, Sunila
> <Sunila.Sahu@cavium.com>; Challa, Mahipal <Mahipal.Challa@cavium.com>; Jain, Deepak K
> <deepak.k.jain@intel.com>; Hemant Agrawal <hemant.agrawal@nxp.com>; Roy Pledge
> <roy.pledge@nxp.com>; Youri Querry <youri.querry_1@nxp.com>; Daly, Lee <lee.daly@intel.com>;
> Jozwiak, TomaszX <tomaszx.jozwiak@intel.com>
> Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
> 
> 
> 
> >-----Original Message-----
> >From: Trahe, Fiona [mailto:fiona.trahe@intel.com]
> >Sent: 13 March 2018 21:22
> >To: Verma, Shally <Shally.Verma@cavium.com>; Ahmed Mansour <ahmed.mansour@nxp.com>;
> dev@dpdk.org
> >Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Athreya, Narayana Prasad
> <NarayanaPrasad.Athreya@cavium.com>;
> >Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>; Challa, Mahipal
> ><Mahipal.Challa@cavium.com>; Jain, Deepak K <deepak.k.jain@intel.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>; Roy
> >Pledge <roy.pledge@nxp.com>; Youri Querry <youri.querry_1@nxp.com>; Daly, Lee
> <lee.daly@intel.com>; Jozwiak, TomaszX
> ><tomaszx.jozwiak@intel.com>; Trahe, Fiona <fiona.trahe@intel.com>
> >Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
> >
> >Hi Shally,
> >
> >> -----Original Message-----
> >> From: Verma, Shally [mailto:Shally.Verma@cavium.com]
> >> Sent: Tuesday, March 13, 2018 8:15 AM
> >> To: Trahe, Fiona <fiona.trahe@intel.com>; Ahmed Mansour <ahmed.mansour@nxp.com>;
> dev@dpdk.org
> >> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Athreya, Narayana Prasad
> >> <NarayanaPrasad.Athreya@cavium.com>; Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu, Sunila
> >> <Sunila.Sahu@cavium.com>; Challa, Mahipal <Mahipal.Challa@cavium.com>; Jain, Deepak K
> >> <deepak.k.jain@intel.com>; Hemant Agrawal <hemant.agrawal@nxp.com>; Roy Pledge
> >> <roy.pledge@nxp.com>; Youri Querry <youri.querry_1@nxp.com>; fiona.trahe@gmail.com; Daly, Lee
> >> <lee.daly@intel.com>; Jozwiak, TomaszX <tomaszx.jozwiak@intel.com>
> >> Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
> >>
> >> HI Fiona
> >>
> >> So I understand we're moving away from mbufs because of its size limitation (64k) and cacheline
> overhead
> >> and their more suitability to n/w applications. Given that, I understand benefit of having another
> structure
> >> to input data but then what is proposal for ipcomp like application where mbuf usage may be a better
> >> option? Should we keep support for both (mbuf and this structure) so that apps can use appropriate
> data
> >> structure depending on their requirement.
> >[Fiona] An application can use pass buffers from an mbuf or mbuf chain to compressdev by filling in the
> >compressdev struct fields with the mbuf meta-data, using rte_pktmbuf_data_len(),
> >rte_pktmbuf_mtod(), rte_pktmbuf_mtophys(), etc
> >For simplicity I'd prefer to offer only 1 rather than 2 data formats on the API.
> >We see storage applications rather than IPComp as the main use-case for compressdev, so would prefer
> >to optimise for that.
> >Do you think otherwise?
> 
> [Shally] Yea. We plan to use it for ipcomp and other such possible n/w apps. So, we envision mbuf support
> as necessary. So, I think we can add two APIs one which process on rte_comp_op and other on rte_mbufs
> to make it simpler.
> 
> >
> >>
> >> Further comments, on github.
> >>
> >> Thanks
> >> Shally
> >>
> >> >-----Original Message-----
> >> >From: Trahe, Fiona [mailto:fiona.trahe@intel.com]
> >> >Sent: 12 March 2018 21:31
> >> >To: Ahmed Mansour <ahmed.mansour@nxp.com>; Verma, Shally <Shally.Verma@cavium.com>;
> >> dev@dpdk.org
> >> >Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Athreya, Narayana Prasad
> >> <NarayanaPrasad.Athreya@cavium.com>;
> >> >Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>; Challa,
> Mahipal
> >> ><Mahipal.Challa@cavium.com>; Jain, Deepak K <deepak.k.jain@intel.com>; Hemant Agrawal
> >> <hemant.agrawal@nxp.com>; Roy
> >> >Pledge <roy.pledge@nxp.com>; Youri Querry <youri.querry_1@nxp.com>; fiona.trahe@gmail.com;
> Daly,
> >> Lee <lee.daly@intel.com>;
> >> >Jozwiak, TomaszX <tomaszx.jozwiak@intel.com>
> >> >Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
> >> >
> >> >Hi Shally, Ahmed, and anyone else interested in compressdev,
> >> >
> >> >I mentioned last week that we've been exploring using something other than mbufs to pass src/dst
> >> buffers to compressdev PMDs.
> >> >
> >> >Reasons:
> >> > - mbuf data is limited to 64k-1 in each segment of a chained mbuf. Data for compression
> >> >    can be greater and it would add cycles to have to break up into smaller segments.
> >> > - data may originate in mbufs, but is more likely, particularly for storage use-cases,  to
> >> >    originate in other data structures.
> >> > - There's a 2 cache-line overhead for every segment in a chain, most of this data
> >> >    is network-related, not needed by compressdev
> >> >So moving to a custom structure would minimise memory overhead, remove restriction on 64k-1 size
> and
> >> give more flexibility if
> >> >compressdev ever needs any comp-specific meta-data.
> >> >
> >> >We've come up with a compressdev-specific structure using the struct iovec from sys/uio.h, which is
> >> commonly used by storage
> >> >applications. This would replace the src and dest mbufs in the  op.
> >> >I'll not include the code here - Pablo will push that to github shortly and we'd appreciate review
> >> comments there.
> >> >https://github.com/pablodelara/dpdk-draft-compressdev
> >> >Just posting on the mailing list to give a heads-up and ensure this reaches a wider audience than may
> see
> >> it on github.
> >> >
> >> >Note : We also considered having no data structures in the op, instead the application
> >> >would supply a callback which the PMD would use to retrieve meta-data (virt address, iova, length)
> >> >for each next segment as needed. While this is quite flexible and allow the application
> >> >to keep its data in its native structures, it's likely to cost more cycles.
> >> >So we're not proposing this at the moment, but hope to benchmark it later while the API is still
> >> experimental.
> >> >
> >> >General feedback on direction is welcome here on the mailing list.
> >> >For feedback on the details of implementation we would appreciate comments on github.
> >> >
> >> >Regards,
> >> >Fiona.

Re: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative

[Ahmed Mansour]

Hi All,

Sticking with mbufs until at least 1805 works for us. We also see
storage as the main use case, but ipcomp maybe an important customer use
case in the future. Nonetheless, I see the mbuf formatting as inherently
external to the compressdev APIs. An application doing ipcomp should
just do the mbuf packaging outside of compressdev. At least that is what
current software implementation of ipcomp do when using zlib.net. I am
assuming that transferring from mbuf to regular buffers and back does
not involve some time consuming work like data copying and such.

Thanks,

Ahmed

On 3/14/2018 2:39 PM, Trahe, Fiona wrote:
> Hi Shally, Ahmed, et al,
>
> Following internal and community feedback we've decided that there's still too much churn in this. 
> We're proposing, in the interest of getting the API out in 18.05, to stick with mbufs - acknowledging
> that they're not optimal for storage and we may propose changes in 18.08. 
> Compressdev will start as an experimental API in 18.05 - we'll POC and benchmark alternatives 
> or API extensions once we get time to do so.
>
> Regards,
> Fiona
>
>> -----Original Message-----
>> From: Verma, Shally [mailto:Shally.Verma@cavium.com]
>> Sent: Wednesday, March 14, 2018 12:51 PM
>> To: Trahe, Fiona <fiona.trahe@intel.com>; Ahmed Mansour <ahmed.mansour@nxp.com>; dev@dpdk.org
>> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Athreya, Narayana Prasad
>> <NarayanaPrasad.Athreya@cavium.com>; Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu, Sunila
>> <Sunila.Sahu@cavium.com>; Challa, Mahipal <Mahipal.Challa@cavium.com>; Jain, Deepak K
>> <deepak.k.jain@intel.com>; Hemant Agrawal <hemant.agrawal@nxp.com>; Roy Pledge
>> <roy.pledge@nxp.com>; Youri Querry <youri.querry_1@nxp.com>; Daly, Lee <lee.daly@intel.com>;
>> Jozwiak, TomaszX <tomaszx.jozwiak@intel.com>
>> Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
>>
>>
>>
>>> -----Original Message-----
>>> From: Trahe, Fiona [mailto:fiona.trahe@intel.com]
>>> Sent: 13 March 2018 21:22
>>> To: Verma, Shally <Shally.Verma@cavium.com>; Ahmed Mansour <ahmed.mansour@nxp.com>;
>> dev@dpdk.org
>>> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Athreya, Narayana Prasad
>> <NarayanaPrasad.Athreya@cavium.com>;
>>> Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>; Challa, Mahipal
>>> <Mahipal.Challa@cavium.com>; Jain, Deepak K <deepak.k.jain@intel.com>; Hemant Agrawal
>> <hemant.agrawal@nxp.com>; Roy
>>> Pledge <roy.pledge@nxp.com>; Youri Querry <youri.querry_1@nxp.com>; Daly, Lee
>> <lee.daly@intel.com>; Jozwiak, TomaszX
>>> <tomaszx.jozwiak@intel.com>; Trahe, Fiona <fiona.trahe@intel.com>
>>> Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
>>>
>>> Hi Shally,
>>>
>>>> -----Original Message-----
>>>> From: Verma, Shally [mailto:Shally.Verma@cavium.com]
>>>> Sent: Tuesday, March 13, 2018 8:15 AM
>>>> To: Trahe, Fiona <fiona.trahe@intel.com>; Ahmed Mansour <ahmed.mansour@nxp.com>;
>> dev@dpdk.org
>>>> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Athreya, Narayana Prasad
>>>> <NarayanaPrasad.Athreya@cavium.com>; Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu, Sunila
>>>> <Sunila.Sahu@cavium.com>; Challa, Mahipal <Mahipal.Challa@cavium.com>; Jain, Deepak K
>>>> <deepak.k.jain@intel.com>; Hemant Agrawal <hemant.agrawal@nxp.com>; Roy Pledge
>>>> <roy.pledge@nxp.com>; Youri Querry <youri.querry_1@nxp.com>; fiona.trahe@gmail.com; Daly, Lee
>>>> <lee.daly@intel.com>; Jozwiak, TomaszX <tomaszx.jozwiak@intel.com>
>>>> Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
>>>>
>>>> HI Fiona
>>>>
>>>> So I understand we're moving away from mbufs because of its size limitation (64k) and cacheline
>> overhead
>>>> and their more suitability to n/w applications. Given that, I understand benefit of having another
>> structure
>>>> to input data but then what is proposal for ipcomp like application where mbuf usage may be a better
>>>> option? Should we keep support for both (mbuf and this structure) so that apps can use appropriate
>> data
>>>> structure depending on their requirement.
>>> [Fiona] An application can use pass buffers from an mbuf or mbuf chain to compressdev by filling in the
>>> compressdev struct fields with the mbuf meta-data, using rte_pktmbuf_data_len(),
>>> rte_pktmbuf_mtod(), rte_pktmbuf_mtophys(), etc
>>> For simplicity I'd prefer to offer only 1 rather than 2 data formats on the API.
>>> We see storage applications rather than IPComp as the main use-case for compressdev, so would prefer
>>> to optimise for that.
>>> Do you think otherwise?
>> [Shally] Yea. We plan to use it for ipcomp and other such possible n/w apps. So, we envision mbuf support
>> as necessary. So, I think we can add two APIs one which process on rte_comp_op and other on rte_mbufs
>> to make it simpler.
>>
>>>> Further comments, on github.
>>>>
>>>> Thanks
>>>> Shally
>>>>
>>>>> -----Original Message-----
>>>>> From: Trahe, Fiona [mailto:fiona.trahe@intel.com]
>>>>> Sent: 12 March 2018 21:31
>>>>> To: Ahmed Mansour <ahmed.mansour@nxp.com>; Verma, Shally <Shally.Verma@cavium.com>;
>>>> dev@dpdk.org
>>>>> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Athreya, Narayana Prasad
>>>> <NarayanaPrasad.Athreya@cavium.com>;
>>>>> Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>; Challa,
>> Mahipal
>>>>> <Mahipal.Challa@cavium.com>; Jain, Deepak K <deepak.k.jain@intel.com>; Hemant Agrawal
>>>> <hemant.agrawal@nxp.com>; Roy
>>>>> Pledge <roy.pledge@nxp.com>; Youri Querry <youri.querry_1@nxp.com>; fiona.trahe@gmail.com;
>> Daly,
>>>> Lee <lee.daly@intel.com>;
>>>>> Jozwiak, TomaszX <tomaszx.jozwiak@intel.com>
>>>>> Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
>>>>>
>>>>> Hi Shally, Ahmed, and anyone else interested in compressdev,
>>>>>
>>>>> I mentioned last week that we've been exploring using something other than mbufs to pass src/dst
>>>> buffers to compressdev PMDs.
>>>>> Reasons:
>>>>> - mbuf data is limited to 64k-1 in each segment of a chained mbuf. Data for compression
>>>>>    can be greater and it would add cycles to have to break up into smaller segments.
>>>>> - data may originate in mbufs, but is more likely, particularly for storage use-cases,  to
>>>>>    originate in other data structures.
>>>>> - There's a 2 cache-line overhead for every segment in a chain, most of this data
>>>>>    is network-related, not needed by compressdev
>>>>> So moving to a custom structure would minimise memory overhead, remove restriction on 64k-1 size
>> and
>>>> give more flexibility if
>>>>> compressdev ever needs any comp-specific meta-data.
>>>>>
>>>>> We've come up with a compressdev-specific structure using the struct iovec from sys/uio.h, which is
>>>> commonly used by storage
>>>>> applications. This would replace the src and dest mbufs in the  op.
>>>>> I'll not include the code here - Pablo will push that to github shortly and we'd appreciate review
>>>> comments there.
>>>>> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpablodelara%2Fdpdk-draft-compressdev&data=02%7C01%7Cahmed.mansour%40nxp.com%7C6a8977f9b3714d58621708d589dae567%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C636566495639618830&sdata=wmFrxeUNyXdxI5%2Fp5gCmyIRfeDnbHebBJXbztqdsMrc%3D&reserved=0
>>>>> Just posting on the mailing list to give a heads-up and ensure this reaches a wider audience than may
>> see
>>>> it on github.
>>>>> Note : We also considered having no data structures in the op, instead the application
>>>>> would supply a callback which the PMD would use to retrieve meta-data (virt address, iova, length)
>>>>> for each next segment as needed. While this is quite flexible and allow the application
>>>>> to keep its data in its native structures, it's likely to cost more cycles.
>>>>> So we're not proposing this at the moment, but hope to benchmark it later while the API is still
>>>> experimental.
>>>>> General feedback on direction is welcome here on the mailing list.
>>>>> For feedback on the details of implementation we would appreciate comments on github.
>>>>>
>>>>> Regards,
>>>>> Fiona.

Re: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative

[Verma, Shally]

@Trahe, Fiona>> We're proposing, in the interest of getting the API out in 18.05, to stick with mbufs - acknowledging
>> that they're not optimal for storage and we may propose changes in 18.08.
[Shally] Sounds good to us too.

@Ahmed Mansour . I am assuming that transferring from mbuf to regular buffers and back does
>not involve some time consuming work like data copying and such.
[Shally] I too assume copying shouldn't be a need and a big no-no. We normally extract and pass buf_addr from mbuf as it is to HW.
So implicit assumption is data memory is dma-able to device.

Thanks
Shally

>-----Original Message-----
>From: Ahmed Mansour [mailto:ahmed.mansour@nxp.com]
>Sent: 15 March 2018 00:32
>To: Trahe, Fiona <fiona.trahe@intel.com>; Verma, Shally <Shally.Verma@cavium.com>; dev@dpdk.org
>Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Athreya, Narayana Prasad <NarayanaPrasad.Athreya@cavium.com>;
>Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>; Challa, Mahipal
><Mahipal.Challa@cavium.com>; Jain, Deepak K <deepak.k.jain@intel.com>; Hemant Agrawal <hemant.agrawal@nxp.com>; Roy
>Pledge <roy.pledge@nxp.com>; Youri Querry <youri.querry_1@nxp.com>; Daly, Lee <lee.daly@intel.com>; Jozwiak, TomaszX
><tomaszx.jozwiak@intel.com>; Alok Makhariya <alok.makhariya@nxp.com>; Shreyansh Jain <shreyansh.jain@nxp.com>
>Subject: Re: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
>
>Hi All,
>
>Sticking with mbufs until at least 1805 works for us. We also see
>storage as the main use case, but ipcomp maybe an important customer use
>case in the future. Nonetheless, I see the mbuf formatting as inherently
>external to the compressdev APIs. An application doing ipcomp should
>just do the mbuf packaging outside of compressdev. At least that is what
>current software implementation of ipcomp do when using zlib.net. I am
>assuming that transferring from mbuf to regular buffers and back does
>not involve some time consuming work like data copying and such.
>
>Thanks,
>
>Ahmed
>
>On 3/14/2018 2:39 PM, Trahe, Fiona wrote:
>> Hi Shally, Ahmed, et al,
>>
>> Following internal and community feedback we've decided that there's still too much churn in this.
>> We're proposing, in the interest of getting the API out in 18.05, to stick with mbufs - acknowledging
>> that they're not optimal for storage and we may propose changes in 18.08.
>> Compressdev will start as an experimental API in 18.05 - we'll POC and benchmark alternatives
>> or API extensions once we get time to do so.
>>
>> Regards,
>> Fiona
>>
>>> -----Original Message-----
>>> From: Verma, Shally [mailto:Shally.Verma@cavium.com]
>>> Sent: Wednesday, March 14, 2018 12:51 PM
>>> To: Trahe, Fiona <fiona.trahe@intel.com>; Ahmed Mansour <ahmed.mansour@nxp.com>; dev@dpdk.org
>>> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Athreya, Narayana Prasad
>>> <NarayanaPrasad.Athreya@cavium.com>; Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu, Sunila
>>> <Sunila.Sahu@cavium.com>; Challa, Mahipal <Mahipal.Challa@cavium.com>; Jain, Deepak K
>>> <deepak.k.jain@intel.com>; Hemant Agrawal <hemant.agrawal@nxp.com>; Roy Pledge
>>> <roy.pledge@nxp.com>; Youri Querry <youri.querry_1@nxp.com>; Daly, Lee <lee.daly@intel.com>;
>>> Jozwiak, TomaszX <tomaszx.jozwiak@intel.com>
>>> Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
>>>
>>>
>>>
>>>> -----Original Message-----
>>>> From: Trahe, Fiona [mailto:fiona.trahe@intel.com]
>>>> Sent: 13 March 2018 21:22
>>>> To: Verma, Shally <Shally.Verma@cavium.com>; Ahmed Mansour <ahmed.mansour@nxp.com>;
>>> dev@dpdk.org
>>>> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Athreya, Narayana Prasad
>>> <NarayanaPrasad.Athreya@cavium.com>;
>>>> Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>; Challa, Mahipal
>>>> <Mahipal.Challa@cavium.com>; Jain, Deepak K <deepak.k.jain@intel.com>; Hemant Agrawal
>>> <hemant.agrawal@nxp.com>; Roy
>>>> Pledge <roy.pledge@nxp.com>; Youri Querry <youri.querry_1@nxp.com>; Daly, Lee
>>> <lee.daly@intel.com>; Jozwiak, TomaszX
>>>> <tomaszx.jozwiak@intel.com>; Trahe, Fiona <fiona.trahe@intel.com>
>>>> Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
>>>>
>>>> Hi Shally,
>>>>
>>>>> -----Original Message-----
>>>>> From: Verma, Shally [mailto:Shally.Verma@cavium.com]
>>>>> Sent: Tuesday, March 13, 2018 8:15 AM
>>>>> To: Trahe, Fiona <fiona.trahe@intel.com>; Ahmed Mansour <ahmed.mansour@nxp.com>;
>>> dev@dpdk.org
>>>>> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Athreya, Narayana Prasad
>>>>> <NarayanaPrasad.Athreya@cavium.com>; Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu, Sunila
>>>>> <Sunila.Sahu@cavium.com>; Challa, Mahipal <Mahipal.Challa@cavium.com>; Jain, Deepak K
>>>>> <deepak.k.jain@intel.com>; Hemant Agrawal <hemant.agrawal@nxp.com>; Roy Pledge
>>>>> <roy.pledge@nxp.com>; Youri Querry <youri.querry_1@nxp.com>; fiona.trahe@gmail.com; Daly, Lee
>>>>> <lee.daly@intel.com>; Jozwiak, TomaszX <tomaszx.jozwiak@intel.com>
>>>>> Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
>>>>>
>>>>> HI Fiona
>>>>>
>>>>> So I understand we're moving away from mbufs because of its size limitation (64k) and cacheline
>>> overhead
>>>>> and their more suitability to n/w applications. Given that, I understand benefit of having another
>>> structure
>>>>> to input data but then what is proposal for ipcomp like application where mbuf usage may be a better
>>>>> option? Should we keep support for both (mbuf and this structure) so that apps can use appropriate
>>> data
>>>>> structure depending on their requirement.
>>>> [Fiona] An application can use pass buffers from an mbuf or mbuf chain to compressdev by filling in the
>>>> compressdev struct fields with the mbuf meta-data, using rte_pktmbuf_data_len(),
>>>> rte_pktmbuf_mtod(), rte_pktmbuf_mtophys(), etc
>>>> For simplicity I'd prefer to offer only 1 rather than 2 data formats on the API.
>>>> We see storage applications rather than IPComp as the main use-case for compressdev, so would prefer
>>>> to optimise for that.
>>>> Do you think otherwise?
>>> [Shally] Yea. We plan to use it for ipcomp and other such possible n/w apps. So, we envision mbuf support
>>> as necessary. So, I think we can add two APIs one which process on rte_comp_op and other on rte_mbufs
>>> to make it simpler.
>>>
>>>>> Further comments, on github.
>>>>>
>>>>> Thanks
>>>>> Shally
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Trahe, Fiona [mailto:fiona.trahe@intel.com]
>>>>>> Sent: 12 March 2018 21:31
>>>>>> To: Ahmed Mansour <ahmed.mansour@nxp.com>; Verma, Shally <Shally.Verma@cavium.com>;
>>>>> dev@dpdk.org
>>>>>> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Athreya, Narayana Prasad
>>>>> <NarayanaPrasad.Athreya@cavium.com>;
>>>>>> Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>; Challa,
>>> Mahipal
>>>>>> <Mahipal.Challa@cavium.com>; Jain, Deepak K <deepak.k.jain@intel.com>; Hemant Agrawal
>>>>> <hemant.agrawal@nxp.com>; Roy
>>>>>> Pledge <roy.pledge@nxp.com>; Youri Querry <youri.querry_1@nxp.com>; fiona.trahe@gmail.com;
>>> Daly,
>>>>> Lee <lee.daly@intel.com>;
>>>>>> Jozwiak, TomaszX <tomaszx.jozwiak@intel.com>
>>>>>> Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
>>>>>>
>>>>>> Hi Shally, Ahmed, and anyone else interested in compressdev,
>>>>>>
>>>>>> I mentioned last week that we've been exploring using something other than mbufs to pass src/dst
>>>>> buffers to compressdev PMDs.
>>>>>> Reasons:
>>>>>> - mbuf data is limited to 64k-1 in each segment of a chained mbuf. Data for compression
>>>>>>    can be greater and it would add cycles to have to break up into smaller segments.
>>>>>> - data may originate in mbufs, but is more likely, particularly for storage use-cases,  to
>>>>>>    originate in other data structures.
>>>>>> - There's a 2 cache-line overhead for every segment in a chain, most of this data
>>>>>>    is network-related, not needed by compressdev
>>>>>> So moving to a custom structure would minimise memory overhead, remove restriction on 64k-1 size
>>> and
>>>>> give more flexibility if
>>>>>> compressdev ever needs any comp-specific meta-data.
>>>>>>
>>>>>> We've come up with a compressdev-specific structure using the struct iovec from sys/uio.h, which is
>>>>> commonly used by storage
>>>>>> applications. This would replace the src and dest mbufs in the  op.
>>>>>> I'll not include the code here - Pablo will push that to github shortly and we'd appreciate review
>>>>> comments there.
>>>>>> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpablodelara%2Fdpdk-draft-
>compressdev&data=02%7C01%7Cahmed.mansour%40nxp.com%7C6a8977f9b3714d58621708d589dae567%7C686ea1d3bc2b4c6fa92cd
>99c5c301635%7C0%7C0%7C636566495639618830&sdata=wmFrxeUNyXdxI5%2Fp5gCmyIRfeDnbHebBJXbztqdsMrc%3D&reserved=0
>>>>>> Just posting on the mailing list to give a heads-up and ensure this reaches a wider audience than may
>>> see
>>>>> it on github.
>>>>>> Note : We also considered having no data structures in the op, instead the application
>>>>>> would supply a callback which the PMD would use to retrieve meta-data (virt address, iova, length)
>>>>>> for each next segment as needed. While this is quite flexible and allow the application
>>>>>> to keep its data in its native structures, it's likely to cost more cycles.
>>>>>> So we're not proposing this at the moment, but hope to benchmark it later while the API is still
>>>>> experimental.
>>>>>> General feedback on direction is welcome here on the mailing list.
>>>>>> For feedback on the details of implementation we would appreciate comments on github.
>>>>>>
>>>>>> Regards,
>>>>>> Fiona.
>

Re: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative

[Trahe, Fiona]

> -----Original Message-----
> From: Verma, Shally [mailto:Shally.Verma@cavium.com]
> Sent: Thursday, March 15, 2018 4:12 AM
> To: Ahmed Mansour <ahmed.mansour@nxp.com>; Trahe, Fiona <fiona.trahe@intel.com>; dev@dpdk.org
> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Athreya, Narayana Prasad
> <NarayanaPrasad.Athreya@cavium.com>; Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu, Sunila
> <Sunila.Sahu@cavium.com>; Challa, Mahipal <Mahipal.Challa@cavium.com>; Jain, Deepak K
> <deepak.k.jain@intel.com>; Hemant Agrawal <hemant.agrawal@nxp.com>; Roy Pledge
> <roy.pledge@nxp.com>; Youri Querry <youri.querry_1@nxp.com>; Daly, Lee <lee.daly@intel.com>;
> Jozwiak, TomaszX <tomaszx.jozwiak@intel.com>; Alok Makhariya <alok.makhariya@nxp.com>; Shreyansh
> Jain <shreyansh.jain@nxp.com>
> Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
> 
> @Trahe, Fiona>> We're proposing, in the interest of getting the API out in 18.05, to stick with mbufs -
> acknowledging
> >> that they're not optimal for storage and we may propose changes in 18.08.
> [Shally] Sounds good to us too.
> 
> @Ahmed Mansour . I am assuming that transferring from mbuf to regular buffers and back does
> >not involve some time consuming work like data copying and such.
> [Shally] I too assume copying shouldn't be a need and a big no-no. We normally extract and pass buf_addr
> from mbuf as it is to HW.
> So implicit assumption is data memory is dma-able to device.
[Fiona] agreed

> 
> Thanks
> Shally
> 
> >-----Original Message-----
> >From: Ahmed Mansour [mailto:ahmed.mansour@nxp.com]
> >Sent: 15 March 2018 00:32
> >To: Trahe, Fiona <fiona.trahe@intel.com>; Verma, Shally <Shally.Verma@cavium.com>; dev@dpdk.org
> >Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Athreya, Narayana Prasad
> <NarayanaPrasad.Athreya@cavium.com>;
> >Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>; Challa, Mahipal
> ><Mahipal.Challa@cavium.com>; Jain, Deepak K <deepak.k.jain@intel.com>; Hemant Agrawal
> <hemant.agrawal@nxp.com>; Roy
> >Pledge <roy.pledge@nxp.com>; Youri Querry <youri.querry_1@nxp.com>; Daly, Lee
> <lee.daly@intel.com>; Jozwiak, TomaszX
> ><tomaszx.jozwiak@intel.com>; Alok Makhariya <alok.makhariya@nxp.com>; Shreyansh Jain
> <shreyansh.jain@nxp.com>
> >Subject: Re: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
> >
> >Hi All,
> >
> >Sticking with mbufs until at least 1805 works for us. We also see
> >storage as the main use case, but ipcomp maybe an important customer use
> >case in the future. Nonetheless, I see the mbuf formatting as inherently
> >external to the compressdev APIs. An application doing ipcomp should
> >just do the mbuf packaging outside of compressdev. At least that is what
> >current software implementation of ipcomp do when using zlib.net. I am
> >assuming that transferring from mbuf to regular buffers and back does
> >not involve some time consuming work like data copying and such.
> >
> >Thanks,
> >
> >Ahmed
> >
> >On 3/14/2018 2:39 PM, Trahe, Fiona wrote:
> >> Hi Shally, Ahmed, et al,
> >>
> >> Following internal and community feedback we've decided that there's still too much churn in this.
> >> We're proposing, in the interest of getting the API out in 18.05, to stick with mbufs - acknowledging
> >> that they're not optimal for storage and we may propose changes in 18.08.
> >> Compressdev will start as an experimental API in 18.05 - we'll POC and benchmark alternatives
> >> or API extensions once we get time to do so.
> >>
> >> Regards,
> >> Fiona
> >>
> >>> -----Original Message-----
> >>> From: Verma, Shally [mailto:Shally.Verma@cavium.com]
> >>> Sent: Wednesday, March 14, 2018 12:51 PM
> >>> To: Trahe, Fiona <fiona.trahe@intel.com>; Ahmed Mansour <ahmed.mansour@nxp.com>;
> dev@dpdk.org
> >>> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Athreya, Narayana Prasad
> >>> <NarayanaPrasad.Athreya@cavium.com>; Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu, Sunila
> >>> <Sunila.Sahu@cavium.com>; Challa, Mahipal <Mahipal.Challa@cavium.com>; Jain, Deepak K
> >>> <deepak.k.jain@intel.com>; Hemant Agrawal <hemant.agrawal@nxp.com>; Roy Pledge
> >>> <roy.pledge@nxp.com>; Youri Querry <youri.querry_1@nxp.com>; Daly, Lee <lee.daly@intel.com>;
> >>> Jozwiak, TomaszX <tomaszx.jozwiak@intel.com>
> >>> Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
> >>>
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: Trahe, Fiona [mailto:fiona.trahe@intel.com]
> >>>> Sent: 13 March 2018 21:22
> >>>> To: Verma, Shally <Shally.Verma@cavium.com>; Ahmed Mansour <ahmed.mansour@nxp.com>;
> >>> dev@dpdk.org
> >>>> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Athreya, Narayana Prasad
> >>> <NarayanaPrasad.Athreya@cavium.com>;
> >>>> Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>; Challa,
> Mahipal
> >>>> <Mahipal.Challa@cavium.com>; Jain, Deepak K <deepak.k.jain@intel.com>; Hemant Agrawal
> >>> <hemant.agrawal@nxp.com>; Roy
> >>>> Pledge <roy.pledge@nxp.com>; Youri Querry <youri.querry_1@nxp.com>; Daly, Lee
> >>> <lee.daly@intel.com>; Jozwiak, TomaszX
> >>>> <tomaszx.jozwiak@intel.com>; Trahe, Fiona <fiona.trahe@intel.com>
> >>>> Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
> >>>>
> >>>> Hi Shally,
> >>>>
> >>>>> -----Original Message-----
> >>>>> From: Verma, Shally [mailto:Shally.Verma@cavium.com]
> >>>>> Sent: Tuesday, March 13, 2018 8:15 AM
> >>>>> To: Trahe, Fiona <fiona.trahe@intel.com>; Ahmed Mansour <ahmed.mansour@nxp.com>;
> >>> dev@dpdk.org
> >>>>> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Athreya, Narayana Prasad
> >>>>> <NarayanaPrasad.Athreya@cavium.com>; Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu,
> Sunila
> >>>>> <Sunila.Sahu@cavium.com>; Challa, Mahipal <Mahipal.Challa@cavium.com>; Jain, Deepak K
> >>>>> <deepak.k.jain@intel.com>; Hemant Agrawal <hemant.agrawal@nxp.com>; Roy Pledge
> >>>>> <roy.pledge@nxp.com>; Youri Querry <youri.querry_1@nxp.com>; fiona.trahe@gmail.com; Daly,
> Lee
> >>>>> <lee.daly@intel.com>; Jozwiak, TomaszX <tomaszx.jozwiak@intel.com>
> >>>>> Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
> >>>>>
> >>>>> HI Fiona
> >>>>>
> >>>>> So I understand we're moving away from mbufs because of its size limitation (64k) and cacheline
> >>> overhead
> >>>>> and their more suitability to n/w applications. Given that, I understand benefit of having another
> >>> structure
> >>>>> to input data but then what is proposal for ipcomp like application where mbuf usage may be a
> better
> >>>>> option? Should we keep support for both (mbuf and this structure) so that apps can use appropriate
> >>> data
> >>>>> structure depending on their requirement.
> >>>> [Fiona] An application can use pass buffers from an mbuf or mbuf chain to compressdev by filling in
> the
> >>>> compressdev struct fields with the mbuf meta-data, using rte_pktmbuf_data_len(),
> >>>> rte_pktmbuf_mtod(), rte_pktmbuf_mtophys(), etc
> >>>> For simplicity I'd prefer to offer only 1 rather than 2 data formats on the API.
> >>>> We see storage applications rather than IPComp as the main use-case for compressdev, so would
> prefer
> >>>> to optimise for that.
> >>>> Do you think otherwise?
> >>> [Shally] Yea. We plan to use it for ipcomp and other such possible n/w apps. So, we envision mbuf
> support
> >>> as necessary. So, I think we can add two APIs one which process on rte_comp_op and other on
> rte_mbufs
> >>> to make it simpler.
> >>>
> >>>>> Further comments, on github.
> >>>>>
> >>>>> Thanks
> >>>>> Shally
> >>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: Trahe, Fiona [mailto:fiona.trahe@intel.com]
> >>>>>> Sent: 12 March 2018 21:31
> >>>>>> To: Ahmed Mansour <ahmed.mansour@nxp.com>; Verma, Shally <Shally.Verma@cavium.com>;
> >>>>> dev@dpdk.org
> >>>>>> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Athreya, Narayana Prasad
> >>>>> <NarayanaPrasad.Athreya@cavium.com>;
> >>>>>> Gupta, Ashish <Ashish.Gupta@cavium.com>; Sahu, Sunila <Sunila.Sahu@cavium.com>; Challa,
> >>> Mahipal
> >>>>>> <Mahipal.Challa@cavium.com>; Jain, Deepak K <deepak.k.jain@intel.com>; Hemant Agrawal
> >>>>> <hemant.agrawal@nxp.com>; Roy
> >>>>>> Pledge <roy.pledge@nxp.com>; Youri Querry <youri.querry_1@nxp.com>; fiona.trahe@gmail.com;
> >>> Daly,
> >>>>> Lee <lee.daly@intel.com>;
> >>>>>> Jozwiak, TomaszX <tomaszx.jozwiak@intel.com>
> >>>>>> Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
> >>>>>>
> >>>>>> Hi Shally, Ahmed, and anyone else interested in compressdev,
> >>>>>>
> >>>>>> I mentioned last week that we've been exploring using something other than mbufs to pass src/dst
> >>>>> buffers to compressdev PMDs.
> >>>>>> Reasons:
> >>>>>> - mbuf data is limited to 64k-1 in each segment of a chained mbuf. Data for compression
> >>>>>>    can be greater and it would add cycles to have to break up into smaller segments.
> >>>>>> - data may originate in mbufs, but is more likely, particularly for storage use-cases,  to
> >>>>>>    originate in other data structures.
> >>>>>> - There's a 2 cache-line overhead for every segment in a chain, most of this data
> >>>>>>    is network-related, not needed by compressdev
> >>>>>> So moving to a custom structure would minimise memory overhead, remove restriction on 64k-1
> size
> >>> and
> >>>>> give more flexibility if
> >>>>>> compressdev ever needs any comp-specific meta-data.
> >>>>>>
> >>>>>> We've come up with a compressdev-specific structure using the struct iovec from sys/uio.h, which
> is
> >>>>> commonly used by storage
> >>>>>> applications. This would replace the src and dest mbufs in the  op.
> >>>>>> I'll not include the code here - Pablo will push that to github shortly and we'd appreciate review
> >>>>> comments there.
> >>>>>>
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpablodelara%2Fd
> pdk-draft-
> >compressdev&data=02%7C01%7Cahmed.mansour%40nxp.com%7C6a8977f9b3714d58621708d589dae
> 567%7C686ea1d3bc2b4c6fa92cd
> >99c5c301635%7C0%7C0%7C636566495639618830&sdata=wmFrxeUNyXdxI5%2Fp5gCmyIRfeDnbHebBJ
> XbztqdsMrc%3D&reserved=0
> >>>>>> Just posting on the mailing list to give a heads-up and ensure this reaches a wider audience than
> may
> >>> see
> >>>>> it on github.
> >>>>>> Note : We also considered having no data structures in the op, instead the application
> >>>>>> would supply a callback which the PMD would use to retrieve meta-data (virt address, iova, length)
> >>>>>> for each next segment as needed. While this is quite flexible and allow the application
> >>>>>> to keep its data in its native structures, it's likely to cost more cycles.
> >>>>>> So we're not proposing this at the moment, but hope to benchmark it later while the API is still
> >>>>> experimental.
> >>>>>> General feedback on direction is welcome here on the mailing list.
> >>>>>> For feedback on the details of implementation we would appreciate comments on github.
> >>>>>>
> >>>>>> Regards,
> >>>>>> Fiona.
> >

Re: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative

[Ananyev, Konstantin]

Hi Fiona,

> Hi Shally, Ahmed, and anyone else interested in compressdev,
> 
> I mentioned last week that we've been exploring using something other than mbufs to pass src/dst buffers to compressdev PMDs.
> 
> Reasons:
>  - mbuf data is limited to 64k-1 in each segment of a chained mbuf. Data for compression
>     can be greater and it would add cycles to have to break up into smaller segments.
>  - data may originate in mbufs, but is more likely, particularly for storage use-cases,  to
>     originate in other data structures.
>  - There's a 2 cache-line overhead for every segment in a chain, most of this data
>     is network-related, not needed by compressdev
> So moving to a custom structure would minimise memory overhead, remove restriction on 64k-1 size and give more flexibility if
> compressdev ever needs any comp-specific meta-data.
> 
> We've come up with a compressdev-specific structure using the struct iovec from sys/uio.h, which is commonly used by storage
> applications. This would replace the src and dest mbufs in the  op.
> I'll not include the code here - Pablo will push that to github shortly and we'd appreciate review comments there.
> https://github.com/pablodelara/dpdk-draft-compressdev
> Just posting on the mailing list to give a heads-up and ensure this reaches a wider audience than may see it on github.
> 
> Note : We also considered having no data structures in the op, instead the application
> would supply a callback which the PMD would use to retrieve meta-data (virt address, iova, length)
> for each next segment as needed. While this is quite flexible and allow the application
> to keep its data in its native structures, it's likely to cost more cycles.

As I said in different thread it will not only slowdown things, but will make it difficult
(if possible at all) for compressdev PMDs to support DPDK MP model.
Konstantin

> So we're not proposing this at the moment, but hope to benchmark it later while the API is still experimental.
> 
> General feedback on direction is welcome here on the mailing list.
> For feedback on the details of implementation we would appreciate comments on github.
> 
> Regards,
> Fiona.