Re: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative

["Trahe, Fiona" <fiona.trahe@intel.com>]

Hi Shally, Ahmed, and anyone else interested in compressdev,

I mentioned last week that we've been exploring using something other than mbufs to pass src/dst buffers to compressdev PMDs.

Reasons:
 - mbuf data is limited to 64k-1 in each segment of a chained mbuf. Data for compression
    can be greater and it would add cycles to have to break up into smaller segments.
 - data may originate in mbufs, but is more likely, particularly for storage use-cases,  to 
    originate in other data structures.
 - There's a 2 cache-line overhead for every segment in a chain, most of this data
    is network-related, not needed by compressdev
So moving to a custom structure would minimise memory overhead, remove restriction on 64k-1 size and give more flexibility if compressdev ever needs any comp-specific meta-data.

We've come up with a compressdev-specific structure using the struct iovec from sys/uio.h, which is commonly used by storage applications. This would replace the src and dest mbufs in the  op.
I'll not include the code here - Pablo will push that to github shortly and we'd appreciate review comments there.
https://github.com/pablodelara/dpdk-draft-compressdev
Just posting on the mailing list to give a heads-up and ensure this reaches a wider audience than may see it on github.  

Note : We also considered having no data structures in the op, instead the application
would supply a callback which the PMD would use to retrieve meta-data (virt address, iova, length) 
for each next segment as needed. While this is quite flexible and allow the application
to keep its data in its native structures, it's likely to cost more cycles.
So we're not proposing this at the moment, but hope to benchmark it later while the API is still experimental.

General feedback on direction is welcome here on the mailing list.
For feedback on the details of implementation we would appreciate comments on github.

Regards,
Fiona.