[dpdk-dev] Fwd: dpdk-iokit: Turn dpdk into the IO field.

[dpdk-dev] Fwd: dpdk-iokit: Turn dpdk into the IO field.

[March Jane]

Hi,
  dpdk folks,
    I am working on a project which is offering a complete IO stack for dpdk, particularly a SCSI target stack in dpdk — dpdk-iokit, once I finished my initial code, will upload to github. I would like to know if there is any other people is working on that purpose? 
    

  - march

[dpdk-dev] Scenario of dpdk-iokit //Re: Fwd: dpdk-iokit: Turn dpdk into the IO field.

[March Jane]

 — Nowadays dilemmas in enterprise-class storage system & motivation -- 
 
     	The model of most storage system is ‘Front-end cards + CPU + back-end magnetic media hard drive’, in such system, the hard drive is very slow in terms of its capability in random accessing - roughly 200IO per second, in contrast, the processor is very fast, therefore many software stack can tolerate waste of CPU cycle for exploiting capabilities of hard drive. 
	
	In addition, regularly software for SAN is developed under kernel space in order to achieve low latency - few milliseconds per request, and IO pressure in scenario of heavy workload. Usually software in user-space with POSIX is slow. However kernel-space developing is nightmare for engineers, even we have approaches to simulate these code under user-space. 
     
    However, so far flash is coming to popular, the gap between CPU and media is overturned, a single flash card can be easy to reach 1M iops, if plug 10 such cards inside a server, the processor is hard to back. Thus, today’s challenge in flash storage is to exploit the capability of processor, ironically.  In addition to this purpose, move to user-space is also a motivation, if move to user-space is OS-bypass rather than moving to POSIX.


 	Best

	- March






On Nov 14, 2013, at 8:03 PM, St Leger, Jim <jim.st.leger@intel.com> wrote:

> March:
> How long have you been working on the project?
> What is your goal for final implementation?
> Do you have any progress so far?
> Jim
>  
> From: March Jane <unix.march@gmail.com>
> Date: November 12, 2013 at 2:27:37 AM PST
> To: <dev@dpdk.org>
> Subject: [dpdk-dev] Fwd: dpdk-iokit: Turn dpdk into the IO field.
> 
> Hi,
>  dpdk folks,
>    I am working on a project which is offering a complete IO stack for dpdk, particularly a SCSI target stack in dpdk — dpdk-iokit, once I finished my initial code, will upload to github. I would like to know if there is any other people is working on that purpose? 
> 
> 
>  - march
> 

Re: [dpdk-dev] Scenario of dpdk-iokit //Re: Fwd: dpdk-iokit: Turn dpdk into the IO field.

[March Jane]

[-- Attachment #1: Type: text/plain, Size: 293 bytes --]


 	Gratitude to reply. 
		
	FCP and Full-offload iSCSI will be nice, for example Qlogic 16Gbps FCP.
	
	But IO scenario is very different from IP scenario as my understanding.

	March	

	BTW: Do you have people at Bay Area?


	PS:
		Already finished header file for dpdk-iokit/libiokit_sctgt

	

[-- Attachment #2: Type: text/plain, Size: 4 bytes --]


	
	

[-- Attachment #3: Type: text/plain, Size: 2048 bytes --]





On Nov 19, 2013, at 5:45 PM, Vincent JARDIN <vincent.jardin@6wind.com> wrote:

> (off list since it coul become a troll ;) )
> 
> At 6WIND, we developed a librte_crypto as a generic framework to manage any crypto framework:
>  - Intel's QuickAssist
>  - Cavium' Nitrox II
>  - AES/NI SW crypto
> 
> it was required in order to manage high rate of PCI IOs for Cryptos (IPsec, SSL, etc...)
> 
> So, after the librte_pmd_mlx4, Virtio, Vmxnet3, that's more than 6 ultra low latency/very efficient drivers that we added into the DPDK and promoted.
> 
> Which storage drivers would you foresee first to be run in userland?
> 
> Best regards,
> 
> 
> On 19/11/2013 02:22, March Jane wrote:
>> 
>>  — Nowadays dilemmas in enterprise-class storage system & motivation --
>> 
>>      	The model of most storage system is ‘Front-end cards + CPU + back-end magnetic media hard drive’, in such system, the hard drive is very slow in terms of its capability in random accessing - roughly 200IO per second, in contrast, the processor is very fast, therefore many software stack can tolerate waste of CPU cycle for exploiting capabilities of hard drive.
>> 	
>> 	In addition, regularly software for SAN is developed under kernel space in order to achieve low latency - few milliseconds per request, and IO pressure in scenario of heavy workload. Usually software in user-space with POSIX is slow. However kernel-space developing is nightmare for engineers, even we have approaches to simulate these code under user-space.
>> 
>>     However, so far flash is coming to popular, the gap between CPU and media is overturned, a single flash card can be easy to reach 1M iops, if plug 10 such cards inside a server, the processor is hard to back. Thus, today’s challenge in flash storage is to exploit the capability of processor, ironically.  In addition to this purpose, move to user-space is also a motivation, if move to user-space is OS-bypass rather than moving to POSIX.
>> 
>> 
>>  	Best
>> 
>> 	- March
>> 
>> 
>> 

Re: [dpdk-dev] Scenario of dpdk-iokit //Re: Fwd: dpdk-iokit: Turn dpdk into the IO field.

[Vincent JARDIN]

so on the list if you prefer to do so ;) did you try a "perf top" on 
your system under load?


On 19/11/2013 10:56, March Jane wrote:
>
>   	Gratitude to reply.
> 		
> 	FCP and Full-offload iSCSI will be nice, for example Qlogic 16Gbps FCP.
> 	
> 	But IO scenario is very different from IP scenario as my understanding.
>
> 	March	
>
> 	BTW: Do you have people at Bay Area?
>
>
> 	PS:
> 		Already finished header file for dpdk-iokit/libiokit_sctgt
>
> 	
>
>
>
>
> 	
> 	
>
>
>
>
>
>
>
> On Nov 19, 2013, at 5:45 PM, Vincent JARDIN <vincent.jardin@6wind.com> wrote:
>
>> (off list since it coul become a troll ;) )
>>
>> At 6WIND, we developed a librte_crypto as a generic framework to manage any crypto framework:
>>   - Intel's QuickAssist
>>   - Cavium' Nitrox II
>>   - AES/NI SW crypto
>>
>> it was required in order to manage high rate of PCI IOs for Cryptos (IPsec, SSL, etc...)
>>
>> So, after the librte_pmd_mlx4, Virtio, Vmxnet3, that's more than 6 ultra low latency/very efficient drivers that we added into the DPDK and promoted.
>>
>> Which storage drivers would you foresee first to be run in userland?
>>
>> Best regards,
>>
>>
>> On 19/11/2013 02:22, March Jane wrote:
>>>
>>>   — Nowadays dilemmas in enterprise-class storage system & motivation --
>>>
>>>       	The model of most storage system is ‘Front-end cards + CPU + back-end magnetic media hard drive’, in such system, the hard drive is very slow in terms of its capability in random accessing - roughly 200IO per second, in contrast, the processor is very fast, therefore many software stack can tolerate waste of CPU cycle for exploiting capabilities of hard drive.
>>> 	
>>> 	In addition, regularly software for SAN is developed under kernel space in order to achieve low latency - few milliseconds per request, and IO pressure in scenario of heavy workload. Usually software in user-space with POSIX is slow. However kernel-space developing is nightmare for engineers, even we have approaches to simulate these code under user-space.
>>>
>>>      However, so far flash is coming to popular, the gap between CPU and media is overturned, a single flash card can be easy to reach 1M iops, if plug 10 such cards inside a server, the processor is hard to back. Thus, today’s challenge in flash storage is to exploit the capability of processor, ironically.  In addition to this purpose, move to user-space is also a motivation, if move to user-space is OS-bypass rather than moving to POSIX.
>>>
>>>
>>>   	Best
>>>
>>> 	- March
>>>
>>>
>>>
>