From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from out1134-243.mail.aliyun.com (out1134-243.mail.aliyun.com [42.120.134.243]) by dpdk.org (Postfix) with ESMTP id C03E57E9D for ; Tue, 11 Nov 2014 04:43:42 +0100 (CET) X-Alimail-AntiSpam: AC=CONTINUE; BC=0.03870059|-1; FP=0|0|0|0|0|0|0|0; HT=r46d02012; MF=liang.xu@cinfotech.cn; PH=DW; RN=2; RT=2; SR=0; Received: from WS-web (liang.xu@cinfotech.cn[203.110.175.218]) by r41g03043.xy2.aliyun.com at Tue, 11 Nov 2014 11:53:22 +0800 Date: Tue, 11 Nov 2014 11:53:22 +0800 From: "XU Liang" To: "=?UTF-8?B?QnVyYWtvdiwgQW5hdG9seQ==?=" , "dev@dpdk.org" Message-ID: X-Mailer: Alimail-Mailagent revision 2667797 MIME-Version: 1.0 References: <1415347284-15468-1-git-send-email-liang.xu@cinfotech.cn> <1415619272-8281-1-git-send-email-anatoly.burakov@intel.com>, C6ECDF3AB251BE4894318F4E4512369780C07EEB@IRSMSX109.ger.corp.intel.com In-Reply-To: C6ECDF3AB251BE4894318F4E4512369780C07EEB@IRSMSX109.ger.corp.intel.com x-aliyun-mail-creator: Webmail4_2670074_hLSTW96aWxsYS81LjAgKFdpbmRvd3MgTlQgNi4xOyBXT1c2NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzMzLjAuMTc1MC4xNDkgU2FmYXJpLzUzNy4zNg==2I Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] =?utf-8?q?=5BPATCH_v7=5D_eal=3A_map_PCI_memory_resourc?= =?utf-8?q?es_after_hugepages?= X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: XU Liang List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Nov 2014 03:43:45 -0000 I had finished some tests. The patch works fine. My tests are included :* sing= le process=C2=A0 + uio + vfio * single process=C2=A0 + uio + vfio + base-virta= ddr * multiple processes + uio + vfio *=C2=A0multiple processes + uio + vfio += base-virtaddr My unlucky multiple process application still got error without= =C2=A0base-virtaddr when initial hugepages. See the attchments: primary.txt an= d secondary.txt.With=C2=A0base-virtaddr the patch worked, both hugepages and p= ci resources were mapped into base-virtaddr, My application is happy.=C2=A0See= the attchments: base-virtaddr_primary.txt and base-virtaddr_secondary.txt. -= -----------------------------------------------------------------From:Burakov,= Anatoly Time:2014 Nov 10 (Mon) 21 : 34To:Burakov, = Anatoly , dev@dpdk.org Subject:Re: [d= pdk-dev] [PATCH v7] eal: map PCI memory resources after hugepages=0ANak, there= are issues with the patch. There is another patch already, but I'll submit it= whenever Liang verifies it works with his setup.=0A=0AThanks,=0AAnatoly=0A=0A= -----Original Message-----=0AFrom: dev [mailto:dev-bounces@dpdk.org] On Behalf= Of Anatoly Burakov=0ASent: Monday, November 10, 2014 11:35 AM=0ATo: dev@dpdk.= org=0ASubject: [dpdk-dev] [PATCH v7] eal: map PCI memory resources after hugep= ages=0A=0AMulti-process DPDK application must mmap hugepages and pci resources= =0Ainto the same virtual address space. By default the virtual addresses=0Aare= chosen by the primary process automatically when calling the mmap.=0ABut some= times the chosen virtual addresses aren't usable in secondary=0Aprocess - for = example, secondary process is linked with more libraries=0Athan primary proces= s, and the library occupies the same address space=0Athat the primary process = has requested for PCI mappings.=0A=0AThis patch makes EAL map PCI BARs right a= fter the hugepages (instead of=0Alocation chosen by mmap) in virtual memory.=0A= =0ASigned-off-by: Anatoly Burakov =0ASigned-off-by:= Liang Xu =0A---=0A lib/librte_eal/linuxapp/eal/eal_pci= .c | 19 +++++++++++++++++++=0A lib/librte_eal/linuxapp/eal/eal_pc= i_uio.c | 9 ++++++++-=0A lib/librte_eal/linuxapp/eal/eal_pci_vfio.c = | 13 +++++++++++--=0A lib/librte_eal/linuxapp/eal/include/eal_pci_init= .h | 6 ++++++=0A 4 files changed, 44 insertions(+), 3 deletions(-)=0A=0Adiff = --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/ea= l_pci.c=0Aindex 5fe3961..dae8739 100644=0A--- a/lib/librte_eal/linuxapp/eal/ea= l_pci.c=0A+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c=0A@@ -97,6 +97,25 @@ err= or:=0A return -1;=0A }=0A =0A+void *=0A+pci_find_max_end_va(void)=0A+{=0A+ co= nst struct rte_memseg *seg =3D rte_eal_get_physmem_layout();=0A+ const struct = rte_memseg *last =3D seg;=0A+ unsigned i =3D 0;=0A+=0A+ for (i =3D 0; i < RTE_= MAX_MEMSEG; i++, seg++) {=0A+ if (seg->addr =3D=3D NULL)=0A+ break;=0A+=0A+= if (seg->addr > last->addr)=0A+ last =3D seg;=0A+=0A+ }=0A+ return RTE_PTR= _ADD(last->addr, last->len);=0A+}=0A+=0A+=0A /* map a particular resource from= a file */=0A void *=0A pci_map_resource(void *requested_addr, int fd, off_t o= ffset, size_t size)=0Adiff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b= /lib/librte_eal/linuxapp/eal/eal_pci_uio.c=0Aindex 7e62266..5090bf1 100644=0A-= -- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c=0A+++ b/lib/librte_eal/linuxapp= /eal/eal_pci_uio.c=0A@@ -48,6 +48,8 @@=0A =0A static int pci_parse_sysfs_value= (const char *filename, uint64_t *val);=0A =0A+void *pci_map_addr =3D NULL;=0A+= =0A =0A #define OFF_MAX ((uint64_t)(off_t)-1)=0A static int=0A@@ = -371,10 +373,15 @@ pci_uio_map_resource(struct rte_pci_device *dev)=0A if (= maps[j].addr !=3D NULL)=0A fail =3D 1;=0A else {=0A- mapaddr =3D pci= _map_resource(NULL, fd, (off_t)offset,=0A+ if (pci_map_addr =3D=3D NULL)=0A= + pci_map_addr =3D pci_find_max_end_va();=0A+=0A+ mapaddr =3D pci_map_r= esource(pci_map_addr, fd, (off_t)offset,=0A (size_t)maps[j].size);=0A = if (mapaddr =3D=3D NULL)=0A fail =3D 1;=0A+=0A+ pci_map_addr =3D RTE= _PTR_ADD(pci_map_addr, maps[j].size);=0A }=0A =0A if (fail) {=0Adiff --g= it a/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c b/lib/librte_eal/linuxapp/eal/= eal_pci_vfio.c=0Aindex c776ddc..fb6ee7a 100644=0A--- a/lib/librte_eal/linuxapp= /eal/eal_pci_vfio.c=0A+++ b/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c=0A@@ -7= 20,8 +720,17 @@ pci_vfio_map_resource(struct rte_pci_device *dev)=0A if (i =3D= =3D msix_bar)=0A continue;=0A =0A- bar_addr =3D pci_map_resource(maps[i].a= ddr, vfio_dev_fd, reg.offset,=0A- reg.size);=0A+ if (internal_config.proce= ss_type =3D=3D RTE_PROC_PRIMARY) {=0A+ if (pci_map_addr =3D=3D NULL)=0A+ = pci_map_addr =3D pci_find_max_end_va();=0A+=0A+ bar_addr =3D pci_map_resourc= e(pci_map_addr, vfio_dev_fd, reg.offset,=0A+ reg.size);=0A+ pci_map_addr= =3D RTE_PTR_ADD(pci_map_addr, reg.size);=0A+ } else {=0A+ bar_addr =3D pci= _map_resource(maps[i].addr, vfio_dev_fd, reg.offset,=0A+ reg.size);=0A+ }= =0A =0A if (bar_addr =3D=3D NULL) {=0A RTE_LOG(ERR, EAL, " %s mapping BA= R%i failed: %s\n", pci_addr, i,=0Adiff --git a/lib/librte_eal/linuxapp/eal/inc= lude/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/include/eal_pci_init.h=0Aind= ex d758bee..1070eb8 100644=0A--- a/lib/librte_eal/linuxapp/eal/include/eal_pci= _init.h=0A+++ b/lib/librte_eal/linuxapp/eal/include/eal_pci_init.h=0A@@ -59,6 = +59,12 @@ struct mapped_pci_resource {=0A TAILQ_HEAD(mapped_pci_res_list, mapp= ed_pci_resource);=0A extern struct mapped_pci_res_list *pci_res_list;=0A =0A+/= *=0A+ * Helper function to map PCI resources right after hugepages in virtual = memory=0A+ */=0A+extern void *pci_map_addr;=0A+void *pci_find_max_end_va(void)= ;=0A+=0A void *pci_map_resource(void *requested_addr, int fd, off_t offset,=0A= size_t size);=0A =0A-- =0A1.8.1.4 >From jijiang.liu@intel.com Tue Nov 11 06:21:09 2014 Return-Path: Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 5836C7F0D for ; Tue, 11 Nov 2014 06:21:09 +0100 (CET) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga102.fm.intel.com with ESMTP; 10 Nov 2014 21:30:56 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.07,358,1413270000"; d="scan'208";a="620365999" Received: from kmsmsx153.gar.corp.intel.com ([172.21.73.88]) by fmsmga001.fm.intel.com with ESMTP; 10 Nov 2014 21:30:55 -0800 Received: from shsmsx104.ccr.corp.intel.com (10.239.110.15) by KMSMSX153.gar.corp.intel.com (172.21.73.88) with Microsoft SMTP Server (TLS) id 14.3.195.1; Tue, 11 Nov 2014 13:29:19 +0800 Received: from shsmsx101.ccr.corp.intel.com ([169.254.1.130]) by SHSMSX104.ccr.corp.intel.com ([169.254.5.62]) with mapi id 14.03.0195.001; Tue, 11 Nov 2014 13:29:19 +0800 From: "Liu, Jijiang" To: Olivier MATZ Thread-Topic: [dpdk-dev] [PATCH v8 10/10] app/testpmd:test VxLAN Tx checksum offload Thread-Index: AQHP8Yux0VUY397yik2xjVM3DjesSpxPpsiAgAGaY5CAABv8AIAIEl8ggAAq4wCAAVxhEA== Date: Tue, 11 Nov 2014 05:29:18 +0000 Message-ID: <1ED644BD7E0A5F4091CF203DAFB8E4CC01D8F7A7@SHSMSX101.ccr.corp.intel.com> References: <1414376006-31402-1-git-send-email-jijiang.liu@intel.com> <1414376006-31402-11-git-send-email-jijiang.liu@intel.com> <54588BF7.309@6wind.com> <1ED644BD7E0A5F4091CF203DAFB8E4CC01D8510E@SHSMSX101.ccr.corp.intel.com> <5459FBB2.1040408@6wind.com> <1ED644BD7E0A5F4091CF203DAFB8E4CC01D8F399@SHSMSX101.ccr.corp.intel.com> <5460E512.1050609@6wind.com> In-Reply-To: <5460E512.1050609@6wind.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] [PATCH v8 10/10] app/testpmd:test VxLAN Tx checksum offload X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Nov 2014 05:21:09 -0000 > -----Original Message----- > From: Olivier MATZ [mailto:olivier.matz@6wind.com] > Sent: Tuesday, November 11, 2014 12:17 AM > To: Liu, Jijiang > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH v8 10/10] app/testpmd:test VxLAN Tx checks= um > offload >=20 > Hi Jijiang, >=20 > On 11/10/2014 07:03 AM, Liu, Jijiang wrote: > >> Another thing is surprising me. > >> > >> - if PKT_TX_VXLAN_CKSUM is not set (legacy use case), then the > >> driver use l2_len and l3_len to offload inner IP/UDP/TCP checksums. > > If the flag is not set, and imply that it is not VXLAN packet, and do > > TX checksum offload as regular packet. > > > >> - if PKT_TX_VXLAN_CKSUM is set, then the driver has to use > >> inner_l{23}_len instead of l{23}_len for the same operation. > > Your understanding is not fully correct. > > The l{23}_len is still used for TX checksum offload, please refer to > i40e_txd_enable_checksum() implementation. >=20 > This fields are part of public mbuf API. You cannot say to refer to i40e = PMD code > to understand how to use it. >=20 > >> Adding PKT_TX_VXLAN_CKSUM changes the semantic of l2_len and l3_len. > >> To fix this, I suggest to remove the new fields inner_l{23}_len then > >> add outer_l{23}_len instead. Therefore, the semantic of l2_len and > >> l3_len would not change, and a driver would always use the same field = for a > specific offload. > > Oh... >=20 > Does it mean you agree? I don't agree to change inner_l{23}_len the name. The reason is that using the "inner" word means incoming packet is tunnel= ing packet or encapsulation packet. if we add "outer"{2,3}_len , which will cause confusion when processing n= on-tunneling packet. > >> For my TSO development, I will follow the current semantic. > > For TSO, you still can use l{2,3} _len . > > When I develop tunneling TSO, I will use inner_l3_len/inner_l4_len. >=20 > I've just submitted a first version, please feel free to comment it. >=20 >=20 > Regards, > Olivier