From: "Burakov, Anatoly" <anatoly.burakov@intel.com> To: "Tone Zhang (Arm Technology China)" <Tone.Zhang@arm.com>, "dev@dpdk.org" <dev@dpdk.org> Cc: "Gavin Hu (Arm Technology China)" <Gavin.Hu@arm.com>, Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>, Steve Capper <Steve.Capper@arm.com>, nd <nd@arm.com> Subject: Re: [dpdk-dev] [PATCH v2] pci_vfio: Support 64KB kernel page_size with vfio-pci driver Date: Fri, 16 Nov 2018 10:36:31 +0000 Message-ID: <03d60745-ad79-c469-5c63-3a3d7eadfddd@intel.com> (raw) In-Reply-To: <AM0PR08MB34094D67C6DD620DB9B455A18FDD0@AM0PR08MB3409.eurprd08.prod.outlook.com> On 16-Nov-18 2:34 AM, Tone Zhang (Arm Technology China) wrote: > Hi Anatoly, > > I have some comments. > >> -----Original Message----- >> From: Tone Zhang (Arm Technology China) >> Sent: Thursday, November 15, 2018 8:49 AM >> To: Burakov, Anatoly <anatoly.burakov@intel.com>; dev@dpdk.org >> Cc: Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>; Honnappa >> Nagarahalli <Honnappa.Nagarahalli@arm.com>; Steve Capper >> <Steve.Capper@arm.com>; nd <nd@arm.com> >> Subject: RE: [PATCH v2] pci_vfio: Support 64KB kernel page_size with vfio-pci >> driver >> >> Hi Anatoly, >> >> Sorry for the late response. >> >>> -----Original Message----- >>> From: Burakov, Anatoly <anatoly.burakov@intel.com> >>> Sent: Friday, November 9, 2018 8:15 PM >>> To: Tone Zhang (Arm Technology China) <Tone.Zhang@arm.com>; >>> dev@dpdk.org >>> Cc: Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>; Honnappa >>> Nagarahalli <Honnappa.Nagarahalli@arm.com>; Steve Capper >>> <Steve.Capper@arm.com>; nd <nd@arm.com> >>> Subject: Re: [PATCH v2] pci_vfio: Support 64KB kernel page_size with >>> vfio-pci driver >>> >>> On 09-Nov-18 5:57 AM, tone.zhang wrote: >>>> With a larger PAGE_SIZE it is possible for the MSI table to very >>>> close to the end of the BAR s.t. when we align the start and end of >>>> the MSI table to the PAGE_SIZE, the end offset of the MSI table is >>>> out of the PCI BAR boundary. >>>> >>>> This patch addresses the issue by comparing both the start and the >>>> end offset of the MSI table with the BAR size, and skip the mapping >>>> if it is out of Bar scope. >>>> >>>> The patch fixes the debug log as below: >>>> EAL: Skipping BAR0 >>>> >>>> Signed-off-by: tone.zhang <tone.zhang@arm.com> >>>> Reviewed-by: Gavin Hu <Gavin.Hu@arm.com> >>>> Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com> >>>> Reviewed-by: Steve Capper <Steve.Capper@arm.com> >>>> Reviewed-by: Burakov Anatoly <anatoly.burakov@intel.com> >>> >>> In the future, please don't include my Reviewed tag unless i actually >>> sent one :) >> >> Thanks a lot! Will keep in mind. 😊 >> >>> >>>> --- >>>> drivers/bus/pci/linux/pci_vfio.c | 36 >>>> +++++++++++++++++++++++++++++++--- >>> -- >>>> 1 file changed, 31 insertions(+), 5 deletions(-) >>>> >>>> diff --git a/drivers/bus/pci/linux/pci_vfio.c >>>> b/drivers/bus/pci/linux/pci_vfio.c >>>> index 305cc06..9a0affe 100644 >>>> --- a/drivers/bus/pci/linux/pci_vfio.c >>>> +++ b/drivers/bus/pci/linux/pci_vfio.c >>>> @@ -445,9 +445,11 @@ pci_vfio_mmap_bar(int vfio_dev_fd, struct >>> mapped_pci_resource *vfio_res, >>>> struct pci_msix_table *msix_table = &vfio_res->msix_table; >>>> struct pci_map *bar = &vfio_res->maps[bar_index]; >>>> >>>> - if (bar->size == 0) >>>> + if (bar->size == 0) { >>>> /* Skip this BAR */ >>>> + RTE_LOG(INFO, EAL, "Skipping BAR%d\n", bar_index); >>>> return 0; >>>> + } >>>> >>>> if (msix_table->bar_index == bar_index) { >>>> /* >>>> @@ -456,8 +458,22 @@ pci_vfio_mmap_bar(int vfio_dev_fd, struct >>> mapped_pci_resource *vfio_res, >>>> */ >>>> uint32_t table_start = msix_table->offset; >>>> uint32_t table_end = table_start + msix_table->size; >>>> - table_end = (table_end + ~PAGE_MASK) & PAGE_MASK; >>>> - table_start &= PAGE_MASK; >>>> + table_end = RTE_ALIGN(table_end, PAGE_SIZE); >>>> + table_start = RTE_ALIGN(table_start, PAGE_SIZE); >>>> + /* after rounding to PAGE_SIZE, it is over the bar->size, >>>> + * fall back to the MSI-X table offset in the bar and >>>> + * align with PAGE_SIZE. >>>> + */ >>> >>> Minor nitpick - wording of comment could be better, for example: >>> >>> if page-aligned start of MSI-X table is beyond BAR size, shrink the >>> mapping size to MSI-X table start address. >>> >>> Also, probably needs newline before comment. >>> >> >> Will update the code in next version. Thanks! >> >>>> + if (table_start >= bar->size) { >>>> + table_start = RTE_ALIGN_FLOOR(msix_table->offset, >>>> + PAGE_SIZE); >>>> + /* after aligning with PAGE_SIZE, if it is less than >>>> + * the MSI-X table offset, continue falling back to >>>> + * the actual MSI-X table offset in the bar. >>>> + */ >>> >>> Same here, wording could probably be improved. Suggested rewording: >>> >>> If MSI-X table address, floor-aligned by page size, is lower than >>> actual MSI-X table offset, fall back to using MSI-X table offset as table start. >>> >>> Now that i think of it, this could really be expressed like this: >>> >>> uint32_t aligned = RTE_ALIGN_FLOOR(msix_table->offset, PAGE_SIZE); >>> table_start = RTE_MAX(aligned, msix_table_offset); >>> >>> I believe this would be much clearer. >>> >> >> Will update the patch. >> > > When enter the judgement, it implies the "msix_table->offset" is NOT page size aligned, I think we can replace the code in the judgement with one line: table_start = msix_table->offset; > It looks more simple. What's your opinion? Thanks! Agree, what was i thinking :D > >>>> + if (table_start < msix_table->offset) >>>> + table_start = msix_table->offset; >>>> + } >>>> >>>> if (table_start == 0 && table_end >= bar->size) { >>>> /* Cannot map this BAR */ >>>> @@ -469,8 +485,18 @@ pci_vfio_mmap_bar(int vfio_dev_fd, struct >>>> mapped_pci_resource *vfio_res, >>>> >>>> memreg[0].offset = bar->offset; >>>> memreg[0].size = table_start; >>>> - memreg[1].offset = bar->offset + table_end; >>>> - memreg[1].size = bar->size - table_end; >>>> + if (bar->size < table_end) { >>>> + /* >>>> + * after rounding to PAGE_SIZE we don't have any space >>>> + * left after the MSI table, so don't try and map it. >>>> + */ >>> >>> Suggested rewording: >>> >>> If MSI-X table end is beyond BAR end, don't attempt to perform second >> mapping. >>> >> >> Thanks a lot. Will update. >> >>>> + memreg[1].offset = 0; >>>> + memreg[1].size = 0; >>>> + } >>>> + else { >>>> + memreg[1].offset = bar->offset + table_end; >>>> + memreg[1].size = bar->size - table_end; >>>> + } >>>> >>>> RTE_LOG(DEBUG, EAL, >>>> "Trying to map BAR%d that contains the MSI-X " >>>> >>> >>> However, the patch can go in as is if needed, so >>> >>> Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com> >>> >> >> Thanks! 😉 >> >>> -- >>> Thanks, >>> Anatoly >> >> Br, >> Tone > > Br, > Tone > -- Thanks, Anatoly
next prev parent reply other threads:[~2018-11-16 10:36 UTC|newest] Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-10-24 2:20 [dpdk-dev] [PATCH] " tone.zhang 2018-10-24 9:09 ` Burakov, Anatoly 2018-11-01 2:33 ` Tone Zhang (Arm Technology China) 2018-11-01 10:01 ` Burakov, Anatoly [not found] ` <DB7PR08MB33859242951014EF340C897AE9C80@DB7PR08MB3385.eurprd08.prod.outlook.com> 2018-11-03 5:46 ` Tone Zhang (Arm Technology China) 2018-11-06 11:03 ` Burakov, Anatoly 2018-11-07 4:55 ` Tone Zhang (Arm Technology China) 2018-11-07 10:12 ` Burakov, Anatoly 2018-11-08 0:45 ` Tone Zhang (Arm Technology China) 2018-11-09 5:57 ` [dpdk-dev] [PATCH v2] " tone.zhang 2018-11-09 12:15 ` Burakov, Anatoly 2018-11-15 0:49 ` Tone Zhang (Arm Technology China) 2018-11-16 2:34 ` Tone Zhang (Arm Technology China) 2018-11-16 10:36 ` Burakov, Anatoly [this message] 2018-11-19 2:37 ` [dpdk-dev] [PATCH v3] " tone.zhang 2018-12-03 7:25 ` Tone Zhang (Arm Technology China) 2018-12-10 11:40 ` Burakov, Anatoly 2018-12-10 11:45 ` Burakov, Anatoly 2018-12-10 15:55 ` Stephen Hemminger 2018-12-12 10:49 ` Tone Zhang (Arm Technology China) 2018-12-12 10:48 ` Tone Zhang (Arm Technology China) 2018-12-12 11:25 ` [dpdk-dev] [PATCH v4] " tone.zhang 2018-12-12 11:27 ` Burakov, Anatoly 2018-12-19 23:05 ` Thomas Monjalon
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=03d60745-ad79-c469-5c63-3a3d7eadfddd@intel.com \ --to=anatoly.burakov@intel.com \ --cc=Gavin.Hu@arm.com \ --cc=Honnappa.Nagarahalli@arm.com \ --cc=Steve.Capper@arm.com \ --cc=Tone.Zhang@arm.com \ --cc=dev@dpdk.org \ --cc=nd@arm.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
DPDK patches and discussions This inbox may be cloned and mirrored by anyone: git clone --mirror https://inbox.dpdk.org/dev/0 dev/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 dev dev/ https://inbox.dpdk.org/dev \ dev@dpdk.org public-inbox-index dev Example config snippet for mirrors. Newsgroup available over NNTP: nntp://inbox.dpdk.org/inbox.dpdk.dev AGPL code for this site: git clone https://public-inbox.org/public-inbox.git