DPDK patches and discussions
 help / color / mirror / Atom feed
From: "Burakov, Anatoly" <anatoly.burakov@intel.com>
To: "Tone Zhang (Arm Technology China)" <Tone.Zhang@arm.com>,
	"dev@dpdk.org" <dev@dpdk.org>
Cc: "Gavin Hu (Arm Technology China)" <Gavin.Hu@arm.com>,
	Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>,
	Steve Capper <Steve.Capper@arm.com>, nd <nd@arm.com>
Subject: Re: [dpdk-dev] [PATCH v2] pci_vfio: Support 64KB kernel page_size with vfio-pci driver
Date: Fri, 16 Nov 2018 10:36:31 +0000
Message-ID: <03d60745-ad79-c469-5c63-3a3d7eadfddd@intel.com> (raw)
In-Reply-To: <AM0PR08MB34094D67C6DD620DB9B455A18FDD0@AM0PR08MB3409.eurprd08.prod.outlook.com>

On 16-Nov-18 2:34 AM, Tone Zhang (Arm Technology China) wrote:
> Hi Anatoly,
> 
> I have some comments.
> 
>> -----Original Message-----
>> From: Tone Zhang (Arm Technology China)
>> Sent: Thursday, November 15, 2018 8:49 AM
>> To: Burakov, Anatoly <anatoly.burakov@intel.com>; dev@dpdk.org
>> Cc: Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>; Honnappa
>> Nagarahalli <Honnappa.Nagarahalli@arm.com>; Steve Capper
>> <Steve.Capper@arm.com>; nd <nd@arm.com>
>> Subject: RE: [PATCH v2] pci_vfio: Support 64KB kernel page_size with vfio-pci
>> driver
>>
>> Hi Anatoly,
>>
>> Sorry for the late response.
>>
>>> -----Original Message-----
>>> From: Burakov, Anatoly <anatoly.burakov@intel.com>
>>> Sent: Friday, November 9, 2018 8:15 PM
>>> To: Tone Zhang (Arm Technology China) <Tone.Zhang@arm.com>;
>>> dev@dpdk.org
>>> Cc: Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>; Honnappa
>>> Nagarahalli <Honnappa.Nagarahalli@arm.com>; Steve Capper
>>> <Steve.Capper@arm.com>; nd <nd@arm.com>
>>> Subject: Re: [PATCH v2] pci_vfio: Support 64KB kernel page_size with
>>> vfio-pci driver
>>>
>>> On 09-Nov-18 5:57 AM, tone.zhang wrote:
>>>> With a larger PAGE_SIZE it is possible for the MSI table to very
>>>> close to the end of the BAR s.t. when we align the start and end of
>>>> the MSI table to the PAGE_SIZE, the end offset of the MSI table is
>>>> out of the PCI BAR boundary.
>>>>
>>>> This patch addresses the issue by comparing both the start and the
>>>> end offset of the MSI table with the BAR size, and skip the mapping
>>>> if it is out of Bar scope.
>>>>
>>>> The patch fixes the debug log as below:
>>>> EAL: Skipping BAR0
>>>>
>>>> Signed-off-by: tone.zhang <tone.zhang@arm.com>
>>>> Reviewed-by: Gavin Hu <Gavin.Hu@arm.com>
>>>> Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
>>>> Reviewed-by: Steve Capper <Steve.Capper@arm.com>
>>>> Reviewed-by: Burakov Anatoly <anatoly.burakov@intel.com>
>>>
>>> In the future, please don't include my Reviewed tag unless i actually
>>> sent one :)
>>
>> Thanks a lot! Will keep in mind. 😊
>>
>>>
>>>> ---
>>>>    drivers/bus/pci/linux/pci_vfio.c | 36
>>>> +++++++++++++++++++++++++++++++---
>>> --
>>>>    1 file changed, 31 insertions(+), 5 deletions(-)
>>>>
>>>> diff --git a/drivers/bus/pci/linux/pci_vfio.c
>>>> b/drivers/bus/pci/linux/pci_vfio.c
>>>> index 305cc06..9a0affe 100644
>>>> --- a/drivers/bus/pci/linux/pci_vfio.c
>>>> +++ b/drivers/bus/pci/linux/pci_vfio.c
>>>> @@ -445,9 +445,11 @@ pci_vfio_mmap_bar(int vfio_dev_fd, struct
>>> mapped_pci_resource *vfio_res,
>>>>    	struct pci_msix_table *msix_table = &vfio_res->msix_table;
>>>>    	struct pci_map *bar = &vfio_res->maps[bar_index];
>>>>
>>>> -	if (bar->size == 0)
>>>> +	if (bar->size == 0) {
>>>>    		/* Skip this BAR */
>>>> +		RTE_LOG(INFO, EAL, "Skipping BAR%d\n", bar_index);
>>>>    		return 0;
>>>> +	}
>>>>
>>>>    	if (msix_table->bar_index == bar_index) {
>>>>    		/*
>>>> @@ -456,8 +458,22 @@ pci_vfio_mmap_bar(int vfio_dev_fd, struct
>>> mapped_pci_resource *vfio_res,
>>>>    		 */
>>>>    		uint32_t table_start = msix_table->offset;
>>>>    		uint32_t table_end = table_start + msix_table->size;
>>>> -		table_end = (table_end + ~PAGE_MASK) & PAGE_MASK;
>>>> -		table_start &= PAGE_MASK;
>>>> +		table_end = RTE_ALIGN(table_end, PAGE_SIZE);
>>>> +		table_start = RTE_ALIGN(table_start, PAGE_SIZE);
>>>> +		/* after rounding to PAGE_SIZE, it is over the bar->size,
>>>> +		 * fall back to the MSI-X table offset in the bar and
>>>> +		 * align with PAGE_SIZE.
>>>> +		 */
>>>
>>> Minor nitpick - wording of comment could be better, for example:
>>>
>>> if page-aligned start of MSI-X table is beyond BAR size, shrink the
>>> mapping size to MSI-X table start address.
>>>
>>> Also, probably needs newline before comment.
>>>
>>
>> Will update the code in next version. Thanks!
>>
>>>> +		if (table_start >= bar->size) {
>>>> +			table_start = RTE_ALIGN_FLOOR(msix_table->offset,
>>>> +							PAGE_SIZE);
>>>> +			/* after aligning with PAGE_SIZE, if it is less than
>>>> +			 * the MSI-X table offset, continue falling back to
>>>> +			 * the actual MSI-X table offset in the bar.
>>>> +			 */
>>>
>>> Same here, wording could probably be improved. Suggested rewording:
>>>
>>> If MSI-X table address, floor-aligned by page size, is lower than
>>> actual MSI-X table offset, fall back to using MSI-X table offset as table start.
>>>
>>> Now that i think of it, this could really be expressed like this:
>>>
>>> uint32_t aligned = RTE_ALIGN_FLOOR(msix_table->offset, PAGE_SIZE);
>>> table_start = RTE_MAX(aligned, msix_table_offset);
>>>
>>> I believe this would be much clearer.
>>>
>>
>> Will update the patch.
>>
> 
> When enter the judgement, it implies the "msix_table->offset" is NOT page size aligned, I think we can replace the code in the judgement with one line: table_start = msix_table->offset;
> It looks more simple. What's your opinion? Thanks!

Agree, what was i thinking :D

> 
>>>> +			if (table_start < msix_table->offset)
>>>> +				table_start = msix_table->offset;
>>>> +		}
>>>>
>>>>    		if (table_start == 0 && table_end >= bar->size) {
>>>>    			/* Cannot map this BAR */
>>>> @@ -469,8 +485,18 @@ pci_vfio_mmap_bar(int vfio_dev_fd, struct
>>>> mapped_pci_resource *vfio_res,
>>>>
>>>>    		memreg[0].offset = bar->offset;
>>>>    		memreg[0].size = table_start;
>>>> -		memreg[1].offset = bar->offset + table_end;
>>>> -		memreg[1].size = bar->size - table_end;
>>>> +		if (bar->size < table_end) {
>>>> +			/*
>>>> +			 * after rounding to PAGE_SIZE we don't have any space
>>>> +			 * left after the MSI table, so don't try and map it.
>>>> +			 */
>>>
>>> Suggested rewording:
>>>
>>> If MSI-X table end is beyond BAR end, don't attempt to perform second
>> mapping.
>>>
>>
>> Thanks a lot. Will update.
>>
>>>> +			memreg[1].offset = 0;
>>>> +			memreg[1].size = 0;
>>>> +		}
>>>> +		else {
>>>> +			memreg[1].offset = bar->offset + table_end;
>>>> +			memreg[1].size = bar->size - table_end;
>>>> +		}
>>>>
>>>>    		RTE_LOG(DEBUG, EAL,
>>>>    			"Trying to map BAR%d that contains the MSI-X "
>>>>
>>>
>>> However, the patch can go in as is if needed, so
>>>
>>> Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com>
>>>
>>
>> Thanks! 😉
>>
>>> --
>>> Thanks,
>>> Anatoly
>>
>> Br,
>> Tone
> 
> Br,
> Tone
> 


-- 
Thanks,
Anatoly

  reply	other threads:[~2018-11-16 10:36 UTC|newest]

Thread overview: 24+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-10-24  2:20 [dpdk-dev] [PATCH] " tone.zhang
2018-10-24  9:09 ` Burakov, Anatoly
2018-11-01  2:33   ` Tone Zhang (Arm Technology China)
2018-11-01 10:01     ` Burakov, Anatoly
     [not found]       ` <DB7PR08MB33859242951014EF340C897AE9C80@DB7PR08MB3385.eurprd08.prod.outlook.com>
2018-11-03  5:46         ` Tone Zhang (Arm Technology China)
2018-11-06 11:03           ` Burakov, Anatoly
2018-11-07  4:55             ` Tone Zhang (Arm Technology China)
2018-11-07 10:12               ` Burakov, Anatoly
2018-11-08  0:45                 ` Tone Zhang (Arm Technology China)
2018-11-09  5:57 ` [dpdk-dev] [PATCH v2] " tone.zhang
2018-11-09 12:15   ` Burakov, Anatoly
2018-11-15  0:49     ` Tone Zhang (Arm Technology China)
2018-11-16  2:34       ` Tone Zhang (Arm Technology China)
2018-11-16 10:36         ` Burakov, Anatoly [this message]
2018-11-19  2:37   ` [dpdk-dev] [PATCH v3] " tone.zhang
2018-12-03  7:25     ` Tone Zhang (Arm Technology China)
2018-12-10 11:40       ` Burakov, Anatoly
2018-12-10 11:45     ` Burakov, Anatoly
2018-12-10 15:55       ` Stephen Hemminger
2018-12-12 10:49         ` Tone Zhang (Arm Technology China)
2018-12-12 10:48       ` Tone Zhang (Arm Technology China)
2018-12-12 11:25     ` [dpdk-dev] [PATCH v4] " tone.zhang
2018-12-12 11:27       ` Burakov, Anatoly
2018-12-19 23:05         ` Thomas Monjalon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=03d60745-ad79-c469-5c63-3a3d7eadfddd@intel.com \
    --to=anatoly.burakov@intel.com \
    --cc=Gavin.Hu@arm.com \
    --cc=Honnappa.Nagarahalli@arm.com \
    --cc=Steve.Capper@arm.com \
    --cc=Tone.Zhang@arm.com \
    --cc=dev@dpdk.org \
    --cc=nd@arm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK patches and discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ https://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git