From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <jianbo.liu@linaro.org>
Received: from mail-yw0-f174.google.com (mail-yw0-f174.google.com
 [209.85.161.174]) by dpdk.org (Postfix) with ESMTP id 328636CD6
 for <dev@dpdk.org>; Fri, 13 May 2016 03:43:30 +0200 (CEST)
Received: by mail-yw0-f174.google.com with SMTP id t10so102022125ywa.0
 for <dev@dpdk.org>; Thu, 12 May 2016 18:43:30 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;
 h=mime-version:in-reply-to:references:date:message-id:subject:from:to
 :cc; bh=PZRZTJ9nt/C61CJ/qc9k/PGsT9pUp6gFxu2qSYdIuRw=;
 b=gXm3l83b2WQqcyg0sHng3LqvJmlmOXQfHtEJPv1fBjsF52TYu2xomFITUb/kyAmg8Q
 180ELBuFhqjBZYwVtHNBcJO7EBd7jPCyA6b57fJ6g0l7LZHxH5afMB20+bBHgKuCVoOF
 6B5sDdiljsZRmOsqWSJq3zQTgEMa9Y4CyZdCs=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20130820;
 h=x-gm-message-state:mime-version:in-reply-to:references:date
 :message-id:subject:from:to:cc;
 bh=PZRZTJ9nt/C61CJ/qc9k/PGsT9pUp6gFxu2qSYdIuRw=;
 b=OJcMQhV2DtChBM4ER6YA1erLIPXJKtyzurBesyeXCEUsqNDf8ebq+FkuBCsVb20qvg
 4Wnde1mIxIT+hpCriFxyogN4Dlpox9FNYzpKcIk7asYWCtxEZ/knZdwxHs9Pie8eK7yv
 saZ4WEO4RPs80I/gfc24W1Mi5k5wcmP7Vcj5jdu/NMGj3rCcl0oqUUvHdFDBeWaC09Qk
 GWV7yv9pMiiMdr0IBVCmwsFN/tOFhdqarE8sUNEKKejMWCj0Di2MG6uJJmK/lhVRvSh+
 jOYar5h+hCqXsmvqxDXabiNulB4aH7frzndehkyByYjpMDYAC/A9v8EJP80yjT1iZRmE
 q4ew==
X-Gm-Message-State: AOPr4FX1mOTIdb+hYMtOphdxoMMz8VcJxj6JfdLuvbG6LyOgb+kYUVvLRx/xNg9JTHzd+o2Jdgw7WWi/AsIZidyd
MIME-Version: 1.0
X-Received: by 10.13.232.134 with SMTP id r128mr6387106ywe.163.1463103809567; 
 Thu, 12 May 2016 18:43:29 -0700 (PDT)
Received: by 10.37.223.2 with HTTP; Thu, 12 May 2016 18:43:29 -0700 (PDT)
In-Reply-To: <20160512103103.GA16360@santosh-Latitude-E5530-non-vPro>
References: <20160511082259.42905f98@xeon-e3>
 <20160511170215.GA1637@localhost.localdomain>
 <20160511112559.69dcff13@xeon-e3>
 <CAP4Qi39YnbOAv8ToaNjc7PnBf=ZDhD3Ofep4SFNnvNmrDBjquQ@mail.gmail.com>
 <20160512031642.GA5855@santosh-Latitude-E5530-non-vPro>
 <CAP4Qi3_7jw1Jy=YUZ6fjzhb+kfpx3noPRbYRs0CgnZLyQ7Ygcw@mail.gmail.com>
 <20160512050638.GA7301@santosh-Latitude-E5530-non-vPro>
 <CAP4Qi3-8vee6wb8TDBt+kEQ-LHEg=0-AEWKgcDm=cZRNWGTQ+Q@mail.gmail.com>
 <20160512083342.GA12841@santosh-Latitude-E5530-non-vPro>
 <CAP4Qi38m9U3rfRzMNLfB2Wx7-_mhRw5a3ahhs_68LjzxxEMe-A@mail.gmail.com>
 <20160512103103.GA16360@santosh-Latitude-E5530-non-vPro>
Date: Fri, 13 May 2016 09:43:29 +0800
Message-ID: <CAP4Qi3_eO-TRw=jkRws_X_iwTBYaNndknoUuEkGw1L3DimLeWg@mail.gmail.com>
From: Jianbo Liu <jianbo.liu@linaro.org>
To: Santosh Shukla <santosh.shukla@caviumnetworks.com>
Cc: Stephen Hemminger <stephen@networkplumber.org>, 
 Jerin Jacob <jerin.jacob@caviumnetworks.com>,
 Hemant Agrawal <hemant.agrawal@nxp.com>, dev@dpdk.org, 
 Thomas Monjalon <thomas.monjalon@6wind.com>
Content-Type: text/plain; charset=UTF-8
Subject: Re: [dpdk-dev] [PATCHv3 1/2] config/armv8a: disable igb_uio
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: patches and discussions about DPDK <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Fri, 13 May 2016 01:43:30 -0000

On 12 May 2016 at 18:31, Santosh Shukla
<santosh.shukla@caviumnetworks.com> wrote:
> On Thu, May 12, 2016 at 05:52:54PM +0800, Jianbo Liu wrote:
>> On 12 May 2016 at 16:57, Santosh Shukla
>> <santosh.shukla@caviumnetworks.com> wrote:
>> > On Thu, May 12, 2016 at 01:54:13PM +0800, Jianbo Liu wrote:
>> >> On 12 May 2016 at 13:06, Santosh Shukla
>> >> <santosh.shukla@caviumnetworks.com> wrote:
>> >> > On Thu, May 12, 2016 at 11:42:26AM +0800, Jianbo Liu wrote:
>> >> >> On 12 May 2016 at 11:17, Santosh Shukla
>> >> >> <santosh.shukla@caviumnetworks.com> wrote:
>> >> >> > On Thu, May 12, 2016 at 10:01:05AM +0800, Jianbo Liu wrote:
>> >> >> >> On 12 May 2016 at 02:25, Stephen Hemminger <stephen@networkplumber.org> wrote:
>> >> >> >> > On Wed, 11 May 2016 22:32:16 +0530
>> >> >> >> > Jerin Jacob <jerin.jacob@caviumnetworks.com> wrote:
>> >> >> >> >
>> >> >> >> >> On Wed, May 11, 2016 at 08:22:59AM -0700, Stephen Hemminger wrote:
>> >> >> >> >> > On Wed, 11 May 2016 19:17:58 +0530
>> >> >> >> >> > Hemant Agrawal <hemant.agrawal@nxp.com> wrote:
>> >> >> >> >> >
>> >> >> >> >> > > IGB_UIO not supported for arm64 arch in kernel so disable.
>> >> >> >> >> > >
>> >> >> >> >> > > Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
>> >> >> >> >> > > Reviewed-by: Santosh Shukla <santosh.shukla@caviumnetworks.com>
>> >> >> >> >> >
>> >> >> >> >> > Really, I have use IGB_UIO on ARM64
>> >> >> >> >>
>> >> >> >> >> May I know what is the technical use case for igb_uio on arm64
>> >> >> >> >> which cannot be addressed through vfio or vfioionommu.
>> >> >> >> >
>> >> >> >> > I was running on older kernel which did not support vfioionommu mode.
>> >> >> >>
>> >> >> >> As I said, most of DPDK developers are not kernel developers. They may
>> >> >> >> have their own kernel tree, and couldn't like to upgrade to latest
>> >> >> >> kernel.
>> >> >> >> They can choose to use or not use igb_uio when binding the driver. But
>> >> >> >> blindly disabling it in the base config seems unreasonable.
>> >> >> >
>> >> >> > if user keeping his own kernel so they could also keep IGB_UIO=y in their local
>> >> >> Most likely they don't have local dpdk tree. They write their own
>> >> >> applications, complie and link to dpdk lib, then done.
>> >> >>
>> >> >> > dpdk tree. Why are you imposing user-x custome depedancy on upstream dpdk base
>> >> >> Customer requiremnts is important. I want they can choose the way they like.
>> >> >>
>> >> >
>> >> > so you choose to keep igb_uio option, provided arch doesn't support?
>> >> > new user did reported issues with igb_uio for arm64, refer this thread [1], as
>> >> > well hemanth too faced issues. we want to avoid that.
>> >> >
>> >> > If customer maintaing out-of-tree kernel then he can also switch to vfio-way.
>> >> > isn;t it?
>> >> >
>> >> >> > config. Is it not enough for explanation that - Base config ie.. armv8 doesn;t
>> >> >> > support pci mmap, so igb_uio is n/a. New user wont able to build/run dpdk/arm64
>> >> >> > in igb_uio-way, He'll prefer to use upstream stuff. I think, you are not making
>> >> >> You are wrong, he can build dpdk. If he like to use upstream without
>> >> >> patching, he can use vfio.
>> >> >
>> >> > I disagree, we want to avoid [1] for new user.
>> >> >
>> >> >> But you can't ignore the need from old user which is more comfortable
>> >> >> with older kernel.
>> >> >>
>> >> > arm/arm64 dpdk support recently added and I am guessing, most likely customer
>> >> > using near latest kernel, switching to vfio won't be so difficult.
>> >> >
>> >> > Or can you take up responsibility of upstreaming pci mmap patch, then we don't
>> >> > need this patch.
>> >> >
>> >> > [1] http://dpdk.org/ml/archives/dev/2016-January/031313.html
>> >>
>> >> Can you read carefully about the guide at
>> >> http://dpdk.org/doc/guides/linux_gsg/build_dpdk.html? It says to use
>> >> uio_pci_generic, igb_uio or vfio-pci.
>> >
>> > *** applicable and works for x86 only, not for arm64: because pci mmap support
>> > not present for arm64, in that case we should update the doc.
>> >
>> >> Could it be possible that the user in that thread has already read and
>> >> tried them all and found that he can't enable vifo with his kernel,
>> >> and igb_uio is the easy way for him and asked for help from community?
>> >> If so, we have no choice but keeping igb_uio enabled.
>> >
>> > By then vfionoiommu support was wip progress in dpdk/linux. but now it merged
>> > and it works. So no need to retain igb_uio in base config for which to work -
>> > user need to use mmap patch at linux side.
>>
>> We can't decide which kernel user will use.
>>
>
> yes, we can't decide kernel for user but we should be explicit to user on - what
> works for dpdk/linux out-of-box vs what could work with use of out-of-tree
> patch/RFC's.. example igb_uio.
>

OK, please persuade Stephen Hemminger and the other guy to use
upstream kernel first!

>> >
>> > Or can you maintain out-of-tree pci mmap patch/ kerne source and make it
>> > explicit somewhere in dpdk build doc that - if user want igb_uio way then
>> > use kernel/mmap patch from x location.
>>
>> The patch is in the kernel maillist, and user google it.
>
> there are feature specific rfc's in plenty in lkml/qemu mailing list,  and you
> suggest- user to hunt for all those information. Is this how we;re officially
> supporting igb_uio for arm64.. that let user to google?
>

Sorry I don't know you are offically support users here.
And you also don't know what they really want.

>> And isn't funny to ask someone to do something again and again (3
>> times) in this thread?
>>
>
> I am asking becasue your in favour of keeping igb_uio for arm64 but not
> agreeing to streamline (writing a note in dpdk doc for igb_uio for arm64 or
> pointing to working tree).. so that user don;t need to grep or google for known
> findings.
>
> I find discussion going in circle and nothing will conclude, So given up.
>> >
>> >> He use lsmod to show us the modules, most likely he know vifo-pci.
>> >>
>> >> Below are the details on modules, hugepages and device binding.
>> >> root at arm64:~# lsmod
>> >> Module                  Size  Used by
>> >> rte_kni               292795  0
>> >> igb_uio                 4338  0
>> >> ixgbe                 184456  0