From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <matan@mellanox.com>
Received: from EUR01-DB5-obe.outbound.protection.outlook.com
(mail-eopbgr60042.outbound.protection.outlook.com [40.107.6.42])
by dpdk.org (Postfix) with ESMTP id 404C21B2B9
for <dev@dpdk.org>; Thu, 18 Jan 2018 15:00:24 +0100 (CET)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com;
s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
bh=G9vAI1mojSdztCVUSH5Pp21wfhc77M8nNnF9Qk/DtqE=;
b=KrTIvuql1p8ruI7OHk82Bv4EMuCOV3YLy0pIAVN28Oh596sCiTw7NqbTTvbDE0c/3+MR195yyMDn8JvUYLuZCKCzcunmymfyDvtxb9BPpe7meGqrNyFvsg9Pu8Co3N5dWrhMUKzGVxwFLZPhZW0RBCzPmHVJilcO5Wjhh/K/X3M=
Received: from AM6PR0502MB3797.eurprd05.prod.outlook.com (52.133.21.26) by
AM6PR0502MB4007.eurprd05.prod.outlook.com (52.133.30.142) with Microsoft SMTP
Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id
15.20.428.17; Thu, 18 Jan 2018 14:00:23 +0000
Received: from AM6PR0502MB3797.eurprd05.prod.outlook.com
([fe80::6c28:c6b3:de94:a733]) by AM6PR0502MB3797.eurprd05.prod.outlook.com
([fe80::6c28:c6b3:de94:a733%13]) with mapi id 15.20.0428.014; Thu, 18 Jan
2018 14:00:23 +0000
From: Matan Azrad <matan@mellanox.com>
To: Neil Horman <nhorman@tuxdriver.com>, "Ananyev, Konstantin"
<konstantin.ananyev@intel.com>
CC: Thomas Monjalon <thomas@monjalon.net>, Gaetan Rivet
<gaetan.rivet@6wind.com>, "Wu, Jingjing" <jingjing.wu@intel.com>,
"dev@dpdk.org" <dev@dpdk.org>, "Richardson, Bruce"
<bruce.richardson@intel.com>
Thread-Topic: [PATCH v2 2/6] ethdev: add port ownership
Thread-Index: AQHTihf/M9xg8LYorUSRFqZtTc27hqNtNdVQgAFomACAAAOCwIAAuwQAgABvY+CABQwJgIAAEk3ggABinoCAANUIAIAAxQGAgAAGCnCAAQnKAIAAAmNwgAApSwCAADKGAIABUdOAgAAMD/A=
Date: Thu, 18 Jan 2018 14:00:23 +0000
Message-ID: <AM6PR0502MB3797474DC219560F2E0FDE0AD2E80@AM6PR0502MB3797.eurprd05.prod.outlook.com>
References: <2601191342CEEE43887BDE71AB9772588627DC25@irsmsx105.ger.corp.intel.com>
<AM6PR0502MB3797650D307664AD9024D927D2EB0@AM6PR0502MB3797.eurprd05.prod.outlook.com>
<2601191342CEEE43887BDE71AB9772588627DE30@irsmsx105.ger.corp.intel.com>
<AM6PR0502MB3797CBF03D656EE2B103E640D2EA0@AM6PR0502MB3797.eurprd05.prod.outlook.com>
<2601191342CEEE43887BDE71AB9772588627E954@irsmsx105.ger.corp.intel.com>
<AM6PR0502MB3797F16A8B4FE5FF9AE47822D2EA0@AM6PR0502MB3797.eurprd05.prod.outlook.com>
<2601191342CEEE43887BDE71AB9772588627EE60@irsmsx105.ger.corp.intel.com>
<AM6PR0502MB3797DAA020B77E44DD599688D2E90@AM6PR0502MB3797.eurprd05.prod.outlook.com>
<20180117140020.GA5432@hmswarspite.think-freely.org>
<2601191342CEEE43887BDE71AB9772588627F0E9@irsmsx105.ger.corp.intel.com>
<20180118131017.GA1622@hmswarspite.think-freely.org>
In-Reply-To: <20180118131017.GA1622@hmswarspite.think-freely.org>
Accept-Language: en-US, he-IL
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: spf=none (sender IP is )
smtp.mailfrom=matan@mellanox.com;
x-originating-ip: [193.47.165.251]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; AM6PR0502MB4007;
7:2QUpfos/L8e5lUm4YMFhkFuCaaW966CLpLDQ6vbpcIcgzhE4QS+qVjo3iALfgOwn5BGCtaSneBurzz58uqIUBrWmhOJ9i3/T+A7OUgRUPvBMQSGgcNtCLS1rFLD9h2Mns+qya3zOuBTohq+hIypsqUR0bRgHMI99OBk94iIvyxGVtS6DfrQbk5i6kIuwf2MgIL+ri9meRe+dASwSrTPOoJDMLB3/iOfe0DauVvPg6nun6DO2uUBhZWXi6M4YYqCK
x-ms-exchange-antispam-srfa-diagnostics: SSOS;
x-ms-office365-filtering-correlation-id: 17983eb6-e492-4309-711a-08d55e7bd184
x-ms-office365-filtering-ht: Tenant
x-microsoft-antispam: UriScan:; BCL:0; PCL:0;
RULEID:(7020095)(4652020)(5600026)(4604075)(3008032)(48565401081)(2017052603307)(7153060)(7193020);
SRVR:AM6PR0502MB4007;
x-ms-traffictypediagnostic: AM6PR0502MB4007:
x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr
x-microsoft-antispam-prvs: <AM6PR0502MB40071672D8FD5D550C9C24D4D2E80@AM6PR0502MB4007.eurprd05.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:(60795455431006)(228905959029699);
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0;
RULEID:(6040470)(2401047)(8121501046)(5005006)(3231023)(2400062)(944501161)(10201501046)(93006095)(93001095)(3002001)(6055026)(6041268)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(20161123562045)(20161123560045)(20161123558120)(6072148)(201708071742011);
SRVR:AM6PR0502MB4007; BCL:0; PCL:0; RULEID:(100000803101)(100110400095);
SRVR:AM6PR0502MB4007;
x-forefront-prvs: 05568D1FF7
x-forefront-antispam-report: SFV:NSPM;
SFS:(10009020)(39860400002)(39380400002)(346002)(376002)(396003)(366004)(199004)(189003)(51444003)(13464003)(81156014)(2906002)(8676002)(81166006)(102836004)(14454004)(7736002)(2900100001)(86362001)(6116002)(305945005)(105586002)(33656002)(3846002)(26005)(229853002)(6436002)(55016002)(9686003)(3660700001)(53936002)(8936002)(97736004)(5250100002)(478600001)(93886005)(4326008)(316002)(110136005)(25786009)(99286004)(74316002)(3280700002)(6246003)(68736007)(54906003)(53546011)(66066001)(76176011)(59450400001)(7696005)(5660300001)(2950100002)(106356001)(6506007);
DIR:OUT; SFP:1101; SCL:1; SRVR:AM6PR0502MB4007;
H:AM6PR0502MB3797.eurprd05.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords;
A:1; MX:1; LANG:en;
received-spf: None (protection.outlook.com: mellanox.com does not designate
permitted sender hosts)
x-microsoft-antispam-message-info: Bowp4jyXoQf5hpTD9CbzHGsKycdMyLg7PFkiTGwF7zNiRMgqwLPpOvdtYPVHIx199EoEQspq08zEn/I5m9i/bg==
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: Mellanox.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 17983eb6-e492-4309-711a-08d55e7bd184
X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Jan 2018 14:00:23.0649 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR0502MB4007
Subject: Re: [dpdk-dev] [PATCH v2 2/6] ethdev: add port ownership
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://dpdk.org/ml/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://dpdk.org/ml/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Thu, 18 Jan 2018 14:00:25 -0000
Hi Neil
From: Neil Horman, Thursday, January 18, 2018 3:10 PM
> On Wed, Jan 17, 2018 at 05:01:10PM +0000, Ananyev, Konstantin wrote:
> >
> >
> > > -----Original Message-----
> > > From: Neil Horman [mailto:nhorman@tuxdriver.com]
> > > Sent: Wednesday, January 17, 2018 2:00 PM
> > > To: Matan Azrad <matan@mellanox.com>
> > > Cc: Ananyev, Konstantin <konstantin.ananyev@intel.com>; Thomas
> > > Monjalon <thomas@monjalon.net>; Gaetan Rivet
> > > <gaetan.rivet@6wind.com>; Wu, Jingjing <jingjing.wu@intel.com>;
> > > dev@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>
> > > Subject: Re: [PATCH v2 2/6] ethdev: add port ownership
> > >
> > > On Wed, Jan 17, 2018 at 12:05:42PM +0000, Matan Azrad wrote:
> > > >
> > > > Hi Konstantin
> > > > From: Ananyev, Konstantin, Sent: Wednesday, January 17, 2018 1:24
> > > > PM
> > > > > Hi Matan,
> > > > >
> > > > > > Hi Konstantin
> > > > > >
> > > > > > From: Ananyev, Konstantin, Tuesday, January 16, 2018 9:11 PM
> > > > > > > Hi Matan,
> > > > > > >
> > > > > > > >
> > > > > > > > Hi Konstantin
> > > > > > > > From: Ananyev, Konstantin, Monday, January 15, 2018 8:44
> > > > > > > > PM
> > > > > > > > > Hi Matan,
> > > > > > > > > > Hi Konstantin
> > > > > > > > > > From: Ananyev, Konstantin, Monday, January 15, 2018
> > > > > > > > > > 1:45 PM
> > > > > > > > > > > Hi Matan,
> > > > > > > > > > > > Hi Konstantin
> > > > > > > > > > > > From: Ananyev, Konstantin, Friday, January 12,
> > > > > > > > > > > > 2018 2:02 AM
> > > > > > > > > > > > > Hi Matan,
> > > > > > > > > > > > > > Hi Konstantin
> > > > > > > > > > > > > > From: Ananyev, Konstantin, Thursday, January
> > > > > > > > > > > > > > 11, 2018
> > > > > > > > > > > > > > 2:40 PM
> > > > > > > > > > > > > > > Hi Matan,
> > > > > > > > > > > > > > > > Hi Konstantin
> > > > > > > > > > > > > > > > From: Ananyev, Konstantin, Wednesday,
> > > > > > > > > > > > > > > > January 10,
> > > > > > > > > > > > > > > > 2018
> > > > > > > > > > > > > > > > 3:36 PM
> > > > > > > > > > > > > > > > > Hi Matan,
> > > > > > > > <snip>
> > > > > > > > > > > > > > > > > It is good to see that now
> > > > > > > > > > > > > > > > > scanning/updating rte_eth_dev_data[] is
> > > > > > > > > > > > > > > > > lock protected, but it might be not very
> > > > > > > > > > > > > > > > > plausible to protect both data[] and
> > > > > > > > > > > > > > > > > next_owner_id using the
> > > > > > > > > > > same lock.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > I guess you mean to the owner structure in
> > > > > > > > > > > rte_eth_dev_data[port_id].
> > > > > > > > > > > > > > > > The next_owner_id is read by ownership
> > > > > > > > > > > > > > > > APIs(for owner validation), so it
> > > > > > > > > > > > > > > makes sense to use the same lock.
> > > > > > > > > > > > > > > > Actually, why not?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Well to me next_owner_id and
> > > > > > > > > > > > > > > rte_eth_dev_data[] are not directly
> > > > > > > > > > > > > related.
> > > > > > > > > > > > > > > You may create new owner_id but it doesn't
> > > > > > > > > > > > > > > mean you would update rte_eth_dev_data[]
> immediately.
> > > > > > > > > > > > > > > And visa-versa - you might just want to
> > > > > > > > > > > > > > > update rte_eth_dev_data[].name or .owner_id.
> > > > > > > > > > > > > > > It is not very good coding practice to use
> > > > > > > > > > > > > > > same lock for non-related data structures.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > I see the relation like next:
> > > > > > > > > > > > > > Since the ownership mechanism synchronization
> > > > > > > > > > > > > > is in ethdev responsibility, we must protect
> > > > > > > > > > > > > > against user mistakes as much as we can by
> > > > > > > > > > > > > using the same lock.
> > > > > > > > > > > > > > So, if user try to set by invalid owner
> > > > > > > > > > > > > > (exactly the ID which currently is
> > > > > > > > > > > > > allocated) we can protect on it.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Hmm, not sure why you can't do same checking
> > > > > > > > > > > > > with different lock or atomic variable?
> > > > > > > > > > > > >
> > > > > > > > > > > > The set ownership API is protected by ownership
> > > > > > > > > > > > lock and checks the owner ID validity By reading th=
e next
> owner ID.
> > > > > > > > > > > > So, the owner ID allocation and set API should use
> > > > > > > > > > > > the same atomic
> > > > > > > > > > > mechanism.
> > > > > > > > > > >
> > > > > > > > > > > Sure but all you are doing for checking validity, is
> > > > > > > > > > > check that owner_id > 0 &&& owner_id < next_ownwe_id,
> right?
> > > > > > > > > > > As you don't allow owner_id overlap (16/3248 bits)
> > > > > > > > > > > you can safely do same check with just
> atomic_get(&next_owner_id).
> > > > > > > > > > >
> > > > > > > > > > It will not protect it, scenario:
> > > > > > > > > > - current next_id is X.
> > > > > > > > > > - call set ownership of port A with owner id X by
> > > > > > > > > > thread 0(by user
> > > > > > > mistake).
> > > > > > > > > > - context switch
> > > > > > > > > > - allocate new id by thread 1 and get X and change
> > > > > > > > > > next_id to
> > > > > > > > > > X+1
> > > > > > > > > atomically.
> > > > > > > > > > - context switch
> > > > > > > > > > - Thread 0 validate X by atomic_read and succeed to
> > > > > > > > > > take
> > > > > ownership.
> > > > > > > > > > - The system loosed the port(or will be managed by two
> > > > > > > > > > entities) -
> > > > > > > crash.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Ok, and how using lock will protect you with such scenari=
o?
> > > > > > > >
> > > > > > > > The owner set API validation by thread 0 should fail
> > > > > > > > because the owner
> > > > > > > validation is included in the protected section.
> > > > > > >
> > > > > > > Then your validation function would fail even if you'll use
> > > > > > > atomic ops instead of lock.
> > > > > > No.
> > > > > > With atomic this specific scenario will cause the validation to=
pass.
> > > > >
> > > > > Can you explain to me how?
> > > > >
> > > > > rte_eth_is_valid_owner_id(uint16_t owner_id) {
> > > > > int32_t cur_owner_id =3D
> > > > > RTE_MIN(rte_atomic32_get(next_owner_id),
> > > > > UINT16_MAX);
> > > > >
> > > > > if (owner_id =3D=3D RTE_ETH_DEV_NO_OWNER || owner >
> > > > > cur_owner_id) {
> > > > > RTE_LOG(ERR, EAL, "Invalid owner_id=3D%d.\n", owner_id);
> > > > > return 0;
> > > > > }
> > > > > return 1;
> > > > > }
> > > > >
> > > > > Let say your next_owne_id=3D=3DX, and you invoke
> > > > > rte_eth_is_valid_owner_id(owner_id=3DX+1) - it would fail.
> > > >
> > > > Explanation:
> > > > The scenario with locks:
> > > > next_owner_id =3D X.
> > > > Thread 0 call to set API(with invalid owner Y=3DX) and take lock.
> > > > Context switch.
> > > > Thread 1 call to owner_new and stuck in the lock.
> > > > Context switch.
> > > > Thread 0 does owner id validation and failed(Y>=3DX) - unlock the l=
ock and
> return failure to the user.
> > > > Context switch.
> > > > Thread 1 take the lock and update X to X+1, then, unlock the lock.
> > > > Everything is OK!
> > > >
> > > > The same scenario with atomics:
> > > > next_owner_id =3D X.
> > > > Thread 0 call to set API(with invalid owner Y=3DX) and take lock.
> > > > Context switch.
> > > > Thread 1 call to owner_new and change X to X+1(atomically).
> > > > Context switch.
> > > > Thread 0 does owner id validation and success(Y<(atomic)X+1) - unlo=
ck
> the lock and return success to the user.
> > > > Problem!
> > > >
> > >
> > >
> > > Matan is correct here, there is no way to preform parallel set
> > > operations using just and atomic variable here, because multiple
> > > reads of next_owner_id need to be preformed while it is stable.
> > > That is to say rte_eth_next_owner_id must be compared to
> > > RTE_ETH_DEV_NO_OWNER and owner_id in rte_eth_is_valid_owner_id.
> If
> > > you were to only use an atomic_read on such a variable, it could be
> > > incremented by the owner_new function between the checks and an
> > > invalid owner value could become valid because a third thread
> > > incremented the next value. The state of next_owner_id must be kept
> > > stable during any validity checks
> >
> > It could still be incremented between the checks - if let say
> > different thread will invoke new_onwer_id, grab the lock update
> > counter, release the lock - all that before the check.
> I don't see how all of the contents of rte_eth_dev_owner_set is protected
> under rte_eth_dev_ownership_lock, as is rte_eth_dev_owner_new.
> Next_owner might increment between another threads calls to owner_new
> and owner_set, but that will just cause a transition from an ownership id
> being valid to invalid, and thats ok, as long as there is consistency in =
the
> model that enforces a single valid owner at a time (in that case the
> subsequent caller to owner_new).
>=20
I'm not sure I fully understand you, but see:
we can't protect all of the user mistakes(using the wrong owner id).
But we are doing the maximum for it.
> Though this confusion does underscore my assertion I think that this API =
is
> overly complicated
>=20
I really don't think it is complicated. - just take ownership of a port(by =
owner id allocation and set APIs) and manage the port as you want.=20
> Neil