From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6D3E7428BD; Mon, 3 Apr 2023 08:55:07 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0492240A7E; Mon, 3 Apr 2023 08:55:07 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by mails.dpdk.org (Postfix) with ESMTP id 388D74014F; Mon, 3 Apr 2023 08:55:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1680504905; x=1712040905; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=wctKMnp1OV2VSsMDo8OX5H1NXm9448+3KhkZlOmssjc=; b=chyHThUiEeoFhINrYZ8y30hoQ0tbrqtIEbSHIdTeK4mw626veObrRMDx WA2J3/CwEleV5NzAUumnPaO39g6IZK8yoQf0DQb8q36mTooD6zIakDxD/ xY6s6KRLglc+Z7d9wdK4aSDioax3fkpkjpeZrsmIrTviYUC44sHZS27Ol FeJI4Qb4ho/jRJc1HYprd/q9igQ4i4wDeKXabL2ewCTEn7xSu0130wQKh 9rFhiU23v6eQhxeCXlPTak7CdPS+91QUJqBSoXZpCci+ZdH+DQfLWy5ZV XE1Qybjv20l8J/0U0/FkdPqiq1QRsHebW6GAwK6gKLQJ5MUwRH2Cn09oZ w==; X-IronPort-AV: E=McAfee;i="6600,9927,10668"; a="321486480" X-IronPort-AV: E=Sophos;i="5.98,314,1673942400"; d="scan'208";a="321486480" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Apr 2023 23:55:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10668"; a="679344148" X-IronPort-AV: E=Sophos;i="5.98,314,1673942400"; d="scan'208";a="679344148" Received: from fmsmsx603.amr.corp.intel.com ([10.18.126.83]) by orsmga007.jf.intel.com with ESMTP; 02 Apr 2023 23:55:02 -0700 Received: from fmsmsx611.amr.corp.intel.com (10.18.126.91) by fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.21; Sun, 2 Apr 2023 23:55:02 -0700 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx611.amr.corp.intel.com (10.18.126.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.21; Sun, 2 Apr 2023 23:55:01 -0700 Received: from fmsedg601.ED.cps.intel.com (10.1.192.135) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.21 via Frontend Transport; Sun, 2 Apr 2023 23:55:01 -0700 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (104.47.66.45) by edgegateway.intel.com (192.55.55.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.21; Sun, 2 Apr 2023 23:54:59 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FqAu6vxHu+ftjkA7Gcg6KvWsD0Zya+CH7A8o+bCHTRf9INsboKf5OIbArmDD4DNE5zzYCzhtgipiW7woirkVpVAxDFrQBtJAQ3u2juxUdPanL6iCagGA2Nd/GkmzuI7uT0uEZt8eZINu8/4GpCxiIKEXQurQ9shh3v2FTYx6BTACJitc6dOaqGgRmd9zgTecIiUihvQ0kR5Kbxd3pljJ4WjMqR+Rca1aQe7DTinfk0GteeytqzPwNI2EiUk8rl6YdAg0fQxbQ17aZpS2zAZDiT7ZUvpRXldjpSn6C/d7DMp6h4o4vk5TUHPWqURb3PqxSFD/zScyu30F4TR27fY0Aw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4apb6Zk6DD8Zz8jI49GvUwSTnBXe9GEu8GaysYBD5/8=; b=gDQip/ylqsimWmU5O/OI/LZakw/MSY1HwCy4dEHUNKlYhB3C8gdhcid/Hw/kabciOzG/9O0IqF7IZJA9eqMsbTxhav3YjiyfGn8OQ3NVcTpTM3P9btW9EYBzv1a3X+KMkPCHRQtEMJkS0SuV2vZMoVjJkGORkwIKHywFyZ4xm9A426qHqcvh3WI0uieJKMCxmr9hiCtj21rUnfXyW4aCNi1HQ2ECXmXC6tZDLbeqxCP4r6xMi/BpGXF0f8RPmYyooujbmGjxhXnYm+1EaXNFmkwhD/eZ2XcFGDN2C2hNe9SkeoWcDlpufKJOYiwsV4jJ9fhVjEtBNkrJeH1gFNUSkw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from DM4PR11MB5994.namprd11.prod.outlook.com (2603:10b6:8:5d::20) by SA3PR11MB8047.namprd11.prod.outlook.com (2603:10b6:806:2fc::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6254.29; Mon, 3 Apr 2023 06:54:57 +0000 Received: from DM4PR11MB5994.namprd11.prod.outlook.com ([fe80::c03b:8d85:1c75:7046]) by DM4PR11MB5994.namprd11.prod.outlook.com ([fe80::c03b:8d85:1c75:7046%3]) with mapi id 15.20.6254.030; Mon, 3 Apr 2023 06:54:57 +0000 From: "Zhang, Qi Z" To: "Ye, MingjinX" , "dev@dpdk.org" CC: "Yang, Qiming" , "stable@dpdk.org" , "Zhou, YidingX" , "Zhang, Ke1X" Subject: RE: [PATCH v6] net/ice: fix ice dcf control thread crash Thread-Topic: [PATCH v6] net/ice: fix ice dcf control thread crash Thread-Index: AQHZXIOzoJKoL97vqkuQbHW7J/CIGq8ZMMdw Date: Mon, 3 Apr 2023 06:54:57 +0000 Message-ID: References: <20230320094030.18949-1-mingjinx.ye@intel.com> <20230322055606.84874-1-mingjinx.ye@intel.com> In-Reply-To: <20230322055606.84874-1-mingjinx.ye@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: DM4PR11MB5994:EE_|SA3PR11MB8047:EE_ x-ms-office365-filtering-correlation-id: 74e070fd-7885-4f39-e94f-08db3410562f x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: V9SayZmczlm3ln4+t90Uh0bJ6IHjdzkPCWtplpnb5vO0d+FnCyoTLbcoeFq7eCES/5cxOaiApLDiRa3y81N1Q86MPdctg/rBsOGrUr9jBIO+f3yWzhqKNLMoIWGEqZXR9MdddsHPJ1PJVLy8yWNAiqe0H85vGZAiVEkCexXVV/ATH1JgfpbMdVeHkpYU4yF9nn168IIzRcEe8Im46o32+b4SCWdnkPgdnNmqhH42Bn1mu/LpeBHc2vCktb1GKYxxYslXUnfdegj5hvVWBdmRkzFRc0Sctce9nb+qdE0pHDg0rALxDLij5mldR2kQUq7R8cpN/HCBpDb17+5v+KUAv5QCRZhe9hzQNwzoUO0qx6ugoGokld4oQ3jfevIFoXbygX+kpA3khv8uT/MHYzSBNaQfNbncwxjzHGqKjVuVgpWGLaA8V9lSMeS8XflCb635KVdN/1Qy11rAsJWmMkGX4o1GJ5zWLazLD7sHzhj3xkCGu26tS37EOsgVLL3kxENmJhN3WqgEokayqDdTAG2dG8StdsBuJaH+fGsDJt7fvsPFSu30eFZ4xwf16vafGkks60baIfQjZ67lir9DC4eHwRPW6j6b788l7IGRN9tME0bItn/yDdOLi6clv+yL4ydQ x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM4PR11MB5994.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(366004)(346002)(396003)(376002)(136003)(39860400002)(451199021)(53546011)(54906003)(55016003)(83380400001)(107886003)(26005)(9686003)(6506007)(186003)(5660300002)(71200400001)(7696005)(41300700001)(38100700002)(8936002)(86362001)(82960400001)(38070700005)(122000001)(52536014)(478600001)(66446008)(66946007)(450100002)(66476007)(76116006)(64756008)(66556008)(316002)(8676002)(4326008)(33656002)(110136005)(2906002); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?n0weY8yAz/FNLawjrObYRuxtcUJ/AGjMzbE0mKzomd9ARUj2wV4vPv8hnmsf?= =?us-ascii?Q?q9A5f8H+UhhRiNJcnxJnpAIQ4YcKPuKWcgdESDWglhIwaRa1VWZekJ3KiWON?= =?us-ascii?Q?XQylv/Ow4A/4syzHyKjqDI0XwzCZ2mi0Ro6YuejkgFv9UJJ5CC2YM8ZhEcd8?= =?us-ascii?Q?7vEeIe9lJbsN2LJKk4Bqii5kWBV4H2Us+iFmz5U2acSikFaQLnromF4d5oE0?= =?us-ascii?Q?3RxSJ/5xc22Z4NpI3XE6oXOmuo8g3amIes/p01G7UTFBqaljU9Gm4i2a193t?= =?us-ascii?Q?zFVd+QcpwxrJ0fSdOKEf2bU4U4umH1lxqgiasbH7Z/40Q2oQFuPQa96BLFje?= =?us-ascii?Q?vdmAJFFq7NQO/fdtEapCd4TDuRMCO7ELDcq8ERXhxkHuNalPxVvYDnQ+pc/C?= =?us-ascii?Q?Aw9Z9vYYYcvGytuItrST8Kcj9KcWs3DawBNLldznqgjP0AYf7vYjuRUN7lJc?= =?us-ascii?Q?aSSn7EJL33nv3sTYpVftLrS5a4JnkjdMBGJmGD7C+QUKxljeR0gb9ye4WWZF?= =?us-ascii?Q?O1cNRcSUwcX/ylnJ0++6nxC/of8bpjfkESwJp+O03z8AZAFSWIz3MWkSzprg?= =?us-ascii?Q?dOFg8axDU3momxGg+OYldgfggRbk+KRlBRBz7p8QQwz15Bm13JXeIKo1ZK9O?= =?us-ascii?Q?L2wUpz+jPDQBG3Wi9sEOrbuDvREzMq7A9TeBxl5K6UY7ee5tn2bv2cvFfiwA?= =?us-ascii?Q?t00FQSwF20HVXT9jleGFpygk0Oik9u8euP/kIV2y2bbZxXy+rMoX5LVPBGk1?= =?us-ascii?Q?5/jwwJvF4mcOthYCsJnTFe1KizLliCU8jJDk34V6xAawaW9ht+2YcpkDIekA?= =?us-ascii?Q?mr4zJqjr7FLaJXv5nnPh1h2b/MQ6xzLUr2TdeB3BRele5hgBx0r7J62Go8wl?= =?us-ascii?Q?O1etE2UiKPob37Ntcvtff1AMxtNegfcEHcctAHlRY7UhFDS2yrI5bDBN6jMX?= =?us-ascii?Q?6F+uK5/bIYhu3NivuEMDN851NjOF8fqqnC9+KohpT0nfoGR6fjHc/0uf8KtM?= =?us-ascii?Q?jDQSQNruL5Xnbf+mCKohe7g8o8sfnY+kJEI5xH2gAslweF/ciVUxv46fMxhr?= =?us-ascii?Q?QyJqkgsCP+ZGSPGyxw/5ONIhpT5v6OR5mYtJjYFivUKAg+KhYO/8NF2TwVPI?= =?us-ascii?Q?jsJoqFonY5oZtcHBk76iSn+4qCm9BOiya7yMcC6BEIZjYxqjIw8BoDYmUOD0?= =?us-ascii?Q?7XaHX76n3RHvK63ANSW/ZnxFWW9tv2K0lgMBlI1VhuCc73+c0JE0FDrdiQip?= =?us-ascii?Q?TfmN6Kn1ona7T7ZCp5+rBCGNjYlH0zbyhzlk7+aveIEOx33m3nkmm9JI8AjX?= =?us-ascii?Q?JGXVATc6jc87IM2vjjVHHcdTBGC4cnSMOCxl/iox4DfCZYZL3l8+PK1Cwyd1?= =?us-ascii?Q?1KIV+5cJ0lCI9iIznlDgJKPCHkVmTD7btmxk4LzpXJGvXdxjxH/4ZUAarQ0r?= =?us-ascii?Q?AbRcoPcbpQNw4EIjRDdu/62gagKKLFAJ7NxntxEU1e8YRrCdcXVnqQr5sFC/?= =?us-ascii?Q?MgLcNGXg9+aOXvbgOGVZl0y2+RR4julizw0wmJliLFTlfRznjWKFV3azMaI3?= =?us-ascii?Q?h0yymS4T95qraROS7HJpnzbrsJXxmg4qqDRPqtwc?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB5994.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 74e070fd-7885-4f39-e94f-08db3410562f X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Apr 2023 06:54:57.2692 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: LEmMy4wRJXUHCWesJHGz5pRIW3eFrUkY07bnQEFg/5jY2Tf2LUGZsLd3OzD6j8j8O8vKabMbi8jhnnRAWnYN7A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR11MB8047 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: Ye, MingjinX > Sent: Wednesday, March 22, 2023 1:56 PM > To: dev@dpdk.org > Cc: Yang, Qiming ; stable@dpdk.org; Zhou, YidingX > ; Ye, MingjinX ; Zhang, > Ke1X ; Zhang, Qi Z > Subject: [PATCH v6] net/ice: fix ice dcf control thread crash >=20 > The control thread accesses the hardware resources after the resources we= re > released, which results in a segment error. >=20 > The 'ice-reset' threads are detached, so thread resources cannot be > reclaimed by `pthread_join` calls. >=20 > This commit synchronizes the number of "ice-reset" threads by adding a > variable ("vsi_update_thread_num") to the "struct ice_dcf_hw" and > performing an atomic operation on this variable. When releasing HW > resources, we wait for the number of "ice-reset" threads to be reduced to= 0 > before releasing the resources. >=20 > Fixes: c7e1a1a3bfeb ("net/ice: refactor DCF VLAN handling") > Fixes: 7564d5509611 ("net/ice: add DCF hardware initialization") > Fixes: 0b02c9519432 ("net/ice: handle PF initialization by DCF") > Cc: stable@dpdk.org >=20 > Signed-off-by: Ke Zhang > Signed-off-by: Mingjin Ye > --- > v2: add pthread_exit() for windows > --- > v3: Optimization. It is unsafe for a thread to forcibly exit, which will = cause the > spin lock to not be released correctly > --- > v4: Safely wait for all event threads to end > --- > v5: Spinlock moved to struct ice_dcf_hw > --- > v6: Spinlock changed to atomic > --- > drivers/net/ice/ice_dcf.c | 9 +++++++++ > drivers/net/ice/ice_dcf.h | 2 ++ > drivers/net/ice/ice_dcf_parent.c | 6 ++++++ > 3 files changed, 17 insertions(+) >=20 > diff --git a/drivers/net/ice/ice_dcf.c b/drivers/net/ice/ice_dcf.c index > 1c3d22ae0f..adf2cf2cb6 100644 > --- a/drivers/net/ice/ice_dcf.c > +++ b/drivers/net/ice/ice_dcf.c > @@ -32,6 +32,8 @@ > #define ICE_DCF_ARQ_MAX_RETRIES 200 > #define ICE_DCF_ARQ_CHECK_TIME 2 /* msecs */ >=20 > +#define ICE_DCF_CHECK_INTERVAL 100 /* 100ms */ > + > #define ICE_DCF_VF_RES_BUF_SZ \ > (sizeof(struct virtchnl_vf_resource) + \ > IAVF_MAX_VF_VSI * sizeof(struct virtchnl_vsi_resource)) @@ > -639,6 +641,8 @@ ice_dcf_init_hw(struct rte_eth_dev *eth_dev, struct > ice_dcf_hw *hw) > rte_spinlock_init(&hw->vc_cmd_queue_lock); > TAILQ_INIT(&hw->vc_cmd_queue); >=20 > + __atomic_store_n(&hw->vsi_update_thread_num, 0, > __ATOMIC_RELAXED); > + > hw->arq_buf =3D rte_zmalloc("arq_buf", ICE_DCF_AQ_BUF_SZ, 0); > if (hw->arq_buf =3D=3D NULL) { > PMD_INIT_LOG(ERR, "unable to allocate AdminQ buffer > memory"); @@ -760,6 +764,11 @@ ice_dcf_uninit_hw(struct rte_eth_dev > *eth_dev, struct ice_dcf_hw *hw) > rte_intr_callback_unregister(intr_handle, > ice_dcf_dev_interrupt_handler, hw); >=20 > + /* Wait for all `ice-thread` threads to exit. */ > + while (__atomic_load_n(&hw->vsi_update_thread_num, > + __ATOMIC_ACQUIRE) !=3D 0) > + rte_delay_ms(ICE_DCF_CHECK_INTERVAL); > + > ice_dcf_mode_disable(hw); > iavf_shutdown_adminq(&hw->avf); >=20 > diff --git a/drivers/net/ice/ice_dcf.h b/drivers/net/ice/ice_dcf.h index > 7f42ebabe9..7becf6d187 100644 > --- a/drivers/net/ice/ice_dcf.h > +++ b/drivers/net/ice/ice_dcf.h > @@ -105,6 +105,8 @@ struct ice_dcf_hw { > void (*vc_event_msg_cb)(struct ice_dcf_hw *dcf_hw, > uint8_t *msg, uint16_t msglen); >=20 > + int vsi_update_thread_num; > + > uint8_t *arq_buf; >=20 > uint16_t num_vfs; > diff --git a/drivers/net/ice/ice_dcf_parent.c > b/drivers/net/ice/ice_dcf_parent.c > index 01e390ddda..6ee0fbcd2b 100644 > --- a/drivers/net/ice/ice_dcf_parent.c > +++ b/drivers/net/ice/ice_dcf_parent.c > @@ -154,6 +154,9 @@ ice_dcf_vsi_update_service_handler(void *param) >=20 > free(param); >=20 > + __atomic_fetch_sub(&hw->vsi_update_thread_num, 1, > + __ATOMIC_RELEASE); > + > return NULL; > } >=20 > @@ -183,6 +186,9 @@ start_vsi_reset_thread(struct ice_dcf_hw *dcf_hw, > bool vfr, uint16_t vf_id) > PMD_DRV_LOG(ERR, "Failed to start the thread for reset > handling"); > free(param); > } > + > + __atomic_fetch_add(&dcf_hw->vsi_update_thread_num, 1, > + __ATOMIC_RELAXED); Is this correct? you increment the counter after you start the new thread w= ith rte_ctrl_thread_create,=20 It is possible, __atomic_fetch_sub in ice_dcf_vsi_update_service_handler wi= ll be executed before __atomic_fetch_add? Why not moving __atomic_fetch_add to the service_handler thread? So the ord= er of "add" and "sub" can be guaranteed ? > } >=20 > static uint32_t > -- > 2.25.1