From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id C7032A0096 for ; Thu, 9 May 2019 09:39:08 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A761D4F90; Thu, 9 May 2019 09:39:08 +0200 (CEST) Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-eopbgr40092.outbound.protection.outlook.com [40.107.4.92]) by dpdk.org (Postfix) with ESMTP id 3F5E94CA0 for ; Thu, 9 May 2019 09:39:07 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Enea.onmicrosoft.com; s=selector1-Enea-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5JKD0YZwCJQJyaJxWELjasvgkz9JbSmCx41IAgiGp8s=; b=YaabjtGYdm2u65sMNiBqZn4+qqPy7LYRgMkdpnEkj0n6DGif1Pt5+ZK2cR1IacX3L1iBZ4ii5OugSllF3nHmShbVwTom4HSqzun/jd2MMq7drNxVr3SIkyodkTz4hacmtONOANPxDNU13WKpIkYjy4KnWynVBCLXiwa+1R7PIX0= Received: from VI1PR0701MB2142.eurprd07.prod.outlook.com (10.169.136.140) by VI1PR0701MB2655.eurprd07.prod.outlook.com (10.173.78.149) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1900.5; Thu, 9 May 2019 07:39:05 +0000 Received: from VI1PR0701MB2142.eurprd07.prod.outlook.com ([fe80::954e:297b:7a96:6049]) by VI1PR0701MB2142.eurprd07.prod.outlook.com ([fe80::954e:297b:7a96:6049%3]) with mapi id 15.20.1878.019; Thu, 9 May 2019 07:39:05 +0000 From: Greg O'Rawe To: "users@dpdk.org" Thread-Topic: [dpdk-users] DPDK not working with ConnectX-3 card on Openstack virtual setup Thread-Index: AdUA/Rj9B1UhzaKZTta32W3FPp7xPAFPIZug Date: Thu, 9 May 2019 07:39:05 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=greg.orawe@owmobility.com; x-originating-ip: [2a00:2381:757:60:1459:1775:3f5:de58] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: d2c05314-7c9a-44f1-f873-08d6d45169ed x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600141)(711020)(4605104)(2017052603328)(7193020); SRVR:VI1PR0701MB2655; x-ms-traffictypediagnostic: VI1PR0701MB2655: x-ms-exchange-purlcount: 1 x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:93; x-forefront-prvs: 003245E729 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(346002)(366004)(136003)(376002)(396003)(39850400004)(13464003)(199004)(189003)(99286004)(6246003)(2501003)(68736007)(55016002)(6306002)(9686003)(5660300002)(81166006)(81156014)(76176011)(1730700003)(53546011)(6506007)(478600001)(5640700003)(229853002)(14454004)(71200400001)(71190400001)(86362001)(305945005)(74316002)(25786009)(966005)(2906002)(8936002)(2351001)(7736002)(6436002)(102836004)(11346002)(446003)(76116006)(73956011)(66946007)(52536014)(186003)(53936002)(64756008)(66446008)(256004)(5024004)(14444005)(7696005)(46003)(316002)(6916009)(486006)(476003)(6116002)(33656002)(66556008)(66476007); DIR:OUT; SFP:1102; SCL:1; SRVR:VI1PR0701MB2655; H:VI1PR0701MB2142.eurprd07.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: owmobility.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: h/R8u+VbvtPyuuzAJmeF6ABagu28is4N7k5xICro37YWo6WtDXbpLTg7yFWWc5BWuwaw3VC2cXp20j7ZtW/fxL03aOIcavayH/RNqjWX7pFTLukeDFKaEru2e5qWPkNNr9/BdF6bNBGZvznDI9tTb5VzPfdb1cH2ND/AqK2Wg2eZm/OfVgInEIyrXWx0qfj6YGQRPkLdBOnS8pOAv1RiPCAtCYt3eSsS1Pzn8dDrsgNoSA1omTtBULb9Drx/f5sCxtIJhVbq7aR7KnocnTPhy+6KZaddz3emCCGR3jorcdH44Fg5RzmhmtBE53Pnhq+cjxXRi+ulxLvE0OYsR+9YRMNO+z2jpU0PwM69ifqo+SHkEqJEY2c4DHCFMHCkroDCuZ//B8BSbBPjGeNNJ3rH+AYeEutlpbi/tJ3WIuyBdJ0= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: owmobility.com X-MS-Exchange-CrossTenant-Network-Message-Id: d2c05314-7c9a-44f1-f873-08d6d45169ed X-MS-Exchange-CrossTenant-originalarrivaltime: 09 May 2019 07:39:05.2515 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 8cdaed89-c47e-40b8-90b5-f28ad2c3fdca X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0701MB2655 Subject: Re: [dpdk-users] DPDK not working with ConnectX-3 card on Openstack virtual setup X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Sender: "users" Hi, I see the same issue was raised here on this thread https://mails.dpdk.org/= archives/users/2018-October/003620.html Any ideas on a possible solution? I didn't see any response on that thread. Many thanks Greg -----Original Message----- From: users On Behalf Of Greg O'Rawe Sent: 02 May 2019 16:42 To: users@dpdk.org Subject: [dpdk-users] DPDK not working with ConnectX-3 card on Openstack vi= rtual setup Hi, I am trying to get DPDK 17.11.4 to run with a ConnectX-3 card on a virtual = environment using Openstack. This uses the VFIO driver which initialises correctly (though running in no= -IOMMU mode). However starting DPDK via the testpmd binary fails trying to = add default flows to the device. mlx_fe-fe-0$ /root/testpmd -c 0xf -n 4 -w 0000:00:06.0 -w 0000:00:08.0 -- = --rxq=3D2 --txq=3D2 -i EAL: Detected 12 lcore(s) EAL: No free hugepages reported in hugepages-1048576kB EAL: Probing VFIO support... EAL: WARNING: cpu flags constant_tsc=3Dyes nonstop_tsc=3Dno -> using unreli= able clock cycles ! EAL: PCI device 0000:00:06.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 15b3:1004 net_mlx4 PMD: net_mlx4: PCI information matches, using device "mlx4_0" (VF: true) PMD: net_mlx4: 1 port(s) detected PMD: net_mlx4: port 1 MAC address is fa:16:3e:c6:5b:df EAL: PCI device 0000:00:08.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 15b3:1004 net_mlx4 PMD: net_mlx4: PCI information matches, using device "mlx4_2" (VF: true) PMD: net_mlx4: 1 port(s) detected PMD: net_mlx4: port 1 MAC address is fa:16:3e:d9:f9:9d Interactive-mode sel= ected USER1: create a new mbuf pool : n=3D171456, size=3D2176= , socket=3D0 Configuring Port 0 (socket 0) PMD: net_mlx4: 0x55e3598ee200: cannot attach flow rules (code 12, "Cannot a= llocate memory"), flow error type 2, cause 0x13de8ee40, message: flow rule = rejected by device Fail to start port 0 Configuring Port 1 (socket 0) PMD: net_mlx4: 0x55e3598f2280: cannot attach flow rules (code 12, "Cannot a= llocate memory"), flow error type 2, cause 0x13de8acc0, message: flow rule = rejected by device Fail to start port 1 Please stop the ports first Done testpmd> /var/log/messages contains the following errors: May 2 15:38:56 mlx_fe-fe-0 kernel: __mlx4_ib_create_flow: mcg ta= ble is full. Fail to register network rule. May 2 15:38:56 mlx_fe-fe-0 testpmd[27582]: PMD: net_mlx4: 0x55e3598ee200: = cannot attach flow rules (code 12, "Cannot allocate memory"), flow error ty= pe 2, cause 0x13de8ee40, message: flow rule rejected by device May 2 15:38= :56 mlx_fe-fe-0 kernel: __mlx4_ib_create_flow: mcg table is full.= Fail to register network rule. May 2 15:38:56 mlx_fe-fe-0 testpmd[27582]: PMD: net_mlx4: 0x55e3598f2280: = cannot attach flow rules (code 12, "Cannot allocate memory"), flow error ty= pe 2, cause 0x13de8acc0, message: flow rule rejected by device It seems that adding default flows fails due to the mcg table being full. What could be the cause of this error? How is the mcg table configured? Thanks Greg O'Rawe This message, including attachments, is CONFIDENTIAL. It may also be privil= eged or otherwise protected by law. If you received this email by mistake p= lease let us know by reply and then delete it from your system; you should = not copy it or disclose its contents to anyone. All messages sent to and fr= om Enea may be monitored to ensure compliance with internal policies and to= protect our business. Emails are not secure and cannot be guaranteed to be= error free as they can be intercepted, a mended, lost or destroyed, or con= tain viruses. The sender therefore does not accept liability for any errors= or omissions in the contents of this message, which arise as a result of e= mail transmission. Anyone who communicates with us by email accepts these r= isks. This message, including attachments, is CONFIDENTIAL. It may also be privil= eged or otherwise protected by law. If you received this email by mistake p= lease let us know by reply and then delete it from your system; you should = not copy it or disclose its contents to anyone. All messages sent to and fr= om Enea may be monitored to ensure compliance with internal policies and to= protect our business. Emails are not secure and cannot be guaranteed to be= error free as they can be intercepted, a mended, lost or destroyed, or con= tain viruses. The sender therefore does not accept liability for any errors= or omissions in the contents of this message, which arise as a result of e= mail transmission. Anyone who communicates with us by email accepts these r= isks.