From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 033C2A0AC5 for ; Thu, 2 May 2019 17:42:10 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C35961B128; Thu, 2 May 2019 17:42:10 +0200 (CEST) Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-eopbgr150103.outbound.protection.outlook.com [40.107.15.103]) by dpdk.org (Postfix) with ESMTP id C91131B115 for ; Thu, 2 May 2019 17:42:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Enea.onmicrosoft.com; s=selector1-owmobility-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=V+VpxDoaC/q53JPGsBnxtKCRNUAk9EoaeANh7WSyeCM=; b=SqogNUhhsNinM5x8aKQ5uJqOY+TYTwqUA+JiVeaSGnhuI8RXuFJLr6jtuAr6Kur+JRCxA+m14/sNHQHmXT5xNvNf5FBWd8TRYAQFK6vkA36pGD6/cK7h9hheSXVQ4Q1MTjUZdlBDgrgTH++mcrGQhNf0nde/hV8lWhD+8RHK3uA= Received: from VI1PR0701MB2142.eurprd07.prod.outlook.com (10.169.136.140) by VI1PR0701MB2207.eurprd07.prod.outlook.com (10.169.137.134) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1856.5; Thu, 2 May 2019 15:42:06 +0000 Received: from VI1PR0701MB2142.eurprd07.prod.outlook.com ([fe80::2d93:5295:1e83:8a5d]) by VI1PR0701MB2142.eurprd07.prod.outlook.com ([fe80::2d93:5295:1e83:8a5d%7]) with mapi id 15.20.1856.008; Thu, 2 May 2019 15:42:06 +0000 From: Greg O'Rawe To: "users@dpdk.org" Thread-Topic: DPDK not working with ConnectX-3 card on Openstack virtual setup Thread-Index: AdUA/Rj9B1UhzaKZTta32W3FPp7xPA== Date: Thu, 2 May 2019 15:42:06 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=greg.orawe@owmobility.com; x-originating-ip: [2a02:c7f:8680:3700:3c15:197f:5d9d:51ed] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 6b8ec3e8-d09f-4fb0-95d0-08d6cf14bb4f x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600141)(711020)(4605104)(2017052603328)(7193020); SRVR:VI1PR0701MB2207; x-ms-traffictypediagnostic: VI1PR0701MB2207: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:41; x-forefront-prvs: 0025434D2D x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(39850400004)(366004)(346002)(136003)(396003)(376002)(189003)(199004)(256004)(7696005)(14444005)(5024004)(66556008)(476003)(99286004)(54896002)(66946007)(6306002)(9686003)(46003)(52536014)(186003)(790700001)(2351001)(86362001)(66476007)(73956011)(53936002)(76116006)(64756008)(66446008)(5640700003)(486006)(55016002)(14454004)(9326002)(6116002)(5660300002)(1730700003)(71200400001)(316002)(71190400001)(81166006)(2906002)(8676002)(81156014)(7736002)(8936002)(6436002)(478600001)(33656002)(6916009)(25786009)(68736007)(74316002)(102836004)(2501003)(6506007); DIR:OUT; SFP:1102; SCL:1; SRVR:VI1PR0701MB2207; H:VI1PR0701MB2142.eurprd07.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: owmobility.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: bGooHeFnzeO1xrM5kHenFW6YxAMjqFfhurEXJ44Y4yUn++aYsId9WcVQ44cGDoaRfdfZGZ3auPJG6gHgqGwzLdQ+oSzShLkMaAUeRHG0/P/7ufRZW5uYf8wgzgeYEB17D/ksh3vwN3jyOij3MscRAGQWEdoynUhmwvWxzAZBkeBS2TOI0YKC0DrGeS9vc/GXi8ld3DNlBb3g5AKhk6F+SNV0z+NCGRTaDS4Nwedgen34cFdssrbYyxbPeMSptlfa0+/lb8SWmyDaHEBy1Nrvc4EDHQdiRYV8UtV2V+x9F+2Qtyc47VrUPek/r/8qXqfeuopC9IDTNMIqGsv63uBq9LdUPS/1MT5d6nErv7IUSiW0Wd8YT00VOIzNG58KoSns4zVlk0rxYtT5z7FlAjH0++9dwOoxI4IM+uQCbS8a1Tw= MIME-Version: 1.0 X-OriginatorOrg: owmobility.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6b8ec3e8-d09f-4fb0-95d0-08d6cf14bb4f X-MS-Exchange-CrossTenant-originalarrivaltime: 02 May 2019 15:42:06.6923 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 8cdaed89-c47e-40b8-90b5-f28ad2c3fdca X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0701MB2207 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-users] DPDK not working with ConnectX-3 card on Openstack virtual setup X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Sender: "users" Hi, I am trying to get DPDK 17.11.4 to run with a ConnectX-3 card on a virtual = environment using Openstack. This uses the VFIO driver which initialises correctly (though running in no= -IOMMU mode). However starting DPDK via the testpmd binary fails trying to = add default flows to the device. mlx_fe-fe-0$ /root/testpmd -c 0xf -n 4 -w 0000:00:06.0 -w 0000:00:08.0 -- = --rxq=3D2 --txq=3D2 -i EAL: Detected 12 lcore(s) EAL: No free hugepages reported in hugepages-1048576kB EAL: Probing VFIO support... EAL: WARNING: cpu flags constant_tsc=3Dyes nonstop_tsc=3Dno -> using unreli= able clock cycles ! EAL: PCI device 0000:00:06.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 15b3:1004 net_mlx4 PMD: net_mlx4: PCI information matches, using device "mlx4_0" (VF: true) PMD: net_mlx4: 1 port(s) detected PMD: net_mlx4: port 1 MAC address is fa:16:3e:c6:5b:df EAL: PCI device 0000:00:08.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 15b3:1004 net_mlx4 PMD: net_mlx4: PCI information matches, using device "mlx4_2" (VF: true) PMD: net_mlx4: 1 port(s) detected PMD: net_mlx4: port 1 MAC address is fa:16:3e:d9:f9:9d Interactive-mode selected USER1: create a new mbuf pool : n=3D171456, size=3D2176= , socket=3D0 Configuring Port 0 (socket 0) PMD: net_mlx4: 0x55e3598ee200: cannot attach flow rules (code 12, "Cannot a= llocate memory"), flow error type 2, cause 0x13de8ee40, message: flow rule = rejected by device Fail to start port 0 Configuring Port 1 (socket 0) PMD: net_mlx4: 0x55e3598f2280: cannot attach flow rules (code 12, "Cannot a= llocate memory"), flow error type 2, cause 0x13de8acc0, message: flow rule = rejected by device Fail to start port 1 Please stop the ports first Done testpmd> /var/log/messages contains the following errors: May 2 15:38:56 mlx_fe-fe-0 kernel: __mlx4_ib_create_flow: mcg ta= ble is full. Fail to register network rule. May 2 15:38:56 mlx_fe-fe-0 testpmd[27582]: PMD: net_mlx4: 0x55e3598ee200: = cannot attach flow rules (code 12, "Cannot allocate memory"), flow error ty= pe 2, cause 0x13de8ee40, message: flow rule rejected by device May 2 15:38:56 mlx_fe-fe-0 kernel: __mlx4_ib_create_flow: mcg ta= ble is full. Fail to register network rule. May 2 15:38:56 mlx_fe-fe-0 testpmd[27582]: PMD: net_mlx4: 0x55e3598f2280: = cannot attach flow rules (code 12, "Cannot allocate memory"), flow error ty= pe 2, cause 0x13de8acc0, message: flow rule rejected by device It seems that adding default flows fails due to the mcg table being full. What could be the cause of this error? How is the mcg table configured? Thanks Greg O'Rawe This message, including attachments, is CONFIDENTIAL. It may also be privil= eged or otherwise protected by law. If you received this email by mistake p= lease let us know by reply and then delete it from your system; you should = not copy it or disclose its contents to anyone. All messages sent to and fr= om Enea may be monitored to ensure compliance with internal policies and to= protect our business. Emails are not secure and cannot be guaranteed to be= error free as they can be intercepted, a mended, lost or destroyed, or con= tain viruses. The sender therefore does not accept liability for any errors= or omissions in the contents of this message, which arise as a result of e= mail transmission. Anyone who communicates with us by email accepts these r= isks.