From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id C81B4A04B3; Sat, 4 Jan 2020 11:59:03 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0FD571D44C; Sat, 4 Jan 2020 11:59:03 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 312841D44B for ; Sat, 4 Jan 2020 11:59:01 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 004AuVxl031155; Sat, 4 Jan 2020 02:59:00 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=pfpt0818; bh=I8ozvrAEl8qjaFl7+JUw+ABzL2WBeH4DtWxbwhnDvgY=; b=gl7l5ti5TI9irZus3xDboPB+QHw3iEfOg4qewoYy9R9MLlfBWFpzD0aWcb3gAovJAGqy HL81h4HL52Vf9duXsl/zaV6GA8A/Z8/1eJiEO3VIC54ODd21fZKmssM47s+PqZx8w/8P qE8ffp9lErMudC12dBk435RG2iSjrZm0Hs+xNziJzKgrAnNJkfUS4ALYvsEC4ESNO6oO dBc+WUc14G5y1KYSLWT2L0IjtM74RNZShE+UxJCIRqfnsjU8TGsUjvw3FHp+AwYCSUgr Cf6XzLUBaJUAUyNSgCWGqkwCa5tN8mNZHUdFJ/6ijo5jyEdJjIJBqxGe73+dMbjxLRbD CA== Received: from sc-exch03.marvell.com ([199.233.58.183]) by mx0a-0016f401.pphosted.com with ESMTP id 2xarxv01qb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sat, 04 Jan 2020 02:59:00 -0800 Received: from SC-EXCH04.marvell.com (10.93.176.84) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 4 Jan 2020 02:58:58 -0800 Received: from NAM02-BL2-obe.outbound.protection.outlook.com (104.47.38.58) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Sat, 4 Jan 2020 02:58:58 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ibE9ForRD1TLcmyfSUu9XoeKKBdCmCorqfG8kpF8HvheADWU9ph7glHc3WY50Q8+6pwBwPrFUWXL6ZrL9PcJCPbYHVejySlwACJ1KtkkWfXB4PUluRCcesXrpVRyy8pNYH2KM7UrEgH+sUy4Sol/UV9p89sw8I/ClA8/cE3PtXf2DKR9IKsG42uMKbOSLmedexuI1kv/qKxQUBrrrKTWF1wJKoYKlMCh5SArZRRdBlUA7ykPHPIiEPJBCascEddM6SxZ9r4OT8ZFUfRWEOlW8UBsMpNY9sfQzqOFN3Fb9XRv6PkpsXBfH29p1uG+GVT0q7YYVeVP+0op9RXbEMaZQQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=I8ozvrAEl8qjaFl7+JUw+ABzL2WBeH4DtWxbwhnDvgY=; b=Uoyjz+4rcSGVDS6TPSW5/zOJrUDWdMnQRN3JNxYj1WhDEFOR8vITL++te8zINfPe/ndTPYiaoVyUvuClyZHr4jXYTX2FK1g5fC3RVIkZc48IcuRuN5w7v6R9SHLwt4FBFp+x0/IoUw2AUT5RNYFXDmxGGstvDZvzLTH7ye9fTZlMkITJmaCgn609/It0lGRyij+kes7hLHQtwDmguX+8Zd7KDh/NWplsYtk7dbxTyhYB4UCN4CH2w8RC4KylUiqgZzVbNVmgMKJA6aMcR3bxRb5yWK+SC6od25jfWLUez9Lh9mydE2/fTYlpxqyCNDk+cpEbpZ3PJBLEn9UrT3L2gQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=I8ozvrAEl8qjaFl7+JUw+ABzL2WBeH4DtWxbwhnDvgY=; b=NBkKmIHteq9c7eS/4guAzv2NukmatYlADUS97t+seSbd7kCOycNIBbHL3S9WOGgRXsLBqxw+nI+EVSO1xNYjQnGR/SnZHV5rGJ1nwMeN6FI/WiMR8o7VsU88AivIruMVA7CRTgTKUq+et7vyWkQQ77kBL9eT4CozIeG1xttuS20= Received: from MN2PR18MB2877.namprd18.prod.outlook.com (20.179.20.218) by MN2PR18MB2926.namprd18.prod.outlook.com (20.179.21.33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2602.13; Sat, 4 Jan 2020 10:58:56 +0000 Received: from MN2PR18MB2877.namprd18.prod.outlook.com ([fe80::5db5:d179:8a01:4636]) by MN2PR18MB2877.namprd18.prod.outlook.com ([fe80::5db5:d179:8a01:4636%7]) with mapi id 15.20.2602.012; Sat, 4 Jan 2020 10:58:56 +0000 From: Anoob Joseph To: "Ananyev, Konstantin" , Akhil Goyal , "Nicolau, Radu" , "Thomas Monjalon" CC: Ankur Dwivedi , Jerin Jacob Kollanukkaran , Narayana Prasad Raju Athreya , Archana Muniganti , Tejasree Kondoj , Vamsi Krishna Attunuru , Lukas Bartosik , "dev@dpdk.org" Thread-Topic: [PATCH 12/14] examples/ipsec-secgw: add driver outbound worker Thread-Index: AQHVrcOLu44JOaT3nEuR4uUFYjJTFqfIES8AgBDSGAA= Date: Sat, 4 Jan 2020 10:58:56 +0000 Message-ID: References: <1575808249-31135-1-git-send-email-anoobj@marvell.com> <1575808249-31135-13-git-send-email-anoobj@marvell.com> In-Reply-To: Accept-Language: en-IN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [111.125.206.172] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: de3ef244-9c2e-435c-e09c-08d791051864 x-ms-traffictypediagnostic: MN2PR18MB2926: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:6790; x-forefront-prvs: 02723F29C4 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(376002)(366004)(136003)(39850400004)(346002)(396003)(13464003)(199004)(189003)(55016002)(5660300002)(52536014)(26005)(186003)(7696005)(66946007)(86362001)(33656002)(71200400001)(76116006)(66446008)(55236004)(64756008)(66556008)(66476007)(9686003)(8676002)(54906003)(478600001)(8936002)(110136005)(316002)(2906002)(53546011)(81166006)(81156014)(4326008)(6506007); DIR:OUT; SFP:1101; SCL:1; SRVR:MN2PR18MB2926; H:MN2PR18MB2877.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: marvell.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: sZwgy540DTsFRZGR2gd4ii0V0MAyTCaQLil2uSfKlRuL+1Dcs+Ln/7SSvmkakWNale6e2PSNk4e6hnX0z+0QIXyN8x50G9N6kyowZHD0e9FIwR+w+ChZqeaEZvgEf15H5RlhQHv0HcLMCyEnrBlWXPtfhijIigSuhTK4I0oEEi5FgQxr/HKl3/L2bUZr8fwtEQiZylEfeLKWb+eqsZ9DZveVtdVRvgTond2ocDYmGxXoar4tQUv/g+V66qwhd5dx4w5fUc2poGsaYR+zf0/Igf2dkCrM1WMUB2f07bzV8FIzuUecD9K7rPJ7BegkJnSC+q0m7pKNyvVbrBZhmJkBsk1jelKvv61+nCJpliAYVmyKJIxM+H2hBInlm9ysVYSke6yoF9H9hUfZ+CbEi8Ue2mrFT1WT6s8LQZU31DHNVYpy8s5uCU4fqJ2fmvIBknkjfGEsz7OZWghgKAaHa3T+iHDH7LJuKyEuTt+wr+0CDCvhETAHAAR37FrIwKDZDJm+ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: de3ef244-9c2e-435c-e09c-08d791051864 X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Jan 2020 10:58:56.4933 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: oLdHNgKc5H+o3srG2sVACsANnOK3LrF5IGnnEFXctDeGF0mmwt6Mv8cn4/A0FzPfbODQlfynxxdxw+dRG+75mA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR18MB2926 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2020-01-04_01:2020-01-02,2020-01-04 signatures=0 Subject: Re: [dpdk-dev] [PATCH 12/14] examples/ipsec-secgw: add driver outbound worker X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Konstantin, Please see inline. Thanks, Anoob > -----Original Message----- > From: Ananyev, Konstantin > Sent: Monday, December 23, 2019 10:58 PM > To: Anoob Joseph ; Akhil Goyal > ; Nicolau, Radu ; Thomas > Monjalon > Cc: Ankur Dwivedi ; Jerin Jacob Kollanukkaran > ; Narayana Prasad Raju Athreya > ; Archana Muniganti ; > Tejasree Kondoj ; Vamsi Krishna Attunuru > ; Lukas Bartosik ; > dev@dpdk.org > Subject: [EXT] RE: [PATCH 12/14] examples/ipsec-secgw: add driver > outbound worker >=20 > External Email >=20 > ---------------------------------------------------------------------- >=20 > > This patch adds the driver outbound worker thread for ipsec-secgw. > > In this mode the security session is a fixed one and sa update is not > > done. > > > > Signed-off-by: Ankur Dwivedi > > Signed-off-by: Anoob Joseph > > Signed-off-by: Lukasz Bartosik > > --- > > examples/ipsec-secgw/ipsec-secgw.c | 12 +++++ > > examples/ipsec-secgw/ipsec.c | 9 ++++ > > examples/ipsec-secgw/ipsec_worker.c | 90 > > ++++++++++++++++++++++++++++++++++++- > > 3 files changed, 110 insertions(+), 1 deletion(-) > > > > diff --git a/examples/ipsec-secgw/ipsec-secgw.c > > b/examples/ipsec-secgw/ipsec-secgw.c > > index 2e7d4d8..76719f2 100644 > > --- a/examples/ipsec-secgw/ipsec-secgw.c > > +++ b/examples/ipsec-secgw/ipsec-secgw.c > > @@ -2011,6 +2011,18 @@ cryptodevs_init(void) > > i++; > > } > > > > + /* > > + * Set the queue pair to at least the number of ethernet > > + * devices for inline outbound. > > + */ > > + qp =3D RTE_MAX(rte_eth_dev_count_avail(), qp); >=20 >=20 > Not sure, what for? > Why we can't process packets from several eth devs on the same crypto-dev > queue? [Anoob] This is because of a limitation in our hardware. In our hardware, i= t's the crypto queue pair which would be submitting to the ethernet queue f= or Tx. But in DPDK spec, the security processing is done by the ethernet PM= D Tx routine alone. We manage to do this by sharing the crypto queue intern= ally. The crypto queues initialized during crypto_configure() gets mapped t= o various ethernet ports. Because of this, we need to have atleast as many = crypto queues as the number of eth ports. The above change is required because here we limit the number of crypto qps= based on the number of cores etc. So when tried on single core, the qps ge= t limited to 1, which causes session_create() to fail for all ports other t= han the first one. >=20 > > + > > + /* > > + * The requested number of queues should never exceed > > + * the max available > > + */ > > + qp =3D RTE_MIN(qp, max_nb_qps); > > + > > if (qp =3D=3D 0) > > continue; > > > > diff --git a/examples/ipsec-secgw/ipsec.c > > b/examples/ipsec-secgw/ipsec.c index e529f68..9ff8a63 100644 > > --- a/examples/ipsec-secgw/ipsec.c > > +++ b/examples/ipsec-secgw/ipsec.c > > @@ -141,6 +141,10 @@ create_lookaside_session(struct ipsec_ctx > *ipsec_ctx, struct ipsec_sa *sa, > > return 0; > > } > > > > +uint16_t sa_no; > > +#define MAX_FIXED_SESSIONS 10 > > +struct rte_security_session *sec_session_fixed[MAX_FIXED_SESSIONS]; > > + > > int > > create_inline_session(struct socket_ctx *skt_ctx, struct ipsec_sa *sa, > > struct rte_ipsec_session *ips) > > @@ -401,6 +405,11 @@ create_inline_session(struct socket_ctx *skt_ctx, > > struct ipsec_sa *sa, > > > > ips->security.ol_flags =3D sec_cap->ol_flags; > > ips->security.ctx =3D sec_ctx; > > + if (sa_no < MAX_FIXED_SESSIONS) { > > + sec_session_fixed[sa_no] =3D > > + ipsec_get_primary_session(sa)- > >security.ses; > > + sa_no++; > > + } > > } >=20 > Totally lost what is the purpose of these changes... > Why first 10 inline-proto are special and need to be saved inside global = array > (sec_session_fixed)? > Why later, in ipsec_worker.c this array is referenced by eth port_id? > What would happen if number of inline-proto sessions is less than number = of > eth ports? [Anoob] This is required for the outbound driver mode. The 'driver mode' is= more like 'single_sa' mode of the existing application. The idea is to ski= p all the lookups etc done in the s/w and perform ipsec processing fully in= h/w. In outbound, following is roughly what we should do for driver mode, pkt =3D rx_burst(); /* set_pkt_metadata() */ pkt-> udata64 =3D session; tx_burst(pkt); The session is created on eth ports. And so, if we have single SA, then the= entire traffic will have to be forwarded on the same port. The above chang= e is to make sure we could send traffic on all ports. Currently we just use the first 10 SAs and save it in the array. So the use= r has to set the conf properly and make sure the SAs are distributed such. = Will update this to save the first parsed outbound SA for a port in the arr= ay. That way the size of the array will be RTE_MAX_ETHPORTS. Is the above approach fine? >=20 > > set_cdev_id: > > diff --git a/examples/ipsec-secgw/ipsec_worker.c > > b/examples/ipsec-secgw/ipsec_worker.c > > index 2af9475..e202277 100644 > > --- a/examples/ipsec-secgw/ipsec_worker.c > > +++ b/examples/ipsec-secgw/ipsec_worker.c > > @@ -263,7 +263,7 @@ process_ipsec_ev_inbound(struct ipsec_ctx *ctx, > struct route_table *rt, > > */ > > > > /* Workers registered */ > > -#define IPSEC_EVENTMODE_WORKERS 2 > > +#define IPSEC_EVENTMODE_WORKERS 3 > > > > /* > > * Event mode worker > > @@ -423,6 +423,84 @@ > ipsec_wrkr_non_burst_int_port_app_mode_inb(struct eh_event_link_info > *links, > > return; > > } > > > > +/* > > + * Event mode worker > > + * Operating parameters : non-burst - Tx internal port - driver mode > > +- outbound */ extern struct rte_security_session > > +*sec_session_fixed[]; static void > > +ipsec_wrkr_non_burst_int_port_drvr_mode_outb(struct > eh_event_link_info *links, > > + uint8_t nb_links) > > +{ > > + unsigned int nb_rx =3D 0; > > + struct rte_mbuf *pkt; > > + unsigned int port_id; > > + struct rte_event ev; > > + uint32_t lcore_id; > > + > > + /* Check if we have links registered for this lcore */ > > + if (nb_links =3D=3D 0) { > > + /* No links registered - exit */ > > + goto exit; > > + } > > + > > + /* Get core ID */ > > + lcore_id =3D rte_lcore_id(); > > + > > + RTE_LOG(INFO, IPSEC, > > + "Launching event mode worker (non-burst - Tx internal port - > " > > + "driver mode - outbound) on lcore %d\n", lcore_id); > > + > > + /* We have valid links */ > > + > > + /* Check if it's single link */ > > + if (nb_links !=3D 1) { > > + RTE_LOG(INFO, IPSEC, > > + "Multiple links not supported. Using first link\n"); > > + } > > + > > + RTE_LOG(INFO, IPSEC, " -- lcoreid=3D%u event_port_id=3D%u\n", > lcore_id, > > + links[0].event_port_id); > > + while (!force_quit) { > > + /* Read packet from event queues */ > > + nb_rx =3D rte_event_dequeue_burst(links[0].eventdev_id, > > + links[0].event_port_id, > > + &ev, /* events */ > > + 1, /* nb_events */ > > + 0 /* timeout_ticks */); > > + > > + if (nb_rx =3D=3D 0) > > + continue; > > + > > + port_id =3D ev.queue_id; > > + pkt =3D ev.mbuf; > > + > > + rte_prefetch0(rte_pktmbuf_mtod(pkt, void *)); > > + > > + /* Process packet */ > > + ipsec_event_pre_forward(pkt, port_id); > > + > > + pkt->udata64 =3D (uint64_t) sec_session_fixed[port_id]; > > + > > + /* Mark the packet for Tx security offload */ > > + pkt->ol_flags |=3D PKT_TX_SEC_OFFLOAD; > > + > > + /* > > + * Since tx internal port is available, events can be > > + * directly enqueued to the adapter and it would be > > + * internally submitted to the eth device. > > + */ > > + rte_event_eth_tx_adapter_enqueue(links[0].eventdev_id, > > + links[0].event_port_id, > > + &ev, /* events */ > > + 1, /* nb_events */ > > + 0 /* flags */); > > + } > > + > > +exit: > > + return; > > +} > > + > > static uint8_t > > ipsec_eventmode_populate_wrkr_params(struct > eh_app_worker_params > > *wrkrs) { @@ -449,6 +527,16 @@ > > ipsec_eventmode_populate_wrkr_params(struct eh_app_worker_params > *wrkrs) > > wrkr->cap.ipsec_dir =3D EH_IPSEC_DIR_TYPE_INBOUND; > > wrkr->worker_thread =3D > ipsec_wrkr_non_burst_int_port_app_mode_inb; > > > > + wrkr++; > > + nb_wrkr_param++; > > + > > + /* Non-burst - Tx internal port - driver mode - outbound */ > > + wrkr->cap.burst =3D EH_RX_TYPE_NON_BURST; > > + wrkr->cap.tx_internal_port =3D EH_TX_TYPE_INTERNAL_PORT; > > + wrkr->cap.ipsec_mode =3D EH_IPSEC_MODE_TYPE_DRIVER; > > + wrkr->cap.ipsec_dir =3D EH_IPSEC_DIR_TYPE_OUTBOUND; > > + wrkr->worker_thread =3D > ipsec_wrkr_non_burst_int_port_drvr_mode_outb; > > + > > nb_wrkr_param++; > > return nb_wrkr_param; > > } > > -- > > 2.7.4