From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50078.outbound.protection.outlook.com [40.107.5.78]) by dpdk.org (Postfix) with ESMTP id 00E301EB87 for ; Thu, 14 Jun 2018 09:59:21 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rF8UxSia4YKaih0n7lRPLMSOYoA3XmYanuoqediGBuw=; b=lu1gIb0x1drACEZTGMGz5j2Cb+MNA7LscmLo1jNWqSOtJRdHP1jwyku26ZWeGnDHbLE08dVHa/oiNnO0EofbVopNH4VkRkJcY5FwYdmpo8RcHBbT0mkTnI2FdfG8HW6f+6NkGGLHUc9/PLflWFLgqEjI3mwRQY28lH7qSZDz5Ag= Received: from HE1PR0501MB2314.eurprd05.prod.outlook.com (10.168.34.19) by HE1PR0501MB2249.eurprd05.prod.outlook.com (10.168.33.152) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.863.17; Thu, 14 Jun 2018 07:59:20 +0000 Received: from HE1PR0501MB2314.eurprd05.prod.outlook.com ([fe80::9c25:4bf1:cc2a:7e54]) by HE1PR0501MB2314.eurprd05.prod.outlook.com ([fe80::9c25:4bf1:cc2a:7e54%3]) with mapi id 15.20.0841.019; Thu, 14 Jun 2018 07:59:20 +0000 From: Ophir Munk To: "Wiles, Keith" CC: DPDK , Pascal Mazon , Thomas Monjalon , Olga Shern Thread-Topic: [PATCH v4 2/2] net/tap: support TSO (TCP Segment Offload) Thread-Index: AQHUAmrmBywwODBpg0uWcjlxc4qHN6ReW4wAgAEIEpA= Date: Thu, 14 Jun 2018 07:59:20 +0000 Message-ID: References: <1520629826-23055-2-git-send-email-ophirmu@mellanox.com> <1528821108-12405-1-git-send-email-ophirmu@mellanox.com> <1528821108-12405-3-git-send-email-ophirmu@mellanox.com> <37D23262-6A5D-4931-A874-1733643C7F95@intel.com> In-Reply-To: <37D23262-6A5D-4931-A874-1733643C7F95@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=ophirmu@mellanox.com; x-originating-ip: [193.47.165.251] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; HE1PR0501MB2249; 7:xuTmK6GCXAzvUOrqixhwpWQyXpdP0LQIC8S6ibSOy+RnUuf3dm8iMuQ+lfmXVIqiPo8xbdTfY7Po7OwGbam9vSArcsQpJegYfXJskVOJGGVhwYTyJiFuxN6fpqSMxPbUekcJFGN1sGBFvowS0A29I7Sg9cgohduKVzp+cN/Ow+YAsSs3AIEWh4MNZHlJqzoExClopsxAql26rP9s4P/YRaTpOp1gOrGlyU+6pVAs59clDJxTFb0fjQ6dJuRGmF7v x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 17e99b08-ab8e-476c-9f4f-08d5d1ccbc2a x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(48565401081)(5600026)(711020)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:HE1PR0501MB2249; x-ms-traffictypediagnostic: HE1PR0501MB2249: x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(66839620246622)(228905959029699); x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(10201501046)(93006095)(93001095)(3002001)(3231254)(2232076)(944501410)(52105095)(6055026)(149027)(150027)(6041310)(20161123558120)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123564045)(20161123560045)(6072148)(201708071742011)(7699016); SRVR:HE1PR0501MB2249; BCL:0; PCL:0; RULEID:; SRVR:HE1PR0501MB2249; x-forefront-prvs: 0703B549E4 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(396003)(39380400002)(39860400002)(346002)(376002)(366004)(199004)(189003)(13464003)(316002)(66066001)(54906003)(68736007)(59450400001)(5660300001)(97736004)(6916009)(105586002)(4326008)(3660700001)(14454004)(99286004)(6506007)(102836004)(9686003)(53546011)(6116002)(3846002)(93886005)(7696005)(476003)(446003)(76176011)(486006)(11346002)(6436002)(86362001)(575784001)(55016002)(8676002)(186003)(26005)(7736002)(2906002)(305945005)(33656002)(3280700002)(8936002)(2900100001)(81156014)(81166006)(478600001)(6246003)(106356001)(5250100002)(25786009)(229853002)(107886003)(74316002)(53936002)(59010400001); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR0501MB2249; H:HE1PR0501MB2314.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: C5NucyxSj9yir+PaWuCOYH54VRWmwzrkLwpynq9i9CFdpzvvMqhccLpUIJqSJHKHBbffFwNYyl4FR7BHvn5Pgi00Hkf+OmEjzfH3Ayg+kLO0mCI3d1oahIUzgFV+vpKqiwhKFysD4hu7V4cgl6G6v2K7HHf0tajfFVSdQhXxtppDe6XGOjgoMsS8Iw8WIznQsd3vr59HJZVF93tx4joSHA8NicKkYxlTCVuEBR07Q8BthplyWRsAAuX/2efILxUYC4UCgZJLc05jxHVEOQucH2TuVHSqLGCTaxgY1kzow60DeJrOazqnnIK7QMJf2oYFIpYT9/vgucrAk1OtfGdYHA== spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 17e99b08-ab8e-476c-9f4f-08d5d1ccbc2a X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Jun 2018 07:59:20.2612 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0501MB2249 Subject: Re: [dpdk-dev] [PATCH v4 2/2] net/tap: support TSO (TCP Segment Offload) X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jun 2018 07:59:22 -0000 > -----Original Message----- > From: Wiles, Keith [mailto:keith.wiles@intel.com] > Sent: Wednesday, June 13, 2018 7:04 PM > To: Ophir Munk > Cc: DPDK ; Pascal Mazon ; > Thomas Monjalon ; Olga Shern > > Subject: Re: [PATCH v4 2/2] net/tap: support TSO (TCP Segment Offload) >=20 >=20 >=20 > > On Jun 12, 2018, at 11:31 AM, Ophir Munk > wrote: > > > > This commit implements TCP segmentation offload in TAP. > > librte_gso library is used to segment large TCP payloads (e.g. packets > > of 64K bytes size) into smaller MTU size buffers. > > By supporting TSO offload capability in software a TAP device can be > > used as a failsafe sub device and be paired with another PCI device > > which supports TSO capability in HW. > > > > For more details on librte_gso implementation please refer to dpdk > > documentation. > > The number of newly generated TCP TSO segments is limited to 64. > > > > Reviewed-by: Raslan Darawsheh > > Signed-off-by: Ophir Munk > > --- > > drivers/net/tap/Makefile | 2 +- > > drivers/net/tap/rte_eth_tap.c | 159 > +++++++++++++++++++++++++++++++++++------- > > drivers/net/tap/rte_eth_tap.h | 3 + > > mk/rte.app.mk | 4 +- > > 4 files changed, 138 insertions(+), 30 deletions(-) > > > > diff --git a/drivers/net/tap/Makefile b/drivers/net/tap/Makefile index > > ccc5c5f..3243365 100644 > > --- a/drivers/net/tap/Makefile > > +++ b/drivers/net/tap/Makefile > > @@ -24,7 +24,7 @@ CFLAGS +=3D -I. > > CFLAGS +=3D $(WERROR_FLAGS) > > LDLIBS +=3D -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring LDLIBS +=3D > > -lrte_ethdev -lrte_net -lrte_kvargs -lrte_hash -LDLIBS +=3D > > -lrte_bus_vdev > > +LDLIBS +=3D -lrte_bus_vdev -lrte_gso > > > > CFLAGS +=3D -DTAP_MAX_QUEUES=3D$(TAP_MAX_QUEUES) > > > > diff --git a/drivers/net/tap/rte_eth_tap.c > > b/drivers/net/tap/rte_eth_tap.c index c19f053..62b931f 100644 > > --- a/drivers/net/tap/rte_eth_tap.c > > +++ b/drivers/net/tap/rte_eth_tap.c > > @@ -17,6 +17,7 @@ > > #include > > #include > > > > +#include > > #include > > #include > > #include > > @@ -55,6 +56,9 @@ > > #define ETH_TAP_CMP_MAC_FMT "0123456789ABCDEFabcdef" > > #define ETH_TAP_MAC_ARG_FMT ETH_TAP_MAC_FIXED "|" > ETH_TAP_USR_MAC_FMT > > > > +#define TAP_GSO_MBUFS_NUM 128 > > +#define TAP_GSO_MBUF_SEG_SIZE 128 > > + > > static struct rte_vdev_driver pmd_tap_drv; static struct > > rte_vdev_driver pmd_tun_drv; > > > > @@ -412,7 +416,8 @@ tap_tx_offload_get_queue_capa(void) > > return DEV_TX_OFFLOAD_MULTI_SEGS | > > DEV_TX_OFFLOAD_IPV4_CKSUM | > > DEV_TX_OFFLOAD_UDP_CKSUM | > > - DEV_TX_OFFLOAD_TCP_CKSUM; > > + DEV_TX_OFFLOAD_TCP_CKSUM | > > + DEV_TX_OFFLOAD_TCP_TSO; > > } > > > > /* Finalize l4 checksum calculation */ @@ -479,23 +484,15 @@ > > tap_tx_l3_cksum(char *packet, uint64_t ol_flags, unsigned int l2_len, > > } > > } > > > > -/* Callback to handle sending packets from the tap interface > > - */ > > -static uint16_t > > -pmd_tx_burst(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) > > +static inline void > > +tap_write_mbufs(struct tx_queue *txq, uint16_t num_mbufs, > > + struct rte_mbuf **pmbufs, uint16_t l234_hlen, > > + uint16_t *num_packets, unsigned long > *num_tx_bytes) > > { > > - struct tx_queue *txq =3D queue; > > - uint16_t num_tx =3D 0; > > - unsigned long num_tx_bytes =3D 0; > > - uint32_t max_size; > > int i; > > > > - if (unlikely(nb_pkts =3D=3D 0)) > > - return 0; > > - > > - max_size =3D *txq->mtu + (ETHER_HDR_LEN + ETHER_CRC_LEN + 4); > > - for (i =3D 0; i < nb_pkts; i++) { > > - struct rte_mbuf *mbuf =3D bufs[num_tx]; > > + for (i =3D 0; i < num_mbufs; i++) { > > + struct rte_mbuf *mbuf =3D pmbufs[i]; > > struct iovec iovecs[mbuf->nb_segs + 2]; > > struct tun_pi pi =3D { .flags =3D 0, .proto =3D 0x00 }; > > struct rte_mbuf *seg =3D mbuf; > > @@ -503,8 +500,7 @@ pmd_tx_burst(void *queue, struct rte_mbuf > **bufs, uint16_t nb_pkts) > > int proto; > > int n; > > int j; > > - int k; /* first index in iovecs for copying segments */ > > - uint16_t l234_hlen; /* length of layers 2,3,4 headers */ > > + int k; /* current index in iovecs for copying segments */ > > uint16_t seg_len; /* length of first segment */ > > uint16_t nb_segs; > > uint16_t *l4_cksum; /* l4 checksum (pseudo header + > payload) */ @@ > > -512,10 +508,6 @@ pmd_tx_burst(void *queue, struct rte_mbuf **bufs, > uint16_t nb_pkts) > > uint16_t l4_phdr_cksum =3D 0; /* TCP/UDP pseudo header > checksum */ > > uint16_t is_cksum =3D 0; /* in case cksum should be offloaded > */ > > > > - /* stats.errs will be incremented */ > > - if (rte_pktmbuf_pkt_len(mbuf) > max_size) > > - break; > > - > > l4_cksum =3D NULL; > > if (txq->type =3D=3D ETH_TUNTAP_TYPE_TUN) { > > /* > > @@ -554,9 +546,8 @@ pmd_tx_burst(void *queue, struct rte_mbuf > **bufs, uint16_t nb_pkts) > > l234_hlen =3D mbuf->l2_len + mbuf->l3_len + mbuf- > >l4_len; > > if (seg_len < l234_hlen) > > break; > > - > > - /* To change checksums, work on a > > - * copy of l2, l3 l4 headers. >=20 > Adding back in the blank line above would be nice for readability. > > + /* To change checksums, work on a * copy of l2, l3 > > + * headers + l4 pseudo header > > */ > > rte_memcpy(m_copy, rte_pktmbuf_mtod(mbuf, void > *), > > l234_hlen); > > @@ -598,13 +589,80 @@ pmd_tx_burst(void *queue, struct rte_mbuf > **bufs, uint16_t nb_pkts) > > n =3D writev(txq->fd, iovecs, j); > > if (n <=3D 0) > > break; > > + (*num_packets)++; > > + (*num_tx_bytes) +=3D rte_pktmbuf_pkt_len(mbuf); > > + } > > +} > > + > > +/* Callback to handle sending packets from the tap interface */ > > +static uint16_t pmd_tx_burst(void *queue, struct rte_mbuf **bufs, > > +uint16_t nb_pkts) { > > + struct tx_queue *txq =3D queue; > > + uint16_t num_tx =3D 0; > > + uint16_t num_packets =3D 0; > > + unsigned long num_tx_bytes =3D 0; > > + uint16_t tso_segsz =3D 0; > > + uint16_t hdrs_len; > > + uint32_t max_size; > > + int i; > > + uint64_t tso; > > + int ret; > > > > + if (unlikely(nb_pkts =3D=3D 0)) > > + return 0; > > + > > + struct rte_mbuf *gso_mbufs[MAX_GSO_MBUFS]; > > + max_size =3D *txq->mtu + (ETHER_HDR_LEN + ETHER_CRC_LEN + 4); > > + for (i =3D 0; i < nb_pkts; i++) { > > + struct rte_mbuf *mbuf_in =3D bufs[num_tx]; > > + struct rte_mbuf **mbuf; > > + uint16_t num_mbufs; > > + > > + tso =3D mbuf_in->ol_flags & PKT_TX_TCP_SEG; > > + if (tso) { > > + struct rte_gso_ctx *gso_ctx =3D &txq->gso_ctx; >=20 > Missing blank line here, this one is not optional. > > + assert(gso_ctx !=3D NULL); >=20 > Blank line would be nice. > > + /* TCP segmentation implies TCP checksum offload > */ > > + mbuf_in->ol_flags |=3D PKT_TX_TCP_CKSUM; >=20 > Blank line would be nice. > > + /* gso size is calculated without ETHER_CRC_LEN */ > > + hdrs_len =3D mbuf_in->l2_len + mbuf_in->l3_len + > > + mbuf_in->l4_len; > > + tso_segsz =3D mbuf_in->tso_segsz + hdrs_len; > > + if (unlikely(tso_segsz =3D=3D hdrs_len) || > > + tso_segsz > *txq->mtu) { > > + txq->stats.errs++; > > + break; > > + } > > + gso_ctx->gso_size =3D tso_segsz; > > + ret =3D rte_gso_segment(mbuf_in, /* packet to > segment */ > > + gso_ctx, /* gso control block */ > > + (struct rte_mbuf **)&gso_mbufs, /* out > mbufs */ > > + RTE_DIM(gso_mbufs)); /* max tso mbufs */ > > + > > + /* ret contains the number of new created mbufs */ > > + if (ret < 0) > > + break; > > + > > + mbuf =3D gso_mbufs; > > + num_mbufs =3D ret; > > + } else { > > + /* stats.errs will be incremented */ > > + if (rte_pktmbuf_pkt_len(mbuf_in) > max_size) > > + break; > > + > > + mbuf =3D &mbuf_in; > > + num_mbufs =3D 1; > > + } > > + > > + tap_write_mbufs(txq, num_mbufs, mbuf, hdrs_len, > > + &num_packets, &num_tx_bytes); > > num_tx++; > > - num_tx_bytes +=3D mbuf->pkt_len; > > - rte_pktmbuf_free(mbuf); > > + rte_pktmbuf_free(mbuf_in); > > } > > > > - txq->stats.opackets +=3D num_tx; > > + txq->stats.opackets +=3D num_packets; > > txq->stats.errs +=3D nb_pkts - num_tx; > > txq->stats.obytes +=3D num_tx_bytes; > > > > @@ -1066,31 +1124,73 @@ tap_mac_set(struct rte_eth_dev *dev, struct > > ether_addr *mac_addr) } > > > > static int > > +tap_gso_ctx_setup(struct rte_gso_ctx *gso_ctx, struct rte_eth_dev > > +*dev) { > > + uint32_t gso_types; > > + char pool_name[64]; > > + > > + /* > > + * Create private mbuf pool with TAP_GSO_MBUF_SEG_SIZE bytes > > + * size per mbuf use this pool for both direct and indirect mbufs > > + */ > > + > > + struct rte_mempool *mp; /* Mempool for GSO packets */ > > + /* initialize GSO context */ > > + gso_types =3D DEV_TX_OFFLOAD_TCP_TSO; > > + snprintf(pool_name, sizeof(pool_name), "mp_%s", dev->device- > >name); > > + mp =3D rte_mempool_lookup((const char *)pool_name); > > + if (!mp) { > > + mp =3D rte_pktmbuf_pool_create(pool_name, > TAP_GSO_MBUFS_NUM, > > + 0, 0, RTE_PKTMBUF_HEADROOM + > TAP_GSO_MBUF_SEG_SIZE, > > + SOCKET_ID_ANY); >=20 > You have setup the mempool with no cache size, which means you have to > take a lock for each allocate. This could changed to have a small cache p= er > lcore say 8, but the total number of mbufs needs to be large enough to no= t > allow starvation for a lcore. total_mbufs =3D (max_num_ports * cache_siz= e) + > some_extra mbufs; >=20 I will set cache_size as 4.=20 The total_mbufs should be mbufs_per_core(128) * cache_size(4) where the max= _num_ports=20 is already taken into consideration in mbufs_per_core. For example, for a T= CP packet of 1024 bytes and TSO max seg size of 256 bytes GSO will allocate= 5 mbufs (one direct and four indirect) regardless of the number of ports. > > + if (!mp) { > > + struct pmd_internals *pmd =3D dev->data- > >dev_private; > > + RTE_LOG(DEBUG, PMD, "%s: failed to create mbuf > pool for device %s\n", > > + pmd->name, dev->device->name); > > + return -1; > > + } > > + } > > + > > + gso_ctx->direct_pool =3D mp; > > + gso_ctx->indirect_pool =3D mp; > > + gso_ctx->gso_types =3D gso_types; > > + gso_ctx->gso_size =3D 0; /* gso_size is set in tx_burst() per packet = */ > > + gso_ctx->flag =3D 0; > > + > > + return 0; > > +} > > + > > +static int > > tap_setup_queue(struct rte_eth_dev *dev, > > struct pmd_internals *internals, > > uint16_t qid, > > int is_rx) > > { > > + int ret; > > int *fd; > > int *other_fd; > > const char *dir; > > struct pmd_internals *pmd =3D dev->data->dev_private; > > struct rx_queue *rx =3D &internals->rxq[qid]; > > struct tx_queue *tx =3D &internals->txq[qid]; > > + struct rte_gso_ctx *gso_ctx; > > > > if (is_rx) { > > fd =3D &rx->fd; > > other_fd =3D &tx->fd; > > dir =3D "rx"; > > + gso_ctx =3D NULL; > > } else { > > fd =3D &tx->fd; > > other_fd =3D &rx->fd; > > dir =3D "tx"; > > + gso_ctx =3D &tx->gso_ctx; > > } > > if (*fd !=3D -1) { > > /* fd for this queue already exists */ > > TAP_LOG(DEBUG, "%s: fd %d for %s queue qid %d exists", > > pmd->name, *fd, dir, qid); > > + gso_ctx =3D NULL; > > } else if (*other_fd !=3D -1) { > > /* Only other_fd exists. dup it */ > > *fd =3D dup(*other_fd); > > @@ -1115,6 +1215,11 @@ tap_setup_queue(struct rte_eth_dev *dev, > > > > tx->mtu =3D &dev->data->mtu; > > rx->rxmode =3D &dev->data->dev_conf.rxmode; > > + if (gso_ctx) { > > + ret =3D tap_gso_ctx_setup(gso_ctx, dev); > > + if (ret) > > + return -1; > > + } > > > > tx->type =3D pmd->type; > > > > diff --git a/drivers/net/tap/rte_eth_tap.h > > b/drivers/net/tap/rte_eth_tap.h index 7b21d0d..44e2773 100644 > > --- a/drivers/net/tap/rte_eth_tap.h > > +++ b/drivers/net/tap/rte_eth_tap.h > > @@ -15,6 +15,7 @@ > > > > #include > > #include > > +#include > > #include "tap_log.h" > > > > #ifdef IFF_MULTI_QUEUE > > @@ -22,6 +23,7 @@ > > #else > > #define RTE_PMD_TAP_MAX_QUEUES 1 > > #endif > > +#define MAX_GSO_MBUFS 64 > > > > enum rte_tuntap_type { > > ETH_TUNTAP_TYPE_UNKNOWN, > > @@ -59,6 +61,7 @@ struct tx_queue { > > uint16_t *mtu; /* Pointer to MTU from dev_data */ > > uint16_t csum:1; /* Enable checksum offloading */ > > struct pkt_stats stats; /* Stats for this TX queue */ > > + struct rte_gso_ctx gso_ctx; /* GSO context */ > > }; > > > > struct pmd_internals { > > diff --git a/mk/rte.app.mk b/mk/rte.app.mk index 1e32c83..e2ee879 > > 100644 > > --- a/mk/rte.app.mk > > +++ b/mk/rte.app.mk > > @@ -38,8 +38,6 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_PORT) +=3D - > lrte_port > > _LDLIBS-$(CONFIG_RTE_LIBRTE_PDUMP) +=3D -lrte_pdump > > _LDLIBS-$(CONFIG_RTE_LIBRTE_DISTRIBUTOR) +=3D -lrte_distributor > > _LDLIBS-$(CONFIG_RTE_LIBRTE_IP_FRAG) +=3D -lrte_ip_frag > > -_LDLIBS-$(CONFIG_RTE_LIBRTE_GRO) +=3D -lrte_gro > > -_LDLIBS-$(CONFIG_RTE_LIBRTE_GSO) +=3D -lrte_gso > > _LDLIBS-$(CONFIG_RTE_LIBRTE_METER) +=3D -lrte_meter > > _LDLIBS-$(CONFIG_RTE_LIBRTE_LPM) +=3D -lrte_lpm > > # librte_acl needs --whole-archive because of weak functions @@ -61,6 > > +59,8 @@ endif _LDLIBS-y +=3D --whole-archive > > > > _LDLIBS-$(CONFIG_RTE_LIBRTE_CFGFILE) +=3D -lrte_cfgfile > > +_LDLIBS-$(CONFIG_RTE_LIBRTE_GRO) +=3D -lrte_gro > > +_LDLIBS-$(CONFIG_RTE_LIBRTE_GSO) +=3D -lrte_gso > > _LDLIBS-$(CONFIG_RTE_LIBRTE_HASH) +=3D -lrte_hash > > _LDLIBS-$(CONFIG_RTE_LIBRTE_MEMBER) +=3D -lrte_member > > _LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST) +=3D -lrte_vhost > > -- > > 2.7.4 > > >=20 > Regards, > Keith