From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM04-BN3-obe.outbound.protection.outlook.com (mail-eopbgr680088.outbound.protection.outlook.com [40.107.68.88]) by dpdk.org (Postfix) with ESMTP id 7ED08160 for ; Wed, 18 Jul 2018 17:06:03 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lhMCLGtJBrVcQNx3l6yaY8KCxSfwdme72fkeqi5+4jU=; b=Cw+Xt1TtAI1kCwXJi73lucymh4H3bzshDjgQRUm5HYdLrywYAUaWGgSbkMwBgqtZn93s0/eKjHMU7j3cMXIxtladKm9SPIhBTmCmuocIZdHoyMrgKvSuc4SpqXk2KZ97/6OeKFfecedzEsxw5BR5VdTG4HGA07/T5aE1580nh2Y= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Pavan.Bhagavatula@cavium.com; Received: from ltp-pvn.caveonetworks.com (111.93.218.67) by MWHPR07MB3471.namprd07.prod.outlook.com (2603:10b6:301:63::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.973.16; Wed, 18 Jul 2018 15:05:58 +0000 From: Pavan Nikhilesh To: jerin.jacob@caviumnetworks.com, santosh.shukla@caviumnetworks.com, rkudurumalla@caviumnetworks.com, ferruh.yigit@intel.com Cc: dev@dpdk.org, Pavan Nikhilesh Date: Wed, 18 Jul 2018 20:35:01 +0530 Message-Id: <20180718150502.19162-1-pbhagavatula@caviumnetworks.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180701164637.978-1-pbhagavatula@caviumnetworks.com> References: <20180701164637.978-1-pbhagavatula@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [111.93.218.67] X-ClientProxiedBy: BM1PR0101CA0056.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:19::18) To MWHPR07MB3471.namprd07.prod.outlook.com (2603:10b6:301:63::22) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 04d167f3-8739-4424-211d-08d5ecbff942 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989117)(5600053)(711020)(2017052603328)(7153060)(7193020); SRVR:MWHPR07MB3471; X-Microsoft-Exchange-Diagnostics: 1; MWHPR07MB3471; 3:eFrvVWjXZXNaVu1jQpBc+YAVy9TskiUZ55mgto77M0+pCevUcEt0IewkzPq6P3AAw5wM07jPnQQ/4o3OmY5CMauRvOxwbeGEjZJYmyzvcs/Bi5Y97i7JAVenhZDiiYURxfryAPDoHOo8ZMaANoP3nfFQ0Y3ewpdZ2lodoayJNE02bsQLBIcL+rhwQw9QhiwzX7FKbRA8xeOhlGbEleO52IHyJAKwlcKpg7GrnZCE3Um1cpVk6FeILmxx3Dt1UPjM; 25:0p9HRLJNVvQ5RKYCoLvDD8NZdIDpUw7snGkTL3PMsPk0vr+3pKlafyCBdA5kLv0DgRpl29upJaMQ4SjOBjp+P3cLZbCf6g0QR1NX7Zpd5cQ7iw2zihWG111QBb4zlpvBBBCo93rVyXs26gQfv4wXKxp+3GFcTTL9LAaCz7KFIbO/mvdIEVp02KklCrmmdtLanZ9IOcmzbgr3kTFmrYXMa/G7+fr/8tqG8ziU9dZFZNiIQ1elMxQxaL/t1HIvFHcDpph6C/JGo68Fc7gfxW4AkFdwkHxYrKbKbFZAu9HlgYZfmhcjzzTutfA/l54N72hrhEF8PSqGuV1/z2oc8zvHdQ==; 31:poWJaapJKEj+9ZAbxjk8iMDkXbKoV0TjN6LR2CZT32vdTQ8QOuT7pYFgpbt/0dMN6ysrMIalIIlVWHgiuDycEI5Y+Imq4kPYkbqFAz6n+8C/LwH+Hr8/0sK17FjcNZAht4j3qKwJCBDv9fNZWBbAWwIM4ZHeM9P+o7Pb3S8YHsn02eo+Wx438Vs+CbU0j+0xjoit887Uqr2tPZFFusGYH31U30HUCmYnjw22WIT4Qn4= X-MS-TrafficTypeDiagnostic: MWHPR07MB3471: X-Microsoft-Exchange-Diagnostics: 1; MWHPR07MB3471; 20:BETInPIgy44Gy10J3pU3ZrdcxccDT0TsejWRJNlEFZe1PNvdj+sPJ4apFWmaWykDHog767mbxHtp0nKWl4utMz6zgo8urLl+FfwiyKtyP04kXunhzaQDYcenY08m1aijqqXRAh9STlq3daq1UC2h/0hXVYgryzK3oHPjvFyfQwyucnlItIN2zEpOLjHd6pVnuRM30TN85CumEBqsD8GpkuXFptau09tP5F/YjfvFzvRnSO20s6ahIEdSYlWeh7GzS2443rOKXDZ8ye0npqqiG3cj1DJPf8UNPr1ODJ1hCkWE79ePgz2G7zc1Tg3/7mqwfmuNgZGnASNzZdHoueKsSsHN6ofPWQY13bQgOqbWsLCpCTXVXKYIDcCYijR2Kz10t6Olwye0LPiUrod7JnMZXmMybl9oBj/M5pW/FRZ/3FQsFGaaUoQJJ8O4GV5ARbNI/T8mIdd2sodfWOtlfp0h5H8h5ZVKdjQNRy7ZjLb6JQkCjYx5WUT0UjOKxOgYnGGPs6aHwNjX08YXuosPy3TamyFLR6uDiJvsjzSB9xIwCHKKafWLM8GkGqWePl5S4TNe+VEh6UJalq65zuYQlHD20jYh+bwQmGUKPCkICNcvrxU=; 4:Gz/XHi+5nYB2as4lXCtI6bq/F7ts+Aj4jyhqGBA43G20m0aIsJGLhCMtEqlByJCyBOss/ZIXR+GVQUjUP4j+FVV8MzXuhL+Wq77YSvIWEHbPQBQ5qniVkr4bFwGAJsfMFZm0lGmP4zgyi6uiPooqClSd51mJnx1NMgOCsJji1BEhuBSOOlc4w/awKcA2sc6NRaEwQzu51rgRdnxlV58tw/C5IPk2J2/9soO0n19Cz1wcxUp0K+2pJ7q0Gfhiqq7o/+wO1B5Phl+ue8jPUmCtQg== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(93006095)(10201501046)(3231311)(944501410)(52105095)(3002001)(149027)(150027)(6041310)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(20161123560045)(20161123558120)(6072148)(201708071742011)(7699016); SRVR:MWHPR07MB3471; BCL:0; PCL:0; RULEID:; SRVR:MWHPR07MB3471; X-Forefront-PRVS: 0737B96801 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(979002)(39860400002)(136003)(376002)(346002)(396003)(366004)(199004)(189003)(69596002)(5660300001)(36756003)(105586002)(316002)(7736002)(48376002)(6486002)(25786009)(305945005)(53936002)(6666003)(11346002)(6512007)(4326008)(3846002)(1076002)(107886003)(6116002)(50466002)(53416004)(14444005)(16586007)(8676002)(72206003)(478600001)(68736007)(81166006)(81156014)(47776003)(106356001)(8936002)(76176011)(42882007)(6506007)(66066001)(52116002)(956004)(2616005)(476003)(386003)(446003)(51416003)(16526019)(5009440100003)(26005)(1857600001)(50226002)(2906002)(486006)(97736004)(42262002)(969003)(989001)(999001)(1009001)(1019001); DIR:OUT; SFP:1101; SCL:1; SRVR:MWHPR07MB3471; H:ltp-pvn.caveonetworks.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; MWHPR07MB3471; 23:T8dtp+yNod0o+FTrt6D3RXMqVa3H/0Hlpyvu3aAjp?= =?us-ascii?Q?C1bND4y3aHMHYhC+6UYv1EFTj9U9jZcFzzf5JrnPPJA8gsC5/NJw+16r4wf3?= =?us-ascii?Q?el/Gba/pw4jZXDt68cRqCZl3K55cw41yjjfFixolpITCrQ3vC8mEzSFRd3HS?= =?us-ascii?Q?R2WJIuPdJTHaeF9PeJMGGiud1wYzP61XDw7Z1Ajr6CLRJS5RrWIQdQhyhxP7?= =?us-ascii?Q?t58yapaMmp++UtYN2XTGlUWaDEdNJxCVV/K9geGtclOQuiBotDIXAMyyEa8X?= =?us-ascii?Q?jSYqQRvKEI3tj4hFKXllM5/SklWf2qX/4yKyDIwTdP7pc0GzENRFLwyQPwV3?= =?us-ascii?Q?KHJ1Um6FEV7YqSS+nN4pc/urq7S64q4B41Vn/xitm3149jV4jgTLMZH/iAqw?= =?us-ascii?Q?s+CxharnpUZePmat2QejWeEFwCCQBe89rlnaoSec5jgK6Q8YnhDKeR+T2/fQ?= =?us-ascii?Q?NtRnY/nTRt5EnKChcjiykg7T990WCGNFD7Bqxd8B7BGvlSnC0/Zz78xme+Lb?= =?us-ascii?Q?gKyXDn00cTN/g/XW3zjryDRHvp5jycjqieKvtlBXu6mtliB4Evd5PWiAL+HO?= =?us-ascii?Q?BOPFV+hhKkTkifgZu6UXgE6845LemhL78PLvRgRcw7lJjA46c936a1b+mTaf?= =?us-ascii?Q?vFS/13iqsKEh/rvrqC7nxVVbuBXFHTD+1VROhQjRnAJPBSRtE/H8H9NslzKl?= =?us-ascii?Q?YAJiIW3tcZ/DvRKI3F8iDvMOIkORph3Ag/Aw/JmlBRgrHlhXQXbVIkUhbshR?= =?us-ascii?Q?7iPpJxuZJNIQeinK9MxBNtS6+lnlVRWXPZkSu7t1Ak86QlEoY2e6s+3yAK+y?= =?us-ascii?Q?CGxGGsI7EnNknC48qx0l+QdhMBUdMsAIAFuTtCi42CF+wsAQWtbV4Y6rL73p?= =?us-ascii?Q?HZgfdAs4IM4bME/dMogzit2KG76kP11zPwTh1wBPVd/6CqT+OttLypwerhIn?= =?us-ascii?Q?uqUxcw4lB5c8JFZfa5pbUs+Py6pIXKNBvxcQY4PUavBBJtRMhIODHgsE2bqn?= =?us-ascii?Q?12wT7S/qrCzOyIPEiWZFnxUzDfgzflk6fzgpWhoqzv6hWlZA5wezImSYEkff?= =?us-ascii?Q?cKKVCXOoQJGu0UKEaKlTuHt8ZTN8JQwX5PqCpTnhCcTsxf6S2ceF5vt72HYK?= =?us-ascii?Q?S7gW61J3Ih/AKjjzPfA9iTkCMicnv4Yl5VxwXbcY6KjVRe4rmx8moFlK9PI7?= =?us-ascii?Q?fDMHfy1mhbpR8iYqVi32454uAjEC1HHBnYJAj/aQ4VYD3X8jDIiyMGZmnFXT?= =?us-ascii?Q?F0MCeXOaa4O/txB1U/au27OQUT3l2Xe8m3Y4eAmp9IveMsZJLZwwe4ImT5Fe?= =?us-ascii?Q?tVavuhySFSj3NzSavGTckRt2xlwOl4oNBQIx9rWa1Rfu0BqjmwzjOHxWjIdF?= =?us-ascii?Q?jUsvPRPflRuSzF//Xr0mTiUlBp4d7SdeIV/nXDy1SYMnF4AkOfQ1Zfv353ww?= =?us-ascii?Q?z/LNaRoZw=3D=3D?= X-Microsoft-Antispam-Message-Info: tD+RwAFFU+V7dPErg2x4K5+X4oScx44vEwn4xKgvHBS7E8FWIP3P0b2lQxzUNPY4MJyPMuH7tPpeecQREQYSv/8F+2mlZPHOFAGMo2Qr/jDnxgRFE0QUelZ5vLeX/AmAXJV4PYVzrd10LA0AI1bxUA6n385JNrg2unhtEoeSarP0SXYha/3TOhb9sgbIr6Aov/HnZWnHxJPqXoXKrJjm2bBan2SeUMXN6UYMPjCy/lsmW+qZzzNPt6tP81vzahy8i26zfr2eWBqW8kbUBp19D4w9zt9X0BCFE7s13uTSXwHGgWoV0+0QzdT1I0iU2PlD+sJwvx33JkF4/pU+Y/Km5ICpHbOIqGY3VCd1N1xVXhU= X-Microsoft-Exchange-Diagnostics: 1; MWHPR07MB3471; 6:UYDm7nk2+CKg3iMmQScbojfVWNMR0uU0+VsgyfJPgAqQ0a/DbVqjfO9t+vkbjXngbG5lEyCIo4FWxHnDljMxBYWUZhw4wJIp7UcYLm0ejevVXEhJ+5YzTPrWk97LRGJYONu0Ke0Od5OCmzgfNQ+bhkUkp8qYObNmhUBfE0YOVwVw6mXF/2gXMWbcTmcXxW+6rc2Y0yhnGAu9A+XwIpTV0tkSoTJUVPFqT5aBeVprCA7rNAmgMjmcjikmqp3MfYcuqumfj0ALtWHARJJUlVy5O8uiSM7t+GHfK7H1B8vMLEwWtSiYa+CTUBVIXwVPMd9OCvl6u03WEsHrBIpHe4FtZ6S5W84d+TSR0PftbYCTPm4trpVjnxI7sFu2y9QZs09drBOsXiQBnQ1ECp1CiRN1XrPD22C/fGWqKrQGoU4mJtClcyD/xWCBQ6zllH1iNwz4KEcDr8unedWyGTQZldyPUA==; 5:W9tk0aC2paitHZXPibB+F1lcsyss1Ynzi1Gx0gjU75//InVyfTWTj7mv8rCdYfYewZt9M350oMAxcXIqWtUVShulCFjJfuqqB+kUQi0gXYRaFXr+SxAkCorgiqRGJtHRfAvcsBWbYRf5LNomvm7goVBsD6o6kvqxhWPK1xulTrc=; 7:JyFECU9U7cNmGqWTGFDYvJNH2h+SYCZbrGq85PkEjuuU1gnhWo88vrzVlC/ArkJIm1IPewpqC+DRkBqMH/pweAMxGR067dEhLmujTDZShnPmZxTgYNeRAUjQziE+TFUU9hsXQh6ayYaBhu5Wo18w8KStC1C7ZPD0Chx1CNtM03TfY2GJMLgDg/QSCgRUnrzNXwTF7NMdwAZLg/83Phy2ua1suwOwSBsrGluy2FB26yJhE7Gv5gO5kzuKmzklmTop SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jul 2018 15:05:58.6311 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 04d167f3-8739-4424-211d-08d5ecbff942 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR07MB3471 Subject: [dpdk-dev] [PATCH v3 1/2] net/thunderx: enable Rx checksum offload X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jul 2018 15:06:04 -0000 Add L3/L4 Rx checksum offload and update capabilities. Signed-off-by: Pavan Nikhilesh --- v3 Changes: - rebase on top of next-net v2 Changes: - Add Rx checksum offload support for l3fwd. drivers/net/thunderx/nicvf_ethdev.c | 33 ++++++++----- drivers/net/thunderx/nicvf_ethdev.h | 1 + drivers/net/thunderx/nicvf_rxtx.c | 73 +++++++++++++++++++++++++---- drivers/net/thunderx/nicvf_rxtx.h | 15 ++++-- drivers/net/thunderx/nicvf_struct.h | 27 ++++++----- 5 files changed, 113 insertions(+), 36 deletions(-) diff --git a/drivers/net/thunderx/nicvf_ethdev.c b/drivers/net/thunderx/nicvf_ethdev.c index 5e15a88a5..eba05fdf0 100644 --- a/drivers/net/thunderx/nicvf_ethdev.c +++ b/drivers/net/thunderx/nicvf_ethdev.c @@ -355,11 +355,9 @@ nicvf_dev_supported_ptypes_get(struct rte_eth_dev *dev) } memcpy((char *)ptypes + copied, &ptypes_end, sizeof(ptypes_end)); - if (dev->rx_pkt_burst == nicvf_recv_pkts || - dev->rx_pkt_burst == nicvf_recv_pkts_multiseg) - return ptypes; - return NULL; + /* All Ptypes are supported in all Rx functions. */ + return ptypes; } static void @@ -916,13 +914,18 @@ nicvf_set_tx_function(struct rte_eth_dev *dev) static void nicvf_set_rx_function(struct rte_eth_dev *dev) { - if (dev->data->scattered_rx) { - PMD_DRV_LOG(DEBUG, "Using multi-segment rx callback"); - dev->rx_pkt_burst = nicvf_recv_pkts_multiseg; - } else { - PMD_DRV_LOG(DEBUG, "Using single-segment rx callback"); - dev->rx_pkt_burst = nicvf_recv_pkts; - } + struct nicvf *nic = nicvf_pmd_priv(dev); + + const eth_rx_burst_t rx_burst_func[2][2] = { + /* [NORMAL/SCATTER] [NO_CKSUM/CKSUM] */ + [0][0] = nicvf_recv_pkts_no_offload, + [0][1] = nicvf_recv_pkts_cksum, + [1][0] = nicvf_recv_pkts_multiseg_no_offload, + [1][1] = nicvf_recv_pkts_multiseg_cksum, + }; + + dev->rx_pkt_burst = + rx_burst_func[dev->data->scattered_rx][nic->offload_cksum]; } static int @@ -1243,6 +1246,9 @@ nicvf_rxq_mbuf_setup(struct nicvf_rxq *rxq) offsetof(struct rte_mbuf, data_off) != 4); RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, port) - offsetof(struct rte_mbuf, data_off) != 6); + RTE_BUILD_BUG_ON(offsetof(struct nicvf_rxq, rxq_fastpath_data_end) - + offsetof(struct nicvf_rxq, + rxq_fastpath_data_start) > 128); mb_def.nb_segs = 1; mb_def.data_off = RTE_PKTMBUF_HEADROOM + (nic->skip_bytes); mb_def.port = rxq->port_id; @@ -1743,7 +1749,7 @@ nicvf_dev_start(struct rte_eth_dev *dev) return ret; } - /* Configure callbacks based on scatter mode */ + /* Configure callbacks based on offloads */ nicvf_set_tx_function(dev); nicvf_set_rx_function(dev); @@ -1962,6 +1968,9 @@ nicvf_dev_configure(struct rte_eth_dev *dev) } } + if (rxmode->offloads & DEV_RX_OFFLOAD_CHECKSUM) + nic->offload_cksum = 1; + PMD_INIT_LOG(DEBUG, "Configured ethdev port%d hwcap=0x%" PRIx64, dev->data->port_id, nicvf_hw_cap(nic)); diff --git a/drivers/net/thunderx/nicvf_ethdev.h b/drivers/net/thunderx/nicvf_ethdev.h index 9af508803..ae440fef2 100644 --- a/drivers/net/thunderx/nicvf_ethdev.h +++ b/drivers/net/thunderx/nicvf_ethdev.h @@ -38,6 +38,7 @@ DEV_TX_OFFLOAD_MULTI_SEGS) #define NICVF_RX_OFFLOAD_CAPA ( \ + DEV_RX_OFFLOAD_CHECKSUM | \ DEV_RX_OFFLOAD_VLAN_STRIP | \ DEV_RX_OFFLOAD_CRC_STRIP | \ DEV_RX_OFFLOAD_JUMBO_FRAME | \ diff --git a/drivers/net/thunderx/nicvf_rxtx.c b/drivers/net/thunderx/nicvf_rxtx.c index 6e075e23c..4980dab79 100644 --- a/drivers/net/thunderx/nicvf_rxtx.c +++ b/drivers/net/thunderx/nicvf_rxtx.c @@ -331,6 +331,20 @@ nicvf_rx_classify_pkt(cqe_rx_word0_t cqe_rx_w0) return ptype_table[cqe_rx_w0.l3_type][cqe_rx_w0.l4_type]; } +static inline uint64_t __hot +nicvf_set_olflags(const cqe_rx_word0_t cqe_rx_w0) +{ + static const uint64_t flag_table[3] __rte_cache_aligned = { + PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_GOOD, + PKT_RX_IP_CKSUM_BAD | PKT_RX_L4_CKSUM_UNKNOWN, + PKT_RX_IP_CKSUM_GOOD | PKT_RX_L4_CKSUM_BAD, + }; + + const uint8_t idx = (cqe_rx_w0.err_opcode == CQE_RX_ERR_L4_CHK) << 1 | + (cqe_rx_w0.err_opcode == CQE_RX_ERR_IP_CHK); + return flag_table[idx]; +} + static inline int __hot nicvf_fill_rbdr(struct nicvf_rxq *rxq, int to_fill) { @@ -389,11 +403,13 @@ nicvf_rx_offload(cqe_rx_word0_t cqe_rx_w0, cqe_rx_word2_t cqe_rx_w2, if (likely(cqe_rx_w0.rss_alg)) { pkt->hash.rss = cqe_rx_w2.rss_tag; pkt->ol_flags |= PKT_RX_RSS_HASH; + } } -uint16_t __hot -nicvf_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) +static __rte_always_inline uint16_t +nicvf_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts, + const uint32_t flag) { uint32_t i, to_process; struct cqe_rx_t *cqe_rx; @@ -424,7 +440,11 @@ nicvf_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) rb0_ptr = *((uint64_t *)cqe_rx + rbptr_offset); pkt = (struct rte_mbuf *)nicvf_mbuff_phy2virt (rb0_ptr - cqe_rx_w1.align_pad, mbuf_phys_off); - pkt->ol_flags = 0; + + if (flag & NICVF_RX_OFFLOAD_NONE) + pkt->ol_flags = 0; + if (flag & NICVF_RX_OFFLOAD_CKSUM) + pkt->ol_flags = nicvf_set_olflags(cqe_rx_w0); pkt->data_len = cqe_rx_w3.rb0_sz; pkt->pkt_len = cqe_rx_w3.rb0_sz; pkt->packet_type = nicvf_rx_classify_pkt(cqe_rx_w0); @@ -449,11 +469,27 @@ nicvf_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) return to_process; } -static inline uint16_t __hot +uint16_t __hot +nicvf_recv_pkts_no_offload(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t nb_pkts) +{ + return nicvf_recv_pkts(rx_queue, rx_pkts, nb_pkts, + NICVF_RX_OFFLOAD_NONE); +} + +uint16_t __hot +nicvf_recv_pkts_cksum(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t nb_pkts) +{ + return nicvf_recv_pkts(rx_queue, rx_pkts, nb_pkts, + NICVF_RX_OFFLOAD_CKSUM); +} + +static __rte_always_inline uint16_t __hot nicvf_process_cq_mseg_entry(struct cqe_rx_t *cqe_rx, uint64_t mbuf_phys_off, struct rte_mbuf **rx_pkt, uint8_t rbptr_offset, - uint64_t mbuf_init) + uint64_t mbuf_init, const uint32_t flag) { struct rte_mbuf *pkt, *seg, *prev; cqe_rx_word0_t cqe_rx_w0; @@ -471,12 +507,15 @@ nicvf_process_cq_mseg_entry(struct cqe_rx_t *cqe_rx, pkt = (struct rte_mbuf *)nicvf_mbuff_phy2virt (rb_ptr[0] - cqe_rx_w1.align_pad, mbuf_phys_off); - pkt->ol_flags = 0; pkt->pkt_len = cqe_rx_w1.pkt_len; pkt->data_len = rb_sz[nicvf_frag_num(0)]; nicvf_mbuff_init_mseg_update( pkt, mbuf_init, cqe_rx_w1.align_pad, nb_segs); pkt->packet_type = nicvf_rx_classify_pkt(cqe_rx_w0); + if (flag & NICVF_RX_OFFLOAD_NONE) + pkt->ol_flags = 0; + if (flag & NICVF_RX_OFFLOAD_CKSUM) + pkt->ol_flags = nicvf_set_olflags(cqe_rx_w0); nicvf_rx_offload(cqe_rx_w0, cqe_rx_w2, pkt); *rx_pkt = pkt; @@ -495,9 +534,9 @@ nicvf_process_cq_mseg_entry(struct cqe_rx_t *cqe_rx, return nb_segs; } -uint16_t __hot +static __rte_always_inline uint16_t __hot nicvf_recv_pkts_multiseg(void *rx_queue, struct rte_mbuf **rx_pkts, - uint16_t nb_pkts) + uint16_t nb_pkts, const uint32_t flag) { union cq_entry_t *cq_entry; struct cqe_rx_t *cqe_rx; @@ -519,7 +558,7 @@ nicvf_recv_pkts_multiseg(void *rx_queue, struct rte_mbuf **rx_pkts, cq_entry = &desc[cqe_head]; cqe_rx = (struct cqe_rx_t *)cq_entry; nb_segs = nicvf_process_cq_mseg_entry(cqe_rx, mbuf_phys_off, - rx_pkts + i, rbptr_offset, mbuf_init); + rx_pkts + i, rbptr_offset, mbuf_init, flag); buffers_consumed += nb_segs; cqe_head = (cqe_head + 1) & cqe_mask; nicvf_prefetch_store_keep(rx_pkts[i]); @@ -539,6 +578,22 @@ nicvf_recv_pkts_multiseg(void *rx_queue, struct rte_mbuf **rx_pkts, return to_process; } +uint16_t __hot +nicvf_recv_pkts_multiseg_no_offload(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t nb_pkts) +{ + return nicvf_recv_pkts_multiseg(rx_queue, rx_pkts, nb_pkts, + NICVF_RX_OFFLOAD_NONE); +} + +uint16_t __hot +nicvf_recv_pkts_multiseg_cksum(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t nb_pkts) +{ + return nicvf_recv_pkts_multiseg(rx_queue, rx_pkts, nb_pkts, + NICVF_RX_OFFLOAD_CKSUM); +} + uint32_t nicvf_dev_rx_queue_count(struct rte_eth_dev *dev, uint16_t queue_idx) { diff --git a/drivers/net/thunderx/nicvf_rxtx.h b/drivers/net/thunderx/nicvf_rxtx.h index 8bdd582ed..72daffb60 100644 --- a/drivers/net/thunderx/nicvf_rxtx.h +++ b/drivers/net/thunderx/nicvf_rxtx.h @@ -8,6 +8,9 @@ #include #include +#define NICVF_RX_OFFLOAD_NONE 0x1 +#define NICVF_RX_OFFLOAD_CKSUM 0x2 + #define NICVF_TX_OFFLOAD_MASK (PKT_TX_IP_CKSUM | PKT_TX_L4_MASK) #ifndef __hot @@ -86,9 +89,15 @@ nicvf_mbuff_init_mseg_update(struct rte_mbuf *pkt, const uint64_t mbuf_init, uint32_t nicvf_dev_rx_queue_count(struct rte_eth_dev *dev, uint16_t queue_idx); uint32_t nicvf_dev_rbdr_refill(struct rte_eth_dev *dev, uint16_t queue_idx); -uint16_t nicvf_recv_pkts(void *rxq, struct rte_mbuf **rx_pkts, uint16_t pkts); -uint16_t nicvf_recv_pkts_multiseg(void *rx_queue, struct rte_mbuf **rx_pkts, - uint16_t nb_pkts); +uint16_t nicvf_recv_pkts_no_offload(void *rxq, struct rte_mbuf **rx_pkts, + uint16_t pkts); +uint16_t nicvf_recv_pkts_cksum(void *rxq, struct rte_mbuf **rx_pkts, + uint16_t pkts); + +uint16_t nicvf_recv_pkts_multiseg_no_offload(void *rx_queue, + struct rte_mbuf **rx_pkts, uint16_t nb_pkts); +uint16_t nicvf_recv_pkts_multiseg_cksum(void *rx_queue, + struct rte_mbuf **rx_pkts, uint16_t nb_pkts); uint16_t nicvf_xmit_pkts(void *txq, struct rte_mbuf **tx_pkts, uint16_t pkts); uint16_t nicvf_xmit_pkts_multiseg(void *txq, struct rte_mbuf **tx_pkts, diff --git a/drivers/net/thunderx/nicvf_struct.h b/drivers/net/thunderx/nicvf_struct.h index cf98f7c1a..a770e6b54 100644 --- a/drivers/net/thunderx/nicvf_struct.h +++ b/drivers/net/thunderx/nicvf_struct.h @@ -55,25 +55,27 @@ union mbuf_initializer { }; struct nicvf_rxq { + MARKER rxq_fastpath_data_start; + uint8_t rbptr_offset; + uint16_t rx_free_thresh; + uint32_t head; + uint32_t qlen_mask; + int32_t recv_buffers; + int32_t available_space; uint64_t mbuf_phys_off; uintptr_t cq_status; uintptr_t cq_door; - union mbuf_initializer mbuf_initializer; - nicvf_iova_addr_t phys; - union cq_entry_t *desc; struct nicvf_rbdr *shared_rbdr; - struct nicvf *nic; struct rte_mempool *pool; - uint32_t head; - uint32_t qlen_mask; - int32_t available_space; - int32_t recv_buffers; - uint16_t rx_free_thresh; - uint16_t queue_id; - uint16_t precharge_cnt; + union cq_entry_t *desc; + union mbuf_initializer mbuf_initializer; + MARKER rxq_fastpath_data_end; uint8_t rx_drop_en; + uint16_t precharge_cnt; uint16_t port_id; - uint8_t rbptr_offset; + uint16_t queue_id; + struct nicvf *nic; + nicvf_iova_addr_t phys; } __rte_cache_aligned; struct nicvf { @@ -85,6 +87,7 @@ struct nicvf { bool loopback_supported; bool pf_acked:1; bool pf_nacked:1; + bool offload_cksum:1; uint64_t hwcap; uint8_t link_up; uint8_t duplex; -- 2.18.0