From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AE90DA034C for ; Mon, 28 Mar 2022 02:03:59 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4C48A4068F; Mon, 28 Mar 2022 02:03:59 +0200 (CEST) Received: from mail-ua1-f46.google.com (mail-ua1-f46.google.com [209.85.222.46]) by mails.dpdk.org (Postfix) with ESMTP id 6E0694068B for ; Mon, 28 Mar 2022 02:03:58 +0200 (CEST) Received: by mail-ua1-f46.google.com with SMTP id a28so5621628uaf.7 for ; Sun, 27 Mar 2022 17:03:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:from:date:message-id:subject:to; bh=DG2xAPVpCkgacm/OQvWeALdsrpo4uo67/Hu5yQDTUBw=; b=GbTbyruOD0iTQt1LfKExisrGK6OeweiuopBScfIinOxT980JxC4qZHbjttqYVqH7Gs TBl3oyE1w6XPk942vmdtJpqSJk3RXbyA4Py5tprcia47YWiLv5cnxQzQfQlNzbvtOGbe 6j0yExdIRdsmfzN3VTYJdoGadpYCl10MXmSTjn2NUjLUxQuuwAZJSfdpnIacRLSlPqEM kCs7XhdDa2BdG8XqtU9aDAZo7xcLpNAOnNL/syMJyQ+uk++OKovJ5rWSzhLjPOo5DAub Zdsu42zPdc+ubDKyJgRGG+GUveLmnw7gGOQFFATBYBgRP6aZtX2noQQ2bf+EObuXvYPV eeFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=DG2xAPVpCkgacm/OQvWeALdsrpo4uo67/Hu5yQDTUBw=; b=BTyd+xB2sOJelHKyC8HXbu1H8t3ZzmptHQwN2dVMy7+tpjL+xjcAr3RIiW++xxPYsv NOEUmkArjmQaMXYjz6L6W8+dpTtrHFNteGuYN0/yEAOXhRYxmfdwijVeJl60Xsr4rmHw eU3ebvbkG9atuFT1qE27iOQ2dmNzdabjwGuFr3/1W3pZpYDNAlPdRZ6j/VQRM2HVb6J7 ODQI2Dsa+b0EV1pas75s3Yei3qh7PP22XBVYQdkVVna+7eafmhdDBQkoWpSOAcZ8X1pH iTvIvTMboVBisaB+UQXImO8mXvWhKgmKTpi6oO8I0TfHoycA6ZYZvpGG2eLBqZ5QhX84 8Rww== X-Gm-Message-State: AOAM5304nD9aoTxFduHzEqOEFYkK6H2YCovtbzOdv8DfRb2jAJN61N2o Gu1e6Nesoni3M6AWhwDRfyh0lF3vNoa4f08x14Er68NLaPY= X-Google-Smtp-Source: ABdhPJwJiCRRUckc8ULUhyivEm8yxJxaZGhw/rsHDJZOmo5kRLFNr/KZGZcoGFdEs0OLGDsfKMZhEVdYYmWKzgKqG70= X-Received: by 2002:ab0:15ae:0:b0:359:6107:43b8 with SMTP id i43-20020ab015ae000000b00359610743b8mr8592302uae.23.1648425837504; Sun, 27 Mar 2022 17:03:57 -0700 (PDT) MIME-Version: 1.0 From: fwefew 4t4tg <7532yahoo@gmail.com> Date: Sun, 27 Mar 2022 20:03:46 -0400 Message-ID: Subject: Preliminary AWS ENA Benchmark Results on DPDK To: users@dpdk.org Content-Type: multipart/alternative; boundary="0000000000007bacee05db3c0cb3" X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --0000000000007bacee05db3c0cb3 Content-Type: text/plain; charset="UTF-8" https://github.com/rodgarrison/reinvent#benchmarks I show two c5n.metal boxes with AWS ENA NICs running under DPDK in the same data center running 1 TXQ and 1 RXQ transceiving 72-byte UDP packets (32-byte payload): - transmit around 874,410 packets/sec/queue If the NIC's TXQ ring is not full the application code can hand off packets to the NICs output buffer as fast as 415 ns/pkt or some 2.4 million packets/sec. But once the TXQ ring or HW buffers get full, throughput drops about 3x. I will update the link shortly with the RX states. In short, performance seems decent except when rings are full. But since AWS ENA NICs are virtual --- they are not a bonafide NIC card plugged into the computer's PCI bus, I'm not sure if these numbers are good, average, or suck. Indeed as per: https://www.amazon.com/Data-Plane-Development-Kit-DPDK/dp/0367373955/ref=sr_1_1?keywords=data+plane+development+kit&qid=1648425589&s=books&sprefix=data+plane+devel%2Cstripbooks%2C85&sr=1-1 chapter 12 (virtio) page 236 virtio is at least partially interrupt driven. I had no idea of the details but interrupt driven surely has to under perform a vanilla case where the CPU/NIC/PCI work in unison without all the virtualization technology. I'm not even sure, by extension, whether the ENA driver is really 100% poll-driven. Any context/feedback here would be appreciated. --0000000000007bacee05db3c0cb3 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
https://github.com/rodgarrison/reinvent#benchmarks

I show t= wo c5n.metal boxes with AWS ENA NICs running under DPDK in the same data ce= nter running 1 TXQ and 1 RXQ transceiving 72-byte UDP packets (32-byte payl= oad):

- transmit around=C2=A0874,410 packets/sec/queue

If th= e=C2=A0NIC's TXQ ring is not full the application code can hand off pac= kets to the NICs output buffer as fast as 415 ns/pkt or=C2=A0 some 2.4 mill= ion packets/sec. But once the TXQ ring or HW buffers get full, throughput d= rops about 3x. I will update the link shortly with the RX states. In short,= performance seems decent except when rings are full.

But since AWS = ENA NICs are virtual --- they are not a bonafide NIC card plugged into the = computer's PCI bus, I'm not sure if these numbers are good, average= , or suck. Indeed as per:

https://www.amazon.com/Data-Plane-Deve= lopment-Kit-DPDK/dp/0367373955/ref=3Dsr_1_1?keywords=3Ddata+plane+developme= nt+kit&qid=3D1648425589&s=3Dbooks&sprefix=3Ddata+plane+devel%2C= stripbooks%2C85&sr=3D1-1

chapter 12 (virtio) page 236 virtio= is at least partially interrupt driven. I had no idea of the details but i= nterrupt driven surely has to under perform a vanilla case where the CPU/NI= C/PCI work in unison without all the virtualization technology. I'm not= even sure, by extension, whether the ENA driver is really 100% poll-driven= .

Any context/feedback here would be appreciated.<= /div>
--0000000000007bacee05db3c0cb3--