From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id D7894A0096 for ; Mon, 3 Jun 2019 15:37:28 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B384D1B950; Mon, 3 Jun 2019 15:37:28 +0200 (CEST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id 8B5322C57 for ; Mon, 3 Jun 2019 15:37:26 +0200 (CEST) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 031043086239; Mon, 3 Jun 2019 13:37:26 +0000 (UTC) Received: from dhcp-25.97.bos.redhat.com (unknown [10.18.25.61]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 38BA410013D9; Mon, 3 Jun 2019 13:37:25 +0000 (UTC) From: Aaron Conole To: Jakub Grajciar Cc: References: <20190520101841.17708-1-jgrajcia@cisco.com> <20190531062247.5952-1-jgrajcia@cisco.com> Date: Mon, 03 Jun 2019 09:37:24 -0400 In-Reply-To: <20190531062247.5952-1-jgrajcia@cisco.com> (Jakub Grajciar's message of "Fri, 31 May 2019 08:22:47 +0200") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.42]); Mon, 03 Jun 2019 13:37:26 +0000 (UTC) Subject: Re: [dpdk-dev] [PATCH v10] net/memif: introduce memory interface (memif) PMD X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Jakub Grajciar writes: > Memory interface (memif), provides high performance > packet transfer over shared memory. > > Signed-off-by: Jakub Grajciar > --- > MAINTAINERS | 6 + > config/common_base | 5 + > config/common_linux | 1 + > doc/guides/nics/features/memif.ini | 14 + > doc/guides/nics/index.rst | 1 + > doc/guides/nics/memif.rst | 234 ++++ > doc/guides/rel_notes/release_19_08.rst | 5 + > drivers/net/Makefile | 1 + > drivers/net/memif/Makefile | 31 + > drivers/net/memif/memif.h | 179 +++ > drivers/net/memif/memif_socket.c | 1124 +++++++++++++++++ > drivers/net/memif/memif_socket.h | 105 ++ > drivers/net/memif/meson.build | 15 + > drivers/net/memif/rte_eth_memif.c | 1206 +++++++++++++++++++ > drivers/net/memif/rte_eth_memif.h | 212 ++++ > drivers/net/memif/rte_pmd_memif_version.map | 4 + > drivers/net/meson.build | 1 + > mk/rte.app.mk | 1 + > 18 files changed, 3145 insertions(+) > create mode 100644 doc/guides/nics/features/memif.ini > create mode 100644 doc/guides/nics/memif.rst > create mode 100644 drivers/net/memif/Makefile > create mode 100644 drivers/net/memif/memif.h > create mode 100644 drivers/net/memif/memif_socket.c > create mode 100644 drivers/net/memif/memif_socket.h > create mode 100644 drivers/net/memif/meson.build > create mode 100644 drivers/net/memif/rte_eth_memif.c > create mode 100644 drivers/net/memif/rte_eth_memif.h > create mode 100644 drivers/net/memif/rte_pmd_memif_version.map > > requires patch: http://patchwork.dpdk.org/patch/51511/ > Memif uses a unix domain socket as control channel (cc). > rte_interrupts.h is used to handle interrupts for this cc. > This patch is required, because the message received can > be a reason for disconnecting. > > v3: > - coding style fixes > - documentation > - doxygen comments > - use strlcpy() instead of strncpy() > - use RTE_BUILD_BUG_ON instead of _Static_assert() > - fix meson build deps > > v4: > - coding style fixes > - doc update (desc format, messaging, features, ...) > - pointer arithmetic fix > - __rte_packed and __rte_aligned instead of __attribute__() > - fixed multi-queue > - support additional archs (memfd_create syscall) > > v5: > - coding style fixes > - add release notes > - remove unused dependencies > - doc update > > v6: > - remove socket on termination of testpmd application > - disallow memif probing in secondary process (return error), multi-proce= ss support will be implemented in separate patch > - fix chained buffer packet length > - use single region for rings and buffers (non-zero-copy) > - doc update > > v7: > - coding style fixes > > v8: > - release notes: 19.05 -> 19.08 > > v9: > - rearrange elements in structures to remove holes > - default socket file in /run > - #define name sizes > - implement dev_close op > > v10: > - fix shared lib build > - clear memory allocated by PMD in .dev_close > - fix NULL pointer string 'error: =E2=80=98%s=E2=80=99 directive argument= is null' > > diff --git a/MAINTAINERS b/MAINTAINERS > index 15d0829c5..3698c146b 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -841,6 +841,12 @@ F: drivers/net/softnic/ > F: doc/guides/nics/features/softnic.ini > F: doc/guides/nics/softnic.rst > > +Memif PMD > +M: Jakub Grajciar > +F: drivers/net/memif/ > +F: doc/guides/nics/memif.rst > +F: doc/guides/nics/features/memif.ini > + > > Crypto Drivers > -------------- > diff --git a/config/common_base b/config/common_base > index 6f19ad5d2..e406e7836 100644 > --- a/config/common_base > +++ b/config/common_base > @@ -444,6 +444,11 @@ CONFIG_RTE_LIBRTE_PMD_AF_PACKET=3Dn > # > CONFIG_RTE_LIBRTE_PMD_AF_XDP=3Dn > > +# > +# Compile Memory Interface PMD driver (Linux only) > +# > +CONFIG_RTE_LIBRTE_PMD_MEMIF=3Dn > + > # > # Compile link bonding PMD library > # > diff --git a/config/common_linux b/config/common_linux > index 75334273d..87514fe4f 100644 > --- a/config/common_linux > +++ b/config/common_linux > @@ -19,6 +19,7 @@ CONFIG_RTE_LIBRTE_VHOST_POSTCOPY=3Dn > CONFIG_RTE_LIBRTE_PMD_VHOST=3Dy > CONFIG_RTE_LIBRTE_IFC_PMD=3Dy > CONFIG_RTE_LIBRTE_PMD_AF_PACKET=3Dy > +CONFIG_RTE_LIBRTE_PMD_MEMIF=3Dy > CONFIG_RTE_LIBRTE_PMD_SOFTNIC=3Dy > CONFIG_RTE_LIBRTE_PMD_TAP=3Dy > CONFIG_RTE_LIBRTE_AVP_PMD=3Dy > diff --git a/doc/guides/nics/features/memif.ini b/doc/guides/nics/feature= s/memif.ini > new file mode 100644 > index 000000000..807d9ecdc > --- /dev/null > +++ b/doc/guides/nics/features/memif.ini > @@ -0,0 +1,14 @@ > +; > +; Supported features of the 'memif' network poll mode driver. > +; > +; Refer to default.ini for the full list of available PMD features. > +; > +[Features] > +Link status =3D Y > +Basic stats =3D Y > +Jumbo frame =3D Y > +ARMv8 =3D Y > +Power8 =3D Y > +x86-32 =3D Y > +x86-64 =3D Y > +Usage doc =3D Y > diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst > index 2221c35f2..691e7209f 100644 > --- a/doc/guides/nics/index.rst > +++ b/doc/guides/nics/index.rst > @@ -36,6 +36,7 @@ Network Interface Controller Drivers > intel_vf > kni > liquidio > + memif > mlx4 > mlx5 > mvneta > diff --git a/doc/guides/nics/memif.rst b/doc/guides/nics/memif.rst > new file mode 100644 > index 000000000..de2d481eb > --- /dev/null > +++ b/doc/guides/nics/memif.rst > @@ -0,0 +1,234 @@ > +.. SPDX-License-Identifier: BSD-3-Clause > + Copyright(c) 2018-2019 Cisco Systems, Inc. > + > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > +Memif Poll Mode Driver > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > + > +Shared memory packet interface (memif) PMD allows for DPDK and any other= client > +using memif (DPDK, VPP, libmemif) to communicate using shared memory. Me= mif is > +Linux only. > + > +The created device transmits packets in a raw format. It can be used with > +Ethernet mode, IP mode, or Punt/Inject. At this moment, only Ethernet mo= de is > +supported in DPDK memif implementation. > + > +Memif works in two roles: master and slave. Slave connects to master ove= r an > +existing socket. It is also a producer of shared memory file and initial= izes > +the shared memory. Each interface can be connected to one peer interface > +at same time. The peer interface is identified by id parameter. Master > +creates the socket and listens for any slave connection requests. The so= cket > +may already exist on the system. Be sure to remove any such sockets, if = you > +are creating a master interface, or you will see an "Address already in = use" > +error. Function ``rte_pmd_memif_remove()``, which removes memif interfac= e, > +will also remove a listener socket, if it is not being used by any other > +interface. > + > +The method to enable one or more interfaces is to use the > +``--vdev=3Dnet_memif0`` option on the DPDK application command line. Each > +``--vdev=3Dnet_memif1`` option given will create an interface named net_= memif0, > +net_memif1, and so on. Memif uses unix domain socket to transmit control > +messages. Each memif has a unique id per socket. This id is used to iden= tify > +peer interface. If you are connecting multiple > +interfaces using same socket, be sure to specify unique ids ``id=3D0``, = ``id=3D1``, > +etc. Note that if you assign a socket to a master interface it becomes a > +listener socket. Listener socket can not be used by a slave interface on= same > +client. > + > +.. csv-table:: **Memif configuration options** > + :header: "Option", "Description", "Default", "Valid value" > + > + "id=3D0", "Used to identify peer interface", "0", "uint32_t" > + "role=3Dmaster", "Set memif role", "slave", "master|slave" > + "bsize=3D1024", "Size of single packet buffer", "2048", "uint16_t" > + "rsize=3D11", "Log2 of ring size. If rsize is 10, actual ring size is= 1024", "10", "1-14" > + "socket=3D/tmp/memif.sock", "Socket filename", "/tmp/memif.sock", "st= ring len 256" > + "mac=3D01:23:45:ab:cd:ef", "Mac address", "01:ab:23:cd:45:ef", "" > + "secret=3Dabc123", "Secret is an optional security option, which if s= pecified, must be matched by peer", "", "string len 24" > + "zero-copy=3Dyes", "Enable/disable zero-copy slave mode", "no", "yes|= no" > + > +**Connection establishment** > + > +In order to create memif connection, two memif interfaces, each in separ= ate > +process, are needed. One interface in ``master`` role and other in > +``slave`` role. It is not possible to connect two interfaces in a single > +process. Each interface can be connected to one interface at same time, > +identified by matching id parameter. > + > +Memif driver uses unix domain socket to exchange required information be= tween > +memif interfaces. Socket file path is specified at interface creation see > +*Memif configuration options* table above. If socket is used by ``master= `` > +interface, it's marked as listener socket (in scope of current process) = and > +listens to connection requests from other processes. One socket can be u= sed by > +multiple interfaces. One process can have ``slave`` and ``master`` inter= faces > +at the same time, provided each role is assigned unique socket. > + > +For detailed information on memif control messages, see: net/memif/memif= .h. > + > +Slave interface attempts to make a connection on assigned socket. Process > +listening on this socket will extract the connection request and create = a new > +connected socket (control channel). Then it sends the 'hello' message > +(``MEMIF_MSG_TYPE_HELLO``), containing configuration boundaries. Slave i= nterface > +adjusts its configuration accordingly, and sends 'init' message > +(``MEMIF_MSG_TYPE_INIT``). This message among others contains interface = id. Driver > +uses this id to find master interface, and assigns the control channel t= o this > +interface. If such interface is found, 'ack' message (``MEMIF_MSG_TYPE_A= CK``) is > +sent. Slave interface sends 'add region' message (``MEMIF_MSG_TYPE_ADD_R= EGION``) for > +every region allocated. Master responds to each of these messages with '= ack' > +message. Same behavior applies to rings. Slave sends 'add ring' message > +(``MEMIF_MSG_TYPE_ADD_RING``) for every initialized ring. Master again r= esponds to > +each message with 'ack' message. To finalize the connection, slave inter= face > +sends 'connect' message (``MEMIF_MSG_TYPE_CONNECT``). Upon receiving thi= s message > +master maps regions to its address space, initializes rings and responds= with > +'connected' message (``MEMIF_MSG_TYPE_CONNECTED``). Disconnect > +(``MEMIF_MSG_TYPE_DISCONNECT``) can be sent by both master and slave int= erfaces at > +any time, due to driver error or if the interface is being deleted. > + > +Files > + > +- net/memif/memif.h *- control messages definitions* > +- net/memif/memif_socket.h > +- net/memif/memif_socket.c > + > +Shared memory > +~~~~~~~~~~~~~ > + > +**Shared memory format** > + > +Slave is producer and master is consumer. Memory regions, are mapped sha= red memory files, > +created by memif slave and provided to master at connection establishmen= t. > +Regions contain rings and buffers. Rings and buffers can also be separat= ed into multiple > +regions. For no-zero-copy, rings and buffers are stored inside single me= mory > +region to reduce the number of opened files. > + > +region n (no-zero-copy): > + > ++-----------------------+-----------------------------------------------= --------------------------+ > +| Rings | Buffers = | > ++-----------+-----------+-----------------+---+-------------------------= --------------------------+ > +| S2M rings | M2S rings | packet buffer 0 | . | pb ((1 << pmd->run.log2_= ring_size)*(s2m + m2s))-1 | > ++-----------+-----------+-----------------+---+-------------------------= --------------------------+ > + > +S2M OR M2S Rings: > + > ++--------+--------+-----------------------+ > +| ring 0 | ring 1 | ring num_s2m_rings - 1| > ++--------+--------+-----------------------+ > + > +ring 0: > + > ++-------------+---------------------------------------+ > +| ring header | (1 << pmd->run.log2_ring_size) * desc | > ++-------------+---------------------------------------+ > + > +Descriptors are assigned packet buffers in order of rings creation. If w= e have one ring > +in each direction and ring size is 1024, then first 1024 buffers will be= long to S2M ring and > +last 1024 will belong to M2S ring. In case of zero-copy, buffers are deq= ueued and > +enqueued as needed. > + > +**Descriptor format** > + > ++----+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+= -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > +|Quad|6| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |3|3|= | | | | | | | | | | | | | |1|1| | | | | | | | | | | | | | | | > +| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+= -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > +|Word|3| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |2|1|= | | | | | | | | | | | | | |6|5| | | | | | | | | | | | | | |0| > ++----+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+= -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > +|0 |length |re= gion |flags | > ++----+---------------------------------------------------------------+--= -----------------------------+-------------------------------+ > +|1 |metadata |of= fset | > ++----+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+= -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > +| |6| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |3|3|= | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | > +| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+= -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > +| |3| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |2|1|= | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |0| > ++----+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+= -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > + > +**Flags field - flags (Quad Word 0, bits 0:15)** > + > ++-----+--------------------+--------------------------------------------= ----------------------------------------------------+ > +|Bits |Name |Functionality = | > ++=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ > +|0 |MEMIF_DESC_FLAG_NEXT|Is chained buffer. When set, the packet is d= ivided into multiple buffers. May not be contiguous.| > ++-----+--------------------+--------------------------------------------= ----------------------------------------------------+ > + > +**Region index - region (Quad Word 0, 16:31)** > + > +Index of memory region, the buffer is located in. > + > +**Data length - length (Quad Word 0, 32:63)** > + > +Length of transmitted/received data. > + > +**Data Offset - offset (Quad Word 1, 0:31)** > + > +Data start offset from memory region address. *.regions[desc->region].ad= dr + desc->offset* > + > +**Metadata - metadata (Quad Word 1, 32:63)** > + > +Buffer metadata. > + > +Files > + > +- net/memif/memif.h *- descriptor and ring definitions* > +- net/memif/rte_eth_memif.c *- eth_memif_rx() eth_memif_tx()* > + > +Example: testpmd > +---------------------------- > +In this example we run two instances of testpmd application and transmit= packets over memif. > + > +First create ``master`` interface:: > + > + #./build/app/testpmd -l 0-1 --proc-type=3Dprimary --file-prefix=3Dpm= d1 --vdev=3Dnet_memif,role=3Dmaster -- -i > + > +Now create ``slave`` interface (master must be already running so the sl= ave will connect):: > + > + #./build/app/testpmd -l 2-3 --proc-type=3Dprimary --file-prefix=3Dpm= d2 --vdev=3Dnet_memif -- -i > + > +Start forwarding packets:: > + > + Slave: > + testpmd> start > + > + Master: > + testpmd> start tx_first > + > +Show status:: > + > + testpmd> show port stats 0 > + > +For more details on testpmd please refer to :doc:`../testpmd_app_ug/inde= x`. > + > +Example: testpmd and VPP > +------------------------ > +For information on how to get and run VPP please see ``_. > + > +Start VPP in interactive mode (should be by default). Create memif maste= r interface in VPP:: > + > + vpp# create interface memif id 0 master no-zero-copy > + vpp# set interface state memif0/0 up > + vpp# set interface ip address memif0/0 192.168.1.1/24 > + > +To see socket filename use show memif command:: > + > + vpp# show memif > + sockets > + id listener filename > + 0 yes (1) /run/vpp/memif.sock > + ... > + > +Now create memif interface by running testpmd with these command line op= tions:: > + > + #./testpmd --vdev=3Dnet_memif,socket=3D/run/vpp/memif.sock -- -i > + > +Testpmd should now create memif slave interface and try to connect to ma= ster. > +In testpmd set forward option to icmpecho and start forwarding:: > + > + testpmd> set fwd icmpecho > + testpmd> start > + > +Send ping from VPP:: > + > + vpp# ping 192.168.1.2 > + 64 bytes from 192.168.1.2: icmp_seq=3D2 ttl=3D254 time=3D36.2918 ms > + 64 bytes from 192.168.1.2: icmp_seq=3D3 ttl=3D254 time=3D23.3927 ms > + 64 bytes from 192.168.1.2: icmp_seq=3D4 ttl=3D254 time=3D24.2975 ms > + 64 bytes from 192.168.1.2: icmp_seq=3D5 ttl=3D254 time=3D17.7049 ms > diff --git a/doc/guides/rel_notes/release_19_08.rst b/doc/guides/rel_note= s/release_19_08.rst > index a17e7dea5..46efd13f0 100644 > --- a/doc/guides/rel_notes/release_19_08.rst > +++ b/doc/guides/rel_notes/release_19_08.rst > @@ -54,6 +54,11 @@ New Features > Also, make sure to start the actual text at the margin. > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > +* **Added memif PMD.** > + > + Added the new Shared Memory Packet Interface (``memif``) PMD. > + See the :doc:`../nics/memif` guide for more details on this new driver. > + > > Removed Items > ------------- > diff --git a/drivers/net/Makefile b/drivers/net/Makefile > index 3a72cf38c..78cb10fc6 100644 > --- a/drivers/net/Makefile > +++ b/drivers/net/Makefile > @@ -35,6 +35,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_ICE_PMD) +=3D ice > DIRS-$(CONFIG_RTE_LIBRTE_IPN3KE_PMD) +=3D ipn3ke > DIRS-$(CONFIG_RTE_LIBRTE_IXGBE_PMD) +=3D ixgbe > DIRS-$(CONFIG_RTE_LIBRTE_LIO_PMD) +=3D liquidio > +DIRS-$(CONFIG_RTE_LIBRTE_PMD_MEMIF) +=3D memif > DIRS-$(CONFIG_RTE_LIBRTE_MLX4_PMD) +=3D mlx4 > DIRS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) +=3D mlx5 > DIRS-$(CONFIG_RTE_LIBRTE_MVNETA_PMD) +=3D mvneta > diff --git a/drivers/net/memif/Makefile b/drivers/net/memif/Makefile > new file mode 100644 > index 000000000..c3119625c > --- /dev/null > +++ b/drivers/net/memif/Makefile > @@ -0,0 +1,31 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright 2018-2019 Cisco Systems, Inc. All rights reserved. > + > +include $(RTE_SDK)/mk/rte.vars.mk > + > +# > +# library name > +# > +LIB =3D librte_pmd_memif.a > + > +EXPORT_MAP :=3D rte_pmd_memif_version.map > + > +LIBABIVER :=3D 1 > + > +CFLAGS +=3D -O3 > +CFLAGS +=3D $(WERROR_FLAGS) > +CFLAGS +=3D -DALLOW_EXPERIMENTAL_API > +# Experimantal APIs: > +# - rte_intr_callback_unregister_pending > +LDLIBS +=3D -lrte_eal -lrte_mbuf -lrte_mempool > +LDLIBS +=3D -lrte_ethdev -lrte_kvargs > +LDLIBS +=3D -lrte_hash > +LDLIBS +=3D -lrte_bus_vdev > + > +# > +# all source are stored in SRCS-y > +# > +SRCS-$(CONFIG_RTE_LIBRTE_PMD_MEMIF) +=3D rte_eth_memif.c > +SRCS-$(CONFIG_RTE_LIBRTE_PMD_MEMIF) +=3D memif_socket.c > + > +include $(RTE_SDK)/mk/rte.lib.mk > diff --git a/drivers/net/memif/memif.h b/drivers/net/memif/memif.h > new file mode 100644 > index 000000000..3948b1f50 > --- /dev/null > +++ b/drivers/net/memif/memif.h > @@ -0,0 +1,179 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright 2018-2019 Cisco Systems, Inc. All rights reserved. > + */ > + > +#ifndef _MEMIF_H_ > +#define _MEMIF_H_ > + > +#define MEMIF_COOKIE 0x3E31F20 > +#define MEMIF_VERSION_MAJOR 2 > +#define MEMIF_VERSION_MINOR 0 > +#define MEMIF_VERSION ((MEMIF_VERSION_MAJOR << 8) | MEMIF_VERSION_MINOR) > +#define MEMIF_NAME_SZ 32 > + > +/* > + * S2M: direction slave -> master > + * M2S: direction master -> slave > + */ > + > +/* > + * Type definitions > + */ > + > +typedef enum memif_msg_type { > + MEMIF_MSG_TYPE_NONE, > + MEMIF_MSG_TYPE_ACK, > + MEMIF_MSG_TYPE_HELLO, > + MEMIF_MSG_TYPE_INIT, > + MEMIF_MSG_TYPE_ADD_REGION, > + MEMIF_MSG_TYPE_ADD_RING, > + MEMIF_MSG_TYPE_CONNECT, > + MEMIF_MSG_TYPE_CONNECTED, > + MEMIF_MSG_TYPE_DISCONNECT, > +} memif_msg_type_t; > + > +typedef enum { > + MEMIF_RING_S2M, /**< buffer ring in direction slave -> master */ > + MEMIF_RING_M2S, /**< buffer ring in direction master -> slave */ > +} memif_ring_type_t; > + > +typedef enum { > + MEMIF_INTERFACE_MODE_ETHERNET, > + MEMIF_INTERFACE_MODE_IP, > + MEMIF_INTERFACE_MODE_PUNT_INJECT, > +} memif_interface_mode_t; > + > +typedef uint16_t memif_region_index_t; > +typedef uint32_t memif_region_offset_t; > +typedef uint64_t memif_region_size_t; > +typedef uint16_t memif_ring_index_t; > +typedef uint32_t memif_interface_id_t; > +typedef uint16_t memif_version_t; > +typedef uint8_t memif_log2_ring_size_t; > + > +/* > + * Socket messages > + */ > + > + /** > + * M2S > + * Contains master interfaces configuration. > + */ > +typedef struct __rte_packed { > + uint8_t name[MEMIF_NAME_SZ]; /**< Client app name. In this case DPDK ve= rsion */ > + memif_version_t min_version; /**< lowest supported memif version */ > + memif_version_t max_version; /**< highest supported memif version */ > + memif_region_index_t max_region; /**< maximum num of regions */ > + memif_ring_index_t max_m2s_ring; /**< maximum num of M2S ring */ > + memif_ring_index_t max_s2m_ring; /**< maximum num of S2M rings */ > + memif_log2_ring_size_t max_log2_ring_size; /**< maximum ring size (as l= og2) */ > +} memif_msg_hello_t; > + > +/** > + * S2M > + * Contains information required to identify interface > + * to which the slave wants to connect. > + */ > +typedef struct __rte_packed { > + memif_version_t version; /**< memif version */ > + memif_interface_id_t id; /**< interface id */ > + memif_interface_mode_t mode:8; /**< interface mode */ > + uint8_t secret[24]; /**< optional security parameter */ > + uint8_t name[MEMIF_NAME_SZ]; /**< Client app name. In this case DPDK ve= rsion */ > +} memif_msg_init_t; > + > +/** > + * S2M > + * Request master to add new shared memory region to master interface. > + * Shared files file descriptor is passed in cmsghdr. > + */ > +typedef struct __rte_packed { > + memif_region_index_t index; /**< shm regions index */ > + memif_region_size_t size; /**< shm region size */ > +} memif_msg_add_region_t; > + > +/** > + * S2M > + * Request master to add new ring to master interface. > + */ > +typedef struct __rte_packed { > + uint16_t flags; /**< flags */ > +#define MEMIF_MSG_ADD_RING_FLAG_S2M 1 /**< ring is in S2M direction */ > + memif_ring_index_t index; /**< ring index */ > + memif_region_index_t region; /**< region index on which this ring is lo= cated */ > + memif_region_offset_t offset; /**< buffer start offset */ > + memif_log2_ring_size_t log2_ring_size; /**< ring size (log2) */ > + uint16_t private_hdr_size; /**< used for private metadata */ > +} memif_msg_add_ring_t; > + > +/** > + * S2M > + * Finalize connection establishment. > + */ > +typedef struct __rte_packed { > + uint8_t if_name[MEMIF_NAME_SZ]; /**< slave interface name */ > +} memif_msg_connect_t; > + > +/** > + * M2S > + * Finalize connection establishment. > + */ > +typedef struct __rte_packed { > + uint8_t if_name[MEMIF_NAME_SZ]; /**< master interface name */ > +} memif_msg_connected_t; > + > +/** > + * S2M & M2S > + * Disconnect interfaces. > + */ > +typedef struct __rte_packed { > + uint32_t code; /**< error code */ > + uint8_t string[96]; /**< disconnect reason */ > +} memif_msg_disconnect_t; > + > +typedef struct __rte_packed __rte_aligned(128) > +{ > + memif_msg_type_t type:16; > + union { > + memif_msg_hello_t hello; > + memif_msg_init_t init; > + memif_msg_add_region_t add_region; > + memif_msg_add_ring_t add_ring; > + memif_msg_connect_t connect; > + memif_msg_connected_t connected; > + memif_msg_disconnect_t disconnect; > + }; > +} memif_msg_t; > + > +/* > + * Ring and Descriptor Layout > + */ > + > +/** > + * Buffer descriptor. > + */ > +typedef struct __rte_packed { > + uint16_t flags; /**< flags */ > +#define MEMIF_DESC_FLAG_NEXT 1 /**< is chained buffer */ > + memif_region_index_t region; /**< region index on which the buffer is l= ocated */ > + uint32_t length; /**< buffer length */ > + memif_region_offset_t offset; /**< buffer offset */ > + uint32_t metadata; > +} memif_desc_t; > + > +#define MEMIF_CACHELINE_ALIGN_MARK(mark) \ > + uint8_t mark[0] __rte_aligned(RTE_CACHE_LINE_SIZE) > + > +typedef struct { > + MEMIF_CACHELINE_ALIGN_MARK(cacheline0); > + uint32_t cookie; /**< MEMIF_COOKIE */ > + uint16_t flags; /**< flags */ > +#define MEMIF_RING_FLAG_MASK_INT 1 /**< disable interrupt mode */ > + volatile uint16_t head; /**< pointer to ring buffer head */ > + MEMIF_CACHELINE_ALIGN_MARK(cacheline1); > + volatile uint16_t tail; /**< pointer to ring buffer tail */ > + MEMIF_CACHELINE_ALIGN_MARK(cacheline2); > + memif_desc_t desc[0]; /**< buffer descriptors */ > +} memif_ring_t; > + > +#endif /* _MEMIF_H_ */ > diff --git a/drivers/net/memif/memif_socket.c b/drivers/net/memif/memif_s= ocket.c > new file mode 100644 > index 000000000..98e8ceefd > --- /dev/null > +++ b/drivers/net/memif/memif_socket.c > @@ -0,0 +1,1124 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright 2018-2019 Cisco Systems, Inc. All rights reserved. > + */ > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +#include "rte_eth_memif.h" > +#include "memif_socket.h" > + > +static void memif_intr_handler(void *arg); > + > +static ssize_t > +memif_msg_send(int fd, memif_msg_t *msg, int afd) > +{ > + struct msghdr mh =3D { 0 }; > + struct iovec iov[1]; > + struct cmsghdr *cmsg; > + char ctl[CMSG_SPACE(sizeof(int))]; > + > + iov[0].iov_base =3D msg; > + iov[0].iov_len =3D sizeof(memif_msg_t); > + mh.msg_iov =3D iov; > + mh.msg_iovlen =3D 1; > + > + if (afd > 0) { > + memset(&ctl, 0, sizeof(ctl)); > + mh.msg_control =3D ctl; > + mh.msg_controllen =3D sizeof(ctl); > + cmsg =3D CMSG_FIRSTHDR(&mh); > + cmsg->cmsg_len =3D CMSG_LEN(sizeof(int)); > + cmsg->cmsg_level =3D SOL_SOCKET; > + cmsg->cmsg_type =3D SCM_RIGHTS; > + rte_memcpy(CMSG_DATA(cmsg), &afd, sizeof(int)); > + } > + > + return sendmsg(fd, &mh, 0); > +} > + > +static int > +memif_msg_send_from_queue(struct memif_control_channel *cc) > +{ > + ssize_t size; > + int ret =3D 0; > + struct memif_msg_queue_elt *e; > + > + e =3D TAILQ_FIRST(&cc->msg_queue); > + if (e =3D=3D NULL) > + return 0; > + > + size =3D memif_msg_send(cc->intr_handle.fd, &e->msg, e->fd); > + if (size !=3D sizeof(memif_msg_t)) { > + MIF_LOG(ERR, "sendmsg fail: %s.", strerror(errno)); > + ret =3D -1; > + } else { > + MIF_LOG(DEBUG, "Sent msg type %u.", e->msg.type); > + } > + TAILQ_REMOVE(&cc->msg_queue, e, next); > + rte_free(e); > + > + return ret; > +} > + > +static struct memif_msg_queue_elt * > +memif_msg_enq(struct memif_control_channel *cc) > +{ > + struct memif_msg_queue_elt *e; > + > + e =3D rte_zmalloc("memif_msg", sizeof(struct memif_msg_queue_elt), 0); > + if (e =3D=3D NULL) { > + MIF_LOG(ERR, "Failed to allocate control message."); > + return NULL; > + } > + > + e->fd =3D -1; > + TAILQ_INSERT_TAIL(&cc->msg_queue, e, next); > + > + return e; > +} > + > +void > +memif_msg_enq_disconnect(struct memif_control_channel *cc, const char *r= eason, > + int err_code) > +{ > + struct memif_msg_queue_elt *e; > + struct pmd_internals *pmd; > + memif_msg_disconnect_t *d; > + > + if (cc =3D=3D NULL) { > + MIF_LOG(DEBUG, "Missing control channel."); > + return; > + } > + > + e =3D memif_msg_enq(cc); > + if (e =3D=3D NULL) { > + MIF_LOG(WARNING, "Failed to enqueue disconnect message."); > + return; > + } > + > + d =3D &e->msg.disconnect; > + > + e->msg.type =3D MEMIF_MSG_TYPE_DISCONNECT; > + d->code =3D err_code; > + > + if (reason !=3D NULL) { > + strlcpy((char *)d->string, reason, sizeof(d->string)); > + if (cc->dev !=3D NULL) { > + pmd =3D cc->dev->data->dev_private; > + strlcpy(pmd->local_disc_string, reason, > + sizeof(pmd->local_disc_string)); > + } > + } > +} > + > +static int > +memif_msg_enq_hello(struct memif_control_channel *cc) > +{ > + struct memif_msg_queue_elt *e =3D memif_msg_enq(cc); > + memif_msg_hello_t *h; > + > + if (e =3D=3D NULL) > + return -1; > + > + h =3D &e->msg.hello; > + > + e->msg.type =3D MEMIF_MSG_TYPE_HELLO; > + h->min_version =3D MEMIF_VERSION; > + h->max_version =3D MEMIF_VERSION; > + h->max_s2m_ring =3D ETH_MEMIF_MAX_NUM_Q_PAIRS; > + h->max_m2s_ring =3D ETH_MEMIF_MAX_NUM_Q_PAIRS; > + h->max_region =3D ETH_MEMIF_MAX_REGION_NUM - 1; > + h->max_log2_ring_size =3D ETH_MEMIF_MAX_LOG2_RING_SIZE; > + > + strlcpy((char *)h->name, rte_version(), sizeof(h->name)); > + > + return 0; > +} > + > +static int > +memif_msg_receive_hello(struct rte_eth_dev *dev, memif_msg_t *msg) > +{ > + struct pmd_internals *pmd =3D dev->data->dev_private; > + memif_msg_hello_t *h =3D &msg->hello; > + > + if (h->min_version > MEMIF_VERSION || h->max_version < MEMIF_VERSION) { > + memif_msg_enq_disconnect(pmd->cc, "Incompatible memif version", 0); > + return -1; > + } > + > + /* Set parameters for active connection */ > + pmd->run.num_s2m_rings =3D RTE_MIN(h->max_s2m_ring + 1, > + pmd->cfg.num_s2m_rings); > + pmd->run.num_m2s_rings =3D RTE_MIN(h->max_m2s_ring + 1, > + pmd->cfg.num_m2s_rings); > + pmd->run.log2_ring_size =3D RTE_MIN(h->max_log2_ring_size, > + pmd->cfg.log2_ring_size); > + pmd->run.pkt_buffer_size =3D pmd->cfg.pkt_buffer_size; > + > + strlcpy(pmd->remote_name, (char *)h->name, sizeof(pmd->remote_name)); > + > + MIF_LOG(DEBUG, "%s: Connecting to %s.", > + rte_vdev_device_name(pmd->vdev), pmd->remote_name); > + > + return 0; > +} > + > +static int > +memif_msg_receive_init(struct memif_control_channel *cc, memif_msg_t *ms= g) > +{ > + memif_msg_init_t *i =3D &msg->init; > + struct memif_socket_dev_list_elt *elt; > + struct pmd_internals *pmd; > + struct rte_eth_dev *dev; > + > + if (i->version !=3D MEMIF_VERSION) { > + memif_msg_enq_disconnect(cc, "Incompatible memif version", 0); > + return -1; > + } > + > + if (cc->socket =3D=3D NULL) { > + memif_msg_enq_disconnect(cc, "Device error", 0); > + return -1; > + } > + > + /* Find device with requested ID */ > + TAILQ_FOREACH(elt, &cc->socket->dev_queue, next) { > + dev =3D elt->dev; > + pmd =3D dev->data->dev_private; > + if (((pmd->flags & ETH_MEMIF_FLAG_DISABLED) =3D=3D 0) && > + pmd->id =3D=3D i->id) { > + /* assign control channel to device */ > + cc->dev =3D dev; > + pmd->cc =3D cc; > + > + if (i->mode !=3D MEMIF_INTERFACE_MODE_ETHERNET) { > + memif_msg_enq_disconnect(pmd->cc, > + "Only ethernet mode supported", > + 0); > + return -1; > + } > + > + if (pmd->flags & (ETH_MEMIF_FLAG_CONNECTING | > + ETH_MEMIF_FLAG_CONNECTED)) { > + memif_msg_enq_disconnect(pmd->cc, > + "Already connected", 0); > + return -1; > + } > + strlcpy(pmd->remote_name, (char *)i->name, > + sizeof(pmd->remote_name)); > + > + if (*pmd->secret !=3D '\0') { > + if (*i->secret =3D=3D '\0') { > + memif_msg_enq_disconnect(pmd->cc, > + "Secret required", 0); > + return -1; > + } > + if (strncmp(pmd->secret, (char *)i->secret, > + ETH_MEMIF_SECRET_SIZE) !=3D 0) { > + memif_msg_enq_disconnect(pmd->cc, > + "Incorrect secret", 0); > + return -1; > + } > + } > + > + pmd->flags |=3D ETH_MEMIF_FLAG_CONNECTING; > + return 0; > + } > + } > + > + /* ID not found on this socket */ > + MIF_LOG(DEBUG, "ID %u not found.", i->id); > + memif_msg_enq_disconnect(cc, "ID not found", 0); > + return -1; > +} > + > +static int > +memif_msg_receive_add_region(struct rte_eth_dev *dev, memif_msg_t *msg, > + int fd) > +{ > + struct pmd_internals *pmd =3D dev->data->dev_private; > + memif_msg_add_region_t *ar =3D &msg->add_region; > + struct memif_region *r; > + > + if (fd < 0) { > + memif_msg_enq_disconnect(pmd->cc, "Missing region fd", 0); > + return -1; > + } > + > + if (ar->index >=3D ETH_MEMIF_MAX_REGION_NUM || ar->index !=3D pmd->regi= ons_num || > + pmd->regions[ar->index] !=3D NULL) { > + memif_msg_enq_disconnect(pmd->cc, "Invalid region index", 0); > + return -1; > + } > + > + r =3D rte_zmalloc("region", sizeof(struct memif_region), 0); > + if (r =3D=3D NULL) { > + MIF_LOG(ERR, "%s: Failed to alloc memif region.", > + rte_vdev_device_name(pmd->vdev)); > + return -ENOMEM; > + } > + > + r->fd =3D fd; > + r->region_size =3D ar->size; > + r->addr =3D NULL; > + > + pmd->regions[ar->index] =3D r; > + pmd->regions_num++; > + > + return 0; > +} > + > +static int > +memif_msg_receive_add_ring(struct rte_eth_dev *dev, memif_msg_t *msg, in= t fd) > +{ > + struct pmd_internals *pmd =3D dev->data->dev_private; > + memif_msg_add_ring_t *ar =3D &msg->add_ring; > + struct memif_queue *mq; > + > + if (fd < 0) { > + memif_msg_enq_disconnect(pmd->cc, "Missing interrupt fd", 0); > + return -1; > + } > + > + /* check if we have enough queues */ > + if (ar->flags & MEMIF_MSG_ADD_RING_FLAG_S2M) { > + if (ar->index >=3D pmd->cfg.num_s2m_rings) { > + memif_msg_enq_disconnect(pmd->cc, "Invalid ring index", 0); > + return -1; > + } > + pmd->run.num_s2m_rings++; > + } else { > + if (ar->index >=3D pmd->cfg.num_m2s_rings) { > + memif_msg_enq_disconnect(pmd->cc, "Invalid ring index", 0); > + return -1; > + } > + pmd->run.num_m2s_rings++; > + } > + > + mq =3D (ar->flags & MEMIF_MSG_ADD_RING_FLAG_S2M) ? > + dev->data->rx_queues[ar->index] : dev->data->tx_queues[ar->index]; > + > + mq->intr_handle.fd =3D fd; > + mq->log2_ring_size =3D ar->log2_ring_size; > + mq->region =3D ar->region; > + mq->ring_offset =3D ar->offset; > + > + return 0; > +} > + > +static int > +memif_msg_receive_connect(struct rte_eth_dev *dev, memif_msg_t *msg) > +{ > + struct pmd_internals *pmd =3D dev->data->dev_private; > + memif_msg_connect_t *c =3D &msg->connect; > + int ret; > + > + ret =3D memif_connect(dev); > + if (ret < 0) > + return ret; > + > + strlcpy(pmd->remote_if_name, (char *)c->if_name, > + sizeof(pmd->remote_if_name)); > + MIF_LOG(INFO, "%s: Remote interface %s connected.", > + rte_vdev_device_name(pmd->vdev), pmd->remote_if_name); > + > + return 0; > +} > + > +static int > +memif_msg_receive_connected(struct rte_eth_dev *dev, memif_msg_t *msg) > +{ > + struct pmd_internals *pmd =3D dev->data->dev_private; > + memif_msg_connected_t *c =3D &msg->connected; > + int ret; > + > + ret =3D memif_connect(dev); > + if (ret < 0) > + return ret; > + > + strlcpy(pmd->remote_if_name, (char *)c->if_name, > + sizeof(pmd->remote_if_name)); > + MIF_LOG(INFO, "%s: Remote interface %s connected.", > + rte_vdev_device_name(pmd->vdev), pmd->remote_if_name); > + > + return 0; > +} > + > +static int > +memif_msg_receive_disconnect(struct rte_eth_dev *dev, memif_msg_t *msg) > +{ > + struct pmd_internals *pmd =3D dev->data->dev_private; > + memif_msg_disconnect_t *d =3D &msg->disconnect; > + > + memset(pmd->remote_disc_string, 0, ETH_MEMIF_DISC_STRING_SIZE); > + strlcpy(pmd->remote_disc_string, (char *)d->string, > + sizeof(pmd->remote_disc_string)); > + > + MIF_LOG(INFO, "%s: Disconnect received: %s", > + rte_vdev_device_name(pmd->vdev), pmd->remote_disc_string); > + > + memset(pmd->local_disc_string, 0, ETH_MEMIF_DISC_STRING_SIZE); > + memif_disconnect(rte_eth_dev_allocated > + (rte_vdev_device_name(pmd->vdev))); > + return 0; > +} > + > +static int > +memif_msg_enq_ack(struct rte_eth_dev *dev) > +{ > + struct pmd_internals *pmd =3D dev->data->dev_private; > + struct memif_msg_queue_elt *e =3D memif_msg_enq(pmd->cc); > + if (e =3D=3D NULL) > + return -1; > + > + e->msg.type =3D MEMIF_MSG_TYPE_ACK; > + > + return 0; > +} > + > +static int > +memif_msg_enq_init(struct rte_eth_dev *dev) > +{ > + struct pmd_internals *pmd =3D dev->data->dev_private; > + struct memif_msg_queue_elt *e =3D memif_msg_enq(pmd->cc); > + memif_msg_init_t *i =3D &e->msg.init; > + > + if (e =3D=3D NULL) > + return -1; > + > + i =3D &e->msg.init; > + e->msg.type =3D MEMIF_MSG_TYPE_INIT; > + i->version =3D MEMIF_VERSION; > + i->id =3D pmd->id; > + i->mode =3D MEMIF_INTERFACE_MODE_ETHERNET; > + > + strlcpy((char *)i->name, rte_version(), sizeof(i->name)); > + > + if (*pmd->secret !=3D '\0') > + strlcpy((char *)i->secret, pmd->secret, sizeof(i->secret)); > + > + return 0; > +} > + > +static int > +memif_msg_enq_add_region(struct rte_eth_dev *dev, uint8_t idx) > +{ > + struct pmd_internals *pmd =3D dev->data->dev_private; > + struct memif_msg_queue_elt *e =3D memif_msg_enq(pmd->cc); > + memif_msg_add_region_t *ar; > + struct memif_region *mr =3D pmd->regions[idx]; > + > + if (e =3D=3D NULL) > + return -1; > + > + ar =3D &e->msg.add_region; > + e->msg.type =3D MEMIF_MSG_TYPE_ADD_REGION; > + e->fd =3D mr->fd; > + ar->index =3D idx; > + ar->size =3D mr->region_size; > + > + return 0; > +} > + > +static int > +memif_msg_enq_add_ring(struct rte_eth_dev *dev, uint8_t idx, > + memif_ring_type_t type) > +{ > + struct pmd_internals *pmd =3D dev->data->dev_private; > + struct memif_msg_queue_elt *e =3D memif_msg_enq(pmd->cc); > + struct memif_queue *mq; > + memif_msg_add_ring_t *ar; > + > + if (e =3D=3D NULL) > + return -1; > + > + ar =3D &e->msg.add_ring; > + mq =3D (type =3D=3D MEMIF_RING_S2M) ? dev->data->tx_queues[idx] : > + dev->data->rx_queues[idx]; > + > + e->msg.type =3D MEMIF_MSG_TYPE_ADD_RING; > + e->fd =3D mq->intr_handle.fd; > + ar->index =3D idx; > + ar->offset =3D mq->ring_offset; > + ar->region =3D mq->region; > + ar->log2_ring_size =3D mq->log2_ring_size; > + ar->flags =3D (type =3D=3D MEMIF_RING_S2M) ? MEMIF_MSG_ADD_RING_FLAG_S2= M : 0; > + ar->private_hdr_size =3D 0; > + > + return 0; > +} > + > +static int > +memif_msg_enq_connect(struct rte_eth_dev *dev) > +{ > + struct pmd_internals *pmd =3D dev->data->dev_private; > + struct memif_msg_queue_elt *e =3D memif_msg_enq(pmd->cc); > + const char *name =3D rte_vdev_device_name(pmd->vdev); > + memif_msg_connect_t *c; > + > + if (e =3D=3D NULL) > + return -1; > + > + c =3D &e->msg.connect; > + e->msg.type =3D MEMIF_MSG_TYPE_CONNECT; > + strlcpy((char *)c->if_name, name, sizeof(c->if_name)); > + > + return 0; > +} > + > +static int > +memif_msg_enq_connected(struct rte_eth_dev *dev) > +{ > + struct pmd_internals *pmd =3D dev->data->dev_private; > + struct memif_msg_queue_elt *e =3D memif_msg_enq(pmd->cc); > + const char *name =3D rte_vdev_device_name(pmd->vdev); > + memif_msg_connected_t *c; > + > + if (e =3D=3D NULL) > + return -1; > + > + c =3D &e->msg.connected; > + e->msg.type =3D MEMIF_MSG_TYPE_CONNECTED; > + strlcpy((char *)c->if_name, name, sizeof(c->if_name)); > + > + return 0; > +} > + > +static void > +memif_intr_unregister_handler(struct rte_intr_handle *intr_handle, void = *arg) > +{ > + struct memif_msg_queue_elt *elt; > + struct memif_control_channel *cc =3D arg; > + > + /* close control channel fd */ > + close(intr_handle->fd); > + /* clear message queue */ > + while ((elt =3D TAILQ_FIRST(&cc->msg_queue)) !=3D NULL) { > + TAILQ_REMOVE(&cc->msg_queue, elt, next); > + rte_free(elt); > + } > + /* free control channel */ > + rte_free(cc); > +} > + > +void > +memif_disconnect(struct rte_eth_dev *dev) > +{ > + struct pmd_internals *pmd =3D dev->data->dev_private; > + struct memif_msg_queue_elt *elt, *next; > + struct memif_queue *mq; > + struct rte_intr_handle *ih; > + int i; > + int ret; > + > + if (pmd->cc !=3D NULL) { > + /* Clear control message queue (except disconnect message if any). */ > + for (elt =3D TAILQ_FIRST(&pmd->cc->msg_queue); elt !=3D NULL; elt =3D = next) { > + next =3D TAILQ_NEXT(elt, next); > + if (elt->msg.type !=3D MEMIF_MSG_TYPE_DISCONNECT) { > + TAILQ_REMOVE(&pmd->cc->msg_queue, elt, next); > + rte_free(elt); > + } > + } > + /* send disconnect message (if there is any in queue) */ > + memif_msg_send_from_queue(pmd->cc); > + > + /* at this point, there should be no more messages in queue */ > + if (TAILQ_FIRST(&pmd->cc->msg_queue) !=3D NULL) { > + MIF_LOG(WARNING, > + "%s: Unexpected message(s) in message queue.", > + rte_vdev_device_name(pmd->vdev)); > + } > + > + ih =3D &pmd->cc->intr_handle; > + if (ih->fd > 0) { > + ret =3D rte_intr_callback_unregister(ih, > + memif_intr_handler, > + pmd->cc); > + /* > + * If callback is active (disconnecting based on > + * received control message). > + */ > + if (ret =3D=3D -EAGAIN) { > + ret =3D rte_intr_callback_unregister_pending(ih, > + memif_intr_handler, > + pmd->cc, > + memif_intr_unregister_handler); > + } else if (ret > 0) { > + close(ih->fd); > + rte_free(pmd->cc); > + } > + pmd->cc =3D NULL; > + if (ret <=3D 0) > + MIF_LOG(WARNING, "%s: Failed to unregister " > + "control channel callback.", > + rte_vdev_device_name(pmd->vdev)); > + } > + } > + > + /* unconfig interrupts */ > + for (i =3D 0; i < pmd->cfg.num_s2m_rings; i++) { > + if (pmd->role =3D=3D MEMIF_ROLE_SLAVE) { > + if (dev->data->tx_queues !=3D NULL) > + mq =3D dev->data->tx_queues[i]; > + else > + continue; > + } else { > + if (dev->data->rx_queues !=3D NULL) > + mq =3D dev->data->rx_queues[i]; > + else > + continue; > + } > + if (mq->intr_handle.fd > 0) { > + close(mq->intr_handle.fd); > + mq->intr_handle.fd =3D -1; > + } > + mq->ring =3D NULL; > + } > + for (i =3D 0; i < pmd->cfg.num_m2s_rings; i++) { > + if (pmd->role =3D=3D MEMIF_ROLE_MASTER) { > + if (dev->data->tx_queues !=3D NULL) > + mq =3D dev->data->tx_queues[i]; > + else > + continue; > + } else { > + if (dev->data->rx_queues !=3D NULL) > + mq =3D dev->data->rx_queues[i]; > + else > + continue; > + } > + if (mq->intr_handle.fd > 0) { > + close(mq->intr_handle.fd); > + mq->intr_handle.fd =3D -1; > + } > + mq->ring =3D NULL; > + } > + > + memif_free_regions(pmd); > + > + /* reset connection configuration */ > + memset(&pmd->run, 0, sizeof(pmd->run)); > + > + dev->data->dev_link.link_status =3D ETH_LINK_DOWN; > + pmd->flags &=3D ~ETH_MEMIF_FLAG_CONNECTING; > + pmd->flags &=3D ~ETH_MEMIF_FLAG_CONNECTED; > + MIF_LOG(DEBUG, "%s: Disconnected.", rte_vdev_device_name(pmd->vdev)); > +} > + > +static int > +memif_msg_receive(struct memif_control_channel *cc) > +{ > + char ctl[CMSG_SPACE(sizeof(int)) + > + CMSG_SPACE(sizeof(struct ucred))] =3D { 0 }; > + struct msghdr mh =3D { 0 }; > + struct iovec iov[1]; > + memif_msg_t msg =3D { 0 }; > + ssize_t size; > + int ret =3D 0; > + struct ucred *cr __rte_unused; > + cr =3D 0; > + struct cmsghdr *cmsg; > + int afd =3D -1; > + int i; > + struct pmd_internals *pmd; > + > + iov[0].iov_base =3D (void *)&msg; > + iov[0].iov_len =3D sizeof(memif_msg_t); > + mh.msg_iov =3D iov; > + mh.msg_iovlen =3D 1; > + mh.msg_control =3D ctl; > + mh.msg_controllen =3D sizeof(ctl); > + > + size =3D recvmsg(cc->intr_handle.fd, &mh, 0); > + if (size !=3D sizeof(memif_msg_t)) { > + MIF_LOG(DEBUG, "Invalid message size."); > + memif_msg_enq_disconnect(cc, "Invalid message size", 0); > + return -1; > + } > + MIF_LOG(DEBUG, "Received msg type: %u.", msg.type); > + > + cmsg =3D CMSG_FIRSTHDR(&mh); > + while (cmsg) { > + if (cmsg->cmsg_level =3D=3D SOL_SOCKET) { > + if (cmsg->cmsg_type =3D=3D SCM_CREDENTIALS) > + cr =3D (struct ucred *)CMSG_DATA(cmsg); > + else if (cmsg->cmsg_type =3D=3D SCM_RIGHTS) > + afd =3D *(int *)CMSG_DATA(cmsg); I see that GCC from Xenial: gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609 Complains as follows: ../drivers/net/memif/memif_socket.c: In function =E2=80=98memif_msg_receive= =E2=80=99: ../drivers/net/memif/memif_socket.c:665:33: error: dereferencing type-punne= d pointer will break strict-aliasing rules [-Werror=3Dstrict-aliasing] afd =3D *(int *)CMSG_DATA(cmsg); ^ Here's the travis build: https://travis-ci.com/ovsrobot/dpdk/builds/113842433 > + } > + cmsg =3D CMSG_NXTHDR(&mh, cmsg); > + } > + > + if (cc->dev =3D=3D NULL && msg.type !=3D MEMIF_MSG_TYPE_INIT) { > + MIF_LOG(DEBUG, "Unexpected message."); > + memif_msg_enq_disconnect(cc, "Unexpected message", 0); > + return -1; > + } > + > + /* get device from hash data */ > + switch (msg.type) { > + case MEMIF_MSG_TYPE_ACK: > + break; > + case MEMIF_MSG_TYPE_HELLO: > + ret =3D memif_msg_receive_hello(cc->dev, &msg); > + if (ret < 0) > + goto exit; > + ret =3D memif_init_regions_and_queues(cc->dev); > + if (ret < 0) > + goto exit; > + ret =3D memif_msg_enq_init(cc->dev); > + if (ret < 0) > + goto exit; > + pmd =3D cc->dev->data->dev_private; > + for (i =3D 0; i < pmd->regions_num; i++) { > + ret =3D memif_msg_enq_add_region(cc->dev, i); > + if (ret < 0) > + goto exit; > + } > + for (i =3D 0; i < pmd->run.num_s2m_rings; i++) { > + ret =3D memif_msg_enq_add_ring(cc->dev, i, > + MEMIF_RING_S2M); > + if (ret < 0) > + goto exit; > + } > + for (i =3D 0; i < pmd->run.num_m2s_rings; i++) { > + ret =3D memif_msg_enq_add_ring(cc->dev, i, > + MEMIF_RING_M2S); > + if (ret < 0) > + goto exit; > + } > + ret =3D memif_msg_enq_connect(cc->dev); > + if (ret < 0) > + goto exit; > + break; > + case MEMIF_MSG_TYPE_INIT: > + /* > + * This cc does not have an interface asociated with it. > + * If suitable interface is found it will be assigned here. > + */ > + ret =3D memif_msg_receive_init(cc, &msg); > + if (ret < 0) > + goto exit; > + ret =3D memif_msg_enq_ack(cc->dev); > + if (ret < 0) > + goto exit; > + break; > + case MEMIF_MSG_TYPE_ADD_REGION: > + ret =3D memif_msg_receive_add_region(cc->dev, &msg, afd); > + if (ret < 0) > + goto exit; > + ret =3D memif_msg_enq_ack(cc->dev); > + if (ret < 0) > + goto exit; > + break; > + case MEMIF_MSG_TYPE_ADD_RING: > + ret =3D memif_msg_receive_add_ring(cc->dev, &msg, afd); > + if (ret < 0) > + goto exit; > + ret =3D memif_msg_enq_ack(cc->dev); > + if (ret < 0) > + goto exit; > + break; > + case MEMIF_MSG_TYPE_CONNECT: > + ret =3D memif_msg_receive_connect(cc->dev, &msg); > + if (ret < 0) > + goto exit; > + ret =3D memif_msg_enq_connected(cc->dev); > + if (ret < 0) > + goto exit; > + break; > + case MEMIF_MSG_TYPE_CONNECTED: > + ret =3D memif_msg_receive_connected(cc->dev, &msg); > + break; > + case MEMIF_MSG_TYPE_DISCONNECT: > + ret =3D memif_msg_receive_disconnect(cc->dev, &msg); > + if (ret < 0) > + goto exit; > + break; > + default: > + memif_msg_enq_disconnect(cc, "Unknown message type", 0); > + ret =3D -1; > + goto exit; > + } > + > + exit: > + return ret; > +} > + > +static void > +memif_intr_handler(void *arg) > +{ > + struct memif_control_channel *cc =3D arg; > + int ret; > + > + ret =3D memif_msg_receive(cc); > + /* if driver failed to assign device */ > + if (cc->dev =3D=3D NULL) { > + ret =3D rte_intr_callback_unregister_pending(&cc->intr_handle, > + memif_intr_handler, > + cc, > + memif_intr_unregister_handler); > + if (ret < 0) > + MIF_LOG(WARNING, > + "Failed to unregister control channel callback."); > + return; > + } > + /* if memif_msg_receive failed */ > + if (ret < 0) > + goto disconnect; > + > + ret =3D memif_msg_send_from_queue(cc); > + if (ret < 0) > + goto disconnect; > + > + return; > + > + disconnect: > + if (cc->dev =3D=3D NULL) { > + MIF_LOG(WARNING, "eth dev not allocated"); > + return; > + } > + memif_disconnect(cc->dev); > +} > + > +static void > +memif_listener_handler(void *arg) > +{ > + struct memif_socket *socket =3D arg; > + int sockfd; > + int addr_len; > + struct sockaddr_un client; > + struct memif_control_channel *cc; > + int ret; > + > + addr_len =3D sizeof(client); > + sockfd =3D accept(socket->intr_handle.fd, (struct sockaddr *)&client, > + (socklen_t *)&addr_len); > + if (sockfd < 0) { > + MIF_LOG(ERR, > + "Failed to accept connection request on socket fd %d", > + socket->intr_handle.fd); > + return; > + } > + > + MIF_LOG(DEBUG, "%s: Connection request accepted.", socket->filename); > + > + cc =3D rte_zmalloc("memif-cc", sizeof(struct memif_control_channel), 0); > + if (cc =3D=3D NULL) { > + MIF_LOG(ERR, "Failed to allocate control channel."); > + goto error; > + } > + > + cc->intr_handle.fd =3D sockfd; > + cc->intr_handle.type =3D RTE_INTR_HANDLE_EXT; > + cc->socket =3D socket; > + cc->dev =3D NULL; > + TAILQ_INIT(&cc->msg_queue); > + > + ret =3D rte_intr_callback_register(&cc->intr_handle, memif_intr_handler= , cc); > + if (ret < 0) { > + MIF_LOG(ERR, "Failed to register control channel callback."); > + goto error; > + } > + > + ret =3D memif_msg_enq_hello(cc); > + if (ret < 0) { > + MIF_LOG(ERR, "Failed to enqueue hello message."); > + goto error; > + } > + ret =3D memif_msg_send_from_queue(cc); > + if (ret < 0) > + goto error; > + > + return; > + > + error: > + if (sockfd > 0) { > + close(sockfd); > + sockfd =3D -1; > + } > + if (cc !=3D NULL) > + rte_free(cc); > +} > + > +static struct memif_socket * > +memif_socket_create(struct pmd_internals *pmd, char *key, uint8_t listen= er) > +{ > + struct memif_socket *sock; > + struct sockaddr_un un; > + int sockfd; > + int ret; > + int on =3D 1; > + > + sock =3D rte_zmalloc("memif-socket", sizeof(struct memif_socket), 0); > + if (sock =3D=3D NULL) { > + MIF_LOG(ERR, "Failed to allocate memory for memif socket"); > + return NULL; > + } > + > + sock->listener =3D listener; > + rte_memcpy(sock->filename, key, 256); > + TAILQ_INIT(&sock->dev_queue); > + > + if (listener !=3D 0) { > + sockfd =3D socket(AF_UNIX, SOCK_SEQPACKET, 0); > + if (sockfd < 0) > + goto error; > + > + un.sun_family =3D AF_UNIX; > + memcpy(un.sun_path, sock->filename, > + sizeof(un.sun_path) - 1); > + > + ret =3D setsockopt(sockfd, SOL_SOCKET, SO_PASSCRED, &on, > + sizeof(on)); > + if (ret < 0) > + goto error; > + ret =3D bind(sockfd, (struct sockaddr *)&un, sizeof(un)); > + if (ret < 0) > + goto error; > + ret =3D listen(sockfd, 1); > + if (ret < 0) > + goto error; > + > + MIF_LOG(DEBUG, "%s: Memif listener socket %s created.", > + rte_vdev_device_name(pmd->vdev), sock->filename); > + > + sock->intr_handle.fd =3D sockfd; > + sock->intr_handle.type =3D RTE_INTR_HANDLE_EXT; > + ret =3D rte_intr_callback_register(&sock->intr_handle, > + memif_listener_handler, sock); > + if (ret < 0) { > + MIF_LOG(ERR, "%s: Failed to register interrupt " > + "callback for listener socket", > + rte_vdev_device_name(pmd->vdev)); > + return NULL; > + } > + } > + > + return sock; > + > + error: > + MIF_LOG(ERR, "%s: Failed to setup socket %s: %s", > + rte_vdev_device_name(pmd->vdev), key, strerror(errno)); > + if (sock !=3D NULL) > + rte_free(sock); > + return NULL; > +} > + > +static struct rte_hash * > +memif_create_socket_hash(void) > +{ > + struct rte_hash_parameters params =3D { 0 }; > + params.name =3D MEMIF_SOCKET_HASH_NAME; > + params.entries =3D 256; > + params.key_len =3D 256; > + params.hash_func =3D rte_jhash; > + params.hash_func_init_val =3D 0; > + return rte_hash_create(¶ms); > +} > + > +int > +memif_socket_init(struct rte_eth_dev *dev, const char *socket_filename) > +{ > + struct pmd_internals *pmd =3D dev->data->dev_private; > + struct memif_socket *socket =3D NULL; > + struct memif_socket_dev_list_elt *elt; > + struct pmd_internals *tmp_pmd; > + struct rte_hash *hash; > + int ret; > + char key[256]; > + > + hash =3D rte_hash_find_existing(MEMIF_SOCKET_HASH_NAME); > + if (hash =3D=3D NULL) { > + hash =3D memif_create_socket_hash(); > + if (hash =3D=3D NULL) { > + MIF_LOG(ERR, "Failed to create memif socket hash."); > + return -1; > + } > + } > + > + memset(key, 0, 256); > + rte_memcpy(key, socket_filename, strlen(socket_filename)); > + ret =3D rte_hash_lookup_data(hash, key, (void **)&socket); > + if (ret < 0) { > + socket =3D memif_socket_create(pmd, key, > + (pmd->role =3D=3D > + MEMIF_ROLE_SLAVE) ? 0 : 1); > + if (socket =3D=3D NULL) > + return -1; > + ret =3D rte_hash_add_key_data(hash, key, socket); > + if (ret < 0) { > + MIF_LOG(ERR, "Failed to add socket to socket hash."); > + return ret; > + } > + } > + pmd->socket_filename =3D socket->filename; > + > + if (socket->listener !=3D 0 && pmd->role =3D=3D MEMIF_ROLE_SLAVE) { > + MIF_LOG(ERR, "Socket is a listener."); > + return -1; > + } else if ((socket->listener =3D=3D 0) && (pmd->role =3D=3D MEMIF_ROLE_= MASTER)) { > + MIF_LOG(ERR, "Socket is not a listener."); > + return -1; > + } > + > + TAILQ_FOREACH(elt, &socket->dev_queue, next) { > + tmp_pmd =3D elt->dev->data->dev_private; > + if (tmp_pmd->id =3D=3D pmd->id) { > + MIF_LOG(ERR, "Memif device with id %d already " > + "exists on socket %s", > + pmd->id, socket->filename); > + return -1; > + } > + } > + > + elt =3D rte_malloc("pmd-queue", sizeof(struct memif_socket_dev_list_elt= ), 0); > + if (elt =3D=3D NULL) { > + MIF_LOG(ERR, "%s: Failed to add device to socket device list.", > + rte_vdev_device_name(pmd->vdev)); > + return -1; > + } > + elt->dev =3D dev; > + TAILQ_INSERT_TAIL(&socket->dev_queue, elt, next); > + > + return 0; > +} > + > +void > +memif_socket_remove_device(struct rte_eth_dev *dev) > +{ > + struct pmd_internals *pmd =3D dev->data->dev_private; > + struct memif_socket *socket =3D NULL; > + struct memif_socket_dev_list_elt *elt, *next; > + struct rte_hash *hash; > + > + hash =3D rte_hash_find_existing(MEMIF_SOCKET_HASH_NAME); > + if (hash =3D=3D NULL) > + return; > + > + if (pmd->socket_filename =3D=3D NULL) > + return; > + > + if (rte_hash_lookup_data(hash, pmd->socket_filename, (void **)&socket) = < 0) > + return; > + > + for (elt =3D TAILQ_FIRST(&socket->dev_queue); elt !=3D NULL; elt =3D ne= xt) { > + next =3D TAILQ_NEXT(elt, next); > + if (elt->dev =3D=3D dev) { > + TAILQ_REMOVE(&socket->dev_queue, elt, next); > + rte_free(elt); > + pmd->socket_filename =3D NULL; > + } > + } > + > + /* remove socket, if this was the last device using it */ > + if (TAILQ_EMPTY(&socket->dev_queue)) { > + rte_hash_del_key(hash, socket->filename); > + if (socket->listener) { > + /* remove listener socket file, > + * so we can create new one later. > + */ > + remove(socket->filename); > + } > + rte_free(socket); > + } > +} > + > +int > +memif_connect_master(struct rte_eth_dev *dev) > +{ > + struct pmd_internals *pmd =3D dev->data->dev_private; > + > + memset(pmd->local_disc_string, 0, ETH_MEMIF_DISC_STRING_SIZE); > + memset(pmd->remote_disc_string, 0, ETH_MEMIF_DISC_STRING_SIZE); > + pmd->flags &=3D ~ETH_MEMIF_FLAG_DISABLED; > + return 0; > +} > + > +int > +memif_connect_slave(struct rte_eth_dev *dev) > +{ > + int sockfd; > + int ret; > + struct sockaddr_un sun; > + struct pmd_internals *pmd =3D dev->data->dev_private; > + > + memset(pmd->local_disc_string, 0, ETH_MEMIF_DISC_STRING_SIZE); > + memset(pmd->remote_disc_string, 0, ETH_MEMIF_DISC_STRING_SIZE); > + pmd->flags &=3D ~ETH_MEMIF_FLAG_DISABLED; > + > + sockfd =3D socket(AF_UNIX, SOCK_SEQPACKET, 0); > + if (sockfd < 0) { > + MIF_LOG(ERR, "%s: Failed to open socket.", > + rte_vdev_device_name(pmd->vdev)); > + return -1; > + } > + > + sun.sun_family =3D AF_UNIX; > + > + memcpy(sun.sun_path, pmd->socket_filename, sizeof(sun.sun_path) - 1); > + > + ret =3D connect(sockfd, (struct sockaddr *)&sun, > + sizeof(struct sockaddr_un)); > + if (ret < 0) { > + MIF_LOG(ERR, "%s: Failed to connect socket: %s.", > + rte_vdev_device_name(pmd->vdev), pmd->socket_filename); > + goto error; > + } > + > + MIF_LOG(DEBUG, "%s: Memif socket: %s connected.", > + rte_vdev_device_name(pmd->vdev), pmd->socket_filename); > + > + pmd->cc =3D rte_zmalloc("memif-cc", > + sizeof(struct memif_control_channel), 0); > + if (pmd->cc =3D=3D NULL) { > + MIF_LOG(ERR, "%s: Failed to allocate control channel.", > + rte_vdev_device_name(pmd->vdev)); > + goto error; > + } > + > + pmd->cc->intr_handle.fd =3D sockfd; > + pmd->cc->intr_handle.type =3D RTE_INTR_HANDLE_EXT; > + pmd->cc->socket =3D NULL; > + pmd->cc->dev =3D dev; > + TAILQ_INIT(&pmd->cc->msg_queue); > + > + ret =3D rte_intr_callback_register(&pmd->cc->intr_handle, > + memif_intr_handler, pmd->cc); > + if (ret < 0) { > + MIF_LOG(ERR, "%s: Failed to register interrupt callback " > + "for control fd", rte_vdev_device_name(pmd->vdev)); > + goto error; > + } > + > + return 0; > + > + error: > + if (sockfd > 0) { > + close(sockfd); > + sockfd =3D -1; > + } > + if (pmd->cc !=3D NULL) { > + rte_free(pmd->cc); > + pmd->cc =3D NULL; > + } > + return -1; > +} > diff --git a/drivers/net/memif/memif_socket.h b/drivers/net/memif/memif_s= ocket.h > new file mode 100644 > index 000000000..c60614721 > --- /dev/null > +++ b/drivers/net/memif/memif_socket.h > @@ -0,0 +1,105 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright 2018-2019 Cisco Systems, Inc. All rights reserved. > + */ > + > +#ifndef _MEMIF_SOCKET_H_ > +#define _MEMIF_SOCKET_H_ > + > +#include > + > +/** > + * Remove device from socket device list. If no device is left on the so= cket, > + * remove the socket as well. > + * > + * @param pmd > + * device internals > + */ > +void memif_socket_remove_device(struct rte_eth_dev *dev); > + > +/** > + * Enqueue disconnect message to control channel message queue. > + * > + * @param cc > + * control channel > + * @param reason > + * const string stating disconnect reason (96 characters) > + * @param err_code > + * error code > + */ > +void memif_msg_enq_disconnect(struct memif_control_channel *cc, const ch= ar *reason, > + int err_code); > + > +/** > + * Initialize memif socket for specified device. If socket doesn't exist= , create socket. > + * > + * @param dev > + * memif ethernet device > + * @param socket_filename > + * socket filename > + * @return > + * - On success, zero. > + * - On failure, a negative value. > + */ > +int memif_socket_init(struct rte_eth_dev *dev, const char *socket_filena= me); > + > +/** > + * Disconnect memif device. Close control channel and shared memory. > + * > + * @param dev > + * ethernet device > + */ > +void memif_disconnect(struct rte_eth_dev *dev); > + > +/** > + * If device is properly configured, enable connection establishment. > + * > + * @param dev > + * memif ethernet device > + * @return > + * - On success, zero. > + * - On failure, a negative value. > + */ > +int memif_connect_master(struct rte_eth_dev *dev); > + > +/** > + * If device is properly configured, send connection request. > + * > + * @param dev > + * memif ethernet device > + * @return > + * - On success, zero. > + * - On failure, a negative value. > + */ > +int memif_connect_slave(struct rte_eth_dev *dev); > + > +struct memif_socket_dev_list_elt { > + TAILQ_ENTRY(memif_socket_dev_list_elt) next; > + struct rte_eth_dev *dev; /**< pointer to device internals */ > + char dev_name[RTE_ETH_NAME_MAX_LEN]; > +}; > + > +#define MEMIF_SOCKET_HASH_NAME "memif-sh" > +struct memif_socket { > + struct rte_intr_handle intr_handle; /**< interrupt handle */ > + char filename[256]; /**< socket filename */ > + > + TAILQ_HEAD(, memif_socket_dev_list_elt) dev_queue; > + /**< Queue of devices using this socket */ > + uint8_t listener; /**< if not zero socket is listener */ > +}; > + > +/* Control message queue. */ > +struct memif_msg_queue_elt { > + memif_msg_t msg; /**< control message */ > + TAILQ_ENTRY(memif_msg_queue_elt) next; > + int fd; /**< fd to be sent to peer */ > +}; > + > +struct memif_control_channel { > + struct rte_intr_handle intr_handle; /**< interrupt handle */ > + TAILQ_HEAD(, memif_msg_queue_elt) msg_queue; /**< control message queue= */ > + struct memif_socket *socket; /**< pointer to socket */ > + struct rte_eth_dev *dev; /**< pointer to device */ > +}; > + > +#endif /* MEMIF_SOCKET_H */ > diff --git a/drivers/net/memif/meson.build b/drivers/net/memif/meson.build > new file mode 100644 > index 000000000..287a30ef2 > --- /dev/null > +++ b/drivers/net/memif/meson.build > @@ -0,0 +1,15 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright 2018-2019 Cisco Systems, Inc. All rights reserved. > + > +if host_machine.system() !=3D 'linux' > + build =3D false > +endif > + > +sources =3D files('rte_eth_memif.c', > + 'memif_socket.c') > + > +allow_experimental_apis =3D true > +# Experimantal APIs: > +# - rte_intr_callback_unregister_pending > + > +deps +=3D ['hash'] > diff --git a/drivers/net/memif/rte_eth_memif.c b/drivers/net/memif/rte_et= h_memif.c > new file mode 100644 > index 000000000..3b9c1ce66 > --- /dev/null > +++ b/drivers/net/memif/rte_eth_memif.c > @@ -0,0 +1,1206 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright 2018-2019 Cisco Systems, Inc. All rights reserved. > + */ > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +#include "rte_eth_memif.h" > +#include "memif_socket.h" > + > +#define ETH_MEMIF_ID_ARG "id" > +#define ETH_MEMIF_ROLE_ARG "role" > +#define ETH_MEMIF_PKT_BUFFER_SIZE_ARG "bsize" > +#define ETH_MEMIF_RING_SIZE_ARG "rsize" > +#define ETH_MEMIF_SOCKET_ARG "socket" > +#define ETH_MEMIF_MAC_ARG "mac" > +#define ETH_MEMIF_ZC_ARG "zero-copy" > +#define ETH_MEMIF_SECRET_ARG "secret" > + > +static const char *valid_arguments[] =3D { > + ETH_MEMIF_ID_ARG, > + ETH_MEMIF_ROLE_ARG, > + ETH_MEMIF_PKT_BUFFER_SIZE_ARG, > + ETH_MEMIF_RING_SIZE_ARG, > + ETH_MEMIF_SOCKET_ARG, > + ETH_MEMIF_MAC_ARG, > + ETH_MEMIF_ZC_ARG, > + ETH_MEMIF_SECRET_ARG, > + NULL > +}; > + > +const char * > +memif_version(void) > +{ > + return ("memif-" RTE_STR(MEMIF_VERSION_MAJOR) "." RTE_STR(MEMIF_VERSION= _MINOR)); > +} > + > +static void > +memif_dev_info(struct rte_eth_dev *dev __rte_unused, struct rte_eth_dev_= info *dev_info) > +{ > + dev_info->max_mac_addrs =3D 1; > + dev_info->max_rx_pktlen =3D (uint32_t)ETH_FRAME_LEN; > + dev_info->max_rx_queues =3D ETH_MEMIF_MAX_NUM_Q_PAIRS; > + dev_info->max_tx_queues =3D ETH_MEMIF_MAX_NUM_Q_PAIRS; > + dev_info->min_rx_bufsize =3D 0; > +} > + > +static memif_ring_t * > +memif_get_ring(struct pmd_internals *pmd, memif_ring_type_t type, uint16= _t ring_num) > +{ > + /* rings only in region 0 */ > + void *p =3D pmd->regions[0]->addr; > + int ring_size =3D sizeof(memif_ring_t) + sizeof(memif_desc_t) * > + (1 << pmd->run.log2_ring_size); > + > + p =3D (uint8_t *)p + (ring_num + type * pmd->run.num_s2m_rings) * ring_= size; > + > + return (memif_ring_t *)p; > +} > + > +static void * > +memif_get_buffer(struct pmd_internals *pmd, memif_desc_t *d) > +{ > + return ((uint8_t *)pmd->regions[d->region]->addr + d->offset); > +} > + > +static int > +memif_pktmbuf_chain(struct rte_mbuf *head, struct rte_mbuf *cur_tail, > + struct rte_mbuf *tail) > +{ > + /* Check for number-of-segments-overflow */ > + if (unlikely(head->nb_segs + tail->nb_segs > RTE_MBUF_MAX_NB_SEGS)) > + return -EOVERFLOW; > + > + /* Chain 'tail' onto the old tail */ > + cur_tail->next =3D tail; > + > + /* accumulate number of segments and total length. */ > + head->nb_segs =3D (uint16_t)(head->nb_segs + tail->nb_segs); > + > + tail->pkt_len =3D tail->data_len; > + head->pkt_len +=3D tail->pkt_len; > + > + return 0; > +} > + > +static uint16_t > +eth_memif_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) > +{ > + struct memif_queue *mq =3D queue; > + struct pmd_internals *pmd =3D mq->pmd; > + memif_ring_t *ring =3D mq->ring; > + uint16_t cur_slot, last_slot, n_slots, ring_size, mask, s0; > + uint16_t n_rx_pkts =3D 0; > + uint16_t mbuf_size =3D rte_pktmbuf_data_room_size(mq->mempool) - > + RTE_PKTMBUF_HEADROOM; > + uint16_t src_len, src_off, dst_len, dst_off, cp_len; > + memif_ring_type_t type =3D mq->type; > + memif_desc_t *d0; > + struct rte_mbuf *mbuf, *mbuf_head, *mbuf_tail; > + uint64_t b; > + ssize_t size __rte_unused; > + uint16_t head; > + int ret; > + > + if (unlikely((pmd->flags & ETH_MEMIF_FLAG_CONNECTED) =3D=3D 0)) > + return 0; > + if (unlikely(ring =3D=3D NULL)) > + return 0; > + > + /* consume interrupt */ > + if ((ring->flags & MEMIF_RING_FLAG_MASK_INT) =3D=3D 0) > + size =3D read(mq->intr_handle.fd, &b, sizeof(b)); > + > + ring_size =3D 1 << mq->log2_ring_size; > + mask =3D ring_size - 1; > + > + cur_slot =3D (type =3D=3D MEMIF_RING_S2M) ? mq->last_head : mq->last_ta= il; > + last_slot =3D (type =3D=3D MEMIF_RING_S2M) ? ring->head : ring->tail; > + if (cur_slot =3D=3D last_slot) > + goto refill; > + n_slots =3D last_slot - cur_slot; > + > + while (n_slots && n_rx_pkts < nb_pkts) { > + mbuf_head =3D rte_pktmbuf_alloc(mq->mempool); > + if (unlikely(mbuf_head =3D=3D NULL)) > + goto no_free_bufs; > + mbuf =3D mbuf_head; > + mbuf->port =3D mq->in_port; > + > +next_slot: > + s0 =3D cur_slot & mask; > + d0 =3D &ring->desc[s0]; > + > + src_len =3D d0->length; > + dst_off =3D 0; > + src_off =3D 0; > + > + do { > + dst_len =3D mbuf_size - dst_off; > + if (dst_len =3D=3D 0) { > + dst_off =3D 0; > + dst_len =3D mbuf_size; > + > + /* store pointer to tail */ > + mbuf_tail =3D mbuf; > + mbuf =3D rte_pktmbuf_alloc(mq->mempool); > + if (unlikely(mbuf =3D=3D NULL)) > + goto no_free_bufs; > + mbuf->port =3D mq->in_port; > + ret =3D memif_pktmbuf_chain(mbuf_head, mbuf_tail, mbuf); > + if (unlikely(ret < 0)) { > + MIF_LOG(ERR, "%s: number-of-segments-overflow", > + rte_vdev_device_name(pmd->vdev)); > + rte_pktmbuf_free(mbuf); > + goto no_free_bufs; > + } > + } > + cp_len =3D RTE_MIN(dst_len, src_len); > + > + rte_pktmbuf_data_len(mbuf) +=3D cp_len; > + rte_pktmbuf_pkt_len(mbuf) =3D rte_pktmbuf_data_len(mbuf); > + if (mbuf !=3D mbuf_head) > + rte_pktmbuf_pkt_len(mbuf_head) +=3D cp_len; > + > + memcpy(rte_pktmbuf_mtod_offset(mbuf, void *, dst_off), > + (uint8_t *)memif_get_buffer(pmd, d0) + src_off, cp_len); > + > + src_off +=3D cp_len; > + dst_off +=3D cp_len; > + src_len -=3D cp_len; > + } while (src_len); > + > + cur_slot++; > + n_slots--; > + > + if (d0->flags & MEMIF_DESC_FLAG_NEXT) > + goto next_slot; > + > + mq->n_bytes +=3D rte_pktmbuf_pkt_len(mbuf_head); > + *bufs++ =3D mbuf_head; > + n_rx_pkts++; > + } > + > +no_free_bufs: > + if (type =3D=3D MEMIF_RING_S2M) { > + rte_mb(); > + ring->tail =3D cur_slot; > + mq->last_head =3D cur_slot; > + } else { > + mq->last_tail =3D cur_slot; > + } > + > +refill: > + if (type =3D=3D MEMIF_RING_M2S) { > + head =3D ring->head; > + n_slots =3D ring_size - head + mq->last_tail; > + > + while (n_slots--) { > + s0 =3D head++ & mask; > + d0 =3D &ring->desc[s0]; > + d0->length =3D pmd->run.pkt_buffer_size; > + } > + rte_mb(); > + ring->head =3D head; > + } > + > + mq->n_pkts +=3D n_rx_pkts; > + return n_rx_pkts; > +} > + > +static uint16_t > +eth_memif_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) > +{ > + struct memif_queue *mq =3D queue; > + struct pmd_internals *pmd =3D mq->pmd; > + memif_ring_t *ring =3D mq->ring; > + uint16_t slot, saved_slot, n_free, ring_size, mask, n_tx_pkts =3D 0; > + uint16_t src_len, src_off, dst_len, dst_off, cp_len; > + memif_ring_type_t type =3D mq->type; > + memif_desc_t *d0; > + struct rte_mbuf *mbuf; > + struct rte_mbuf *mbuf_head; > + uint64_t a; > + ssize_t size; > + > + if (unlikely((pmd->flags & ETH_MEMIF_FLAG_CONNECTED) =3D=3D 0)) > + return 0; > + if (unlikely(ring =3D=3D NULL)) > + return 0; > + > + ring_size =3D 1 << mq->log2_ring_size; > + mask =3D ring_size - 1; > + > + n_free =3D ring->tail - mq->last_tail; > + mq->last_tail +=3D n_free; > + slot =3D (type =3D=3D MEMIF_RING_S2M) ? ring->head : ring->tail; > + > + if (type =3D=3D MEMIF_RING_S2M) > + n_free =3D ring_size - ring->head + mq->last_tail; > + else > + n_free =3D ring->head - ring->tail; > + > + while (n_tx_pkts < nb_pkts && n_free) { > + mbuf_head =3D *bufs++; > + mbuf =3D mbuf_head; > + > + saved_slot =3D slot; > + d0 =3D &ring->desc[slot & mask]; > + dst_off =3D 0; > + dst_len =3D (type =3D=3D MEMIF_RING_S2M) ? > + pmd->run.pkt_buffer_size : d0->length; > + > +next_in_chain: > + src_off =3D 0; > + src_len =3D rte_pktmbuf_data_len(mbuf); > + > + while (src_len) { > + if (dst_len =3D=3D 0) { > + if (n_free) { > + slot++; > + n_free--; > + d0->flags |=3D MEMIF_DESC_FLAG_NEXT; > + d0 =3D &ring->desc[slot & mask]; > + dst_off =3D 0; > + dst_len =3D (type =3D=3D MEMIF_RING_S2M) ? > + pmd->run.pkt_buffer_size : d0->length; > + d0->flags =3D 0; > + } else { > + slot =3D saved_slot; > + goto no_free_slots; > + } > + } > + cp_len =3D RTE_MIN(dst_len, src_len); > + > + memcpy((uint8_t *)memif_get_buffer(pmd, d0) + dst_off, > + rte_pktmbuf_mtod_offset(mbuf, void *, src_off), > + cp_len); > + > + mq->n_bytes +=3D cp_len; > + src_off +=3D cp_len; > + dst_off +=3D cp_len; > + src_len -=3D cp_len; > + dst_len -=3D cp_len; > + > + d0->length =3D dst_off; > + } > + > + if (rte_pktmbuf_is_contiguous(mbuf) =3D=3D 0) { > + mbuf =3D mbuf->next; > + goto next_in_chain; > + } > + > + n_tx_pkts++; > + slot++; > + n_free--; > + rte_pktmbuf_free(mbuf_head); > + } > + > +no_free_slots: > + rte_mb(); > + if (type =3D=3D MEMIF_RING_S2M) > + ring->head =3D slot; > + else > + ring->tail =3D slot; > + > + if ((ring->flags & MEMIF_RING_FLAG_MASK_INT) =3D=3D 0) { > + a =3D 1; > + size =3D write(mq->intr_handle.fd, &a, sizeof(a)); > + if (unlikely(size < 0)) { > + MIF_LOG(WARNING, > + "%s: Failed to send interrupt. %s", > + rte_vdev_device_name(pmd->vdev), strerror(errno)); > + } > + } > + > + mq->n_err +=3D nb_pkts - n_tx_pkts; > + mq->n_pkts +=3D n_tx_pkts; > + return n_tx_pkts; > +} > + > +void > +memif_free_regions(struct pmd_internals *pmd) > +{ > + int i; > + struct memif_region *r; > + > + /* regions are allocated contiguously, so it's > + * enough to loop until 'pmd->regions_num' > + */ > + for (i =3D 0; i < pmd->regions_num; i++) { > + r =3D pmd->regions[i]; > + if (r !=3D NULL) { > + if (r->addr !=3D NULL) { > + munmap(r->addr, r->region_size); > + if (r->fd > 0) { > + close(r->fd); > + r->fd =3D -1; > + } > + } > + rte_free(r); > + pmd->regions[i] =3D NULL; > + } > + } > + pmd->regions_num =3D 0; > +} > + > +static int > +memif_region_init_shm(struct pmd_internals *pmd, uint8_t has_buffers) > +{ > + char shm_name[ETH_MEMIF_SHM_NAME_SIZE]; > + int ret =3D 0; > + struct memif_region *r; > + > + if (pmd->regions_num >=3D ETH_MEMIF_MAX_REGION_NUM) { > + MIF_LOG(ERR, "%s: Too many regions.", rte_vdev_device_name(pmd->vdev)); > + return -1; > + } > + > + r =3D rte_zmalloc("region", sizeof(struct memif_region), 0); > + if (r =3D=3D NULL) { > + MIF_LOG(ERR, "%s: Failed to alloc memif region.", > + rte_vdev_device_name(pmd->vdev)); > + return -ENOMEM; > + } > + > + /* calculate buffer offset */ > + r->pkt_buffer_offset =3D (pmd->run.num_s2m_rings + pmd->run.num_m2s_rin= gs) * > + (sizeof(memif_ring_t) + sizeof(memif_desc_t) * > + (1 << pmd->run.log2_ring_size)); > + > + r->region_size =3D r->pkt_buffer_offset; > + /* if region has buffers, add buffers size to region_size */ > + if (has_buffers =3D=3D 1) > + r->region_size +=3D (uint32_t)(pmd->run.pkt_buffer_size * > + (1 << pmd->run.log2_ring_size) * > + (pmd->run.num_s2m_rings + > + pmd->run.num_m2s_rings)); > + > + memset(shm_name, 0, sizeof(char) * ETH_MEMIF_SHM_NAME_SIZE); > + snprintf(shm_name, ETH_MEMIF_SHM_NAME_SIZE, "memif_region_%d", > + pmd->regions_num); > + > + r->fd =3D memfd_create(shm_name, MFD_ALLOW_SEALING); > + if (r->fd < 0) { > + MIF_LOG(ERR, "%s: Failed to create shm file: %s.", > + rte_vdev_device_name(pmd->vdev), > + strerror(errno)); > + ret =3D -1; > + goto error; > + } > + > + ret =3D fcntl(r->fd, F_ADD_SEALS, F_SEAL_SHRINK); > + if (ret < 0) { > + MIF_LOG(ERR, "%s: Failed to add seals to shm file: %s.", > + rte_vdev_device_name(pmd->vdev), > + strerror(errno)); > + goto error; > + } > + > + ret =3D ftruncate(r->fd, r->region_size); > + if (ret < 0) { > + MIF_LOG(ERR, "%s: Failed to truncate shm file: %s.", > + rte_vdev_device_name(pmd->vdev), > + strerror(errno)); > + goto error; > + } > + > + r->addr =3D mmap(NULL, r->region_size, PROT_READ | > + PROT_WRITE, MAP_SHARED, r->fd, 0); > + if (r->addr =3D=3D MAP_FAILED) { > + MIF_LOG(ERR, "%s: Failed to mmap shm region: %s.", > + rte_vdev_device_name(pmd->vdev), > + strerror(ret)); > + ret =3D -1; > + goto error; > + } > + > + pmd->regions[pmd->regions_num] =3D r; > + pmd->regions_num++; > + > + return ret; > + > +error: > + if (r->fd > 0) > + close(r->fd); > + r->fd =3D -1; > + > + return ret; > +} > + > +static int > +memif_regions_init(struct pmd_internals *pmd) > +{ > + int ret; > + > + /* create one buffer region */ > + ret =3D memif_region_init_shm(pmd, /* has buffer */ 1); > + if (ret < 0) > + return ret; > + > + return 0; > +} > + > +static void > +memif_init_rings(struct rte_eth_dev *dev) > +{ > + struct pmd_internals *pmd =3D dev->data->dev_private; > + memif_ring_t *ring; > + int i, j; > + uint16_t slot; > + > + for (i =3D 0; i < pmd->run.num_s2m_rings; i++) { > + ring =3D memif_get_ring(pmd, MEMIF_RING_S2M, i); > + ring->head =3D 0; > + ring->tail =3D 0; > + ring->cookie =3D MEMIF_COOKIE; > + ring->flags =3D 0; > + for (j =3D 0; j < (1 << pmd->run.log2_ring_size); j++) { > + slot =3D i * (1 << pmd->run.log2_ring_size) + j; > + ring->desc[j].region =3D 0; > + ring->desc[j].offset =3D pmd->regions[0]->pkt_buffer_offset + > + (uint32_t)(slot * pmd->run.pkt_buffer_size); > + ring->desc[j].length =3D pmd->run.pkt_buffer_size; > + } > + } > + > + for (i =3D 0; i < pmd->run.num_m2s_rings; i++) { > + ring =3D memif_get_ring(pmd, MEMIF_RING_M2S, i); > + ring->head =3D 0; > + ring->tail =3D 0; > + ring->cookie =3D MEMIF_COOKIE; > + ring->flags =3D 0; > + for (j =3D 0; j < (1 << pmd->run.log2_ring_size); j++) { > + slot =3D (i + pmd->run.num_s2m_rings) * > + (1 << pmd->run.log2_ring_size) + j; > + ring->desc[j].region =3D 0; > + ring->desc[j].offset =3D pmd->regions[0]->pkt_buffer_offset + > + (uint32_t)(slot * pmd->run.pkt_buffer_size); > + ring->desc[j].length =3D pmd->run.pkt_buffer_size; > + } > + } > +} > + > +/* called only by slave */ > +static void > +memif_init_queues(struct rte_eth_dev *dev) > +{ > + struct pmd_internals *pmd =3D dev->data->dev_private; > + struct memif_queue *mq; > + int i; > + > + for (i =3D 0; i < pmd->run.num_s2m_rings; i++) { > + mq =3D dev->data->tx_queues[i]; > + mq->ring =3D memif_get_ring(pmd, MEMIF_RING_S2M, i); > + mq->log2_ring_size =3D pmd->run.log2_ring_size; > + /* queues located only in region 0 */ > + mq->region =3D 0; > + mq->ring_offset =3D (uint8_t *)mq->ring - (uint8_t *)pmd->regions[0]->= addr; > + mq->last_head =3D 0; > + mq->last_tail =3D 0; > + mq->intr_handle.fd =3D eventfd(0, EFD_NONBLOCK); > + if (mq->intr_handle.fd < 0) { > + MIF_LOG(WARNING, > + "%s: Failed to create eventfd for tx queue %d: %s.", > + rte_vdev_device_name(pmd->vdev), i, > + strerror(errno)); > + } > + } > + > + for (i =3D 0; i < pmd->run.num_m2s_rings; i++) { > + mq =3D dev->data->rx_queues[i]; > + mq->ring =3D memif_get_ring(pmd, MEMIF_RING_M2S, i); > + mq->log2_ring_size =3D pmd->run.log2_ring_size; > + /* queues located only in region 0 */ > + mq->region =3D 0; > + mq->ring_offset =3D (uint8_t *)mq->ring - (uint8_t *)pmd->regions[0]->= addr; > + mq->last_head =3D 0; > + mq->last_tail =3D 0; > + mq->intr_handle.fd =3D eventfd(0, EFD_NONBLOCK); > + if (mq->intr_handle.fd < 0) { > + MIF_LOG(WARNING, > + "%s: Failed to create eventfd for rx queue %d: %s.", > + rte_vdev_device_name(pmd->vdev), i, > + strerror(errno)); > + } > + } > +} > + > +int > +memif_init_regions_and_queues(struct rte_eth_dev *dev) > +{ > + int ret; > + > + ret =3D memif_regions_init(dev->data->dev_private); > + if (ret < 0) > + return ret; > + > + memif_init_rings(dev); > + > + memif_init_queues(dev); > + > + return 0; > +} > + > +int > +memif_connect(struct rte_eth_dev *dev) > +{ > + struct pmd_internals *pmd =3D dev->data->dev_private; > + struct memif_region *mr; > + struct memif_queue *mq; > + int i; > + > + for (i =3D 0; i < pmd->regions_num; i++) { > + mr =3D pmd->regions[i]; > + if (mr !=3D NULL) { > + if (mr->addr =3D=3D NULL) { > + if (mr->fd < 0) > + return -1; > + mr->addr =3D mmap(NULL, mr->region_size, > + PROT_READ | PROT_WRITE, > + MAP_SHARED, mr->fd, 0); > + if (mr->addr =3D=3D NULL) > + return -1; > + } > + } > + } > + > + for (i =3D 0; i < pmd->run.num_s2m_rings; i++) { > + mq =3D (pmd->role =3D=3D MEMIF_ROLE_SLAVE) ? > + dev->data->tx_queues[i] : dev->data->rx_queues[i]; > + mq->ring =3D (memif_ring_t *)((uint8_t *)pmd->regions[mq->region]->add= r + > + mq->ring_offset); > + if (mq->ring->cookie !=3D MEMIF_COOKIE) { > + MIF_LOG(ERR, "%s: Wrong cookie", > + rte_vdev_device_name(pmd->vdev)); > + return -1; > + } > + mq->ring->head =3D 0; > + mq->ring->tail =3D 0; > + mq->last_head =3D 0; > + mq->last_tail =3D 0; > + /* enable polling mode */ > + if (pmd->role =3D=3D MEMIF_ROLE_MASTER) > + mq->ring->flags =3D MEMIF_RING_FLAG_MASK_INT; > + } > + for (i =3D 0; i < pmd->run.num_m2s_rings; i++) { > + mq =3D (pmd->role =3D=3D MEMIF_ROLE_SLAVE) ? > + dev->data->rx_queues[i] : dev->data->tx_queues[i]; > + mq->ring =3D (memif_ring_t *)((uint8_t *)pmd->regions[mq->region]->add= r + > + mq->ring_offset); > + if (mq->ring->cookie !=3D MEMIF_COOKIE) { > + MIF_LOG(ERR, "%s: Wrong cookie", > + rte_vdev_device_name(pmd->vdev)); > + return -1; > + } > + mq->ring->head =3D 0; > + mq->ring->tail =3D 0; > + mq->last_head =3D 0; > + mq->last_tail =3D 0; > + /* enable polling mode */ > + if (pmd->role =3D=3D MEMIF_ROLE_SLAVE) > + mq->ring->flags =3D MEMIF_RING_FLAG_MASK_INT; > + } > + > + pmd->flags &=3D ~ETH_MEMIF_FLAG_CONNECTING; > + pmd->flags |=3D ETH_MEMIF_FLAG_CONNECTED; > + dev->data->dev_link.link_status =3D ETH_LINK_UP; > + MIF_LOG(INFO, "%s: Connected.", rte_vdev_device_name(pmd->vdev)); > + return 0; > +} > + > +static int > +memif_dev_start(struct rte_eth_dev *dev) > +{ > + struct pmd_internals *pmd =3D dev->data->dev_private; > + int ret =3D 0; > + > + switch (pmd->role) { > + case MEMIF_ROLE_SLAVE: > + ret =3D memif_connect_slave(dev); > + break; > + case MEMIF_ROLE_MASTER: > + ret =3D memif_connect_master(dev); > + break; > + default: > + MIF_LOG(ERR, "%s: Unknown role: %d.", > + rte_vdev_device_name(pmd->vdev), pmd->role); > + ret =3D -1; > + break; > + } > + > + return ret; > +} > + > +static void > +memif_dev_close(struct rte_eth_dev *dev) > +{ > + struct pmd_internals *pmd =3D dev->data->dev_private; > + > + memif_msg_enq_disconnect(pmd->cc, "Device closed", 0); > + memif_disconnect(dev); > + > + memif_socket_remove_device(dev); > +} > + > +static int > +memif_dev_configure(struct rte_eth_dev *dev) > +{ > + struct pmd_internals *pmd =3D dev->data->dev_private; > + > + /* > + * SLAVE - TXQ > + * MASTER - RXQ > + */ > + pmd->cfg.num_s2m_rings =3D (pmd->role =3D=3D MEMIF_ROLE_SLAVE) ? > + dev->data->nb_tx_queues : dev->data->nb_rx_queues; > + > + /* > + * SLAVE - RXQ > + * MASTER - TXQ > + */ > + pmd->cfg.num_m2s_rings =3D (pmd->role =3D=3D MEMIF_ROLE_SLAVE) ? > + dev->data->nb_rx_queues : dev->data->nb_tx_queues; > + > + return 0; > +} > + > +static int > +memif_tx_queue_setup(struct rte_eth_dev *dev, > + uint16_t qid, > + uint16_t nb_tx_desc __rte_unused, > + unsigned int socket_id __rte_unused, > + const struct rte_eth_txconf *tx_conf __rte_unused) > +{ > + struct pmd_internals *pmd =3D dev->data->dev_private; > + struct memif_queue *mq; > + > + mq =3D rte_zmalloc("tx-queue", sizeof(struct memif_queue), 0); > + if (mq =3D=3D NULL) { > + MIF_LOG(ERR, "%s: Failed to allocate tx queue id: %u", > + rte_vdev_device_name(pmd->vdev), qid); > + return -ENOMEM; > + } > + > + mq->type =3D > + (pmd->role =3D=3D MEMIF_ROLE_SLAVE) ? MEMIF_RING_S2M : MEMIF_RING_M= 2S; > + mq->n_pkts =3D 0; > + mq->n_bytes =3D 0; > + mq->n_err =3D 0; > + mq->intr_handle.fd =3D -1; > + mq->intr_handle.type =3D RTE_INTR_HANDLE_EXT; > + mq->pmd =3D pmd; > + dev->data->tx_queues[qid] =3D mq; > + > + return 0; > +} > + > +static int > +memif_rx_queue_setup(struct rte_eth_dev *dev, > + uint16_t qid, > + uint16_t nb_rx_desc __rte_unused, > + unsigned int socket_id __rte_unused, > + const struct rte_eth_rxconf *rx_conf __rte_unused, > + struct rte_mempool *mb_pool) > +{ > + struct pmd_internals *pmd =3D dev->data->dev_private; > + struct memif_queue *mq; > + > + mq =3D rte_zmalloc("rx-queue", sizeof(struct memif_queue), 0); > + if (mq =3D=3D NULL) { > + MIF_LOG(ERR, "%s: Failed to allocate rx queue id: %u", > + rte_vdev_device_name(pmd->vdev), qid); > + return -ENOMEM; > + } > + > + mq->type =3D (pmd->role =3D=3D MEMIF_ROLE_SLAVE) ? MEMIF_RING_M2S : MEM= IF_RING_S2M; > + mq->n_pkts =3D 0; > + mq->n_bytes =3D 0; > + mq->n_err =3D 0; > + mq->intr_handle.fd =3D -1; > + mq->intr_handle.type =3D RTE_INTR_HANDLE_EXT; > + mq->mempool =3D mb_pool; > + mq->in_port =3D dev->data->port_id; > + mq->pmd =3D pmd; > + dev->data->rx_queues[qid] =3D mq; > + > + return 0; > +} > + > +static void > +memif_queue_release(void *queue) > +{ > + struct memif_queue *mq =3D (struct memif_queue *)queue; > + > + if (!mq) > + return; > + > + rte_free(mq); > +} > + > +static int > +memif_link_update(struct rte_eth_dev *dev __rte_unused, > + int wait_to_complete __rte_unused) > +{ > + return 0; > +} > + > +static int > +memif_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) > +{ > + struct pmd_internals *pmd =3D dev->data->dev_private; > + struct memif_queue *mq; > + int i; > + uint8_t tmp, nq; > + > + stats->ipackets =3D 0; > + stats->ibytes =3D 0; > + stats->opackets =3D 0; > + stats->obytes =3D 0; > + stats->oerrors =3D 0; > + > + tmp =3D (pmd->role =3D=3D MEMIF_ROLE_SLAVE) ? pmd->run.num_s2m_rings : > + pmd->run.num_m2s_rings; > + nq =3D (tmp < RTE_ETHDEV_QUEUE_STAT_CNTRS) ? tmp : > + RTE_ETHDEV_QUEUE_STAT_CNTRS; > + > + /* RX stats */ > + for (i =3D 0; i < nq; i++) { > + mq =3D dev->data->rx_queues[i]; > + stats->q_ipackets[i] =3D mq->n_pkts; > + stats->q_ibytes[i] =3D mq->n_bytes; > + stats->ipackets +=3D mq->n_pkts; > + stats->ibytes +=3D mq->n_bytes; > + } > + > + tmp =3D (pmd->role =3D=3D MEMIF_ROLE_SLAVE) ? pmd->run.num_m2s_rings : > + pmd->run.num_s2m_rings; > + nq =3D (tmp < RTE_ETHDEV_QUEUE_STAT_CNTRS) ? tmp : > + RTE_ETHDEV_QUEUE_STAT_CNTRS; > + > + /* TX stats */ > + for (i =3D 0; i < nq; i++) { > + mq =3D dev->data->tx_queues[i]; > + stats->q_opackets[i] =3D mq->n_pkts; > + stats->q_obytes[i] =3D mq->n_bytes; > + stats->opackets +=3D mq->n_pkts; > + stats->obytes +=3D mq->n_bytes; > + stats->oerrors +=3D mq->n_err; > + } > + return 0; > +} > + > +static void > +memif_stats_reset(struct rte_eth_dev *dev) > +{ > + struct pmd_internals *pmd =3D dev->data->dev_private; > + int i; > + struct memif_queue *mq; > + > + for (i =3D 0; i < pmd->run.num_s2m_rings; i++) { > + mq =3D (pmd->role =3D=3D MEMIF_ROLE_SLAVE) ? dev->data->tx_queues[i] : > + dev->data->rx_queues[i]; > + mq->n_pkts =3D 0; > + mq->n_bytes =3D 0; > + mq->n_err =3D 0; > + } > + for (i =3D 0; i < pmd->run.num_m2s_rings; i++) { > + mq =3D (pmd->role =3D=3D MEMIF_ROLE_SLAVE) ? dev->data->rx_queues[i] : > + dev->data->tx_queues[i]; > + mq->n_pkts =3D 0; > + mq->n_bytes =3D 0; > + mq->n_err =3D 0; > + } > +} > + > +static int > +memif_rx_queue_intr_enable(struct rte_eth_dev *dev, uint16_t qid __rte_u= nused) > +{ > + struct pmd_internals *pmd =3D dev->data->dev_private; > + > + MIF_LOG(WARNING, "%s: Interrupt mode not supported.", > + rte_vdev_device_name(pmd->vdev)); > + > + return -1; > +} > + > +static int > +memif_rx_queue_intr_disable(struct rte_eth_dev *dev, uint16_t qid __rte_= unused) > +{ > + struct pmd_internals *pmd __rte_unused =3D dev->data->dev_private; > + > + return 0; > +} > + > +static const struct eth_dev_ops ops =3D { > + .dev_start =3D memif_dev_start, > + .dev_close =3D memif_dev_close, > + .dev_infos_get =3D memif_dev_info, > + .dev_configure =3D memif_dev_configure, > + .tx_queue_setup =3D memif_tx_queue_setup, > + .rx_queue_setup =3D memif_rx_queue_setup, > + .rx_queue_release =3D memif_queue_release, > + .tx_queue_release =3D memif_queue_release, > + .rx_queue_intr_enable =3D memif_rx_queue_intr_enable, > + .rx_queue_intr_disable =3D memif_rx_queue_intr_disable, > + .link_update =3D memif_link_update, > + .stats_get =3D memif_stats_get, > + .stats_reset =3D memif_stats_reset, > +}; > + > +static int > +memif_create(struct rte_vdev_device *vdev, enum memif_role_t role, > + memif_interface_id_t id, uint32_t flags, > + const char *socket_filename, > + memif_log2_ring_size_t log2_ring_size, > + uint16_t pkt_buffer_size, const char *secret, > + struct rte_ether_addr *ether_addr) > +{ > + int ret =3D 0; > + struct rte_eth_dev *eth_dev; > + struct rte_eth_dev_data *data; > + struct pmd_internals *pmd; > + const unsigned int numa_node =3D vdev->device.numa_node; > + const char *name =3D rte_vdev_device_name(vdev); > + > + if (flags & ETH_MEMIF_FLAG_ZERO_COPY) { > + MIF_LOG(ERR, "Zero-copy slave not supported."); > + return -1; > + } > + > + eth_dev =3D rte_eth_vdev_allocate(vdev, sizeof(*pmd)); > + if (eth_dev =3D=3D NULL) { > + MIF_LOG(ERR, "%s: Unable to allocate device struct.", name); > + return -1; > + } > + > + pmd =3D eth_dev->data->dev_private; > + memset(pmd, 0, sizeof(*pmd)); > + > + pmd->vdev =3D vdev; > + pmd->id =3D id; > + pmd->flags =3D flags; > + pmd->flags |=3D ETH_MEMIF_FLAG_DISABLED; > + pmd->role =3D role; > + > + ret =3D memif_socket_init(eth_dev, socket_filename); > + if (ret < 0) > + return ret; > + > + memset(pmd->secret, 0, sizeof(char) * ETH_MEMIF_SECRET_SIZE); > + if (secret !=3D NULL) > + strlcpy(pmd->secret, secret, sizeof(pmd->secret)); > + > + pmd->cfg.log2_ring_size =3D log2_ring_size; > + /* set in .dev_configure() */ > + pmd->cfg.num_s2m_rings =3D 0; > + pmd->cfg.num_m2s_rings =3D 0; > + > + pmd->cfg.pkt_buffer_size =3D pkt_buffer_size; > + > + data =3D eth_dev->data; > + data->dev_private =3D pmd; > + data->numa_node =3D numa_node; > + data->mac_addrs =3D ether_addr; > + > + eth_dev->dev_ops =3D &ops; > + eth_dev->device =3D &vdev->device; > + eth_dev->rx_pkt_burst =3D eth_memif_rx; > + eth_dev->tx_pkt_burst =3D eth_memif_tx; > + > + eth_dev->data->dev_flags |=3D RTE_ETH_DEV_CLOSE_REMOVE; > + > + rte_eth_dev_probing_finish(eth_dev); > + > + return 0; > +} > + > +static int > +memif_set_role(const char *key __rte_unused, const char *value, > + void *extra_args) > +{ > + enum memif_role_t *role =3D (enum memif_role_t *)extra_args; > + > + if (strstr(value, "master") !=3D NULL) { > + *role =3D MEMIF_ROLE_MASTER; > + } else if (strstr(value, "slave") !=3D NULL) { > + *role =3D MEMIF_ROLE_SLAVE; > + } else { > + MIF_LOG(ERR, "Unknown role: %s.", value); > + return -EINVAL; > + } > + return 0; > +} > + > +static int > +memif_set_zc(const char *key __rte_unused, const char *value, void *extr= a_args) > +{ > + uint32_t *flags =3D (uint32_t *)extra_args; > + > + if (strstr(value, "yes") !=3D NULL) { > + *flags |=3D ETH_MEMIF_FLAG_ZERO_COPY; > + } else if (strstr(value, "no") !=3D NULL) { > + *flags &=3D ~ETH_MEMIF_FLAG_ZERO_COPY; > + } else { > + MIF_LOG(ERR, "Failed to parse zero-copy param: %s.", value); > + return -EINVAL; > + } > + return 0; > +} > + > +static int > +memif_set_id(const char *key __rte_unused, const char *value, void *extr= a_args) > +{ > + memif_interface_id_t *id =3D (memif_interface_id_t *)extra_args; > + > + /* even if parsing fails, 0 is a valid id */ > + *id =3D strtoul(value, NULL, 10); > + return 0; > +} > + > +static int > +memif_set_bs(const char *key __rte_unused, const char *value, void *extr= a_args) > +{ > + unsigned long tmp; > + uint16_t *pkt_buffer_size =3D (uint16_t *)extra_args; > + > + tmp =3D strtoul(value, NULL, 10); > + if (tmp =3D=3D 0 || tmp > 0xFFFF) { > + MIF_LOG(ERR, "Invalid buffer size: %s.", value); > + return -EINVAL; > + } > + *pkt_buffer_size =3D tmp; > + return 0; > +} > + > +static int > +memif_set_rs(const char *key __rte_unused, const char *value, void *extr= a_args) > +{ > + unsigned long tmp; > + memif_log2_ring_size_t *log2_ring_size =3D > + (memif_log2_ring_size_t *)extra_args; > + > + tmp =3D strtoul(value, NULL, 10); > + if (tmp =3D=3D 0 || tmp > ETH_MEMIF_MAX_LOG2_RING_SIZE) { > + MIF_LOG(ERR, "Invalid ring size: %s (max %u).", > + value, ETH_MEMIF_MAX_LOG2_RING_SIZE); > + return -EINVAL; > + } > + *log2_ring_size =3D tmp; > + return 0; > +} > + > +/* check if directory exists and if we have permission to read/write */ > +static int > +memif_check_socket_filename(const char *filename) > +{ > + char *dir =3D NULL, *tmp; > + uint32_t idx; > + int ret =3D 0; > + > + tmp =3D strrchr(filename, '/'); > + if (tmp !=3D NULL) { > + idx =3D tmp - filename; > + dir =3D rte_zmalloc("memif_tmp", sizeof(char) * (idx + 1), 0); > + if (dir =3D=3D NULL) { > + MIF_LOG(ERR, "Failed to allocate memory."); > + return -1; > + } > + strlcpy(dir, filename, sizeof(char) * (idx + 1)); > + } > + > + if (dir =3D=3D NULL || (faccessat(-1, dir, F_OK | R_OK | > + W_OK, AT_EACCESS) < 0)) { > + MIF_LOG(ERR, "Invalid socket directory."); > + ret =3D -EINVAL; > + } > + > + if (dir !=3D NULL) > + rte_free(dir); > + > + return ret; > +} > + > +static int > +memif_set_socket_filename(const char *key __rte_unused, const char *valu= e, > + void *extra_args) > +{ > + const char **socket_filename =3D (const char **)extra_args; > + > + *socket_filename =3D value; > + return memif_check_socket_filename(*socket_filename); > +} > + > +static int > +memif_set_mac(const char *key __rte_unused, const char *value, void *ext= ra_args) > +{ > + struct rte_ether_addr *ether_addr =3D (struct rte_ether_addr *)extra_ar= gs; > + int ret =3D 0; > + > + ret =3D sscanf(value, "%hhx:%hhx:%hhx:%hhx:%hhx:%hhx", > + ðer_addr->addr_bytes[0], ðer_addr->addr_bytes[1], > + ðer_addr->addr_bytes[2], ðer_addr->addr_bytes[3], > + ðer_addr->addr_bytes[4], ðer_addr->addr_bytes[5]); > + if (ret !=3D 6) > + MIF_LOG(WARNING, "Failed to parse mac '%s'.", value); > + return 0; > +} > + > +static int > +memif_set_secret(const char *key __rte_unused, const char *value, void *= extra_args) > +{ > + const char **secret =3D (const char **)extra_args; > + > + *secret =3D value; > + return 0; > +} > + > +static int > +rte_pmd_memif_probe(struct rte_vdev_device *vdev) > +{ > + RTE_BUILD_BUG_ON(sizeof(memif_msg_t) !=3D 128); > + RTE_BUILD_BUG_ON(sizeof(memif_desc_t) !=3D 16); > + int ret =3D 0; > + struct rte_kvargs *kvlist; > + const char *name =3D rte_vdev_device_name(vdev); > + enum memif_role_t role =3D MEMIF_ROLE_SLAVE; > + memif_interface_id_t id =3D 0; > + uint16_t pkt_buffer_size =3D ETH_MEMIF_DEFAULT_PKT_BUFFER_SIZE; > + memif_log2_ring_size_t log2_ring_size =3D ETH_MEMIF_DEFAULT_RING_SIZE; > + const char *socket_filename =3D ETH_MEMIF_DEFAULT_SOCKET_FILENAME; > + uint32_t flags =3D 0; > + const char *secret =3D NULL; > + struct rte_ether_addr *ether_addr =3D rte_zmalloc("", sizeof(struct rte= _ether_addr), 0); > + > + rte_eth_random_addr(ether_addr->addr_bytes); > + > + MIF_LOG(INFO, "Initialize MEMIF: %s.", name); > + > + if (rte_eal_process_type() =3D=3D RTE_PROC_SECONDARY) { > + MIF_LOG(ERR, "Multi-processing not supported for memif."); > + /* TODO: > + * Request connection information. > + * > + * Once memif in the primary process is connected, > + * broadcast connection information. > + */ > + return -1; > + } > + > + kvlist =3D rte_kvargs_parse(rte_vdev_device_args(vdev), valid_arguments= ); > + > + /* parse parameters */ > + if (kvlist !=3D NULL) { > + ret =3D rte_kvargs_process(kvlist, ETH_MEMIF_ROLE_ARG, > + &memif_set_role, &role); > + if (ret < 0) > + goto exit; > + ret =3D rte_kvargs_process(kvlist, ETH_MEMIF_ID_ARG, > + &memif_set_id, &id); > + if (ret < 0) > + goto exit; > + ret =3D rte_kvargs_process(kvlist, ETH_MEMIF_PKT_BUFFER_SIZE_ARG, > + &memif_set_bs, &pkt_buffer_size); > + if (ret < 0) > + goto exit; > + ret =3D rte_kvargs_process(kvlist, ETH_MEMIF_RING_SIZE_ARG, > + &memif_set_rs, &log2_ring_size); > + if (ret < 0) > + goto exit; > + ret =3D rte_kvargs_process(kvlist, ETH_MEMIF_SOCKET_ARG, > + &memif_set_socket_filename, > + (void *)(&socket_filename)); > + if (ret < 0) > + goto exit; > + ret =3D rte_kvargs_process(kvlist, ETH_MEMIF_MAC_ARG, > + &memif_set_mac, ether_addr); > + if (ret < 0) > + goto exit; > + ret =3D rte_kvargs_process(kvlist, ETH_MEMIF_ZC_ARG, > + &memif_set_zc, &flags); > + if (ret < 0) > + goto exit; > + ret =3D rte_kvargs_process(kvlist, ETH_MEMIF_SECRET_ARG, > + &memif_set_secret, (void *)(&secret)); > + if (ret < 0) > + goto exit; > + } > + > + /* create interface */ > + ret =3D memif_create(vdev, role, id, flags, socket_filename, > + log2_ring_size, pkt_buffer_size, secret, ether_addr); > + > +exit: > + if (kvlist !=3D NULL) > + rte_kvargs_free(kvlist); > + return ret; > +} > + > +static int > +rte_pmd_memif_remove(struct rte_vdev_device *vdev) > +{ > + struct rte_eth_dev *eth_dev; > + int i; > + > + eth_dev =3D rte_eth_dev_allocated(rte_vdev_device_name(vdev)); > + if (eth_dev =3D=3D NULL) > + return 0; > + > + for (i =3D 0; i < eth_dev->data->nb_rx_queues; i++) > + (*eth_dev->dev_ops->rx_queue_release)(eth_dev->data->rx_queues[i]); > + for (i =3D 0; i < eth_dev->data->nb_tx_queues; i++) > + (*eth_dev->dev_ops->rx_queue_release)(eth_dev->data->tx_queues[i]); > + > + rte_free(eth_dev->process_private); > + eth_dev->process_private =3D NULL; > + > + rte_eth_dev_close(eth_dev->data->port_id); > + > + return 0; > +} > + > +static struct rte_vdev_driver pmd_memif_drv =3D { > + .probe =3D rte_pmd_memif_probe, > + .remove =3D rte_pmd_memif_remove, > +}; > + > +RTE_PMD_REGISTER_VDEV(net_memif, pmd_memif_drv); > + > +RTE_PMD_REGISTER_PARAM_STRING(net_memif, > + ETH_MEMIF_ID_ARG "=3D" > + ETH_MEMIF_ROLE_ARG "=3Dmaster|slave" > + ETH_MEMIF_PKT_BUFFER_SIZE_ARG "=3D" > + ETH_MEMIF_RING_SIZE_ARG "=3D" > + ETH_MEMIF_SOCKET_ARG "=3D" > + ETH_MEMIF_MAC_ARG "=3Dxx:xx:xx:xx:xx:xx" > + ETH_MEMIF_ZC_ARG "=3Dyes|no" > + ETH_MEMIF_SECRET_ARG "=3D"); > + > +int memif_logtype; > + > +RTE_INIT(memif_init_log) > +{ > + memif_logtype =3D rte_log_register("pmd.net.memif"); > + if (memif_logtype >=3D 0) > + rte_log_set_level(memif_logtype, RTE_LOG_NOTICE); > +} > diff --git a/drivers/net/memif/rte_eth_memif.h b/drivers/net/memif/rte_et= h_memif.h > new file mode 100644 > index 000000000..8cee2643c > --- /dev/null > +++ b/drivers/net/memif/rte_eth_memif.h > @@ -0,0 +1,212 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright 2018-2019 Cisco Systems, Inc. All rights reserved. > + */ > + > +#ifndef _RTE_ETH_MEMIF_H_ > +#define _RTE_ETH_MEMIF_H_ > + > +#ifndef _GNU_SOURCE > +#define _GNU_SOURCE > +#endif /* GNU_SOURCE */ > + > +#include > + > +#include > +#include > +#include > + > +#include "memif.h" > + > +#define ETH_MEMIF_DEFAULT_SOCKET_FILENAME "/run/memif.sock" > +#define ETH_MEMIF_DEFAULT_RING_SIZE 10 > +#define ETH_MEMIF_DEFAULT_PKT_BUFFER_SIZE 2048 > + > +#define ETH_MEMIF_MAX_NUM_Q_PAIRS 255 > +#define ETH_MEMIF_MAX_LOG2_RING_SIZE 14 > +#define ETH_MEMIF_MAX_REGION_NUM 256 > + > +#define ETH_MEMIF_SHM_NAME_SIZE 32 > +#define ETH_MEMIF_DISC_STRING_SIZE 96 > +#define ETH_MEMIF_SECRET_SIZE 24 > + > +extern int memif_logtype; > + > +#define MIF_LOG(level, fmt, args...) \ > + rte_log(RTE_LOG_ ## level, memif_logtype, \ > + "%s(): " fmt "\n", __func__, ##args) > + > +enum memif_role_t { > + MEMIF_ROLE_MASTER, > + MEMIF_ROLE_SLAVE, > +}; > + > +struct memif_region { > + void *addr; /**< shared memory address */ > + memif_region_size_t region_size; /**< shared memory size */ > + int fd; /**< shared memory file descriptor */ > + uint32_t pkt_buffer_offset; > + /**< offset from 'addr' to first packet buffer */ > +}; > + > +struct memif_queue { > + struct rte_mempool *mempool; /**< mempool for RX packets */ > + struct pmd_internals *pmd; /**< device internals */ > + > + memif_ring_type_t type; /**< ring type */ > + memif_region_index_t region; /**< shared memory region index */ > + > + uint16_t in_port; /**< port id */ > + > + memif_region_offset_t ring_offset; > + /**< ring offset from start of shm region (ring - memif_region.addr) */ > + > + uint16_t last_head; /**< last ring head */ > + uint16_t last_tail; /**< last ring tail */ > + > + /* rx/tx info */ > + uint64_t n_pkts; /**< number of rx/tx packets */ > + uint64_t n_bytes; /**< number of rx/tx bytes */ > + uint64_t n_err; /**< number of tx errors */ > + > + memif_ring_t *ring; /**< pointer to ring */ > + > + struct rte_intr_handle intr_handle; /**< interrupt handle */ > + > + memif_log2_ring_size_t log2_ring_size; /**< log2 of ring size */ > +}; > + > +struct pmd_internals { > + memif_interface_id_t id; /**< unique id */ > + enum memif_role_t role; /**< device role */ > + uint32_t flags; /**< device status flags */ > +#define ETH_MEMIF_FLAG_CONNECTING (1 << 0) > +/**< device is connecting */ > +#define ETH_MEMIF_FLAG_CONNECTED (1 << 1) > +/**< device is connected */ > +#define ETH_MEMIF_FLAG_ZERO_COPY (1 << 2) > +/**< device is zero-copy enabled */ > +#define ETH_MEMIF_FLAG_DISABLED (1 << 3) > +/**< device has not been configured and can not accept connection reques= ts */ > + > + char *socket_filename; /**< pointer to socket filename */ > + char secret[ETH_MEMIF_SECRET_SIZE]; /**< secret (optional security para= meter) */ > + > + struct memif_control_channel *cc; /**< control channel */ > + > + struct memif_region *regions[ETH_MEMIF_MAX_REGION_NUM]; > + /**< shared memory regions */ > + memif_region_index_t regions_num; /**< number of regions */ > + > + /* remote info */ > + char remote_name[RTE_DEV_NAME_MAX_LEN]; /**< remote app name */ > + char remote_if_name[RTE_DEV_NAME_MAX_LEN]; /**< remote peer name */ > + > + struct { > + memif_log2_ring_size_t log2_ring_size; /**< log2 of ring size */ > + uint8_t num_s2m_rings; /**< number of slave to master rings */ > + uint8_t num_m2s_rings; /**< number of master to slave rings */ > + uint16_t pkt_buffer_size; /**< buffer size */ > + } cfg; /**< Configured parameters (max values) */ > + > + struct { > + memif_log2_ring_size_t log2_ring_size; /**< log2 of ring size */ > + uint8_t num_s2m_rings; /**< number of slave to master rings */ > + uint8_t num_m2s_rings; /**< number of master to slave rings */ > + uint16_t pkt_buffer_size; /**< buffer size */ > + } run; > + /**< Parameters used in active connection */ > + > + char local_disc_string[ETH_MEMIF_DISC_STRING_SIZE]; > + /**< local disconnect reason */ > + char remote_disc_string[ETH_MEMIF_DISC_STRING_SIZE]; > + /**< remote disconnect reason */ > + > + struct rte_vdev_device *vdev; /**< vdev handle */ > +}; > + > +/** > + * Unmap shared memory and free regions from memory. > + * > + * @param pmd > + * device internals > + */ > +void memif_free_regions(struct pmd_internals *pmd); > + > +/** > + * Finalize connection establishment process. Map shared memory file > + * (master role), initialize ring queue, set link status up. > + * > + * @param pmd > + * device internals > + * @return > + * - On success, zero. > + * - On failure, a negative value. > + */ > +int memif_connect(struct rte_eth_dev *dev); > + > +/** > + * Create shared memory file and initialize ring queue. > + * Only called by slave when establishing connection > + * > + * @param pmd > + * device internals > + * @return > + * - On success, zero. > + * - On failure, a negative value. > + */ > +int memif_init_regions_and_queues(struct rte_eth_dev *dev); > + > +/** > + * Get memif version string. > + * > + * @return > + * - memif version string > + */ > +const char *memif_version(void); > + > +#ifndef MFD_HUGETLB > +#ifndef __NR_memfd_create > + > +#if defined __x86_64__ > +#define __NR_memfd_create 319 > +#elif defined __x86_32__ > +#define __NR_memfd_create 1073742143 > +#elif defined __arm__ > +#define __NR_memfd_create 385 > +#elif defined __aarch64__ > +#define __NR_memfd_create 279 > +#elif defined __powerpc__ > +#define __NR_memfd_create 360 > +#elif defined __i386__ > +#define __NR_memfd_create 356 > +#else > +#error "__NR_memfd_create unknown for this architecture" > +#endif > + > +#endif /* __NR_memfd_create */ > + > +static inline int memfd_create(const char *name, unsigned int flags) > +{ > + return syscall(__NR_memfd_create, name, flags); > +} > +#endif /* MFD_HUGETLB */ > + > +#ifndef F_LINUX_SPECIFIC_BASE > +#define F_LINUX_SPECIFIC_BASE 1024 > +#endif > + > +#ifndef MFD_ALLOW_SEALING > +#define MFD_ALLOW_SEALING 0x0002U > +#endif > + > +#ifndef F_ADD_SEALS > +#define F_ADD_SEALS (F_LINUX_SPECIFIC_BASE + 9) > +#define F_GET_SEALS (F_LINUX_SPECIFIC_BASE + 10) > + > +#define F_SEAL_SEAL 0x0001 /* prevent further seals from being set */ > +#define F_SEAL_SHRINK 0x0002 /* prevent file from shrinking */ > +#define F_SEAL_GROW 0x0004 /* prevent file from growing */ > +#define F_SEAL_WRITE 0x0008 /* prevent writes */ > +#endif > + > +#endif /* RTE_ETH_MEMIF_H */ > diff --git a/drivers/net/memif/rte_pmd_memif_version.map b/drivers/net/me= mif/rte_pmd_memif_version.map > new file mode 100644 > index 000000000..8861484fb > --- /dev/null > +++ b/drivers/net/memif/rte_pmd_memif_version.map > @@ -0,0 +1,4 @@ > +DPDK_19.08 { > + > + local: *; > +}; > diff --git a/drivers/net/meson.build b/drivers/net/meson.build > index ed99896c3..b57073483 100644 > --- a/drivers/net/meson.build > +++ b/drivers/net/meson.build > @@ -24,6 +24,7 @@ drivers =3D ['af_packet', > 'ixgbe', > 'kni', > 'liquidio', > + 'memif', > 'mlx4', > 'mlx5', > 'mvneta', > diff --git a/mk/rte.app.mk b/mk/rte.app.mk > index 7c9b4b538..326b2a30b 100644 > --- a/mk/rte.app.mk > +++ b/mk/rte.app.mk > @@ -174,6 +174,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_KNI),y) > _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_KNI) +=3D -lrte_pmd_kni > endif > _LDLIBS-$(CONFIG_RTE_LIBRTE_LIO_PMD) +=3D -lrte_pmd_lio > +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_MEMIF) +=3D -lrte_pmd_memif > _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX4_PMD) +=3D -lrte_pmd_mlx4 > _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) +=3D -lrte_pmd_mlx5 -lmnl > ifeq ($(CONFIG_RTE_IBVERBS_LINK_DLOPEN),y) > -- > 2.17.1