From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4316CA0C43; Fri, 8 Oct 2021 05:28:48 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BB8EB4067E; Fri, 8 Oct 2021 05:28:47 +0200 (CEST) Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by mails.dpdk.org (Postfix) with ESMTP id 7BA974003C for ; Fri, 8 Oct 2021 05:28:45 +0200 (CEST) Received: from dggemv703-chm.china.huawei.com (unknown [172.30.72.56]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4HQYSy5kpYzbnB2; Fri, 8 Oct 2021 11:24:18 +0800 (CST) Received: from dggpeml500024.china.huawei.com (7.185.36.10) by dggemv703-chm.china.huawei.com (10.3.19.46) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.8; Fri, 8 Oct 2021 11:28:42 +0800 Received: from [127.0.0.1] (10.67.100.224) by dggpeml500024.china.huawei.com (7.185.36.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.8; Fri, 8 Oct 2021 11:28:42 +0800 To: Matan Azrad , NBU-Contact-Thomas Monjalon , "ferruh.yigit@intel.com" , "bruce.richardson@intel.com" , "jerinj@marvell.com" , "jerinjacobk@gmail.com" , "andrew.rybchenko@oktetlabs.ru" CC: "dev@dpdk.org" , "mb@smartsharesystems.com" , "nipun.gupta@nxp.com" , "hemant.agrawal@nxp.com" , "maxime.coquelin@redhat.com" , "honnappa.nagarahalli@arm.com" , "david.marchand@redhat.com" , "sburla@marvell.com" , "pkapoor@marvell.com" , "konstantin.ananyev@intel.com" , "conor.walsh@intel.com" , "kevin.laatz@intel.com" References: <1625231891-2963-1-git-send-email-fengchengwen@huawei.com> <20210924105357.15386-1-fengchengwen@huawei.com> <20210924105357.15386-3-fengchengwen@huawei.com> From: fengchengwen Message-ID: Date: Fri, 8 Oct 2021 11:28:42 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.11.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset="utf-8" Content-Language: en-US Content-Transfer-Encoding: 7bit X-Originating-IP: [10.67.100.224] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpeml500024.china.huawei.com (7.185.36.10) X-CFilter-Loop: Reflected Subject: Re: [dpdk-dev] [PATCH v23 2/6] dmadev: add control plane function support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 2021/10/5 18:16, Matan Azrad wrote: > Hi Chengwen > > API looks good to me, thanks! > > I have some questions below. > >> This patch add control plane functions for dmadev. >> > >> +/** >> + * DMA transfer direction defines. >> + * >> + * @see struct rte_dma_vchan_conf::direction */ enum rte_dma_direction >> +{ >> + /** DMA transfer direction - from memory to memory. >> + * >> + * @see struct rte_dma_vchan_conf::direction >> + */ >> + RTE_DMA_DIR_MEM_TO_MEM, >> + /** DMA transfer direction - from memory to device. >> + * In a typical scenario, the SoCs are installed on host servers as >> + * iNICs through the PCIe interface. In this case, the SoCs works in >> + * EP(endpoint) mode, it could initiate a DMA move request from >> memory >> + * (which is SoCs memory) to device (which is host memory). >> + * >> + * @see struct rte_dma_vchan_conf::direction >> + */ >> + RTE_DMA_DIR_MEM_TO_DEV, > > > I don't understand precisely the meaning of mem and dev. > > What does it mean SoCs memory? > > What does it mean host memory? > > What is the memory HW in these two types? > > How does the user get the addresses of SoCs memory? > > How does the user get the addresses of host memory? > Hi Matan, System Bus | ----------PCIe module---------- | Bus | Interface | ----- ------------------ | | | | PCIe Core0 | DDR3 ------| | | | | ----------- | | | | PF-0 -- VF-0 | | Host A | | | |--------| |- VF-1 |--------| Root | ---DDR1 | | | | PF-1 | | Complex | | | | | PF-2 | ----------- | | | ------------------ | |DMA| | | | ------------------ | | | | PCIe Core1 | | | | | | ----------- | | | | PF-0 -- VF-0 | | Host B | |-----| |--------| PF-1 -- VF-0 |--------| Root | ---DDR2 | | | | |- VF-1 | | Complex | | | | | PF-2 | ----------- | | | ------------------ | ----- ---------------------SOC------------------------- -------HOST-------- As shown in the above figure, the SOC is connected to two hosts. The DDR3 is the SOC's memory, the DDR1 is HOST-A's memory, and DDR2 is HOST-B's memory. To access DDR3 memory, APP could mmap and pass its IOVA address to DMA. To access DDR1/2 memory, some devices use parameterized descriptor which contain the function from which the request is sent and the destination address. In this case, it is up to the user to confirm the function and destination address which could passed in as parameters. Thanks. > > Can dpdk app here access physical memory not mapped\allocated to the app? > > Matan > > > >> + /** DMA transfer direction - from device to memory. >> + * In a typical scenario, the SoCs are installed on host servers as >> + * iNICs through the PCIe interface. In this case, the SoCs works in >> + * EP(endpoint) mode, it could initiate a DMA move request from device >> + * (which is host memory) to memory (which is SoCs memory). >> + * >> + * @see struct rte_dma_vchan_conf::direction >> + */ >> + RTE_DMA_DIR_DEV_TO_MEM, >> + /** DMA transfer direction - from device to device. >> + * In a typical scenario, the SoCs are installed on host servers as >> + * iNICs through the PCIe interface. In this case, the SoCs works in >> + * EP(endpoint) mode, it could initiate a DMA move request from device >> + * (which is host memory) to the device (which is another host >> memory). >> + * >> + * @see struct rte_dma_vchan_conf::direction >> + */ >> + RTE_DMA_DIR_DEV_TO_DEV, >> +}; >> + >> +/** >> + * DMA access port type defines. >> + * >> + * @see struct rte_dma_port_param::port_type */ enum >> rte_dma_port_type >> +{ >> + RTE_DMA_PORT_NONE, >> + RTE_DMA_PORT_PCIE, /**< The DMA access port is PCIe. */ }; >> + >> +/** >> + * A structure used to descript DMA access port parameters. >> + * >> + * @see struct rte_dma_vchan_conf::src_port >> + * @see struct rte_dma_vchan_conf::dst_port */ struct >> +rte_dma_port_param { >> + /** The device access port type. >> + * >> + * @see enum rte_dma_port_type >> + */ >> + enum rte_dma_port_type port_type; >> + union { >> + /** PCIe access port parameters. >> + * >> + * The following model shows SoC's PCIe module connects to >> + * multiple PCIe hosts and multiple endpoints. The PCIe module >> + * has an integrated DMA controller. >> + * >> + * If the DMA wants to access the memory of host A, it can be >> + * initiated by PF1 in core0, or by VF0 of PF0 in core0. >> + * >> + * \code{.unparsed} >> + * System Bus >> + * | ----------PCIe module---------- >> + * | Bus >> + * | Interface >> + * | ----- ------------------ >> + * | | | | PCIe Core0 | >> + * | | | | | ----------- >> + * | | | | PF-0 -- VF-0 | | Host A | >> + * | | |--------| |- VF-1 |--------| Root | >> + * | | | | PF-1 | | Complex | >> + * | | | | PF-2 | ----------- >> + * | | | ------------------ >> + * | | | >> + * | | | ------------------ >> + * | | | | PCIe Core1 | >> + * | | | | | ----------- >> + * | | | | PF-0 -- VF-0 | | Host B | >> + * |-----| |--------| PF-1 -- VF-0 |--------| Root | >> + * | | | | |- VF-1 | | Complex | >> + * | | | | PF-2 | ----------- >> + * | | | ------------------ >> + * | | | >> + * | | | ------------------ >> + * | |DMA| | | ------ >> + * | | | | |--------| EP | >> + * | | |--------| PCIe Core2 | ------ >> + * | | | | | ------ >> + * | | | | |--------| EP | >> + * | | | | | ------ >> + * | ----- ------------------ >> + * >> + * \endcode >> + * >> + * @note If some fields can not be supported by the >> + * hardware/driver, then the driver ignores those fields. >> + * Please check driver-specific documentation for limitations >> + * and capablites. >> + */ >> + struct { >> + uint64_t coreid : 4; /**< PCIe core id used. */ >> + uint64_t pfid : 8; /**< PF id used. */ >> + uint64_t vfen : 1; /**< VF enable bit. */ >> + uint64_t vfid : 16; /**< VF id used. */ >> + /** The pasid filed in TLP packet. */ >> + uint64_t pasid : 20; >> + /** The attributes filed in TLP packet. */ >> + uint64_t attr : 3; >> + /** The processing hint filed in TLP packet. */ >> + uint64_t ph : 2; >> + /** The steering tag filed in TLP packet. */ >> + uint64_t st : 16; >> + } pcie; >> + }; >> + uint64_t reserved[2]; /**< Reserved for future fields. */ }; >> + >> +/** >> + * A structure used to configure a virtual DMA channel. >> + * >> + * @see rte_dma_vchan_setup >> + */ >> +struct rte_dma_vchan_conf { >> + /** Transfer direction >> + * >> + * @see enum rte_dma_direction >> + */ >> + enum rte_dma_direction direction; >> + /** Number of descriptor for the virtual DMA channel */ >> + uint16_t nb_desc; >> + /** 1) Used to describes the device access port parameter in the >> + * device-to-memory transfer scenario. >> + * 2) Used to describes the source device access port parameter in the >> + * device-to-device transfer scenario. >> + * >> + * @see struct rte_dma_port_param >> + */ >> + struct rte_dma_port_param src_port; >> + /** 1) Used to describes the device access port parameter in the >> + * memory-to-device transfer scenario. >> + * 2) Used to describes the destination device access port parameter in >> + * the device-to-device transfer scenario. >> + * >> + * @see struct rte_dma_port_param >> + */ >> + struct rte_dma_port_param dst_port; }; >> + > >