From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0CF30433A3; Thu, 23 Nov 2023 06:25:18 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D1B3C402BD; Thu, 23 Nov 2023 06:25:17 +0100 (CET) Received: from mail-oo1-f52.google.com (mail-oo1-f52.google.com [209.85.161.52]) by mails.dpdk.org (Postfix) with ESMTP id 60C2240041 for ; Thu, 23 Nov 2023 06:25:16 +0100 (CET) Received: by mail-oo1-f52.google.com with SMTP id 006d021491bc7-58d18d5c687so243409eaf.3 for ; Wed, 22 Nov 2023 21:25:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700717115; x=1701321915; darn=dpdk.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=0Ia6JDXcqoaC/tRFyQXxoysFrK4KxUcTuquUHA6ym/Y=; b=WN/Lj3+AqHIaZC2sPsRS0VVI7Ci9OUO5JWvSlhkqanxu6Oa9QB+8P7pVQZ/dy9Sltl OR6nLFJ0GKs7xDrNz4NfV5+yF9uALc0VpirH7LXj9z4gDoj6o6Ov2hHnNDp3Jc9kNyPA kNemI6iprKXj8IuJL9Kyqibwso/7J2Hy1hQq1EQnjNvTt8Utoa9qR6VqD1JUsohCk00V DAuIeWemeqv5vDWs6NGg9MvlPJxqWvzDqbKlJQYwjahKT5SJqteYtl7TaFQGWOAMNUif gK9csz10XsSMS5jk3rWJi3tz8LZsHMLFApOCA4qdm1rw0A7V7x1xYBUXBTwO2fuUMLL0 nVTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700717115; x=1701321915; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0Ia6JDXcqoaC/tRFyQXxoysFrK4KxUcTuquUHA6ym/Y=; b=iAZsbZvrul/l4cTNjeH3kvk12kscvO1fk8eGzr8MH5uiNI/7DssV+eC2aihlZdsURD dU0hFMsC1Ejc3BdmpILJhPPftWVr77PIluv8LXHYt6tnNp8/HQ7a7OI7PeusKtJBl+LE +jizpNSu+3iOiUdoeoRTWrIQCbe3q3YrDZe71ydsynAzNlBYUddfSsdumLi+s87kr3SF OaT6qHT2TcliZpYHxBcZ///8hy21k8IBbubjVuDIIS0cjK8JdUa7N9okC2yFnPKksMF2 +4QDCaJLy2JDHT6RBIBgW79lvV0vXDmE9/clpxszhqawhfFQjW5g+JSREltkXPAuhplc 7BBg== X-Gm-Message-State: AOJu0Yya+yuj1Zt6sG2Pl691wIF6By19pRonG+JNgdX9zV2AuLy1SBZo N1UqJ7K01Lw+/13wtVpTA1NTb8hlggD6MxGgfgA= X-Google-Smtp-Source: AGHT+IF4NrOP+tYiFr6FIAmEKEODsK/rwItpxV+6vdzL2S0QtUBP/O68cMtjCUOMs5SSlYfomB5JprtwSRxleNfFJI8= X-Received: by 2002:a05:6358:f5c:b0:169:9586:9195 with SMTP id c28-20020a0563580f5c00b0016995869195mr3143507rwj.31.1700717115533; Wed, 22 Nov 2023 21:25:15 -0800 (PST) MIME-Version: 1.0 References: <8866a5c7ea36e476b2a92e3e4cea6c2c127ab82f.1691768110.git.anatoly.burakov@intel.com> <8ce1cf14-17e4-4092-1102-305f8fe25a36@huawei.com> <3504aaf1-b4e0-44f5-883a-5e4bc0197283@intel.com> In-Reply-To: <3504aaf1-b4e0-44f5-883a-5e4bc0197283@intel.com> From: Jerin Jacob Date: Thu, 23 Nov 2023 10:54:49 +0530 Message-ID: Subject: Re: [PATCH v1 1/3] dmadev: add inter-domain operations To: "Medvedkin, Vladimir" Cc: fengchengwen , sburla@marvell.com, anoobj@marvell.com, Anatoly Burakov , dev@dpdk.org, Kevin Laatz , Bruce Richardson Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Fri, Oct 27, 2023 at 7:16=E2=80=AFPM Medvedkin, Vladimir wrote: > > Hi Satananda, Anoob, Chengwen, Jerin, all, > > After a number of internal discussions we have decided that we're going > to postpone this feature/patchset till next release. > > >[Satananda] Have you considered extending rte_dma_port_param and > rte_dma_vchan_conf to represent interdomain memory transfer setup as a > separate port type like RTE_DMA_PORT_INTER_DOMAIN ? > > >[Anoob] Can we move this 'src_handle' and 'dst_handle' registration to > rte_dma_vchan_setup so that the 'src_handle' and 'dst_handle' can be > configured in control path and the existing datapath APIs can work as is. > > >[Jerin] Or move src_handle/dst_hanel to vchan config > > We've listened to feedback on implementation, and have prototyped a > vchan-based interface. This has a number of advantages and > disadvantages, both in terms of API usage and in terms of our specific > driver. > > Setting up inter-domain operations as separate vchans allow us to store > data inside the PMD and not duplicate any API paths, so having multiple > vchans addresses that problem. However, this also means that any new > vchans added while the PMD is active (such as attaching to a new This could be mitigated by setup max number of vchan up front before start(= ) and use as demanded. > process) will have to be gated by start/stop. This is probably fine from > API point of view, but a hassle for user (previously, we could've just > started using the new inter-domain handle right away). > > Another usability issue with multiple vchan approach is that now, each > vchan will have its own enqueue/submit/completion cycle, so any use case > relying on one thread communicating with many processes will have to > process each vchan separately, instead of everything going into one > vchan - again, looks fine API-wise, but a hassle for the user, since > this requires calling submit and completion for each vchan, and in some > cases it requires maintaining some kind of reordering queue. (On the > other hand, it would be much easier to separate operations intended for > different processes with this approach, so perhaps this is not such a > big issue) IMO, The design principle behind vchan was, -A single HW queue be serving N number of vchan -A vchan is nothing, but it creates desired HW instruction format as template in slow path to use in fast path or write some slow path registers to define the attribute of vchan. IMO, The above-mentioned usability constraints will be there in all PMD as vchan is muxing a single HW queue. IMO, Decision for vchan vs fast path API could be a) Number of vchan is required - In this case, we are using for VM to VM or Container to Container copy. So I think, it is limited b) HW support - Some HW's in order to reduce size of HW descriptor size, some features will be configured as slow path only(can not be changed at runtime, without reconfiguring the vchan).