From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D602042B22; Tue, 16 May 2023 13:45:33 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A7900410EE; Tue, 16 May 2023 13:45:33 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 6453240A8B for ; Tue, 16 May 2023 13:45:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684237531; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8Y4qVjgIixK5Rnnj3+pDfyLweoiDmyHj/fzazM0RJp8=; b=Vm70X76yqTzGQntpBcIjc4FJHP1xybzrRY1hzlNSGWKCAz6X4UsnZh+EEVVFX0bqwm7qbk /kQSPnfA6jokynasL9bl13tiD/8N3G4mzoMa8fisRC5PjRBf9GXfrzM2ZIkJEwvG9zHiv3 8Svs4Wv2g7xmE31oLAJuMtJoKlPrrMk= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-487-oIv9J6RJOKO-VYuntNpUjw-1; Tue, 16 May 2023 07:45:30 -0400 X-MC-Unique: oIv9J6RJOKO-VYuntNpUjw-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 71BA789174B; Tue, 16 May 2023 11:45:30 +0000 (UTC) Received: from [10.39.208.21] (unknown [10.39.208.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6BBB1492B00; Tue, 16 May 2023 11:45:29 +0000 (UTC) Message-ID: Date: Tue, 16 May 2023 13:45:27 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 To: Eelco Chaudron , David Marchand Cc: chenbo.xia@intel.com, dev@dpdk.org References: <168069838578.833254.4856666346839028593.stgit@ebuild.local> <168069850278.833254.17836553051894685658.stgit@ebuild.local> <683D0767-DCE8-45F4-B904-CD2DD574DE53@redhat.com> <0E248797-C6C2-4C5A-B96F-1693B9B2091C@redhat.com> From: Maxime Coquelin Subject: Re: [PATCH v2 3/3] vhost: add device op to offload the interrupt kick In-Reply-To: <0E248797-C6C2-4C5A-B96F-1693B9B2091C@redhat.com> X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 5/16/23 13:36, Eelco Chaudron wrote: > > > On 16 May 2023, at 12:12, David Marchand wrote: > >> On Tue, May 16, 2023 at 10:53 AM Eelco Chaudron wrote: >>> On 10 May 2023, at 13:44, David Marchand wrote: >> >> [snip] >> >>>>> @@ -846,6 +848,14 @@ vhost_user_socket_mem_free(struct vhost_user_socket *vsocket) >>>>> vsocket->path = NULL; >>>>> } >>>>> >>>>> + if (vsocket && vsocket->alloc_notify_ops) { >>>>> +#pragma GCC diagnostic push >>>>> +#pragma GCC diagnostic ignored "-Wcast-qual" >>>>> + free((struct rte_vhost_device_ops *)vsocket->notify_ops); >>>>> +#pragma GCC diagnostic pop >>>>> + vsocket->notify_ops = NULL; >>>>> + } >>>> >>>> Rather than select the behavior based on a boolean (and here force the >>>> compiler to close its eyes), I would instead add a non const pointer >>>> to ops (let's say alloc_notify_ops) in vhost_user_socket. >>>> The code can then unconditionnally call free(vsocket->alloc_notify_ops); >>> >>> Good idea, I will make the change in v3. >> >> Feel free to use a better name for this field :-). >> >>> >>>>> + >>>>> if (vsocket) { >>>>> free(vsocket); >>>>> vsocket = NULL; >> >> [snip] >> >>>>> + /* >>>>> + * Although the ops structure is a const structure, we do need to >>>>> + * override the guest_notify operation. This is because with the >>>>> + * previous APIs it was "reserved" and if any garbage value was passed, >>>>> + * it could crash the application. >>>>> + */ >>>>> + if (ops && !ops->guest_notify) { >>>> >>>> Hum, as described in the comment above, I don't think we should look >>>> at ops->guest_notify value at all. >>>> Checking ops != NULL should be enough. >>> >>> Not sure I get you here. If the guest_notify passed by the user is NULL, it means the previously ‘reserved[1]’ field is NULL, so we do not need to use a new structure. >>> >>> I guess your comment would be true if we would introduce a new field in the data structure, not replacing a reserved one. >> >> Hum, I don't understand my comment either o_O'. >> Too many days off... or maybe my evil twin took over the keyboard. >> >> >>> >>>>> + struct rte_vhost_device_ops *new_ops; >>>>> + >>>>> + new_ops = malloc(sizeof(*new_ops)); >>>> >>>> Strictly speaking, we lose the numa affinity of "ops" by calling malloc. >>>> I am unclear of the impact though. >>> >>> Don’t think there is a portable API that we can use to determine the NUMA for the ops memory and then allocate this on the same numa? >>> >>> Any thoughts or ideas on how to solve this? I hope most people will memset() the ops structure and the reserved[1] part is zero, but it might be a problem in the future if more extensions get added. >> >> Determinining current numa is doable, via 'ops' >> get_mempolicy(MPOL_F_NODE | MPOL_F_ADDR), like what is done for vq in >> numa_realloc(). >> The problem is how to allocate on this numa with the libc allocator >> for which I have no idea... >> We could go with the dpdk allocator (again, like numa_realloc()). >> >> >> In practice, the passed ops will be probably from a const variable in >> the program .data section (for which I think fields are set to 0 >> unless explicitly initialised), or a memset() will be called for a >> dynamic allocation from good citizens. >> So we can probably live with the current proposal. >> Plus, this is only for one release, since in 23.11 with the ABI bump, >> we will drop this compat code. >> >> Maxime, Chenbo, what do you think? > > Wait for their response, but for now I assume we can just keep the numa unaware malloc(). Let's keep it as is as we'll get rid of it in 23.11. >> >> [snip] >> >>>> >>>> But putting indentation aside, is this change equivalent? >>>> - if ((vhost_need_event(vhost_used_event(vq), new, old) && >>>> - (vq->callfd >= 0)) || >>>> - unlikely(!signalled_used_valid)) { >>>> + if ((vhost_need_event(vhost_used_event(vq), new, old) || >>>> + unlikely(!signalled_used_valid)) && >>>> + vq->callfd >= 0) { >>> >>> They are not equal, but in the past eventfd_write() should also not have been called with callfd < 0, guess this was an existing bug ;) >> >> I think this should be a separate fix. > > ACK, will add a separate patch in this series to fix it. I also caught & fixed it while implementing my VDUSE series [0]. You can pick it in your series, and I will rebase my series on top of it. Thanks, Maxime [0]: https://gitlab.com/mcoquelin/dpdk-next-virtio/-/commit/b976e1f226db5c09834148847d994045eb89be93 > >> >>> >>>>> + vhost_vring_inject_irq(dev, vq); >> >> >> -- >> David Marchand >