From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-it0-f43.google.com (mail-it0-f43.google.com [209.85.214.43]) by dpdk.org (Postfix) with ESMTP id 9D03072FB for ; Wed, 14 Sep 2016 04:43:50 +0200 (CEST) Received: by mail-it0-f43.google.com with SMTP id n143so4772480ita.1 for ; Tue, 13 Sep 2016 19:43:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:from:date:message-id:subject:to; bh=8S1q5FjRMVjbldmPv0rLRbpkhlpOYtClzRH3rAv7NnI=; b=tl0e43SUA7CtPrxMMZIyfV3Z46jE9LprpcNR48jVh1OZFe7auYmjGYDmqCzPELKPVR gmrDyOTHoB+Uj7UyoKJLUv7pWSHJOAq4hGevEP3yGEdCXd7+3Ty+rJSdSs6CYiyY19Ab AET8+1HEv34oTtAqYcLs/VouPwPDnQBCZTprY4638NzHwu2DEfZA0hL716S21P/6kL7X 6b+4FlftGYEhvA8WwFq4T/dF+hXsMQtDuIR3OtcBVKoqoAQG+ompgQ6JPJFXwiA5CUuT rP2Ge4GfepNQUcHMaZztWjmJ8zNXfWGccW/BtMt1YGrlScV79x1IfE784vadxxN/KOhK ognA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:from:date:message-id:subject :to; bh=8S1q5FjRMVjbldmPv0rLRbpkhlpOYtClzRH3rAv7NnI=; b=ZaqEyYS+cwGFLkXAir5NlZ+4JFx/Okv09nJ7rxZcJKku+wYp4CTkhl6N+R665fX+62 v6GZSvNH/SeAQRoctpcde0pMRNAx9Xi+xLfrtanXZNyPI3vdS5EbQ5Qlx8IvBi8rHkfy Jdd4KWiVg7zA8zrREGfg+sTF0zP6nPEVEYkq2RqD6MnHAm9AmjxijWkjwIjxV0HCJVRi EEZfXzXAjD6ArJ8/TjM8bo0LvxeiKoM08CGylhWXcq/7f6I/hDr1Kajg6GVqCiJ1+gQy CppqD7e9hK0p/X2AQ02EjJbNK4WIRbypQi94g/DTf4AmT7Hnr8298DPdSz/x+FIrShCk gFUw== X-Gm-Message-State: AE9vXwNCawtFbBS3IDVRTZxpwyTnLYw4JpR3eRCO7Ez+6N+GNPsTkSY0pWByLM1+5QvObE6JqofvZgjdfgT8MQ== X-Received: by 10.107.30.194 with SMTP id e185mr1482291ioe.128.1473821029908; Tue, 13 Sep 2016 19:43:49 -0700 (PDT) MIME-Version: 1.0 Sender: jsnell@gmail.com Received: by 10.64.66.200 with HTTP; Tue, 13 Sep 2016 19:43:49 -0700 (PDT) From: Juho Snellman Date: Wed, 14 Sep 2016 04:43:49 +0200 X-Google-Sender-Auth: PqhVgTHAYGtirv6097kCGZym6ls Message-ID: To: dev@dpdk.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-dev] Legacy virtio vs. multi-process mode X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Sep 2016 02:43:50 -0000 I've been having trouble getting virtio (in legacy mode) working with the multi-process model. The latest issue is as follows: - The primary process calls vtpci_init() with a struct rte_pci_device that's only valid in that process - For legacy mode vtpci_init() calls rte_eal_pci_ioport() with that pci device and a the struct rte_pci_ioport that's in shared memory - The secondary processes will of course not call vtpci_init() - When a secondary process tries to notify the vqueue in either the the RX or TX path, it will do so via the shared memory ioport. - When rte_eal_pci_ioport_write() tries to dispatch the ioport to the right place, it'll need to access the PCI device. But that's a garbage pointer in the context of this process. (Usually it's still pointing to some allocated memory, so you don't even get a segfault but the ioport write is just silently aborted). - This causes the queues to stall, unless the primary process happens to notify them as well for some reason. Is there a clean solution? My current workaround is to allocate rte_pci_device structs in shared memory rather than the heap. (Though all processes still have their own copy). But that requires rearranging the initialization order in rte_eal_init() to delay the PCI init, and I'm guessing there's a good reason for the current order. -- Juho Snellman