From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk0-f177.google.com (mail-qk0-f177.google.com [209.85.220.177]) by dpdk.org (Postfix) with ESMTP id 0B24F2FDD for ; Tue, 18 Jul 2017 11:36:16 +0200 (CEST) Received: by mail-qk0-f177.google.com with SMTP id d136so11598503qkg.3 for ; Tue, 18 Jul 2017 02:36:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=fn34B4AEX9iga6jV8CQ+RtXwOO7DbXUO4cOBelsP80k=; b=qRQaOh2+Tokj0wdBCWgV/i7/XhMJV+I6e+6eEQnODrnnmJOodXAB9dQXRoJuNmShNx 2llPzETSqTRzPQnd+TerJ6IU/r7Db2eV/Sves/YFL/8Nwov8HzGe9BBeN4fKEbgdBBS2 3Mei/Qnnm9LTyl7dVricw0SC+FJOmxub57oEowrLfNCHhny+nUD8LGjIi1GTPMSeQTCo QmjCqkJU7kO3c0Ml/hyyrxC2qId21mJ185yn6sNz5iXzu+AEYTFXiuVPTkeciuHHTD5U tGaCAkjiQtEY4GWnzKkDO2wVTwwlDwfaFbpIxqFs7jY5MnmQ7zWbdiTH0SRX8KPrjZlH /npA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=fn34B4AEX9iga6jV8CQ+RtXwOO7DbXUO4cOBelsP80k=; b=HCnAGZUu1uvUpjTo/ZDwn7OTHuwl20D+pDglqzBI3BVobNun27bgU/9nmiTIueNDu5 cC5MsWu1o4edY3ezWJiqCDXOMqAliaPrzr0Z/zKrq1tEULpCmE8dfhRg4LmBnK+VFfbv pSLoMdM8tMypIY7tFCNvyZkPua+OvuY0kv3l2cXhgi3/uTFYAJKcSILWqd+Ttc4EiZeB 230ZuiKmGBwzfrXdAiJE5jVQ4j8MaYlV9yd0EgQU53spKBkQDsHJvoJAAewEbnNN4Z6f dWCIka1KXsYQ0MMq5VH89eZA6Q1cufJOk5bZgF9Ivpc8HpoXcEgPFR5TxK/LNH4N81nu u9Dg== X-Gm-Message-State: AIVw1115/ryQmvudbATy8ywITGHarUCIg2aqwSWDnSCbBm0ZMnVP+zsO 7gVDRCAA3KSIsYFqkDuMn8RzffAxXg== X-Received: by 10.55.135.196 with SMTP id j187mr630425qkd.221.1500370576342; Tue, 18 Jul 2017 02:36:16 -0700 (PDT) MIME-Version: 1.0 Received: by 10.237.61.51 with HTTP; Tue, 18 Jul 2017 02:36:15 -0700 (PDT) In-Reply-To: References: From: Harold Demure Date: Tue, 18 Jul 2017 11:36:15 +0200 Message-ID: To: Shyam Shrivastav Cc: Pavel Shirshov , users@dpdk.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-users] Strange packet loss with multi-frame payloads X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Jul 2017 09:36:17 -0000 Hello Shyam, Thank you for your suggestion. I will try what you say. However, this problem arises only with specific workloads. For example, if the clients only send requests of 1 frame, everything runs smoothly even with 16 active queues. My problem arises only with bigger payloads and multiple queues. Shouldn't this suggest that the problem is not "simply" that my NIC drops packets with > X active queues? Regards, Harold 2017-07-18 7:50 GMT+02:00 Shyam Shrivastav : > As I understand the problem disappears with 1 RX queue on server. You can > reduce number of queues on server from 8 and arrive at an optimal value > without packet loss. > For intel 82599 NIC packet loss is experienced with more than 4 RX queues, > this was reported in dpdk dev or user mailing list, read in archives > sometime back while looking for similar information with 82599. > > On Tue, Jul 18, 2017 at 4:54 AM, Harold Demure > wrote: > >> Hello again, >> I tried to convert my statically defined buffers into buffers allocated >> through rte_malloc (as discussed in the previous email, see quoted text). >> Unfortunately, the problem is still there :( >> Regards, >> Harold >> >> >> >> > >> > 2. How do you know you have the packet loss? >> > >> > >> > *I know it because some fragmented packets never get reassembled fully. >> If >> > I print the packets seen by the server I see something like "PCKT_ID 10 >> > FRAG 250, PCKT_ID 10 FRAG 252". And FRAG 251 is never printed.* >> > >> > *Actually, something strange that happens sometimes is that a core >> > receives fragments of two packets and, say, receives frag 1 of packet >> X, >> > frag 2 of packet Y, frag 3 of packet X, frag 4 of packet Y.* >> > *Or that, after "losing" a fragment for packet X, I only see printed >> > fragments with EVEN frag_id for that packet X. At least for a while.* >> > >> > *This led me also to consider a bug in my implementation (I don't >> > experience this problem if I run with a SINGLE client thread). However, >> > with smaller payloads, even fragmented, everything runs smoothly.* >> > *If you have any suggestions for tests to run to spot a possible bug in >> my >> > implementation, It'd be more than welcome!* >> > >> > *MORE ON THIS: the buffers in which I store the packets taken from RX >> are >> > statically defined arrays, like struct rte_mbuf* temp_mbuf[SIZE]. SIZE >> > can be pretty high (say, 10K entries), and there are 3 of those arrays >> per >> > core. Can it be that, somehow, they mess up the memory layout (e.g., >> they >> > intersect)?* >> > >> > >