From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
	by inbox.dpdk.org (Postfix) with ESMTP id 48891A00BE;
	Thu, 10 Mar 2022 21:25:26 +0100 (CET)
Received: from [217.70.189.124] (localhost [127.0.0.1])
	by mails.dpdk.org (Postfix) with ESMTP id 3860F4113F;
	Thu, 10 Mar 2022 21:25:26 +0100 (CET)
Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com
 [209.85.210.171])
 by mails.dpdk.org (Postfix) with ESMTP id 843D34113E
 for <dev@dpdk.org>; Thu, 10 Mar 2022 21:25:25 +0100 (CET)
Received: by mail-pf1-f171.google.com with SMTP id g1so6099777pfv.1
 for <dev@dpdk.org>; Thu, 10 Mar 2022 12:25:25 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=networkplumber-org.20210112.gappssmtp.com; s=20210112;
 h=date:from:to:cc:subject:message-id:in-reply-to:references
 :mime-version:content-transfer-encoding;
 bh=CRFREhpfcLIeERhanA/ckrYUTXSw9utjbQ2I/q8zYUQ=;
 b=wyAjVRkXwOjv7zHJfKDZlb88bMxsqQ3u6bx/jHltIk62MaFcRtUAvC/lV+Dlp5a2G9
 lobtJoA4a/YNz0zhJFHvW17CJInadfW75U5lKnCiE4sN2bsbpbf+PgVgyiSZC5h44pPw
 MDnrCHwaKEkZhMIdsIgjUS8zV5uBj9C03a0/M6QQq/KXp5GMN463h61oC6u+qNdjO9xH
 9BsOP3S7op1ZKo3NN2bSh5THGLa1g4WIQzEYt6X3WqXnGtDWodRDXfxwHM9owcu3Ap8o
 LxNprjcpZAfofeWvU1CXhGBhz3aiHuVAhadG8YuBbdoi93e07S9dODueq5LvWTZ1ugh1
 jhIQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to
 :references:mime-version:content-transfer-encoding;
 bh=CRFREhpfcLIeERhanA/ckrYUTXSw9utjbQ2I/q8zYUQ=;
 b=2cd5ecSWggRP1mLztbV70i+9k6cZc4E/FqEwMimWrKBK5tAt82atjKTuEwdXTJzboE
 K7/CnaR4dyYHwKHxUjGb3BPxyQjiCsfaT/VhlSIAAVsJJHtLA2sippv3lqhkHPRBw0Hy
 1vVM/GS+FQ03VPfZC84grYKC5RPgaR/CRQ1vBsXAnR4aFcRzcfBbQn35QG9BKdrQ4zMF
 7qUpz1LyDVuILJzNFWRdYlevqkN6R/XOGweo0CD7nkxvFzcf/cVplO7AVQN5uq9J0Ykw
 bLgwr8QMkFUZQEOrtQQGft1G3eunyUPJTJNLHPxGrsmKeOmF6JpqrvlM9hXh9UjIkPoX
 T1RA==
X-Gm-Message-State: AOAM530tEJ8mnk51lMtVVt4cgEZJsanasrUSUCnlUxhpBLrXBb0YhRip
 4I88b76Wtrpkcai+kK0uIRBqKw==
X-Google-Smtp-Source: ABdhPJwVeZXfdIKydaRi7L9kzCQ52h4emAAiv7nlCojE6JRTjDjSAH9ty0vzNxmm7zkZXiZgukzytA==
X-Received: by 2002:a05:6a00:890:b0:4f6:686e:a8a9 with SMTP id
 q16-20020a056a00089000b004f6686ea8a9mr6344328pfj.83.1646943924576; 
 Thu, 10 Mar 2022 12:25:24 -0800 (PST)
Received: from hermes.local (204-195-112-199.wavecable.com. [204.195.112.199])
 by smtp.gmail.com with ESMTPSA id
 2-20020a620502000000b004f6d2975cbesm7307206pff.116.2022.03.10.12.25.23
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 10 Mar 2022 12:25:24 -0800 (PST)
Date: Thu, 10 Mar 2022 12:23:25 -0800
From: Stephen Hemminger <stephen@networkplumber.org>
To: Megha Ajmera <megha.ajmera@intel.com>
Cc: dev@dpdk.org, john.mcnamara@intel.com, jasvinder.singh@intel.com,
 cristian.dumitrescu@intel.com, ferruh.yigit@intel.com,
 sham.singh.thakur@intel.com
Subject: Re: [PATCH v1] sched: enable/disable TC OV at runtime
Message-ID: <20220310122325.2e928c7d@hermes.local>
In-Reply-To: <20220310195426.4121953-1-megha.ajmera@intel.com>
References: <20220310195426.4121953-1-megha.ajmera@intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org

On Thu, 10 Mar 2022 19:54:26 +0000
Megha Ajmera <megha.ajmera@intel.com> wrote:

> diff --git a/lib/sched/rte_sched.c b/lib/sched/rte_sched.c
> index ec74bee939..1d05089d00 100644
> --- a/lib/sched/rte_sched.c
> +++ b/lib/sched/rte_sched.c
> @@ -155,6 +155,7 @@ struct rte_sched_subport {
>  	uint64_t tc_credits[RTE_SCHED_TRAFFIC_CLASSES_PER_PIPE];
>  
>  	/* TC oversubscription */
> +	uint8_t is_tc_ov_enabled;
>  	uint64_t tc_ov_wm;
>  	uint64_t tc_ov_wm_min;
>  	uint64_t tc_ov_wm_max;

Putting the field there creates a hole in the structure.
Put it after tc_ov and fill an existing hole.

This is pahole of current code, looks like this struct could
use some work to be better packed and aligned.

struct rte_sched_port {
	uint32_t                   n_subports_per_port;  /*     0     4 */
	uint32_t                   n_pipes_per_subport;  /*     4     4 */
	uint32_t                   n_pipes_per_subport_log2; /*     8     4 */
	uint16_t                   pipe_queue[13];       /*    12    26 */
	uint8_t                    pipe_tc[16];          /*    38    16 */
	uint8_t                    tc_queue[16];         /*    54    16 */

	/* XXX 2 bytes hole, try to pack */

	/* --- cacheline 1 boundary (64 bytes) was 8 bytes ago --- */
	uint32_t                   n_subport_profiles;   /*    72     4 */
	uint32_t                   n_max_subport_profiles; /*    76     4 */
	uint64_t                   rate;                 /*    80     8 */
	uint32_t                   mtu;                  /*    88     4 */
	uint32_t                   frame_overhead;       /*    92     4 */
	int                        socket;               /*    96     4 */

	/* XXX 4 bytes hole, try to pack */

	uint64_t                   time_cpu_cycles;      /*   104     8 */
	uint64_t                   time_cpu_bytes;       /*   112     8 */
	uint64_t                   time;                 /*   120     8 */
	/* --- cacheline 2 boundary (128 bytes) --- */
	struct rte_reciprocal      inv_cycles_per_byte;  /*   128     8 */

	/* XXX last struct has 2 bytes of padding */

	uint64_t                   cycles_per_byte;      /*   136     8 */
	struct rte_mbuf * *        pkts_out;             /*   144     8 */
	uint32_t                   n_pkts_out;           /*   152     4 */
	uint32_t                   subport_id;           /*   156     4 */
	struct rte_sched_subport_profile * subport_profiles; /*   160     8 */

	/* XXX 24 bytes hole, try to pack */

	/* --- cacheline 3 boundary (192 bytes) --- */
	struct rte_sched_subport * subports[] __attribute__((__aligned__(64))); /*   192     0 */

	/* size: 192, cachelines: 3, members: 22 */
	/* sum members: 162, holes: 3, sum holes: 30 */
	/* paddings: 1, sum paddings: 2 */
	/* forced alignments: 1, forced holes: 1, sum forced holes: 24 */
} __attribute__((__aligned__(64)));