From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 814A9A04B1; Thu, 24 Sep 2020 14:20:18 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id EEB051DF97; Thu, 24 Sep 2020 14:14:01 +0200 (CEST) Received: from dispatch1-us1.ppe-hosted.com (dispatch1-us1.ppe-hosted.com [148.163.129.52]) by dpdk.org (Postfix) with ESMTP id 59AEB1DE2B for ; Thu, 24 Sep 2020 14:13:03 +0200 (CEST) Received: from mx1-us1.ppe-hosted.com (unknown [10.7.65.64]) by dispatch1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTP id D7BDD60052 for ; Thu, 24 Sep 2020 12:13:02 +0000 (UTC) Received: from us4-mdac16-2.ut7.mdlocal (unknown [10.7.65.70]) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTP id D6F1D2009B for ; Thu, 24 Sep 2020 12:13:02 +0000 (UTC) X-Virus-Scanned: Proofpoint Essentials engine Received: from mx1-us1.ppe-hosted.com (unknown [10.7.65.174]) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 47ADA22004F for ; Thu, 24 Sep 2020 12:13:02 +0000 (UTC) Received: from webmail.solarflare.com (uk.solarflare.com [193.34.186.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id F1F631C0068 for ; Thu, 24 Sep 2020 12:13:01 +0000 (UTC) Received: from ukex01.SolarFlarecom.com (10.17.10.4) by ukex01.SolarFlarecom.com (10.17.10.4) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 24 Sep 2020 13:12:47 +0100 Received: from opal.uk.solarflarecom.com (10.17.10.1) by ukex01.SolarFlarecom.com (10.17.10.4) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 24 Sep 2020 13:12:47 +0100 Received: from ukv-loginhost.uk.solarflarecom.com (ukv-loginhost.uk.solarflarecom.com [10.17.10.39]) by opal.uk.solarflarecom.com (8.13.8/8.13.8) with ESMTP id 08OCCljn026050 for ; Thu, 24 Sep 2020 13:12:47 +0100 Received: from ukv-loginhost.uk.solarflarecom.com (localhost [127.0.0.1]) by ukv-loginhost.uk.solarflarecom.com (Postfix) with ESMTP id DC9501613AB for ; Thu, 24 Sep 2020 13:12:47 +0100 (BST) From: Andrew Rybchenko To: Date: Thu, 24 Sep 2020 13:12:17 +0100 Message-ID: <1600949555-28043-43-git-send-email-arybchenko@solarflare.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1600949555-28043-1-git-send-email-arybchenko@solarflare.com> References: <1600764594-14752-1-git-send-email-arybchenko@solarflare.com> <1600949555-28043-1-git-send-email-arybchenko@solarflare.com> MIME-Version: 1.0 Content-Type: text/plain X-TM-AS-Product-Ver: SMEX-12.5.0.1300-8.6.1012-25674.003 X-TM-AS-Result: No-6.374300-8.000000-10 X-TMASE-MatchedRID: jsC3uNckhg4ij1rXhFZBDNB/IoRhBzVHSoCG4sefl8Q7OtcKFS0EBBb/ j4AKLa01VZ/zZu2XjbKRW72+zOTrrr3VqvDjHOXN4h8r8l3l4eZvV3/OnMClWhQUOSCpbPwOaXM mCtN+yuTZccnQviM4hCz/fUbMqROxCKGbCJcIygY/ApMPW/xhXkyQ5fRSh265AqmSGaN08VCJmy WBjMsxkBfb/g6/AVAtueD2AKNg3y6D66TlvUkVYODN5dBHl70NA96VLx8Kcq68YDH/UBNnmzUee vPcUyNZJBrWAcLM7pB6pBzYp496ZuKU/rnfWHv6CWlWR223da5SQLJ/PYofeDbpMgyAfh26cOQF qW7TulVYaCfj+DnsCOWjw11a7ymC9KnJ9ZqX1X3MbQu1fPiCD0NWaKIdBIV4MAmzHUrPkB5OzKQ Ys+5RjoZsgRXKjrxDU+XXDdnqpV7LkQcFIcXJTX84FZpy/6JVfXRAc+s3RFd1+BANR76ZZ22L9P qwEA2bMEiQlkRQZB1HBaYvF0hxKHUxqy9pUmSjyl2yL95kDWN9LQinZ4QefMdVaUMTROUqJdW0G bI6Iq6OhzOa6g8Kra0jvZMGDW1pK4Md0GDzOcLhmcfLMEu5oI+CWj5H8dgtXg9mK8GW8ypDDKa3 G4nrLQ== X-TM-AS-User-Approved-Sender: Yes X-TM-AS-User-Blocked-Sender: No X-TMASE-Result: 10--6.374300-8.000000 X-TMASE-Version: SMEX-12.5.0.1300-8.6.1012-25674.003 X-MDID: 1600949582-6wdXdH0HVwJ7 Subject: [dpdk-dev] [PATCH v3 42/60] common/sfc_efx/base: choose smallest Rx prefix on Riverhead X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Riverhead supports many Rx prefixes. The Rx prefix may be chosen based on which information is required. To have better performance choose the smallest Rx prefix which meets our requirements. Right now there is no way to specify requirements on Rx queue creation, but it could be added in the future. Signed-off-by: Andrew Rybchenko Reviewed-by: Andy Moreton --- drivers/common/sfc_efx/base/efx_impl.h | 1 + drivers/common/sfc_efx/base/efx_mcdi.c | 10 +- drivers/common/sfc_efx/base/rhead_rx.c | 333 ++++++++++++++++++++++++- 3 files changed, 338 insertions(+), 6 deletions(-) diff --git a/drivers/common/sfc_efx/base/efx_impl.h b/drivers/common/sfc_efx/base/efx_impl.h index c373192554..fc0a654f80 100644 --- a/drivers/common/sfc_efx/base/efx_impl.h +++ b/drivers/common/sfc_efx/base/efx_impl.h @@ -1439,6 +1439,7 @@ typedef struct efx_mcdi_init_rxq_params_s { uint32_t es_max_dma_len; uint32_t es_buf_stride; uint32_t hol_block_timeout; + uint32_t prefix_id; } efx_mcdi_init_rxq_params_t; LIBEFX_INTERNAL diff --git a/drivers/common/sfc_efx/base/efx_mcdi.c b/drivers/common/sfc_efx/base/efx_mcdi.c index aa19c7c759..b8e45b458d 100644 --- a/drivers/common/sfc_efx/base/efx_mcdi.c +++ b/drivers/common/sfc_efx/base/efx_mcdi.c @@ -2692,8 +2692,8 @@ efx_mcdi_init_rxq( { efx_nic_cfg_t *encp = &(enp->en_nic_cfg); efx_mcdi_req_t req; - EFX_MCDI_DECLARE_BUF(payload, MC_CMD_INIT_RXQ_V4_IN_LEN, - MC_CMD_INIT_RXQ_V4_OUT_LEN); + EFX_MCDI_DECLARE_BUF(payload, MC_CMD_INIT_RXQ_V5_IN_LEN, + MC_CMD_INIT_RXQ_V5_OUT_LEN); int npages = efx_rxq_nbufs(enp, ndescs); int i; efx_qword_t *dma_addr; @@ -2747,9 +2747,9 @@ efx_mcdi_init_rxq( req.emr_cmd = MC_CMD_INIT_RXQ; req.emr_in_buf = payload; - req.emr_in_length = MC_CMD_INIT_RXQ_V4_IN_LEN; + req.emr_in_length = MC_CMD_INIT_RXQ_V5_IN_LEN; req.emr_out_buf = payload; - req.emr_out_length = MC_CMD_INIT_RXQ_V4_OUT_LEN; + req.emr_out_length = MC_CMD_INIT_RXQ_V5_OUT_LEN; MCDI_IN_SET_DWORD(req, INIT_RXQ_EXT_IN_SIZE, ndescs); MCDI_IN_SET_DWORD(req, INIT_RXQ_EXT_IN_TARGET_EVQ, eep->ee_index); @@ -2787,6 +2787,8 @@ efx_mcdi_init_rxq( MCDI_IN_SET_DWORD(req, INIT_RXQ_V4_IN_BUFFER_SIZE_BYTES, params->buf_size); + MCDI_IN_SET_DWORD(req, INIT_RXQ_V5_IN_RX_PREFIX_ID, params->prefix_id); + dma_addr = MCDI_IN2(req, efx_qword_t, INIT_RXQ_IN_DMA_ADDR); addr = EFSYS_MEM_ADDR(esmp); diff --git a/drivers/common/sfc_efx/base/rhead_rx.c b/drivers/common/sfc_efx/base/rhead_rx.c index 38c905444a..d683f280ce 100644 --- a/drivers/common/sfc_efx/base/rhead_rx.c +++ b/drivers/common/sfc_efx/base/rhead_rx.c @@ -10,6 +10,12 @@ #if EFSYS_OPT_RIVERHEAD +/* + * Maximum number of Rx prefixes supported by Rx prefix choice to be + * returned from firmware. + */ +#define RHEAD_RX_PREFIX_IDS_MAX 16 + /* * Default Rx prefix layout on Riverhead if FW does not support Rx * prefix choice using MC_CMD_GET_RX_PREFIX_ID and query its layout @@ -265,6 +271,312 @@ rhead_rx_qenable( _NOTE(ARGUNUSED(erp)) } +static __checkReturn efx_rc_t +efx_mcdi_get_rx_prefix_ids( + __in efx_nic_t *enp, + __in uint32_t mcdi_fields_mask, + __in unsigned int max_ids, + __out unsigned int *nids, + __out_ecount_part(max_ids, *nids) uint32_t *idsp) +{ + efx_mcdi_req_t req; + EFX_MCDI_DECLARE_BUF(payload, MC_CMD_GET_RX_PREFIX_ID_IN_LEN, + MC_CMD_GET_RX_PREFIX_ID_OUT_LENMAX); + efx_rc_t rc; + uint32_t num; + + req.emr_cmd = MC_CMD_GET_RX_PREFIX_ID; + req.emr_in_buf = payload; + req.emr_in_length = MC_CMD_GET_RX_PREFIX_ID_IN_LEN; + req.emr_out_buf = payload; + req.emr_out_length = MC_CMD_GET_RX_PREFIX_ID_OUT_LENMAX; + + MCDI_IN_SET_DWORD(req, GET_RX_PREFIX_ID_IN_FIELDS, mcdi_fields_mask); + + efx_mcdi_execute(enp, &req); + + if (req.emr_rc != 0) { + rc = req.emr_rc; + goto fail1; + } + + if (req.emr_out_length_used < MC_CMD_GET_RX_PREFIX_ID_OUT_LENMIN) { + rc = EMSGSIZE; + goto fail2; + } + + num = MCDI_OUT_DWORD(req, GET_RX_PREFIX_ID_OUT_NUM_RX_PREFIX_IDS); + + if (req.emr_out_length_used != MC_CMD_GET_RX_PREFIX_ID_OUT_LEN(num)) { + rc = EMSGSIZE; + goto fail3; + } + + *nids = MIN(num, max_ids); + + EFX_STATIC_ASSERT(sizeof (idsp[0]) == + MC_CMD_GET_RX_PREFIX_ID_OUT_RX_PREFIX_ID_LEN); + memcpy(idsp, + MCDI_OUT2(req, uint32_t, GET_RX_PREFIX_ID_OUT_RX_PREFIX_ID), + *nids * sizeof (idsp[0])); + + return (0); + +fail3: + EFSYS_PROBE(fail3); +fail2: + EFSYS_PROBE(fail2); +fail1: + EFSYS_PROBE1(fail1, efx_rc_t, rc); + + return (rc); +} + +static __checkReturn efx_rx_prefix_field_t +efx_mcdi_rx_prefix_field_map(unsigned int mcdi_idx) +{ + static const efx_rx_prefix_field_t efx_mcdi_to_rx_prefix_field[] = { +#define EFX_MCDI_TO_RX_PREFIX_FIELD(_field) \ + [RX_PREFIX_FIELD_INFO_ ## _field] = EFX_RX_PREFIX_FIELD_ ## _field + + EFX_MCDI_TO_RX_PREFIX_FIELD(LENGTH), + EFX_MCDI_TO_RX_PREFIX_FIELD(RSS_HASH_VALID), + EFX_MCDI_TO_RX_PREFIX_FIELD(USER_FLAG), + EFX_MCDI_TO_RX_PREFIX_FIELD(CLASS), + EFX_MCDI_TO_RX_PREFIX_FIELD(PARTIAL_TSTAMP), + EFX_MCDI_TO_RX_PREFIX_FIELD(RSS_HASH), + EFX_MCDI_TO_RX_PREFIX_FIELD(USER_MARK), + EFX_MCDI_TO_RX_PREFIX_FIELD(INGRESS_VPORT), + EFX_MCDI_TO_RX_PREFIX_FIELD(CSUM_FRAME), + EFX_MCDI_TO_RX_PREFIX_FIELD(VLAN_STRIP_TCI), + +#undef EFX_MCDI_TO_RX_PREFIX_FIELD + }; + + if (mcdi_idx >= EFX_ARRAY_SIZE(efx_mcdi_to_rx_prefix_field)) + return (EFX_RX_PREFIX_NFIELDS); + + return (efx_mcdi_to_rx_prefix_field[mcdi_idx]); +} + +static __checkReturn int +efx_rx_prefix_field_map_to_mcdi( + __in efx_rx_prefix_field_t field) +{ + static const int efx_rx_prefix_field_to_mcdi[] = { + [EFX_RX_PREFIX_FIELD_LENGTH] = + EFX_LOW_BIT(MC_CMD_GET_RX_PREFIX_ID_IN_LENGTH), + [EFX_RX_PREFIX_FIELD_ORIG_LENGTH] = -1, + [EFX_RX_PREFIX_FIELD_CLASS] = + EFX_LOW_BIT(MC_CMD_GET_RX_PREFIX_ID_IN_CLASS), + [EFX_RX_PREFIX_FIELD_RSS_HASH] = + EFX_LOW_BIT(MC_CMD_GET_RX_PREFIX_ID_IN_RSS_HASH), + [EFX_RX_PREFIX_FIELD_RSS_HASH_VALID] = + EFX_LOW_BIT(MC_CMD_GET_RX_PREFIX_ID_IN_RSS_HASH_VALID), + [EFX_RX_PREFIX_FIELD_PARTIAL_TSTAMP] = + EFX_LOW_BIT(MC_CMD_GET_RX_PREFIX_ID_IN_PARTIAL_TSTAMP), + [EFX_RX_PREFIX_FIELD_VLAN_STRIP_TCI] = + EFX_LOW_BIT(MC_CMD_GET_RX_PREFIX_ID_IN_VLAN_STRIP_TCI), + [EFX_RX_PREFIX_FIELD_INNER_VLAN_STRIP_TCI] = -1, + [EFX_RX_PREFIX_FIELD_USER_FLAG] = + EFX_LOW_BIT(MC_CMD_GET_RX_PREFIX_ID_IN_USER_FLAG), + [EFX_RX_PREFIX_FIELD_USER_MARK] = + EFX_LOW_BIT(MC_CMD_GET_RX_PREFIX_ID_IN_USER_MARK), + [EFX_RX_PREFIX_FIELD_USER_MARK_VALID] = -1, + [EFX_RX_PREFIX_FIELD_CSUM_FRAME] = + EFX_LOW_BIT(MC_CMD_GET_RX_PREFIX_ID_IN_CSUM_FRAME), + [EFX_RX_PREFIX_FIELD_INGRESS_VPORT] = + EFX_LOW_BIT(MC_CMD_GET_RX_PREFIX_ID_IN_INGRESS_VPORT), + }; + + if (field >= EFX_ARRAY_SIZE(efx_rx_prefix_field_to_mcdi)) + return (-1); + + return (efx_rx_prefix_field_to_mcdi[field]); +} + +static __checkReturn efx_rc_t +efx_rx_prefix_fields_mask_to_mcdi( + __in uint32_t fields_mask, + __out uint32_t *mcdi_fields_maskp) +{ + uint32_t mcdi_fields_mask = 0; + unsigned int i; + + for (i = 0; i < EFX_RX_PREFIX_NFIELDS; ++i) { + if (fields_mask & (1U << i)) { + int mcdi_field = efx_rx_prefix_field_map_to_mcdi(i); + + if (mcdi_field < 0) + return (EINVAL); + + mcdi_fields_mask |= (1U << mcdi_field); + } + } + + *mcdi_fields_maskp = mcdi_fields_mask; + return (0); +} + +static __checkReturn efx_rc_t +efx_mcdi_query_rx_prefix_id( + __in efx_nic_t *enp, + __in uint32_t prefix_id, + __out efx_rx_prefix_layout_t *erplp) +{ + efx_mcdi_req_t req; + EFX_MCDI_DECLARE_BUF(payload, MC_CMD_QUERY_RX_PREFIX_ID_IN_LEN, + MC_CMD_QUERY_RX_PREFIX_ID_OUT_LENMAX); + efx_rc_t rc; + size_t response_len; + const efx_dword_t *resp; + const efx_dword_t *finfo; + unsigned int num_fields; + unsigned int mcdi_field; + efx_rx_prefix_field_t field; + unsigned int i; + + req.emr_cmd = MC_CMD_QUERY_RX_PREFIX_ID; + req.emr_in_buf = payload; + req.emr_in_length = MC_CMD_QUERY_RX_PREFIX_ID_IN_LEN; + req.emr_out_buf = payload; + req.emr_out_length = MC_CMD_QUERY_RX_PREFIX_ID_OUT_LENMAX; + + MCDI_IN_SET_DWORD(req, QUERY_RX_PREFIX_ID_IN_RX_PREFIX_ID, prefix_id); + + efx_mcdi_execute(enp, &req); + + if (req.emr_rc != 0) { + rc = req.emr_rc; + goto fail1; + } + + if (req.emr_out_length_used < MC_CMD_QUERY_RX_PREFIX_ID_OUT_LENMIN) { + rc = EMSGSIZE; + goto fail2; + } + + if (MCDI_OUT_BYTE(req, QUERY_RX_PREFIX_ID_OUT_RESPONSE_TYPE) != + MC_CMD_QUERY_RX_PREFIX_ID_OUT_RESPONSE_TYPE_FIXED) { + rc = ENOTSUP; + goto fail3; + } + + EFX_STATIC_ASSERT(MC_CMD_QUERY_RX_PREFIX_ID_OUT_LENMIN >= + MC_CMD_QUERY_RX_PREFIX_ID_OUT_RESPONSE_OFST); + response_len = req.emr_out_length_used - + MC_CMD_QUERY_RX_PREFIX_ID_OUT_RESPONSE_OFST; + + if (response_len < RX_PREFIX_FIXED_RESPONSE_LENMIN) { + rc = EMSGSIZE; + goto fail4; + } + + resp = MCDI_OUT2(req, efx_dword_t, QUERY_RX_PREFIX_ID_OUT_RESPONSE); + + memset(erplp, 0, sizeof (*erplp)); + erplp->erpl_id = prefix_id; + erplp->erpl_length = + EFX_DWORD_FIELD(*resp, RX_PREFIX_FIXED_RESPONSE_PREFIX_LENGTH_BYTES); + num_fields = + EFX_DWORD_FIELD(*resp, RX_PREFIX_FIXED_RESPONSE_FIELD_COUNT); + + if (response_len < RX_PREFIX_FIXED_RESPONSE_LEN(num_fields)) { + rc = EMSGSIZE; + goto fail5; + } + + finfo = (const efx_dword_t *)((const uint8_t *)resp + + RX_PREFIX_FIXED_RESPONSE_FIELDS_OFST); + + for (i = 0; i < num_fields; ++i, ++finfo) { + mcdi_field = EFX_DWORD_FIELD(*finfo, RX_PREFIX_FIELD_INFO_TYPE); + + field = efx_mcdi_rx_prefix_field_map(mcdi_field); + if (field >= EFX_RX_PREFIX_NFIELDS) + continue; + + erplp->erpl_fields[field].erpfi_offset_bits = + EFX_DWORD_FIELD(*finfo, RX_PREFIX_FIELD_INFO_OFFSET_BITS); + erplp->erpl_fields[field].erpfi_width_bits = + EFX_DWORD_FIELD(*finfo, RX_PREFIX_FIELD_INFO_WIDTH_BITS); + } + + return (0); + +fail5: + EFSYS_PROBE(fail5); +fail4: + EFSYS_PROBE(fail4); +fail3: + EFSYS_PROBE(fail3); +fail2: + EFSYS_PROBE(fail2); +fail1: + EFSYS_PROBE1(fail1, efx_rc_t, rc); + + return (rc); +} + +static __checkReturn efx_rc_t +rhead_rx_choose_prefix_id( + __in efx_nic_t *enp, + __in uint32_t fields_mask, + __out efx_rx_prefix_layout_t *erplp) +{ + efx_rx_prefix_layout_t erpl; + uint32_t prefix_ids[RHEAD_RX_PREFIX_IDS_MAX]; + uint32_t mcdi_fields_mask; + unsigned int num = 0; + unsigned int i; + efx_rc_t rc; + + rc = efx_rx_prefix_fields_mask_to_mcdi(fields_mask, &mcdi_fields_mask); + if (rc != 0) + goto fail1; + + memset(erplp, 0, sizeof (*erplp)); + + rc = efx_mcdi_get_rx_prefix_ids(enp, mcdi_fields_mask, + EFX_ARRAY_SIZE(prefix_ids), &num, prefix_ids); + if (rc == ENOTSUP) { + /* Not supported MCDI, use default prefix ID */ + *erplp = rhead_default_rx_prefix_layout; + goto done; + } + if (rc != 0) + goto fail2; + + if (num == 0) { + rc = ENOTSUP; + goto fail3; + } + + for (i = 0; i < num; ++i) { + rc = efx_mcdi_query_rx_prefix_id(enp, prefix_ids[i], &erpl); + if (rc != 0) + goto fail4; + + /* Choose the smallest prefix which meets our requirements */ + if (i == 0 || erpl.erpl_length < erplp->erpl_length) + *erplp = erpl; + } + +done: + return (0); + +fail4: + EFSYS_PROBE(fail4); +fail3: + EFSYS_PROBE(fail3); +fail2: + EFSYS_PROBE(fail2); +fail1: + EFSYS_PROBE1(fail1, efx_rc_t, rc); + + return (rc); +} + __checkReturn efx_rc_t rhead_rx_qcreate( __in efx_nic_t *enp, @@ -281,6 +593,7 @@ rhead_rx_qcreate( { const efx_nic_cfg_t *encp = efx_nic_cfg_get(enp); efx_mcdi_init_rxq_params_t params; + efx_rx_prefix_layout_t erpl; efx_rc_t rc; _NOTE(ARGUNUSED(id)) @@ -310,6 +623,20 @@ rhead_rx_qcreate( else params.disable_scatter = encp->enc_rx_disable_scatter_supported; + /* + * LENGTH is required in EF100 host interface, as receive events + * do not include the packet length. + * NOTE: Required fields are hard-wired now. Future designs will + * want to allow the client (driver) code to have control over + * which fields are required or may be allow to request so-called + * default Rx prefix (which ID is equal to 0). + */ + if ((rc = rhead_rx_choose_prefix_id(enp, + (1U << EFX_RX_PREFIX_FIELD_LENGTH), &erpl)) != 0) + goto fail3; + + params.prefix_id = erpl.erpl_id; + /* * Ignore EFX_RXQ_FLAG_INNER_CLASSES since in accordance with * EF100 host interface both inner and outer classes are provided @@ -318,15 +645,17 @@ rhead_rx_qcreate( if ((rc = efx_mcdi_init_rxq(enp, ndescs, eep, label, index, esmp, ¶ms)) != 0) - goto fail3; + goto fail4; erp->er_eep = eep; erp->er_label = label; erp->er_buf_size = params.buf_size; - erp->er_prefix_layout = rhead_default_rx_prefix_layout; + erp->er_prefix_layout = erpl; return (0); +fail4: + EFSYS_PROBE(fail4); fail3: EFSYS_PROBE(fail3); fail2: -- 2.17.1