From patchwork Tue Jun 25 21:15:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 167756 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp40276ilk; Tue, 25 Jun 2019 14:15:53 -0700 (PDT) X-Google-Smtp-Source: APXvYqzL5zNYwTDcS+TwM4lHUA75hgBbP/QY4iub1kTL6dLZGmXtwguL9UKl5la7hUcY5kR6HCuH X-Received: by 2002:a1c:ab06:: with SMTP id u6mr255090wme.125.1561497353225; Tue, 25 Jun 2019 14:15:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561497353; cv=none; d=google.com; s=arc-20160816; b=ACt0aD7RAtX+5QZCyTXClZO96leqjDrf3FlmGwSUXfy/F5fT0KerwQJOhDTkO4EcE7 6gs/LGDD+xl/rgRhnVd2m24Kj0l8cCvIpu4xE/L31QL6vb04o0D8eQQAPAlU8O9Mat8q jC9hXiqR20nlSR39mb+xmoH4LbSM1sqTVhmGEl5wkFj2IZQB+RYYq/7EH9XY2OJawNDC 1jdipQOq1nVm//R7ofiQ0527oE8SnvyHcKsctSg94vgdC2KAoI1GXGdRZJFJnKQuSdae xY9XSFO1JGGHPkbi7J9yVKGzu39qocsp9I7XIi2vyWkpeUp+WwOeqM9b6HyllkM4Q5yE b3HA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=Crijhf6udlqXzMyz4gRBZmepexs9+f0+ADdCg58IDL8=; b=BgtSWtg6oBGlYUaSK+aV/teTG+tmxcLPkapQlt04SnaBqbEeXl5b0nV+G0ilmDUFmN V096XbPq5ZPeevTZQl1vBPwbFSK/I05QyM253Vq01cRaKTRI42PUCj1+lGjwexeJMP21 MorRGF2cQXK+p5WExGvs1GigIjHuIbDgpO4ED5uALKqqTbdLf4ZY7n2XmLqFcUIl70T2 mTxtRdtjJL+YSq0//rlbnlHaZYu3rokifcPbuynWSdoy3U1IwyzLDHrI4k9IM8HDkv5V LFTeC0icUjmt+6WRavrcBrMdODl+pmXXaMoGNG3qZRNs0P7gkmAm8sw5YfCNWARF1c3s zZDw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Return-Path: Received: from dpdk.org (dpdk.org. [92.243.14.124]) by mx.google.com with ESMTP id x2si156697wmb.168.2019.06.25.14.15.52; Tue, 25 Jun 2019 14:15:53 -0700 (PDT) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) client-ip=92.243.14.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 343321B9A1; Tue, 25 Jun 2019 23:15:44 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id BAC8A1B993; Tue, 25 Jun 2019 23:15:40 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 03DDBC0A; Tue, 25 Jun 2019 14:15:40 -0700 (PDT) Received: from qc2400f-1.austin.arm.com (qc2400f-1.austin.arm.com [10.118.12.65]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E4F163F7D8; Tue, 25 Jun 2019 14:15:39 -0700 (PDT) From: Honnappa Nagarahalli To: yipeng1.wang@intel.com, sameh.gobriel@intel.com, bruce.richardson@intel.com, pablo.de.lara.guarch@intel.com, honnappa.nagarahalli@arm.com Cc: gavin.hu@arm.com, ruifeng.wang@arm.com, dev@dpdk.org, nd@arm.com, stable@dpdk.org Date: Tue, 25 Jun 2019 16:15:18 -0500 Message-Id: <20190625211520.43181-2-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190625211520.43181-1-honnappa.nagarahalli@arm.com> References: <20190625211520.43181-1-honnappa.nagarahalli@arm.com> Subject: [dpdk-dev] [PATCH 1/3] lib/hash: use ordered loads only if signature matches X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Relaxed signature comparison is done first. Further ordered loads are done only if the signature matches. Any false positives are caught by the full key comparison. Fixes: e605a1d36 ("hash: add lock-free r/w concurrency") Cc: stable@dpdk.org Signed-off-by: Honnappa Nagarahalli Reviewed-by: Gavin Hu Tested-by: Ruifeng Wang --- lib/librte_hash/rte_cuckoo_hash.c | 35 ++++++++++++++++++------------- 1 file changed, 21 insertions(+), 14 deletions(-) -- 2.17.1 Acked-by: Yipeng Wang diff --git a/lib/librte_hash/rte_cuckoo_hash.c b/lib/librte_hash/rte_cuckoo_hash.c index 953928f27..f37f6957d 100644 --- a/lib/librte_hash/rte_cuckoo_hash.c +++ b/lib/librte_hash/rte_cuckoo_hash.c @@ -1188,22 +1188,29 @@ search_one_bucket_lf(const struct rte_hash *h, const void *key, uint16_t sig, struct rte_hash_key *k, *keys = h->key_store; for (i = 0; i < RTE_HASH_BUCKET_ENTRIES; i++) { - key_idx = __atomic_load_n(&bkt->key_idx[i], + /* Signature comparison is done before the acquire-load + * of the key index to achieve better performance. + * Any false positives will be caught in full comparison + * of the key. + */ + if (bkt->sig_current[i] == sig) { + key_idx = __atomic_load_n(&bkt->key_idx[i], __ATOMIC_ACQUIRE); - if (bkt->sig_current[i] == sig && key_idx != EMPTY_SLOT) { - k = (struct rte_hash_key *) ((char *)keys + - key_idx * h->key_entry_size); - pdata = __atomic_load_n(&k->pdata, - __ATOMIC_ACQUIRE); + if (key_idx != EMPTY_SLOT) { + k = (struct rte_hash_key *) ((char *)keys + + key_idx * h->key_entry_size); + pdata = __atomic_load_n(&k->pdata, + __ATOMIC_ACQUIRE); - if (rte_hash_cmp_eq(key, k->key, h) == 0) { - if (data != NULL) - *data = pdata; - /* - * Return index where key is stored, - * subtracting the first dummy index - */ - return key_idx - 1; + if (rte_hash_cmp_eq(key, k->key, h) == 0) { + if (data != NULL) + *data = pdata; + /* + * Return index where key is stored, + * subtracting the first dummy index + */ + return key_idx - 1; + } } } } From patchwork Tue Jun 25 21:15:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 167758 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp40570ilk; Tue, 25 Jun 2019 14:16:09 -0700 (PDT) X-Google-Smtp-Source: APXvYqzI4a5RN/OcvdC/KVyOHLh0lmWSzhz+uNPhlR5bmcmpIlqqFMU9P7frauC+MfKhOf9WSAFP X-Received: by 2002:adf:b1ce:: with SMTP id r14mr189101wra.0.1561497369689; Tue, 25 Jun 2019 14:16:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561497369; cv=none; d=google.com; s=arc-20160816; b=rcVKZvciHfmf586pdtIj9HvbAkGJAxTHodQTb0NyAOilMR8ram13DbEqW7Hs8YbGxi hHqYNQOMudcchRK5BTjugbDqVzccZrpeof6e+//lXjb+aaz1PD8eA+9DsyJIAgbtcixo CRS4qR5eNHnFiCB+woqz8X+M491nZDtZqhMpzH1q7nzUazER89IJETVgSIJQ3E1P/P5R XRpccKywGlEuSzc23J6ULggGsWSWiY/X0pHV1C4H2YQtJKEwpEKSUYJzX3zvFqpc54RL ZnUdkLM/nv7riktJGqXKXCJPXuPFHPtj/GDIeJ6NTOlBguti6TfMPWxD/Mlvboqm2nX7 Gh3A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=e2FB29ie2MxmyFLOrAfpeWR5vZkgOrU0MSm0F4ZuGeU=; b=DFtbgqcG1XaBQ2ZCj61sN+ZxxKsQRFbZrYbYl5jUbLJ61YM2qpUl1xYEM8UjGpAzN2 wTCtJOKz+E66Dn7j/zJwdwjrdXFb2ncjKTxABqehnTiIj7XggUnCN2N5KlulCxBByENC 01myBex2Ed+akTfBuiAUGsceCouFNW1JkUR6Bcrr+MJ9N+JnE560hMI0CGG1NYQiAzK8 Rk7+x8zCmtlrDrhzwCvPpYYrLRR7rcxricr28DvdZAwCLFQjJnkxc9ks1i7YVuMNzLce YIGjVs6JBxe5rYx5TbIrFltCVKcBvhrt072HNKMbOOJIen+2GEFxbbXNAwoicrnz12Zm z8Eg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Return-Path: Received: from dpdk.org (dpdk.org. [92.243.14.124]) by mx.google.com with ESMTP id b141si173395wmd.88.2019.06.25.14.16.09; Tue, 25 Jun 2019 14:16:09 -0700 (PDT) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) client-ip=92.243.14.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6E5B81B9B4; Tue, 25 Jun 2019 23:15:49 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id C1D9E1B99C; Tue, 25 Jun 2019 23:15:40 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1A4B7ED1; Tue, 25 Jun 2019 14:15:40 -0700 (PDT) Received: from qc2400f-1.austin.arm.com (qc2400f-1.austin.arm.com [10.118.12.65]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 04B923F246; Tue, 25 Jun 2019 14:15:40 -0700 (PDT) From: Honnappa Nagarahalli To: yipeng1.wang@intel.com, sameh.gobriel@intel.com, bruce.richardson@intel.com, pablo.de.lara.guarch@intel.com, honnappa.nagarahalli@arm.com Cc: gavin.hu@arm.com, ruifeng.wang@arm.com, dev@dpdk.org, nd@arm.com, stable@dpdk.org Date: Tue, 25 Jun 2019 16:15:19 -0500 Message-Id: <20190625211520.43181-3-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190625211520.43181-1-honnappa.nagarahalli@arm.com> References: <20190625211520.43181-1-honnappa.nagarahalli@arm.com> Subject: [dpdk-dev] [PATCH 2/3] lib/hash: load pData after full key compare X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" When a hash entry is added, there are 2 sets of stores. 1) The application writes its data to memory (whose address is provided in rte_hash_add_key_with_hash_data API (or NULL)) 2) The rte_hash library writes to its own internal data structures; key store entry and the hash table. The only ordering requirement between these 2 is that - the store to the application data must complete before the store to key_index. There are no ordering requirements between the stores to the key/signature and store to application data. The synchronization point for application data can be any point between the 'store to application data' and 'store to the key_index'. So, pData should not be a guard variable for the data in hash table. It should be a guard variable only for the application data written to the memory location pointed by pData. Hence, pData can be loaded after full key comparison. Fixes: e605a1d36 ("hash: add lock-free r/w concurrency") Cc: stable@dpdk.org Signed-off-by: Honnappa Nagarahalli Reviewed-by: Gavin Hu Tested-by: Ruifeng Wang --- lib/librte_hash/rte_cuckoo_hash.c | 67 +++++++++++++++---------------- 1 file changed, 32 insertions(+), 35 deletions(-) -- 2.17.1 Acked-by: Yipeng Wang diff --git a/lib/librte_hash/rte_cuckoo_hash.c b/lib/librte_hash/rte_cuckoo_hash.c index f37f6957d..077328fed 100644 --- a/lib/librte_hash/rte_cuckoo_hash.c +++ b/lib/librte_hash/rte_cuckoo_hash.c @@ -649,9 +649,11 @@ search_and_update(const struct rte_hash *h, void *data, const void *key, k = (struct rte_hash_key *) ((char *)keys + bkt->key_idx[i] * h->key_entry_size); if (rte_hash_cmp_eq(key, k->key, h) == 0) { - /* 'pdata' acts as the synchronization point - * when an existing hash entry is updated. - * Key is not updated in this case. + /* The store to application data at *data + * should not leak after the store to pdata + * in the key store. i.e. pdata is the guard + * variable. Release the application data + * to the readers. */ __atomic_store_n(&k->pdata, data, @@ -711,11 +713,10 @@ rte_hash_cuckoo_insert_mw(const struct rte_hash *h, /* Check if slot is available */ if (likely(prim_bkt->key_idx[i] == EMPTY_SLOT)) { prim_bkt->sig_current[i] = sig; - /* Key can be of arbitrary length, so it is - * not possible to store it atomically. - * Hence the new key element's memory stores - * (key as well as data) should be complete - * before it is referenced. + /* Store to signature and key should not + * leak after the store to key_idx. i.e. + * key_idx is the guard variable for signature + * and key. */ __atomic_store_n(&prim_bkt->key_idx[i], new_idx, @@ -990,17 +991,15 @@ __rte_hash_add_key_with_hash(const struct rte_hash *h, const void *key, new_k = RTE_PTR_ADD(keys, (uintptr_t)slot_id * h->key_entry_size); new_idx = (uint32_t)((uintptr_t) slot_id); - /* Copy key */ - memcpy(new_k->key, key, h->key_len); - /* Key can be of arbitrary length, so it is not possible to store - * it atomically. Hence the new key element's memory stores - * (key as well as data) should be complete before it is referenced. - * 'pdata' acts as the synchronization point when an existing hash - * entry is updated. + /* The store to application data (by the application) at *data should + * not leak after the store of pdata in the key store. i.e. pdata is + * the guard variable. Release the application data to the readers. */ __atomic_store_n(&new_k->pdata, data, __ATOMIC_RELEASE); + /* Copy key */ + memcpy(new_k->key, key, h->key_len); /* Find an empty slot and insert */ ret = rte_hash_cuckoo_insert_mw(h, prim_bkt, sec_bkt, key, data, @@ -1064,8 +1063,10 @@ __rte_hash_add_key_with_hash(const struct rte_hash *h, const void *key, /* Check if slot is available */ if (likely(cur_bkt->key_idx[i] == EMPTY_SLOT)) { cur_bkt->sig_current[i] = short_sig; - /* Store to signature should not leak after - * the store to key_idx + /* Store to signature and key should not + * leak after the store to key_idx. i.e. + * key_idx is the guard variable for signature + * and key. */ __atomic_store_n(&cur_bkt->key_idx[i], new_idx, @@ -1087,8 +1088,9 @@ __rte_hash_add_key_with_hash(const struct rte_hash *h, const void *key, bkt_id = (uint32_t)((uintptr_t)ext_bkt_id) - 1; /* Use the first location of the new bucket */ (h->buckets_ext[bkt_id]).sig_current[0] = short_sig; - /* Store to signature should not leak after - * the store to key_idx + /* Store to signature and key should not leak after + * the store to key_idx. i.e. key_idx is the guard variable + * for signature and key. */ __atomic_store_n(&(h->buckets_ext[bkt_id]).key_idx[0], new_idx, @@ -1184,7 +1186,6 @@ search_one_bucket_lf(const struct rte_hash *h, const void *key, uint16_t sig, { int i; uint32_t key_idx; - void *pdata; struct rte_hash_key *k, *keys = h->key_store; for (i = 0; i < RTE_HASH_BUCKET_ENTRIES; i++) { @@ -1199,12 +1200,13 @@ search_one_bucket_lf(const struct rte_hash *h, const void *key, uint16_t sig, if (key_idx != EMPTY_SLOT) { k = (struct rte_hash_key *) ((char *)keys + key_idx * h->key_entry_size); - pdata = __atomic_load_n(&k->pdata, - __ATOMIC_ACQUIRE); if (rte_hash_cmp_eq(key, k->key, h) == 0) { - if (data != NULL) - *data = pdata; + if (data != NULL) { + *data = __atomic_load_n( + &k->pdata, + __ATOMIC_ACQUIRE); + } /* * Return index where key is stored, * subtracting the first dummy index @@ -1902,7 +1904,6 @@ __rte_hash_lookup_bulk_lf(const struct rte_hash *h, const void **keys, uint32_t prim_hitmask[RTE_HASH_LOOKUP_BULK_MAX] = {0}; uint32_t sec_hitmask[RTE_HASH_LOOKUP_BULK_MAX] = {0}; struct rte_hash_bucket *cur_bkt, *next_bkt; - void *pdata[RTE_HASH_LOOKUP_BULK_MAX]; uint32_t cnt_b, cnt_a; /* Prefetch first keys */ @@ -2004,10 +2005,6 @@ __rte_hash_lookup_bulk_lf(const struct rte_hash *h, const void **keys, (const char *)h->key_store + key_idx * h->key_entry_size); - if (key_idx != EMPTY_SLOT) - pdata[i] = __atomic_load_n( - &key_slot->pdata, - __ATOMIC_ACQUIRE); /* * If key index is 0, do not compare key, * as it is checking the dummy slot @@ -2016,7 +2013,9 @@ __rte_hash_lookup_bulk_lf(const struct rte_hash *h, const void **keys, !rte_hash_cmp_eq( key_slot->key, keys[i], h)) { if (data != NULL) - data[i] = pdata[i]; + data[i] = __atomic_load_n( + &key_slot->pdata, + __ATOMIC_ACQUIRE); hits |= 1ULL << i; positions[i] = key_idx - 1; @@ -2038,10 +2037,6 @@ __rte_hash_lookup_bulk_lf(const struct rte_hash *h, const void **keys, (const char *)h->key_store + key_idx * h->key_entry_size); - if (key_idx != EMPTY_SLOT) - pdata[i] = __atomic_load_n( - &key_slot->pdata, - __ATOMIC_ACQUIRE); /* * If key index is 0, do not compare key, * as it is checking the dummy slot @@ -2051,7 +2046,9 @@ __rte_hash_lookup_bulk_lf(const struct rte_hash *h, const void **keys, !rte_hash_cmp_eq( key_slot->key, keys[i], h)) { if (data != NULL) - data[i] = pdata[i]; + data[i] = __atomic_load_n( + &key_slot->pdata, + __ATOMIC_ACQUIRE); hits |= 1ULL << i; positions[i] = key_idx - 1; From patchwork Tue Jun 25 21:15:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 167757 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp40446ilk; Tue, 25 Jun 2019 14:16:02 -0700 (PDT) X-Google-Smtp-Source: APXvYqwQWEDhFNayGhTHKeR+mhUiFXO5qygtGKFnTrpsLJ8x4S7jAoluIey5HTuvzoetiaEIavzR X-Received: by 2002:a1c:5f87:: with SMTP id t129mr298987wmb.150.1561497362734; Tue, 25 Jun 2019 14:16:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561497362; cv=none; d=google.com; s=arc-20160816; b=Tvv2PW0ZOPV8RGb9Zk+speUaa7IDsuao+WrQAwn/FH/B5pQ5lzTAPt9pwtYyVTSGCt y+DZfF0J5WEcRbmZdSn9SUH7fHYL0TmlhR4LoT93qoaokigAmCh7p+4YZ/nrv4PdamHg 1uxfK1ij7WMHhIBMTsCi46DK/0H9roH5rRUmWvkJw+zAGqXv0/iunU7U+V5ll/PVg7wN ee0MtR+n2wGj/w4CQedY+VxTu7Dksiy/gbhsuN7z9M8gLSqVDX559Vn/bB9cboxuOuys p29pEQSmrL9nCorE9Um5s7A9dkgqLhhAx/ntlp0F484clmXauAs3lC36YGspTrUl1gp9 Emig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=49OU9oztnzmhPSrA/rdADjzSaSU9BFWOV9g8IWmdHEY=; b=N8fvupmJfFOaCOSr3PQpxCkxsNZsP4+kJhzlhM1xn2kMGHt4I72rVhOrzhsRzx+Vbb sr5EoD0gYyAPFT6oRga6gBwHY38PB9IbA/jvySgpq07+H4cX4+lCmtQ6NTJQY3DAP+eY ttGX9xPTkwbqaJcxdtK4kuf7DYEKY6+QvWwt8/oMJgHiYDnlemTmvdCveLV/gkmxbYib oHKztLLQHOEauoVSYv9iW5xan9jyUgMSsmiHqoEUFm1kjlq5TnALJt5KfCN0WZTfRNYY u4vmnCIpUfonbeBQ69/Xzy6TrkGNppAPKzgzK6Y1DuzSn3LDSmtK90nkenroDMESyUWq fW7w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Return-Path: Received: from dpdk.org (dpdk.org. [92.243.14.124]) by mx.google.com with ESMTP id k8si1174983wrx.382.2019.06.25.14.16.02; Tue, 25 Jun 2019 14:16:02 -0700 (PDT) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) client-ip=92.243.14.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E76771B9AF; Tue, 25 Jun 2019 23:15:46 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id C4C581B99D; Tue, 25 Jun 2019 23:15:40 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2AD89106F; Tue, 25 Jun 2019 14:15:40 -0700 (PDT) Received: from qc2400f-1.austin.arm.com (qc2400f-1.austin.arm.com [10.118.12.65]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 195F73F7D8; Tue, 25 Jun 2019 14:15:40 -0700 (PDT) From: Honnappa Nagarahalli To: yipeng1.wang@intel.com, sameh.gobriel@intel.com, bruce.richardson@intel.com, pablo.de.lara.guarch@intel.com, honnappa.nagarahalli@arm.com Cc: gavin.hu@arm.com, ruifeng.wang@arm.com, dev@dpdk.org, nd@arm.com, stable@dpdk.org Date: Tue, 25 Jun 2019 16:15:20 -0500 Message-Id: <20190625211520.43181-4-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190625211520.43181-1-honnappa.nagarahalli@arm.com> References: <20190625211520.43181-1-honnappa.nagarahalli@arm.com> Subject: [dpdk-dev] [PATCH 3/3] lib/hash: adjust tbl_chng_cnt position X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" tbl_chng_cnt is one of the first elements of the structure used in the lookup. Move it to the beginning of the cache line to gain performance. Fixes: e605a1d36 ("hash: add lock-free r/w concurrency") Cc: stable@dpdk.org Signed-off-by: Honnappa Nagarahalli Reviewed-by: Gavin Hu Tested-by: Ruifeng Wang --- lib/librte_hash/rte_cuckoo_hash.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) -- 2.17.1 diff --git a/lib/librte_hash/rte_cuckoo_hash.h b/lib/librte_hash/rte_cuckoo_hash.h index fb19bb27d..af6451b5c 100644 --- a/lib/librte_hash/rte_cuckoo_hash.h +++ b/lib/librte_hash/rte_cuckoo_hash.h @@ -170,7 +170,9 @@ struct rte_hash { /* Fields used in lookup */ - uint32_t key_len __rte_cache_aligned; + uint32_t *tbl_chng_cnt __rte_cache_aligned; + /**< Indicates if the hash table changed from last read. */ + uint32_t key_len; /**< Length of hash key. */ uint8_t hw_trans_mem_support; /**< If hardware transactional memory is used. */ @@ -218,8 +220,6 @@ struct rte_hash { * is piggy-backed to freeing of the key index. */ uint32_t *ext_bkt_to_free; - uint32_t *tbl_chng_cnt; - /**< Indicates if the hash table changed from last read. */ } __rte_cache_aligned; struct queue_node {