From patchwork Thu May 19 13:02:35 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Forrest Shi X-Patchwork-Id: 68106 Delivered-To: patch@linaro.org Received: by 10.140.92.199 with SMTP id b65csp3048156qge; Thu, 19 May 2016 00:35:20 -0700 (PDT) X-Received: by 10.55.22.5 with SMTP id g5mr12966068qkh.101.1463643320712; Thu, 19 May 2016 00:35:20 -0700 (PDT) Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id 137si11040864qhr.116.2016.05.19.00.35.20; Thu, 19 May 2016 00:35:20 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=pass (p=NONE dis=NONE) header.from=linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 2E3B3617EF; Thu, 19 May 2016 07:35:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: * X-Spam-Status: No, score=1.1 required=5.0 tests=BAYES_00, DATE_IN_FUTURE_03_06, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 3E57F61645; Thu, 19 May 2016 07:35:11 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id A82956174B; Thu, 19 May 2016 07:34:54 +0000 (UTC) Received: from na01-bl2-obe.outbound.protection.outlook.com (mail-bl2on0089.outbound.protection.outlook.com [65.55.169.89]) by lists.linaro.org (Postfix) with ESMTPS id 6345561645 for ; Thu, 19 May 2016 07:32:52 +0000 (UTC) Received: from BN3PR0301CA0029.namprd03.prod.outlook.com (10.160.180.167) by DM2PR03MB477.namprd03.prod.outlook.com (10.141.85.19) with Microsoft SMTP Server (TLS) id 15.1.497.12; Thu, 19 May 2016 07:32:52 +0000 Received: from BL2FFO11FD020.protection.gbl (2a01:111:f400:7c09::116) by BN3PR0301CA0029.outlook.office365.com (2a01:111:e400:4000::39) with Microsoft SMTP Server (TLS) id 15.1.497.12 via Frontend Transport; Thu, 19 May 2016 07:32:51 +0000 Received-SPF: SoftFail (protection.outlook.com: domain of transitioning linaro.org discourages use of 192.88.168.50 as permitted sender) Received: from tx30smr01.am.freescale.net (192.88.168.50) by BL2FFO11FD020.mail.protection.outlook.com (10.173.161.38) with Microsoft SMTP Server (TLS) id 15.1.492.8 via Frontend Transport; Thu, 19 May 2016 07:32:50 +0000 Received: from netperf2.ap.freescale.net ([10.232.133.164]) by tx30smr01.am.freescale.net (8.14.3/8.14.0) with ESMTP id u4J7WlNR014483; Thu, 19 May 2016 00:32:49 -0700 From: To: Date: Thu, 19 May 2016 18:32:35 +0530 Message-ID: <1463662955-1241-1-git-send-email-forrest.shi@linaro.org> X-Mailer: git-send-email 2.8.2 X-EOPAttributedMessage: 0 X-Matching-Connectors: 131081167710934857; (91ab9b29-cfa4-454e-5278-08d120cd25b8); () X-Forefront-Antispam-Report: CIP:192.88.168.50; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(2980300002)(189002)(199003)(9170700003)(189998001)(4326007)(110136002)(19580395003)(36756003)(48376002)(6806005)(92566002)(104016004)(2906002)(50226002)(8676002)(8936002)(77096005)(50986999)(586003)(105596002)(5008740100001)(86152002)(1220700001)(5003940100001)(2876002)(106466001)(19580405001)(87936001)(33646002)(86362001)(229853001)(2351001)(81166006)(50466002); DIR:OUT; SFP:1101; SCL:1; SRVR:DM2PR03MB477; H:tx30smr01.am.freescale.net; FPR:; SPF:SoftFail; MLV:sfv; MX:1; A:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BL2FFO11FD020; 1:xZBDnI2u0WqCZmU0701M0DChF3NrBysxfNhzAYuTqDVNLHst3BaZTbNw9ktMigP1fYITkbnrkjo31GV9hm9RGtewRt4u5i/mD14QL8g1Xbl4eC4jjsUynOjZ700fCzoFdkXg+DbZ1GRHrgTa79clF5aJVrZX8nSUNE6GsJhn5d6ovRQErUQJVDcQb2keR8VEM92wr/zlzHwvEZ0pX9pZQC9uzOgCHE7YlLwilAGqRQgUFgGl98fR/pQgZahFXWwc9rBe8H+Jt/jh1fIia7RClgit9MijC8MIccZfOFGr8IxLs/69kevCRlFlS45CoXe+FFHP+9F94DYLVujKe5t/yR4bC6O5iKfKY+UuMyWFzfgbezE9BcyspBFtvvVwaYkx3UUw7sUrLrjROf5XgT7prJ/65xdnhzdKYGsgTb9NgdKx4wnwCxotzt5K44rS16sOflrG874BmrIRXGU082oIGnS1vI3Do74Fg4K8qJgWxw7OhPC//635Ff4eVIqy9EKBL393Jmd8+qK48yrOHjyH5Up8n/5VVrPH+23M5rlzSfdAaiGECVgCOxsjbUwXwkhyyXoipmft5IhLLKeUM+bm8Y4HqxzhCJCavwvaNJyjj1Jt6iPWYSOXC+qnwWXdvsDF MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 25d0791f-1319-4e53-f5ea-08d37fb7c89b X-Microsoft-Exchange-Diagnostics: 1; DM2PR03MB477; 2:cHS+67P1Q6hozyAoEM8bfry1CEodQqxR4S61u1QiUC40qlDeWiT6S6Vr1FLs13CEWo3huzsjndUrIUsOLf90MynYZFPvFDDoj+NiWPGJT7mxZW8C6rL8lnV+S108ICXvbhrN7kWl8c01aXsghRlmsDnZ67JH45ylicKnb7Xxtu7jys/jW/4NPxH+TPVWN7cr; 3:vnpC//oYtdAJwFKU8gQ5GGiSBx+svqt01Rq4RpWyfj8eXNfqNg74SaX3QjijRdcS5ji0/r25HGvxK9eakb+m8WoXSRxG46FKyXWjlnoySPDklwMZed0lc/ilXFbJb8W17y6NAWXRNyFVOKLAtbDITw2U3OOQUFLojsoTX4gdZ+qRwEyUtvhT9jKFE+kClEjWGfy7MStcCWlCepGFD+G6ge1KkMnHbNi8Zg1zAUfbZ90=; 25:+whCfShlNqdOJ5gFrDaM7RQvwdMnDYTdI4ZLLIMyiVkDwhblzc4VgWCREO3veXwMXctg4yKKeDd3v/QMKom4qCYKXR4s+g4qEDYrWktuqUu0eegsSo/SlC5MCK0uO1cYga6uFQCiqdLTmXzdzwwBLgpyFLe4J8Cg2+d96RQIahAgrvih0TvSnD3YLu7BgKJewoHDK3ldvDuQRA+neAapdHNOUmy/YBGCFvS+MXqFw0EKs8MtULClLrcR8CRKwM5WVUMmG22SQoKCKTj9iiehK8UA21jj7g9bJfnRMiw7BdAFelCJh1ZhiG4tU5rTbNquNgdk3jLwMZr2jSjGWsI9MkffnvOwFqRQYT5FxE3NAUlsR5D04rnJt0zIiuqrhfs1zaM6Ja8b/mtLd/yiF9UrzS+sGl49E9e5UOXVCuzriy4= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:DM2PR03MB477; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(13024025)(13023025)(13018025)(13017025)(13015025)(5005006)(8121501046)(3002001)(10201501046)(6055026); SRVR:DM2PR03MB477; BCL:0; PCL:0; RULEID:(400006); SRVR:DM2PR03MB477; X-Microsoft-Exchange-Diagnostics: 1; DM2PR03MB477; 4:EF2WBI4+PVDG+g8hvZtyfTcUheCG/pLPcNsAx/WaoxVktYi4R7BjRDdgMDRJlRed/RcmLAW5y45XXuqMrQyK2kTCn3Y4QmDQ+B8yPYRNEI3nu6/gfxRWbieUf95wkoPKDBqKS+izRIRiLeacKwwtRJiFy7tIZ0gh1amC5Fy4RmKb7FMaxg323xstG1ROpCREg5AN85AszE2AHFPwIknCbOT7YhiJt2oyL6GWz5z7/27D43nwZUdt8vlvxyxJa5wNM8Uk5TLjplgUgyiBeb7ysvGyNlo6BrboRKA9Y8+8hEnVL4FaXORsFRIbWFmA8bsdOVoKncTR1e2S3pP1C//08kExde4BCNdncByGz/g5f27v+yam6sh/fGZeQ8TbfeVK+YXl697MR74icsXafRnlVIzX3bhizVkWjbQMQlbLRYLlFLd5H6uaFGB9zKCrpmNCBW2xn/+mGeNOKhjOR6vqY6jbIHV37Ec4Y71WUvw32D0= X-Forefront-PRVS: 094700CA91 X-Microsoft-Exchange-Diagnostics: 1; DM2PR03MB477; 23:Zd2N4PtLqE94OP1jf01gCwCSvLbP9jbUmGO4JXIRWg64tic74zYSiiTE9yoOou4eLxgMrCabhw9re8F2p3fFsl+HvABCKk6MX7iI2svBKc0GBOsHbZc3udsh7KsgeSkvG7r6JYcP/bkSGk0m5Ws45gU8wpl1qnjCgyBhNlSsY+Zme4+1CcKo7r3dNrE3RC2rC+xGXo5OlXfkDR77ydXrZ91JsHPjsLFOraj9M8LTW6t3jg2Z7jKzHQDwqPVc7XFEG77wDtHw3cCPtksdkr9yWcNydFWhohR/393mQAVG1z4DFRBTeN5KSAhpuMv3mOEOIOAmZFXkjDZs5oC5tHurmXEyv5PJzXJ4CHozQsJCPHDV5nT8RmcfzkSMr0tM75/R8XVuX/S1xr4dwc5uLSOikRkDX3p/VoWWANM9FIQij1CXDWuDnnsMkXSk4bv7L7StS+eejDmdtAKgWEmMJYm0i2iGbYW/gjoH6wTLRejWYyk0elrd6uOHnekknkGJ7yF15Ypgpy+dtemccC49FtWcN0te+3mPVce5XztMnWwlBElQ4rD2wDrJ14U4P8yOgklz+dpmrFbpqXX6mj8sfk6D/YzRNUsiae+8V3hgfJ+3stsNoXDHiQwxfYGTFFNrkr42lmDxYOG0YODN7DGoBpOngk2BPmXwzJK/ABO8YVkxAebDBIG6inIVW3MSZZL/3IkKbHt0mtEtY2nK7QHlSoyuzbW8a86+KJSRDtYQBWhYEMvdKPV0bIzpAHloqstRMs4HDvBiUelErTp890GwYaLGQ/CSoVSZscQcBhGAQTcTw7ZziYyZvM+A2hj3UXEJ362Ixv/73pGJFAfqqRSRm/jIVZJC31I9nKjvbiymgM2aeXHSGJrLTvXY1qC7A+TtI9iu4CfOqJlDZgUsQQNGa7KiX7J+vtz3Xh/s6wEFNMwH6YkUwg7X/tbazowrPpZkb99p X-Microsoft-Exchange-Diagnostics: 1; DM2PR03MB477; 5:cnd59YcKkonoO6SiupFvOcfQnRbYhyLKfmNAxI+8xCiPJUnK+e8HzsyLnSkRX72GCETrxdmhf/p3pqOp/6uCwpSw6cNAJI4iPmZuMfuqY6lCqwwlFey+yvGYq5/uxmgANEb5WP4tN1/K5thNJlV7GSB0Y9PI3HDnqECjO10vkpo=; 24:xGg46uHMFufsQ2U5ykCTLFhujJSg0Sv6vIV0irMhzj9/9ZyC3dHeE33frIx4QwN3+ReNBo5wP4tVY8rs20aOt2PyRAfkS+TziTIXlhMolVU=; 7:7zwaawXWAZB+SEnDAGOL+p7wRXRuEHwdjK8IN3JKFfJtaQFNWN6sI6u5sjscTXJG67wv++AFiEWEvQQKGzmM2VgbjHpnt+Jf+bobCPfwfGXZqLy8ZjVKU+u4Knfz9je1IulBRSZdRO5Ba/YzZPckncrJXq0ahuM0YkPPNobxrMgKUwKzRWLAFKTEOxhX+ZAT SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 May 2016 07:32:50.9062 (UTC) X-MS-Exchange-CrossTenant-Id: 5afe0b00-7697-4969-b663-5eab37d5f47e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5afe0b00-7697-4969-b663-5eab37d5f47e; Ip=[192.88.168.50]; Helo=[tx30smr01.am.freescale.net] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM2PR03MB477 X-Topics: patch Subject: [lng-odp] [PATCH 2/5] example:l3fwd: add forward cache X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Xuelin Shi add a lookup cache based on flow hash before lookup fwd_db, lookup the cache first Signed-off-by: Xuelin Shi --- example/l3fwd/odp_l3fwd.c | 27 +++++++-- example/l3fwd/odp_l3fwd_db.c | 140 ++++++++++++++++++++++++++++++++++++++++--- example/l3fwd/odp_l3fwd_db.h | 10 ++-- 3 files changed, 159 insertions(+), 18 deletions(-) diff --git a/example/l3fwd/odp_l3fwd.c b/example/l3fwd/odp_l3fwd.c index 704d33e..5f8eaa0 100644 --- a/example/l3fwd/odp_l3fwd.c +++ b/example/l3fwd/odp_l3fwd.c @@ -11,6 +11,8 @@ #include #include #include +#include +#include #include "odp_l3fwd_db.h" @@ -105,20 +107,33 @@ static void *run_worker(void *arg ODP_UNUSED) odp_packet_t pkt = pkt_tbl[i]; odph_ethhdr_t *eth; odph_ipv4hdr_t *ip; + odph_udphdr_t *udp; uint32_t len; - uint32_t dst_ip; fwd_db_entry_t *entry; odp_pktout_queue_t outq; + ipv4_tuple5_t key; - if (odp_unlikely(!odp_packet_has_l3(pkt))) { - printf("warning: packet has no ip header\n"); + if (odp_unlikely(!odp_packet_has_ipv4(pkt))) { + printf("warning: packet has no ipv4 header\n"); return NULL; } /*TODO: ipv6 need to be done */ ip = (odph_ipv4hdr_t *)odp_packet_l3_ptr(pkt, &len); - dst_ip = odp_be_to_cpu_32(ip->dst_addr); - entry = find_fwd_db_entry(dst_ip); + key.dst_ip = odp_be_to_cpu_32(ip->dst_addr); + key.src_ip = odp_be_to_cpu_32(ip->src_addr); + key.proto = ip->proto; + if (odp_packet_has_udp(pkt) || + odp_packet_has_tcp(pkt)) { + /* UDP or TCP*/ + void *ptr = odp_packet_l4_ptr(pkt, NULL); + + udp = (odph_udphdr_t *)ptr; + key.src_port = odp_be_to_cpu_16(udp->src_port); + key.dst_port = odp_be_to_cpu_16(udp->dst_port); + } + + entry = find_fwd_db_entry(&key); if (!entry) { pkt_tbl_drop[need_to_drop] = pkt; need_to_drop++; @@ -130,6 +145,8 @@ static void *run_worker(void *arg ODP_UNUSED) return NULL; } + ip->ttl--; + ip->chksum = odph_ipv4_csum_update(pkt); eth = (odph_ethhdr_t *)odp_packet_l2_ptr(pkt, NULL); memcpy(eth->src.addr, entry->src_mac, ODPH_ETHADDR_LEN); memcpy(eth->dst.addr, entry->dst_mac, ODPH_ETHADDR_LEN); diff --git a/example/l3fwd/odp_l3fwd_db.c b/example/l3fwd/odp_l3fwd_db.c index 2140cca..7fde4ae 100644 --- a/example/l3fwd/odp_l3fwd_db.c +++ b/example/l3fwd/odp_l3fwd_db.c @@ -22,15 +22,14 @@ * Compute hash value from a flow */ static inline -uint64_t odp_l3fwd_calc_hash(ipv4_tuple5_t *flow) +uint64_t odp_l3fwd_calc_hash(ipv4_tuple5_t *key) { uint64_t l4_ports = 0; - ipv4_tuple5_t key; + uint32_t dst_ip, src_ip; - key = *flow; - - key.dst_ip += JHASH_GOLDEN_RATIO; - ODP_BJ3_MIX(key.src_ip, key.dst_ip, l4_ports); + src_ip = key->src_ip; + dst_ip = key->dst_ip + JHASH_GOLDEN_RATIO; + ODP_BJ3_MIX(src_ip, dst_ip, l4_ports); return l4_ports; } @@ -157,6 +156,118 @@ char *mac_addr_str(char *b, uint8_t *mac) return b; } +/** + * Flow cache table entry + */ +typedef struct flow_entry_s { + ipv4_tuple5_t key; /**< match key */ + struct flow_entry_s *next; /**< next entry on the list */ + fwd_db_entry_t *fwd_entry; /**< entry info in db */ +} flow_entry_t; + +/** + * Flow cache table bucket + */ +typedef struct flow_bucket_s { + odp_spinlock_t lock; /**< Bucket lock*/ + flow_entry_t *next; /**< Pointer to first flow entry in bucket*/ +} flow_bucket_t; + +/** + * Flow hash table, fast lookup cache + */ +typedef struct flow_table_s { + flow_bucket_t *bucket; + uint32_t count; +} flow_table_t; + +static flow_table_t fwd_lookup_cache; + +static inline +void init_fwd_cache(void) +{ + odp_shm_t hash_shm; + flow_bucket_t *bucket; + uint32_t bucket_count; + uint32_t i; + + bucket_count = ODP_DEF_BUCKET_COUNT; + + /*Reserve memory for Routing hash table*/ + hash_shm = odp_shm_reserve("route_table", + sizeof(flow_bucket_t) * bucket_count, + ODP_CACHE_LINE_SIZE, 0); + + bucket = odp_shm_addr(hash_shm); + if (!bucket) { + EXAMPLE_ERR("Error: shared mem alloc failed.\n"); + exit(-1); + } + + fwd_lookup_cache.bucket = bucket; + fwd_lookup_cache.count = bucket_count; + + /*Inialize Locks*/ + for (i = 0; i < bucket_count; i++) { + bucket = &fwd_lookup_cache.bucket[i]; + odp_spinlock_init(&bucket->lock); + bucket->next = NULL; + } +} + +static inline +int match_key_flow(ipv4_tuple5_t *key, flow_entry_t *flow) +{ + if (key->src_ip == flow->key.src_ip && + key->dst_ip == flow->key.dst_ip && + key->src_port == flow->key.src_port && + key->dst_port == flow->key.dst_port && + key->proto == flow->key.proto) + return 1; + + return 0; +} + +static inline +flow_entry_t *lookup_fwd_cache(ipv4_tuple5_t *key, flow_bucket_t *bucket) +{ + flow_entry_t *rst; + + for (rst = bucket->next; rst != NULL; rst = rst->next) { + if (match_key_flow(key, rst)) + break; + } + + return rst; +} + +static inline +flow_entry_t *insert_fwd_cache(ipv4_tuple5_t *key, + flow_bucket_t *bucket, + fwd_db_entry_t *entry) +{ + flow_entry_t *flow; + + flow = lookup_fwd_cache(key, bucket); + if (flow) + return flow; + + flow = malloc(sizeof(flow_entry_t)); + flow->key = *key; + flow->fwd_entry = entry; + + odp_spinlock_lock(&bucket->lock); + if (!bucket->next) { + bucket->next = flow; + } else { + flow->next = bucket->next; + bucket->next = flow; + } + odp_spinlock_unlock(&bucket->lock); + + return flow; +} + /** Global pointer to fwd db */ fwd_db_t *fwd_db; @@ -176,6 +287,8 @@ void init_fwd_db(void) exit(EXIT_FAILURE); } memset(fwd_db, 0, sizeof(*fwd_db)); + + init_fwd_cache(); } int create_fwd_db_entry(char *input) @@ -287,12 +400,23 @@ void dump_fwd_db(void) dump_fwd_db_entry(entry); } -fwd_db_entry_t *find_fwd_db_entry(uint32_t dst_ip) +fwd_db_entry_t *find_fwd_db_entry(ipv4_tuple5_t *key) { fwd_db_entry_t *entry; + flow_entry_t *flow; + flow_bucket_t *bucket; + uint64_t hash; + + /* first find in cache */ + hash = odp_l3fwd_calc_hash(key); + hash &= fwd_lookup_cache.count - 1; + bucket = &fwd_lookup_cache.bucket[hash]; + flow = lookup_fwd_cache(key, bucket); + if (flow) + return flow->fwd_entry; for (entry = fwd_db->list; NULL != entry; entry = entry->next) - if (entry->subnet.addr == (dst_ip & entry->subnet.mask)) + if (entry->subnet.addr == (key->dst_ip & entry->subnet.mask)) break; return entry; diff --git a/example/l3fwd/odp_l3fwd_db.h b/example/l3fwd/odp_l3fwd_db.h index bd27b2a..c632f5b 100644 --- a/example/l3fwd/odp_l3fwd_db.h +++ b/example/l3fwd/odp_l3fwd_db.h @@ -21,12 +21,12 @@ extern "C" { /** * Default number of flows */ -#define ODP_MAX_FLOW_COUNT 100000 +#define ODP_DEF_FLOW_COUNT 100000 /** * Default Hash bucket number */ -#define ODP_MAX_BUCKET_COUNT (ODP_MAX_FLOW_COUNT / 8) +#define ODP_DEF_BUCKET_COUNT (ODP_DEF_FLOW_COUNT / 8) /** * Hash calculation utility @@ -75,7 +75,7 @@ typedef struct fwd_db_entry_s { } fwd_db_entry_t; /** - * Forwarding data base hash structure + * Forwarding data base */ typedef struct fwd_db_s { uint32_t index; /**< Next available entry */ @@ -124,11 +124,11 @@ void dump_fwd_db(void); /** * Find a matching forwarding database entry * - * @param dst_ip Destination IPv4 address + * @param key ipv4 tuple * * @return pointer to forwarding DB entry else NULL */ -fwd_db_entry_t *find_fwd_db_entry(uint32_t dst_ip); +fwd_db_entry_t *find_fwd_db_entry(ipv4_tuple5_t *key); #ifdef __cplusplus }