From patchwork Thu May 19 13:02:53 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Forrest Shi X-Patchwork-Id: 68107 Delivered-To: patch@linaro.org Received: by 10.140.92.199 with SMTP id b65csp3048429qge; Thu, 19 May 2016 00:36:07 -0700 (PDT) X-Received: by 10.55.181.67 with SMTP id e64mr12906527qkf.112.1463643367843; Thu, 19 May 2016 00:36:07 -0700 (PDT) Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id n125si11046599qkb.263.2016.05.19.00.36.07; Thu, 19 May 2016 00:36:07 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=pass (p=NONE dis=NONE) header.from=linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 7F44E617F0; Thu, 19 May 2016 07:36:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: * X-Spam-Status: No, score=1.1 required=5.0 tests=BAD_ENC_HEADER,BAYES_00, DATE_IN_FUTURE_03_06, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 9B00A61776; Thu, 19 May 2016 07:35:11 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id BF26761694; Thu, 19 May 2016 07:34:57 +0000 (UTC) Received: from na01-bl2-obe.outbound.protection.outlook.com (mail-bl2on0090.outbound.protection.outlook.com [65.55.169.90]) by lists.linaro.org (Postfix) with ESMTPS id 4FD28616D6 for ; Thu, 19 May 2016 07:33:12 +0000 (UTC) Received: from BY2PR03CA078.namprd03.prod.outlook.com (10.141.249.51) by BY2PR03MB475.namprd03.prod.outlook.com (10.141.141.150) with Microsoft SMTP Server (TLS) id 15.1.492.11; Thu, 19 May 2016 07:33:10 +0000 Received: from BY2FFO11FD032.protection.gbl (2a01:111:f400:7c0c::171) by BY2PR03CA078.outlook.office365.com (2a01:111:e400:2c5d::51) with Microsoft SMTP Server (TLS) id 15.1.497.12 via Frontend Transport; Thu, 19 May 2016 07:33:10 +0000 Received-SPF: SoftFail (protection.outlook.com: domain of transitioning linaro.org discourages use of 192.88.168.50 as permitted sender) Received: from tx30smr01.am.freescale.net (192.88.168.50) by BY2FFO11FD032.mail.protection.outlook.com (10.1.14.210) with Microsoft SMTP Server (TLS) id 15.1.497.8 via Frontend Transport; Thu, 19 May 2016 07:33:09 +0000 Received: from netperf2.ap.freescale.net ([10.232.133.164]) by tx30smr01.am.freescale.net (8.14.3/8.14.0) with ESMTP id u4J7X6j9015041; Thu, 19 May 2016 00:33:08 -0700 From: To: Date: Thu, 19 May 2016 18:32:53 +0530 Message-ID: <1463662973-1290-1-git-send-email-forrest.shi@linaro.org> X-Mailer: git-send-email 2.8.2 X-EOPAttributedMessage: 0 X-Matching-Connectors: 131081167897715454; (91ab9b29-cfa4-454e-5278-08d120cd25b8); () X-Forefront-Antispam-Report: CIP:192.88.168.50; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(2980300002)(189002)(199003)(9170700003)(2876002)(2351001)(36756003)(2906002)(4326007)(11100500001)(47776003)(8676002)(6806005)(50986999)(229853001)(586003)(86362001)(33646002)(86152002)(1220700001)(575784001)(50226002)(106466001)(104016004)(48376002)(189998001)(50466002)(77096005)(87936001)(19580395003)(19580405001)(5003940100001)(110136002)(105596002)(8936002)(5008740100001)(81166006)(92566002)(414714003)(473944003); DIR:OUT; SFP:1101; SCL:1; SRVR:BY2PR03MB475; H:tx30smr01.am.freescale.net; FPR:; SPF:SoftFail; MLV:nov; MX:1; A:1; PTR:InfoDomainNonexistent; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BY2FFO11FD032; 1:Io2TSR9vEeInfL7bDTR1OHg4CDyD/pT730MU85v5/XaFzIhPfIIsDxfCIFP1mWk8unVCabcayhv2PLkvwJcIn/SoYKLVXAlApMzfQEZbZ/WbscHJKlSrQRNSG26F88zp2YiWYF1BBkalO/ylc7gWT5K5zMCFcWvAL7Sky/vaFQewBeBIvTBHe5Kl4k5aq+wShj1Ri42hxYPZLEYjogfyBTfQ2SvahEjO3WRtyzY/Y1aZAs3ueVHcMe28kxOsfAckkVLH9DYc5Y9gSQCSwdmd11RDJuHSdhOzPG1ZU3kBTHl2ucraH+3ZH3uERcTJsD41jsELis12RVK0A+a/KVlGuX9OHNNcvqSOXXh/2nvd1lc3I19/8g2tbNgeGkGxOJtumA0yQNQTsjEwRKljHO8dvPHRKaeIPM9gSXFiWNvoBqUtfCNbiQgxnmINWJkOzpmLvpQ+wUWM+AZjOhCgCJpJMaOmM7ONfNsmsBzpU/qLv9Ur1+n3whvJKk2bwUp5sZqTNdyB78LJm29QQakjg4fzEaHR9f8zuGuyR9J4QehynjAfhQRiQf6FrgnI2zStWdZhRBzuDVPFuI4lZu2GwR7YCg== MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 30a0561b-0cbd-4b5e-62fe-08d37fb7d3c0 X-Microsoft-Exchange-Diagnostics: 1; BY2PR03MB475; 2:dY3DYsyuWm2E2GhS3WPHcC2mW6/zPPqo3G9cz1Dz34ImlmqrRqbFEqDUUjXkBqgC5so/E8E73PWFTCiGiRLHK1pLTetd4qJkZAOSCtNuzxuHw2rLQS0l9admVvTDb6vjQFv9pHAa/521FzCK5EstiJlx11Ujtv7Aal4z2mr0EAxD3DpJUpWkKRjPgtE2wePC; 3:JQumwndZMCGB35ns+ugu3e8D/vICKUxbB48J4zaoWReAXMPwRF7m2Kdh6WbNYkOO0yP+K5QXueAbpXOB4FOiLs+1KLZOw0RwYSqO7UM7GtlDlrV7/phlOD+pEmo5trK6n6qt8MIq1txVe5iEidRA3uQpWKTQ4KHpRwXs1Z4tchoUGe1vl424MWFBKZDWw0cSvb/qt2JJSZ+pXf2kPqCDhsnSaTO8qkp0M925A8MwQnA=; 25:1QoStn2NSmKi+FtNGhbN/0IssP/JIA1kVSqXsb1jj4kHu81vf0PPGstCFg8G22LaAPcS7+MQAfz08ZgDXu88q9E2WNoxhO+bw/WymOOkIaUsTHpeI5WlC05Vsc0PjIG8G/7n7NKeJRMeGiSlrfsnsdI9B3ET6E1zLiXwPwJPupdOW038Ft59TVyM8D4OLZi8lbls7h15nWjM+JZ7ciwKk/BsJk3VO84OK87HufzUKSUg0o7F05W945bihnv/4DJz6QcBTIyYYo/KU9c2UWYk2mM3FhI1qhx+Aq00oXCOsrjhSF3LVCCcNZxynw/AT2cnEDsa0CW6W1cvOFx12QbkLBaIBL6WCWH/L4ilWeKT2UrBmvigQNboW5WJGFqtdQTGGTN3vO3tbYMuTaVOcbqTIeaEk5GF4l/PbpuALBb6DiY= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BY2PR03MB475; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(13023025)(8121501046)(13017025)(13018025)(5005006)(13015025)(13024025)(3002001)(10201501046)(6055026); SRVR:BY2PR03MB475; BCL:0; PCL:0; RULEID:(400006); SRVR:BY2PR03MB475; X-Microsoft-Exchange-Diagnostics: 1; BY2PR03MB475; 4:z+RyhZ2O1KKm0k+X0bIoO6CFdQZv0DhLH13tcle5iAXM4KFKjbVZmMkY9yuCxqNQ9TVlIUd5oqUJSDbrkhm6JIsvgRaK724c7UyLB92Emx+F4HW99IJPOEKJJ469V59+ERgMETQHPsOGhtX+Rjoc7cvx1KkyJ39zGJsPmz9/r9TKMr0oOADbDlN3kdg8SQKoHnk8KDm9eYo6gMDbRs5tVD3N0uS8m+qx7ardsrJPXiY9eJ7DMYp2wzUsg++mhp+3W/iwsHl1gGN8oLKp3Ne24Tw/SKvqITownZL7vnxjNnbUBLNs13mplv5OuehK7W70foKkuuAaw47gtxT7ZcP/PRbYyJA8PKnytUcAQ8hX3Wr3sTeBDkeRD5GhVL4nws2J7+iftMY0iy5dW5OxpNDkwqkI1tdMEBHzTwQJ6hqUy1siD2Kr5YO9GzSSR2VTeBts6YTCdbIuL9Z0m+IKmGdDZVCelKob96dfuY0jLzq5O4I= X-Forefront-PRVS: 094700CA91 X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BY2PR03MB475; 23:j2yB7+B7l/4dSLJ419ohRLCh8W/xh39QEi9bMJZ6Pn?= =?us-ascii?Q?sirJ7HP3y7GVUVpe50HH8xCh8EZsYFnPHaQpzTD8rjbGKygYjn/1XrwYuUBA?= =?us-ascii?Q?LrIVHyEvxPv+WIN0w075Z8oaIXpFqAlPS1EC6tl/11TPNGagf8zQ7FS2CxQ4?= =?us-ascii?Q?HDapHiDSxDKPNGlgES94dZY3vu0+p/LBbLVKHJCbQrcGIrFWxtUJ7uP+Pxp5?= =?us-ascii?Q?UIQgE+dm2Ar+iKwYxZxtDu+7VGk3Krl4y0bFkkQKcSYvNY3Xo9MaX9Q3SJww?= =?us-ascii?Q?HLFDEhLwA7rKcdhxU3c6lhwvDaC1vqm/GaZAjLPr55u74y2krz1SVf5C8k4U?= =?us-ascii?Q?XZbEyOdGVmE8EammveLW/HYuRiwOqfSPr1LSHKY3LVo7knMDK0jfOigu5CsJ?= =?us-ascii?Q?Sz3HRY4uYgJ/tDL9RNY0HV+dusSBihKOdTEqel/47vHTHmLmvyKSZ3zJ6nPo?= =?us-ascii?Q?U1IAVMt49yxaUr53h8MBo972UVtHr+5ltX8R+MEFlq8xCgNUVHgtyjobTqlq?= =?us-ascii?Q?UM7CBiandjvJg5M/reycJfVEX5X0ObIrI7LGwOpBhVdxWSQ1ZO2Z6XpWD46w?= =?us-ascii?Q?ssbnIyLlCFha8N4gTXcXUzFZdJP7z3AsGoxulaAgIO044H/vhkq5Yp1K62Mo?= =?us-ascii?Q?cJ8hrHuxLvjqvgXMFaQZCmncZg/MaC5+49ZEg/g/hql7aLYU07QKEbhbFqiX?= =?us-ascii?Q?4XvK4PHKWhkiZYUp50FGHs38I0rKud1oNJ4b1A6VhkKtwiHOJtDVmrsIV4rh?= =?us-ascii?Q?QezBBZW1AvdOOZTdQ2ZPBxlOtcl5naq0Gz7EKlmxQ+27L8iIfJoBGz6pbHvv?= =?us-ascii?Q?GhtzFxh0QPo/8zJ8NJpv7IW+1HiRmIs1RMzYnsXZp2rD7mAmi8aeZGQ+VY0j?= =?us-ascii?Q?5uEv0KsaPU7JVOmKrGFIopSbRu26jWepUi1mdTifZD81e2yHrNOKqlY6MgFk?= =?us-ascii?Q?wNENUnrVeYngbQH8BJ3g0DC3y74hDGLZWqPnam73BSpcobOTtzqwXf9y+bpa?= =?us-ascii?Q?rYLR4XW84ehMcmwZFZYWbDiUHFt98aVmWZBCyfHujyVRx3Jb5+n0f8xD/sug?= =?us-ascii?Q?9pvynFJr5fi0JrXz+x1kWlTdG3?= X-Microsoft-Exchange-Diagnostics: 1; BY2PR03MB475; 5:K9krNu+BybWmII1G4ribU2NVn/sqryOnXZWFqgXA05iXTm+jXEVV+SNMdSa6uAluEZLrdgLw/qSPkqbY6BnqGLBvJ1UetxVW5HvAuoEc+J6EH/+MtsEwiY80MOob7yBHaefGjH6tqTLwNf+/fCU05nt05l+CpYR0FTzwojGU1bk=; 24:WNyHfZwaLJS8XnQ/W7iGQCPgiIhH2tvrb37lUwFme+DA1cQ3rGM6zUVI1JJ6SBImaCKwJz5BS6u0B33dACiqaXHadz0MfZA9dulfQete3CI=; 7:xNX8UaCVkbBIl/wkYiLInZAl4x0GZssIpXgZFq8TMZu9nGvaXCgqyEDJrAldMZf52+u4HZGS1PRGWf5eYittX/YZLz/qDRQ3o9OowcE/ZiCm9+WEnG/WRBwznfL20J6WVd5vBYweuBhPgA293PtzlDl3jpk/bMV8kVOXbnNccTVGTNg47Ub4Q8Me1O9elECi SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 May 2016 07:33:09.5687 (UTC) X-MS-Exchange-CrossTenant-Id: 5afe0b00-7697-4969-b663-5eab37d5f47e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5afe0b00-7697-4969-b663-5eab37d5f47e; Ip=[192.88.168.50]; Helo=[tx30smr01.am.freescale.net] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2PR03MB475 X-Topics: patch Subject: [lng-odp] [PATCH 3/5] example:l3fwd: support bi-directional forwarding X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Xuelin Shi Two threads are used to do packet processing. each port is assigned one thread. Signed-off-by: Xuelin Shi --- example/l3fwd/odp_l3fwd.c | 93 +++++++++++++++++++++++++++++++---------------- 1 file changed, 62 insertions(+), 31 deletions(-) diff --git a/example/l3fwd/odp_l3fwd.c b/example/l3fwd/odp_l3fwd.c index 5f8eaa0..29da045 100644 --- a/example/l3fwd/odp_l3fwd.c +++ b/example/l3fwd/odp_l3fwd.c @@ -25,11 +25,25 @@ static const char * const route_str[] = { "2.1.1.0/24:fm1-mac2:00.e0.0c.00.85.01", }; +#define MAX_NB_WORKER 8 +#define MAX_NB_PKTIO 4 + +struct l3fwd_pktio_s { + odp_pktio_t pktio; + odp_pktin_queue_t ifin; + odp_pktout_queue_t ifout; +}; + +struct thread_arg_s { + uint32_t if_idx; +}; + struct { - odp_pktio_t if0, if1; - odp_pktin_queue_t if0in, if1in; - odp_pktout_queue_t if0out, if1out; - odph_ethaddr_t src, dst; + struct l3fwd_pktio_s l3fwd_pktios[MAX_NB_PKTIO]; + odph_linux_pthread_t l3fwd_workers[MAX_NB_WORKER]; + struct thread_arg_s worker_args[MAX_NB_WORKER]; + uint32_t nb_pktio; /* effective pktios */ + uint32_t nb_worker; /* effective workers */ } global; static odp_pktio_t create_pktio(const char *name, odp_pool_t pool, @@ -77,30 +91,28 @@ static odp_pktio_t create_pktio(const char *name, odp_pool_t pool, return pktio; } -static void *run_worker(void *arg ODP_UNUSED) +static void *run_worker(void *arg) { odp_packet_t pkt_tbl[MAX_PKT_BURST]; odp_packet_t pkt_tbl_drop[MAX_PKT_BURST]; - int pkts, i; + uint32_t pkts, i; + struct l3fwd_pktio_s *port; - if (odp_pktio_start(global.if0)) { - printf("unable to start input interface\n"); - exit(1); - } - printf("started input interface\n"); - if (odp_pktio_start(global.if1)) { - printf("unable to start output interface\n"); + i = ((struct thread_arg_s *)arg)->if_idx; + port = &global.l3fwd_pktios[i]; + + if (odp_pktio_start(port->pktio)) { + printf("unable to start interface: %d\n", i); exit(1); } - printf("started output interface\n"); - printf("started all\n"); + printf("start interface: %d\n", i); for (;;) { int need_to_drop = 0; memset(pkt_tbl, 0, sizeof(pkt_tbl_drop)); - pkts = odp_pktin_recv(global.if0in, pkt_tbl, MAX_PKT_BURST); + pkts = odp_pktin_recv(port->ifin, pkt_tbl, MAX_PKT_BURST); if (odp_unlikely(pkts <= 0)) continue; for (i = 0; i < pkts; i++) { @@ -164,10 +176,9 @@ int main(int argc, char **argv) odp_pool_t pool; odp_pool_param_t params; odp_cpumask_t cpumask; - odph_linux_pthread_t thd; odp_instance_t instance; odph_linux_thr_params_t thr_params; - size_t i; + uint32_t cpu, i; uint8_t mac[ODPH_ETHADDR_LEN]; if (argc != 3) { @@ -187,6 +198,9 @@ int main(int argc, char **argv) exit(1); } + /* Clear global argument */ + memset(&global, 0, sizeof(global)); + /* Init l3fwd tale */ init_fwd_db(); @@ -212,24 +226,41 @@ int main(int argc, char **argv) exit(1); } - global.if0 = create_pktio(argv[1], pool, &global.if0in, &global.if0out); - global.if1 = create_pktio(argv[2], pool, &global.if1in, &global.if1out); - - /* resolve route table */ - odp_pktio_mac_addr(global.if0, mac, ODPH_ETHADDR_LEN); - resolve_fwd_db(argv[1], global.if0, mac); - odp_pktio_mac_addr(global.if1, mac, ODPH_ETHADDR_LEN); - resolve_fwd_db(argv[2], global.if1, mac); - - odp_cpumask_default_worker(&cpumask, 1); + /* TODO: parse cmdline to get pktio number, name */ + global.nb_pktio = 2; + for (i = 0; i < global.nb_pktio; i++) { + struct l3fwd_pktio_s *port; + char *ifname = argv[i + 1]; + + port = &global.l3fwd_pktios[i]; + port->pktio = create_pktio(ifname, pool, &port->ifin, + &port->ifout); + odp_pktio_mac_addr(port->pktio, mac, ODPH_ETHADDR_LEN); + resolve_fwd_db(ifname, port->pktio, mac); + } + global.nb_worker = 2; memset(&thr_params, 0, sizeof(thr_params)); thr_params.start = run_worker; - thr_params.arg = NULL; thr_params.thr_type = ODP_THREAD_WORKER; thr_params.instance = instance; - odph_linux_pthread_create(&thd, &cpumask, &thr_params); - odph_linux_pthread_join(&thd, 1); + odp_cpumask_default_worker(&cpumask, global.nb_worker); + cpu = odp_cpumask_first(&cpumask); + for (i = 0; i < global.nb_worker; i++) { + struct thread_arg_s *arg; + odp_cpumask_t thr_mask; + + odp_cpumask_zero(&thr_mask); + odp_cpumask_set(&thr_mask, cpu); + arg = &global.worker_args[i]; + arg->if_idx = i; + thr_params.arg = arg; + odph_linux_pthread_create(&global.l3fwd_workers[i], &thr_mask, + &thr_params); + cpu = odp_cpumask_next(&cpumask, cpu); + } + odph_linux_pthread_join(&global.l3fwd_workers[0], global.nb_worker); + return 0; }