From patchwork Thu Jun 1 09:03:31 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Eremin-Solenikov X-Patchwork-Id: 100852 Delivered-To: patch@linaro.org Received: by 10.140.96.100 with SMTP id j91csp709661qge; Thu, 1 Jun 2017 02:15:30 -0700 (PDT) X-Received: by 10.55.137.7 with SMTP id l7mr484608qkd.122.1496308530405; Thu, 01 Jun 2017 02:15:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1496308530; cv=none; d=google.com; s=arc-20160816; b=PMBvdGwjncv103A0ivtXEw8tCJ3UEjIDJOKwzvOGJmLk+vzMBaupySrc08VXQj4xiT FWrpPY3vvjYt9uwDYOhoRWjBJW4n4YhjSPcQp9dAVQu5azC5E3VwzQgNingnblzsNlfh f1G3z3TKJYHoHGm/qwCTiVahor1NSoCUQL9lqZzj/4Bj86BfDibZwfaF+vYdEB4yumSq C/vWIEsLvZTcMOprX7MJm1K3WeFE6P04qcrMQcvm8CSKrS0SVAtVNjwKP+GrwiafVUSM ZNXl36EH03gaBjOzIRG6Al6ep0TnG7wQl5eG+c+OGAKbe1P4y9bjNc79DEYi9Dz/NZ/M 8zJA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:delivered-to:arc-authentication-results; bh=yt5T/9lpoLc8JBh3XPJN5tSxbEttVbPTl8cMHvEsMh8=; b=JHhF2qa201dWEad4kmBnNDPJ+NA0QNgpAT4J1vfMnrHJST1TOiYvWsPnxIJc3y7UFk MRMidcMm76mjYdseJXxQbv+TP0of/V4aurLkXC19moK4KWfZ3qnmkFNGg/MGC3EdKw7j QIaSwm6wYI5FgHGTNPIV3wtiPy3gbWbEOI8x9c/q+0rP3Wm+je9Ya61+vJ7w6/g46STn BuyAdiTMhJElH5EV3blPWwx0lgH3tDN82Yy+HIGTdyCL5WssJMt02NCrUt25enUNmdj0 QKkn8pd0lQrd7u9u56Pok1+02MYLsLE5AMqB6uhanVG0nbZ3jLFGZuIbaykj9D6JQJjQ uY3Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id z42si19177675qta.41.2017.06.01.02.15.30; Thu, 01 Jun 2017 02:15:30 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 0FFB4608DD; Thu, 1 Jun 2017 09:15:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-1.4 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, RCVD_IN_SORBS_SPAM, URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 4207D609A7; Thu, 1 Jun 2017 09:06:10 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 7C00F60D35; Thu, 1 Jun 2017 09:06:02 +0000 (UTC) Received: from mail-lf0-f44.google.com (mail-lf0-f44.google.com [209.85.215.44]) by lists.linaro.org (Postfix) with ESMTPS id 908A060C42 for ; Thu, 1 Jun 2017 09:04:00 +0000 (UTC) Received: by mail-lf0-f44.google.com with SMTP id c184so17852179lfe.2 for ; Thu, 01 Jun 2017 02:04:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=yt5T/9lpoLc8JBh3XPJN5tSxbEttVbPTl8cMHvEsMh8=; b=NNNZ1CmrvRrGnC7UUV2kuglnq8FtdJBKLMOlNO1gk4KvQvyivklLDUzVLIVtuo+Mql 1gswc5MWJWgZzoppneOdY6qgYpnoQ1kOQWRtKxRFxgMeOhFNEy5/sNQSpB3YcWgl+iV8 E42tK/yGfJL+DctAwAWcmvel8iTYaVlPHouN8uvYl638EEYqbUh013H+Kb3cTQr+F1r4 geFtMA+qtWy5uaWpI+/6RZlAatlIK3cunfCnv6wkz+K5feTBUUK5yAktDX0xT4imGmuY mvxQCLhf9OnHwdfZfK7r4xowC8Z74vdm6/HBDbp6vXLAmBjn49XRbBnKv4krRFKPfEfm b2vg== X-Gm-Message-State: AODbwcCChe7I4vV6vikcgmYPGlS1Ok8BwXOBrzkBzAVUIiiUtzzWY+Sr mZMfxX5c2SWNnEHp2wu91Vm+ X-Received: by 10.46.82.151 with SMTP id n23mr183015lje.2.1496307839138; Thu, 01 Jun 2017 02:03:59 -0700 (PDT) Received: from forlindon.lumag.auriga.ru ([188.162.64.12]) by smtp.gmail.com with ESMTPSA id 129sm4232819ljf.36.2017.06.01.02.03.57 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 01 Jun 2017 02:03:58 -0700 (PDT) From: Dmitry Eremin-Solenikov To: lng-odp@lists.linaro.org Date: Thu, 1 Jun 2017 12:03:31 +0300 Message-Id: <20170601090339.9313-16-dmitry.ereminsolenikov@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170601090339.9313-1-dmitry.ereminsolenikov@linaro.org> References: <20170601090339.9313-1-dmitry.ereminsolenikov@linaro.org> Subject: [lng-odp] [PATCH API-NEXT v5 15/23] linux-generic: crypto: make en/decryption work across packet segments X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" Signed-off-by: Dmitry Eremin-Solenikov --- platform/linux-generic/odp_crypto.c | 184 +++++++++++++++++++++++++++--------- 1 file changed, 138 insertions(+), 46 deletions(-) -- 2.11.0 diff --git a/platform/linux-generic/odp_crypto.c b/platform/linux-generic/odp_crypto.c index 7ea90fb92b48..ea7c702650f6 100644 --- a/platform/linux-generic/odp_crypto.c +++ b/platform/linux-generic/odp_crypto.c @@ -190,14 +190,122 @@ odp_crypto_alg_err_t auth_check(odp_crypto_op_param_t *param, } static +int internal_encrypt(EVP_CIPHER_CTX *ctx, odp_crypto_op_param_t *param) +{ + odp_packet_t pkt = param->out_pkt; + unsigned in_pos = param->cipher_range.offset; + unsigned out_pos = param->cipher_range.offset; + unsigned in_len = param->cipher_range.length; + uint8_t block[2 * EVP_MAX_BLOCK_LENGTH]; + unsigned block_len = EVP_CIPHER_block_size(EVP_CIPHER_CTX_cipher(ctx)); + int cipher_len; + int ret; + + while (in_len > 0) { + uint32_t seglen = 0; /* GCC */ + uint8_t *insegaddr = odp_packet_offset(pkt, in_pos, + &seglen, NULL); + unsigned inseglen = in_len < seglen ? in_len : seglen; + + /* There should be at least 1 additional block in out buffer */ + if (inseglen > block_len) { + unsigned part = inseglen - block_len; + + EVP_EncryptUpdate(ctx, insegaddr, &cipher_len, + insegaddr, part); + in_pos += part; + in_len -= part; + insegaddr += part; + inseglen -= part; + + out_pos += cipher_len; + } + + /* Use temporal storage */ + if (inseglen > 0) { + unsigned part = inseglen; + + EVP_EncryptUpdate(ctx, block, &cipher_len, + insegaddr, part); + in_pos += part; + in_len -= part; + insegaddr += part; + inseglen -= part; + + odp_packet_copy_from_mem(pkt, out_pos, + cipher_len, block); + out_pos += cipher_len; + } + } + + ret = EVP_EncryptFinal_ex(ctx, block, &cipher_len); + odp_packet_copy_from_mem(pkt, out_pos, cipher_len, block); + + return ret; +} + +static +int internal_decrypt(EVP_CIPHER_CTX *ctx, odp_crypto_op_param_t *param) +{ + odp_packet_t pkt = param->out_pkt; + unsigned in_pos = param->cipher_range.offset; + unsigned out_pos = param->cipher_range.offset; + unsigned in_len = param->cipher_range.length; + uint8_t block[2 * EVP_MAX_BLOCK_LENGTH]; + unsigned block_len = EVP_CIPHER_block_size(EVP_CIPHER_CTX_cipher(ctx)); + int cipher_len; + int ret; + + while (in_len > 0) { + uint32_t seglen = 0; /* GCC */ + uint8_t *insegaddr = odp_packet_offset(pkt, in_pos, + &seglen, NULL); + unsigned inseglen = in_len < seglen ? in_len : seglen; + + /* There should be at least 1 additional block in out buffer */ + if (inseglen > block_len) { + unsigned part = inseglen - block_len; + + EVP_DecryptUpdate(ctx, insegaddr, &cipher_len, + insegaddr, part); + in_pos += part; + in_len -= part; + insegaddr += part; + inseglen -= part; + + out_pos += cipher_len; + } + + /* Use temporal storage */ + if (inseglen > 0) { + unsigned part = inseglen; + + EVP_DecryptUpdate(ctx, block, &cipher_len, + insegaddr, part); + in_pos += part; + in_len -= part; + insegaddr += part; + inseglen -= part; + + odp_packet_copy_from_mem(pkt, out_pos, + cipher_len, block); + out_pos += cipher_len; + } + } + + ret = EVP_DecryptFinal_ex(ctx, block, &cipher_len); + odp_packet_copy_from_mem(pkt, out_pos, cipher_len, block); + + return ret; +} + +static odp_crypto_alg_err_t cipher_encrypt(odp_crypto_op_param_t *param, odp_crypto_generic_session_t *session) { EVP_CIPHER_CTX *ctx; - uint8_t *data = odp_packet_data(param->out_pkt); - uint32_t plain_len = param->cipher_range.length; void *iv_ptr; - int cipher_len = 0; + int ret; if (param->override_iv_ptr) iv_ptr = param->override_iv_ptr; @@ -206,9 +314,6 @@ odp_crypto_alg_err_t cipher_encrypt(odp_crypto_op_param_t *param, else return ODP_CRYPTO_ALG_ERR_IV_INVALID; - /* Adjust pointer for beginning of area to cipher/auth */ - data += param->cipher_range.offset; - /* Encrypt it */ ctx = EVP_CIPHER_CTX_new(); EVP_EncryptInit_ex(ctx, session->cipher.evp_cipher, NULL, @@ -216,13 +321,12 @@ odp_crypto_alg_err_t cipher_encrypt(odp_crypto_op_param_t *param, EVP_EncryptInit_ex(ctx, NULL, NULL, NULL, iv_ptr); EVP_CIPHER_CTX_set_padding(ctx, 0); - EVP_EncryptUpdate(ctx, data, &cipher_len, data, plain_len); - - EVP_EncryptFinal_ex(ctx, data + cipher_len, &cipher_len); + ret = internal_encrypt(ctx, param); EVP_CIPHER_CTX_free(ctx); - return ODP_CRYPTO_ALG_ERR_NONE; + return ret <= 0 ? ODP_CRYPTO_ALG_ERR_DATA_SIZE : + ODP_CRYPTO_ALG_ERR_NONE; } static @@ -230,10 +334,8 @@ odp_crypto_alg_err_t cipher_decrypt(odp_crypto_op_param_t *param, odp_crypto_generic_session_t *session) { EVP_CIPHER_CTX *ctx; - uint8_t *data = odp_packet_data(param->out_pkt); - uint32_t cipher_len = param->cipher_range.length; - int plain_len = 0; void *iv_ptr; + int ret; if (param->override_iv_ptr) iv_ptr = param->override_iv_ptr; @@ -242,9 +344,6 @@ odp_crypto_alg_err_t cipher_decrypt(odp_crypto_op_param_t *param, else return ODP_CRYPTO_ALG_ERR_IV_INVALID; - /* Adjust pointer for beginning of area to cipher/auth */ - data += param->cipher_range.offset; - /* Decrypt it */ ctx = EVP_CIPHER_CTX_new(); EVP_DecryptInit_ex(ctx, session->cipher.evp_cipher, NULL, @@ -252,13 +351,12 @@ odp_crypto_alg_err_t cipher_decrypt(odp_crypto_op_param_t *param, EVP_DecryptInit_ex(ctx, NULL, NULL, NULL, iv_ptr); EVP_CIPHER_CTX_set_padding(ctx, 0); - EVP_DecryptUpdate(ctx, data, &plain_len, data, cipher_len); - - EVP_DecryptFinal_ex(ctx, data + plain_len, &plain_len); + ret = internal_decrypt(ctx, param); EVP_CIPHER_CTX_free(ctx); - return ODP_CRYPTO_ALG_ERR_NONE; + return ret <= 0 ? ODP_CRYPTO_ALG_ERR_DATA_SIZE : + ODP_CRYPTO_ALG_ERR_NONE; } static int process_cipher_param(odp_crypto_generic_session_t *session, @@ -293,13 +391,12 @@ odp_crypto_alg_err_t aes_gcm_encrypt(odp_crypto_op_param_t *param, odp_crypto_generic_session_t *session) { EVP_CIPHER_CTX *ctx; - uint8_t *data = odp_packet_data(param->out_pkt); - uint32_t plain_len = param->cipher_range.length; const uint8_t *aad_head = param->aad.ptr; uint32_t aad_len = param->aad.length; void *iv_ptr; - int cipher_len = 0; - uint8_t *tag = data + param->hash_result_offset; + int dummy_len = 0; + uint8_t block[EVP_MAX_MD_SIZE]; + int ret; if (param->override_iv_ptr) iv_ptr = param->override_iv_ptr; @@ -308,9 +405,6 @@ odp_crypto_alg_err_t aes_gcm_encrypt(odp_crypto_op_param_t *param, else return ODP_CRYPTO_ALG_ERR_IV_INVALID; - /* Adjust pointer for beginning of area to cipher/auth */ - data += param->cipher_range.offset; - /* Encrypt it */ ctx = EVP_CIPHER_CTX_new(); EVP_EncryptInit_ex(ctx, session->cipher.evp_cipher, NULL, @@ -322,18 +416,20 @@ odp_crypto_alg_err_t aes_gcm_encrypt(odp_crypto_op_param_t *param, /* Authenticate header data (if any) without encrypting them */ if (aad_len > 0) - EVP_EncryptUpdate(ctx, NULL, &cipher_len, + EVP_EncryptUpdate(ctx, NULL, &dummy_len, aad_head, aad_len); - EVP_EncryptUpdate(ctx, data, &cipher_len, data, plain_len); + ret = internal_encrypt(ctx, param); - EVP_EncryptFinal_ex(ctx, data + cipher_len, &cipher_len); EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_GCM_GET_TAG, - session->p.auth_digest_len, tag); + session->p.auth_digest_len, block); + odp_packet_copy_from_mem(param->out_pkt, param->hash_result_offset, + session->p.auth_digest_len, block); EVP_CIPHER_CTX_free(ctx); - return ODP_CRYPTO_ALG_ERR_NONE; + return ret <= 0 ? ODP_CRYPTO_ALG_ERR_DATA_SIZE : + ODP_CRYPTO_ALG_ERR_NONE; } static @@ -341,13 +437,12 @@ odp_crypto_alg_err_t aes_gcm_decrypt(odp_crypto_op_param_t *param, odp_crypto_generic_session_t *session) { EVP_CIPHER_CTX *ctx; - uint8_t *data = odp_packet_data(param->out_pkt); - uint32_t cipher_len = param->cipher_range.length; const uint8_t *aad_head = param->aad.ptr; uint32_t aad_len = param->aad.length; - int plain_len = 0; + int dummy_len = 0; void *iv_ptr; - uint8_t *tag = data + param->hash_result_offset; + uint8_t block[EVP_MAX_MD_SIZE]; + int ret; if (param->override_iv_ptr) iv_ptr = param->override_iv_ptr; @@ -356,9 +451,6 @@ odp_crypto_alg_err_t aes_gcm_decrypt(odp_crypto_op_param_t *param, else return ODP_CRYPTO_ALG_ERR_IV_INVALID; - /* Adjust pointer for beginning of area to cipher/auth */ - data += param->cipher_range.offset; - /* Decrypt it */ ctx = EVP_CIPHER_CTX_new(); EVP_DecryptInit_ex(ctx, session->cipher.evp_cipher, NULL, @@ -368,22 +460,22 @@ odp_crypto_alg_err_t aes_gcm_decrypt(odp_crypto_op_param_t *param, EVP_DecryptInit_ex(ctx, NULL, NULL, NULL, iv_ptr); EVP_CIPHER_CTX_set_padding(ctx, 0); + odp_packet_copy_to_mem(param->out_pkt, param->hash_result_offset, + session->p.auth_digest_len, block); EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_GCM_SET_TAG, - session->p.auth_digest_len, tag); + session->p.auth_digest_len, block); /* Authenticate header data (if any) without encrypting them */ if (aad_len > 0) - EVP_DecryptUpdate(ctx, NULL, &plain_len, + EVP_DecryptUpdate(ctx, NULL, &dummy_len, aad_head, aad_len); - EVP_DecryptUpdate(ctx, data, &plain_len, data, cipher_len); - - if (EVP_DecryptFinal_ex(ctx, data + plain_len, &plain_len) <= 0) - return ODP_CRYPTO_ALG_ERR_ICV_CHECK; + ret = internal_decrypt(ctx, param); EVP_CIPHER_CTX_free(ctx); - return ODP_CRYPTO_ALG_ERR_NONE; + return ret <= 0 ? ODP_CRYPTO_ALG_ERR_ICV_CHECK : + ODP_CRYPTO_ALG_ERR_NONE; } static int process_aes_gcm_param(odp_crypto_generic_session_t *session,