From patchwork Mon May 22 22:00:07 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Github ODP bot X-Patchwork-Id: 100333 Delivered-To: patch@linaro.org Received: by 10.140.96.100 with SMTP id j91csp49590qge; Mon, 22 May 2017 15:15:45 -0700 (PDT) X-Received: by 10.36.33.142 with SMTP id e136mr6430ita.72.1495491345122; Mon, 22 May 2017 15:15:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1495491345; cv=none; d=google.com; s=arc-20160816; b=QN9JvmJpCvwxBpye3eoZahqWOtSguaCVQEQ1AQGRiEet0OEosiTyxlnsxEpzLV2mta jARHAXRBqwppjhm8eVQOWul2cQ+q/bGwqyrHQ2lHe8+vNiFJy4MuTFMUKcdX5hAFLYxQ 044XLpi4DZH4mbdX1IGaMBv5Rb/KlxFDSENVr852JZb7cH5NgW/8uV6lzBF09ZdadCcZ hZ/1Lyr5eW3lShA+42aM+khEcQFcHGeQdPFvR837EN4VsrrQbJpngBk+BmhHyO66FrAv oh/9WWaHTYX7LsL8p2BuN8lNgTlybtWm1/r4LZ/DI7VK1+jR6e3XMsqTzHwZ4wLpvgBi XdBg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:github-pr-num :references:in-reply-to:message-id:date:to:from:delivered-to :arc-authentication-results; bh=pz5Ajll1rLBs/+e1Q+v/F7Y9VmC+6fIggTPN8Vc47r0=; b=VHOGqrI4dinHl9u8wCAD5auVjJv5V8yKHhTnb5Bo34RtafZZGDHOwcriHtjU2oZEm8 CU/9CTL4lg+6h5le3P3NuYiTpXLdHZ3wcDmXhTIXZ6nzYbq23a7Rnai4rTKHBlVtBvgd EFiFYFgM/pF5ZWBSueUJz6VV5Lb+ZRNccq5jLh/K6AGd4C6FYKshxbP7/ZoGljCEpdUp nZU7UAJ6Et2cIr3jKcL5jH99LNY/tUnSq48hOjHDW/vgjWkw1QS29raIudOPCJ45j44d ChqDsuoyaAGFOQdMbNzbtV1xKyhXNFLqCbtfSUVy68FqOfXw5qBAeXpkvf/ZiLCVmSk1 N5BA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id 99si18780985ioj.212.2017.05.22.15.15.44; Mon, 22 May 2017 15:15:45 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Received: by lists.linaro.org (Postfix, from userid 109) id DECCB6067D; Mon, 22 May 2017 22:15:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_LOW,URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 931D160C3E; Mon, 22 May 2017 22:02:31 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 0E4C060C37; Mon, 22 May 2017 22:02:24 +0000 (UTC) Received: from forward2h.cmail.yandex.net (forward2h.cmail.yandex.net [87.250.230.17]) by lists.linaro.org (Postfix) with ESMTPS id 393F360AF6 for ; Mon, 22 May 2017 22:00:31 +0000 (UTC) Received: from smtp3m.mail.yandex.net (smtp3m.mail.yandex.net [77.88.61.130]) by forward2h.cmail.yandex.net (Yandex) with ESMTP id D299820623 for ; Tue, 23 May 2017 01:00:29 +0300 (MSK) Received: from smtp3m.mail.yandex.net (localhost.localdomain [127.0.0.1]) by smtp3m.mail.yandex.net (Yandex) with ESMTP id B36072840B67 for ; Tue, 23 May 2017 01:00:29 +0300 (MSK) Received: by smtp3m.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id ll1QrMKPBI-0TEeUdYM; Tue, 23 May 2017 01:00:29 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) X-Yandex-Suid-Status: 1 0 From: Github ODP bot To: lng-odp@lists.linaro.org Date: Tue, 23 May 2017 01:00:07 +0300 Message-Id: <1495490409-30066-19-git-send-email-odpbot@yandex.ru> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1495490409-30066-1-git-send-email-odpbot@yandex.ru> References: <1495490409-30066-1-git-send-email-odpbot@yandex.ru> Github-pr-num: 34 Subject: [lng-odp] [PATCH API-NEXT v2 18/20] linux-generic: crypto: make en/decryption work across packet segments X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Dmitry Eremin-Solenikov Signed-off-by: Dmitry Eremin-Solenikov --- /** Email created from pull request 34 (lumag:crypto-update-main-new) ** https://github.com/Linaro/odp/pull/34 ** Patch: https://github.com/Linaro/odp/pull/34.patch ** Base sha: 826ee894aa0ebd09d42a17e1de077c46bc5b366a ** Merge commit sha: 7c49c61063e2d57f049a5436cf12a3c36710bb34 **/ platform/linux-generic/odp_crypto.c | 162 ++++++++++++++++++++++++++---------- 1 file changed, 116 insertions(+), 46 deletions(-) diff --git a/platform/linux-generic/odp_crypto.c b/platform/linux-generic/odp_crypto.c index a147f271..e5468c40 100644 --- a/platform/linux-generic/odp_crypto.c +++ b/platform/linux-generic/odp_crypto.c @@ -195,15 +195,101 @@ odp_crypto_alg_err_t auth_check(odp_crypto_op_param_t *param, return ODP_CRYPTO_ALG_ERR_NONE; } +#define IN_CONSUME(len) \ + in_pos += (len); \ + in_len -= (len); \ + insegaddr += (len); \ + inseglen -= (len); + +static +int internal_encrypt(EVP_CIPHER_CTX *ctx, odp_crypto_op_param_t *param) +{ + odp_packet_t pkt = param->out_pkt; + unsigned in_pos = param->cipher_range.offset; + unsigned out_pos = param->cipher_range.offset; + unsigned in_len = param->cipher_range.length; + uint8_t block[2 * EVP_MAX_BLOCK_LENGTH]; + unsigned block_len = EVP_CIPHER_block_size(EVP_CIPHER_CTX_cipher(ctx)); + int cipher_len; + int ret; + + while (in_len > 0) { + uint32_t seglen = 0; /* GCC */ + uint8_t *insegaddr = odp_packet_offset(pkt, in_pos, &seglen, NULL); + unsigned inseglen = in_len < seglen ? in_len : seglen; + + /* There should be at least 1 additional block in out buffer */ + if (inseglen > block_len) { + unsigned part = inseglen - block_len; + EVP_EncryptUpdate(ctx, insegaddr, &cipher_len, insegaddr, part); + IN_CONSUME(part); + out_pos += cipher_len; + } + + /* Use temporal storage */ + if (inseglen > 0) { + unsigned part = inseglen; + EVP_EncryptUpdate(ctx, block, &cipher_len, insegaddr, part); + IN_CONSUME(part); + odp_packet_copy_from_mem(pkt, out_pos, cipher_len, block); + out_pos += cipher_len; + } + } + + ret = EVP_EncryptFinal_ex(ctx, block, &cipher_len); + odp_packet_copy_from_mem(pkt, out_pos, cipher_len, block); + + return ret; +} + +static +int internal_decrypt(EVP_CIPHER_CTX *ctx, odp_crypto_op_param_t *param) +{ + odp_packet_t pkt = param->out_pkt; + unsigned in_pos = param->cipher_range.offset; + unsigned out_pos = param->cipher_range.offset; + unsigned in_len = param->cipher_range.length; + uint8_t block[2 * EVP_MAX_BLOCK_LENGTH]; + unsigned block_len = EVP_CIPHER_block_size(EVP_CIPHER_CTX_cipher(ctx)); + int cipher_len; + int ret; + + while (in_len > 0) { + uint32_t seglen = 0; /* GCC */ + uint8_t *insegaddr = odp_packet_offset(pkt, in_pos, &seglen, NULL); + unsigned inseglen = in_len < seglen ? in_len : seglen; + + /* There should be at least 1 additional block in out buffer */ + if (inseglen > block_len) { + unsigned part = inseglen - block_len; + EVP_DecryptUpdate(ctx, insegaddr, &cipher_len, insegaddr, part); + IN_CONSUME(part); + out_pos += cipher_len; + } + + /* Use temporal storage */ + if (inseglen > 0) { + unsigned part = inseglen; + EVP_DecryptUpdate(ctx, block, &cipher_len, insegaddr, part); + IN_CONSUME(part); + odp_packet_copy_from_mem(pkt, out_pos, cipher_len, block); + out_pos += cipher_len; + } + } + + ret = EVP_DecryptFinal_ex(ctx, block, &cipher_len); + odp_packet_copy_from_mem(pkt, out_pos, cipher_len, block); + + return ret; +} + static odp_crypto_alg_err_t cipher_encrypt(odp_crypto_op_param_t *param, odp_crypto_generic_session_t *session) { EVP_CIPHER_CTX *ctx; - uint8_t *data = odp_packet_data(param->out_pkt); - uint32_t plain_len = param->cipher_range.length; void *iv_ptr; - int cipher_len = 0; + int ret; if (param->override_iv_ptr) iv_ptr = param->override_iv_ptr; @@ -212,9 +298,6 @@ odp_crypto_alg_err_t cipher_encrypt(odp_crypto_op_param_t *param, else return ODP_CRYPTO_ALG_ERR_IV_INVALID; - /* Adjust pointer for beginning of area to cipher/auth */ - data += param->cipher_range.offset; - /* Encrypt it */ ctx = EVP_CIPHER_CTX_new(); EVP_EncryptInit_ex(ctx, session->cipher.evp_cipher, NULL, @@ -222,13 +305,12 @@ odp_crypto_alg_err_t cipher_encrypt(odp_crypto_op_param_t *param, EVP_EncryptInit_ex(ctx, NULL, NULL, NULL, iv_ptr); EVP_CIPHER_CTX_set_padding(ctx, 0); - EVP_EncryptUpdate(ctx, data, &cipher_len, data, plain_len); - - EVP_EncryptFinal_ex(ctx, data + cipher_len, &cipher_len); + ret = internal_encrypt(ctx, param); EVP_CIPHER_CTX_free(ctx); - return ODP_CRYPTO_ALG_ERR_NONE; + return ret <= 0 ? ODP_CRYPTO_ALG_ERR_DATA_SIZE : + ODP_CRYPTO_ALG_ERR_NONE; } static @@ -236,10 +318,8 @@ odp_crypto_alg_err_t cipher_decrypt(odp_crypto_op_param_t *param, odp_crypto_generic_session_t *session) { EVP_CIPHER_CTX *ctx; - uint8_t *data = odp_packet_data(param->out_pkt); - uint32_t cipher_len = param->cipher_range.length; - int plain_len = 0; void *iv_ptr; + int ret; if (param->override_iv_ptr) iv_ptr = param->override_iv_ptr; @@ -248,9 +328,6 @@ odp_crypto_alg_err_t cipher_decrypt(odp_crypto_op_param_t *param, else return ODP_CRYPTO_ALG_ERR_IV_INVALID; - /* Adjust pointer for beginning of area to cipher/auth */ - data += param->cipher_range.offset; - /* Decrypt it */ ctx = EVP_CIPHER_CTX_new(); EVP_DecryptInit_ex(ctx, session->cipher.evp_cipher, NULL, @@ -258,13 +335,12 @@ odp_crypto_alg_err_t cipher_decrypt(odp_crypto_op_param_t *param, EVP_DecryptInit_ex(ctx, NULL, NULL, NULL, iv_ptr); EVP_CIPHER_CTX_set_padding(ctx, 0); - EVP_DecryptUpdate(ctx, data, &plain_len, data, cipher_len); - - EVP_DecryptFinal_ex(ctx, data + plain_len, &plain_len); + ret = internal_decrypt(ctx, param); EVP_CIPHER_CTX_free(ctx); - return ODP_CRYPTO_ALG_ERR_NONE; + return ret <= 0 ? ODP_CRYPTO_ALG_ERR_DATA_SIZE : + ODP_CRYPTO_ALG_ERR_NONE; } static int process_cipher_param(odp_crypto_generic_session_t *session, @@ -299,13 +375,12 @@ odp_crypto_alg_err_t aes_gcm_encrypt(odp_crypto_op_param_t *param, odp_crypto_generic_session_t *session) { EVP_CIPHER_CTX *ctx; - uint8_t *data = odp_packet_data(param->out_pkt); - uint32_t plain_len = param->cipher_range.length; const uint8_t *aad_head = param->aad.ptr; uint32_t aad_len = param->aad.length; void *iv_ptr; - int cipher_len = 0; - uint8_t *tag = data + param->hash_result_offset; + int dummy_len = 0; + uint8_t block[EVP_MAX_MD_SIZE]; + int ret; if (param->override_iv_ptr) iv_ptr = param->override_iv_ptr; @@ -314,9 +389,6 @@ odp_crypto_alg_err_t aes_gcm_encrypt(odp_crypto_op_param_t *param, else return ODP_CRYPTO_ALG_ERR_IV_INVALID; - /* Adjust pointer for beginning of area to cipher/auth */ - data += param->cipher_range.offset; - /* Encrypt it */ ctx = EVP_CIPHER_CTX_new(); EVP_EncryptInit_ex(ctx, session->cipher.evp_cipher, NULL, @@ -328,18 +400,20 @@ odp_crypto_alg_err_t aes_gcm_encrypt(odp_crypto_op_param_t *param, /* Authenticate header data (if any) without encrypting them */ if (aad_len > 0) - EVP_EncryptUpdate(ctx, NULL, &cipher_len, + EVP_EncryptUpdate(ctx, NULL, &dummy_len, aad_head, aad_len); - EVP_EncryptUpdate(ctx, data, &cipher_len, data, plain_len); + ret = internal_encrypt(ctx, param); - EVP_EncryptFinal_ex(ctx, data + cipher_len, &cipher_len); EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_GCM_GET_TAG, - session->p.auth_digest_len, tag); + session->p.auth_digest_len, block); + odp_packet_copy_from_mem(param->out_pkt, param->hash_result_offset, + session->p.auth_digest_len, block); EVP_CIPHER_CTX_free(ctx); - return ODP_CRYPTO_ALG_ERR_NONE; + return ret <= 0 ? ODP_CRYPTO_ALG_ERR_DATA_SIZE : + ODP_CRYPTO_ALG_ERR_NONE; } static @@ -347,13 +421,12 @@ odp_crypto_alg_err_t aes_gcm_decrypt(odp_crypto_op_param_t *param, odp_crypto_generic_session_t *session) { EVP_CIPHER_CTX *ctx; - uint8_t *data = odp_packet_data(param->out_pkt); - uint32_t cipher_len = param->cipher_range.length; const uint8_t *aad_head = param->aad.ptr; uint32_t aad_len = param->aad.length; - int plain_len = 0; + int dummy_len = 0; void *iv_ptr; - uint8_t *tag = data + param->hash_result_offset; + uint8_t block[EVP_MAX_MD_SIZE]; + int ret; if (param->override_iv_ptr) iv_ptr = param->override_iv_ptr; @@ -362,9 +435,6 @@ odp_crypto_alg_err_t aes_gcm_decrypt(odp_crypto_op_param_t *param, else return ODP_CRYPTO_ALG_ERR_IV_INVALID; - /* Adjust pointer for beginning of area to cipher/auth */ - data += param->cipher_range.offset; - /* Decrypt it */ ctx = EVP_CIPHER_CTX_new(); EVP_DecryptInit_ex(ctx, session->cipher.evp_cipher, NULL, @@ -374,22 +444,22 @@ odp_crypto_alg_err_t aes_gcm_decrypt(odp_crypto_op_param_t *param, EVP_DecryptInit_ex(ctx, NULL, NULL, NULL, iv_ptr); EVP_CIPHER_CTX_set_padding(ctx, 0); + odp_packet_copy_to_mem(param->out_pkt, param->hash_result_offset, + session->p.auth_digest_len, block); EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_GCM_SET_TAG, - session->p.auth_digest_len, tag); + session->p.auth_digest_len, block); /* Authenticate header data (if any) without encrypting them */ if (aad_len > 0) - EVP_DecryptUpdate(ctx, NULL, &plain_len, + EVP_DecryptUpdate(ctx, NULL, &dummy_len, aad_head, aad_len); - EVP_DecryptUpdate(ctx, data, &plain_len, data, cipher_len); - - if (EVP_DecryptFinal_ex(ctx, data + plain_len, &plain_len) <= 0) - return ODP_CRYPTO_ALG_ERR_ICV_CHECK; + ret = internal_decrypt(ctx, param); EVP_CIPHER_CTX_free(ctx); - return ODP_CRYPTO_ALG_ERR_NONE; + return ret <= 0 ? ODP_CRYPTO_ALG_ERR_ICV_CHECK: + ODP_CRYPTO_ALG_ERR_NONE; } static int process_aes_gcm_param(odp_crypto_generic_session_t *session,