From patchwork Mon Nov 18 16:46:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 179595 Delivered-To: patch@linaro.org Received: by 2002:a92:38d5:0:0:0:0:0 with SMTP id g82csp3127176ilf; Mon, 18 Nov 2019 08:47:15 -0800 (PST) X-Google-Smtp-Source: APXvYqw7kcWQ7X/W5lwT110Hls/bufZaWiPIr9ywP4G1BIX1K7sCu6ciqLfYc4aCmvZtnJygTJRx X-Received: by 2002:a17:90b:4006:: with SMTP id ie6mr490890pjb.50.1574095635439; Mon, 18 Nov 2019 08:47:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1574095635; cv=none; d=google.com; s=arc-20160816; b=SWf5Rpf0bArWCAM30qu0JdHxlL33+Dg+qWL2Kl/DQVDzm5YECYMz/YC11gwVqB8epv t7/PgiJBDVJXKaYzg26nbYVZm8qJELVhehIwOfObcLKa7KGxmuKdZmPrbrkeDUNiLO1u oc95YI61oWiUoBLlPCeJluHl1wt1fHfkqVENbFkFLhvEdKLGaYioKLk+YykGbUaKKU1M 09D/y5xzRiimu4R1FZWSTFDXaHPVRhlv3avVnG84ilmlptVaxLyoy2UJkAA+Eb2hxNn4 RpJkurY0rTg1FkSNuOmer8rMrE9NMSI0fBmQuqxf74iE8/7Lo01Um+9H/ByQkikFhyxA +oXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:sender:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:delivered-to; bh=yRU08dYY9JT+LbfqLvKEETbJxRCp475H/XPpEl072YI=; b=sxqxqO4MUTIw7V2tWryBHy33qHuQwM7Sf70SjhfdC3WUrwarlMLOTM/szuuf/hWpvk w5pbFAa4uu5kUJohZrbTFQMXoZshrhPfr9g3H37nAM2CCL8+zW4BaiXMmbwPYxXrb9rR tP4Z4OOZAJ2icmwCMWXgSEcLN6yLc/dfgjK80gpx4//uud+T9BxIIEkns24r+kcWYt/y 2CSCzV0VFGtjdBd6NKpBGbuOXGBPOnQASdsHVsZ51J7ljuWuW5AqsChqMy4AMMvbaBrI CTRDysspr2f+7YNF3nwIyoXj571zPmV/J14R1kyX/dJLyEjXsn0th9Bcggnq8OhjFY/7 Ft9A== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b=vlibkkv2; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from mail.openembedded.org (mail.openembedded.org. [140.211.169.62]) by mx.google.com with ESMTP id w128si21133989pfw.27.2019.11.18.08.47.15; Mon, 18 Nov 2019 08:47:15 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) client-ip=140.211.169.62; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel-com.20150623.gappssmtp.com header.s=20150623 header.b=vlibkkv2; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from ec2-34-214-78-129.us-west-2.compute.amazonaws.com (localhost [127.0.0.1]) by mail.openembedded.org (Postfix) with ESMTP id 42E027F8BC; Mon, 18 Nov 2019 16:47:00 +0000 (UTC) X-Original-To: openembedded-core@lists.openembedded.org Delivered-To: openembedded-core@lists.openembedded.org Received: from mail-wm1-f67.google.com (mail-wm1-f67.google.com [209.85.128.67]) by mail.openembedded.org (Postfix) with ESMTP id B5EBA7F8CC for ; Mon, 18 Nov 2019 16:46:55 +0000 (UTC) Received: by mail-wm1-f67.google.com with SMTP id b17so19717993wmj.2 for ; Mon, 18 Nov 2019 08:46:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=bXpTy66ZhmzjFqRhzkqbURNmX/UVYlGxO0A5tVJj/8M=; b=vlibkkv2d6MKO7QsI4chhZbn0VBvKUJSNvu/o7t+0xJPAEcwLAL2ApToFdl0zU63dL 9o5i/SKw/abiFjhLQJ0JbCG8pShXAW/d9lQPdCoq9W4Bj5zLuUqXXjMmKM8DuLCMwRsJ 72CMzPSy8X1NlmkE6abIQu/gajmuL17DoB9gbLykaNYpRE2YqApCGHyVOq8dgqQ77zDr RQ0m72cBe8KD50ntsaGq8B/7hicqEZcyx1H3CDLteUon2RsVHpxezBoURjdU0cZz/OBw p3L8tlXGMFqN5Kf2Pvs9cd7gHNkce+YjIIvedDBHBtbsA4MDqBBJBMYn/6+Cv+kSwoj/ JSdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bXpTy66ZhmzjFqRhzkqbURNmX/UVYlGxO0A5tVJj/8M=; b=fr+r01ihFLun3dVbUgrUSVtPJ6DZBSV0TbbmF+Bc/l2UXU2j9WkO4HgjDbpZjfUFX0 fzbb9wfjxbuL8QicSdwEMmqc2dbd1NSKy/RfTDwqQF5TiibB93C6BQswCQutyVDIVDnD 5vLFIzdUsxd3h4eAUGfXIlsdpkH/7S0u3t+JswXO4pU6BumtX9mwpC/3FD+5xAM6Cu7T SBXwzCHutGlS8uV9BYkcD9goJQnrL58z5bxWiUiwIoX9ajuY7Bsg5Dx1A49WUk3Xngvq RVqz1BLbzC0VsAe+pxZ7sTA0AQ+P9WuS+K+F2RNgqjYPE6z/yyS2GOuHnGeNfJV3ow1Y tbFg== X-Gm-Message-State: APjAAAXmVtTzjbHPw2Dob25pCrWKI+B0GuJdxH6vAOuTZrQoAUut7v5D 5WRSOY58NDgaQLnHFg+hi3roJDOZcXQ= X-Received: by 2002:a7b:c776:: with SMTP id x22mr29090101wmk.144.1574095616145; Mon, 18 Nov 2019 08:46:56 -0800 (PST) Received: from flashheart.burtonini.com (35.106.2.81.in-addr.arpa. [81.2.106.35]) by smtp.gmail.com with ESMTPSA id y189sm20936208wmb.13.2019.11.18.08.46.55 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Nov 2019 08:46:55 -0800 (PST) From: Ross Burton To: openembedded-core@lists.openembedded.org Date: Mon, 18 Nov 2019 16:46:45 +0000 Message-Id: <20191118164647.29409-4-ross.burton@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20191118164647.29409-1-ross.burton@intel.com> References: <20191118164647.29409-1-ross.burton@intel.com> MIME-Version: 1.0 Subject: [OE-core] [PATCH 4/6] cve-check: rewrite look to fix false negatives X-BeenThere: openembedded-core@lists.openembedded.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Patches and discussions about the oe-core layer List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: openembedded-core-bounces@lists.openembedded.org Errors-To: openembedded-core-bounces@lists.openembedded.org A previous optimisation was premature and resulted in false-negatives in the report. Rewrite the checking algorithm to first get the list of potential CVEs by vendor:product, then iterate through every matching CPE for that CVE to determine if the bounds match or not. By doing this in two stages we can know if we've checked every CPE, instead of accidentally breaking out of the scan too early. Signed-off-by: Ross Burton --- meta/classes/cve-check.bbclass | 63 ++++++++++++++++++---------------- 1 file changed, 34 insertions(+), 29 deletions(-) -- 2.20.1 -- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass index 3326944d791..c1cbdbde7b7 100644 --- a/meta/classes/cve-check.bbclass +++ b/meta/classes/cve-check.bbclass @@ -165,7 +165,6 @@ def check_cves(d, patched_cves): """ Connect to the NVD database and find unpatched cves. """ - import ast, csv, tempfile, subprocess, io from distutils.version import LooseVersion cves_unpatched = [] @@ -187,68 +186,74 @@ def check_cves(d, patched_cves): cve_whitelist = d.getVar("CVE_CHECK_WHITELIST").split() import sqlite3 - db_file = d.getVar("CVE_CHECK_DB_FILE") - conn = sqlite3.connect(db_file) + db_file = d.expand("file:${CVE_CHECK_DB_FILE}?mode=ro") + conn = sqlite3.connect(db_file, uri=True) + # For each of the known product names (e.g. curl has CPEs using curl and libcurl)... for product in products: - c = conn.cursor() if ":" in product: vendor, product = product.split(":", 1) - c.execute("SELECT * FROM PRODUCTS WHERE PRODUCT IS ? AND VENDOR IS ?", (product, vendor)) else: - c.execute("SELECT * FROM PRODUCTS WHERE PRODUCT IS ?", (product,)) + vendor = "%" - for row in c: - cve = row[0] - version_start = row[3] - operator_start = row[4] - version_end = row[5] - operator_end = row[6] + # Find all relevant CVE IDs. + for cverow in conn.execute("SELECT DISTINCT ID FROM PRODUCTS WHERE PRODUCT IS ? AND VENDOR LIKE ?", (product, vendor)): + cve = cverow[0] if cve in cve_whitelist: bb.note("%s-%s has been whitelisted for %s" % (product, pv, cve)) # TODO: this should be in the report as 'whitelisted' patched_cves.add(cve) + continue elif cve in patched_cves: bb.note("%s has been patched" % (cve)) - else: - to_append = False + continue + + vulnerable = False + for row in conn.execute("SELECT * FROM PRODUCTS WHERE ID IS ? AND PRODUCT IS ? AND VENDOR LIKE ?", (cve, product, vendor)): + (_, _, _, version_start, operator_start, version_end, operator_end) = row + #bb.debug(2, "Evaluating row " + str(row)) + if (operator_start == '=' and pv == version_start): - to_append = True + vulnerable = True else: if operator_start: try: - to_append_start = (operator_start == '>=' and LooseVersion(pv) >= LooseVersion(version_start)) - to_append_start |= (operator_start == '>' and LooseVersion(pv) > LooseVersion(version_start)) + vulnerable_start = (operator_start == '>=' and LooseVersion(pv) >= LooseVersion(version_start)) + vulnerable_start |= (operator_start == '>' and LooseVersion(pv) > LooseVersion(version_start)) except: bb.warn("%s: Failed to compare %s %s %s for %s" % (product, pv, operator_start, version_start, cve)) - to_append_start = False + vulnerable_start = False else: - to_append_start = False + vulnerable_start = False if operator_end: try: - to_append_end = (operator_end == '<=' and LooseVersion(pv) <= LooseVersion(version_end)) - to_append_end |= (operator_end == '<' and LooseVersion(pv) < LooseVersion(version_end)) + vulnerable_end = (operator_end == '<=' and LooseVersion(pv) <= LooseVersion(version_end)) + vulnerable_end |= (operator_end == '<' and LooseVersion(pv) < LooseVersion(version_end)) except: bb.warn("%s: Failed to compare %s %s %s for %s" % (product, pv, operator_end, version_end, cve)) - to_append_end = False + vulnerable_end = False else: - to_append_end = False + vulnerable_end = False if operator_start and operator_end: - to_append = to_append_start and to_append_end + vulnerable = vulnerable_start and vulnerable_end else: - to_append = to_append_start or to_append_end + vulnerable = vulnerable_start or vulnerable_end - if to_append: + if vulnerable: bb.note("%s-%s is vulnerable to %s" % (product, pv, cve)) cves_unpatched.append(cve) - else: - bb.note("%s-%s is not vulnerable to %s" % (product, pv, cve)) - patched_cves.add(cve) + break + + if not vulnerable: + bb.note("%s-%s is not vulnerable to %s" % (product, pv, cve)) + # TODO: not patched but not vulnerable + patched_cves.add(cve) + conn.close() return (list(patched_cves), cves_unpatched)