From patchwork Sun Dec 8 18:35:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niko Mauno X-Patchwork-Id: 180969 Delivered-To: patch@linaro.org Received: by 2002:a92:3001:0:0:0:0:0 with SMTP id x1csp3449574ile; Sun, 8 Dec 2019 10:41:10 -0800 (PST) X-Google-Smtp-Source: APXvYqwBF5gUpIADfIutWhdS1HFIlLj3FiZRmE38mgL8zh1tnqMpRQtAcLUE64ZxnLDKr6BEaqxP X-Received: by 2002:a65:66c4:: with SMTP id c4mr15063500pgw.429.1575830469151; Sun, 08 Dec 2019 10:41:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1575830469; cv=none; d=google.com; s=arc-20160816; b=x00xV9IJpTogBMz9ISRTK1BRsOmfhGP1csCnDNysGhXrQgZBJ1b1ndly79puXjO/Cs Bv1928mFqs1vodzP/OipdvihG3y7ew0ZwucJBN13juN4VpZbj2QbTZ+VVyaotLOrXDDx c4Us34/JrBRL/BcXLyDsUwaTQ8MznM1zv2PWLuTZ5LtHhh1j0g3cI61LVA55Zw8OdMmp KLzNtEDmjjlVwxtnQkfZ90sdWFmHJzkWHwKkA1xkUZl7+NPkkVVCIbVLYjasPMCQAIrH ba0mJ9CPOQNIGp6CqF/eKTQ0X2eDvW29wVSvRKnNzEQ2fFDN/eNoKuYjw3EO+xwKxRf5 RpBw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:sender:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :delivered-to; bh=gdFzUZnwnF2tDHt2qJLlHMIzqxHuCCwATvUeR9WiAdc=; b=iaL611ONrM12oO7qHOxSFR+3yaWs0DGgiraSVzA1wDHGcWrdIEKcsGvkH0Da+UNzCV m9Di2K+Jrqac5X6X4AKlqm4Y2MnDRQFObD0wmpa9NxdBCoSS+eM0oHINo2ulRYun6Wr2 tu00mPrHehOzaCXAU82gjJ5CF7jzOYQrPppjEwaUwGovQC9DPvi9ubI0iW8QZDhOsctv l/qx2PYL/Dq0i0VRskS8yxm3htPD7aPLnwCWaFZCPkafXOTdU7tFo1ZEJYkWcGkKd9UR htraGA4OwxKpAiHl+UkB6KqXFesINOGQ4Debh26B1XxvbainC1tDTKKrvdt+1YSakkWA SI1Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=iki.fi Return-Path: Received: from mail.openembedded.org (mail.openembedded.org. [140.211.169.62]) by mx.google.com with ESMTP id d9si16789059plj.318.2019.12.08.10.41.08; Sun, 08 Dec 2019 10:41:09 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) client-ip=140.211.169.62; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) smtp.mailfrom=openembedded-core-bounces@lists.openembedded.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=iki.fi Received: from ec2-34-214-78-129.us-west-2.compute.amazonaws.com (localhost [127.0.0.1]) by mail.openembedded.org (Postfix) with ESMTP id 055187FBE2; Sun, 8 Dec 2019 18:41:04 +0000 (UTC) X-Original-To: openembedded-core@lists.openembedded.org Delivered-To: openembedded-core@lists.openembedded.org Received: from sypressi2.dnainternet.net (sypressi2.dnainternet.net [83.102.40.154]) by mail.openembedded.org (Postfix) with ESMTP id DE9557F849 for ; Sun, 8 Dec 2019 18:40:55 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by sypressi2.dnainternet.net (Postfix) with ESMTP id 9C3EF14E51; Sun, 8 Dec 2019 20:40:56 +0200 (EET) X-Virus-Scanned: DNA Internet at dnainternet.net X-Spam-Flag: NO X-Spam-Score: 0.653 X-Spam-Level: X-Spam-Status: No, score=0.653 tagged_above=-9999 required=6 tests=[SPF_HELO_NONE=0.001, SPF_NEUTRAL=0.652] autolearn=disabled Received: from sypressi2.dnainternet.net ([83.102.40.154]) by localhost (sypressi2.dnainternet.net [127.0.0.1]) (DNA Internet, port 10041) with ESMTP id Pu3TWmUkL8zn; Sun, 8 Dec 2019 20:40:56 +0200 (EET) Received: from luumupuu2.dnainternet.net (luumupuu2.dnainternet.net [83.102.40.55]) by sypressi2.dnainternet.net (Postfix) with ESMTP id 11144149A7; Sun, 8 Dec 2019 20:40:56 +0200 (EET) Received: from localhost.localdomain (62-78-161-185.bb.dnainternet.fi [62.78.161.185]) by luumupuu2.dnainternet.net (Postfix) with ESMTP id 1A15F89; Sun, 8 Dec 2019 20:40:50 +0200 (EET) From: Niko Mauno To: openembedded-core@lists.openembedded.org Date: Sun, 8 Dec 2019 20:35:54 +0200 Message-Id: <20191208183557.32589-8-niko.mauno@iki.fi> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20191208183557.32589-1-niko.mauno@iki.fi> References: <20191208183557.32589-1-niko.mauno@iki.fi> MIME-Version: 1.0 Subject: [OE-core] [thud-next][PATCH 08/11] cve-update-db-native: clean up proxy handling X-BeenThere: openembedded-core@lists.openembedded.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Patches and discussions about the oe-core layer List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: openembedded-core-bounces@lists.openembedded.org Errors-To: openembedded-core-bounces@lists.openembedded.org From: Ross Burton urllib handles adding proxy handlers if the proxies are set in the environment, so call bb.utils.export_proxies() to do that and remove the manual setup. (From OE-Core rev: 6b73004668b3b71c9c38814b79fbb58c893ed434) Signed-off-by: Ross Burton Signed-off-by: Richard Purdie --- .../recipes-core/meta/cve-update-db-native.bb | 31 +++---------------- 1 file changed, 5 insertions(+), 26 deletions(-) -- 2.20.1 -- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core diff --git a/meta/recipes-core/meta/cve-update-db-native.bb b/meta/recipes-core/meta/cve-update-db-native.bb index 08b18f064f..db1d69a28e 100644 --- a/meta/recipes-core/meta/cve-update-db-native.bb +++ b/meta/recipes-core/meta/cve-update-db-native.bb @@ -21,10 +21,12 @@ python do_populate_cve_db() { """ Update NVD database with json data feed """ - + import bb.utils import sqlite3, urllib, urllib.parse, shutil, gzip from datetime import date + bb.utils.export_proxies(d) + BASE_URL = "https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-" YEAR_START = 2002 @@ -40,16 +42,6 @@ python do_populate_cve_db() { except OSError: pass - proxy = d.getVar("https_proxy") - if proxy: - # instantiate an opener but do not install it as the global - # opener unless if we're really sure it's applicable for all - # urllib requests - proxy_handler = urllib.request.ProxyHandler({'https': proxy}) - proxy_opener = urllib.request.build_opener(proxy_handler) - else: - proxy_opener = None - cve_f = open(os.path.join(d.getVar("TMPDIR"), 'cve_check'), 'a') if not os.path.isdir(db_dir): @@ -67,15 +59,7 @@ python do_populate_cve_db() { json_url = year_url + ".json.gz" # Retrieve meta last modified date - - response = None - - if proxy_opener: - response = proxy_opener.open(meta_url) - else: - req = urllib.request.Request(meta_url) - response = urllib.request.urlopen(req) - + response = urllib.request.urlopen(meta_url) if response: for l in response.read().decode("utf-8").splitlines(): key, value = l.split(":", 1) @@ -95,12 +79,7 @@ python do_populate_cve_db() { # Update db with current year json file try: - if proxy_opener: - response = proxy_opener.open(json_url) - else: - req = urllib.request.Request(json_url) - response = urllib.request.urlopen(req) - + response = urllib.request.urlopen(json_url) if response: update_db(c, gzip.decompress(response.read()).decode('utf-8')) c.execute("insert or replace into META values (?, ?)", [year, last_modified])