[5/5] cve-update-db-native: don't refresh more than once an hour

Message ID 20191107235830.26644-5-ross.burton@intel.com
State Accepted
Commit 50d898fd360c58fe85460517d965f62b7654771a
Headers show
Series
  • [1/5] libsoup: update patch upstream status
Related show

Commit Message

Ross Burton Nov. 7, 2019, 11:58 p.m.
We already fetch the yearly CVE metadata and check that for updates before
downloading the full data, but we can speed up CVE checking further by only
checking the CVE metadata once an hour.

Signed-off-by: Ross Burton <ross.burton@intel.com>

---
 meta/recipes-core/meta/cve-update-db-native.bb | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

-- 
2.20.1

-- 
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Comments

Armin Kuster Nov. 11, 2019, 10:19 p.m. | #1
On 11/7/19 3:58 PM, Ross Burton wrote:
> We already fetch the yearly CVE metadata and check that for updates before

> downloading the full data, but we can speed up CVE checking further by only

> checking the CVE metadata once an hour.

>

> Signed-off-by: Ross Burton <ross.burton@intel.com>

> ---

>  meta/recipes-core/meta/cve-update-db-native.bb | 10 +++++++++-

>  1 file changed, 9 insertions(+), 1 deletion(-)

>

> diff --git a/meta/recipes-core/meta/cve-update-db-native.bb b/meta/recipes-core/meta/cve-update-db-native.bb

> index 2c427a5884f..19875a49b1c 100644

> --- a/meta/recipes-core/meta/cve-update-db-native.bb

> +++ b/meta/recipes-core/meta/cve-update-db-native.bb

> @@ -31,8 +31,16 @@ python do_populate_cve_db() {

>      db_dir = os.path.join(d.getVar("DL_DIR"), 'CVE_CHECK')

>      db_file = os.path.join(db_dir, 'nvdcve_1.0.db')

>      json_tmpfile = os.path.join(db_dir, 'nvd.json.gz')

> -    proxy = d.getVar("https_proxy")

>  

> +    # Don't refresh the database more than once an hour


err, I thought the NVD db is only updated once every two hours. And why
is this not a variable so folks can tweak accordingly?

- armin
> +    try:

> +        import time

> +        if time.time() - os.path.getmtime(db_file) < (60*60):

> +            return

> +    except OSError:

> +        pass

> +

> +    proxy = d.getVar("https_proxy")

>      if proxy:

>          # instantiate an opener but do not install it as the global

>          # opener unless if we're really sure it's applicable for all


-- 
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Ross Burton Nov. 12, 2019, 3:27 p.m. | #2
On 11/11/2019 22:19, akuster808 wrote:
> err, I thought the NVD db is only updated once every two hours. And why

> is this not a variable so folks can tweak accordingly?


This a fast-path so that a refresh isn't even attempted 60 minutes after 
a fetch.  A fetch involves checking the last updates times in the 
metadata anyway so is relatively fast.

Don't see the need for this to be another variable.  Do you have a 
better refresh interval?

Ross
-- 
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Patch

diff --git a/meta/recipes-core/meta/cve-update-db-native.bb b/meta/recipes-core/meta/cve-update-db-native.bb
index 2c427a5884f..19875a49b1c 100644
--- a/meta/recipes-core/meta/cve-update-db-native.bb
+++ b/meta/recipes-core/meta/cve-update-db-native.bb
@@ -31,8 +31,16 @@  python do_populate_cve_db() {
     db_dir = os.path.join(d.getVar("DL_DIR"), 'CVE_CHECK')
     db_file = os.path.join(db_dir, 'nvdcve_1.0.db')
     json_tmpfile = os.path.join(db_dir, 'nvd.json.gz')
-    proxy = d.getVar("https_proxy")
 
+    # Don't refresh the database more than once an hour
+    try:
+        import time
+        if time.time() - os.path.getmtime(db_file) < (60*60):
+            return
+    except OSError:
+        pass
+
+    proxy = d.getVar("https_proxy")
     if proxy:
         # instantiate an opener but do not install it as the global
         # opener unless if we're really sure it's applicable for all