Menu

Michael Gracie

Grab and validate National Vulnerabilities Database updates

Here is a concoction to grab National Vulnerability Database feeds, specifically the Modified JSON and related metadata, then validate the reported sha256 hashes:

import urllib.request
import gzip
import hashlib

#json file
fileurl = 'https://static.nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-modified.json.gz'
json_file = '/Users/laptopuser/Documents/Active/NVD/nvd-data/0326/nvdcve-1.0-modified.json.gz'
urllib.request.urlretrieve(fileurl, json_file)
json_file_open = gzip.open(json_file, 'rb')

#meta file
fileurl = 'https://static.nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-modified.meta'
json_meta_file = '/Users/laptopuser/Documents/Active/NVD/nvd-data/0326/nvdcve-1.0-modified.meta'
urllib.request.urlretrieve(fileurl, json_meta_file)
json_meta_file_open = open(json_meta_file, 'r')

#get hash from meta file
for line in json_meta_file_open:
    li = line.split(':')
    if li[0] == 'sha256':
        ze_sha = li[1].strip('\n')
        print('Meta:', ze_sha)

#calc hash of file
sha256_hash = hashlib.sha256()
with json_file_open as f:
    for byte_block in iter(lambda: f.read(4096),b""):
        sha256_hash.update(byte_block)
    ze_hash = sha256_hash.hexdigest().upper()
    print('Calc:', ze_hash)
    
if ze_sha == ze_hash:
    print('MATCH')

json_file_open.close()
json_meta_file_open.close()

You will get output that looks something like this …

Meta: E3ECE7D603F091E68E60E68CD6E230A28BC9E23EFB7E9B8145E559D1910BE9A6
Calc: E3ECE7D603F091E68E60E68CD6E230A28BC9E23EFB7E9B8145E559D1910BE9A6
MATCH

No apologies for the basic code presentation, nor for using urllib.request.urlretrieve. Feel free to copy and paste into Jupyter notebook or PyCharm if syntax highlighting is desired; as the latter goes, I know that function is supposed to disappear but my application requires keeping a sizable rotation of NIST’s handiwork close by.

MG signing off (to grab and validate some more)

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.