OpenVAS Change Request #33: Change server-side NVT cache from binary dumps to keyfiles
Status: Voted +9. Done. Released with openvas-scanner 3.0.0.
To remove size limitations for various NVT attributes: description, dependencies, required keys etc.
To drop binary formats in favour of text formats that are human readable and can more easily be handled via other programming languages.
To have standard procedures take care instead of self-invented solutions.
The current cache works with fixed sizes, and just dumps binary memory blocks to filesystem and from filesystem to memory. These memory blocks (openvas-libraries/libopenvas/store_internal.h) impose fixed sizes for various attributes of NVTs, such as description, dependencies, required keys etc.
Repeatingly it happened that the description, taken from CVE, had to be shortened, that CVE references list had to be shortened, etc. This lowers the quality of NVTs and should better be improved.
The cache files will be renamed from *.desc to *.nvti, where nvti stands for NVT information. Thus the old *.desc need to be removed explicitely for a clean cache. The *.desc files do no harm other than consuming disc space though.
Given the proposed solution based on keyfiles, the loading times and disc consumption will change:
- openvasd startup time with empty cache: almost the same
- openvasd startup time with filled cache: doubled (drops from ~4000 NVTs per second to ~2000 NVT per second on a simple laptop).
- disc space consumption of cache: drops to ~1/3 of the current solution (from 142M to 48M at ~12000 NVTs)
Design and Implementation
The new cache files will be glib keyfiles.
A new layer for managing the NVT information will be applied between the keyfiles and the current plugin-layer. It is envisioned to replace this plugin layer entirely eventually. This layer is already implemenented in the module openvas-libraries/libopenvascommon/nvti.o but not yet used in openvas-libraries. nvti.o is curently only used by openvas-manager.
Changes for switching the cache mechanism will only be applied to file openvas-libraries/libopenvas/store.c. It is expected that initially, the implementation will be suboptimal in terms of code size. These could be cleaned up when also improving openvas-libraries/libopenvas/plugutils.c.
The change will be introduced to openvas-libraries 2.1-series.
- 2010-01-06 Felix Wolfsteller <email@example.com>:
Updated status as done.
- 2009-06-12 Jan-Oliver Wagner <firstname.lastname@example.org>:
Added voting results.
- 2009-06-05 Jan-Oliver Wagner <email@example.com>: