tissynbe.py
As discussed in another post, tissynbe.py is a script I developed in Python that helped me analyze mountains of Nessus results quickly. The script cleans up the data and can insert it into a database or output to a CSV file. One other thing it does is split up the descriptions from the solutions (as best as possible).
If you encounter any errors with the script, it should send the offending lines to stdout. Simply copy/paste the output and email it to me (sanitized of course) and I will update the script. Of course, you can do this modification yourself as well, by adding the plugin to FIX, but I’d like to keep track of what plugins are causing trouble.
You can also download tissynbe.py directly. For database schema, see nessusdb.sql. I hope you find this tool useful and I encourage comments and suggestions!
Dependencies:
- Python 2.5 (to import __future__ module’s with_statement)
- Python MySQLdb (On Debian-based distros: apt-get install python-mysqldb)
Some changes that have been suggested from the wonderful #python IRC channel, is to create a mapping of strings to be replaced and iterate through it.
Code would look like the following:
s = 'abcabc'foo = [ ('a','foo'), ('b','bar'), ('c','baz') ]
for i, j in foo: s = s.replace( i, j )
Since some of the string replacements depend on previous regex matches and other replacements, I will need to do some testing and perhaps modify some of the regular expression objects to account for this.
Ok, so I applied the changes. For anyone who’s interested, processing a 17MB nbe file takes about 14 seconds to output as a CSV on my Thinkpad T42, 1.7GHz Centrino. YMMV.
Added the following on line 130:
line[4] = strftime("%Y-%m-%d %H:%M:%S", strptime(line[4]))Keep getting this when attempting to execute py script tissynbe.py. I have tested “Hello World” py script to verify python is available.
Ran this…
./tissynbe.py -d nessus -f ./June18scan.nbe
Recevied this error…
File “./tissynbe.py”, line 109
with open(nbe, ‘rU’) as file:
^
SyntaxError: invalid syntax
What version of Python are you using? I am running Python 2.5.1, which the Python 2.5 branch includes the __future__ module with_statement.
Added a small check at line 248 to only output to a CSV file if there is data to be written.
Also added the following plugins to the FIX line:
* 11040
* 11822
* 11865
* 14674
Thanks for the code Marcin, I’ll get you some updated changes to review for version 2.0
Added header row creation for writing the CSV file, and also support for PluginID 21156.
Is there a relationship between the timestamps table and the results table? Can you query the stored values to figure out at what time a given result was produced? Thanks.
You should be able to do a JOIN on the `host` columns and return scan timestamp information by host. Unfortunately, I don’t have a database set up to provide you with the exact SQL statement, but it won’t tell you exactly when results were produced. You can modify the script to insert a unique identifier in both the results and timestamps that you can then JOIN on, but that would have introduced more complexity than what I needed the script to do.
I am having the exact same issue as Randy, problem is discribed below. any ideas?
Randy replied July 8th, 2008 11:14 am #
Keep getting this when attempting to execute py script tissynbe.py. I have tested “Hello World” py script to verify python is available.
Ran this…
./tissynbe.py -d nessus -f ./June18scan.nbe
Recevied this error…
File “./tissynbe.py”, line 109
with open(nbe, ‘rU’) as file:
^
SyntaxError: invalid syntax
@Jon do you have Python 2.5 or later?
Marcin — Thanks for the very useful script.
Any thoughts on integrating support for policy compliance output (plugin 21156)? I noticed there were some lines for the plugin (21156). However, upon testing it doesn’t look like it parses the output field (e.g., control #, passed/failed).
I was going to take a shot at enhancing your script — but wanted to check and see if this is something you’ve attempted already. Their output is pretty unstructured / inconsistent :/
@John The only stuff I’ve ever done around plugin 21156, is comparing one scan output against another. For that, I used http://tssci-security.com/pub/scripts/fdcc-tool.py . Other than that, I think I’m going to rewrite this tool to use .nessus XML files. If you want, you can email me some sanitized output and I can have a look at what kind of output you’re working with.
I load data to mysql using
./tissynbe.py -f all.nbe -d nessus
and I got next messages:
./tissynbe.py:195: Warning: Out of range value adjusted for column ’scriptid’ at row 87
VALUES (%s, %s, %s, %s, %s, %s, %s)”"”, (small_results))
./tissynbe.py:195: Warning: Out of range value adjusted for column ’scriptid’ at row 98
VALUES (%s, %s, %s, %s, %s, %s, %s)”"”, (small_results))
./tissynbe.py:195: Warning: Out of range value adjusted for column ’scriptid’ at row 4
VALUES (%s, %s, %s, %s, %s, %s, %s)”"”, (small_results))
./tissynbe.py:195: Warning: Out of range value adjusted for column ’scriptid’ at row 13
./tissynbe.py:195: Warning: Data truncated for column ‘msg2′ at row 77
VALUES (%s, %s, %s, %s, %s, %s, %s)”"”, (small_results))
./tissynbe.py:195: Warning: Data truncated for column ‘msg2′ at row 82
VALUES (%s, %s, %s, %s, %s, %s, %s)”"”, (small_results))
./tissynbe.py:195: Warning: Data truncated for column ‘msg2′ at row 32
@voa It appears a scriptid is longer than 5 integers in the first half of your error output. The second half indicates data is being truncated because it does not fit within the maximum byte size for `text` column field. See MySQL Storage Requirements for more information.
Okay, I used this one on a recent engagement. Where do I send a token gesture of my appreciation?
It does not work with the newest nessus files. I only get
“Error occurred while processing: no data to write!”
This cannot be as the .nessus files have 2564 to 27269 lines. They definitely have low, medium and high severities. HAs the format of the latest nessus reports changed?
The format expected by tissynbe is the “nbe” format.
Your python script turned up in my hunt for a .nbe to .csv conversion tool, and it does exactly what I needed it to do. Thanks. I echo Shohn’s sentiments – let me know how I can express a small token of my gratitude!