I’ve been doing some work lately with text files and have been using various shell command techniques to manipulate them for whatever purposes I need. This isn’t a HOWTO guide as much as it is a reference for myself and others that just need something quick to work off of.

The first command I reach for is the find utility. If you didn’t know, you can pass arguments to find that will get executed when find finds a match. This command will find all Nmap greppable output files and grep for lines with “Status: Up”:

$ find . -name *.gnmap -exec grep “Status: Up” {} \;
Host: 192.168.1.1 () Status: Up
Host: 192.168.1.10 () Status: Up
Host: 192.168.1.40 () Status: Up
Host: 192.168.1.42 () Status: Up
Host: 192.168.1.102 () Status: Up
Host: 192.168.1.103 () Status: Up

You can improve the above command further with `awk` to only print out the IP addresses that appeared online:

$ find . -name *.gnmap -exec \
awk ‘/Status:\ Up/ {print $2}’ {} \;
192.168.1.1
192.168.1.10
192.168.1.40
192.168.1.42
192.168.1.102
192.168.1.103

If you need to know what file the matching results came from, you can do the following:

$ find . -name *.gnmap -exec \
awk ‘/Status:\ Up/ {print $2}’ {} \; -print
192.168.1.1
192.168.1.10
192.168.1.40
192.168.1.42
192.168.1.102
192.168.1.103
./nmap_scans/192.168.1.0_24.gnmap

Nmap results are pretty easy to go through, but what if you have Nessus nbe files? If you’ve ever seen a Nessus nbe file, it isn’t pretty. The following command will run through a nbe file and print out IP addresses and NetBIOS names in CSV format. Nessus PluginID 10150 identifies scanned hosts’ NetBIOS names.

$ find . -name *.nbe -exec awk -F ‘|’ ‘$5==”10150″‘ {} \; | \
awk ‘{print $63″|”$1}’ | awk -F ‘|’ ‘{print $1″,”$4}’
stacker,192.168.1.10
slaptiva,192.168.1.40
thinker,192.168.1.42

The last script I have found useful is when I need to pull text broken up over several lines back into one really long line. I accomplish this using awk, with the following command:

$ awk ‘BEGIN {RS=”\n\n”; FS=”\n”} {for (i=1;i<=NF;i++) \
printf “%s “, $i ; printf “\n\n” }’

The Handy One-Liners for Awk and Handy One-Liners for Sed have both been awesome references that I keep bookmarked. I have also found myself keeping UNIX Power Tools open all day as well.