Posts Tagged: NIDS

Snort 2.9.8.x on Ubuntu – Part 2: Configure Snort to Run as a NIDS

UPDATE: Snort 2.9.9.x has been released. Please see the updated series of articles here or my quick install guide here.

I am leaving this older guide online for anyone who wants to install this older version of Snort on Ubuntu, but you really should be using the updated guide for the 2.9.9.x version of Snort, since support for older versions of Snort are set to expire, and the updated guide is kept more up to date and includes BASE instead of Snorby for a Web GUI.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  1. Installing Snort
  2. Configure Snort to Run as a NIDS
  3. Writing and Testing a Single Rule With Snort
  4. Installing Barnyard2
  5. Installing PulledPork
  6. Creating Upstart Scripts for Snort
  7. Creating systemD Scripts for Snort
  8. Installing Snorby on Ubuntu 12
  9. Installing Snorby on Ubuntu 14
  10. Installing Snorby on Ubuntu 15
  11. Conclusion

Configure Snort to Run as a NIDS

Overview

This is the second in a set of articles will guide you through the steps of installing and configuring Snort as a Network Intrusion Detection System (NIDS). In the previous article we installed the Snort binary and verified that it correctly executed. In this section, we will configure Snort to run as a NIDS by creating the files and folders that Snort expects when running as a NIDS, and we will learn about the Snort configuration file: snort.conf.

Basic Configuration

First off, for security reasons we want Snort to run as an unprivileged user. We create a snort user and group for this purpose:

sudo groupadd snort
sudo useradd snort -r -s /sbin/nologin -c SNORT_IDS -g snort

Next, we need to create a number of files and folders that Snort expects when running in NIDS mode.  We will then change the ownership of those files to our new snort user. Snort stores configuration files in /etc/snort, rules in /etc/snort/rules, /usr/local/lib/snort_dynamicrules, and stores its logs in /var/log/snort:

# Create the Snort directories:
sudo mkdir /etc/snort
sudo mkdir /etc/snort/rules
sudo mkdir /etc/snort/rules/iplists
sudo mkdir /etc/snort/preproc_rules
sudo mkdir /usr/local/lib/snort_dynamicrules
sudo mkdir /etc/snort/so_rules

# Create some files that stores rules and ip lists
sudo touch /etc/snort/rules/iplists/black_list.rules
sudo touch /etc/snort/rules/iplists/white_list.rules
sudo touch /etc/snort/rules/local.rules
sudo touch /etc/snort/sid-msg.map

# Create our logging directories:
sudo mkdir /var/log/snort
sudo mkdir /var/log/snort/archived_logs

# Adjust permissions:
sudo chmod -R 5775 /etc/snort
sudo chmod -R 5775 /var/log/snort
sudo chmod -R 5775 /var/log/snort/archived_logs
sudo chmod -R 5775 /etc/snort/so_rules
sudo chmod -R 5775 /usr/local/lib/snort_dynamicrules

# Change Ownership on folders:
sudo chown -R snort:snort /etc/snort
sudo chown -R snort:snort /var/log/snort
sudo chown -R snort:snort /usr/local/lib/snort_dynamicrules

We now need to move the following files from the extracted Snort tarball to the snort configuration folder:

  • classification.config describes the types of attack classifications that Snort understands (grouping rules into these types of classifications), such as trojan-activity or system-call-detect. The list of classifications can be found in section 3.4.6 of the Snort Manual
  • file_magic.conf describes rules for identifying file types.
  • reference.config contains urls that are referenced in the rules that provide more information about alerts.
  • snort.conf is the configuration file for Snort, it tells Snort where resources are located, and how to output alerts, among other things.
  • threshold.conf allows you to control the number of events that are required to generate an alert, which can help suppress noisy alerts. More information here.
  • attribute table.dtd lets Snort use outside information to determine protocols and policies. More information here.
  • gen-msg.map tells Snort which pre-processor is used by which rule. More information here.
  • unicode.map provides a mapping between Unicode languages and the identifier. This file is required by Snort in order to start.

Run the commands below to move the files listed above to the /etc/snort folder:

cd ~/snort_src/snort-2.9.8.0/etc/
sudo cp *.conf* /etc/snort
sudo cp *.map /etc/snort
sudo cp *.dtd /etc/snort

cd ~/snort_src/snort-2.9.8.0/src/dynamic-preprocessors/build/usr/local/lib/snort_dynamicpreprocessor/
sudo cp * /usr/local/lib/snort_dynamicpreprocessor/

The Snort configuration folder and file structure should now look like the following:

	user@snortserver:~$ tree /etc/snort
	/etc/snort
	├── attribute_table.dtd
	├── classification.config
	├── file_magic.conf
	├── gen-msg.map
	├── preproc_rules
	├── reference.config
	├── rules
	│.. ├── local.rules
	│.. ├── iplists
	│    .. ├── black_list.rules
	│    .. ├── white_list.rules
	├── snort.conf
	├── so_rules
	├── threshold.conf
	└── unicode.map

Editing the Snort Configuration File

The Snort configuration file is stored at /etc/snort/snort.conf, and contains all the settings that Snort will use when it is run in NIDS mode. This is a large file (well over 500 lines), and contains a number of options for the configuration of Snort. We are interested in only a few settings at this time.

First, we need to comment out the lines that causes Snort to import the default set of rule files. We do this because we will be using PulledPork to manage our rulesets, which saves all the rules into a single file. The easy way to comment out all these lines is to use sed to append the “#” (hash) character to those lines.  This is accomplished by running the following command:

sudo sed -i 's/include \$RULE\_PATH/#include \$RULE\_PATH/' /etc/snort/snort.conf

The result of this command is that lines 547 to 651 in snort.conf will now be commented out, which will prevent Snort from loading those rule files on start-up. These rule files do not exist, and will cause Snort to generate an error if it tries to load a file that doesn’t exist. If you were to manually download the rule files from the snort website and extract them to the /etc/snort/rules folder, then you would want those rules to be un-commented out. We will use PulledPork (configured later) to manage all our rules and save them into a single file, which is why we need all those rule files to be commented out.

Next, we need to manually edit a few lines in the snort.conf file. Use vi (or your favorite editor) to edit /etc/snort/snort.conf:

sudo vi /etc/snort/snort.conf

First, we need to let Snort know the network range of your home network (the assets you are trying to protect) and all other external networks.  We do this by editing lines 45 and 48 of snort.conf to tell it the IP ranges of these two networks. In the example below, our home network is 10.0.0.0 with a 24 bit subnet mask (255.255.255.0), and our external networks are all other networks.

ipvar HOME_NET 10.0.0.0/24 	# (line 45) make this match your internal (friendly) network

Note: it is not recommended to set EXTERNAL_NET to !$HOME NET as recommended in some guides, since it can cause Snort to miss alerts.

Next we need to tell Snort about the locations of all the folders we created earlier.  These settings are also part of the snort.conf file.  I have included the line numbers after the hash so you can more easily find the setting (do not write the line number, just change the path to match what is below):

var RULE_PATH /etc/snort/rules						# line 104
var SO_RULE_PATH /etc/snort/so_rules				# line 105
var PREPROC_RULE_PATH /etc/snort/preproc_rules		# line 106

var WHITE_LIST_PATH /etc/snort/rules/iplists		# line 113
var BLACK_LIST_PATH /etc/snort/rules/iplists		# line 114

Finally, we want to enable one included rule file: /etc/snort/rules/local.rules.  We will use this file to store our own rules, including one rule that we will write in the next article in this series that will allow us to easily check that Snort is correctly generating alerts.  Un-comment the following line (line 545) by deleting the hash from the beginning of the line:

include $RULE_PATH/local.rules

Testing Snort with our Configuration File

Snort has the ability to validate the configuration file, and you should do this whenever you make modifications to snort.conf. Run the following command to have Snort test the configuration file:

sudo snort -T -c /etc/snort/snort.conf -i eth0

The -T tells snort to test, and -c tells snort the path to the configuration file, and you are required to specify an interface you want to listen to with -i (this is a new requirement for the 2.9.8.x version of snort). You should see some output, with the following lines at the end:

    ...
	Snort successfully validated the configuration!
	Snort exiting

Congratulations, if you have output similar to the above then you have successfully Configured Snort to run as a NIDS. Continue to the next section: Writing and Testing a Single Rule With Snort.

The Reputation Preprocessor in Snort – Blacklists and Whitelists

In this article, we are going to look at Snort’s Reputation Preprocessor. We will look at how this preprocessor is used to use IP blacklists and IP whitelists (known together as IP lists) to either block, alert, or allow traffic based on the sender’s and/or recipient’s IP address. I will show you how to configure, test, and troubleshoot the reputation preprocessor and associated IP lists. Finally we will look at how PulledPork can be configured to download blacklists automatically.

History of the Reputation Preprocessor

Before the reputation preprocessor was developed, if you wanted to block or alert on traffic to or from a specific IP address or range, you had to create a rule for that IP address or range of IP addresses. This works well for a very small set of addresses that do not change often. Unfortunately, in today’s environment IP addresses for malicious hosts change rapidly, and there are a very large number of malicious addresses. The administrative overhead of creating and maintaining rules specifically for these addresses became difficult, as well as the problem of the additional processor load on the Snort detection engine with the addition of so many extra rules. The current Talos blacklist has over 40,000 entries, so you can imagine that the effort of using regular Snort rules to block that many IP addresses was difficult, to say the least.

The solution to these difficulties was the reputation preprocessor, first included in the Snort 2.9.1.x release of Snort.

Overview of the Reputation Preprocessor

The reputation preprocessor was created to allow Snort to use a file full of just IP addresses to identify bad hosts and trusted hosts. Malicious IP addresses are stored in blacklists, and trusted IP addresses are stored in whitelists. The reputation preprocessor loads these lists when Snort starts, and compares all traffic against those lists. Snort checks both the sending and receiving IP address in each packet against every entry in the IP lists, and if the IP addresses in the packet matches an IP address on the blacklist, whitelist, or both lists, Snort can take a few different actions: Snort can either generate an alert, block the packet, allow the packet without any other processing (skipping all other rules), or let the packet continue through the rest of the regular rule checks. The action that Snort takes depends on how you have the reputation preprocessor configured, and if Snort is running in IDS or IPS mode (Snort can only drop packets when running in IPS mode, for obvious reasons).

The reputation preprocessor is the first preprocessor that a packet encounters in Snort (after being assembled by the decoder). The reason for this is that since the reputation preprocessor can mark trusted packets to skip the rest of the preprocessors and rule engine, or can drop the packet, it can help to reduce the load on the Snort system.

You can manually create whitelists and blacklists, although you are probably better off using PulledPork to automatically download blacklist files. The good news is that if you are using PulledPork and you’ve got the reputation preprocessor configured correctly, all this just works for you. If you want to change the way things work, are doing something special, or just want to understand Snort better, then the rest this guide is for you.

What Happens When a Packet Matches an Entry in an IP List

Assuming your reputation preprocessor is configured correctly, and you have entries in your whitelist and blacklist files: the reputation preprocessor is the first processor that a packet encounters in Snort after being assembled by the decoder. The reputation preprocessor compares the source and destination IP addresses in the packet against the IP addresses in both the whitelist and blacklist files. If one of the IP addresses (sender or recipient) for the packet is on the blacklist, then an alert is generated (with GID:136, and SID:1) and no further processing is done on the packet (it skips all other processors and the rule engine). If you are running in NIDS mode, only an alert is generated. If you are running inline in IPS mode, then the packet is dropped. If one of the IP addresses is on a whitelist: the packet can either bypass all other preprocessors and the rule engine and continue on, or it can be “unblacked”. When a packet is unblocked, it is treated like a regular packet, being processed by the other preprocessors and rules, even if the address is on the blacklist.

If your Snort server is running as a NIDS (network intrusion detection system) then alerts are generated (we are detecting) for packets that match one of the IP lists. If you are running Snort as a NIPS (network intrusion prevention system), then traffic can be dropped instead of generating alerts when the packet IP address matches an IP in the blacklist . If one IP address is on the whitelist, and the other address is on the blacklist, the action taken will depend on your configuration, namely the priority and white reputation preprocessor options described below.

Configuring the Reputation Preprocessor

The reputation preprocessor is configured in your snort.conf. Many standard Snort installations place this file at /etc/snort/snort.conf. Open this snort configuration file and find the section for the reputation preprocessor. This should be around line number 506 if you haven’t changed your snort.conf much. If the preprocessor is disabled with the hash symbol (#) at the beginning of each line for the preprocessor, you can enable it by removing the hash symbol from the beginning of each line. The preprocessor configuration should look similar to the following when enabled:

# Reputation preprocessor. For more information see README.reputation
preprocessor reputation: \
   memcap 500, \
   priority whitelist, \
   nested_ip inner, \
   whitelist $WHITE_LIST_PATH/white_list.rules, \
   blacklist $BLACK_LIST_PATH/black_list.rules 

There are a few other lines in your snort.conf that relate to IP lists as well. The following two lines tell Snort where the folder is that stores the whitelists and blacklists:

var WHITE_LIST_PATH /etc/snort/rules/iplists		# line 113 in snort.conf
var BLACK_LIST_PATH /etc/snort/rules/iplists		# line 114 in snort.conf

note that you could just use an absolute path for WHITE_LIST_PATH and BLACK_LIST_PATH rater than using the $BLACK_LIST_PATH/filename as in the above two examples.

We also need a folder to hold your IP lists, and the empty whitelist and blacklist. These three items are what we told Snort to use in the above two sections of the snort.conf (create these if they don’t exist, based on your preprocessor configuration):

# these commands will create your whitelist and blacklist files as configured in the above example
sudo mkdir /etc/snort/rules/iplists
sudo touch /etc/snort/rules/iplists/black_list.rules
sudo touch /etc/snort/rules/iplists/white_list.rules

Since you’ve edited your snort.conf, it’s always a good idea to test that you didn’t create any errors. A simple test (change for your system as needed) and make sure no issues are reported:

sudo snort -T -c /etc/snort/snort.conf -i eth0

Manually Adding Entries to IP Lists

If you want to build your own whitelists and blacklists, this is easy. Snort can easily load multiple whitelists and blacklists (see the section below for instructions). The list should be a text document with either plain IP addresses (specifying a single host), or IP addresses in CIDR format, with one entry per line. You can have full-line and inline comments by using the hash (#) symbol. An example of all these options is below:

# This is a full-line comment
# This list could be a whitelist or a blacklist, it only depends on what you tell Snort to treat it as

# Add these single hosts to this list:
10.0.0.120    
10.0.0.222       # This is an inline comment.

# Add these entire subnets (in CIDR format) to the list:
10.2.0.0/24
224.0.0.0/4      # add the entire multicast subnet to this list

Allowing Local IP Addresses

If you want the reputation preprocessor to recognize (not ignore) private network addresses (the ones on your home or internal network) which all fall in the local ranges:

  • 10.0.0.0 – 10.255.255.255 (10.0.0.0/8)
  • 172.16.0.0 – 172.31.255.255 (172.16.0.0/12)
  • 192.168.0.0 – 192.168.255.255 (192.168.0.0/16)

then add the scan_local option to the reputation preprocessor, as show below in line 6:

# Reputation preprocessor. For more information see README.reputation
preprocessor reputation: \
   memcap 500, \
   priority whitelist, \
   nested_ip inner, \
   scan_local, \
   whitelist $WHITE_LIST_PATH/white_list.rules, \
   blacklist $BLACK_LIST_PATH/black_list.rules 

This option allows you to test the reputation preprocessor with private addresses (alert on traffic from the 10.0.0.0/24 subnet for example). Without this option, all IP addresses in your IP lists from a private address will be not be compared against the IP lists.

Configuring IP List Actions and Precedence

The two reputation preprocessor configuration options that determine how IP lists affect the processing of packets are priority and white.

priority: When a packet has one IP on a blacklist and the other IP on a whitelist (sender IP address and receiver IP address), this option determines which is more important. If this is set to blacklist, then the packet will generate an alert. If this is set to whitelist, then the process will be allowed to pass. An example of this setting (truncated for simplicity):

preprocessor reputation: \
   priority whitelist, \
   ...

white: this option can be set to either unblack or trust. When set to unblack, if the packet also has an address that is in the IP blacklist (say the source IP address is in the whitelist and the destination ip address of that same packet is in the blacklist), then the packet will continue to process through the other preprocessors as if it was not on the blacklist. Note that for the packet to continue to be processed, the priority must be set to whitelist. When white is set to trust, then the packet is implicitly trusted and bypasses all further processing. An example of this:

preprocessor reputation: \
   white unblack, \
   ...

Setting up local.rules to Generate Alerts for Blacklist Events

If you are not using PulledPork to manage your rulesets, and have manually configured your whitelists and / or blacklists, you need to tell Snort to generate an alert when it sees packets that match these IP lists.

You need a local.rules file loaded by Snort with the following rules (if you need help setting this up, please see my article here):

alert ( msg: "REPUTATION_EVENT_BLACKLIST"; sid: 1; gid: 136; rev: 1; metadata: rule-type preproc ; classtype:bad-unknown; )
alert ( msg: "REPUTATION_EVENT_WHITELIST"; sid: 2; gid: 136; rev: 1; metadata: rule-type preproc ; classtype:bad-unknown; )

NOTE: if you are using PulledPork to manage rules, you don’t need the above lines, it will add these rules automatically.

Rules with GID 136 are rules triggered by the reputation preprocessor. There are 3 SID’s for that processor:

  1. Packets are blacklisted
  2. Packets are whitelisted
  3. Packets are inspected

We don’t want to create a rule with a SID of 3 because that would be a lot of alerts (essentially all packets).

There is an easy way to test the reputation processor works. First, make sure your reputation preprocessor is properly configured, and you have the two rules listed above in your local.rules file (and make sure that Snort is loading your local.rules).

Next, add the IP address of a second host on your network (other than your snort host) to your black_list.rules file. This IP address will be the address that Snort generates alerts on, due to the IP address being in the blacklist file.

Start Snort with the following command (change for your specific system settings). This will generate alerts to the console:

sudo /usr/local/bin/snort -A console -q -c /etc/snort/snort.conf -i eth0

If you now ping your Snort server from the system that is in your blacklist, you should see alerts display on the console. Use Ctrl-C to stop Snort from running. In the example below, the first alert is from me ssh-ing into the Snort server from the blacklisted computer. Next I pinged the Snort server 8 times, then used wget to try to pull a webpage from the Snort server:

12/09-20:25:10.423907  [**] [136:1:1] (spp_reputation) packets blacklisted [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 10.0.0.72:51312 -> 10.0.0.101:22
12/09-20:25:15.355331  [**] [136:1:1] (spp_reputation) packets blacklisted [**] [Classification: Potentially Bad Traffic] [Priority: 2] {ICMP} 10.0.0.101 -> 10.0.0.105
12/09-20:25:15.355375  [**] [136:1:1] (spp_reputation) packets blacklisted [**] [Classification: Potentially Bad Traffic] [Priority: 2] {ICMP} 10.0.0.105 -> 10.0.0.101
12/09-20:25:16.355231  [**] [136:1:1] (spp_reputation) packets blacklisted [**] [Classification: Potentially Bad Traffic] [Priority: 2] {ICMP} 10.0.0.101 -> 10.0.0.105
12/09-20:25:16.355270  [**] [136:1:1] (spp_reputation) packets blacklisted [**] [Classification: Potentially Bad Traffic] [Priority: 2] {ICMP} 10.0.0.105 -> 10.0.0.101
12/09-20:25:17.355272  [**] [136:1:1] (spp_reputation) packets blacklisted [**] [Classification: Potentially Bad Traffic] [Priority: 2] {ICMP} 10.0.0.101 -> 10.0.0.105
12/09-20:25:17.355310  [**] [136:1:1] (spp_reputation) packets blacklisted [**] [Classification: Potentially Bad Traffic] [Priority: 2] {ICMP} 10.0.0.105 -> 10.0.0.101
12/09-20:25:18.355293  [**] [136:1:1] (spp_reputation) packets blacklisted [**] [Classification: Potentially Bad Traffic] [Priority: 2] {ICMP} 10.0.0.101 -> 10.0.0.105
12/09-20:25:18.355319  [**] [136:1:1] (spp_reputation) packets blacklisted [**] [Classification: Potentially Bad Traffic] [Priority: 2] {ICMP} 10.0.0.105 -> 10.0.0.101
12/09-20:25:26.194898  [**] [136:1:1] (spp_reputation) packets blacklisted [**] [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 10.0.0.101:52671 -> 10.0.0.105:80
^C*** Caught Int-Signal

If you are wondering how I’m able to remote into the server from a blacklisted host, remember that we have configured Snort as an IDS (intrusion detection system), it only detects and alerts for malicious traffic. We have not configured Snort as an IPS (intrusion prevention system). More information on running Snort as an IPS here.

If you don’t see any alerts like above, run the below command to test your snort.conf,

sudo snort -T -c /etc/snort/snort.conf -i eth0

If Snort verifies the configuration file successfully (indicated in the final few lines of output), then scroll up through the output up to see if any IP addresses show in the reputation portion of the output (see line 6 below for our one IP address loaded from the blacklist file):

Reputation config: 
	WARNING: /etc/snort/snort.conf(512) => Keyword priority for whitelist is not applied when white action is unblack.
	    Processing whitelist file /etc/snort/rules/iplists/default.whitelist
	    Reputation entries loaded: 0, invalid: 0, re-defined: 0 (from file /etc/snort/rules/iplists/default.whitelist)
	    Processing blacklist file /etc/snort/rules/iplists/default.blacklist
	    Reputation entries loaded: 1, invalid: 0, re-defined: 0 (from file /etc/snort/rules/iplists/default.blacklist)
	    Reputation total memory usage: 329636 bytes
	    Reputation total entries loaded: 1, invalid: 0, re-defined: 0
	    Memcap: 500 (Default) M bytes 
	    Scan local network: DISABLED (Default)
	    Reputation priority:  whitelist(Default) 
	    Nested IP: inner (Default) 
	    White action: unblack (Default) 
	    Shared memory is Not supported.

You will also want to verify that our two rules (from local.rules) have loaded in the rules section of the output (note that they are recognized as preprocessor rules):

+++++++++++++++++++++++++++++++++++++++++++++++++++
Initializing rule chains...
2 Snort rules read
    0 detection rules
    0 decoder rules
    2 preprocessor rules
2 Option Chains linked into 1 Chain Headers
0 Dynamic rules
+++++++++++++++++++++++++++++++++++++++++++++++++++

+-------------------[Rule Port Counts]---------------------------------------
|             tcp     udp    icmp      ip
|     src       0       0       0       0
|     dst       0       0       0       0
|     any       2       0       0       0
|      nc       2       0       0       0
|     s+d       0       0       0       0
+----------------------------------------------------------------------------

if both of those are correct, start looking at your IP addresses to verify that you have everything correct. If you are using private IP addresses (like 10.x.x.x) then remember that you need to enable scan_local in the reputation preprocessor.

Understanding nested_ip

Packets are often encapsulated in other packets, such as GRE or IP in IP tunnels. If Snort can see the IP header information of the encapsulated (internal) packet, you can choose to compare the inner packet, outer packet, or both inner and outer IP headers against your IP lists, by setting the nested_ip option to one of the following: inner, outer, or both, which tells the reputation preprocessor to check the inner IP address, the outer IP address, or to check both IP addresses (both inner and outer). One example is below:

preprocessor reputation: \
   nested_ip inner, \
   ...

PulledPork and Blacklists

PulledPork can automatically download blacklists (but not whitelists), and is configured to do so by default. When configuring pulledpork.conf (usually located in /etc/snort/), you will need to have the following lines configured correctly.

First we need to tell PulledPork which IP blacklist to download. By default we download the Talos blacklist, which is found at line 24 of pulledpork.conf. No changes are required to this line, since it’s enabled by default:

# pulledpork.conf - Line 24
rule_url=http://talosintel.com/feeds/ip-filter.blf|IPBLACKLIST|open

Line 141 in PulledPork.conf points to local file where you will save the blacklist that you download. This will be the same file you configured in the reputation preprocessor with the directive: BLACK_LIST_PATH in your snort.conf. This is the where PulledPork will write the blacklists to:

# pulledpork.conf - Line 141
black_list=/etc/snort/rules/iplists/black_list.rules

The other configuration item in PulledPork.conf that is related to blacklists is line 150. This is used to have Snort reload the IP lists without a reboot (although that requires a lot more configuration, and re-compiling snort with -enable-shared-rep and –enable-control-socket, which isn’t covered here). You just need to make sure this folder path points to your iplists folder so there are no errors, although we won’t be using this feature:

# pulledpork.conf - Line 150
IPRVersion=/etc/snort/rules/iplists

after running PulledPork, you should see the black_list.rules file be populated with a number of IP addresses (over 40,000 at this time from the Talos blacklist).

Using Multiple IP Lists

You can have the reputation preprocessor load multiple whitelists and blacklists. This is good if you have a personal blacklist that you don’t want overwritten by PulledPork. An example or the reputation preprocessor configured with two whitelists and two blacklists:

preprocessor reputation: \
   memcap 500, \
   priority whitelist, \
   nested_ip inner, \
   whitelist $WHITE_LIST_PATH/white_list.rules, \
   whitelist etc/snort/rules/iplists/some_whitelist.rules, \
   blacklist /etc/snort/rules/iplists/some_blacklist.rules, \
   blacklist $BLACK_LIST_PATH/black_list.rules 

Useful References

Snort’s guide on the reputation preprocessor is here. This explains every option for the preprocessor in detail.
README.reputation: the Snort overview of the reputation preprocessor.

Conclusion

I hope this has been a good overview of the reputation preprocessor in Snort. I wrote this article because I found most of the information on the web to be scattered, incomplete, and sometimes contradictory (as things tend to often be on the internet). I am hoping this article helps to explain the reputation preprocessor at a high-enough level as to make you wiser, as well as deeply enough that you can bend it to your will. If you have any questions or recommendations, please contact me. I can’t always answer questions right away, but I will do my best to get back to you. I welcome all recommendations and corrections.

Snort 2.9.8.x on Ubuntu – Part 3: Writing and Testing a Single Rule With Snort

UPDATE: Snort 2.9.9.x has been released. Please see the updated series of articles here or my quick install guide here.

I am leaving this older guide online for anyone who wants to install this older version of Snort on Ubuntu, but you really should be using the updated guide for the 2.9.9.x version of Snort, since support for older versions of Snort are set to expire, and the updated guide is kept more up to date and includes BASE instead of Snorby for a Web GUI.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  1. Installing Snort
  2. Configure Snort to Run as a NIDS
  3. Writing and Testing a Single Rule With Snort
  4. Installing Barnyard2
  5. Installing PulledPork
  6. Creating Upstart Scripts for Snort
  7. Creating systemD Scripts for Snort
  8. Installing Snorby on Ubuntu 12
  9. Installing Snorby on Ubuntu 14
  10. Installing Snorby on Ubuntu 15
  11. Conclusion

Writing and Testing a Single Rule With Snort

In the previous two articles in this series, we installed Snort an configured it to run as a NIDS. In this article, we are going to create a rule which causes Snort to generate an alert whenever it sees an ICMP message. If you want, you can skip this section, as it is not required to get a Snort NIDS up and running, but it will help you to gain a much better understanding of how Snort rules are created and loaded.

Onward

In the previous article, we created the /etc/snort/rules/local.rules file and left it empty. We also edited the snort.conf file to tell Snort to load this local.rules file (when we un-commented the line: include $RULE_PATH/local.rules in snort.conf). When Snort starts, it will use the include directive in snort.conf to load all rules in local.rules. The local.rules file is a place where we can place rules that are specific to our environment, and is great for testing.

First, we need to edit the local.rules file:

sudo vi /etc/snort/rules/local.rules

input the following text and save the file:

alert icmp any any -> $HOME_NET any (msg:"ICMP test detected"; GID:1; sid:10000001; rev:001; classtype:icmp-event;)

What this rule says is that for any ICMP packets it sees from any network to our HOME_NET, generate an alert with the text ICMP test. The other information here (GID, REV, classtype) are used group the rule, and will be helpful when you install Snorby.

Since we have made changes to the files that snort loads, it is a good idea to test the configuration file again:

sudo snort -T -c /etc/snort/snort.conf -i eth0

If successful, you should be able to scroll up through the output and see that Snort has loaded our rule:

		+++++++++++++++++++++++++++++++++++++++++++++++++++
		Initializing rule chains...
		1 Snort rules read
			1 detection rules
			0 decoder rules
			0 preprocessor rules
		1 Option Chains linked into 1 Chain Headers
		0 Dynamic rules
		+++++++++++++++++++++++++++++++++++++++++++++++++++

		+-------------------[Rule Port Counts]---------------------------------------
		|             tcp     udp    icmp      ip
		|     src       0       0       0       0
		|     dst       0       0       0       0
		|     any       0       0       1       0
		|      nc       0       0       1       0
		|     s+d       0       0       0       0
		+----------------------------------------------------------------------------

Now to test the rule.  We need to verify that Snort generates an alert when it processes an ICMP packet. We will launch Snort with the following options:

-A console                    the console option prints fast mode alerts to stdout
-q                            Quiet. Don't show banner and status report.
-u snort                      run snort as the following user after startup
-g snort                      run snort as the following group after startup
-c /etc/snort/snort.conf      the path to our snort.conf file
-i eth0                       the interface to listen on

Run Snort with the command below, modifying the parameters as required specific for your configuration:

sudo /usr/local/bin/snort -A console -q -u snort -g snort -c /etc/snort/snort.conf -i eth0

Note: If you are running Ubuntu 15.10, remember that your interface name is not eth0.

Once you have started Snort with the above command, you need use another computer or another terminal window to ping the interface that you directed Snort to listen on.  You should see output similar to the below on the terminal of the Snort machine:

10/31-02:27:19.663643  [**] [1:10000001:1] ICMP test detected [∗∗] [Classification: Generic ICMP event] [Priority:3] {ICMP} 10.0.0.74 -> 10.0.0.64
10/31-02:27:19.663675  [**] [1:10000001:1] ICMP test detected [∗∗] [Classification: Generic ICMP event] [Priority:3] {ICMP} 10.0.0.64 -> 10.0.0.74
10/31-02:27:20.658378  [**] [1:10000001:1] ICMP test detected [∗∗] [Classification: Generic ICMP event] [Priority:3] {ICMP} 10.0.0.74 -> 10.0.0.64
10/31-02:27:20.658404  [**] [1:10000001:1] ICMP test detected [∗∗] [Classification: Generic ICMP event] [Priority:3] {ICMP} 10.0.0.64 -> 10.0.0.74
10/31-02:27:21.766521  [**] [1:10000001:1] ICMP test detected [∗∗] [Classification: Generic ICMP event] [Priority:3] {ICMP} 10.0.0.74 -> 10.0.0.64
10/31-02:27:21.766551  [**] [1:10000001:1] ICMP test detected [∗∗] [Classification: Generic ICMP event] [Priority:3] {ICMP} 10.0.0.64 -> 10.0.0.74
10/31-02:27:22.766167  [**] [1:10000001:1] ICMP test detected [∗∗] [Classification: Generic ICMP event] [Priority:3] {ICMP} 10.0.0.74 -> 10.0.0.64
10/31-02:27:22.766197  [**] [1:10000001:1] ICMP test detected [∗∗] [Classification: Generic ICMP event] [Priority:3] {ICMP} 10.0.0.64 -> 10.0.0.74
^C*** Caught Int-Signal

You have to use ctrl-c to stop snort from running after the above output. What the above example shows is the 4 ICMP Echo Request and Reply messages between our Snort server (IP 10.0.0.64) and our other machine (10.0.0.74). If you look in /var/log/snort, you will also see a file with the name snort.log.nnnnnnnnnn (the n’s are replaced by numbers), which contains the same information that Snort printed to the screen.

Congratulations, if you have output similar to the above then you have successfully created a rule for Snort to alert on. Continue to the next section to Install Barnyard2.

Snort 2.9.8.x on Ubuntu – Part 4: Installing Barnyard2

UPDATE: Snort 2.9.9.x has been released. Please see the updated series of articles here or my quick install guide here.

I am leaving this older guide online for anyone who wants to install this older version of Snort on Ubuntu, but you really should be using the updated guide for the 2.9.9.x version of Snort, since support for older versions of Snort are set to expire, and the updated guide is kept more up to date and includes BASE instead of Snorby for a Web GUI.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  1. Installing Snort
  2. Configure Snort to Run as a NIDS
  3. Writing and Testing a Single Rule With Snort
  4. Installing Barnyard2
  5. Installing PulledPork
  6. Creating Upstart Scripts for Snort
  7. Creating systemD Scripts for Snort
  8. Installing Snorby on Ubuntu 12
  9. Installing Snorby on Ubuntu 14
  10. Installing Snorby on Ubuntu 15
  11. Conclusion

Installing Barnyard2

In the previous three articles in this series, we installed Snort, configured it to run as a NIDS, and configured a rule. In this article, we are going to install and configure Barnyard2, which is a dedicated spooler that will help reduce the load on the Snort server.

Notes

You will be prompted to create both a MySQL root password, as well as a password for a MySQL database snort user. In the examples below, we have chose to use MYSQLROOTPASSWORD as the MySQL root password, and MYSQLSNORTPASSWORD as the MySQL database snort user. Please note the differences when working below.

Onward

First, we need to install some pre-requisites:

sudo apt-get install -y mysql-server libmysqlclient-dev mysql-client autoconf libtool

You will be prompted for the MySQL root password. We choose MYSQLROOTPASSWORD for the below examples.

Next, we need to edit the snort.conf:

sudo vi /etc/snort/snort.conf

We need to add a line that tells Snort to output events in binary form (so that Barnyard2 can read them). After line 520 in /etc/snort/snort.conf (a line that is a commented-out example), add the following line and save the file:

output unified2: filename snort.u2, limit 128

This line tells snort to output events in the unified2 binary format (which is easier for snort to output rather than human-readable alerts).

Next we need to get, configure, and install Barnyard2.

Note on Barnyard2 Version: In the commands below, we will be downloading a specific snapshot of Barnyard2 from github: Barnyard2 version 2.1.14 with commits from Oct 21, 2015 (this is the latest version at this time). I chose not to use the latest stable release: 2.1.13 because some patches have been added after that release that are important, and I chose not to use the Head release, because that will change after the release of this guide, and I won’t have had the ability to test it. If you want, you can (and probably will want) to use the current head release of Barnyard2, but if you have issues, you can always come back and use the version I’ve used below which I have verified will work with the other pieces of software in this guide.

cd ~/snort_src
wget https://github.com/firnsy/barnyard2/archive/7254c24702392288fe6be948f88afb74040f6dc9.tar.gz -O barnyard2-2-1.14-336.tar.gz
tar zxvf barnyard2-2-1.14-336.tar.gz
mv barnyard2-7254c24702392288fe6be948f88afb74040f6dc9 barnyard2-2-1.14-336
cd barnyard2-2-1.14-336
autoreconf -fvi -I ./m4

Barnyard2 needs access to the dnet.h library, which we installed with the Ubuntu libdumbnet package earlier. However, Barnyard2 expects a different file name for this library. Create a soft link from dnet.h to dubmnet.h so there are no issues:

sudo ln -s /usr/include/dumbnet.h /usr/include/dnet.h
sudo ldconfig

Depending on the architecture of your system (x86 or x64), choose to run one of the following lines to tell Barnyard2 where the MySQL libraries are:

./configure --with-mysql --with-mysql-libraries=/usr/lib/x86_64-linux-gnu
./configure --with-mysql --with-mysql-libraries=/usr/lib/i386-linux-gnu

Then continue with the install:

make
sudo make install

Barnyard2 is now installed to /usr/local/bin/barnyard2. To configure Snort to use Barnyard2, we need to copy a few files from the source package:

cd ~/snort_src/barnyard2-2-1.14-336
sudo cp etc/barnyard2.conf /etc/snort

# the /var/log/barnyard2 folder is never used or referenced
# but barnyard2 will error without it existing
sudo mkdir /var/log/barnyard2
sudo chown snort.snort /var/log/barnyard2

sudo touch /var/log/snort/barnyard2.waldo
sudo chown snort.snort /var/log/snort/barnyard2.waldo
sudo touch /etc/snort/sid-msg.map

Since Barnyard2 saves alerts to our MySQL database, we need to create that database, as well as a ‘snort’ MySQL user to access that database. Run the following commands to create the database and MySQL user.

When prompted for a password, use the MYSQLROOTPASSWORD . You will also be setting the MySQL snort user password in the fourth mysql command (to MYSQLSNORTPASSWORD), so change it there as well.

$ mysql -u root -p
mysql> create database snort;
mysql> use snort;
mysql> source ~/snort_src/barnyard2-2-1.13/schemas/create_mysql
mysql> CREATE USER 'snort'@'localhost' IDENTIFIED BY 'MYSQLSNORTPASSWORD';
mysql> grant create, insert, select, delete, update on snort.* to 'snort'@'localhost';
mysql> exit

Now that the Snort database has been created, we need to tell Barnyard2 about the details of the database. Edit the Barnyard2 configuration file:

sudo vi /etc/snort/barnyard2.conf

and at the end of the file, append this line:

output database: log, mysql, user=snort password=MYSQLSNORTPASSWORD dbname=snort host=localhost

Since the password is in the barnyard2.conf file, we should prevent other users from reading it:

sudo chmod o-r /etc/snort/barnyard2.conf

Now Barnyard2 is configured to work with Snort. To test, let’s run Snort and Barnyard2 and generate some alerts.  First, we run Snort as a daemon. We use the same parameters as before, with the addition of the -D flag, which tells snort to run as a daemon, and we removed -A Console since we don’t want alerts to show on the screen. Take note of the PID of the process so you can kill it later if needed:

sudo /usr/local/bin/snort -q -u snort -g snort -c /etc/snort/snort.conf -i eth0 -D

Ping the IP address of the interface specified above (eth0). If you check Snort’s log directory, you should see a file called snort.u2.nnnnnnnnnn (the n’s are replaced by numbers). These are the binary alerts that snort has written out for Barnyard2 to process.

Now we want to tell Barnyard2 to look at these events and load into the snort database instance. We run Barnyard2 with the following flags:

-c /etc/snort/barnyard2.conf        the Barnyard2 configuration file
-d /var/log/snort                   the location to look for the snort binary output file
-f snort.u2                         the name of the file to look for.
-w /var/log/snort/barnyard2.waldo   the path to the waldo file (checkpoint file).
-u snort                            run Barnyard2 as the following user after startup
-g snort                            run Barnyard2 as the following group after startup

Run the following command:

sudo barnyard2 -c /etc/snort/barnyard2.conf -d /var/log/snort -f snort.u2 -w /var/log/snort/barnyard2.waldo -g snort -u snort

you should see output similar to the below:

        --== Initialization Complete ==--

  ______   -*> Barnyard2 <*-
 / ,,_  \  Version 2.1.14 (Build 336)
 |o"  )~|  By Ian Firns (SecurixLive): http://www.securixlive.com/
 + '''' +  (C) Copyright 2008-2013 Ian Firns <firnsy@securixlive.com>

Using waldo file '/var/log/snort/barnyard2.waldo':
    spool directory = /var/log/snort
    spool filebase  = snort.u2
    time_stamp      = 1412527313
    record_idx      = 16
Opened spool file '/var/log/snort/snort.u2.1412527313'
Closing spool file '/var/log/snort/snort.u2.1412527313'. Read 16 records
Opened spool file '/var/log/snort/snort.u2.1412528990'
Waiting for new data

Use ctrl-cps to find it as in the example below):

user@snortserver:~$ ps aux | grep snort
      snort     1296  0.0  2.1 297572 43988 ?        Ssl  03:15   0:00 /usr/local/bin/snort -q -u snort -g snort -c /etc/snort/snort.conf -i eth0 -D
      user      1314  0.0  0.0   4444   824 pts/0    S+   03:17   0:00 grep --color=auto snort
user@snortserver:~$ sudo kill 1296
user@snortserver:~$

Congratulations, if you have output similar to the above then you have successfully Configured Barnyard2. Continue to the next section to install PulledPork

Snort 2.9.8.x on Ubuntu – Part 5: Installing PulledPork

UPDATE: Snort 2.9.9.x has been released. Please see the updated series of articles here or my quick install guide here.

I am leaving this older guide online for anyone who wants to install this older version of Snort on Ubuntu, but you really should be using the updated guide for the 2.9.9.x version of Snort, since support for older versions of Snort are set to expire, and the updated guide is kept more up to date and includes BASE instead of Snorby for a Web GUI.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  1. Installing Snort
  2. Configure Snort to Run as a NIDS
  3. Writing and Testing a Single Rule With Snort
  4. Installing Barnyard2
  5. Installing PulledPork
  6. Creating Upstart Scripts for Snort
  7. Creating systemD Scripts for Snort
  8. Installing Snorby on Ubuntu 12
  9. Installing Snorby on Ubuntu 14
  10. Installing Snorby on Ubuntu 15
  11. Conclusion

Onwards

In the previous two sections of this article, we installed Snort and configured it to work as a NIDS with Barnyard2 processing packets that generated alerts based on a rule. In this article, we are going to install a Perl script called PulledPork, which will automatically download the latest rulesets from the Snort website.

Oinkcode

To download the main free ruleset from Snort, you need an oinkcode. Register on the Snort website and save your oinkcode before continuing, as the oinkcode is required for the most popular free ruleset.

Installing PulledPork

Install the PulledPork pre-requisites:

sudo apt-get install -y libcrypt-ssleay-perl liblwp-useragent-determined-perl

Note on PulledPork Version: The command below installs the 0.7.2 version of PulledPork as it was on November 12, 2015 (fixing issue #194). There are issues with the base 0.7.2 version of PulledPork that are fixed with later patches, but a version release (0.7.3) hasn’t been created that includes those patches yet. I don’t want to use the 0.7.2 version of PulledPork because of the issues, and I don’t want to install the current Master version of PulledPork because it may change after the release of this guide (breaking this guide), so I’ve compromised by linking to a current (as of the time of this writing) version that works well and won’t change. As newer releases come out, they should work, but you will need to test if you choose something different. If you have issues running PulledPork, you may need to install newer versions than what I’m using, as they are actively working on the code at this time.

Download PulledPork and install. Here we copy the actual perl file to /usr/local/bin and the needed configuration files to /etc/snort:

cd ~/snort_src
wget https://github.com/finchy/pulledpork/archive/66241690356d54faa509625a78f80f326b75c339.tar.gz -O pulledpork-0.7.2-194.tar.gz
tar xvfvz pulledpork-0.7.2-194.tar.gz
mv pulledpork-66241690356d54faa509625a78f80f326b75c339 pulledpork-0.7.2-194

cd pulledpork-0.7.2-194/
sudo cp pulledpork.pl /usr/local/bin
sudo chmod +x /usr/local/bin/pulledpork.pl
sudo cp etc/*.conf /etc/snort

Test that PulledPork runs by running the following command, looking for the output below:

user@snortserver:~$ /usr/local/bin/pulledpork.pl -V
PulledPork v0.7.2 - E.Coli in your water bottle!

user@snortserver:~$

Now that we are sure that PulledPork works, we need to configure it:

sudo vi /etc/snort/pulledpork.conf

Make the following changes to the pulledpork.conf file. Anywhere you see ‹oinkcode› enter your oinkcode from the Snort website.  I have included line numbers to help you identify the location of these lines in the configuration file.

Line 19 & 26:  enter your oinkcode where appropriate (or comment out if no oinkcode)
Line 29:  Un-comment for Emerging threats ruleset (not tested with this guide)

Line 74:  change to: rule_path=/etc/snort/rules/snort.rules
Line 89:  change to: local_rules=/etc/snort/rules/local.rules
Line 92:  change to: sid_msg=/etc/snort/sid-msg.map
Line 96:  change to: sid_msg_version=2

Line 119:  change to: config_path=/etc/snort/snort.conf

Line 133:  change to: distro=Ubuntu-12-04

Line 141:  change to: black_list=/etc/snort/rules/iplists/black_list.rules
Line 150:  change to: IPRVersion=/etc/snort/rules/iplists

We want to run PulledPork once manually to make sure it works. We use the following flags:

 -c /etc/snort/pulledpork.conf      the location of the snort.conf file
 -l                                 Write detailed logs to /var/log

Run the following command:

sudo /usr/local/bin/pulledpork.pl -c /etc/snort/pulledpork.conf -l 

After this command runs (it takes some time), you should now see snort.rules in /etc/snort/rules, and .so rules in /usr/local/lib/snort_dynamicrules. Pulled Pork combines all the rulesets that it downloads into these two files. You need to make sure to add the line: include $RULE_PATH/snort.rules to the snort.conf file, or the pulled pork rules will never be read into memory when Snort starts:

sudo vi /etc/snort/snort.conf

Add the following line to enable snort to use the rules that PulledPork downloaded (line 547), after the line for local.rules:

include $RULE_PATH/snort.rules

Since we have modified snort.conf, we should test that Snort loads correctly in NIDS mode with the PulledPork rules included:

sudo snort -T -c /etc/snort/snort.conf -i eth0

Once that is successful, we want to test that Snort and Barnyard2 load correctly when run manually as daemons:

sudo /usr/local/bin/snort -u snort -g snort -c /etc/snort/snort.conf -i eth0 -D
sudo barnyard2 -c /etc/snort/barnyard2.conf -d /var/log/snort -f snort.u2 -w /var/log/snort/barnyard2.waldo -g snort -u snort -D

As before, ping the IP address of the Snort eth0 interface, and then check the database for more events (remember to use the MYSQLSNORTPASSWORD):

mysql -u snort -p -D snort -e "select count(*) from event"

The number of events reported should be greater than what you saw the last time you ran this command. Now that we are sure that PulledPork runs correctly, we want to add PulledPork to root’s crontab to run daily:

sudo crontab -e

Choose any editor if prompted, then add the following line and save:

01 04 * * * /usr/local/bin/pulledpork.pl -c /etc/snort/pulledpork.conf -l

Stop the running daemons from earlier testing:

user@snortserver:~$ ps aux | grep snort
snort     1296  0.0  2.1 297572 43988 ?        Ssl  03:15   0:00 /usr/local/bin/snort -q -u snort -g snort -c /etc/snort/snort.conf -i eth0 -D
user      1314  0.0  0.0   4444   824 pts/0    S+   03:17   0:00 grep --color=auto snort
user@snortserver:~$ sudo kill 1296

user@snortserver:~$ ps aux | grep barnyard2
snort     1298  0.0  2.1 297572 43988 ?        Ssl  03:15   0:00 barnyard2 -c /etc/snort/barnyard2.conf -d /var/log/snort -f snort.u2 -w /var/log/snort/barnyard2.waldo -g snort -u snort -D
user      1316  0.0  0.0   4444   824 pts/0    S+   03:17   0:00 grep --color=auto barnyard2
user@snortserver:~$ sudo kill 1298

Note: Snort needs to be reloaded to see the new rules. This can be done with kill -SIGHUP snort-pid, or you can restart the snort service (once that’s created in a later part of this guide).

Additional note about shared object rules: In addition to regular rules, The above section will download Shared object rules. Shared object rules are also known as ”Shared Object rules”, ”SO rules”, ”pre-compiled rules”, or ”Shared Objects”. These are detection rules that are written in the Shared Object rule language, which is similar to C.

These rules are pre-compiled by the provider of the rules, and allow for more complicated rules, and allow for obfuscation of rules (say to detect attacks that haven’t been patched yet, but the vendor wants to allow detection without revealing the vulnerability). These rules are compiled by the vendor for specific systems. One of these systems is Ubuntu 12, and luckily these rules also work on Ubuntu 14 and 15.

Congratulations, if you have output similar to the above then you have successfully Configured PulledPork. Continue to the next section to install startup scripts for Snort and Barnyard2. Choose one of the two following links, depending on your version of Ubuntu. You will create an Upstart scripts for Ubuntu 12 and 14, and a systemD scripts for Ubuntu 15.

Choose One of the following to continue:
Ubuntu 12 and 14: Creating Upstart Scripts for Snort and Barnyard2
Ubuntu 15: Creating systemD Scripts for Snort

Snort 2.9.8.x on Ubuntu – Part 6: Creating Upstart Scripts for Snort 12 and 14

UPDATE: Snort 2.9.9.x has been released. Please see the updated series of articles here or my quick install guide here.

I am leaving this older guide online for anyone who wants to install this older version of Snort on Ubuntu, but you really should be using the updated guide for the 2.9.9.x version of Snort, since support for older versions of Snort are set to expire, and the updated guide is kept more up to date and includes BASE instead of Snorby for a Web GUI.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  1. Installing Snort
  2. Configure Snort to Run as a NIDS
  3. Writing and Testing a Single Rule With Snort
  4. Installing Barnyard2
  5. Installing PulledPork
  6. Creating Upstart Scripts for Snort
  7. Creating systemD Scripts for Snort
  8. Installing Snorby on Ubuntu 12
  9. Installing Snorby on Ubuntu 14
  10. Installing Snorby on Ubuntu 15
  11. Conclusion

Overview

Creating the Upstart Scripts for Ubuntu 12 and 14

In the previous articles in this series, we have created a complete Snort NIDS with a web interface and rulesets that automatically update.  In this article, we will finalize the configuration of our Snort service by creating Upstart scripts for the Snort and Barnyard2 daemons. If you are running Ubuntu 15, you should go see my systemD article instead of this article.

First create the Snort Upstart script:

sudo vi /etc/init/snort.conf

We will insert the below content into this Upstart script.  Note that we are using the same flags that we used in earlier articles, so if Snort ran correctly for you earlier, then you shouldn’t need to change any of these flags:

description "Snort NIDS service"
stop on runlevel [!2345]
start on runlevel [2345]
script
    exec /usr/sbin/snort -q -u snort -g snort -c /etc/snort/snort.conf -i eth0 -D
end script

Now make the script executable, and tell Upstart that the script exists:

sudo chmod +x /etc/init/snort.conf
initctl list | grep snort
	snort stop/waiting

do the same for our Barnyard2 script:

sudo vi /etc/init/barnyard2.conf

with the following content:

description "barnyard2 service"
stop on runlevel [!2345]
start on runlevel [2345]
script
    exec /usr/local/bin/barnyard2 -c /etc/snort/barnyard2.conf -d /var/log/snort -f snort.u2 -w /var/log/snort/barnyard2.waldo -g snort -u snort -D -a /var/log/snort/archived_logs
end script

Note that we have added a new flag here that we didn’t use before: -a /var/log/snort/archived_logs, this will move logs that Barnyard2 has processed to the /var/log/snort/archived_logs/ folder.

Now make the script executable, and tell Upstart that the script exists:

sudo chmod +x /etc/init/barnyard2.conf
initctl list | grep barnyard
	barnyard2 stop/waiting

Reboot the computer and check that both services are started:

user@snortserver:~$ service snort status
snort start/running, process 1116
user@snortserver:~$ service barnyard2 status
barnyard2 start/running, process 1109
user@snortserver:~$

If both services are running, you are ready to move to the next section, where you will install Snorby, a web-based GUI to view and profile alert data. The instructions are broken up into three different versions, based on the version of Ubuntu you are running.

Choose one of the following:
Ubuntu 12: Installing Snorby on Ubuntu 12
Ubuntu 14: Installing Snorby on Ubuntu 14
Ubuntu 15: Installing Snorby on Ubuntu 15

Snort 2.9.8.x on Ubuntu – Part 6: Creating systemD Scripts for Snort

UPDATE: Snort 2.9.9.x has been released. Please see the updated series of articles here or my quick install guide here.

I am leaving this older guide online for anyone who wants to install this older version of Snort on Ubuntu, but you really should be using the updated guide for the 2.9.9.x version of Snort, since support for older versions of Snort are set to expire, and the updated guide is kept more up to date and includes BASE instead of Snorby for a Web GUI.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  1. Installing Snort
  2. Configure Snort to Run as a NIDS
  3. Writing and Testing a Single Rule With Snort
  4. Installing Barnyard2
  5. Installing PulledPork
  6. Creating Upstart Scripts for Snort
  7. Creating systemD Scripts for Snort
  8. Installing Snorby on Ubuntu 12
  9. Installing Snorby on Ubuntu 14
  10. Installing Snorby on Ubuntu 15
  11. Conclusion

Overview

In the previous articles in this series, we have created a complete Snort NIDS with a web interface and rulesets that automatically update.  In this article, we will finalize the configuration of our Snort service by creating systemD scripts for the Snort and Barnyard2 daemons. If you are running Ubuntu 12 or 14, you should go see my Upstart article instead of this article.

Creating a systemD startup script in Ubuntu 15

Ubuntu 15 has moved to systemD for services / daemons. For more information about creating and managing systemD servcies, please see this excellent article.

To create the Snort systemD service, use an editor to create a service file:

sudo vi /lib/systemd/system/snort.service

with the following content (change eth0 if different on your system):

[Unit]
Description=Snort NIDS Daemon
After=syslog.target network.target

[Service]
Type=simple
ExecStart=/usr/local/bin/snort -q -u snort -g snort -c /etc/snort/snort.conf -i eth0

[Install]
WantedBy=multi-user.target

Now we tell systemD that the service should be started at boot:

sudo systemctl enable snort

And start the Snort service:

sudo systemctl start snort

Verify the service is running

systemctl status snort

Next, create the Barnyard2 systemd service. We will add two flags here: -D to run as a daemon, and -a /var/log/snort/archived logs, this will move logs that Barnyard2 has processed to the /var/log/snort/archived/ folder. Use an editor to create a service file:

sudo vi /lib/systemd/system/barnyard2.service

With the following content:

[Unit]
Description=Barnyard2 Daemon
After=syslog.target network.target

[Service]
Type=simple
ExecStart=/usr/local/bin/barnyard2 -c /etc/snort/barnyard2.conf -d /var/log/snort -f snort.u2 -q -w /var/log/snort/barnyard2.waldo -g snort -u snort -D -a /var/log/snort/archived_logs

[Install]
WantedBy=multi-user.target

Now we tell systemD that the service should be started at boot:

sudo systemctl enable barnyard2

And start the barnyard2 service:

sudo systemctl start barnyard2

Verify the service is running

systemctl status barnyard2

Reboot the computer and check that both services are started

user@snortserver:~$ service snort status
snort start/running, process 1116
user@snortserver:~$ service barnyard2 status
barnyard2 start/running, process 1109
user@snortserver:~$

If both services are running, you are ready to move to the next section, where you will install Snorby, a web-based GUI to view and profile alert data. The instructions are broken up into three different versions, based on the version of Ubuntu you are running.

Choose one of the following:
Ubuntu 12: Installing Snorby on Ubuntu 12
Ubuntu 14: Installing Snorby on Ubuntu 14
Ubuntu 15: Installing Snorby on Ubuntu 15

Snort 2.9.8.x on Ubuntu – Quick Install Guide

UPDATE: Snort 2.9.9.x has been released. Please see the updated version my quick install guide here.

I am leaving this older guide online for anyone who wants to install this older version of Snort on Ubuntu, but you really should be using the updated guide for the 2.9.9.x version of Snort, since support for older versions of Snort are set to expire.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The instructions below show how to install Snort 2.9.8.x on both the x86 and x64 architectures for Ubuntu 12, 13, 14, and 15. If you want a more in-depth explanation of the install steps, as well as instructions on how to configure and enhance Snort’s functionality, see my in-depth series for installing Snort on Ubuntu.

If you want to test the new alpha version of Snort (Version 3.0 Alpha 2), please see my articles: Installing Snort 3 Alpha in Ubuntu 14, or Ubuntu 12.

If you want to work with OpenAppID, please see my guide for OpenAppID for Snort 2.9.8.x on Ubuntu.

Let Us Begin:

So let’s get started. First we need to install all the pre-requisites from the Ubuntu repositories:

sudo apt-get install -y build-essential libpcap-dev libpcre3-dev libdumbnet-dev bison flex zlib1g-dev liblzma-dev openssl libssl-dev ethtool

Disable LRO and GRO for all interfaces Snort will listen on under /etc/network/interfaces. using ethtool. An explanation of LRO and GRO are in the The Snort Manual). Use an editor to edit the network interfaces file:

sudo vi /etc/network/interfaces

and for every interface that Snort will listen on (one interface for simple setups, multiple interfaces for more complex setups), add the following two lines, changing eth0 to match the interface:

post-up ethtool -K eth0 gro off
post-up ethtool -K eth0 lro off

for example, my /etc/network/interfaces file looks like this:

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet dhcp
post-up ethtool -K eth0 gro off
post-up ethtool -K eth0 lro off

Reboot the system and verify that LRO and GRO are off:

user@snortserver:~$ ethtool -k eth0 | grep receive-offload
generic-receive-offload: off
large-receive-offload: off

Next we will create a directory to save the downloaded tarball files:

mkdir ~/snort_src
cd ~/snort_src

Download and install Data Acquisition library (DAQ) from the Snort website:

cd ~/snort_src
wget https://www.snort.org/downloads/snort/daq-2.0.6.tar.gz
tar -xvzf daq-2.0.6.tar.gz
cd daq-2.0.6
./configure
make
sudo make install

Now we are ready to install Snort from source:

cd ~/snort_src
wget https://snort.org/downloads/snort/snort-2.9.8.0.tar.gz
tar -xvzf snort-2.9.8.0.tar.gz
cd snort-2.9.8.0
./configure --enable-sourcefire
make
sudo make install

Run the following command to update shared libraries:

sudo ldconfig

Since the Snort installation places the Snort binary at /usr/local/bin/snort, it is common to create a symlink to /usr/sbin/snort:

sudo ln -s /usr/local/bin/snort /usr/sbin/snort

The last step of our Snort installation is to test that the Snort Binary runs. Execute Snort with the -V flag, which causes Snort to print the current version. You should see output similar to the following:

user@snortserver:~$ snort -V

   ,,_     -*> Snort! <*-
  o"  )~   Version 2.9.8.0 GRE (Build 229) 
   ''''    By Martin Roesch & The Snort Team: http://www.snort.org/contact#team
           Copyright (C) 2014-2015 Cisco and/or its affiliates. All rights reserved.
           Copyright (C) 1998-2013 Sourcefire, Inc., et al.
           Using libpcap version 1.7.4
           Using PCRE version: 8.35 2014-04-04
           Using ZLIB version: 1.2.8

user@snortserver:~$

If you have output similar to the above, then Snort is installed and works. If you want to learn more about how to run Snort, and how to install additional software to enhance a Snort system, see my in-depth series for installing Snort on Ubuntu.

If you have any questions or recommendations, please contact me. I can’t always answer questions right away, but I will do my best to get back to you. I welcome all recommendations and corrections.

Snort 2.9.8.x on Ubuntu – Part 1: Installing Snort

UPDATE: Snort 2.9.9.x has been released. Please see the updated series of articles here or my quick install guide here.

I am leaving this older guide online for anyone who wants to install this older version of Snort on Ubuntu, but you really should be using the updated guide for the 2.9.9.x version of Snort, since support for older versions of Snort are set to expire, and the updated guide is kept more up to date and includes BASE instead of Snorby for a Web GUI.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  1. Installing Snort
  2. Configure Snort to Run as a NIDS
  3. Writing and Testing a Single Rule With Snort
  4. Installing Barnyard2
  5. Installing PulledPork
  6. Creating Upstart Scripts for Snort
  7. Creating systemD Scripts for Snort
  8. Installing Snorby on Ubuntu 12
  9. Installing Snorby on Ubuntu 14
  10. Installing Snorby on Ubuntu 15
  11. Conclusion

Overview

This detailed set of articles will guide you through the steps of installing and configuring Snort as a Network Intrusion Detection System (NIDS), along with additional software that extends the functionality of your Snort system.  These articles are based on the Snort Installation guide I wrote, and which was posted in the documents section of the Snort website. If you are instead looking for a quick install guide for Snort on Ubuntu, please see my other standalone article: Snort 2.9.8.x on Ubuntu (quick install guide). If you want to test the new alpha version of Snort, please see my articles: Installing Snort++ (Version 3.0 Alpha 2) in Ubuntu 12 and Ubuntu 14.

These articles are designed to take you step-by-step through the installation, configuration, and testing of each component of a Snort system.  I will explain the design decisions and the purpose of specific commands throughout this guide, which will will help you understand how Snort is installed, configured, tested, executed, and how it interfaces with its supporting software.   You can follow the steps in this guide, but choose to skim the detailed explanations if you would like, and you will still end up with a working Snort system. However, if you take the effort to understand every step you will have a much deeper understanding of Snort, be better able to troubleshoot issues, and fully customize your Snort installation.

Supported Software Versions

This guide has been tested with Snort 2.9.8.0 on both the x86 and x64 architectures of Ubuntu 12, 14, and 15. This guide will probably work on other Ubuntu-derived distributions, and I have been told that it works fairly well (with some modifications) for Debian systems. This guide will note VMware specific configuration options, if you want to run Snort as a virtual machine.  At the time of this writing, the latest version of Snort is 2.9.8.0, and the instructions below are tailored for that version.  If you want to use more recent versions of any of the software installed below (updated versions released after the publication of this guide), it should work without significant changes, but obviously you may encounter issues I can’t foresee.

On its own, Snort runs in standalone mode as a packet sniffer and logger.  With a few additional applications and some configuration, a Snort system becomes much more useful as a NIDS.  The supporting software components we will install in this set of articles are:

  • Barnyard2 is a dedicated spooler for Snort’s unified2 binary output format. Packet processing is very resource intensive, so to reduce the load on the Snort process: we have Snort save suspicious packets to a directory in a native binary format without processing the packets. Barnyard2 then asynchronously processes those packets and saves them in a MySQL database.
  • PulledPork is a Perl script that automatically downloads the latest Snort rulesets. Since the threat landscape is constantly evolving, new rulesets are required by Snort to identify the latest types of suspicious traffic (rulesets are similar to antivirus signatures).
  • Snorby provides a web front-end to query and analyze the alerts coming from a Snort system.

Alternatives to This Guide

If you just want a Snort system installed and running without having to compile and install all the individual components, there are some alternatives:

  • Autosnort: a script that will install Snort and supporting software on your system.
  • Install Snort from the Ubuntu repository: This version of Snort tends to be out of date, and doesn’t give you the flexibility provided by compiling your own version of Snort.
  • Security Onion: A live CD based on Ubuntu with Snort already installed.

Recommendations for Running Snort in a Virtual Machine

If you are running Snort as a VMware ESXi virtual machine, it is recommended that you use the vmxnet 3 network adapter.

Onwards

So let’s get started. First we need to install all the prerequisites from the Ubuntu repositories:

sudo apt-get install -y build-essential libpcap-dev libpcre3-dev libdumbnet-dev bison flex zlib1g-dev liblzma-dev openssl libssl-dev

Breakdown of the packages you are installing:

  • build-essential: provides the build tools (GCC and the like) to compile software.
  • bison, flex: parsers required by DAQ (DAQ is installed later below).
  • libpcap-dev: Library for network traffic capture required by Snort.
  • libpcre3-dev: Library of functions to support regular expressions required by Snort.
  • libdumbnet-dev: the libdnet library provides a simplified, portable interface to several low-level networking routines. Many guides for installing Snort install this library from source, although that is not necessary.
  • zlib1g-dev: A compression library required by Snort.
  • liblzma-dev: Provides decompression of swf files (adobe flash)
  • openssl and libssl-dev: Provides SHA and MD5 file signatures

Next, we need to ensure that the network card does not truncate over-sized packets.  From The Snort Manual:

Some network cards have features named “Large Receive Offload” (lro) and “Generic Receive Offload” (gro). With these features enabled, the network card performs packet reassembly before they’re processed by the kernel. By default, Snort will truncate packets larger than the default snaplen of 1518 bytes. In addition, LRO and GRO may cause issues with Stream5 target-based reassembly. We recommend that you turn off LRO and GRO.

Install ethtool if you are on Ubuntu 12:

sudo apt-get install -y ethtool

now edit /etc/network/interfaces as an admin:

sudo vi /etc/network/interfaces

Append the following two lines for each network interface you will have Snort listen on (See note below for Ubuntu 15):

post-up ethtool -K eth0 gro off
post-up ethtool -K eth0 lro off

Important note for people running Ubuntu 15.10: In Ubuntu 15.10, for new installations (not upgrades), network interfaces no longer follow the ethX standard (eth0, eth1, …). Instead, interfaces names are assigned as Predictable Network Interface Names. This means you need to check the names of your interfaces using ifconfig -a. In my case, what was originally eth0 is now ens160. If you are running Ubuntu 15.10, anywhere in this guide you see eth0, you will need to replace with your new interface name.

an example of how the /etc/network/interfaces file should look for a single interface:

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
source /etc/network/interfaces.d/*
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
auto eth0
iface eth0 inet dhcp
post-up ethtool -K eth0 gro off
post-up ethtool -K eth0 lro off

Next we will create a directory to save the downloaded tarball files:

mkdir ~/snort_src
cd ~/snort_src

Snort uses the Data Acquisition library (DAQ) to abstract calls to packet capture libraries. DAQ is downloaded and installed from the Snort website:

cd ~/snort_src
wget https://www.snort.org/downloads/snort/daq-2.0.6.tar.gz
tar -xvzf daq-2.0.6.tar.gz
cd daq-2.0.6
./configure
make
sudo make install

Now we are ready to install Snort from source. When we configure the build of Snort, we use the –enable-sourcefire flag, which enables Packet Performance Monitoring (PPM), and matches the way the sourcefire team builds Snort.

cd ~/snort_src
wget https://www.snort.org/downloads/snort/snort-2.9.7.6.tar.gz
tar -xvzf snort-2.9.7.6.tar.gz
cd snort-2.9.7.6
./configure --enable-sourcefire
make
sudo make install

Run the following command to update shared libraries:

sudo ldconfig

Since the Snort installation places the Snort binary at /usr/local/bin/snort, it is a good policy to create a symlink to /usr/sbin/snort:

sudo ln -s /usr/local/bin/snort /usr/sbin/snort

The last step of our Snort installation is to test that the Snort Binary runs. Execute Snort with the -V flag, which causes Snort to show the version number:

/usr/sbin/snort -V

and you should see output similar to the following:

user@snortserver:~$ /usr/sbin/snort -V

   ,,_     -*&amp;gt; Snort! &amp;lt;*-
  o&amp;quot;  )~   Version 2.9.8.0 GRE (Build 229)
   ''''    By Martin Roesch &amp;amp; The Snort Team: http://www.snort.org/contact#team
           Copyright (C) 2014 Cisco and/or its affiliates. All rights reserved.
           Copyright (C) 1998-2013 Sourcefire, Inc., et al.
           Using libpcap version 1.5.3
           Using PCRE version: 8.31 2012-07-06
           Using ZLIB version: 1.2.8

user@snortserver:~$

Congratulations, if you have output similar to the above then you have successful installed Snort. Continue to the next section to Configure Snort to Run as a NIDS.

Installing OpenAppID with Snort 2.9.8.x on Ubuntu

UPDATE: Snort 2.9.9.x has been released. Please see the updated of article here.

I am leaving this older guide online for anyone who wants to install this older version of Snort on Ubuntu, but you really should be using the updated guide for the 2.9.9.x version of Snort, since support for older versions of Snort are set to expire, and the updated guide is kept more up to date.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The instructions below show how to install OpenAppId in Snort 2.9.8.x on Ubuntu 14.

If you want a more in-depth explanation of the install steps, as well as instructions on how to configure and enhance Snort’s functionality, see my in-depth series for installing Snort on Ubuntu, or my Quick Install Guide for Snort 2.9.8.x on Ubuntu.

If you want to test the new alpha version of Snort (Version 3.0 Alpha 2), please see my articles: Installing Snort 3 Alpha in Ubuntu 14, or Ubuntu 12.

Let Us Get Started

So let’s get started. First we need to install all the Snort pre-requisites from the Ubuntu repositories:

sudo apt-get install -y build-essential libpcap-dev libpcre3-dev libdumbnet-dev bison flex zlib1g-dev liblzma-dev ethtool

Next we want to install the pre-requisites that are specific to OpenAppID:

sudo apt-get install -y libluajit-5.1-dev pkg-config openssl libssl-dev

Disable LRO and GRO for all interfaces Snort will listen on under /etc/network/interfaces. using ethtool. An explanation of LRO and GRO are in the The Snort Manual). Use an editor to edit the network interfaces file:

sudo vi /etc/network/interfaces

and for every interface that Snort will listen on (one interface for simple setups, multiple interfaces for more complex setups), add the following two lines, changing eth0 to match the interface:

post-up ethtool -K eth0 gro off
post-up ethtool -K eth0 lro off

for example, my /etc/network/interfaces file looks like this:

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet dhcp
post-up ethtool -K eth0 gro off
post-up ethtool -K eth0 lro off

Reboot the system and verify that LRO and GRO are off:

user@snortserver:~$ ethtool -k eth0 | grep receive-offload
generic-receive-offload: off
large-receive-offload: off

Next we will create a directory to save the downloaded tarball files:

mkdir ~/snort_src
cd ~/snort_src

Download and install Data Acquisition library (DAQ) from the Snort website:

cd ~/snort_src
wget https://www.snort.org/downloads/snort/daq-2.0.6.tar.gz
tar -xvzf daq-2.0.6.tar.gz
cd daq-2.0.6
./configure
make
sudo make install

Installing Snort

Now we are ready to install Snort from source. We use the ‑‑enable-open-appid option, which prepares Snort to be built with OpenAppID support. We also use the ‑‑enable-sourcefire option, which enables the Sourcefire-specific build options:

Now we are ready to install Snort from source:

cd ~/snort_src
wget https://snort.org/downloads/snort/snort-2.9.8.0.tar.gz
tar -xvzf snort-2.9.8.0.tar.gz
cd snort-2.9.8.0
./configure --enable-sourcefire --enable-open-appid
make
sudo make install

Run the following command to update shared libraries:

sudo ldconfig

Since the Snort installation places the Snort binary at /usr/local/bin/snort, it is common to create a symlink to /usr/sbin/snort:

sudo ln -s /usr/local/bin/snort /usr/sbin/snort

We need to a few configuration things to prepare Snort for use. More detailed information on the steps below can be found here .

Create the needed directories and empty files:

# Create the Snort directories:
sudo mkdir /etc/snort
sudo mkdir /etc/snort/rules
sudo mkdir /etc/snort/rules/iplists
sudo mkdir /etc/snort/preproc_rules
sudo mkdir /usr/local/lib/snort_dynamicrules
sudo mkdir /etc/snort/so_rules

# Create some files that stores rules and ip lists
sudo touch /etc/snort/rules/iplists/default.blacklist
sudo touch /etc/snort/rules/iplists/default.whitelist
sudo touch /etc/snort/rules/local.rules
sudo touch /etc/snort/sid-msg.map

# Create our logging directories:
sudo mkdir /var/log/snort
sudo mkdir /var/log/snort/archived_logs

# Adjust permissions:
sudo chmod -R 5775 /etc/snort
sudo chmod -R 5775 /var/log/snort
sudo chmod -R 5775 /var/log/snort/archived_logs
sudo chmod -R 5775 /etc/snort/so_rules
sudo chmod -R 5775 /usr/local/lib/snort_dynamicrules

Finally copy some files:

cd ~/snort_src/snort-2.9.8.0/etc/
sudo cp *.conf* /etc/snort
sudo cp *.map /etc/snort
sudo cp *.dtd /etc/snort
cd ~/snort_src/snort-2.9.8.0/src/dynamic-preprocessors/build/usr/local/lib/snort_dynamicpreprocessor/
sudo cp * /usr/local/lib/snort_dynamicpreprocessor/

Comment out the rule files that are automatically loaded by Snort in snort.conf (since we don’t have any rule files downloaded at this time) by running the following command:

sudo sed -i 's/include \$RULE\_PATH/#include \$RULE\_PATH/' /etc/snort/snort.conf

Next we need to edit the /etc/snort/snort.conf Snort configuration file.

Line 45 of /etc/snort/snort.conf: the variable HOME_NET should match your internal (defended) network. In the below example our HOME NET is 10.0.0.0 with a 24-bit subnet mask (255.255.255.0):

ipvar HOME_NET 10.0.0.0/24

Still editing snort.conf, next we need to modify some file paths to match the lines below, beginning at line 104:

var RULE_PATH /etc/snort/rules
var SO_RULE_PATH /etc/snort/so_rules
var PREPROC_RULE_PATH /etc/snort/preproc_rules

var WHITE_LIST_PATH /etc/snort/rules/iplists
var BLACK_LIST_PATH /etc/snort/rules/iplists

Still editing snort.conf, next we need to modify the whitelist and blacklist path, beginning at line 511:

     whitelist $WHITE_LIST_PATH/default.whitelist, \
     blacklist $BLACK_LIST_PATH/default.blacklist

Once you have saved your edits to snort.conf, you should test that snort can load this configuration file without any errors. You do this by running snort with the -T flag to tell snort to test the file, the -c flag to identify the path of the snort.conf file, and the -i flag for a network interface that Snort will listen on. This is shown below. Output has been truncated to the final few lines to show success:

user@snortserver:~$ sudo snort -T -i eth0 -c /etc/snort/snort.conf
   (...)
   Snort successfully validated the configuration!
   Snort exiting
user@snortserver:~$

Download and Extract the Application Detector Package

Now we need to download the Application Detector Package, which contains the rules for detecting types of traffic. You can find this file on the Snort.org download page, listed as snort-openappid.tar.gz. You should download the latest version of this package, the version below is the latest as of writing, but will probably have changed, as the Snort team is updating regularly:

cd ~/snort_src
wget https://snort.org/downloads/openappid/3192 -O snort-openappid.tar.gz
tar -xvzf snort-openappid.tar.gz

The result of the above command will create a odp directory which holds all the application detector files. We want to move that folder under our Snort rules folder:

sudo cp -r ~/snort_src/odp/ /etc/snort/rules/

and create one folder for third-party developed application detectors:

sudo mkdir /usr/local/lib/thirdparty

Editing snort.conf to enable OpenAppID

We need to enable the OpenAppID pre-processor, then we need to have Snort output the AppID data. To enable the pre-processor, edit the snort.conf file (located at /etc/snort/snort.conf), and add the following line before the commented-out section 6 (line 513 for me):

preprocessor appid: app_stats_filename appstats-u2.log, \
   app_stats_period 60, \
   app_detector_dir /etc/snort/rules

This tells Snort the file name of the log to output statistics to (appstats-u2-log), how often to write to the log (every 60 seconds), and where to find the odf folder we downloaded earlier.

While still in the /etc/snort/snort.conf file, add the following lower down (below the commented-out section 6, around line 526 ):

output unified2: filename snort.log, limit 128, appid_event_types

this directive tells Snort to output alerts in the unified2 binary format to the snort.log, the size of the log, and also to output AppID data to the same location.

Now test the Snort configuration file to verify there are no errors:

sudo /usr/local/bin/snort -T -c /etc/snort/snort.conf -i eth0

as above, you should see the text: Snort successfully validated the configuration! If not, fix the errors that are reported.

Collecting OpenAppID Data

Use the below command to start collecting packets (change the interface as needed), and use ctrl-c to stop the collection:

sudo /usr/local/bin/snort -c /etc/snort/snort.conf -i eth0
ctrl-c

To generate OpenAppID data while Snort is running as above, try browsing to a website, making sure the data is visible to the interface that snort is listening on, either by passing that data directly through the Snort interface, or by ensuring that your network infrastructure copies network traffic to the Snort server (span port, port mirroring, or promiscuous mode, for example).

Once you have collected data (remember that we are writing data out every 60 seconds, so wait longer than a minute before cancelling the collection), you should see file(s) in /var/log/snort/ with the name: appstats-u2.log.nnnnnnnnnn (where the n’s are numbers). these are the OpenAppID data files. We can process them with u2openappid, which is located in /usr/local/bin.

A simple example of this processing:

noah@snort:~$ sudo u2openappid /var/log/snort/appstats-u2.log.1449426302 
statTime="1449426240",appName="HTTP",txBytes="0",rxBytes="8152"
statTime="1449426300",appName="HTTP",txBytes="0",rxBytes="9542"
statTime="1449426240",appName="DNS",txBytes="301",rxBytes="0"
statTime="1449426240",appName="__unknown",txBytes="12376",rxBytes="1118"
statTime="1449426300",appName="DNS",txBytes="761",rxBytes="0"

In the above example, I used curl over the same interface snort was listening on to request www.xkcd.com. The various application detectors show the amount of traffic for each detector, DNS, HTTP, and the like.

An more complex example of this processing (from an older version of OpenAppID, but still valid):

noah@snort:~$ sudo /usr/local/bin/u2openappid /var/log/snort/appstats-u2.log.1428300780 
statTime="1428300720",appName="curl",txBytes="740",rxBytes="6894"
statTime="1428300720",appName="http",txBytes="1306",rxBytes="7384"
statTime="1428300720",appName="ubuntu",txBytes="566",rxBytes="490"
statTime="1428300720",appName="python_urllib",txBytes="566",rxBytes="490"
statTime="1428300780",appName="https",txBytes="777",rxBytes="1444"
statTime="1428300780",appName="https",txBytes="1040",rxBytes="2116"
statTime="1428300840",appName="google",txBytes="3001",rxBytes="4684"
statTime="1428300840",appName="facebook",txBytes="66705",rxBytes="1841294"
statTime="1428300840",appName="firefox",txBytes="9080",rxBytes="29282"
statTime="1428300840",appName="google_analytic",txBytes="2441",rxBytes="17912"
statTime="1428300840",appName="http",txBytes="10591",rxBytes="49907"
statTime="1428300840",appName="https",txBytes="68049",rxBytes="1846327"
statTime="1428300840",appName="ssl_client",txBytes="66013",rxBytes="1840694"
statTime="1428300840",appName="linux_mint",txBytes="955",rxBytes="2912"
statTime="1428300840",appName="python_urllib",txBytes="1511",rxBytes="20625"
statTime="1428300720",appName="dns",txBytes="380",rxBytes="538"
statTime="1428300720",appName="ssh",txBytes="10487",rxBytes="24943"
statTime="1428300720",appName="rtp",txBytes="592",rxBytes="0"
statTime="1428300780",appName="dhcp",txBytes="1368",rxBytes="0"
statTime="1428300780",appName="dns",txBytes="482",rxBytes="936"
statTime="1428300780",appName="vnc",txBytes="219685",rxBytes="5131591"
statTime="1428300780",appName="https",txBytes="210284",rxBytes="1373974"
statTime="1428300780",appName="mdns",txBytes="8316",rxBytes="0"
statTime="1428300840",appName="dns",txBytes="1754",rxBytes="5372"
statTime="1428300840",appName="facebook",txBytes="3109",rxBytes="11074"
statTime="1428300840",appName="https",txBytes="3109",rxBytes="11074"
statTime="1428300840",appName="ssl_client",txBytes="3109",rxBytes="11074"

If you have output similar to the above, then Snort is installed and works. To generate the above output, I browsed to xkcd.com with curl on one computer, and to facebook with firefox on another computer. Looking through the output, the applications listed with the same statTime are from the same request. When I used curl to request xkcd.com, snort detected the various types of traffic defined by the various detectors.

If you want to learn more about how to run Snort, and how to install additional software to enhance a Snort system, see my in-depth series on installing Snort on Ubuntu. If you have any feedback (recommendations or corrections), please let me know here.