Building an IDS : installing snorby, suricata and barnyard2

Reference 1: https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Suricata_Snorby_and_Barnyard2_set_up_guide

Reference 2: http://www.aldeid.com/wiki/Suricata/Setting-up-rules

Reference 3: https://www.corelan.be/index.php/2011/02/27/cheat-sheet-installing-snorby-2-2-with-apache2-and-suricata-with-barnyard2-on-ubuntu-10-x/

Reference 4: https://github.com/Snorby/snorby/issues/102#issuecomment-1704653

Reference 5: http://www.aldeid.com/wiki/Snorby

Disclaimer
I claimed no credits for this post, this post is for my own personal reference while installing the components onto the Ubuntu Server 12.04 LTS. No plagiarism is intended! All setup credits go to References above. Please follow the steps from the references to setup one IDS yourself.

Softwares needed for the setup

1. Suricata, the IDS engine.

2. Apache2, the webserver.

3. MySQL, the database server.

4. Barnyard2, the parser which parses unified2 format from Suricata and write them to MySQL database.

5. Snorby, the web interface frontend for managing IDS alerts.

6. Ruby 1.9.3, at least version 1.9.2 is needed to support Snorby.

7. wkhtmltopdf, for export to pdf.

8. Ubuntu Server 12.04 LTS 32-bit, the base Linux OS.

9. Passenger.

Pre-requisite programs
1. gcc – GNU compiler frontend, basically it uses the appropriate the compiler to compile your source code. If your source code is in C++ GCC uses g++.
2. g++ – C++ compiler
3. build-essential – This is an information list to build Debian packages.
4. libssl-dev – Source code for SSL.
5. libreadline6-dev – Source code for readline library. Readline is a GNU software library for line-editing in a CLI, it allows user to move the text cursor and do tab completion.
6. zlib1g-dev – source code for zlib library. Zlib contains library for data compression.
7. linux-headers-generic – Linux header files that are required to compile Linux.
8. libsqlite3-dev – SQLite library source code.
9. libxslt-dev – source code for XLST library.
10. libxml2-dev – Source code for XML library.
11. imagemagick – for displaying and converting image formats.
12. git-core – for downloading softwares and source code. This is needed for downloading snorby.
13. libmysqlclient-dev – SQL client library source code.
14. mysql-server – MySQL server
15. libmagickwand-dev – source code for imagemagick library.
16. default-jre – Java runtime environment for Linux.
17. ruby1.9.3 – Ruby version 1.9.3.

SSH onto the installed Ubuntu server, then copy and paste the packages needed from this guide.

While installation you will be asked to provide root password for mysql.

WKhtmltoPDF

You can obtain the program from here https://code.google.com/p/wkhtmltopdf/

cyruslab@localhost:/tmp$ mkdir wkhtmlpdf
cyruslab@localhost:/tmp$ cd wkhtmlpdf

cyruslab@localhost:/tmp/wkhtmlpdf$ wget https://wkhtmltopdf.googlecode.com/files/wkhtmltoimage-0.11.0_rc1-static-i386.tar.bz2
--2012-10-12 17:01:48--  https://wkhtmltopdf.googlecode.com/files/wkhtmltoimage-0.11.0_rc1-static-i386.tar.bz2
Resolving wkhtmltopdf.googlecode.com (wkhtmltopdf.googlecode.com)... 173.194.72.82, 2404:6800:4008:c00::52
Connecting to wkhtmltopdf.googlecode.com (wkhtmltopdf.googlecode.com)|173.194.72.82|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 11393207 (11M) [application/octet-stream]
Saving to: `wkhtmltoimage-0.11.0_rc1-static-i386.tar.bz2'

100%[======================================>] 11,393,207  1.94M/s   in 7.3s

2012-10-12 17:01:55 (1.49 MB/s) - `wkhtmltoimage-0.11.0_rc1-static-i386.tar.bz2' saved [11393207/11393207]

cyruslab@localhost:/tmp/wkhtmlpdf$ tar -jxvf wkhtmltoimage-0.11.0_rc1-static-i386.tar.bz2
wkhtmltoimage-i386
cyruslab@localhost:/tmp/wkhtmlpdf$ sudo cp wkhtmltoimage-i386 /usr/bin/wkhtmltopdf

Installing and configuring snorby

Ruby Gems required:
1. thor
2. i18n
3. bundler
4. tzinfo
5. builder
6. memcache-client
7. rack
8. rack-test
9. erubis
10. mail
11. text-format
12. rack-mount
13. rails
14. sqlite3

You will see an error for installing text-format gem, because the ruby I installed is higher than the expected version.
Reference Output while installation:

cyruslab@localhost:/tmp/wkhtmlpdf$ sudo gem install thor i18n bundler tzinfo bui                                                                                lder memcache-client rack rack-test erubis mail text-format rack-mount rails sql                                                                                ite3
Fetching: thor-0.16.0.gem (100%)
Successfully installed thor-0.16.0
Fetching: i18n-0.6.1.gem (100%)
Successfully installed i18n-0.6.1
Fetching: bundler-1.2.1.gem (100%)
Successfully installed bundler-1.2.1
Fetching: tzinfo-0.3.33.gem (100%)
Successfully installed tzinfo-0.3.33
Fetching: builder-3.1.3.gem (100%)
Successfully installed builder-3.1.3
Fetching: memcache-client-1.8.5.gem (100%)
Successfully installed memcache-client-1.8.5
Fetching: rack-1.4.1.gem (100%)
Successfully installed rack-1.4.1
Fetching: rack-test-0.6.2.gem (100%)
Successfully installed rack-test-0.6.2
Fetching: erubis-2.7.0.gem (100%)
Successfully installed erubis-2.7.0
Fetching: mime-types-1.19.gem (100%)
Fetching: polyglot-0.3.3.gem (100%)
Fetching: treetop-1.4.11.gem (100%)
Fetching: mail-2.4.4.gem (100%)
Successfully installed mime-types-1.19
Successfully installed polyglot-0.3.3
Successfully installed treetop-1.4.11
Successfully installed mail-2.4.4
Fetching: text-hyphen-1.0.2.gem (100%)
ERROR:  Error installing text-format:
        text-hyphen requires Ruby version < 1.9.
Fetching: rack-mount-0.8.3.gem (100%)
Successfully installed rack-mount-0.8.3
Fetching: multi_json-1.3.6.gem (100%)
Fetching: activesupport-3.2.8.gem (100%)
Fetching: builder-3.0.3.gem (100%)
Fetching: activemodel-3.2.8.gem (100%)
Fetching: rack-cache-1.2.gem (100%)
Fetching: journey-1.0.4.gem (100%)
Fetching: hike-1.2.1.gem (100%)
Fetching: tilt-1.3.3.gem (100%)
Fetching: sprockets-2.1.3.gem (100%)
Fetching: actionpack-3.2.8.gem (100%)
Fetching: arel-3.0.2.gem (100%)
Fetching: activerecord-3.2.8.gem (100%)
Fetching: activeresource-3.2.8.gem (100%)
Fetching: actionmailer-3.2.8.gem (100%)
Fetching: rake-0.9.2.2.gem (100%)
Fetching: rack-ssl-1.3.2.gem (100%)
Fetching: json-1.7.5.gem (100%)
Building native extensions.  This could take a while...
Fetching: rdoc-3.12.gem (100%)
Depending on your version of ruby, you may need to install ruby rdoc/ri data:

<= 1.8.6 : unsupported
 = 1.8.7 : gem install rdoc-data; rdoc-data --install
 = 1.9.1 : gem install rdoc-data; rdoc-data --install
>= 1.9.2 : nothing to do! Yay!
Fetching: railties-3.2.8.gem (100%)
Fetching: rails-3.2.8.gem (100%)
Successfully installed multi_json-1.3.6
Successfully installed activesupport-3.2.8
Successfully installed builder-3.0.3
Successfully installed activemodel-3.2.8
Successfully installed rack-cache-1.2
Successfully installed journey-1.0.4
Successfully installed hike-1.2.1
Successfully installed tilt-1.3.3
Successfully installed sprockets-2.1.3
Successfully installed actionpack-3.2.8
Successfully installed arel-3.0.2
Successfully installed activerecord-3.2.8
Successfully installed activeresource-3.2.8
Successfully installed actionmailer-3.2.8
Successfully installed rake-0.9.2.2
Successfully installed rack-ssl-1.3.2
Successfully installed json-1.7.5
Successfully installed rdoc-3.12
Successfully installed railties-3.2.8
Successfully installed rails-3.2.8
Fetching: sqlite3-1.3.6.gem (100%)
Building native extensions.  This could take a while...
Successfully installed sqlite3-1.3.6
35 gems installed
Installing ri documentation for thor-0.16.0...
Installing ri documentation for i18n-0.6.1...
Installing ri documentation for bundler-1.2.1...
Installing ri documentation for tzinfo-0.3.33...
Installing ri documentation for builder-3.1.3...
Installing ri documentation for memcache-client-1.8.5...
Installing ri documentation for rack-1.4.1...
Installing ri documentation for rack-test-0.6.2...
Installing ri documentation for erubis-2.7.0...
Installing ri documentation for mime-types-1.19...
Installing ri documentation for polyglot-0.3.3...
Installing ri documentation for treetop-1.4.11...
Installing ri documentation for mail-2.4.4...
Installing ri documentation for rack-mount-0.8.3...
Installing ri documentation for multi_json-1.3.6...
Installing ri documentation for activesupport-3.2.8...
Installing ri documentation for builder-3.0.3...
Installing ri documentation for activemodel-3.2.8...
Installing ri documentation for rack-cache-1.2...
Installing ri documentation for journey-1.0.4...
Installing ri documentation for hike-1.2.1...
Installing ri documentation for tilt-1.3.3...
Installing ri documentation for sprockets-2.1.3...
Installing ri documentation for actionpack-3.2.8...
Installing ri documentation for arel-3.0.2...
Installing ri documentation for activerecord-3.2.8...
Installing ri documentation for activeresource-3.2.8...
Installing ri documentation for actionmailer-3.2.8...
Installing ri documentation for rake-0.9.2.2...
Installing ri documentation for rack-ssl-1.3.2...
Installing ri documentation for json-1.7.5...
Installing ri documentation for rdoc-3.12...
Installing ri documentation for railties-3.2.8...
Installing ri documentation for rails-3.2.8...
Installing ri documentation for sqlite3-1.3.6...
Installing RDoc documentation for thor-0.16.0...
Installing RDoc documentation for i18n-0.6.1...
Installing RDoc documentation for bundler-1.2.1...
Installing RDoc documentation for tzinfo-0.3.33...
Installing RDoc documentation for builder-3.1.3...
Installing RDoc documentation for memcache-client-1.8.5...
Installing RDoc documentation for rack-1.4.1...
Installing RDoc documentation for rack-test-0.6.2...
Installing RDoc documentation for erubis-2.7.0...
Installing RDoc documentation for mime-types-1.19...
Installing RDoc documentation for polyglot-0.3.3...
Installing RDoc documentation for treetop-1.4.11...
Installing RDoc documentation for mail-2.4.4...
Installing RDoc documentation for rack-mount-0.8.3...
Installing RDoc documentation for multi_json-1.3.6...
Installing RDoc documentation for activesupport-3.2.8...
Installing RDoc documentation for builder-3.0.3...
Installing RDoc documentation for activemodel-3.2.8...
Installing RDoc documentation for rack-cache-1.2...
Installing RDoc documentation for journey-1.0.4...
Installing RDoc documentation for hike-1.2.1...
Installing RDoc documentation for tilt-1.3.3...
Installing RDoc documentation for sprockets-2.1.3...
Installing RDoc documentation for actionpack-3.2.8...
Installing RDoc documentation for arel-3.0.2...
Installing RDoc documentation for activerecord-3.2.8...
Installing RDoc documentation for activeresource-3.2.8...
Installing RDoc documentation for actionmailer-3.2.8...
Installing RDoc documentation for rake-0.9.2.2...
Installing RDoc documentation for rack-ssl-1.3.2...
Installing RDoc documentation for json-1.7.5...
Installing RDoc documentation for rdoc-3.12...
Installing RDoc documentation for railties-3.2.8...
Installing RDoc documentation for rails-3.2.8...
Installing RDoc documentation for sqlite3-1.3.6...
cyruslab@localhost:/tmp/wkhtmlpdf$

This reference output is to record what are expected while installation, suppose you encountered a different output which you do not know if it is right or wrong you can refer to this reference output for details.

Download snorby with git
Git is a very cool program for downloading packages, it clones whatever path is defined in github. Use sudo git clone http://github.com/Snorby/snorby.git /var/www/snorby to download snorby into /var/www/snorby, you do not need to create sub directory of snorby git will do it for you. 🙂

Reference output is below:

Cloning into '/var/www/snorby'...
remote: Counting objects: 9659, done.
remote: Compressing objects: 100% (3362/3362), done.
remote: Total 9659 (delta 6478), reused 9265 (delta 6134)
Receiving objects: 100% (9659/9659), 7.51 MiB | 247 KiB/s, done.
Resolving deltas: 100% (6478/6478), done.
cyruslab@localhost:/tmp/wkhtmlpdf$ ls -lah /var/www/snorby/
total 100K
drwxr-xr-x 14 root root 4.0K Oct 12 17:12 .
drwxr-xr-x  3 root root 4.0K Oct 12 17:11 ..
drwxr-xr-x  7 root root 4.0K Oct 12 17:12 app
-rw-r--r--  1 root root 7.9K Oct 12 17:12 ChangeLog.md
drwxr-xr-x  5 root root 4.0K Oct 12 17:12 config
-rw-r--r--  1 root root  156 Oct 12 17:12 config.ru
drwxr-xr-x  2 root root 4.0K Oct 12 17:12 db
-rw-r--r--  1 root root 3.5K Oct 12 17:12 Gemfile
-rw-r--r--  1 root root 8.8K Oct 12 17:12 Gemfile.lock
drwxr-xr-x  8 root root 4.0K Oct 12 17:12 .git
-rw-r--r--  1 root root  458 Oct 12 17:12 .gitignore
drwxr-xr-x  4 root root 4.0K Oct 12 17:12 lib
-rw-r--r--  1 root root 1.7K Oct 12 17:12 LICENSE
drwxr-xr-x  2 root root 4.0K Oct 12 17:12 log
drwxr-xr-x  7 root root 4.0K Oct 12 17:12 public
-rw-r--r--  1 root root  307 Oct 12 17:12 Rakefile
-rw-r--r--  1 root root 3.9K Oct 12 17:12 README.md
drwxr-xr-x  2 root root 4.0K Oct 12 17:12 script
drwxr-xr-x  8 root root 4.0K Oct 12 17:12 spec
drwxr-xr-x  6 root root 4.0K Oct 12 17:12 test
drwxr-xr-x  2 root root 4.0K Oct 12 17:12 tmp
drwxr-xr-x  3 root root 4.0K Oct 12 17:12 vendor
cyruslab@localhost:/tmp/wkhtmlpdf$

Configuring database.yml
Make a copy of the example database.yml.

cyruslab@localhost:/$ cd /var/www/snorby/config/ && sudo cp database.yml.example database.yml

Put in your MySQL password so that snorby can connect to mysql to retrieve the database to tabulate its dashboard.

snorby: &snorby
  adapter: mysql
  username: root
  password: "your_sql_passwd" # Example: password: "s3cr3tsauce"
  host: localhost

Make a copy of snorby_config.yml

cyruslab@localhost:/var/www/snorby/config$ sudo cp snorby_config.yml.example snorby_config.yml

Modify the wkhtmltopdf file path:

#
# Production
#
# Change the production configuration for your environment.
#
# USE THIS!
#
production:
  domain: 'localhost:3000'
  wkhtmltopdf: /usr/bin/wkhtmltopdf
  ssl: false
  mailer_sender: 'snorby@snorby.org'
  geoip_uri: "http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz"
  rules:
    - ""
  authentication_mode: database
....
....
....
#
# Only Use For Development
#
development:
  domain: localhost:3000
  wkhtmltopdf: /usr/bin/wkhtmltopdf
  ssl: false
  mailer_sender: 'snorby@snorby.org'
  geoip_uri: "http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz"
  rules:
    - "/Users/mephux/.snort/rules"
    - "/Users/mephux/.snort/so_rules"
  authentication_mode: database
....
....
....
#
# Only Use For Testing
#
test:
  domain: localhost:3000
  wkhtmltopdf: /usr/bin/wkhtmltopdf
  mailer_sender: 'snorby@snorby.org'
  geoip_uri: "http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz"
  authentication_mode: database

Bundle update

Reference output:

cyruslab@localhost:/var/www/snorby/config$ cd ../

cyruslab@localhost:/var/www/snorby$ sudo bundle update activesupport railties rails
Fetching http://github.com/Snorby/snorby_cas_authenticatable.git
remote: Counting objects: 509, done.
remote: Compressing objects: 100% (236/236), done.
remote: Total 509 (delta 276), reused 440 (delta 233)
Receiving objects: 100% (509/509), 58.50 KiB | 108 KiB/s, done.
Resolving deltas: 100% (276/276), done.
Fetching http://github.com/mephux/ezprint.git
remote: Counting objects: 114, done.
remote: Compressing objects: 100% (56/56), done.
remote: Total 114 (delta 43), reused 114 (delta 43)
Receiving objects: 100% (114/114), 14.20 KiB, done.
Resolving deltas: 100% (43/43), done.
Fetching http://github.com/Snorby/delayed_job_data_mapper.git
remote: Counting objects: 55, done.
remote: Compressing objects: 100% (36/36), done.
remote: Total 55 (delta 16), reused 50 (delta 11)
Unpacking objects: 100% (55/55), done.
Fetching gem metadata from http://rubygems.org/......
Fetching gem metadata from http://rubygems.org/..
Installing rake (0.9.2)
Installing Platform (0.4.0) WARNING: Platform-0.4.0 has an invalid nil value for @cert_chain

Installing open4 (1.3.0)
Installing POpen4 (0.1.4)
Installing RedCloth (4.2.9) with native extensions
Installing multi_json (1.0.4)
Installing activesupport (3.1.0)
Installing bcrypt-ruby (3.0.1) with native extensions
Using builder (3.0.3)
Using i18n (0.6.1)
Installing activemodel (3.1.0)
Using erubis (2.7.0)
Installing rack (1.3.6)
Installing rack-cache (1.0.3)
Using rack-mount (0.8.3)
Using rack-test (0.6.2)
Using hike (1.2.1)
Using tilt (1.3.3)
Installing sprockets (2.0.4)
Installing actionpack (3.1.0)
Using mime-types (1.19)
Using polyglot (0.3.3)
Using treetop (1.4.11)
Installing mail (2.3.3)
Installing actionmailer (3.1.0)
Installing arel (2.2.3)
Using tzinfo (0.3.33)
Installing activerecord (3.1.0)
Installing activeresource (3.1.0)
Installing addressable (2.2.8)
Installing ansi (1.4.2)
Using bundler (1.2.1)
Installing highline (1.6.13)
Installing net-ssh (2.5.2)
Installing net-scp (1.0.4)
Installing net-sftp (2.0.5)
Installing net-ssh-gateway (1.1.0)
Installing capistrano (2.12.0)
Installing nokogiri (1.5.4) with native extensions
Installing ffi (1.0.11) with native extensions
Installing childprocess (0.3.2)
Installing libwebsocket (0.1.3)
Installing rubyzip (0.9.8)
Installing selenium-webdriver (2.22.2)
Installing xpath (0.1.4)
Installing capybara (1.1.2)
Installing chronic (0.3.0)
Installing closure-compiler (1.1.6)
Installing daemons (1.1.8)
Installing data_objects (0.10.8)
Installing delayed_job (2.1.4)
Installing dm-core (1.2.0)
Installing dm-aggregates (1.2.0)
Installing dm-observer (1.2.0)
Using delayed_job_data_mapper (1.0.0.rc) from http://github.com/Snorby/delayed_job_data_mapper.git (at master)
Installing orm_adapter (0.0.7)
Installing warden (1.2.1)
Installing devise (1.5.3)
Installing rubycas-client (2.3.8)
Using devise_cas_authenticatable (1.0.0.alpha10) from http://github.com/Snorby/snorby_cas_authenticatable.git (at master)
Installing diff-lcs (1.1.3)
Installing dm-active_model (1.2.1)
Installing dm-ar-finders (1.2.0)
Installing dm-chunked_query (0.3.1)
Installing dm-constraints (1.2.0)
Installing dm-migrations (1.2.0)
Using rack-ssl (1.3.2)
Installing json (1.6.7) with native extensions
Using rdoc (3.12)
Installing thor (0.14.6)
Installing railties (3.1.0)
Installing dm-rails (1.2.1)
Installing fastercsv (1.5.5)
Installing json_pure (1.6.7)
Installing dm-serializer (1.2.1)
Installing dm-timestamps (1.2.0)
Installing dm-devise (1.5.0)
Installing dm-do-adapter (1.2.0)
Installing do_mysql (0.10.8) with native extensions
Installing dm-mysql-adapter (1.2.0)
Installing dm-pager (1.1.0)
Installing dm-transactions (1.2.0)
Installing stringex (1.3.3)
Installing uuidtools (2.1.2)
Installing dm-types (1.2.1)
Installing dm-validations (1.2.0)
Installing ruby-graphviz (1.0.5)
Installing dm-visualizer (0.1.0)
Installing env (0.3.0)
Installing eventmachine (0.12.10) with native extensions
Installing pdfkit (0.4.6)
Using ezprint (0.2.0) from http://github.com/mephux/ezprint.git (at rails3)
Installing geoip (1.1.2)
Installing home_run (1.0.6) with native extensions
Installing yui-compressor (0.9.6)
Installing jammit (0.5.4)
Installing jquery-rails (1.0.19)
Installing launchy (2.1.0)
Installing letter_opener (0.0.2)
Installing minitest (3.1.0)
Installing net-dns (0.6.1)
Installing netaddr (1.5.0)
Installing rails (3.1.0)
Installing rspec-core (2.0.1)
Installing rspec-expectations (2.0.1)
Installing rspec-mocks (2.0.1)
Installing rspec (2.0.1)
Installing rspec-rails (2.0.1)
Installing simple_form (1.2.2)
Installing thin (1.3.1) with native extensions
Installing turn (0.9.5)
Installing whois (2.3.0)
Your bundle is updated! Use `bundle show [gemname]` to see where a bundled gem is installed.
cyruslab@localhost:/var/www/snorby$

Gem install arel and ezprint

Reference output:

cyruslab@localhost:/var/www/snorby$ sudo gem install arel ezprint && sudo bundle install
Successfully installed arel-3.0.2
Fetching: pdfkit-0.5.2.gem (100%)
Fetching: ezprint-0.3.1.gem (100%)
Successfully installed pdfkit-0.5.2
Successfully installed ezprint-0.3.1
3 gems installed
Installing ri documentation for arel-3.0.2...
Installing ri documentation for pdfkit-0.5.2...
Installing ri documentation for ezprint-0.3.1...
Installing RDoc documentation for arel-3.0.2...
Installing RDoc documentation for pdfkit-0.5.2...
Installing RDoc documentation for ezprint-0.3.1...
Using rake (0.9.2)
Using Platform (0.4.0)
Using open4 (1.3.0)
Using POpen4 (0.1.4)
Using RedCloth (4.2.9)
Using multi_json (1.0.4)
Using activesupport (3.1.0)
Using bcrypt-ruby (3.0.1)
Using builder (3.0.3)
Using i18n (0.6.1)
Using activemodel (3.1.0)
Using erubis (2.7.0)
Using rack (1.3.6)
Using rack-cache (1.0.3)
Using rack-mount (0.8.3)
Using rack-test (0.6.2)
Using hike (1.2.1)
Using tilt (1.3.3)
Using sprockets (2.0.4)
Using actionpack (3.1.0)
Using mime-types (1.19)
Using polyglot (0.3.3)
Using treetop (1.4.11)
Using mail (2.3.3)
Using actionmailer (3.1.0)
Using arel (2.2.3)
Using tzinfo (0.3.33)
Using activerecord (3.1.0)
Using activeresource (3.1.0)
Using addressable (2.2.8)
Using ansi (1.4.2)
Using highline (1.6.13)
Using net-ssh (2.5.2)
Using net-scp (1.0.4)
Using net-sftp (2.0.5)
Using net-ssh-gateway (1.1.0)
Using capistrano (2.12.0)
Using nokogiri (1.5.4)
Using ffi (1.0.11)
Using childprocess (0.3.2)
Using libwebsocket (0.1.3)
Using rubyzip (0.9.8)
Using selenium-webdriver (2.22.2)
Using xpath (0.1.4)
Using capybara (1.1.2)
Using chronic (0.3.0)
Using closure-compiler (1.1.6)
Using daemons (1.1.8)
Using data_objects (0.10.8)
Using delayed_job (2.1.4)
Using dm-core (1.2.0)
Using dm-aggregates (1.2.0)
Using dm-observer (1.2.0)
Using delayed_job_data_mapper (1.0.0.rc) from http://github.com/Snorby/delayed_job_data_mapper.git (at master)
Using orm_adapter (0.0.7)
Using warden (1.2.1)
Using devise (1.5.3)
Using rubycas-client (2.3.8)
Using devise_cas_authenticatable (1.0.0.alpha10) from http://github.com/Snorby/snorby_cas_authenticatable.git (at master)
Using diff-lcs (1.1.3)
Using dm-active_model (1.2.1)
Using dm-ar-finders (1.2.0)
Using dm-chunked_query (0.3.1)
Using dm-constraints (1.2.0)
Using dm-migrations (1.2.0)
Using rack-ssl (1.3.2)
Using json (1.6.7)
Using rdoc (3.12)
Using thor (0.14.6)
Using railties (3.1.0)
Using dm-rails (1.2.1)
Using fastercsv (1.5.5)
Using json_pure (1.6.7)
Using dm-serializer (1.2.1)
Using dm-timestamps (1.2.0)
Using dm-devise (1.5.0)
Using dm-do-adapter (1.2.0)
Using do_mysql (0.10.8)
Using dm-mysql-adapter (1.2.0)
Using dm-pager (1.1.0)
Using dm-transactions (1.2.0)
Using stringex (1.3.3)
Using uuidtools (2.1.2)
Using dm-types (1.2.1)
Using dm-validations (1.2.0)
Using ruby-graphviz (1.0.5)
Using dm-visualizer (0.1.0)
Using env (0.3.0)
Using eventmachine (0.12.10)
Using pdfkit (0.4.6)
Using ezprint (0.2.0) from http://github.com/mephux/ezprint.git (at rails3)
Using geoip (1.1.2)
Using home_run (1.0.6)
Using yui-compressor (0.9.6)
Using jammit (0.5.4)
Using jquery-rails (1.0.19)
Using launchy (2.1.0)
Using letter_opener (0.0.2)
Using minitest (3.1.0)
Using net-dns (0.6.1)
Using netaddr (1.5.0)
Using bundler (1.2.1)
Using rails (3.1.0)
Using rspec-core (2.0.1)
Using rspec-expectations (2.0.1)
Using rspec-mocks (2.0.1)
Using rspec (2.0.1)
Using rspec-rails (2.0.1)
Using simple_form (1.2.2)
Using thin (1.3.1)
Using turn (0.9.5)
Using whois (2.3.0)
Your bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed.
cyruslab@localhost:/var/www/snorby$

Setup snorby

Snorby will create database named snorby into MySQL server.

Reference output:

cyruslab@localhost:/var/www/snorby$ sudo bundle exec rake snorby:setup
e0ed1ff88c997fc0e5f6ba13275c27f5ec0c14552179ce1745656000ae123cc6827a55d16162483e2cf6e087dfc81879b256daf0ade7538544a51a87d20eb3c3
[datamapper] Created database 'snorby'
[datamapper] Finished auto_upgrade! for :default repository 'snorby'
[~] Adding `index_timestamp_cid_sid` index to the event table
[~] Adding `id` to the event table
[~] Building `aggregated_events` database view
[~] Building `events_with_join` database view
* Removing old jobs
* Starting the Snorby worker process.
* Adding jobs to the queue
cyruslab@localhost:/var/www/snorby$

Create user for snorby

Login to MySQL server with mysql -u root -p

cyruslab@localhost:/var/www/snorby$ mysql -u root -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 45
Server version: 5.5.24-0ubuntu0.12.04.1 (Ubuntu)

Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>

Create the user for snorby with all privileges to database snorby.

mysql> create user 'admin'@'localhost' identified by 'admin_password';
Query OK, 0 rows affected (0.00 sec)

mysql> grant all privileges on snorby.* to 'admin'@'localhost' with grant option;
Query OK, 0 rows affected (0.02 sec)

mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

mysql>

I will need to modify the database.yml again with the newly created snorby username and password. The database.yml is in /var/www/snorby/config/

# Snorby Database Configuration
#
# Please set your database password/user below
# NOTE: Indentation is important.
#
snorby: &snorby
  adapter: mysql
  username: admin
  password: "admin_password" # Example: password: "s3cr3tsauce"
  host: localhost

development:
  database: snorby
  <<: *snorby

test:
  database: snorby
  <<: *snorby

production:
  database: snorby
  <<: *snorby

Modify MySQL config file my.cnf
By default MySQL only listens to localhost (127.0.0.1), however I want MySQL to listen to from all source addresses.

cyruslab@localhost:/var/www/snorby$ sudo nano /etc/mysql/my.cnf
....
....
....
#
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
bind-address            = 127.0.0.1

Comment the bind-address line. Then restart mysqld service.

#
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
#bind-address           = 127.0.0.1

Restart mysqld:

cyruslab@localhost:/var/www/snorby$ sudo service mysql restart
mysql stop/waiting
mysql start/running, process 5758
cyruslab@localhost:/var/www/snorby$

The *:mysql (Listen) means it listens from all source addresses.

cyruslab@localhost:/var/www/snorby$ sudo lsof -i | grep mysqld
mysqld    5758    mysql   10u  IPv4  41545      0t0  TCP *:mysql (LISTEN)

Installing more packages

1. apache2 – http web server
2. apache2-prefork-dev – pre-fork http web server
3. libapr1-dev – source code for Apache portable runtime library
4. libaprutil1-dev – source code for apache portable runtime utilities library
5. libopenssl-ruby – SSL library for ruby
6. libcurl4-openssl-dev – source code for client-side url transfer library that supports SSL

Install passenger gem

Reference output:

cyruslab@localhost:/var/www/snorby$ sudo gem install --no-ri --no-rdoc passenger
Fetching: fastthread-1.0.7.gem (100%)
Building native extensions.  This could take a while...
Fetching: daemon_controller-1.0.0.gem (100%)
Fetching: passenger-3.0.17.gem (100%)
Successfully installed fastthread-1.0.7
Successfully installed daemon_controller-1.0.0
Successfully installed passenger-3.0.17
3 gems installed
cyruslab@localhost:/var/www/snorby$

Installing apache2 passenger module

Output is too long which I cannot show them here.
sudo /usr/local/bin/passenger-install-apache2-module -a

At the end you will see a guide on how to deploy:

--------------------------------------------
The Apache 2 module was successfully installed.

Please edit your Apache configuration file, and add these lines:

   LoadModule passenger_module /var/lib/gems/1.9.1/gems/passenger-3.0.17/ext/apache2/mod_passenger.so
   PassengerRoot /var/lib/gems/1.9.1/gems/passenger-3.0.17
   PassengerRuby /usr/bin/ruby1.9.1

After you restart Apache, you are ready to deploy any number of Ruby on Rails
applications on Apache, without any further Ruby on Rails-specific
configuration!

Press ENTER to continue.

--------------------------------------------
Deploying a Ruby on Rails application: an example

Suppose you have a Rails application in /somewhere. Add a virtual host to your
Apache configuration file and set its DocumentRoot to /somewhere/public:

   <VirtualHost *:80>
      ServerName www.yourhost.com
      # !!! Be sure to point DocumentRoot to 'public'!
      DocumentRoot /somewhere/public
      <Directory /somewhere/public>
         # This relaxes Apache security settings.
         AllowOverride all
         # MultiViews must be turned off.
         Options -MultiViews
      </Directory>
   </VirtualHost>

And that's it! You may also want to check the Users Guide for security and
optimization tips, troubleshooting and other useful information:

  /var/lib/gems/1.9.1/gems/passenger-3.0.17/doc/Users guide Apache.html

Enjoy Phusion Passenger, a product of Phusion (www.phusion.nl) 🙂
https://www.phusionpassenger.com

Phusion Passenger is a trademark of Hongli Lai & Ninh Bui.
cyruslab@localhost:/var/www/snorby$

Create passenger.load
There is no passenger.load module in /etc/apache2/mods-available, create one using sudo touch /etc/apache2/mods-available/passenger.load.

Look for the passenger module path:

cyruslab@localhost:/var/www/snorby$ sudo find / -name "*mod_passenger*"
/var/lib/gems/1.9.1/gems/passenger-3.0.17/ext/apache2/mod_passenger.so
/var/lib/gems/1.9.1/gems/passenger-3.0.17/ext/apache2/mod_passenger.c
/var/lib/gems/1.9.1/gems/passenger-3.0.17/ext/apache2/mod_passenger.o
cyruslab@localhost:/var/www/snorby$

Put the file location and path onto passenger.load

cyruslab@localhost:/var/www/snorby$ sudo nano /etc/apache2/mods-available/passenger.load

LoadModule passenger_module /var/lib/gems/1.9.1/gems/passenger3.0.17/ext/apache2/mod_passenger.so

<IfModule mod_passenger.c>
   PassengerRoot /var/lib/gems/1.9.1/gems/passenger-3.0.17
   PassengerRuby /usr/bin/ruby
</IfModule>

Activate the modules
Reference output:

cyruslab@localhost:/var/www/snorby$ sudo a2enmod passenger
Enabling module passenger.
To activate the new configuration, you need to run:
  service apache2 restart

cyruslab@localhost:/var/www/snorby$ sudo a2enmod rewrite
Enabling module rewrite.
To activate the new configuration, you need to run:
  service apache2 restart

cyruslab@localhost:/var/www/snorby$ sudo a2enmod ssl
Enabling module ssl.
See /usr/share/doc/apache2.2-common/README.Debian.gz on how to configure SSL and create self-signed certificates.
To activate the new configuration, you need to run:
  service apache2 restart

cyruslab@localhost:/var/www/snorby$ sudo chown www-data:www-data /var/www/snorby -R

Change the owner and group for /var/www/snorby and everything under it.

Modify default

cyruslab@localhost:/var/www/snorby$ cd /etc/apache2/sites-available/
cyruslab@localhost:/etc/apache2/sites-available$ sudo nano default

From the default, modify the below portion only:

<VirtualHost *:80>
        ServerAdmin webmaster@localhost

        DocumentRoot /var/www
        <Directory />
                Options FollowSymLinks
                AllowOverride None
        </Directory>
        <Directory /var/www/>
                Options Indexes FollowSymLinks MultiViews
                AllowOverride None
                Order allow,deny
                allow from all
        </Directory>

I changed default to the below:

<VirtualHost *:80>
        ServerAdmin webmaster@localhost

        DocumentRoot /var/www/snorby/public
        <Directory />
                Options FollowSymLinks
                AllowOverride None
        </Directory>
        <Directory /var/www/snorby/public>
                #Options Indexes FollowSymLinks MultiViews
                Options -MultiViews
                AllowOverride all
                #Order allow,deny
                #allow from all
        </Directory>

Go to /var/www/snorby with cd /var/www/snorby then use this sudo bundle pack && sudo bundle install --path vender/cache

Restart the apache2 web server with sudo service apache2 restart

Test the snorby web interface
Successful snorby installation and apache configuration will land you to this page as shown below.

Installing Barnyard2 and Suricata

Barnyard 2 is a parser program that parses the unified2 format and sends the alerts to MySQL server.

Suricata is the IDS/IPS engine, the alerts are sent out in unified2 format.

Pre-requisites

The below are softwares or libraries or source codes that need to be installed.

cyruslab@localhost:~$ sudo apt-get install -y libpcre3 libpcre3-dbg libpcre3-dev autoconf automake libtool libpcap-dev libnet1-dev libyaml-0-2 libyaml-dev libcap-ng-dev libcap-ng0 pkg-config libnss3-dev libnspr4-dev libmagic-dev

Download and install the Suricata

Download and install Suricata
The below commands are found in OISF guide.

sudo git clone git://phalanx.openinfosecfoundation.org/oisf.git && cd oisf/ && sudo ./autogen.sh && sudo ./configure  --with-libnss-libraries=/usr/lib --with-libnss-includes=/usr/include/nss/ --with-libnspr-libraries=/usr/lib --with-libnspr-includes=/usr/include/nspr && sudo make clean && sudo make  && sudo  make install-full && sudo ldconfig

The Suricata can be downloaded from here, OSIF also contains instructions on how to install.

Install Barnyard2

sudo apt-get install -y mysql-client 

The library file libmysqlclient16-dev has been renamed to libmysqlclient-dev in Ubuntu server 12.04.

Find the location of the libmysqlclient library by using

find / -name libmysql*

/usr/lib/i386-linux-gnu/libmysqlclient.so.18
/usr/lib/i386-linux-gnu/libmysqlclient.a
/usr/lib/i386-linux-gnu/libmysqlclient_r.a
/usr/lib/i386-linux-gnu/libmysqlclient.so
/usr/lib/i386-linux-gnu/libmysqlclient_r.so
/usr/lib/i386-linux-gnu/libmysqlclient_r.so.18.0.0
/usr/lib/i386-linux-gnu/libmysqlclient.so.18.0.0
/usr/lib/i386-linux-gnu/libmysqlclient_r.so.18

specify the mysql client library directory:

cyruslab@localhost:/usr/lib$ cd /tmp && sudo wget http://www.securixlive.com/download/barnyard2/barnyard2-1.9.tar.gz && sudo tar xvfz barnyard2-1.9.tar.gz && cd barnyard2-1.9
sudo ./configure --with-mysql-libraries=/usr/lib/i386-linux-gnu/ && make && sudo make install

Configure Suricata and download the rules

Create the sub directory sudo mkdir /etc/suricata.

Copy the barnyard2.conf sudo cp /tmp/barnyard2-1.9/etc/barnyard2.conf /etc/suricata/
I will configure the files later.

Download the suricata rules:

cd /etc/suricata && sudo wget http://rules.emergingthreats.net/open/suricata/emerging.rules.tar.gz &&  sudo tar zxvf emerging.rules.tar.gz

Copy the reference.config and classification.config to /etc/suricata from /etc/suricata/rules/ directory.

cyruslab@localhost:/etc/suricata/rules$ sudo cp reference.config ../
cyruslab@localhost:/etc/suricata/rules$ sudo cp classification.config ../

Configure barnyard
sudo nano /etc/suricata/barnyard2.conf

config reference_file:      /etc/snort/reference.config
config classification_file: /etc/snort/classification.config
config gen_file:            /etc/snort/gen-msg.map
config sid_file:                /etc/snort/sid-msg.map

config reference_file:      /etc/suricata/reference.config
config classification_file: /etc/suricata/classification.config
config gen_file:            /etc/suricata/rules/gen-msg.map
config sid_file:                /etc/suricata/rules/sid-msg.map

# define the full waldo filepath.
#
#config waldo_file: /tmp/waldo

# define the full waldo filepath.
#
config waldo_file: /var/log/suricata/suricata.waldo

   output database: log, mysql, user=admin password=your_db_passwd dbname=snorby host=localhost
#   output database: alert, postgresql, user=snort dbname=snort
#   output database: log, odbc, user=snort dbname=snort
#   output database: log, mssql, dbname=snort user=snort password=test
#   output database: log, oracle, dbname=snort user=snort password=test
#

# An example of usage would be to configure them to the values of the associated
# snort process whose unified files you are reading.
#
# Example:
#   For a snort process as follows:
#     snort -i eth0 -c /etc/snort.conf
#
#   Typical options would be:
#     config hostname:  thor
#     config interface: eth0
#     config alert_with_interface_name
#
#config hostname:       thor
#config interface:      eth0

#
#config hostname:       thor
config interface:       eth0

cyruslab@localhost:~$ sudo mkdir /var/log/barnyard2
cyruslab@localhost:~$ sudo mkdir /var/log/suricata/ && sudo touch /var/log         ata/suricata.waldo
cyruslab@localhost:~$

cyruslab@localhost:/tmp$ sudo find / -name suricata.yaml
/home/cyruslab/oisf/oisf/suricata.yaml
/usr/local/etc/suricata/suricata.yaml
cyruslab@localhost:/tmp$ sudo cp /usr/local/etc/suricata/suricata.yaml /etc/suricata/
cyruslab@localhost:/tmp$sudo chmod 644 /etc/suricata/suricata.yaml

cyruslab@localhost:/tmp$ ls -lah /var/log/suricata/
total 856K
drwxr-xr-x  2 root root 4.0K Oct 18 12:57 .
drwxr-xr-x 14 root root 4.0K Oct 18 12:45 ..
-rw-r-----  1 root root 242K Oct 18 13:07 fast.log
-rw-r-----  1 root root    0 Oct 18 12:57 http.log
-rw-r--r--  1 root root 246K Oct 18 13:07 stats.log
-rw-r--r--  1 root root 2.1K Oct 18 13:07 suricata.waldo
-rw-r-----  1 root root 340K Oct 18 13:07 unified2.alert.1350536267
cyruslab@localhost:/tmp$