mirror of https://github.com/OISF/suricata
doc: added output section (some fixes still to do)
Andreas Herz
10 years ago
committed by
Victor Julien
parent
2fa4547197
commit
398151ac76
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@ -0,0 +1,54 @@
|
||||
Custom http logging
|
||||
===================
|
||||
|
||||
As of Suricata 1.3.1 you can enable a custom http logging option.
|
||||
|
||||
In your Suricata.yaml, find the http-log section and edit as follows:
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
- http-log:
|
||||
enabled: yes
|
||||
filename: http.log
|
||||
custom: yes # enable the custom logging format (defined by custom format)
|
||||
customformat: "%{%D-%H:%M:%S}t.%z %{X-Forwarded-For}i %{User-agent}i %H %m %h %u %s %B %a:%p -> %A:%P"
|
||||
append: no
|
||||
#extended: yes # enable this for extended logging information
|
||||
#filetype: regular # 'regular', 'unix_stream' or 'unix_dgram'
|
||||
|
||||
And in your http.log file you would get the following, for example:
|
||||
|
||||
::
|
||||
|
||||
8/28/12-22:14:21.101619 - Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:11.0) Gecko/20100101 Firefox/11.0 HTTP/1.1 GET us.cnn.com /video/data/3.0/video/world/2012/08/28/hancocks-korea-typhoon-bolavan.cnn/index.xml 200 16856 192.168.1.91:45111 -> 157.166.255.18:80
|
||||
|
||||
::
|
||||
|
||||
08/28/12-22:14:30.693856 - Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:11.0) Gecko/20100101 Firefox/11.0 HTTP/1.1 GET us.cnn.com /video/data/3.0/video/showbiz/2012/08/28/conan-reports-from-rnc-convention.teamcoco/index.xml 200 15789 192.168.1.91:45108 -> 157.166.255.18:80
|
||||
|
||||
It should be noted that each line represents the http request and response all in one, depending on how you specified your custom format. HTTP responses do not get their own line and if you wish to see that information, you will need to modify the custom format to include it using %{header_field}o for example.
|
||||
|
||||
The list of supported format strings is the following:
|
||||
|
||||
* %h - Host HTTP Header (remote host name). ie: google.com
|
||||
* %H - Request Protocol. ie: HTTP/1.1
|
||||
* %m - Request Method. ie: GET
|
||||
* %u - URL including query string. ie: /search?q=suricata
|
||||
* %{header_name}i - contents of the defined HTTP Request Header name. ie:
|
||||
|
||||
* %{User-agent}i: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:11.0) Gecko/20100101 Firefox/11.0
|
||||
* %{X-Forwarded-For}i: outputs the IP address contained in the X-Forwarded-For HTTP header (inserted by a reverse proxy)
|
||||
|
||||
* %s - return status code. In the case of 301 and 302 it will print the url in brackets. ie: 200
|
||||
* %B - response size in bytes. ie: 15789
|
||||
* %{header_name}o - contents of the defined HTTP Response Header name
|
||||
* %{strftime_format]t - timestamp of the HTTP transaction in the selected strftime format. ie: 08/28/12-22:14:30
|
||||
* %z - precision time in useconds. ie: 693856
|
||||
* %a - client IP address
|
||||
* %p - client port number
|
||||
* %A - server IP address
|
||||
* %P - server port number
|
||||
|
||||
Any non printable character will be represented by its byte value in hexadecimal format (\|XX\|, where XX is the hex code)
|
||||
@ -0,0 +1,80 @@
|
||||
Eve JSON 'jq' Examples
|
||||
======================
|
||||
|
||||
The jq tool is very useful for quickly parsing and filtering JSON files. This page is contains various examples of how it can be used with Suricata's Eve.json.
|
||||
|
||||
The basics are discussed here:
|
||||
|
||||
* https://www.stamus-networks.com/2015/05/18/looking-at-suricata-json-events-on-command-line/
|
||||
|
||||
Colorize output
|
||||
---------------
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
tail -f eve.json | jq -c '.'
|
||||
|
||||
|
||||
DNS NXDOMAIN
|
||||
------------
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
tail -f eve.json|jq -c 'select(.dns.rcode=="NXDOMAIN")'
|
||||
|
||||
Unique HTTP User Agents
|
||||
-----------------------
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
cat eve.json | jq -s '[.[]|.http.http_user_agent]|group_by(.)|map({key:.[0],value:(.|length)})|from_entries'
|
||||
|
||||
Source: https://twitter.com/mattarnao/status/601807374647750657
|
||||
|
||||
|
||||
Data use for a host
|
||||
-------------------
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
tail -n500000 eve.json | jq -s 'map(select(.event_type=="netflow" and .dest_ip=="192.168.1.3").netflow.bytes)|add'|numfmt --to=iec
|
||||
1.3G
|
||||
|
||||
Note: can use a lot of memory.
|
||||
Source: https://twitter.com/pkt_inspector/status/605524218722148352
|
||||
|
||||
|
||||
Monitor part of the stats
|
||||
-------------------------
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
$ tail -f eve.json | jq -c 'select(.event_type=="stats")|.stats.decoder'
|
||||
|
||||
Inspect Alert Data
|
||||
------------------
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
cat eve.json | jq -r -c 'select(.event_type=="alert")|.payload'|base64 --decode
|
||||
|
||||
Top 10 Destination Ports
|
||||
------------------------
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
cat eve.json | jq -c 'select(.event_type=="flow")|[.proto, .dest_port]'|sort |uniq -c|sort -nr|head -n10
|
||||
@ -0,0 +1,196 @@
|
||||
Eve JSON Format
|
||||
===============
|
||||
|
||||
Example:
|
||||
|
||||
::
|
||||
|
||||
|
||||
{
|
||||
"timestamp": "2009-11-24T21:27:09.534255",
|
||||
"event_type": "alert",
|
||||
"src_ip": "192.168.2.7",
|
||||
"src_port": 1041,
|
||||
"dest_ip": "x.x.250.50",
|
||||
"dest_port": 80,
|
||||
"proto": "TCP",
|
||||
"alert": {
|
||||
"action": "allowed",
|
||||
"gid": 1,
|
||||
"signature_id" :2001999,
|
||||
"rev": 9,
|
||||
"signature": "ET MALWARE BTGrab.com Spyware Downloading Ads",
|
||||
"category": "A Network Trojan was detected",
|
||||
"severity": 1
|
||||
}
|
||||
}
|
||||
|
||||
Common Section
|
||||
--------------
|
||||
|
||||
All the JSON log types share a common structure:
|
||||
|
||||
::
|
||||
|
||||
|
||||
{"timestamp":"2009-11-24T21:27:09.534255","event_type":"TYPE", ...tuple... ,"TYPE":{ ... type specific content ... }}
|
||||
|
||||
Event types
|
||||
~~~~~~~~~~~
|
||||
|
||||
The common part has a field "event_type" to indicate the log type.
|
||||
|
||||
::
|
||||
|
||||
|
||||
"event_type":"TYPE"
|
||||
|
||||
Event type: Alert
|
||||
-----------------
|
||||
|
||||
Field action
|
||||
~~~~~~~~~~~~
|
||||
|
||||
Possible values: "allowed" and "blocked"
|
||||
|
||||
Example:
|
||||
|
||||
::
|
||||
|
||||
|
||||
"action":"allowed"
|
||||
|
||||
Action is set to "allowed" unless a rule used the "drop" action and Suricata is in IPS mode, or when the rule used the "reject" action.
|
||||
|
||||
Event type: HTTP
|
||||
----------------
|
||||
|
||||
Fields
|
||||
~~~~~~
|
||||
|
||||
* "hostname": The hostname this HTTP event is attributed to
|
||||
* "url": URL at the hostname that was accessed
|
||||
* "http_user_agent": The user-agent of the software that was used
|
||||
* "http_content_type": The type of data returned (ex: application/x-gzip)
|
||||
* "cookie"
|
||||
|
||||
In addition to these fields, if the extended logging is enabled in the suricata.yaml file the following fields are (can) also included:
|
||||
|
||||
* "length": The content size of the HTTP body
|
||||
* "status": HTTP statuscode
|
||||
* "protocol": Protocol / Version of HTTP (ex: HTTP/1.1)
|
||||
* "http_method": The HTTP method (ex: GET, POST, HEAD)
|
||||
* "http_refer": The referer for this action
|
||||
|
||||
In addition to the extended logging fields one can also choose to enable/add from 47 additional custom logging HTTP fields enabled in the suricata.yaml file. The additional fields can be enabled as following:
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
- eve-log:
|
||||
enabled: yes
|
||||
type: file #file|syslog|unix_dgram|unix_stream
|
||||
filename: eve.json
|
||||
# the following are valid when type: syslog above
|
||||
#identity: "suricata"
|
||||
#facility: local5
|
||||
#level: Info ## possible levels: Emergency, Alert, Critical,
|
||||
## Error, Warning, Notice, Info, Debug
|
||||
types:
|
||||
- alert
|
||||
- http:
|
||||
extended: yes # enable this for extended logging information
|
||||
# custom allows additional http fields to be included in eve-log
|
||||
# the example below adds three additional fields when uncommented
|
||||
#custom: [Accept-Encoding, Accept-Language, Authorization]
|
||||
custom: [accept, accept-charset, accept-encoding, accept-language,
|
||||
accept-datetime, authorization, cache-control, cookie, from,
|
||||
max-forwards, origin, pragma, proxy-authorization, range, te, via,
|
||||
x-requested-with, dnt, x-forwarded-proto, accept-range, age,
|
||||
allow, connection, content-encoding, content-language,
|
||||
content-length, content-location, content-md5, content-range,
|
||||
content-type, date, etags, last-modified, link, location,
|
||||
proxy-authenticate, referrer, refresh, retry-after, server,
|
||||
set-cookie, trailer, transfer-encoding, upgrade, vary, warning,
|
||||
www-authenticate]
|
||||
|
||||
|
||||
The benefits here of using the extended logging is to see if this action for example was a POST or perhaps if a download of an executable actually returned any bytes.
|
||||
|
||||
Examples
|
||||
~~~~~~~~
|
||||
|
||||
Event with non-extended logging:
|
||||
|
||||
::
|
||||
|
||||
|
||||
"http": {
|
||||
"hostname": "www.digip.org",
|
||||
"url" :"\/jansson\/releases\/jansson-2.6.tar.gz",
|
||||
"http_user_agent": "<User-Agent>",
|
||||
"http_content_type": "application\/x-gzip"
|
||||
}
|
||||
|
||||
Event with extended logging:
|
||||
|
||||
::
|
||||
|
||||
|
||||
"http": {
|
||||
"hostname": "direkte.vg.no",
|
||||
"url":".....",
|
||||
"http_user_agent": "<User-Agent>",
|
||||
"http_content_type": "application\/json",
|
||||
"http_refer": "http:\/\/www.vg.no\/",
|
||||
"http_method": "GET",
|
||||
"protocol": "HTTP\/1.1",
|
||||
"status":"200",
|
||||
"length":310
|
||||
}
|
||||
|
||||
Event type: DNS
|
||||
---------------
|
||||
|
||||
Fields
|
||||
~~~~~~
|
||||
|
||||
Outline of fields seen in the different kinds of DNS events:
|
||||
|
||||
* "type": Indicating DNS message type, can be "answer" or "query".
|
||||
* "id": <needs explanation>
|
||||
* "rrname": Resource Record Name (ex: a domain name)
|
||||
* "rrtype": Resource Record Type (ex: A, AAAA, NS, PTR)
|
||||
* "rdata": Resource Data (ex. IP that domain name resolves to)
|
||||
* "ttl": Time-To-Live for this resource record
|
||||
|
||||
|
||||
Examples
|
||||
~~~~~~~~
|
||||
|
||||
Example of a DNS query for the IPv4 address of "twitter.com" (resource record type 'A'):
|
||||
|
||||
::
|
||||
|
||||
|
||||
"dns": {
|
||||
"type": "query",
|
||||
"id": 16000,
|
||||
"rrname": "twitter.com",
|
||||
"rrtype":"A"
|
||||
}
|
||||
|
||||
Example of a DNS answer with an IPv4 (resource record type 'A') return:
|
||||
|
||||
::
|
||||
|
||||
|
||||
"dns": {
|
||||
"type": "answer",
|
||||
"id":16000,
|
||||
"rrname": "twitter.com",
|
||||
"rrtype":"A",
|
||||
"ttl":8,
|
||||
"rdata": "199.16.156.6"
|
||||
}
|
||||
@ -0,0 +1,101 @@
|
||||
Eve JSON Output
|
||||
===============
|
||||
|
||||
JSON output
|
||||
|
||||
Starting in 2.0, Suricata can output alerts, http events, dns events, tls events and file info through json.
|
||||
|
||||
The most common way to use this is through 'EVE', which is a firehose approach where all these logs go into a single file.
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
outputs:
|
||||
- eve-log:
|
||||
enabled: yes
|
||||
type: file #file|syslog|unix_dgram|unix_stream
|
||||
filename: eve.json
|
||||
types:
|
||||
- alert
|
||||
- http:
|
||||
extended: yes # enable this for extended logging information
|
||||
- dns
|
||||
- tls:
|
||||
extended: yes # enable this for extended logging information
|
||||
- files:
|
||||
force-magic: no # force logging magic on all logged files
|
||||
force-md5: no # force logging of md5 checksums
|
||||
#- drop
|
||||
- ssh
|
||||
|
||||
Each alert, http log, etc will go into this one file: 'eve.json'. This file can than be processed by Logstash for example.
|
||||
|
||||
|
||||
Multiple Logger Instances
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
It is possible to have multiple 'EVE' instances, for example the following is valid:
|
||||
|
||||
::
|
||||
|
||||
|
||||
outputs:
|
||||
- eve-log:
|
||||
enabled: yes
|
||||
type: file
|
||||
filename: eve-ips.json
|
||||
types:
|
||||
- alert
|
||||
- drop
|
||||
|
||||
- eve-log:
|
||||
enabled: yes
|
||||
type: file
|
||||
filename: eve-nsm.json
|
||||
types:
|
||||
- http
|
||||
- dns
|
||||
- tls
|
||||
|
||||
So here the alerts and drops go into 'eve-ips.json', while http, dns and tls go into 'eve-nsm.json'.
|
||||
|
||||
In addition to this, each log can be handled completely separately:
|
||||
|
||||
::
|
||||
|
||||
|
||||
outputs:
|
||||
- alert-json-log:
|
||||
enabled: yes
|
||||
filename: alert-json.log
|
||||
- dns-json-log:
|
||||
enabled: yes
|
||||
filename: dns-json.log
|
||||
- drop-json-log:
|
||||
enabled: yes
|
||||
filename: drop-json.log
|
||||
- http-json-log:
|
||||
enabled: yes
|
||||
filename: http-json.log
|
||||
- ssh-json-log:
|
||||
enabled: yes
|
||||
filename: ssh-json.log
|
||||
- tls-json-log:
|
||||
enabled: yes
|
||||
filename: tls-json.log
|
||||
|
||||
For most output types, you can add multiple:
|
||||
|
||||
::
|
||||
|
||||
|
||||
outputs:
|
||||
- alert-json-log:
|
||||
enabled: yes
|
||||
filename: alert-json1.log
|
||||
- alert-json-log:
|
||||
enabled: yes
|
||||
filename: alert-json2.log
|
||||
|
||||
Except for drop and tls, for those only one logger instance is supported.
|
||||
@ -0,0 +1,8 @@
|
||||
EVE
|
||||
======
|
||||
|
||||
.. toctree::
|
||||
|
||||
eve-json-output
|
||||
eve-json-format
|
||||
eve-json-examplesjq
|
||||
Binary file not shown.
@ -0,0 +1,62 @@
|
||||
What to do with files-json.log output
|
||||
=====================================
|
||||
|
||||
.. toctree::
|
||||
|
||||
script-follow-json
|
||||
mysql
|
||||
postgresql
|
||||
useful-queries-for-mysql-and-postgresql
|
||||
mongodb
|
||||
logstash-kibana-and-suricata-json-output
|
||||
|
||||
Suricata has the ability to produce the files-json.log output.
|
||||
Basically this is a JSON style format output logfile with entries like this:
|
||||
|
||||
::
|
||||
|
||||
{
|
||||
"timestamp": "10\/01\/2012-16:52:59.217616",
|
||||
"ipver": 4,
|
||||
"srcip": "80.239.217.171",
|
||||
"dstip": "192.168.42.197",
|
||||
"protocol": 6,
|
||||
"sp": 80,
|
||||
"dp": 32982,
|
||||
"http_uri": "\/frameworks\/barlesque\/2.11.0\/desktop\/3.5\/style\/main.css", "http_host": "static.bbci.co.uk", "http_referer": "http:\/\/www.bbc.com\/", "filename": "\/frameworks\/barl
|
||||
esque\/2.11.0\/desktop\/3.5\/style\/main.css",
|
||||
"magic": "ASCII text, with very long lines, with no line terminators",
|
||||
"state": "CLOSED",
|
||||
"md5": "be7db5e9a4416a4123d556f389b7f4b8",
|
||||
"stored": false,
|
||||
"size": 29261
|
||||
}
|
||||
|
||||
for every single file that crossed your http pipe.
|
||||
This in general is very helpful and informative.
|
||||
In this section we are going to try to explore/suggest approaches for putting it to actual use, since it could aggregate millions of entries in just a week.
|
||||
There are a god few options in general since the JSON style format is pretty common.
|
||||
http://www.json.org/
|
||||
|
||||
|
||||
This guide offers a couple of approaches -
|
||||
use of custom created script with MySQL or PostgreSQL import (bulk or continuous)
|
||||
or importing it directly to MongoDB(native import of JSON files).
|
||||
|
||||
Please read the all the pages before you jump into executing scripts and/or installing/configuring things.
|
||||
Te guide is written using Ubuntu LTS server 12.04
|
||||
|
||||
Thee are 3 options in general that we suggest, that we are going to explain here:
|
||||
|
||||
1. import JSON into MySQL
|
||||
2. import JSON into PostgreSQL
|
||||
3. import JSON into MongoDB
|
||||
|
||||
The suggested approach is
|
||||
configure Suricata.yaml
|
||||
configure your Database
|
||||
run the script (not applicable to MongoDB)
|
||||
and then execute queries against the DB to get the big picture.
|
||||
|
||||
|
||||
Peter Manev
|
||||
@ -0,0 +1,236 @@
|
||||
Logstash Kibana and Suricata JSON output
|
||||
========================================
|
||||
|
||||
With the release of Suricata 2.0rc1 , Suricata introduces all JSON output capability.
|
||||
What is JSON - http://en.wikipedia.org/wiki/JSON
|
||||
|
||||
|
||||
One way to handle easily Suricata's JSON log outputs is through Kibana - http://kibana.org/ :
|
||||
> Kibana is a highly scalable interface for Logstash (http://logstash.net/) and ElasticSearch (http://www.elasticsearch.org/) that allows you to efficiently search, graph, analyze and otherwise make sense of a mountain of logs.
|
||||
>
|
||||
|
||||
The installation is very simple/basic start up with minor specifics for ubuntu. You can be up and running, looking through the logs in under 5 min.
|
||||
The downloads can be found here - http://www.elasticsearch.org/overview/elkdownloads/
|
||||
|
||||
This is what yo need to do.
|
||||
|
||||
Suricata
|
||||
---------
|
||||
|
||||
Make sure your Suricata is compiled/installed with libjansson support enabled:
|
||||
|
||||
::
|
||||
|
||||
|
||||
$ suricata --build-info
|
||||
This is Suricata version 2.0 RELEASE
|
||||
Features: NFQ PCAP_SET_BUFF LIBPCAP_VERSION_MAJOR=1 AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG LIBNET1.1 HAVE_HTP_URI_NORMALIZE_HOOK HAVE_NSS HAVE_LIBJANSSON
|
||||
...
|
||||
libnss support: yes
|
||||
libnspr support: yes
|
||||
libjansson support: --> yes <--
|
||||
Prelude support: no
|
||||
PCRE jit: no
|
||||
libluajit: no
|
||||
libgeoip: yes
|
||||
Non-bundled htp: yes
|
||||
Old barnyard2 support: no
|
||||
CUDA enabled: no
|
||||
...
|
||||
|
||||
If it isn't check out the [[Suricata_installation]] page to install or compile Suricata for your distribution.
|
||||
**NOTE:** you will need these packages installed -> **libjansson4** and *libjansson-dev* before compilation.
|
||||
|
||||
Configure suricata
|
||||
------------------
|
||||
|
||||
In your suricata.yaml
|
||||
|
||||
::
|
||||
|
||||
|
||||
# "United" event log in JSON format
|
||||
- eve-log:
|
||||
enabled: yes
|
||||
type: file #file|syslog|unix_dgram|unix_stream
|
||||
filename: eve.json
|
||||
# the following are valid when type: syslog above
|
||||
#identity: "suricata"
|
||||
#facility: local5
|
||||
#level: Info ## possible levels: Emergency, Alert, Critical,
|
||||
## Error, Warning, Notice, Info, Debug
|
||||
types:
|
||||
- alert
|
||||
- http:
|
||||
extended: yes # enable this for extended logging information
|
||||
- dns
|
||||
- tls:
|
||||
extended: yes # enable this for extended logging information
|
||||
- files:
|
||||
force-magic: yes # force logging magic on all logged files
|
||||
force-md5: yes # force logging of md5 checksums
|
||||
#- drop
|
||||
- ssh
|
||||
- smtp
|
||||
- flow
|
||||
|
||||
Install ELK (elasticsearch, logstash, kibana)
|
||||
---------------------------------------------
|
||||
|
||||
First install the dependencies
|
||||
(
|
||||
**NOTE:**
|
||||
ELK recommends running with Oracle Java - how to ->
|
||||
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/setup-service.html#_installing_the_oracle_jdk
|
||||
)
|
||||
|
||||
Otherwise you can install the openjdk:
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
apt-get install apache2 openjdk-7-jdk openjdk-7-jre-headless
|
||||
|
||||
Then download and install the software.
|
||||
|
||||
Make sure you download the latest versions -
|
||||
http://www.elasticsearch.org/overview/elkdownloads/
|
||||
The installation process is simple (for example):
|
||||
|
||||
::
|
||||
|
||||
|
||||
wget https://download.elasticsearch.org/kibana/kibana/kibana-3.0.0.tar.gz
|
||||
wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.6.1.deb
|
||||
wget https://download.elastic.co/logstash/logstash/packages/debian/logstash_1.5.3-1_all.deb
|
||||
|
||||
tar -C /var/www/ -xzf kibana-3.0.0.tar.gz
|
||||
dpkg -i elasticsearch-1.6.1.deb
|
||||
dpkg -i logstash_1.5.3-1_all.deb
|
||||
|
||||
Logstash configuration
|
||||
----------------------
|
||||
|
||||
Create and save a *logstash.conf* file with the following content in the /etc/logstash/conf.d/ directory :
|
||||
|
||||
::
|
||||
|
||||
|
||||
touch /etc/logstash/conf.d/logstash.conf
|
||||
|
||||
Insert the following(make sure the directory path is correct):
|
||||
|
||||
::
|
||||
|
||||
|
||||
input {
|
||||
file {
|
||||
path => ["/var/log/suricata/eve.json"]
|
||||
sincedb_path => ["/var/lib/logstash/"]
|
||||
codec => json
|
||||
type => "SuricataIDPS"
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
filter {
|
||||
if [type] == "SuricataIDPS" {
|
||||
date {
|
||||
match => [ "timestamp", "ISO8601" ]
|
||||
}
|
||||
ruby {
|
||||
code => "if event['event_type'] == 'fileinfo'; event['fileinfo']['type']=event['fileinfo']['magic'].to_s.split(',')[0]; end;"
|
||||
}
|
||||
}
|
||||
|
||||
if [src_ip] {
|
||||
geoip {
|
||||
source => "src_ip"
|
||||
target => "geoip"
|
||||
#database => "/opt/logstash/vendor/geoip/GeoLiteCity.dat"
|
||||
add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
|
||||
add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}" ]
|
||||
}
|
||||
mutate {
|
||||
convert => [ "[geoip][coordinates]", "float" ]
|
||||
}
|
||||
if ![geoip.ip] {
|
||||
if [dest_ip] {
|
||||
geoip {
|
||||
source => "dest_ip"
|
||||
target => "geoip"
|
||||
#database => "/opt/logstash/vendor/geoip/GeoLiteCity.dat"
|
||||
add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
|
||||
add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}" ]
|
||||
}
|
||||
mutate {
|
||||
convert => [ "[geoip][coordinates]", "float" ]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
output {
|
||||
elasticsearch {
|
||||
host => localhost
|
||||
#protocol => http
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
Configure the start-up services
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
update-rc.d elasticsearch defaults 95 10
|
||||
update-rc.d logstash defaults
|
||||
|
||||
service apache2 restart
|
||||
service elasticsearch start
|
||||
service logstash start
|
||||
|
||||
Enjoy
|
||||
-----
|
||||
|
||||
That's all. Now make sure Suricata is running and you have logs written in your JSON log files and you point your browser towards ->
|
||||
|
||||
::
|
||||
|
||||
|
||||
http://localhost/kibana-3.0.0
|
||||
|
||||
**NOTE:**
|
||||
Some ready to use templates can be found here:
|
||||
|
||||
* https://github.com/pevma/Suricata-Logstash-Templates
|
||||
|
||||
From here on if you would like to customize and familiarize yourself more with the interface you should read the documentation about Kibana and Logstash.
|
||||
Please have in mind that this is a very quick(under 5 min) tutorial. You should customize and review the proper way for you of using it as a service and/or consider using *httpS web interface and reversy proxy with some authentication*.
|
||||
|
||||
Some possible customization of the output of Logstash and Kibana - >
|
||||
|
||||
|
||||
|
||||
.. image:: logstash-kibana-and-suricata-json-output/Logstash1.png
|
||||
|
||||
|
||||
.. image:: logstash-kibana-and-suricata-json-output/Logstash2.png
|
||||
|
||||
|
||||
.. image:: logstash-kibana-and-suricata-json-output/Logstash3.png
|
||||
|
||||
|
||||
.. image:: logstash-kibana-and-suricata-json-output/Logstash4.png
|
||||
|
||||
|
||||
.. image:: logstash-kibana-and-suricata-json-output/Logstash5.png
|
||||
|
||||
|
||||
.. image:: logstash-kibana-and-suricata-json-output/Logstash6.png
|
||||
|
||||
Peter Manev
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 76 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 133 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 29 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 123 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 17 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 81 KiB |
@ -0,0 +1,97 @@
|
||||
MongoDB
|
||||
=======
|
||||
|
||||
If you do not have it installed, follow the istructions here:
|
||||
http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/
|
||||
|
||||
Basically you do:
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10
|
||||
deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen
|
||||
sudo apt-get update && sudo apt-get install mongodb-10gen
|
||||
|
||||
|
||||
The bigest benefit of MongoDB is that it can natively import json.log files:
|
||||
if you have MongoDB installed - all you have to do is:
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
mongoimport --db filejsondb --collection filejson --file files-json.log
|
||||
|
||||
here:
|
||||
|
||||
* --db filejsondb is the database,
|
||||
* --collections filejson is the equivalent of SQL "table"
|
||||
* --file files-json.log - is the json log created and logged into from Suricata.
|
||||
|
||||
last but not least - it would automatically create the db and tables for you.
|
||||
|
||||
this would import a 5 Gb (15 million entries) json log file in about 5-10 minutes - default configuration, without tuning MongoDB for high performance. (your set up and HW will definitely have effect on the import time )
|
||||
|
||||
|
||||
|
||||
MongoDB Example queries (once you have imported the files-json.log - described above - just go ahead with these queries):
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
db.files.group( { cond : {"magic":/.*PDF*./ }, key: {"srcip":true,"http_host":true,"magic":true} ,initial: {count: 0},reduce: function(value, total) {total+=value.count;} } );
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
db.filejson.find({magic:/.*PDF.*/},{srcip:1,http_host:1,magic:1}).sort({srcip:1,http_host:1,magic:1}).limit(20)
|
||||
|
||||
|
||||
Get a sorted table biggest to smallest number hosts of file downloads:
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
> map = function () { emit({srcip:this.srcip,http_host:this.http_host,magic:this.magic}, {count:1}); }
|
||||
function () {
|
||||
emit({srcip:this.srcip, http_host:this.http_host, magic:this.magic}, {count:1});
|
||||
}
|
||||
> reduce = function(k, values) {var result = {count: 0}; values.forEach(function(value) { result.count += value.count; }); return result; }
|
||||
function (k, values) {
|
||||
var result = {count:0};
|
||||
values.forEach(function (value) {result.count += value.count;});
|
||||
return result;
|
||||
}
|
||||
> db.filejson.mapReduce(map,reduce,{out: "myoutput" });
|
||||
{
|
||||
"result" : "myoutput",
|
||||
"timeMillis" : 578806,
|
||||
"counts" : {
|
||||
"input" : 3110871,
|
||||
"emit" : 3110871,
|
||||
"reduce" : 673186,
|
||||
"output" : 219840
|
||||
},
|
||||
"ok" : 1,
|
||||
}
|
||||
> db.myoutput.find().sort({'value.count':-1}).limit(10)
|
||||
{ "_id" : { "srcip" : "184.107.x.x", "http_host" : "arexx.x", "magic" : "very short file (no magic)" }, "value" : { "count" : 42560 } }
|
||||
{ "_id" : { "srcip" : "66.135.210.182", "http_host" : "www.ebay.co.uk", "magic" : "XML document text" }, "value" : { "count" : 30896 } }
|
||||
{ "_id" : { "srcip" : "66.135.210.62", "http_host" : "www.ebay.co.uk", "magic" : "XML document text" }, "value" : { "count" : 27812 } }
|
||||
{ "_id" : { "srcip" : "213.91.x.x", "http_host" : "www.focxxxx.x", "magic" : "HTML document, ISO-8859 text" }, "value" : { "count" : 26301 } }
|
||||
{ "_id" : { "srcip" : "195.168.x.x", "http_host" : "search.etaxxx.x", "magic" : "JPEG image data, JFIF standard 1.01, comment: \"CREATOR: gd-jpeg v1.0 (using IJG JPEG v80), quality = 100\"" }, "value" : { "count" : 16131 } }
|
||||
{ "_id" : { "srcip" : "184.107.x.x", "http_host" : "p2p.arxx.x:2710", "magic" : "ASCII text, with no line terminators" }, "value" : { "count" : 15829 } }
|
||||
{ "_id" : { "srcip" : "213.91.x.x", "http_host" : "www.focxx.x", "magic" : "HTML document, ISO-8859 text" }, "value" : { "count" : 14472 } }
|
||||
{ "_id" : { "srcip" : "64.111.199.222", "http_host" : "syndication.exoclick.com", "magic" : "HTML document, ASCII text, with very long lines, with no line terminators" }, "value" : { "count" : 14009 } }
|
||||
{ "_id" : { "srcip" : "69.171.242.70", "http_host" : "www.facebook.com", "magic" : "ASCII text, with no line terminators" }, "value" : { "count" : 13098 } }
|
||||
{ "_id" : { "srcip" : "69.171.242.74", "http_host" : "www.facebook.com", "magic" : "ASCII text, with no line terminators" }, "value" : { "count" : 12801 } }
|
||||
>
|
||||
|
||||
|
||||
|
||||
Peter Manev
|
||||
@ -0,0 +1,36 @@
|
||||
MySQL
|
||||
=====
|
||||
|
||||
If you do not have MySQL installed - go ahead and do so:
|
||||
|
||||
::
|
||||
|
||||
|
||||
sudo apt-get update && sudo apt-get upgrade
|
||||
sudo apt-get install mysql-server mysql-client
|
||||
|
||||
|
||||
For MySQL make sure you create a db and a table:
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
mysql>create database filejsondb;
|
||||
mysql> create user 'filejson'@'localhost' IDENTIFIED BY 'PASSWORD123';
|
||||
Query OK, 0 rows affected (0.00 sec)
|
||||
mysql> grant all privileges on filejsondb.* to 'filejson'@'localhost' with grant option;
|
||||
mysql> flush privileges;
|
||||
mysql> use filejsondb;
|
||||
|
||||
mysql> CREATE TABLE filejson( time_received VARCHAR(64), ipver VARCHAR(4), srcip VARCHAR(40), dstip VARCHAR(40), protocol SMALLINT UNSIGNED, sp SMALLINT UNSIGNED, dp SMALLINT UNSIGNED, http_uri TEXT, http_host TEXT, http_referer TEXT, filename TEXT, magic TEXT, state VARCHAR(32), md5 VARCHAR(32), stored VARCHAR(32), size BIGINT UNSIGNED);
|
||||
|
||||
mysql> show columns from filejson;
|
||||
|
||||
|
||||
|
||||
OPTIONALLY - if you would like you can add in the MD5 whitelist table and import the data as described here ( [[Filemd5 and white/black listing with MD5]] )
|
||||
|
||||
now you can go ahead and execute the script - [[Script FollowJSON]]
|
||||
|
||||
Peter Manev
|
||||
@ -0,0 +1,80 @@
|
||||
PostgreSQL
|
||||
==========
|
||||
|
||||
If you do not have PostgreSQL installed:
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
sudo apt-get update && sudo apt-get upgrade
|
||||
sudo apt-get install postgresql
|
||||
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
sudo vim /etc/postgresql/9.1/main/pg_hba.conf
|
||||
|
||||
change the line:
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
local all all trust
|
||||
|
||||
to
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
local all all md5
|
||||
|
||||
|
||||
login and change passwords
|
||||
|
||||
::
|
||||
|
||||
|
||||
sudo -u postgres psql postgres
|
||||
\password postgres
|
||||
|
||||
|
||||
Then -
|
||||
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
create database filejsondb;
|
||||
\c filejsondb;
|
||||
create user filejson with password 'PASSWORD123';
|
||||
CREATE TABLE filejson( time_received VARCHAR(64), ipver VARCHAR(4), srcip VARCHAR(40), dstip VARCHAR(40), protocol INTEGER, sp INTEGER, dp INTEGER, http_uri TEXT, http_host TEXT, http_referer TEXT, filename TEXT, magic TEXT, state VARCHAR(32), md5 VARCHAR(32), stored VARCHAR(32), size BIGINT);
|
||||
grant all privileges on database filejsondb to filejson;
|
||||
|
||||
Log out and log in again (with the "filejson" user) to test if everything is ok:
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
psql -d filejson -U filejson
|
||||
|
||||
|
||||
|
||||
|
||||
Optionally you could create and import the MD5 white list data if you wish - generally the same guidance as described in:
|
||||
[[Filemd5 and white/black listing with MD5]]
|
||||
|
||||
Some more general info and basic commands/queries:
|
||||
http://jazstudios.blogspot.se/2010/06/postgresql-login-commands.html
|
||||
http://www.thegeekstuff.com/2009/05/15-advanced-postgresql-commands-with-examples/
|
||||
|
||||
|
||||
now you can go ahead and execute the script - [[Script FollowJSON]]
|
||||
|
||||
Peter Manev
|
||||
@ -0,0 +1,98 @@
|
||||
Script FollowJSON
|
||||
=================
|
||||
|
||||
BEFORE you run the script - make sure you have set up suricata.yaml and your database correctly !!
|
||||
|
||||
Suricata.yaml:
|
||||
|
||||
1. make sure json-log is enabled
|
||||
2. and append is set to yes
|
||||
3. optionally - you have compilled in Suricata with MD5's enabled
|
||||
|
||||
MD5's are enabled and forced in the suricata yaml config ( :ref:`MD5 <md5>` )
|
||||
bottom of the page "Log all MD5s without any rules" .
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
- file-log:
|
||||
enabled: yes
|
||||
filename: files-json.log
|
||||
append: yes
|
||||
#filetype: regular # 'regular', 'unix_stream' or 'unix_dgram'
|
||||
force-magic: yes # force logging magic on all logged files
|
||||
force-md5: yes # force logging of md5 checksums
|
||||
|
||||
|
||||
**Append is set to yes** - this is very important if you "follow" , json.log - if you use the tool to constantly parse and insert logs from files-json.log as they are being written onto the log file.
|
||||
|
||||
|
||||
There is a python script (in BETA now and) available here:
|
||||
|
||||
* https://redmine.openinfosecfoundation.org/attachments/download/843/FollowJSON.tar.gz
|
||||
|
||||
that you can use for helping out in importing files-json.log entries into a MSQL or PostgreSQL database.
|
||||
|
||||
The script would allow you to do the following:
|
||||
|
||||
|
||||
* it contains 2 files
|
||||
* one python executable
|
||||
* one yaml config file
|
||||
* one LICENSE (GPLv2)
|
||||
|
||||
This is what the script does:
|
||||
|
||||
1. Multi-threaded - spawns multiple processes if itself
|
||||
2. uses yaml as configuration
|
||||
3. Can:
|
||||
|
||||
3.1. Read files-json.log file
|
||||
|
||||
3.1.1. - Continuously - as logs are being written in the log file
|
||||
3.1.2. - mass import a stand alone files-json.log into a database
|
||||
|
||||
3.2. Into (your choice)
|
||||
|
||||
3.2.1. MySQL DB (locally/remotely,ip)
|
||||
3.2.2. PostgreSQL DB (locally/remotely,ip)
|
||||
|
||||
4. Customizable number of processes (default is number of cores - if you have more then 16 - suggested value is NumCores/2)
|
||||
5. Customizable "chunk" lines to read at once by every process - suggested (default) value is 10 (16 cores = 16 processes * 10 = 160 entries per second)
|
||||
|
||||
**Please look into the configurational yaml file** for more information.
|
||||
|
||||
The script is in BETA state - it has been tested , it works - but still, you should test it and adjust the configuration accordingly and run it on your test environment first before you put it in production.
|
||||
|
||||
After you have made:
|
||||
|
||||
#. your choices of database type (MySQL or PostgreSQL and installed/configured tables for it),
|
||||
#. created the appropriate database structure and tables (explained in the next tutorial(s) ),
|
||||
#. adjusted the yaml configuration accordingly,
|
||||
#. started Suricata,
|
||||
|
||||
you would need:
|
||||
|
||||
::
|
||||
|
||||
|
||||
sudo apt-get install python-yaml python-mysqldb python-psycopg2
|
||||
|
||||
Then you just run the script, after you have started Suricata:
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
sudo python Follow_JSON_Multi.py
|
||||
|
||||
if you would like to execute the script in the background:
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
sudo python Follow_JSON_Multi.py &
|
||||
|
||||
Peter Manev
|
||||
@ -0,0 +1,138 @@
|
||||
Useful queries - for MySQL and PostgreSQL
|
||||
=========================================
|
||||
|
||||
|
||||
General Purpose and Useful Queries (MySQL - 99% the same for PostgreSQL) for the files-json.log databases and tables:
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
mysql>select srcip,http_host,count(*) as total from filejson where magic like "%PDF document%" group by srcip,http_host order by total DESC limit 10;
|
||||
|
||||
above top 10 source ip from which PDF's where downloaded
|
||||
change srcip with dstip to get top 10 IPs downloading PDFs
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
mysql>select srcip,http_host,count(*) as total from filejson where magic like "%executable%" group by srcip,http_host order by total DESC limit 10;
|
||||
|
||||
above top 10 source ip from which executables where downloaded from,
|
||||
change srcip with dstip to get top 10 IPs downloading executables
|
||||
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
mysql> SELECT srcip,http_host,count(*) AS Total , (COUNT(*) / (SELECT COUNT(*) FROM filejson where magic like "%executable%")) * 100 AS 'Percentage to all items' FROM filejson WHERE magic like "%executable%" GROUP BY srcip,http_host order by total DESC limit 10;
|
||||
|
||||
::
|
||||
|
||||
|
||||
+----------------+----------------------+-------+-------------------------+
|
||||
| srcip | http_host | Total | Percentage to all items |
|
||||
+----------------+----------------------+-------+-------------------------+
|
||||
| 149.5.130.7 | ws.livepcsupport.com | 225 | 9.1167 |
|
||||
..............................
|
||||
.............................
|
||||
|
||||
This would give you a sorted table depicting source ip and host name, number of executable downloads from that host/source ip and what percentage is that of the total executable downloads.
|
||||
Note: the term executable means - dll, exe, com, msi, java ... and so on , NOT just .exe files
|
||||
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
mysql>select count(magic) as totalPDF from filejson where magic like "%PDF%"
|
||||
|
||||
This will give you the total number of PDFs out of all files
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
mysql>SELECT ( select count(magic) from filejson where magic like "%PDF%" ) as "PDF Total" , (select count(magic) from filejson where magic like "%executable%") as "Executables Total" , (select count(magic) from filejson where filename like "%.xls") as "Excel Total";
|
||||
|
||||
This will give you:
|
||||
|
||||
::
|
||||
|
||||
|
||||
+-----------+-------------------+-------------+
|
||||
| PDF Total | Executables Total | Excel Total |
|
||||
+-----------+-------------------+-------------+
|
||||
| 391 | 2468 | 7 |
|
||||
+-----------+-------------------+-------------+
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
mysql> SELECT ( select count(magic) from filejson where magic like "%PDF%" ) as "PDF Total" , (select count(magic) from filejson where magic like "%executable%") as "Executables Total" , (select count(magic) from filejson where filename like "%.xls") as "Excel Total", (select count(magic) from filejson) as "TOTAL NUMER OF FILES";
|
||||
|
||||
::
|
||||
|
||||
|
||||
+-----------+-------------------+-------------+----------------------+
|
||||
| PDF Total | Executables Total | Excel Total | TOTAL NUMER OF FILES |
|
||||
+-----------+-------------------+-------------+----------------------+
|
||||
| 391 | 2468 | 7 | 3743925 |
|
||||
+-----------+-------------------+-------------+----------------------+
|
||||
|
||||
the above query - a breakdown for PDF, executables and files hat have extension .xls
|
||||
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
mysql>select srcip,filename,http_host,count(*) as total from filejson where filename like "%.xls" group by srcip,filename,http_host order by total DESC limit 10;
|
||||
|
||||
the above will select top 10 source ip and document NAMES where excel files (files with extension .xls) were downloaded form
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
mysql>select srcip,http_host,count(*) as total from filejson where filename like "%.exe" group by srcip,http_host order by total DESC limit 10;
|
||||
|
||||
the above will select the top 10 source ips from where ".exe" files where downloaded from
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
mysql>select srcip,http_host,count(*) as total from filejson where filename like "%.doc" group by srcip,http_host order by total DESC limit 10;
|
||||
|
||||
the above for ".doc" files
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
mysql>select magic,http_host,count(*) as count from filejson group by magic,http_host order by count DESC limit 20;
|
||||
|
||||
select top 20 hosts grouped and ordered by count
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
mysql>select dstip,size,count(*) as total from filejson group by dstip,size order by total DESC limit 10;
|
||||
|
||||
the above query will show you he top 10 downloading ips by size of downloads
|
||||
|
||||
|
||||
::
|
||||
|
||||
|
||||
mysql>select dstip,http_host,count(*) as total from filejson where filename like "%.exe" group by dstip order by total DESC limit 5;
|
||||
|
||||
the above query will show you the top 5 downloading ips (and the hosts they downloaded from) that downloaded files with .exe extensions.
|
||||
|
||||
|
||||
Peter Manev
|
||||
@ -0,0 +1,11 @@
|
||||
Output
|
||||
======
|
||||
|
||||
.. toctree::
|
||||
|
||||
eve/index.rst
|
||||
../configuration/lua-output
|
||||
syslog-alerting-comp
|
||||
custom-http-logging
|
||||
../configuration/log-rotation
|
||||
files-json-log-output/files-json-log-output.rst
|
||||
@ -0,0 +1,63 @@
|
||||
Syslog Alerting Compatibility
|
||||
=============================
|
||||
|
||||
Suricata can alert via sylog which is a very handy feature for central log collection, compliance, and reporting to a SIEM. Instructions on setting this up can be found in the .yaml file in the section where you can configure what type of alert (and other) logging you would like.
|
||||
|
||||
However, there are different syslog daemons and there can be parsing issues with the syslog format a SIEM expects and what syslog format Suricata sends. The syslog format from Suricata is dependent on the syslog daemon running on the Suricata sensor but often the format it sends is not the format the SIEM expects and cannot parse it properly.
|
||||
|
||||
Popular syslog daemons
|
||||
----------------------
|
||||
|
||||
* **syslogd** - logs system messages
|
||||
* **syslog-ng** - logs system messages but also suports TCP, TLS, and other enhanced enterprise features
|
||||
* **rsyslogd** - logs system messages but also support TCP, TLS, multi-threading, and other enhanced features
|
||||
* **klogd** - logs kernel messages
|
||||
* **sysklogd** - basically a bundle of syslogd and klogd
|
||||
|
||||
If the syslog format the Suricata sensor is sending is not compatible with what your SIEM or syslog collector expects, you will need to fix this. You can do this on your SIEM if it is capable of being able to be configured to interpret the message, or by configuring the syslog daemon on the Suricata sensor itself to send in a format you SIEM can parse. The latter can be done by applying a template to your syslog config file.
|
||||
|
||||
Finding what syslog daemon you are using
|
||||
----------------------------------------
|
||||
|
||||
There are many ways to find out what syslog daemon you are using but here is one way:
|
||||
|
||||
::
|
||||
|
||||
|
||||
cd /etc/init.d
|
||||
ls | grep syslog
|
||||
|
||||
You should see a file with the word syslog in it, e.g. "syslog", "rsyslogd", etc. Obviously if the name is "rsyslogd" you can be fairly confident you are running rsyslogd. If unsure or the filename is just "syslog", take a look at that file. For example, if it was "rsyslogd", run:
|
||||
|
||||
::
|
||||
|
||||
|
||||
less rsyslogd
|
||||
|
||||
At the top you should see a comment line that looks something like this:
|
||||
|
||||
::
|
||||
|
||||
|
||||
# rsyslog Starts rsyslogd/rklogd.
|
||||
|
||||
Locate those files and look at them to give you clues as to what syslog daemon you are running. Also look in the *start()* section of the file you ran "less" on and see what binaries get started because that can give you clues as well.
|
||||
|
||||
Example
|
||||
-------
|
||||
|
||||
Here is an example where the Suricata sensor is sending syslog messages in rsyslogd format but the SIEM is expecting and parsing them in a sysklogd format. In the syslog configuration file (ususally in /etc with a filename like rsyslog.conf or syslog.conf), first add the template:
|
||||
|
||||
::
|
||||
|
||||
|
||||
$template sysklogd, "<%PRI%>%syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%"
|
||||
|
||||
Then send it to the syslog server with the template applied:
|
||||
|
||||
::
|
||||
|
||||
|
||||
user.alert @10.8.75.24:514;sysklogd
|
||||
|
||||
Of course this is just one example and it will probably be different in your environment depending on what syslog daemons and SIEM you use but hopefully this will point you in the right direction.
|
||||
Loading…
Reference in New Issue