- eBPF filter: any BPF like filter can be developed. An example of filter accepting only packet for some VLANs is provided. A bypass implementation is also provided.
- eBPF load balancing: provide programmable load balancing. A simple ippair load balancing is provided.
- XDP programs: suricata can load XDP programs. A bypass program is provided.
- eBPF load balancing: provide programmable load balancing. Simple ippair load balancing is provided.
- XDP programs: Suricata can load XDP programs. A bypass program is provided.
Bypass can be implemented in eBPF and XDP. The advantage of XDP is that the packets are dropped at the earliest stage
possible. So performance is better. But bypassed packets don't reach the network so you can't use this on regular
@ -42,7 +42,7 @@ More info about XDP:
Requirements
------------
You will need a kernel that supports XDP and, for real performance improvement, a network
You will need a kernel that supports XDP and, for the most performance improvement, a network
card that support XDP in the driver.
Suricata XDP code has been tested with 4.13.10 but 4.15 or later is necessary to use all
@ -62,7 +62,7 @@ This guide has been confirmed on Debian/Ubuntu "LTS" Linux.
Disable irqbalance
~~~~~~~~~~~~~~~~~~
Irqbalance may cause issues in most setups described here, so it is recommended
``irqbalance`` may cause issues in most setups described here, so it is recommended
to deactivate it ::
systemctl stop irqbalance
@ -76,7 +76,7 @@ You need to run a kernel 4.13 or newer.
Clang and dependencies
~~~~~~~~~~~~~~~~~~~~~~
Make sure you have clang (>=3.9) installed on the system ::
Make sure you have ``clang`` (>=3.9) installed on the system ::
sudo apt install clang
@ -101,26 +101,26 @@ Now, you can build and install the library ::
sudo ldconfig
In some cases your system will not find the libbpf library that is installed under
`/usr/lib64` so you may need to modify your ldconfig configuration.
``/usr/lib64`` so you may need to modify your ldconfig configuration.
Compile and install Suricata
----------------------------
To get Suricata source, you can use the usual ::
git clone https://github.com/OISF/suricata.git
git clone https://github.com/OISF/suricata.git
cd suricata && git clone https://github.com/OISF/libhtp.git -b 0.5.x
./autogen.sh
Then you need to add the ebpf flags to configure and specify the Clang
Then you need to add the eBPF flags to configure and specify the Clang
compiler for building all C sources, including the eBPF programs ::
In this guide I'll describe using the Myricom libpcap support. I'm going to assume you installed the card properly, installed the Sniffer driver and made sure that all works. Make sure that in your dmesg you see that the card is in sniffer mode:
In this guide I'll describe using the Myricom libpcap support. I'm going to assume you installed the card properly, installed the Sniffer driver and made sure that all works. Make sure ``dmesg`` shows that the card is in sniffer mode:
::
@ -11,11 +11,10 @@ In this guide I'll describe using the Myricom libpcap support. I'm going to assu
[ 2102.860241] myri_snf INFO: eth4: Link0 is UP
[ 2101.341965] myri_snf INFO: eth5: Link0 is UP
I have installed the Myricom runtime and libraries in /opt/snf
I have installed the Myricom runtime and libraries in ``/opt/snf``
Compile Suricata against Myricom's libpcap:
::
@ -35,7 +34,7 @@ Next, configure the amount of ringbuffers. I'm going to work with 8 here, as my
buffer-size: 512kb
checksum-checks: no
The 8 threads setting makes Suricata create 8 reader threads for eth5. The Myricom driver makes sure each of those is attached to its own ringbuffer.
The 8 threads setting causes Suricata to create 8 reader threads for eth5. The Myricom driver makes sure each of those is attached to its own ringbuffer.
Then start Suricata as follows:
@ -44,18 +43,17 @@ Then start Suricata as follows:
Note that the pcap.buffer-size yaml setting shown above is currently ignored when using Myricom cards. The value is passed through to the pcap_set_buffer_size libpcap API within the Suricata source code. From Myricom support:
Note that the ``pcap.buffer-size`` configuration setting shown above is currently ignored when using Myricom cards. The value is passed through to the ``pcap_set_buffer_size`` libpcap API within the Suricata source code. From Myricom support:
::
"The libpcap interface to Sniffer10G ignores the pcap_set_buffer_size() value. The call to snf_open() uses zero as the dataring_size which informs the Sniffer library to use a default value or the value from the SNF_DATARING_SIZE environment variable."
The following pull request opened by Myricom in the libpcap project indicates that a future SNF software release could provide support for setting the SNF_DATARING_SIZE via the pcap.buffer-size yaml setting:
@ -17,12 +17,13 @@ Given a file-store configuration like::
write-meta: yes # write a .meta file if set to yes
include-pid: yes # include the pid in filenames if set to yes.
the following changes will need to be made to convert to a V2 style configuration:
The following changes will need to be made to convert to a v2 style configuration:
* A``version`` field must be set to 2.
* The``version`` field must be set to 2.
* The ``log-dir`` field should be renamed to ``dir``. It is recommended to use a new directory instead of an existing v1 directory.
* Remove the ``waldo`` option. It is no longer used.
* Remove the ``write-meta`` option. Optionally set ``write-fileinfo`` to enable writing of a metadata file along side the extracted file. Not that this option is disabled by default as a ``fileinfo`` event can be written to the Eve log file.
* Remove the ``write-meta`` option.
* Optionally set ``write-fileinfo`` to enable writing of a metadata file along side the extracted file. Not that this option is disabled by default as a ``fileinfo`` event can be written to the Eve log file.
* Remove the ``include-pid`` option. There is no equivalent to this option in file-store v2.
@ -12,7 +12,7 @@ In case of HTTP, the parser takes care of dechunking and unzipping the request a
This means that settings in the stream engine, reassembly engine and the application layer parsers all affect the workings of the file extraction.
What files are actually extracted and stored to disk is controlled by the rule language.
The rule language controls which files are extracted and stored on disk.
Supported protocols are:
@ -29,7 +29,7 @@ Settings
*file-store.stream-depth* controls how far into a stream reassembly is done. Beyond this value no reassembly will be done. This means that after this value the HTTP session will no longer be tracked. By default a setting of 1 Megabyte is used. 0 sets it to unlimited. If set to no, it is disabled and stream.reassembly.depth is considered. Non-zero values must be greater than ``stream.stream-depth`` to be used.
*libhtp.default-config.request-body-limit* / *libhtp.server-config.<config>.request-body-limit* controls how much of the HTTP request body is tracked for inspection by the http_client_body keyword, but also used to limit file inspection. A value of 0 means unlimited.
*libhtp.default-config.request-body-limit* / *libhtp.server-config.<config>.request-body-limit* controls how much of the HTTP request body is tracked for inspection by the `http_client_body` keyword, but also used to limit file inspection. A value of 0 means unlimited.
*libhtp.default-config.response-body-limit* / *libhtp.server-config.<config>.response-body-limit* is like the request body limit, only it applies to the HTTP response body.
@ -40,8 +40,8 @@ Output
File-Store and Eve Fileinfo
---------------------------
There are two output modules for logging information about files
extracted. The first is ``eve.files`` which is an ``eve`` sub-logger
There are two output modules for logging information about extracted files.
The first is ``eve.files`` which is an ``eve`` sub-logger
that logs ``fileinfo`` records. These ``fileinfo`` records provide
metadata about the file, but not the actual file contents.
@ -114,55 +114,47 @@ Rules
Without rules in place no extraction will happen. The simplest rule would be:
::
alert http any any -> any any (msg:"FILE store all"; filestore; sid:1; rev:1;)
This will simply store all files to disk.
Want to store all files with a pdf extension?
Want to store all files with a pdf extension?
::
alert http any any -> any any (msg:"FILE PDF file claimed"; fileext:"pdf"; filestore; sid:2; rev:1;)
Or rather all actual pdf files?
Or rather all actual pdf files?
::
alert http any any -> any any (msg:"FILE pdf detected"; filemagic:"PDF document"; filestore; sid:3; rev:1;)
Or rather only store files from black list checksum md5 ?
Or rather only store files from black list checksum md5 ?
::
alert http any any -> any any (msg:"Black list checksum match and extract MD5"; filemd5:fileextraction-chksum.list; filestore; sid:4; rev:1;)
Or only store files from black list checksum sha1 ?
Or only store files from black list checksum sha1 ?
::
alert http any any -> any any (msg:"Black list checksum match and extract SHA1"; filesha1:fileextraction-chksum.list; filestore; sid:5; rev:1;)
Or finally store files from black list checksum sha256 ?
Or finally store files from black list checksum sha256 ?
::
alert http any any -> any any (msg:"Black list checksum match and extract SHA256"; filesha256:fileextraction-chksum.list; filestore; sid:6; rev:1;)
Bundled with the Suricata download is a file with more example rules. In the archive, go to the rules/ directory and check the files.rules file.
Bundled with the Suricata download, is a file with more example rules. In the archive, go to the `rules` directory and check the ``files.rules`` file.
@ -125,21 +114,11 @@ Log all MD5s without any rules
If you would like to log MD5s for everything and anything that passes through the traffic that you are inspecting with Suricata, but not log the files themselves, all you have to do is disable file-store and enable only the JSON output with forced MD5s - in suricata.yaml like so:
::
- file-store:
version: 2
enabled: no # set to yes to enable
log-dir: files # directory to store the files
force-magic: yes # force logging magic on all stored files
force-hash: [md5] # force logging of md5 checksums
#waldo: file.waldo # waldo file to store the file_id across runs
- file-log:
enabled: yes
filename: files-json.log
append: no
#filetype: regular # 'regular', 'unix_stream' or 'unix_dgram'
force-magic: yes # force logging magic on all logged files
force-filestore: no
force-hash: [md5] # force logging of md5 checksums
The amount of threads depends on the system and the configuration.
The actual thread count will depend on the system and the configuration.
To see statistics the ``stats.log`` can be checked::
To see statistics, check the ``stats.log`` file::
sudo tail -f /var/log/suricata/stats.log
Every 20 seconds by default it will show updated informations about the current
By default, it is updated every 8 seconds to show updated values with the current
state, like how many packets have been processed and what type of traffic was
decoded.
Alerting
--------
To test the IDS functionality of Suricata it's best to test with a signature. The signature with ID ``2100498`` from the ET Open ruleset is written specific for such test cases.
To test the IDS functionality of Suricata it's best to test with a signature. The signature with
ID ``2100498`` from the ET Open ruleset is written specific for such test cases.
2100498::
@ -123,10 +124,10 @@ To test the IDS functionality of Suricata it's best to test with a signature. Th
The syntax and logic behind those signatures is covered in other chapters. This
will alert on any IP traffic that has the content within its payload. This rule
can be triggered quite easy. Before we trigger it, we will tail on the
``fast.log`` so we see the result.
can be triggered quite easy. Before we trigger it, start ``tail`` to see updates to
``fast.log``.
Ruletrigger::
Ruletrigger::
sudo tail -f /var/log/suricata/fast.log
curl http://testmyids.com/
@ -140,19 +141,20 @@ This should include the timestamp and the IP of your system.
EVE Json
--------
The more advanced output is the EVE Json output which is explained in detail in
The more advanced output is the EVE JSON output which is explained in detail in
:ref:`Eve JSON Output <eve-json-output>`. To see what this looks like it's
recommended to use ``jq`` to parse the JSON output.
New versions of Suricata will occasionally include updated config files:
``classification.config`` and ``reference.config``. Since the Suricata
installation will not overwrite these if they exist, they should be manually
installation will not overwrite these if they exist, they must be manually
updated. If there are no local modifications they can simply be overwritten
by the ones Suricata supplies.
@ -33,11 +33,22 @@ remove features.
Upgrading 5.0 to 6.0
--------------------
- SIP now enabled by default
- EDP now enabled by default
- RDP now enabled by default
- ERSPAN Type I enabled by default.
Major changes
~~~~~~~~~~~~~
- New protocols enabled by default: mqtt, rfb
- SSH Client fingerprinting for SSH clients
- Conditional logging
- Initial HTTP/2 support
- DCERPC logging
- Improved EVE logging performance
Removals
~~~~~~~~
- File-store v1 has been removed. If using file extraction, the file-store configuration will need to be updated to version 2. See :ref:`filestore-update-v1-to-v2`.
- File-store v1 has been removed. If using file extraction, the file-store configuration
will need to be updated to version 2. See :ref:`filestore-update-v1-to-v2`.
Upgrading 4.1 to 5.0
--------------------
@ -60,5 +71,6 @@ Removals
~~~~~~~~
- ``dns-log``, the text dns log. Use EVE.dns instead.
- ``file-log``, the non-EVE JSON file log. Use EVE.files instead.
- ``drop-log``, the non-EVE JSON drop log.
See https://suricata-ids.org/about/deprecation-policy/
Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. It is open source and owned by a community-run non-profit foundation, the Open Information Security Foundation (OISF). Suricata is developed by the OISF.
Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. It is open source and owned by a community-run non-profit foundation, the Open Information Security Foundation (`OISF<https://oisf.net>`__). Suricata is developed by the OISF.
About the Open Information Security Foundation
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The Open Information Security Foundation is a non-profit foundation organized to build community and to support open-source security technologies like Suricata, the world-class IDS/IPS engine.
License
-------
The Suricata source code is licensed under version 2 of the
:doc:`licenses/gnu-gpl-v2.0`. This documentation is licensed under the
Jeff Lucovsky
6 years ago
committed by
Victor Julien