Make sure we register the detect.alerts counter before packet runtime starts
even in delayed detect mode. The registration of new counters at packet
runtime is not supported by the counters api and might lead to crashes as there
is no proper locking to allow for this operation.
This changes how delayed detect works a bit. Now we call the ThreadInit
callback twice. The first call will only register the counter. The 2nd call
will do all the other setup. This way the counter is registered before the
counters api starts operating in the packet runtime.
Fixes the segv reported in ticket #1018.
This variable can be used to indicate to suricata that the host
running is running as a router or is in sniffing only mode.
This will used at least to determine which interfaces are used to
send reject message.
To be able to split code in functions in main, we need to pass
information about the current running Suricata to functions.
For that we create a structure to store suricata run parameters.
In this patch it allows to separate command line parsing and to
treat internal running mode in a switch just after command line
parsing.
The list-keyword and app-layer listing code was spread over all the
init code. This patch introduces a separate file to store non standard
running mode like these ones.
The TILE-Gx processor includes a packet processing engine, called
mPIPE, that can deliver packets directly into user space memory. It
handles buffer allocation and load balancing (either static 5-tuple
hashing, or dynamic flow affinity hashing are used here). The new
packet source code is in source-mpipe.c and source-mpipe.h
A new Tile runmode is added that configures the Suricata pipelines in
worker mode, where each thread does the entire packet processing
pipeline. It scales across all the Gx chips sizes of 9, 16, 36 or 72
cores. The new runmode is in runmode-tile.c and runmode-tile.h
The configure script detects the TILE-Gx architecture and defines
HAVE_MPIPE, which is then used to conditionally enable the code to
support mPIPE packet processing. Suricata runs on TILE-Gx even without
mPIPE support enabled.
The Suricata Packet structures are allocated by the mPIPE hardware by
allocating the Suricata Packet structure immediatley before the mPIPE
packet buffer and then pushing the mPIPE packet buffer pointer onto
the mPIPE buffer stack. This way, mPIPE writes the packet data into
the buffer, returns the mPIPE packet buffer pointer, which is then
converted into a Suricata Packet pointer for processing inside
Suricata. When the Packet is freed, the buffer is returned to mPIPE's
buffer stack, by setting ReleasePacket to an mPIPE release specific
function.
The code checks for the largest Huge page available in Linux when
Suricata is started. TILE-Gx supports Huge pages sizes of 16MB, 64MB,
256MB, 1GB and 4GB. Suricata then divides one of those page into
packet buffers for mPIPE.
The code is not yet optimized for high performance. Performance
improvements will follow shortly.
The code was originally written by Tom Decanio and then further
modified by Tilera.
This code has been tested with Tilera's Multicore Developement
Environment (MDE) version 4.1.5. The TILEncore-Gx36 (PCIe card) and
TILEmpower-Gx (1U Rack mount).
This patch should fix the bug #637. Between pcap files, it uses a
new function HostCleanup() to clear tag and threshold on host with
an IP regputation. An other consequence of this modification is
that Host init and shutdown are now init and shutdown unconditionaly.
This patch adds two commands to unix-command. 'iface-list' displays
the list of interface which are sniffed by Suricata and 'iface-stat'
display the available statistics for a single interface. For now,
this is the number of packets and the number of invalid checksums.
This patch introduces a unix command socket. JSON formatted messages
can be exchanged between suricata and a program connecting to a
dedicated socket.
The protocol is the following:
* Client connects to the socket
* It sends a version message: { "version": "$VERSION_ID" }
* Server answers with { "return": "OK|NOK" }
If server returns OK, the client is now allowed to send command.
The format of command is the following:
{
"command": "pcap-file",
"arguments": { "filename": "smtp-clean.pcap", "output-dir": "/tmp/out" }
}
The server will try to execute the "command" specified with the
(optional) provided "arguments".
The answer by server is the following:
{
"return": "OK|NOK",
"message": JSON_OBJECT or information string
}
A simple script is provided and is available under scripts/suricatasc. It
is not intended to be enterprise-grade tool but it is more a proof of
concept/example code. The first command line argument of suricatasc is
used to specify the socket to connect to.
Configuration of the feature is made in the YAML under the 'unix-command'
section:
unix-command:
enabled: yes
filename: custom.socket
The path specified in 'filename' is not absolute and is relative to the
state directory.
A new running mode called 'unix-socket' is also added.
When starting in this mode, only a unix socket manager
is started. When it receives a 'pcap-file' command, the manager
start a 'pcap-file' running mode which does not really leave at
the end of file but simply exit. The manager is then able to start
a new running mode with a new file.
To start this mode, Suricata must be started with the --unix-socket
option which has an optional argument which fix the file name of the
socket. The path is not absolute and is relative to the state directory.
THe 'pcap-file' command adds a file to the list of files to treat.
For each pcap file, a pcap file running mode is started and the output
directory is changed to what specified in the command. The running
mode specified in the 'runmode' YAML setting is used to select which
running mode must be use for the pcap file treatment.
This requires modification in suricata.c file where initialisation code
is now conditional to the fact 'unix-socket' mode is not used.
Two other commands exists to get info on the remaining tasks:
* pcap-file-number: return the number of files in the waiting queue
* pcap-file-list: return the list of waiting files
'pcap-file-list' returns a structured object as message. The
structure is the following:
{
'count': 2,
'files': ['file1.pcap', 'file2.pcap']
}
The output of the list-keyword is modified to include the url to
the keyword documentation when this is available. All documented
keywords should have their link set.
list-keyword can be used with an optional value:
no option or short: display list of keywords
csv: display a csv output on info an all keywords
all: display a human readable output of keywords info
$KWD: display the info about one keyword.
In list-keywords and list-app-layer mode, suricata now only
displays the messages linked with the feature. This allow users
to redirect the output and easily work on it. For exemple, the
csv output will be easily imported into a spreadsheet.
This patch update the list-keyword command. Without any option,
the previous behavior is conserved. If 'all' is used as option,
suricata print a csv formatted output of keyword information:
name;features;description
If a keyword name is used as argument, suricata print a readable
message:
tls.subject
Features: state inspecting
Description: Match TLS/SSL certificate Subject field
As we don't parse the YAML file when listing of keywords is asked,
suricata make a test on existence of the build-default directory.
So with a non standard (working) install (even a single configure
without option lead to a failure), the keyword listing fails
because the default logging directory does not exist.
This patch creates a pid file per default and use it to avoid to be
able to run two Suricata. Separate pid file have to be provided to
be able to do it.
Removed the Napatech 2GD support
runmode-napatech-3gd.c had an include from runmode-napatech.h which was erroneous and has been removed as well.
Signed-off-by: Matt Keeler <mk@npulsetech.com>
For use with Network Cards from Napatech utilizing the 3GD driver/api.
- Implemented new run modes in runmode-napatech-3gd.*
- Implemented capture/decode threads in source-napatech-3gd.*
- Integrated the new run modes and source into the build infrastructure.
New configure switches
--enabled-napatech-3gd : Turns on the NT 3GD support
--with-napatech-3gd-includes : The directory containing the NT 3GD header files
--with-napatech-3gd-libraries : The directory containing the NT 3GD libraries to link against.
New CLI switch
--napatech-3gd : Uses the Napatech 3GD run mode
Runmodes Supported:
- auto
- autofp
- workers
Notes:
- tested with 1 Gbps sustained traffic (no drops)
Signed-off-by: Matt Keeler <mk@npulsetech.com>