Only use RSS queue count when cluster_qm is used. Only use core count
when cluster_flow is used.
Use a local variable to simplify the check so that we don't have to deal
with the extra flags.
Older system may pretend they can support FANOUT but then fail to
work at runtime. CentOS6 is an example of this. It would fail to
start up with the default configuration with errors like:
[15770] 21/6/2016 -- 16:00:13 - (tm-threads.c:2168) <Notice> (TmThreadWaitOnThreadInit) -- all 4 packet processing threads, 4 management threads initialized, engine started.
[15785] 21/6/2016 -- 16:00:13 - (source-af-packet.c:1907) <Error> (AFPCreateSocket) -- [ERRCODE: SC_ERR_AFP_CREATE(190)] - Coudn't set fanout mode, error Protocol not available
[15785] 21/6/2016 -- 16:00:13 - (source-af-packet.c:1337) <Error> (ReceiveAFPLoop) -- [ERRCODE: SC_ERR_AFP_CREATE(190)] - Couldn't init AF_PACKET socket, fatal error
[15770] 21/6/2016 -- 16:00:13 - (suricata.c:2664) <Notice> (main) -- Signal Received. Stopping engine.
[15787] 21/6/2016 -- 16:00:13 - (source-af-packet.c:1907) <Error> (AFPCreateSocket) -- [ERRCODE: SC_ERR_AFP_CREATE(190)] - Coudn't set fanout mode, error Protocol not available
[15788] 21/6/2016 -- 16:00:13 - (source-af-packet.c:1907) <Error> (AFPCreateSocket) -- [ERRCODE: SC_ERR_AFP_CREATE(190)] - Coudn't set fanout mode, error Protocol not available
[15786] 21/6/2016 -- 16:00:13 - (source-af-packet.c:1907) <Error> (AFPCreateSocket) -- [ERRCODE: SC_ERR_AFP_CREATE(190)] - Coudn't set fanout mode, error Protocol not available
[15789] 21/6/2016 -- 16:00:13 - (flow-manager.c:693) <Perf> (FlowManager) -- 0 new flows, 0 established flows were timed out, 0 flows in closed state
[15787] 21/6/2016 -- 16:00:13 - (source-af-packet.c:1337) <Error> (ReceiveAFPLoop) -- [ERRCODE: SC_ERR_AFP_CREATE(190)] - Couldn't init AF_PACKET socket, fatal error
[15788] 21/6/2016 -- 16:00:13 - (source-af-packet.c:1337) <Error> (ReceiveAFPLoop) -- [ERRCODE: SC_ERR_AFP_CREATE(190)] - Couldn't init AF_PACKET socket, fatal error
[15786] 21/6/2016 -- 16:00:13 - (source-af-packet.c:1337) <Error> (ReceiveAFPLoop) -- [ERRCODE: SC_ERR_AFP_CREATE(190)] - Couldn't init AF_PACKET socket, fatal error
This patch adds a test that if run before the number of threads
is determined. If the test fails, only 1 thread is created.
As AF_PACKET handles csum offloading don't check for this type of
offloading. Other methods like pcap and netmap do require it to be
turned off.
Improve disable command suggestion wording.
Update the code to use mmap capture by default even in unset in
configuration file. mmap capture is now be turned off by using
explicitely 'use-mmap: no' in configuration.
If TPACKET_V3 is not defined then it is not available and we should
not build anything related to tpacket_v3. This will allow us to
activate it dy default and fallback to v2 if not available.
It is used to set the block size in tpacket_v3. It will allow user
to tune the capture depending on his bandwidth.
Default block size value has been updated to a bigger value to
allow more efficient wlak on block.
This patch adds a basic implementation of AF_PACKET tpacket v3. It
is basic in the way it is only working for 'workers' runnning mode.
If not in 'workers' mode there is a fallback to tpacket_v2. Feature
is activated via tpacket-v3 option in the af-packet section of
Suricata YAML.
For af-packet, pf-ring, netmap, and pcap use a generic
lookup function to find the configuration node for an
interface.
The new lookup function does not depend on the ordering
of the items inside the device configuration.
Update FlowManager/Recycler to use global name.
Also add # into thread number.
Update af-packet to use global threadnames.
Update pcap to use global threadnames.
Update pfring to use global threadnames.
Update erf-dag to use global threadnames.
Update nflog to use global threadnames.
Update netmap to use global threadnames.
Update napatech to use global threadnames.
This patch implements the rollover option in af_packet capture.
This should heavily minimize the packet drops as well as the
maximum bandwidth treated for a single flow.
The option has been deactivated by default but it is activated in
the af_packet default section. This ensure there is no change for
old users using an existing YAML. And new users will benefit from
the change.
This option is available since Linux 3.10. An analysis of af_packet
kernel code shows that setting the flag in all cases should not
cause any trouble for older kernel.
This patch implements the fanout load balancing modes available
in kernel 4.0. The more interesting is cluster_qm that does the
load balancing based on the RSS queues. So if the network card
is doing a flow based load balancing then a given socket will
receive all packets of a flow indepently of the CPU affinity.
If we manage to read the number of RSS queues from an interface,
this means that the optimal number of capture threads is equal
to the minimum of this number and of the number of cores on the
system.
This patch implements this logic thanks to the newly introduced
function GetIfaceRSSQueuesNum.
Add a new default value for the 'threads:' setting in af-packet: "auto".
This will create as many capture threads as there are cores.
Default runmode of af-packet to workers.
AF_PACKET is not setting the engine mode to IPS when some
interfaces are peered and use IPS mode. This is due to the
fact, it is possible to peer 2 interfaces and run an IPS on
them and have a third one that is running in normal IDS mode.
In fact this choice is the bad one as unwanted side effect is
that there is no drop log and that stream inline is not used.
To fix that, this patch puts suricata in IPS mode as soon as
there is two interfaces in IPS mode. And it displays a error
message to warn user that the accuracy of detection on IDS only
interfaces will be low.
A cast during the reading of a configuration variable was invalid
because a 16 bit integer was cast to a 32 bit integer. The called
function is only setting the pointer value to 1 or 0 so there is
no real issue there.
When handling error case on SCMallog, SCCalloc or SCStrdup
we are in an unlikely case. This patch adds the unlikely()
expression to indicate this to gcc.
This patch has been obtained via coccinelle. The transformation
is the following:
@istested@
identifier x;
statement S1;
identifier func =~ "(SCMalloc|SCStrdup|SCCalloc)";
@@
x = func(...)
... when != x
- if (x == NULL) S1
+ if (unlikely(x == NULL)) S1
Flush all waiting packets to be in sync with kernel when drop
occurs. This mode can be activated by setting use-emergency-flush
to yes in the interface configuration.
This patch adds a new feature to AF_PACKET capture mode. It is now
possible to use AF_PACKET in IPS and TAP mode: all traffic received
on a interface will be forwarded (at the Ethernet level) to an other
interface. To do so, Suricata create a raw socket and sends the receive
packets to a interface designed in the configuration file.
This patch adds two variables to the configuration of af-packet
interface:
copy-mode: ips or tap
copy-iface: eth1 #the interface where packet are copied
If copy-mode is set to ips then the packet wth action DROP are not
copied to the destination interface. If copy-mode is set to tap,
all packets are copied to the destination interface.
Any other value of copy-mode results in the feature to be unused.
There is no default interface for copy-iface and the variable has
to be set for the ids or tap mode to work.
For now, this feature depends of the release data system. This
implies you need to activate the ring mode and zero copy. Basically
use-mmap has to be set to yes.
This patch adds a peering of AF_PACKET sockets from the thread on
one interface to the threads on another interface. Peering is
necessary as if we use an other socket the capture socket receives
all emitted packets. This is made using a new AFPPeer structure to
avoid direct interaction between AFPTreadVars.
There is currently a bug in Linux kernel (prior to 3.6) and it is
not possible to use multiple threads.
You need to setup two interfaces with equality on the threads
variable. copy-mode variable must be set on the two interfaces
and use-mmap must be set to activated.
A valid configuration for an IPS using eth0 and vboxnet1 interfaces
will look like:
af-packet:
- interface: eth0
threads: 1
defrag: yes
cluster-type: cluster_flow
cluster-id: 98
copy-mode: ips
copy-iface: vboxnet1
buffer-size: 64535
use-mmap: yes
- interface: vboxnet1
threads: 1
cluster-id: 97
defrag: yes
cluster-type: cluster_flow
copy-mode: ips
copy-iface: eth0
buffer-size: 64535
use-mmap: yes
The mmaped mode was using a too small ring buffer size which was
not able to handle burst of packets coming from the network. This
may explain the important packet loss rate observed by Edward
Fjellskål.
This patch increases the default value and adds a ring-size
variable which can be used to manually tune the value.
clang was issuing some warnings related to unused return in function.
This patch adds some needed error treatment and ignore the rest of the
warnings by adding a cast to void.
Victor Julien
Eric Leblond
Jason Ish
maxtors
Zachary Rasmor
Anoop Saldanha
pi-rho