This patch is updating af-packet to discard packets that have been
sent to a socket before all socket in a fanout group have been setup.
Without this, there is no way to assure that all packets for a single
flow will be treated by the same thread.
Tests have been done on a system with an ixgbe network card. When using
'cluster_flow' load balancing and disactivating receive hash on the iface:
ethtool -K IFACE rxhash off
then suricata is behaving as expected and all packets for a single flow
are treated by the same thread.
For some unknown reason, this is not the case when using cluster_cpu. It
seems that in that case the load balancing is not perfect on the card side.
The rxhash offloading has a direct impact on the cluster_flow load balancing
because load balancing is done by using a generic hash key attached to
each skb. This hash can be computed by the network card or can be
computed by the kernel. In the xase of a ixgbe network card, it seems there
is some issue with the hash key for TCP. This explains why it is necessary to
remove the rxhash offloading to have a correct behavior. This could also
explain why cluster_cpu is currently failing because the card is using the
same hash key computation to do the RSS queues load balancing.
This allows for registering a keyword under another name while keeping
the old name active and supported.
Do this for 'luajit', which can now also be used as just 'lua'.
The HTTP module of Eve didn't register itself with the app-layer
for HTTP. This meant that if no other HTTP logger was active, the
HTTP logging in Eve wouldn't work.
This patch makes the HTTP Eve module register itself correctly.
Bug #1133.
To remain constistent with the other logs, set the event type to
the same name as the structure containing the defails. In this
case fileinfo.
Part of bug #1127.
Due to what appears to be an issue in logstash, the 'file' part of
the file event types was masked by a field that logstash-forwarder
added itself.
Since logstash-forwarder is an important part of the logstash stack,
this patch works around the issue by renaming our 'file' structure
to 'fileinfo', thus resolving the naming conflict.
Bug #1127
Move pfring_enable_ring to the start of ReceivePfringLoop() so that
it's guaranteed to be called after all threads have called
pfring_set_cluster first.
This is necessary because pfring will already make packets available
to thread N, while thread N+1 is still registering itself. This leads
to cases where the first packet(s) of a flow are processed by a
different thread in Suricata than the later ones.
This is a race condition only at start up. New flows after the pfring
initialization is complete will not be influenced by this.
Bug #1129.
This patch updates the timestamp format used in eve loggin.
It uses a ISO 8601 comptatible string. This allow tools parsing
the output to easily detect adn/or use the timestamp.
In the EVE JSON output, the value of the timestamp key has been
changed to 'timestamp' (instead of 'time'). This allows tools
like Splunk to detect the timestamp and use it without configuration.
Logstash configuration is simple:
input {
file {
path => [ "/usr/local/var/log/suricata/eve.json" ]
codec => json
type => "suricata-log"
}
}
filter {
if [type] == "suricata-log" {
date {
match => [ "timestamp", "ISO8601" ]
}
}
}
In splunk, auto detection of the fle format is failling and it seems
you need to define a type to parse JSON in
$SPLUNK_DIR/etc/system/local/props.conf:
[suricata]
KV_MODE = json
NO_BINARY_CHECK = 1
TRUNCATE = 0
Then you can simply declare the log file in
$SPLUNK_DIR/etc/system/local/inputs.conf:
[monitor:///usr/local/var/log/suricata/eve.json]
sourcetype = suricata
In both cases the timestamp are correctly imported by
the tools.
PF_RING is delivering the packet with VLAN header stripped. This
patch updates the code to get the information from PF_RING extended
header information.
This patch uses the new function SCKernelVersionIsAtLeast to know
that we've got a old kernel that do not strip the VLAN header from
the message before sending it to userspace.
In case of 'alert ip' rules that have ports, the port checks would
be bypassed for non-port protocols, such as ICMP. This would lead to
a rule matching: a false positive.
This patch adds a check. If the rule has a port setting other than
'any' and the protocol is not TCP, UDP or SCTP, then we rule won't
match.
Rules with 'alert ip' and ports are rare, so the impact should be
minimal.
Bug #611.
Eve-log would call GET_VLAN_ID on the packets vlan header if p->vlan_idx
was bigger than 0. GET_VLAN_ID would then unconditionally dereference
p->vlanh[0] or [1]. However, there are a number of cases in which these
pointers are not set. Defrag pseudo packets, AF_PACKET and in the future
PF_RING, do set the id's, but not the header pointers.
This patch adds 2 new macro's which are wrappers around a function:
VLAN_GET_ID1 and VLAN_GET_ID2 get the id's by calling DecodeVLANGetId.
This function will return the correct id.
Bug #1120.
app-layer-ssh.c:165:5: warning: Value stored to 'input_len' is never read
input_len -= 1;
^ ~
1 warning generated.
app-layer-ssh.c:160:5: warning: Value stored to 'input_len' is never read
input_len -= 4;
^ ~
1 warning generated.
Previously the software version would only contain up to the first
space.
E.g. in SSH-2.0-OpenSSH_4.7p1 Debian-8ubuntu3
It would contain "OpenSSH_4.7p1".
This patch changes the behavior to:
"OpenSSH_4.7p1 Debian-8ubuntu3"
This patch adds a check that was missing when specifying BPF filter
from a file. Suricata behavior should have been the same as when
BPF filter is specified on command line.
Fix warning spotted by clang on FreeBSD:
source-ipfw.c:241:49: warning: use of logical '||' with constant operand [-Wconstant-logical-operand]
if (suricata_ctl_flags & (SURICATA_STOP || SURICATA_KILL)) {
^ ~~~~~~~~~~~~~
source-ipfw.c:241:49: note: use '|' for a bitwise operation
if (suricata_ctl_flags & (SURICATA_STOP || SURICATA_KILL)) {
^~
|
Use same logic as the one used in other capture mode.
Eric Leblond
Victor Julien