No longer set stream events after a gap or wrong thread. We know
we lost sync and are now in 'lets make the best of it'-mode. No
point in flooding the system with stream events.
Ticket #2484
Add a keyword configuration dump-all-headers, with allowed values
{both, request, response}, dumping all HTTP headers in the eve-log http
object. Each header is a single object in the list request_headers
(response_headers) with the following notation:
{
"name": <header name>,
"value": <header value>
}
To avoid forged malicious headers, the header name size is capped at 256
bytes, the header value size at 2048.
By default, dump-all-headers is disabled.
Add a raw-extraction option for smtp. When enabled, this feature will
store the raw e-mail inside a file, including headers, e-mail content,
attachments (base64 encoded). This content is stored in a normal File *,
allowing for normal file detection.
It'd also allow for all-emails extraction if a rule has
detect-filename:"rawmsg" matcher (and filestore).
Note that this feature is in contrast with decode-mime.
This feature is disabled by default, and will be disabled automatically
if decode-mime is enabled.
Previously this was limited to 16, however Netfilter allows
up to 65535 queues. Suricata now is able to create as many
queues as possible, but at the same time warns user if one
specifies more queues than available CPU cores.
This change involves dynamic (de)allocation of NFQ contexts
instead of on-stack arrays to use less memory.
If one needs to use multiple sequential Netfilter queues,
it can be done with a new '-q' option's syntax: "start:end"
(just like it's done with iptables '--queue-balance' option).
If a bad RST was received the stream inspection would not happen
for that packet, but it would still move the 'raw progress' tracker
forward. Following good packets would then fail to detect anything
before the 'raw progress' position.
Bug #2770
Reported-by: Alexey Vishnyakov
In the eve log the decoder events are added as optional counters. This
behaviour is enabled by default. However, lots of the counters are
missing, as the names colide with other counters.
E.g.
decoder.ipv6 counts ipv6 packets
decoder.ipv6.unknown_next_header counts how often an unknown next
header is encountered.
In this example 'ipv6' would be both a json integer and a json object.
It appears that jansson favours the first that is generated, so the
event counters are mostly missing.
This patch registers them as 'decoder.events.<event>' instead. As
these names are generated on the fly, a hash table to contain the
allocated strings was added as well.
Synchronize start was disabled for v2 when v3 was introduced, without
a reason being given.
Re-enable as v2 will otherwise also start reading packets before the
other threads are set up. This will lead to hashing issues.
Part of bug #2788.
The tpacket-v3 implementation of the synchonize start logic would
not correctly consider the timestamp parameter, leading to threads
starting before synchronization between threads was complete.
Bug #2788
Fixes nfqueue and delayed-detect.
On systems with small amount of traffic (or with no traffic at all)
nfqueue with 'delayed-detect' enabled hanged in 'workers' mode.
Bug #2362.
The DCERPC UDP detection would misfire on DNS with transaction
ID 0x0400. This would happen as the protocol detection engine
gives preference to pattern based detection over probing parsers for
performance reasons.
This hack/workaround fixes this specific case by still running the
probing parser if DCERPC has been detected on UDP. The probing
parser result will take precedence.
Bug #2736.
Invalid Teredo can lead to valid DNS traffic (or other UDP traffic)
being misdetected as Teredo. This leads to false negatives in the
UDP payload inspection.
Make the teredo code only consider a packet teredo if the encapsulated
data was decoded without any 'invalid' events being set.
Bug #2736.