A cast during the reading of a configuration variable was invalid
because a 16 bit integer was cast to a 32 bit integer. The called
function is only setting the pointer value to 1 or 0 so there is
no real issue there.
Use test macro instead of direct access to action field.
This patch has been obtained by using the following
spatch file:
@@
Packet *p;
expression E;
@@
- p->action & E
+ TEST_PACKET_ACTION(p, E)
The action field in Packet structure should not be accessed
directly as the tunneled packet needs to update the root packet
and not the initial packet.
This patch is fixing issue #819 where suricata was not able to
drop fragmented packets in AF_PACKET IPS mode. It also fixes
drop capability for tunneled packets.
Bug emanates from byte_test, byte_jump and byte_extract keyword being
unable to handle negative offsets when the inspection pointer is at the
end of the buffer.
Now depth is kept in mind when we inspect chunks in client/server body.
This takes care of FPs originating from inspecting subsequent chunks that
match with depth, but shouldn't.
Add flowint lua functions for incrementing and decrementing flowints.
First use creates the var and inits to 0. So a call:
a = ScFlowintIncr(0)
Results in a == 1.
If the var reached UINT_MAX (2^32), it's not further incremented. If the
var reaches 0 it's not decremented further.
Calling ScFlowintDecr on a uninitialized var will init it to 0.
Example script:
function init (args)
local needs = {}
needs["http.request_headers"] = tostring(true)
needs["flowint"] = {"cnt_incr"}
return needs
end
function match(args)
a = ScFlowintIncr(0);
if a == 23 then
return 1
end
return 0
end
return 0
This script matches the 23rd time it's invoked on a flow.
Expose ScFlowintGet and ScFlowintSet functions to luajit. These set
flowints in real time, regardless of rule and/or script match.
Example:
function init (args)
local needs = {}
needs["http.request_headers"] = tostring(true)
needs["flowint"] = {"cnt"}
return needs
end
function match(args)
a = ScFlowintGet(0);
if a then
ScFlowintSet(0, a + 1)
else
ScFlowintSet(0, 1)
end
a = ScFlowintGet(0);
if a == 23 then
return 1
end
return 0
end
return 0
Script's init call first registers "cnt" at id 0, then 0 is used to use
this var.
sigs(priority wise) inside staging.
Previously we would assign signums before sig ordering, and hence the
order didn't actually reflect the order of the sig in the
sig_list(assuming sig reordering changed the sig_list). Staging would
use the old sig_nums to decide the priority of sigs.
2. Fix sig ordering for flowvar, flowbits, flowint, pktvar sigs. We have
introduced a new priority to treat sigs with set + read as lower
priority compared to set only sigs.
3. Previously we treated sigs with a "priority(keyword)" > another sig's
priority, as a sig with greater priority than the later. We have
reversed it. Now the sig priority ordering is 1,2,.etc. Updated
sigordering unittests to reflect the same.
Improved accuracy, improved performance. Performance improvement
noticeable with http heavy traffic and ruleset.
A lot of other cosmetic changes carried out as well. Wrappers introduced
for a lot of app layer functions.
Failing dce unittests disabled. Will be reintroduced in the updated dce
engine.
Cross transaction matching taken care of. FPs emanating from these
matches have now disappeared. Double inspection of transactions taken
care of as well.
The memset() inside PACKET_INITIALIZE() is redundant in some cases and
it is cleaner to do as part of the memory allocation. This simplifies
changes for integrating Tilera mPIPE support because the size of memory
cleared in that case is different from SIZE_OF_PACKET.
For the cases where Packets are directly allocated and then call
PACKET_INITIALIZE() without memset() first, this patch adds memset() calls.
A further change would use GetPacketFromAlloc() directly.
In case of error, errno is set by sendmsg which is called by
nfnetlink and which is called by libnetfilter_queue. This patch
displays the string expression of errno if verdict has failed.
Normally, there is one verdict per packet, i.e., we receive a packet,
process it, and then tell the kernel what to do with that packet (eg.
DROP or ACCEPT).
recv(), packet id x
send verdict v, packet id x
recv(), packet id x+1
send verdict v, packet id x+1
[..]
recv(), packet id x+n
send verdict v, packet id x+n
An alternative is to process several packets from the queue, and then send
a batch-verdict.
recv(), packet id x
recv(), packet id x+1
[..]
recv(), packet id x+n
send batch verdict v, packet id x+n
A batch verdict affects all previous packets (packet_id <= x+n),
we thus only need to remember the last packet_id seen.
Caveats:
- can't modify payload
- verdict is applied to all packets
- nfmark (if set) will be set for all packets
- increases latency (packets remain queued by the kernel
until batch verdict is sent).
To solve this, we only defer verdict for up to 20 packets and
send pending batch-verdict immediately if:
- no packets are currently queue
- current packet should be dropped
- current packet has different nfmark
- payload of packet was modified
This patch adds a configurable batch verdict support for workers runmode.
The batch verdicts are turned off by default.
Problem is that batch verdicts only work with kernels >= 3.1, i.e.
using newer libnetfilter_queue with an old kernel means non-working
suricata. So the functionnality has to be disabled by default.
currently, the packet payload recv()d from the nfqueue netlink
socket is copied into a new packet buffer.
This is required because the recv-buffer space used is tied
to the current thread, but a packet may be handed off to other
threads, and the recv-buffer can be re-used while the packet
is handled by another thread.
However, in worker runmode, the packet will always be handled
by the current thread, and the recv-buffer will only be reused
after the entire packet processing stack is done with the packet.
Thus, in worker runmode, we can avoid the copy and assign
the packet data area directly.
By randomizing chunk size around the choosen value, it is possible
to escape some evasion technics that are using the fact they know
chunk size to split the attack at the correct place.
This patch activates randomization by default and set the random
interval to chunk size value +- 10%.
Until now, when processing the TCP 3 way handshake (3whs), retransmissions
of SYN/ACKs are silently accepted, unless they are different somehow. If
the SEQ or ACK values are different they are considered wrong and events
are set. The stream events rules will match on this.
In some cases, this is wrong. If the client missed the SYN/ACK, the server
may send a different one with a different SEQ. This commit deals with this.
As it is impossible to predict which one the client will accept, each is
added to a list. Then on receiving the final ACK from the 3whs, the list
is checked and the state is updated according to the queued SYN/ACK.
Bug #802
Flowvars are set from pcre, and lock the flow when being set. However
when HTTP buffers were inspected, flow was already locked: deadlock.
This patch introduces a post-match list in the detection engine thread
ctx, where store candidates are kept. Then a post-match function is used
to finalize the storing if the rule matches.
Solves the deadlock and brings the handling of flowvars more in line
with flowbits and flowints.
Previously we would have forced all users to use nocase with http_host
keywords(since the hostname buffer is lowercase).
We now error out on sigs that has nocase set with http_host set. Also if
the http_host pattern or http_host pcre has an uppercase character set, we
invalidate such sigs. Unittests also updated to reflect the above change.
DetectEngineThreadCtxInit and DetectEngineThreadCtxInitForLiveRuleSwap did
pretty much the same thing, except for a counters registration. As can be
predicted with code duplication like this, things got out of sync. To make
sure this doesn't happen again, I created a helper function that does the
heavy lifting in this function.
Ssn flag STREAMTCP_FLAG_ZERO_TIMESTAMP was used in stream only. Due to
it's value it did not conflict with a real stream flag. Renamed it to
STREAMTCP_STREAM_FLAG_ZERO_TIMESTAMP.
The STREAMTCP_FLAG_TIMESTAMP flag is a ssn flag, however it was used in
the stream flag field. As it has the same value as
STREAMTCP_STREAM_FLAG_DEPTH_REACHED it's possible that stream reassembly
got confused by the timestamp.
Whan running suricata via 'suricata --af-packet', the list of interfaces
was containing the 'default' interface and sniffing it was attempted.
This was not wanted.
Rename struct DetectFigureFPAndId_t_ to DetectFPAndItsId_ and move it's
definition from inside the function where it's used to the global namespace,
as requested on #suricata.
Rename DetectEngineContentModifiedBufferSetup to DetectEngineContentModifierBufferSetup.
Also rename DetectFigureFPAndId() to DetectSetFastPatternAndItsId().
Updated DetectSetFastPatternAndItsId() to not exit on failure and return error.
All fp id assignment now happens in one go.
Also noticing a slight perf increase, probably emanating from improved cache
perf.
Removed irrelevant unittests as well.
In unix socket mode, Suricata was stopping processing pcap files
when a pcap file with an unsupported datalink was treated. This
patch updates error handling to allow Suricata to treat other
pcap files.
This patch adds a 'conf-get' command which get the configuration
value from suricata. Argument of the command is the name of the
variable to fetch.
The command syntax is the following:
{
"command": "conf-get",
"arguments": { "variable":value}
}
Pcap snaplen related modification broke compilation of Suricata for
system having old pcap library. This patch fixes the issue and allow
old pcap library to honour the snaplen value.
As reported in bug #688, htp_config_set_path_decode_u_encoding
function is not included in libhtp header before 0.3.0. Result
is that suricata compilation fail with an external htp library.
The following patch detect the issue and adds the missing
declaration.
Added a napatech section in the yaml configuration.
hba - host buffer allowance
use-all-streams - whether all streams should be used
streams - list of stream numbers to use when use-all-streams is no
The source-napatech.* files were modified to support the host buffer allowance configuration.
The runmode-napatech.c file was modified to support both the host buffer allowance configuration and stream configuration
Signed-off-by: Matt Keeler <mk@npulsetech.com>
This patch introduces 'snaplen' a new YAML variable in the pcap section.
It can be set per-interface to force pcap capture snaplen. If not set
it defaults to interface MTU if MTU can be known via a ioctl call and to
full capture if not.
Main objective of this patch is to use a dynamic snaplen to avoid
to truncate packet at the currently fixed snaplen.
It set snaplen to MTU length if the MTU can be retrieved. If not, it
does not set the snaplen which results in using a 65535 snaplen.
libpcap is trying to use mmaped capture and setup the ring by using buffer_size
as the total memory. It also use "rounded" snaplen as frame size. So if we set
snaplen to MTU when available we are optimal regarding the building of the ring.
This patch fixes an error in pointer arythmetic and add some
comments to increase maintanability of the code. It also
simplify the decoding code as a careful RFC reading indicate
that if we discard packet containing an authentication field,
it is only possible to have a single origin indication field.
Adds support for match-on conditions (src, dst, any, both)
Uses GEOIP_MEMORY_CACHE for performance reasons
Adds support for negation and multiple countries in the same rule
Bug fixes
Changed to take flow direction from rule, if present
Comments addressed. Unit tests added.
This patch introduces a new set of functions to the ConfGetChildValue
family. They permit to look under a default node if looking under
base node as failed. This will be used to access to default parameters
for a data type (for instance, first usage will be interface).