Getting clock through Time Stamp Counter (TSC) can be precise and fast,
however only for a short duration of time.
The implementation across CPUs seems to vary. The original idea is to
increment the counter with every tick. Then dividing the delta of CPU ticks
by the CPU frequency can return the time that passed.
However, the CPU clock/frequency can change over time, resulting in uneven
incrementation of TSC. On some CPUs this is handled by extra logic.
As a result, obtaining time through this method might drift from the real
time.
This commit therefore substitues TSC time retrieval by the standard system
call wrapped in GetTime function - on Linux it is gettimeofday.
Ticket: 7115
So far, when the data size was passed to the THash API, it was sent as
a sizeof(Struct) which works fine for the other data types as they have
a fixed length but not for the StringType.
However, because of the sizeof construct, the length of a string type
dataset was always taken to be 16 Bytes which is only the size of the struct
itself. It did not accomodate the actual size of the string that the
StringType holds. Fix this so that the memuse that is used to determine
whether memcap was reached also takes into consideration the size of the
actual string.
Bug 3910
Don't assume the ntlmssp version field is always present if the flag is
set. Instead keep track of the offsets of the data of the various blobs
and see if there is space for the version.
Inspired by how Wireshark does the parsing.
Bug: #7121.
Enable backoff for most rules. The rules looking at the session start up
use a count of 1 and a multiplier of 2.
Post-3whs rules use a count of 1 and a multiplier of 10.
Implement new `type backoff` for thresholding. This allows alerts to be
limited.
A count of 1 with a multiplier of 10 would generate alerts for matching packets:
1, 10, 100, 1000, 10000, 100000, etc.
A count of 1 with a multiplier of 2 would generate alerts for matching packets:
1, 2, 4, 8, 16, 32, etc.
Like with other thresholds, rule actions like drop and setting of
flowbits will still be performed for each matching packet.
Current implementation is only for the by_flow tracker and for per rule
threshold statements.
Tracking is done using uint32_t. When it reaches this value, the rest of
the packets in the tracker will use the silent match.
Ticket: #7120.
Instead of a Host and IPPair table thresholding layer, use a dedicated
THash to store both. This allows hashing on host+sid+tracker or
ippair+sid+tracker, to create more unique hash keys.
This allows for fewer hash collisions.
The per rule tracking also uses this, so that the single big lock is no
longer a single point of contention.
Reimplement storage for flow thresholds to reuse as much logic as
possible from the host/ippair/rule thresholds.
Ticket: #426.
Thresholding often has 2 stages:
1. recording matches
2. appling an action, like suppress
E.g. with something like:
threshold:type limit, count 10, seconds 3600, track by_src;
the recording state is about counting 10 first hits for an IP,
then followed by the "suppress" state that might last an hour.
By_src/by_dst are expensive, as they do a host table lookup and lock
the host. If many threads require this access, lock contention becomes
a serious problem.
This patch adds a thread local cache to avoid the synchronization
overhead. When the threshold for a host enters the "apply" stage,
a thread local hash entry is added. This entry knows the expiry
time and the action to apply. This way the action can be applied
w/o the synchronization overhead.
A rbtree is used to handle expiration.
Implemented for IPv4.
Sascha Steinbiss
dependabot[bot]
Jeff Lucovsky
Philippe Antoine
Lukas Sismis
Shivani Bhardwaj
Jason Ish
Victor Julien
Philippe Antoine