Commit Graph

91 Commits (e824a8be763135d70e134f94ac04d7c45b6e7917)

Author SHA1 Message Date
Jason Ish 796dd5223b tests: no longer necessary to provide successful return code
1 pass, 0 is fail.
10 years ago
Victor Julien dc008206e3 detect: fix scan-build warning
detect-engine-siggroup.c:700:38: warning: Call to 'malloc' has an allocation size of 0 bytes
    sgh->non_mpm_other_store_array = SCMalloc(non_mpm * sizeof(SignatureNonMpmStore));
                                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./util-mem.h:177:14: note: expanded from macro 'SCMalloc'
    ptrmem = malloc((a)); \
             ^~~~~~~~~~~
1 warning generated.
10 years ago
Victor Julien e0111fbb90 detect grouping: remove debug mem counters 10 years ago
Victor Julien de273d88cc detect: remove unused content minlen tracking 10 years ago
Victor Julien a96fa0fc2f detect: remove unused dport sgh hash 10 years ago
Victor Julien a34be23002 detect: simplify negated mpm handling 10 years ago
Victor Julien b84d6d402f detect grouping: multiple whitelist conditions
Instead of the binary yes/no whitelisting used so far, use different
values for different sorts of whitelist reasons. The port list will
be sorted by whitelist value first, then by rule count.

The goal is to whitelist groups that have weak sigs:

 - 1 byte pattern groups

 - SYN sigs

    Rules that check for SYN packets are mostly scan detection rules.
    They will be checked often as SYN packets are very common.

    e.g. alert tcp any any -> any 22 (flags:S,12; sid:123;)

    This patch adds whitelisting for SYN-sigs, so that the sigs end up
    in as unique groups as possible.

 - negated mpm sigs

    Currently negated mpm sigs are inspected often, so they are quite
    expensive. For this reason, try to whitelist them.

These values are set during 'stage 1', rule preprocessing.
10 years ago
Victor Julien 3c184c19cd detect grouping: port based group whitelisting
Whitelist some ports in grouping to make sure they get their own group.
10 years ago
Victor Julien 2ce03fbabb detect: split non-mpm list into syn/nosyn
Since SYN inspecting rules are expensive, this patch splits the
'non-mpm' list (i.e. the rules that are always considered) into
a 'syn' and 'non-syn' list. The SYN list is only inspected if the
packet has the SYN flag set, otherwise the non-syn list is used.

The syn-list contains _all_ rules. The non-syn list contains all
minus the rules requiring the SYN bit in a packet.
10 years ago
Victor Julien 26517b8b61 detect: mpm store frees mpm_ctx' it owns 10 years ago
Victor Julien fac2cc0560 detect: mpm deduplication
Create hash for mpm's that we can reuse. Have packet/stream mpms
use this.
10 years ago
Victor Julien f0ba00e51d detect: remove old unused code 10 years ago
Victor Julien d82df4eb8b detect-mpm: make sgh setup proto aware
Allow multi-proto, multi-direction sgh's.
10 years ago
Victor Julien 27e63a1e11 detect: track direction and ipproto of sgh
Each SGH has a unique ipproto and direction.
10 years ago
Victor Julien 0d3f671b55 detect: constify mpm/detect funcs 10 years ago
Victor Julien 6b2f831a70 mpm: SGH maxlen was actually minlen, so rename 10 years ago
Victor Julien 2716c78628 mpm: improve SGH content len tracking
SGH's track content length for rule grouping.

This patch changes the logic to only consider the pattern that is
used in the mpm for a sig.
10 years ago
Victor Julien e529ebb50e mpm: redo uri maxlen logic
The mpm_uricontent_maxlen logic was meant to track the shortest
possible pattern in the MPM of a SGH. So a minlen more than a maxlen.

This patch replaces the complicated tracking logic by a simpler
scheme. When the SGH's are finalize, the minlen is calculated.

It also fixes a small corner case where the calculated "maxlen" could
be wrong. This would require a smaller pattern in a rule to be forced
as fast pattern.
10 years ago
Victor Julien 6b2fb3fd5d Fix minor format string issues 10 years ago
Victor Julien a8c16405fb detect: properly size det_ctx::non_mpm_id_array
Track which sgh has the higest non-mpm sig count and use that value
to size the det_ctx::non_mpm_id_array array.
11 years ago
Victor Julien 4c10635dc1 detect: optimize non-mpm mask checking
Store id and mask in a single array of type SignatureNonMpmStore so
that both are loaded into the same cache line.
11 years ago
Victor Julien b5a3127151 detect: add mask check prefilter for non mpm list
Add mask array for non_mpm sigs, so that we can exclude many sigs before
we merge sort.

Shows 50% less non mpm sigs inspected on average.
11 years ago
Victor Julien ef6875d583 detect: Disable unused SignatureHeader code 11 years ago
Ken Steele 65af1f1c5e Remove sgh->mask_array
Not needed by new MPM opt.
11 years ago
Ken Steele 736ac6a459 Use SigIntId as the type for storing signature IDs (Internal)
Previously using uint32_t, but SigIntId is currently uint16_t, so arrays
will take less memory.
11 years ago
Victor Julien 1f57e25c03 detect: Add negated MPM to non-MPM array
Treat negated MPM sigs as if non-MPM, so we consider them always.

As MPM results and non-MPM rules lists are now merged and considered
for further inspection, rules that need to be considerd when a pattern
is absent are caught in the middle.

As a HACK/workaround this patch adds them to the non-MPM list. This
causes them to be inspected each time.
11 years ago
Victor Julien f5df526f9b Detect: create per sgh non-MPM rule array
Array of rule id's that are not using MPM prefiltering. These will be
merged with the MPM results array. Together these should lead to a
list of all the rules that can possibly match.
11 years ago
Ken Steele ff41c1c452 Correct size increase in SigGroupHeadStore()
The code was increasing the size of the allocated memory by 16, but
only increasing the stored size by 10. Now uses one variable for both
places.
11 years ago
Ken Steele 8f1d75039a Enforce function coding standard
Functions should be defined as:

int foo(void)
{
}

Rather than:
int food(void) {
}

All functions where changed by a script to match this standard.
12 years ago
Eric Leblond 1f07d1521e Fix realloc error handling
This patch is fixing realloc error handling. In case of a realloc
failure, it free the initial memory and continue existing error
handling.

The patch has been obtained via the following semantic patch and
a bit oh hand editing:

@@
expression x, E;
identifier f;
@@

f(...)
{
+ void *ptmp;
<+...
- x = SCRealloc(x, E);
+ ptmp = SCRealloc(x, E);
... when != x
- if (x == NULL)
+ if (ptmp == NULL)
{
+ SCFree(x);
+ x = NULL;
...
- }
+ } else {
+     x = ptmp;
+ }
...+>
}

@@
expression x, E;
identifier f;
statement ES;
@@

f(...) {
+ void *ptmp;

<+...
- x = SCRealloc(x, E);
+ ptmp = SCRealloc(x, E);
... when != x
- if (x == NULL) ES
+ if (ptmp == NULL) {
+ SCFree(x);
+ x = NULL;
+ ES
+ } else {
+     x = ptmp;
+ }
...+>

}

@@
expression x, E;
identifier f;
@@

f(...)
{
+ void *ptmp;
<+...
- x = SCRealloc(x, E);
+ ptmp = SCRealloc(x, E);
... when != x
- if (unlikely(x == NULL))
+ if (unlikely(ptmp == NULL))
{
+ SCFree(x);
+ x = NULL;
...
- }
+ } else {
+     x = ptmp;
+ }
...+>
}

@@
expression x, E;
identifier f;
statement ES;
@@

f(...) {
+ void *ptmp;

<+...
- x = SCRealloc(x, E);
+ ptmp = SCRealloc(x, E);
... when != x
- if (unlikely(x == NULL)) ES
+ if (unlikely(ptmp == NULL)) {
+ SCFree(x);
+ x = NULL;
+ ES
+ } else {
+     x = ptmp;
+ }
...+>

}
12 years ago
Victor Julien ecd5c7573b Coverity 1038135 fix
Small cleanup in the error handling. The extra null check confused
Coverity.
12 years ago
Ken Steele 784843b146 Use Tilera SIMD for Signature matching ala SSE3
Makes use of 8-wide byte compare instructions in signature matching.

For allocating aligned memory, _mm_malloc() is SSE only, so added
check for __tile__ to use memalign() instead.

Shows a 13% speed up.
12 years ago
Victor Julien d0c1410cf5 Fix sig grouping bug when certain sigs are mixed. Add tests. 13 years ago
Eric Leblond e176be6fcc Use unlikely for error treatment.
When handling error case on SCMallog, SCCalloc or SCStrdup
we are in an unlikely case. This patch adds the unlikely()
expression to indicate this to gcc.

This patch has been obtained via coccinelle. The transformation
is the following:

@istested@
identifier x;
statement S1;
identifier func =~ "(SCMalloc|SCStrdup|SCCalloc)";
@@

x = func(...)
... when != x
- if (x == NULL) S1
+ if (unlikely(x == NULL)) S1
13 years ago
Victor Julien 8f71333e12 file: implement filesize keyword. #489. 13 years ago
pi-rho 0df4c5838d spelling corrections documented in redmine bug#533 13 years ago
Victor Julien c9e93ec52c filemd5: add support code for md5 handling for signatures. 14 years ago
Victor Julien cdba2f50d1 Various fixes and improvements based on feedback by Coverity analyzer. 14 years ago
Anoop Saldanha 4810ee9c5f All uricontent modified patterns now are DETECT_CONTENT and not DETECT_URICONTENT. Step towards unifying all content based patterns. Makes way for easier management of patterns 14 years ago
Victor Julien d5ed28b065 Remove SIG_FLAG_MPM flag. 14 years ago
Victor Julien 4992f7c417 Remove SIG_FLAG_MPM_URI flag. It was checked but never set. 14 years ago
Victor Julien 9b62ec65ab Make sure filemagic works properly regardless of filestore being in use for a flow. 14 years ago
Victor Julien 23e01d23d3 Implement filestore keyword, including a way for the stateful detection engine to conclude that a file will never have to be stored. 14 years ago
Anoop Saldanha 17f3f36d38 packet keywords only added for packet mpm. Rest in stream mpm. Update detection engine to handle the same 14 years ago
Anoop Saldanha ed3b44b3b5 fix parsing content keywords. We are more strict now. All content keywords need to be enclosed in double quotes. Better validation for sid, priority and rev keywords 14 years ago
Victor Julien 4b52823ab6 Use 64 bit mask on 64-bit systems. 15 years ago
Victor Julien e5b6c0f518 Check 32 masks per run instead of 16 in the SIMD code. 15 years ago
Victor Julien 2dbfdd40af Clean up new SIMD mask checking code, improve non-SIMD checks. 15 years ago
Victor Julien b421019cef Match packet mask against 16 signature masks at once using SIMD instructions for SSE3 and up. 15 years ago
Gerardo Iglesias Galvan 91c001f93b Fix potential crash in initialization cleanup code 15 years ago