Commit Graph

6893 Commits (suricata-3.1.1)
 

Author SHA1 Message Date
Victor Julien a34be23002 detect: simplify negated mpm handling 10 years ago
Victor Julien b84d6d402f detect grouping: multiple whitelist conditions
Instead of the binary yes/no whitelisting used so far, use different
values for different sorts of whitelist reasons. The port list will
be sorted by whitelist value first, then by rule count.

The goal is to whitelist groups that have weak sigs:

 - 1 byte pattern groups

 - SYN sigs

    Rules that check for SYN packets are mostly scan detection rules.
    They will be checked often as SYN packets are very common.

    e.g. alert tcp any any -> any 22 (flags:S,12; sid:123;)

    This patch adds whitelisting for SYN-sigs, so that the sigs end up
    in as unique groups as possible.

 - negated mpm sigs

    Currently negated mpm sigs are inspected often, so they are quite
    expensive. For this reason, try to whitelist them.

These values are set during 'stage 1', rule preprocessing.
10 years ago
Victor Julien 3c184c19cd detect grouping: port based group whitelisting
Whitelist some ports in grouping to make sure they get their own group.
10 years ago
Victor Julien 5772f526dc detect grouping: warn on and fix up bad sigs
Only inspect directionless SYN scan sigs toserver. Issue a warning for
those rules.
10 years ago
Victor Julien 2ce03fbabb detect: split non-mpm list into syn/nosyn
Since SYN inspecting rules are expensive, this patch splits the
'non-mpm' list (i.e. the rules that are always considered) into
a 'syn' and 'non-syn' list. The SYN list is only inspected if the
packet has the SYN flag set, otherwise the non-syn list is used.

The syn-list contains _all_ rules. The non-syn list contains all
minus the rules requiring the SYN bit in a packet.
10 years ago
Victor Julien e48d745ed7 mpm: constify search func args 10 years ago
Victor Julien 26517b8b61 detect: mpm store frees mpm_ctx' it owns 10 years ago
Victor Julien 102a82fc7b detect: use mpm store for app layer mpms
Rework app-layer mpm setup and registration to make this possible.
10 years ago
Victor Julien fac2cc0560 detect: mpm deduplication
Create hash for mpm's that we can reuse. Have packet/stream mpms
use this.
10 years ago
Victor Julien f0ba00e51d detect: remove old unused code 10 years ago
Victor Julien 10b049304f detect: set new defaults for grouping 10 years ago
Victor Julien 2ee9bf2aef detect: rename groupings vars 10 years ago
Victor Julien 1f70ccfc23 detect: remove unused grouping settings 10 years ago
Victor Julien ae80ed5964 detect: make port grouping use config limits 10 years ago
Victor Julien df529b13ce detect: change port grouping
Update port grouping logic. Previously it would create one consistent
list w/o overlap. It largely still does this, except for the 'catch
all' port group at the end of the list. This port group contains all
the sigs that didn't fit into the other groups.
10 years ago
Victor Julien a392812302 detect: sort/group port sigs 10 years ago
Victor Julien e570b10abe detect: display unique sgh count 10 years ago
Victor Julien eda9552e95 detect: group proto sghs 10 years ago
Victor Julien d82df4eb8b detect-mpm: make sgh setup proto aware
Allow multi-proto, multi-direction sgh's.
10 years ago
Victor Julien 4223ce9aba detect: remove obsolete grouping code 10 years ago
Victor Julien 9ae4cb9e02 detect: debug output 10 years ago
Victor Julien fd5a06017d detect: per port and proto rule grouping
Replace tree based approach for rule grouping with a per port (tcp/udp)
and per protocol approach.

Grouping now looks like:

               +----+
               |icmp+--->
               +----+
               |gre +--->
               +----+
               |esp +--->
               +----+
          other|... |
        +----->-----+
        |      |N   +--->
        |      +----+
        |
        | tcp  +----+   +----+
        +----->+ 80 +-->+ 139+-->
        |      +----+   +----+
        |
        | udp  +----+   +----+
    +---+----->+ 53 +-->+ 135+-->
    |          +----+   +----+
    |toserver
+--->
    |toclient
    |
    +--->

So the first 'split' in the rules is the direction: toserver or toclient.
Rules that don't have a direction, are in both branches.

Then the split is between tcp/udp and the other protocols. For tcp and
udp port lists are used. For the other protocols, grouping is simply per
protocol.

The ports used are the destination ports for toserver sigs and source
ports for toclient sigs.
10 years ago
Victor Julien 27e63a1e11 detect: track direction and ipproto of sgh
Each SGH has a unique ipproto and direction.
10 years ago
Victor Julien e75a93b125 detect: pass ipproto to rule grouping funcs 10 years ago
Victor Julien c71c991669 detect: delay sgh cleanup 10 years ago
Victor Julien bb662a65f8 detect: delay sgh mpm setup 10 years ago
Victor Julien 18dd54dfa7 Start rule inspect with mask check 10 years ago
Victor Julien bfe49b60f7 rule analyzer: add no/both direction warning 10 years ago
Victor Julien b7d81fc3b0 detect: SYN flags
Add funcs to see if a rule needs a SYN flag in the packet.
10 years ago
Victor Julien f720dfd21e detect: validate http_method pattern
Leading and trailing spaces and tabs are invalid as these are not part
of the buffer as returned by libhtp.
10 years ago
Victor Julien 66b3dba676 detect: remove dead code 10 years ago
Victor Julien b3dcdb10be detect mpm: remove dead code 10 years ago
Victor Julien 14d9ce7b2e detect/mpm: remove unused max_id param from API 10 years ago
Victor Julien 0d3f671b55 detect: constify mpm/detect funcs 10 years ago
Victor Julien 4f8e1f59a6 mpm: remove obsolete mpm algos
Remove: ac-gfbs, wumanber, b2g, b3g.
10 years ago
Victor Julien 262abbb49f mpm: fix ac-ks compilation on cygwin 10 years ago
Victor Julien 58e533858b detect mpm: ac-tile/ac-ks default to single
Use sgh-mpm-context single is it is set to 'auto' when ac-ks is used.
10 years ago
Victor Julien 0987fd16dc ac-ks: 32bit fixes 10 years ago
Victor Julien 9c2e374a3d ac-ks: fix mem leaks 10 years ago
Victor Julien 887ddf1ed8 mpm: introduce ac-ks
Introduce 'ac-ks' or the Kenneth Steele AC implementation. It's
actually 'ac-tile' written by Ken for the Tilera platform. This
patch adds support for it on other architectures as well.

Enable ac-tile for other archs as 'ac-ks'.

Fix a bunch of OOB reads in the loops that triggered ASAN.
10 years ago
Victor Julien 3781b00dbc Open Suricata 3.1 development branch 10 years ago
Victor Julien 71a3c4caac Update Changelog for 3.0.1 10 years ago
Mats Klepsland f9ac42b36f util-decode-der: fix NULL dereference bug
Make sure that the length is not longer than the size of the buffer
provided.
10 years ago
Mats Klepsland c0dd911591 util-decode-der: fix heap-buffer-overflow 10 years ago
Mats Klepsland d07c495ed1 app-layer-tls-handshake: fix heap-buffer overflow
Fix heap-buffer overflow that occurs when we are given repeatedly
certificates with the length of zero.
10 years ago
Victor Julien ba035e601e hyperscan: fix minor coverity warning 1358023
*** CID 1358023:  Null pointer dereferences  (REVERSE_INULL)
/src/util-mpm-hs.c: 860 in SCHSDestroyThreadCtx()
854         if (thr_ctx->scratch != NULL) {
855             hs_free_scratch(thr_ctx->scratch);
856             mpm_thread_ctx->memory_cnt--;
857             mpm_thread_ctx->memory_size -= thr_ctx->scratch_size;
858         }
859
>>>     CID 1358023:  Null pointer dereferences  (REVERSE_INULL)
>>>     Null-checking "mpm_thread_ctx->ctx" suggests that it may be null, but it has already been dereferenced on all paths leading to the check.
860         if (mpm_thread_ctx->ctx != NULL) {
861             SCFree(mpm_thread_ctx->ctx);
862             mpm_thread_ctx->ctx = NULL;
863             mpm_thread_ctx->memory_cnt--;
864             mpm_thread_ctx->memory_size -= sizeof(SCHSThreadCtx);
865         }
10 years ago
Victor Julien 2ccf028eac hyperscan: fix minor coverity warning 1358024
*** CID 1358024:  Null pointer dereferences  (REVERSE_INULL)
/src/util-mpm-hs.c: 1043 in SCHSPrintInfo()
1037         printf("  SCHSPattern    %" PRIuMAX "\n", (uintmax_t)sizeof(SCHSPattern));
1038         printf("Unique Patterns: %" PRIu32 "\n", mpm_ctx->pattern_cnt);
1039         printf("Smallest:        %" PRIu32 "\n", mpm_ctx->minlen);
1040         printf("Largest:         %" PRIu32 "\n", mpm_ctx->maxlen);
1041         printf("\n");
1042
>>>     CID 1358024:  Null pointer dereferences  (REVERSE_INULL)
>>>     Null-checking "ctx" suggests that it may be null, but it has already been dereferenced on all paths leading to the check.
1043         if (ctx) {
1044             char *db_info = NULL;
1045             if (hs_database_info(pd->hs_db, &db_info) == HS_SUCCESS) {
1046                 printf("HS Database Info: %s\n", db_info);
1047                 SCFree(db_info);
1048             }
10 years ago
browner87 070a35fd3b source-pcap-file: fix stats not being reset
Fix for redmine #1737. Clears stats between each scanned PCAP instead of just 
on load in Unix socket mode.
10 years ago
browner87 d05883af9b Fix for redmine bug 1737
PCAP stats not reset between files in Unix socket mode. Added a memset to the Global Init function to clear these stats.
10 years ago
Jason Ish 06dcaec335 byte-extract: for string, default to base "dec" if not provided
For Snort compatibility, as Snort defaults to "dec" if not provided.

Fixed issue 1697:
https://redmine.openinfosecfoundation.org/issues/1697
10 years ago