Commit Graph

8081 Commits (889a6de926c02c96e2d86f3a0cf9d551c4f13f3a)
 

Author SHA1 Message Date
Victor Julien 3ff5dc3653 nfq: remove obsolete and broken netfilterforwin support 8 years ago
Victor Julien ea99099c64 isdataat: add test for leading space 8 years ago
Victor Julien 6142e88ed5 nflog: compiler warning fix 8 years ago
Victor Julien ab1200fbd7 compiler: more strict compiler warnings
Set flags by default:

    -Wmissing-prototypes
    -Wmissing-declarations
    -Wstrict-prototypes
    -Wwrite-strings
    -Wcast-align
    -Wbad-function-cast
    -Wformat-security
    -Wno-format-nonliteral
    -Wmissing-format-attribute
    -funsigned-char

Fix minor compiler warnings for these new flags on gcc and clang.
8 years ago
Victor Julien 342059835f detect-parse: improve common parser
In preparation of turning input to keyword parsers to const add
options to the common rule parser to enforce and strip double
quotes and parse negation support.

At registration, the keyword can register 3 extra flags:

    SIGMATCH_QUOTES_MANDATORY: value to keyword must be quoted

    SIGMATCH_QUOTES_OPTIONAL: value to keyword may be quoted

    SIGMATCH_HANDLE_NEGATION: leading ! is parsed

In all cases leading spaces are removed. If the 'quote' flags are
set, the quotes are removed from the input as well.
8 years ago
Victor Julien 842dfbc3f8 detect: enforce isdataat:!1,relative earlier
The expression 'isdataat:!1,relative' is used to make sure a match
is at the end of a buffer quite often. This patch optimizes this case
for 'content' followed by the expression. It enforces it by setting
and 'ends with' flag on the content and then taking that flag into
account while doing the pattern match.
8 years ago
Victor Julien c0275c2b29 detect: more content inspection tests 8 years ago
Victor Julien 7eda6beade detect: don't rescan when just distance is used
Content inspection optimization: when just distance is used without
within we don't need to search recursively.

E.g. content:"a"; content:"b"; distance:1; will scan the buffer for
'a' and when it finds 'a' it will scan the remainder for 'b'. Until
now, the failure to find 'b' would lead to looking for the next 'a'
and then for 'b' after that. However, we already inspected the
entire buffer for 'b', so we know this will fail.
8 years ago
Victor Julien 84b97ca155 detect: content-inspection tests
Add tests for the content inspection engine that count the number
of steps it takes to eval a rule.
8 years ago
Victor Julien b9579fbe7d detect: avoid needless recursive scanning
Don't recursively inspect a detect list if the recursion
doesn't increase chance of success.
8 years ago
Victor Julien bc7c01ecbc detect: use BIT_U32 macro for content flags 8 years ago
Victor Julien c65a119cc0 debug: suppress notice message 8 years ago
Victor Julien 276125c1ef cleanup: remove unused ringbuffer code 8 years ago
Victor Julien cda6e0291f cleanup: remove libpcap < 1 support 8 years ago
Victor Julien 119115d3b6 configure: remove CentOS5 pkg-config fix 8 years ago
Victor Julien 0516b5d704 cleanup: from AS_VERSION_COMPARE CentOS5 workaround 8 years ago
Victor Julien d31cb083e9 detect: update tests that mix state/stream inspect 8 years ago
Victor Julien eb5857b68a unittests: add/improve helpers for stream/flow 8 years ago
Victor Julien 15dcac92f2 http_header: enable trailer prefilter engines
Now that the mpm engines run only for the proper 'progress'
value, the trailing headers need their own engine registration.
8 years ago
Victor Julien cf7f819888 state: check progress before calling engine
Make sure progress of an inspect engine is available.
8 years ago
Victor Julien 1bbf555318 detect: improve stateful detection
Now that MPM runs when the TX progress is right, stateful detection
operates differently.

Changes:

1. raw stream inspection is now also an inspect engine

   Since this engine doesn't take the transactions into account, it
   could potentially run multiple times on the same data. To avoid
   this, basic result caching is in place.

2. the engines are sorted by progress, but the 'MPM' engine is first
   even if the progress is higher

   If MPM flags a rule to be inspected, the inspect engine for that
   buffer runs first. If this step fails, the rule is no longer
   evaluated. No state is stored.
8 years ago
Victor Julien d1b7a83905 detect: change mask logic
Previously the MPM/Prefilter engines would suggest the same rule
candidates multiple times.

For example, while processing the request body, the http headers
would be inspected by MPM multiple times.

The mask check was one way to quickly decide which rules could be
skipped.

Now that the MPM engines normally return a rule just once, this
mask check no longer makes sense. If the rule meets the ip/port/
direction based conditions, it needs to be evaluated if the MPM
said so. Even if not all conditions are yet true.

WIP disable mask as it no longer makes sense

WIP redo mask match
8 years ago
Victor Julien a0fad6bb7f mpm: run engines as few times as possible
In various scenarios buffers would be checked my MPM more than
once. This was because the buffers would be inspected for a
certain progress value or higher.

For example, for each packet in a file upload, the engine would
not just rerun the 'http client body' MPM on the new data, it
would also rerun the method, uri, headers, cookie, etc MPMs.

This was obviously inefficent, so this patch changes the logic.

The patch only runs the MPM engines when the progress is exactly
the intended progress. If the progress is beyond the desired
value, it is run once. A tracker is added to the app layer API,
where the completed MPMs are tracked.

Implemented for HTTP, TLS and SSH.
8 years ago
Victor Julien d304be5bc3 detect: register progress in inspect engines
Register required progress so we can stop inspecting as soon
as the progress isn't far enough yet.
8 years ago
Victor Julien bc1698cfbe detect-state: don't use casts to uint 8 years ago
Victor Julien 53b21e5ee1 http_uri: unittest cleanup 8 years ago
Victor Julien 8d2f3b46e6 http_header: add another trailer test 8 years ago
Victor Julien 8d18be1fdb http_header (trailer) test cleanup 8 years ago
Victor Julien 1c46af477e ssh: fix test 8 years ago
Victor Julien a744d00f45 ssh: fix banner state setting 8 years ago
Victor Julien e3bd5f371d detect: more detailed state profiling 8 years ago
Victor Julien 6d562f3b5e app-layer: set stream-depth after stream init 8 years ago
Victor Julien 358e41b935 detect: clean up stateful detect 8 years ago
Victor Julien 9f4884a132 stream: reduce scope of new ssn func 8 years ago
Victor Julien 5c31f22e09 autotools: add src/tests to extra dist 8 years ago
Victor Julien 5a210984d5 stream: move inline tests 8 years ago
Victor Julien bea2b2c00c stream: list management cleanups 8 years ago
Victor Julien 34f7cb2b55 stream: debug improvements 8 years ago
Victor Julien aba9cd7d02 stream inspection: add debug counters 8 years ago
Victor Julien 2b433fab53 stream: pack config struct 8 years ago
Victor Julien 606f515fe9 stream: enforce gap earlier in app reassembly 8 years ago
Victor Julien 314516ffe2 stream: don't call app reassembly if disable flag set 8 years ago
Victor Julien 89af036336 stream: app-layer micro optimizations 8 years ago
Victor Julien 2f77302eeb stream: raw reassembly explicit disable raw handling 8 years ago
Victor Julien d6d7f65050 stream: mpm inspect micro optimizations 8 years ago
Victor Julien 7bddd0e168 stream: improve --disable-detection GAP handling 8 years ago
Victor Julien 6fefe70196 stream: remove unused StreamTcpGetStreamSize function 8 years ago
Victor Julien 422095668e stream: optimize session pruning 8 years ago
Victor Julien 79389558ac doc: update for stream changes 8 years ago
Victor Julien a995734b3a yaml: sync with new stream engine 8 years ago