This commit removes the `_AL_` usage in detect keywords for improved
readability.
Some of the HTTP rule keywords already had counterparts without using
"_AL_". These rule keywords are the legacy content modifier keywords
that now have sticky buffer equivalents.
For these, "_AL_" was removed and a suffix was added to the #define:
src/detect-engine-register.h:151: DETECT_HTTP_COOKIE_CM
src/detect-engine-register.h:153: DETECT_HTTP_METHOD_CM
src/detect-engine-register.h:161: DETECT_HTTP_HEADER_CM
src/detect-engine-register.h:173: DETECT_HTTP_RAW_HEADER_CM
src/detect-engine-register.h:175: DETECT_HTTP_URI_CM
src/detect-engine-register.h:179: DETECT_HTTP_STAT_MSG_CM
src/detect-engine-register.h:181: DETECT_HTTP_STAT_CODE_CM
src/detect-engine-register.h:185: DETECT_HTTP_HOST_CM
Ticket: #6104
And failures should be handled to say that the rule failed to load
Reverts the fix by 299ee6ed55
that was simple, but not complete (memory leak),
to have this bigger API change which simplifies code.
Removes the use of PCRE for performance
Forbids empty negations after a valid pattern
Forbids mixing negative and positive forms as it is irrelevant
Forbids useless repetition of a version
Set flags by default:
-Wmissing-prototypes
-Wmissing-declarations
-Wstrict-prototypes
-Wwrite-strings
-Wcast-align
-Wbad-function-cast
-Wformat-security
-Wno-format-nonliteral
-Wmissing-format-attribute
-funsigned-char
Fix minor compiler warnings for these new flags on gcc and clang.
Remove the 'StreamMsg' approach from the engine. In this approach the
stream engine would create a list of chunks for inspection by the
detection engine. There were several issues:
1. the messages had a fixed size, so blocks of data bigger than ~4k
would be cut into multiple messages
2. it lead to lots of data copying and unnecessary memory use
3. the StreamMsgs used a central pool
The Stream engine switched over to the streaming buffer API, which
means that the reassembled data is always available. This made the
StreamMsg approach even clunkier.
The new approach exposes the streaming buffer data to the detection
engine. It has to pay attention to an important issue though: packet
loss. The data may have gaps. The streaming buffer API tracks the
blocks of continuous data.
To access the data for inspection a callback approach is used. The
'StreamReassembleRaw' function is called with a callback and data.
This way it runs the MPM and individual rule inspection code. At
the end of each detection run the stream engine is notified that it
can move forward it's 'progress'.
To be able to add a transaction counter we will need a ThreadVars
in the AppLayerParserParse function.
This function is massively used in unittests
and this result in an long commit.
This is an intrusive change. This patch modifies the way AMATCH
inspection uses locking.
So far, each keyword did it's own locking. This lead to a situation
where a 'alstate' pointer was passed around that was not always
protected by a lock.
This patch moves the locking to the Stateful detection functions.
app-layer.[ch], app-layer-detect-proto.[ch] and app-layer-parser.[ch].
Things addressed in this commit:
- Brings out a proper separation between protocol detection phase and the
parser phase.
- The dns app layer now is registered such that we don't use "dnstcp" and
"dnsudp" in the rules. A user who previously wrote a rule like this -
"alert dnstcp....." or
"alert dnsudp....."
would now have to use,
alert dns (ipproto:tcp;) or
alert udp (app-layer-protocol:dns;) or
alert ip (ipproto:udp; app-layer-protocol:dns;)
The same rules extend to other another such protocol, dcerpc.
- The app layer parser api now takes in the ipproto while registering
callbacks.
- The app inspection/detection engine also takes an ipproto.
- All app layer parser functions now take direction as STREAM_TOSERVER or
STREAM_TOCLIENT, as opposed to 0 or 1, which was taken by some of the
functions.
- FlowInitialize() and FlowRecycle() now resets proto to 0. This is
needed by unittests, which would try to clean the flow, and that would
call the api, AppLayerParserCleanupParserState(), which would try to
clean the app state, but the app layer now needs an ipproto to figure
out which api to internally call to clean the state, and if the ipproto
is 0, it would return without trying to clean the state.
- A lot of unittests are now updated where if they are using a flow and
they need to use the app layer, we would set a flow ipproto.
- The "app-layer" section in the yaml conf has also been updated as well.
In debug validation mode, it is required to call application layer
parsing and other functions with a lock on flow. This patch updates
the code to do so.
When handling error case on SCMallog, SCCalloc or SCStrdup
we are in an unlikely case. This patch adds the unlikely()
expression to indicate this to gcc.
This patch has been obtained via coccinelle. The transformation
is the following:
@istested@
identifier x;
statement S1;
identifier func =~ "(SCMalloc|SCStrdup|SCCalloc)";
@@
x = func(...)
... when != x
- if (x == NULL) S1
+ if (unlikely(x == NULL)) S1
Philippe Antoine
Jeff Lucovsky
Victor Julien
Philippe Antoine
Victor Julien
Jeff Lucovsky
Shivani Bhardwaj
Mats Klepsland
Giuseppe Longo
Eric Leblond
Jason Ish
Ken Steele
Anoop Saldanha
Anoop Saldanha