Commit Graph

5800 Commits (e2c557cfdb48ac13ed535ee672d49d58b5a772c5)
 

Author SHA1 Message Date
Victor Julien 18f0351d9b detect-state: fix profiling 10 years ago
Victor Julien 0b262cbfda dns: optimize tx list walks
The detection engine and log engines can walk the tx list indirectly,
by looping AppLayerParserGetTx. This would lead to new list walks in
the DNS tx list though. Leading to bad performance.

This patch stores the last returned tx and uses that to determine if
the next tx is what we need next. If so, we can return that w/o list
walk.
10 years ago
Victor Julien 0704ece4d7 detect-reload: enable unconditionally
Reloading is available unconditionally now.
10 years ago
Victor Julien 724c7044e1 detect-reload: 0 detect threads is no error
The reload code would consider 0 detect threads to be an error,
but it's not in case of unix socket mode.
10 years ago
Victor Julien 7c9e015748 unix-socket: implement reload-rules
Implement the reload-rules unix socket command. The unix command
thread signals the main thread to do the reload and it waits for
it to complete.
10 years ago
Victor Julien 71d01f06b9 detect reload: load config
Load the YAML into a prefix "detect-engine-reloads.N" where N is the
reload counter. This way we can load the updated config w/o overwriting
the current one.
10 years ago
Victor Julien b51075e804 detect: remove config at prefix
Remove config at prefix when freeing a detect engine.
10 years ago
Victor Julien 85e12f2bc6 rule vars: support prefix
Support the detection engine's prefix when retrieving rule vars.
10 years ago
Victor Julien 0bc27c7dc7 rule-vars: take detect engine as arg 10 years ago
Victor Julien 3083f51cc6 detect:pass DetectEngineCtx to port parsing
Preparation for prefix handling in port parsing.
10 years ago
Victor Julien 2be9ccfe9c detect: pass DetectEngineCtx to address parsing
Preparation for prefix handling in address parsing.
10 years ago
Victor Julien 7108085d33 detect: initialize detection engine by prefix
Initalize detection engine by configuration prefix.

    DetectEngineCtxInitWithPrefix(const char *prefix)

Takes the detection engine configuration from:
<prefix>.<config>

If prefix is NULL the regular config will be used.

Update sure that DetectLoadCompleteSigPath considers the prefix when
retrieving the configuration.
10 years ago
Victor Julien 97d77e3540 conf: add ConfYamlLoadFileWithPrefix
Add function to load a yaml file and insert it into the conf tree at
a specific prefix.

Example YAML:
somefile: myfile.txt

If loaded using ConfYamlLoadFileWithPrefix with prefix "myprefix", it
can be retrieved by the name of "myprefix.somefile".
10 years ago
Victor Julien a80cc696d7 detect: allow det_ctx->de_ctx to be NULL
When freeing det_ctx, allow de_ctx to be NULL.
10 years ago
Victor Julien c9a8262ccf detect: reload thread init cleanup
Rename the thread init function DetectEngineThreadCtxInitForLiveRuleSwap
to DetectEngineThreadCtxInitForReload and change it's logic to take the
new detection engine as argument and let it return the
DetectEngineThreadCtx or NULL on error.

The old approach used the thread init API format, but it wasn't used in
that way.
10 years ago
Victor Julien 55e7370fc5 detect reload: allow master update during reload
Add DetectEngineReference, which takes a reference to a detect engine,
and make DetectEngineThreadCtxInitForLiveRuleSwap use it. This way
reload will not depend on master staying the same. This allows master
to be updated in between w/o affecting the reload that is in progress.
10 years ago
Victor Julien b1c54a8673 detect: remove old live reload implementation
Remove code that ran the reload in it's own thread. Simplify the
signal handling.
10 years ago
Victor Julien 0c263105cd detect: move reload into main loop
Use new DetectEngineReload() function. It's called from the main loop
instead of it being spawned into it's own temporary thread. This greatly
simplifies the signal handling.

An added advantage is that this seems to improve the memory usage.

Related to bug #1358
10 years ago
Victor Julien e7882da178 detect: introduce 'minimal' detect engine
The minimal detect engine has only the minimal memory use and setup
time. It's to be used for 'delayed' detect where the first detection
engine is essentially empty.

The threads setup are also minimal.
10 years ago
Victor Julien f4617d5357 threading: remove 'dummy' slot logic
Now that delayed detect is not using it anymore, the dummy logic
can be removed.
10 years ago
Victor Julien b0cb4c17ec detect: unify delayed detect and reload
Instead of threading logic with dummy slots and all, use the regular
reload logic for delayed detect.

This means we pass a empty detect engine to the threads and then
reload (live swap) it as soon as the engine is running.
10 years ago
Victor Julien 38b349af1e runmodes: remove DetectEngineCtx passing from API
No longer pass a pointer to the current detection engine to the
runmode API calls.

Note: breaks delayed detect. Will be fixed in a future commit.
10 years ago
Victor Julien b038b6a2f8 unittests: add exception to detect engine setup
Add code to allow for unittests not following the complete api.

Update replace tests as they don't use the unittests runmode that
powers the workaround based on RunmodeIsUnittests().
10 years ago
Victor Julien d66fa1add1 detect: update detect engine management
Update detect engine management to make it easier to reload the detect
engine.

Core of the new approach is a 'master' ctx, that keeps a list of one or
more detect engines. The detect engines will not be passed to any thread
directly, but instead will only be accessed through the detect engine
thread contexts. As we can replace those atomically, replacing a detect
engine becomes easier.

Each thread keeps a reference to its detect context. When a detect engine
is replaced or removed, it's added to a free list. Once its reference
count reaches 0, it is freed.
10 years ago
Victor Julien 664100c074 suricatasc: allow for much longer response times 10 years ago
Victor Julien 092ddc1853 detect: no exit on reference/classification errors
Don't exit on errors during classification and reference parsing.

Add some suppression of error messages when in unittest mode.
10 years ago
Victor Julien 49bad2cfba detect: consolidate more setup into DetectEngineCtxInit
Loading of classifications, references and action order was done
unconditionally, so can be done in one place.
10 years ago
Victor Julien 69f99245c5 unix-command: fix duplicate registration error msg 10 years ago
Victor Julien 60a49657df Bug 1401: on midstream pickup, fix packet direction
On midstream SYN/ACK pickups, we would flip the direction of packets
after the first. This meant the first (pickup) packet's direction
was wrong.

This patch fixes that.
10 years ago
Jason Ish ee7e813256 Bug 1417 - Record sequence nodes as sequences.
Nodes that are sequences weren't being recorded as such, causing
rules to fail to load.

Change sequence test name to reflect better what it tests, and
test that the sequence node is detected as a sequence.
10 years ago
Victor Julien e250040b72 detect-state: implement tx state reset for reload
In case of Detect Reload, we need to reset active tx' state.
10 years ago
Victor Julien da3e8ad8f6 detect-state: split flow and tx state
Use separate data structures for storing TX and FLOW (AMATCH) detect
state.

- move state storing into util funcs
- remove de_state_m
- simplify reset state logic on reload
10 years ago
Victor Julien 840efe17fe modbus: tx de_state 10 years ago
Victor Julien e984a57203 smtp: tx de_state 10 years ago
Victor Julien 2a23627a82 dns: implement tx de_state 10 years ago
Victor Julien 774bb90351 http: clean up tx destate at tx destroy 10 years ago
Victor Julien 6279da0fbd http: support per TX destate storage 10 years ago
Victor Julien 1cf02560c8 app-layer: per tx destate
Add API calls for storing detection state in the TX.
10 years ago
Victor Julien 866d9684ea detect-state: fix profiling 10 years ago
Victor Julien 7e75279977 detect-state: various cleanups 10 years ago
Victor Julien eec22ce19b detect-state: rip per sig detect out of ContinueDetect 10 years ago
Victor Julien bf818b8fb2 detect-state: remove DeStateResetFileInspection
It was effectively unused.
10 years ago
Victor Julien 206f9d4010 detect-state: remove redundant code 10 years ago
Victor Julien e390e24a7c detect-state: add helper to test state
Add little helper function StateIsValid() to test if the state
can be inspected safely.

Cleans up stateful detection loops.
10 years ago
Victor Julien 072ae12771 detect-state: add helper to indicate last tx
Add little helper to indicate current tx is that last we have.
10 years ago
Victor Julien b710f2dd59 detect-state: cleanup ContinueDetection
Only lock f->de_state->m when we start to access it. So after
declaration and initialization of local vars.
10 years ago
Victor Julien 54cb2b6877 detect-state: cleanup retvals
Use DETECT_ENGINE_INSPECT_SIG_* instead of 0, 1, 2 and 3.
10 years ago
Victor Julien 97cab030d7 modbus: shrink data structure 10 years ago
Victor Julien 9f1b417660 file: don't 'close' file if we need to track it 10 years ago
Victor Julien 6723d03c7e http: add inspection engine for http request line
No MPM though.
10 years ago