Commit Graph

90 Commits (3a092f30278b2c6c86c4a0c0f3bca7f77d5922c8)

Author SHA1 Message Date
Philippe Antoine 3a092f3027 detect: allow rule which need both directions to match
Ticket: 5665

This is done with `alert ip any any => any any`
The => operator means that we will need both directions
8 months ago
Philippe Antoine 4ec90bd227 detect: absent keyword to test absence of sticky buffer
Ticket: 2224

It takes an argument to match only if the buffer is absent,
or it can still match if the buffer is present, but we test
the absence of some content.

For multi buffers, absent matches if there are 0 buffers.

For file keywords, absent matches if there is no file.
12 months ago
Philippe Antoine 0d267e29a5 files: remove the need for state in callbacks
As files now belong to transactions
1 year ago
Philippe Antoine 20423fdd38 style: remove some useless return
and remove empty line before end of function
1 year ago
Victor Julien b1fa975503 detect/content-inspect: remove const casting 2 years ago
Victor Julien 7f42506760 detect: reimplement discontinue matching logic
Previously various steps in the content inspection logic would use
a variable in the DetectEngineThreadCtx to flag that matching should
be discontinued.

This patch reimplements this logic by using a new return code instead.

Split content inspection into public and private version, so that
common initialization can be done in a single place.

Update the callsites.
2 years ago
Victor Julien 68a2fcaad3 mpm: thread ctx cleanups
Remove unused thread ctx' from AC variants

Use single thread store in detection.

Minor cleanups.
2 years ago
Victor Julien c5d83d081e detect/file: use util to turn keyword to nocase
This changes the way the pattern is stored by making it lowercase.
2 years ago
Jeff Lucovsky f735e309e2 detect/file_data: Consolidate file handling
Issue: 4145

Consolidate file handling for all protocols that use file objects for
file_data.

Make sure http_server_body / http.response_body for HTTP1 continue
to inspect the actual body. For HTTP2, http.response_body acts as
an internal alias for `file_data`.
2 years ago
Victor Julien 2cac440f7d detect/filemagic: fix thread ctx registration; reloads
Make sure thread ctx registration happens and id remains correct
in case of reloads.

To do so, move id var into the detect ctx.
2 years ago
Victor Julien 1f22b51848 detect/filemagic: switch to file.magic implementation
Replace implementation of the legacy `filemagic` keyword by the
implementation for the `file.magic` variant. This leads to better
performance and hooks the rules into the detection engine better.

Remove unittests that depended on the old logic.

Ticket: #6194.
2 years ago
Philippe Antoine e75956717d detect/files: centralize definition of protocols
Protocols supporting files are only defined in one place, which
gets used by all keywords, which can handle some exceptions
(like HTTP2 not having file names)
2 years ago
Victor Julien da89164ba1 detect: spelling: DetectBufferMpmRegistry 3 years ago
Victor Julien 5890a8a8ab detect: register multi-buffer keywords
Register the keywords that use multi buffer support as such, so that
rule parsing can set them up with multi-instance support.

Ticket: #5784.
3 years ago
Victor Julien 2c0e434bb8 detect: pass de_ctx to DetectBufferSetActiveList 3 years ago
Victor Julien 977c5ea719 detect: initialize empty buffers 3 years ago
Victor Julien 576bfc6bf0 detect/file.magic: update buffer initialization logic 3 years ago
Victor Julien 71bc9e75f5 app-layer: get sbconfg with files 3 years ago
Victor Julien b31ffde6f4 output: remove error codes from output 3 years ago
Victor Julien e042cd785e error: use SC_ENOMEM for alloc errors 3 years ago
Alice Akaki 8614bff017 detect-filemagic: convert unittests to FAIL/PASS APIs
Task: #4034
3 years ago
Victor Julien 41265a859e detect/files: optimize file.data by skipping non-file txs
As well as 'file' txs not in our direction.

Implement the same logic for file.name and file.magic prefilter engines.
3 years ago
Victor Julien 3263202094 detect/tx: add AppLayerTxData to PrefilterTx
In preparation of some file inspection optimizations, for which we need the
tx data.

Update all users.
3 years ago
Victor Julien 79499e4769 app-layer: move files into transactions
Update APIs to store files in transactions instead of the per flow state.

Goal is to avoid the overhead of matching up files and transactions in
cases where there are many of both.

Update all protocol implementations to support this.

Update file logging logic to account for having files in transactions. Instead
of it acting separately on file containers, it is now tied into the
transaction logging.

Update the filestore keyword to consider a match if filestore output not
enabled.
3 years ago
Victor Julien 040404b093 detect/profiling: track bytes scanned by prefilter engines 3 years ago
Victor Julien e250ef6402 debug: remove empty header 3 years ago
Victor Julien af145ad125 detect/file: reduce scope of keyword data structures 3 years ago
Victor Julien 73eb7744d8 detect/file: update copyright years 3 years ago
Philippe Antoine 02f2602dde src: rework includes as per cppclean 3 years ago
Philippe Antoine b8524e70d9 detect: change InspectEngineFuncPtr2 to return uint8_t 3 years ago
Victor Julien 4feb0529a4 detect/file: minor code cleanup
Reduce scope where possible. Suggested by cppcheck.
4 years ago
Victor Julien 47629b7aeb detect/filemagic: don't pass unused pointer 4 years ago
Victor Julien d8d1fbe443 detect/files: fix buffer tracking with multiple files 4 years ago
Victor Julien 3dc50322db detect: fix multi inspect buffer issue; clean up
Fix multi inspect buffer API causing cleanup logic in the single
inspect buffer paths. This could lead to a buffer overrun in the
"to clear" logic.

Multi buffers now use InspectionBufferSetupMulti instead of
InspectionBuffer. This is enforced by a check in debug validation.

Simplify the multi inspect buffer setup code and update the callers.
4 years ago
Philippe Antoine 707f027231 protos: renaming ALPROTO_HTTP* constants
Having now ALPROTO_HTTP1, ALPROTO_HTTP2 and ALPROTO_HTTP

Run with 3 sed commands
git grep ALPROTO_HTTP | cut -d: -f1 | uniq |
 xargs sed -i -e 's/ALPROTO_HTTP/ALPROTO_HTTP1/g'
git grep ALPROTO_HTTP12 | cut -d: -f1 | uniq |
 xargs sed -i -e 's/ALPROTO_HTTP12/ALPROTO_HTTP2/g'
git grep ALPROTO_HTTP1_ANY | cut -d: -f1 | uniq |
 xargs sed -i -e 's/ALPROTO_HTTP1_ANY/ALPROTO_HTTP/g'

and then running clang-format
5 years ago
Victor Julien 13cebb1857 detect: fix heap overflow issue with buffer setup
In some cases, the InspectionBufferGet function would be followed by
a failure to set the buffer up, for example due to a HTTP body limit
not yet being reached. Yet each call to InspectionBufferGet would lead
to the matching list_id to be added to the
DetectEngineThreadCtx::inspect.to_clear_queue. This array is sized to
add each list only once, but in this case the same id could be added
multiple times, potentially overflowing the array.
5 years ago
Angelo Mirabella c72069c3b2 detect/magic: fix crash on rule reloading
This changseset fixes a bug causing a segmentation fault.
When rules are reloaded and a rule using libmagic matches, suricata
crashes due to an improper reinitialization of the thread contexts.

Bug: #3726
5 years ago
Philippe Antoine 1422b18a99 http2: initial support 5 years ago
Victor Julien 6ab323d323 detect: hide RegisterTests behind ifdef UNITTESTS
Update all callers to more aggressively use UNITTESTS guards as well.
5 years ago
Eric Leblond ae5650d443 magic: get rid of global lock
Global magic context was involving a lock that appear to be really
costly for some traffic.
6 years ago
Victor Julien 26bcc97515 detect/keywords: dynamic version part of doc URL 6 years ago
Jeff Lucovsky d3a65fe156 detect: Provide `de_ctx` to free functions
This commit makes sure that the `DetectEngineCtx *` is available
to each detector's "free" function.
6 years ago
Victor Julien a4a4d17ad0 app-layer/files: optimize GetFiles calls
Remove FlowGetProtoMapping calls from the GetFiles wrapper and
get the alstate from the flow directly.
6 years ago
Victor Julien 4dff903b35 detect: introduce pkt mpm engines
Instead of the hardcode L4 matching in MPM that was recently introduced,
add an API similar to the AppLayer MPM and inspect engines.

Share part of the registration code with the AppLayer.

Implement for the tcp.hdr and udp.hdr keywords.
6 years ago
Victor Julien 14896365ef detect: remove Threadvars argument from API calls
Remove it as it's (almost) never used. If it is really needed it can
be accessed through DetectEngineThreadCtx::tv as well.
6 years ago
Victor Julien 32fb7d773a detect/content-inspect: turn void arg into Packet
Replace the 'void *data' argument by a 'Packet *p' as this was
the only user left of the data pointer.
7 years ago
Victor Julien 84881bf1b8 detect/file.magic: add sticky buffer
Add sticky buffer to inspect file magic. Includes mpm support.
7 years ago
Eric Leblond 63f2032ed6 doc: fix way to build URL 7 years ago
Victor Julien 164252e381 detect/file: fix minor scan-build warnings 7 years ago
Andreas Herz 2e8678a5ff docs: replace redmine links and enforce https on oisf urls 8 years ago