Commit Graph

107 Commits (fda72e6f83410b229467ff6cbda072d261138e07)

Author SHA1 Message Date
Philippe Antoine c99e159341 detect/files: support protocols only over udp
Ticket: 7973

Files were supported on both TCP and UDP. But file detection keywords
such as file.data made signatures loading fail if the signature
was using an app-layer protocol that enabled on udp only, even
if the signatures could run smoothly.
1 month ago
Philippe Antoine b298bce0e7 detect: list-keywords cli shows multi-buffers
Ticket: 7571
2 months ago
Philippe Antoine ad0f6e388e src: remove BUG_ON in packet path
Transforms them into debug validations

Ticket: 1484

Packet path is here considered FlowWorker and all lines
covered by SV tests are here fixed
5 months ago
Philippe Antoine b4095bf683 detect/files: increment local_file_id even if buffer is NULL
Ticket: 7579

Otherwise, we will keep on calling again and again GetDataCallback
with the same local_file_id, and we will always get a NULL
buffer even if the next local_file_id would return a non-NULL buffer.
6 months ago
Philippe Antoine b1ef498a81 rust: bindgen detect-parse.h
Ticket: 7667

Currently no functions are exported.

DetectFile* struct are moved to detect-file-data.h where
they make more sense.

ifndef SURICATA_BINDGEN_H is used for bindgen to exclude
pcre2 related code
6 months ago
Philippe Antoine 8545ef2e56 detect: factorize code for DetectSetupDirection
Ticket: 7665

Instead of each keyword calling DetectSetupDirection, use a
new flag SIGMATCH_SUPPORT_DIR so that DetectSetupDirection gets
called, before parsing the rest of the keyword.

Allows to support filesize keyword in transactional signatures
6 months ago
Philippe Antoine dadf9012fc rust: bindgen detect-engine-buffer.h
Ticket: 7667

And prefix SCDetectBufferSetActiveList to be exported

Allows less use of suricata crate in plugin as we get the functions
prototypes from suricata_sys and they are more correct.
7 months ago
Philippe Antoine 0024de5e9e src: new file detect-engine-buffer.h
Taken from detect-engine.h so that we can bindgen this smaller
unit.
7 months ago
Jeff Lucovsky 22ea5ddbb7 detect/transforms: Add engine detect thread ctx to signature
Modify the transform function signature to include the detect engine
thread ctx.
8 months ago
Philippe Antoine 3a092f3027 detect: allow rule which need both directions to match
Ticket: 5665

This is done with `alert ip any any => any any`
The => operator means that we will need both directions
8 months ago
Philippe Antoine 9a8e377a7a http: include htp-libhtp.h from header
Instead of C files requiring it.
In preparation of libhtp rust, to minimize the final commit.
10 months ago
Philippe Antoine 5ee6e7cef0 http: aliases for htp progresses
In preparation of libhtp rust
10 months ago
Philippe Antoine ae1a4ef757 app-layer: make number of alprotos dynamic
Ticket: 5053

The names are now dynamically registered at runtime.
The AppProto alproto enum identifiers are still static for now.

This is the final step before app-layer plugins.
10 months ago
Philippe Antoine 4ec90bd227 detect: absent keyword to test absence of sticky buffer
Ticket: 2224

It takes an argument to match only if the buffer is absent,
or it can still match if the buffer is present, but we test
the absence of some content.

For multi buffers, absent matches if there are 0 buffers.

For file keywords, absent matches if there is no file.
12 months ago
Philippe Antoine deb4a5a8cc detect/file-data: use dynamic number of app-layer protos 1 year ago
Philippe Antoine dc043d0297 detect: remove unused field
content_inspect_window is used in app-layer-smtp, but
not directly in detect-file-data
1 year ago
Philippe Antoine 0d267e29a5 files: remove the need for state in callbacks
As files now belong to transactions
1 year ago
Victor Julien b1fa975503 detect/content-inspect: remove const casting 2 years ago
Victor Julien 7f42506760 detect: reimplement discontinue matching logic
Previously various steps in the content inspection logic would use
a variable in the DetectEngineThreadCtx to flag that matching should
be discontinued.

This patch reimplements this logic by using a new return code instead.

Split content inspection into public and private version, so that
common initialization can be done in a single place.

Update the callsites.
2 years ago
Victor Julien 368adf4599 detect/file-data: simplify content inspect loop 2 years ago
jason taylor c50002978d doc: update file.data keyword documentation
Signed-off-by: jason taylor <jtfas90@gmail.com>
2 years ago
Victor Julien 68a2fcaad3 mpm: thread ctx cleanups
Remove unused thread ctx' from AC variants

Use single thread store in detection.

Minor cleanups.
2 years ago
Jeff Lucovsky f735e309e2 detect/file_data: Consolidate file handling
Issue: 4145

Consolidate file handling for all protocols that use file objects for
file_data.

Make sure http_server_body / http.response_body for HTTP1 continue
to inspect the actual body. For HTTP2, http.response_body acts as
an internal alias for `file_data`.
2 years ago
Jason Ish 4a28f07298 file-data: better error for conflicting keywords
Instead of just erroring out with "rule contains conflicting
keywords", give an error that says what is actually wrong.
2 years ago
Philippe Antoine e75956717d detect/files: centralize definition of protocols
Protocols supporting files are only defined in one place, which
gets used by all keywords, which can handle some exceptions
(like HTTP2 not having file names)
2 years ago
Philippe Antoine 71bab65496 detect/files: reuse AppLayerParserSupportsFiles
rather than relisting the protocols
2 years ago
Philippe Antoine 656cddcf65 detect: http_client_body for HTTP2
By using the file.data logic

Ticket: #4067
3 years ago
Victor Julien da89164ba1 detect: spelling: DetectBufferMpmRegistry 3 years ago
Victor Julien 5890a8a8ab detect: register multi-buffer keywords
Register the keywords that use multi buffer support as such, so that
rule parsing can set them up with multi-instance support.

Ticket: #5784.
3 years ago
Victor Julien 2c0e434bb8 detect: pass de_ctx to DetectBufferSetActiveList 3 years ago
Victor Julien 977c5ea719 detect: initialize empty buffers 3 years ago
Victor Julien 50fd691efb detect/file.data: update buffer initialization logic 3 years ago
Victor Julien 71bc9e75f5 app-layer: get sbconfg with files 3 years ago
Victor Julien b31ffde6f4 output: remove error codes from output 3 years ago
Victor Julien e601ebdfd8 files: always initialize inspect_window and min_inspect_depth
This is to make sure the files buffers are properly managed even
when there are no rules or when there are no file.data rules.

Bug: #5703.
3 years ago
Eric Leblond ea85e2ccf2 detect/filename: fix buffer description 3 years ago
Victor Julien 41265a859e detect/files: optimize file.data by skipping non-file txs
As well as 'file' txs not in our direction.

Implement the same logic for file.name and file.magic prefilter engines.
3 years ago
Victor Julien 3263202094 detect/tx: add AppLayerTxData to PrefilterTx
In preparation of some file inspection optimizations, for which we need the
tx data.

Update all users.
3 years ago
Victor Julien 79499e4769 app-layer: move files into transactions
Update APIs to store files in transactions instead of the per flow state.

Goal is to avoid the overhead of matching up files and transactions in
cases where there are many of both.

Update all protocol implementations to support this.

Update file logging logic to account for having files in transactions. Instead
of it acting separately on file containers, it is now tied into the
transaction logging.

Update the filestore keyword to consider a match if filestore output not
enabled.
3 years ago
Victor Julien 040404b093 detect/profiling: track bytes scanned by prefilter engines 3 years ago
Victor Julien e250ef6402 debug: remove empty header 3 years ago
Philippe Antoine e587f6792a detect: support file.data for HTTP1 to server
That is file sent with POST or PUT

Ticket: #4144
3 years ago
Philippe Antoine b8524e70d9 detect: change InspectEngineFuncPtr2 to return uint8_t 3 years ago
Victor Julien 4feb0529a4 detect/file: minor code cleanup
Reduce scope where possible. Suggested by cppcheck.
4 years ago
Philippe Antoine 16f4e5f31c detect: file_data keyword works on nfs protocol
Ticket: #4839
4 years ago
Philippe Antoine c3339c853e detect: fixes InspectionBuffer id with transforms
When InspectionBufferGet gets called with base_id
Later InspectionBufferSetup must also be called with base_id

In case there were transforms, we had base_id != list_id

Not calling InspectionBufferSetup with the right id
resulted in leaving a dangling pointer,
because it was not added to det_ctx->inspect.to_clear_queue

Bug: #4681.
4 years ago
Victor Julien 3dc50322db detect: fix multi inspect buffer issue; clean up
Fix multi inspect buffer API causing cleanup logic in the single
inspect buffer paths. This could lead to a buffer overrun in the
"to clear" logic.

Multi buffers now use InspectionBufferSetupMulti instead of
InspectionBuffer. This is enforced by a check in debug validation.

Simplify the multi inspect buffer setup code and update the callers.
4 years ago
Victor Julien ea3fb4a465 detect/file.data: fix mixing transforms (http)
Fix handling of file progress tracking for regular file.data along
with transform combinations for the part of the implementation that
uses the HTTP inspection logic.

This is done by implementing the 'base id' logic.

Related tickets: #4361 #4199 #3616
5 years ago
Victor Julien 54ad7de9ce detect/file.data: fix mixing transforms (file api)
Fix handling of file progress tracking for regular file.data along
with transform combinations for the part of the implementation that
uses the File API.

This is done by implementing the 'base id' logic.

Related tickets: #4361 #4199 #3616
5 years ago
Philippe Antoine 707f027231 protos: renaming ALPROTO_HTTP* constants
Having now ALPROTO_HTTP1, ALPROTO_HTTP2 and ALPROTO_HTTP

Run with 3 sed commands
git grep ALPROTO_HTTP | cut -d: -f1 | uniq |
 xargs sed -i -e 's/ALPROTO_HTTP/ALPROTO_HTTP1/g'
git grep ALPROTO_HTTP12 | cut -d: -f1 | uniq |
 xargs sed -i -e 's/ALPROTO_HTTP12/ALPROTO_HTTP2/g'
git grep ALPROTO_HTTP1_ANY | cut -d: -f1 | uniq |
 xargs sed -i -e 's/ALPROTO_HTTP1_ANY/ALPROTO_HTTP/g'

and then running clang-format
5 years ago