Commit Graph

17728 Commits (master)
 

Author SHA1 Message Date
Philippe Antoine ee386ac6eb detect/tag: timeout handling precision improvement
As found by -Wshorten-64-to-32 warnings

Ticket: #6186

Use SCTime_t instead of u32, which increases memory usage for
the structures changed here, while making it more correct.
2 months ago
Philippe Antoine bad7d2f16d detect/xbits: timeout handling precision improvement
As found by -Wshorten-64-to-32 warnings

Ticket: #6186

Use SCTime_t instead of u32, which increases memory usage for
the structures changed here, while making it more correct.
2 months ago
Jason Ish 259a304f3e rust/applayer: collapse nested if let to remove clippy warning
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#collapsible_match
   = note: `#[warn(clippy::collapsible_match)]` on by default
2 months ago
Jason Ish 90297788f6 rust/htp: suppress io_other_error lint
The fix for this lint requires Rust 1.74.
2 months ago
Jason Ish 061ed2d6a4 github-ci: use rust 1.85.0 for clippy check on templates
Currently our CI clippy test is based on 1.85.0, but the clippy test for
the templates was using latest stable.  Unify the clippy checks to
1.85.0.
2 months ago
Shivani Bhardwaj 91353fdb61 mqtt: trigger raw stream reassembly
Internals
---------
Suricata's stream engine returns data for inspection to the detection
engine from the stream when the chunk size is reached.

Bug
---
Inspection triggered only in the specified chunk sizes may be too late
when it comes to inspection of smaller protocol specific data which
could result in delayed inspection, incorrect data logged with a transaction
and logs misindicating the pkt that triggered an alert.

Fix
---
Fix this by making an explicit call from all respective applayer parsers to
trigger raw stream reassembly which shall make the data available for inspection
in the following call of the stream engine. This needs to happen per direction
on the completion of an entity like a request or a response.

Important notes
---------------
1. The above mentioned behavior with and without this patch is
affected internally by the following conditions.
- inspection depth
- stream depth
In these special cases, the inspection window will be affected and
Suricata may not consider all the data that could be expected to be
inspected.
2. This only applies to applayer protocols running over TCP.
3. The inspection window is only considered up to the ACK'd data.
4. This entire issue is about IDS mode only.

MQTT creates a transaction per message per direction, so, a call to
trigger raw stream reassembly has been made on completion of each
transaction in the respective direction.

Optimization 7026
Bug 7004
2 months ago
Shivani Bhardwaj 42978ca9a7 modbus: trigger raw stream reassembly
Internals
---------
Suricata's stream engine returns data for inspection to the detection
engine from the stream when the chunk size is reached.

Bug
---
Inspection triggered only in the specified chunk sizes may be too late
when it comes to inspection of smaller protocol specific data which
could result in delayed inspection, incorrect data logged with a transaction
and logs misindicating the pkt that triggered an alert.

Fix
---
Fix this by making an explicit call from all respective applayer parsers to
trigger raw stream reassembly which shall make the data available for inspection
in the following call of the stream engine. This needs to happen per direction
on the completion of an entity like a request or a response.

Important notes
---------------
1. The above mentioned behavior with and without this patch is
affected internally by the following conditions.
- inspection depth
- stream depth
In these special cases, the inspection window will be affected and
Suricata may not consider all the data that could be expected to be
inspected.
2. This only applies to applayer protocols running over TCP.
3. The inspection window is only considered up to the ACK'd data.
4. This entire issue is about IDS mode only.

Modbus has a classic request response model, so, a call to trigger raw
stream reassembly is added on completion of each request and response.

Optimization 7026
Bug 7004
2 months ago
Shivani Bhardwaj ca7e9f8daf ldap: trigger raw stream reassembly
Internals
---------
Suricata's stream engine returns data for inspection to the detection
engine from the stream when the chunk size is reached.

Bug
---
Inspection triggered only in the specified chunk sizes may be too late
when it comes to inspection of smaller protocol specific data which
could result in delayed inspection, incorrect data logged with a transaction
and logs misindicating the pkt that triggered an alert.

Fix
---
Fix this by making an explicit call from all respective applayer parsers to
trigger raw stream reassembly which shall make the data available for inspection
in the following call of the stream engine. This needs to happen per direction
on the completion of an entity like a request or a response.

Important notes
---------------
1. The above mentioned behavior with and without this patch is
affected internally by the following conditions.
- inspection depth
- stream depth
In these special cases, the inspection window will be affected and
Suricata may not consider all the data that could be expected to be
inspected.
2. This only applies to applayer protocols running over TCP.
3. The inspection window is only considered up to the ACK'd data.
4. This entire issue is about IDS mode only.

LDAP can have multiple responses corresponding to a request. The call to
trigger raw stream reassembly has been added on common call sites that
see the completion of a request or any of the responses.

Optimization 7026
Bug 7004
2 months ago
Shivani Bhardwaj fd683ed151 krb: trigger raw stream reassembly
Internals
---------
Suricata's stream engine returns data for inspection to the detection
engine from the stream when the chunk size is reached.

Bug
---
Inspection triggered only in the specified chunk sizes may be too late
when it comes to inspection of smaller protocol specific data which
could result in delayed inspection, incorrect data logged with a transaction
and logs misindicating the pkt that triggered an alert.

Fix
---
Fix this by making an explicit call from all respective applayer parsers to
trigger raw stream reassembly which shall make the data available for inspection
in the following call of the stream engine. This needs to happen per direction
on the completion of an entity like a request or a response.

Important notes
---------------
1. The above mentioned behavior with and without this patch is
affected internally by the following conditions.
- inspection depth
- stream depth
In these special cases, the inspection window will be affected and
Suricata may not consider all the data that could be expected to be
inspected.
2. This only applies to applayer protocols running over TCP.
3. The inspection window is only considered up to the ACK'd data.
4. This entire issue is about IDS mode only.

KRB5 creates a transaction based on how each input is parsed. It could
be parsed as a request or response but that is the concern of the
parser. The call to trigger raw stream reassembly has been added after
successful parsing of the respective request/response.

Optimization 7026
Bug 7004
2 months ago
Shivani Bhardwaj d0655ed30e http2: trigger raw stream reassembly
Internals
---------
Suricata's stream engine returns data for inspection to the detection
engine from the stream when the chunk size is reached.

Bug
---
Inspection triggered only in the specified chunk sizes may be too late
when it comes to inspection of smaller protocol specific data which
could result in delayed inspection, incorrect data logged with a transaction
and logs misindicating the pkt that triggered an alert.

Fix
---
Fix this by making an explicit call from all respective applayer parsers to
trigger raw stream reassembly which shall make the data available for inspection
in the following call of the stream engine. This needs to happen per direction
on the completion of an entity like a request or a response.

Important notes
---------------
1. The above mentioned behavior with and without this patch is
affected internally by the following conditions.
- inspection depth
- stream depth
In these special cases, the inspection window will be affected and
Suricata may not consider all the data that could be expected to be
inspected.
2. This only applies to applayer protocols running over TCP.
3. The inspection window is only considered up to the ACK'd data.
4. This entire issue is about IDS mode only.

HTTP2 has a classic request response model, so, a call to trigger raw
stream reassembly is added on completion of each request and response.
HTTP2 parser has its own maximum reassembly setting. The call has been
added irrespective of this setting as it is prudent to make all data so
far available for inspection if maximum was reached until the maximum.

Optimization 7026
Bug 7004
2 months ago
Shivani Bhardwaj eca13b8dd8 enip: trigger raw stream reassembly
Internals
---------
Suricata's stream engine returns data for inspection to the detection
engine from the stream when the chunk size is reached.

Bug
---
Inspection triggered only in the specified chunk sizes may be too late
when it comes to inspection of smaller protocol specific data which
could result in delayed inspection, incorrect data logged with a transaction
and logs misindicating the pkt that triggered an alert.

Fix
---
Fix this by making an explicit call from all respective applayer parsers to
trigger raw stream reassembly which shall make the data available for inspection
in the following call of the stream engine. This needs to happen per direction
on the completion of an entity like a request or a response.

Important notes
---------------
1. The above mentioned behavior with and without this patch is
affected internally by the following conditions.
- inspection depth
- stream depth
In these special cases, the inspection window will be affected and
Suricata may not consider all the data that could be expected to be
inspected.
2. This only applies to applayer protocols running over TCP.
3. The inspection window is only considered up to the ACK'd data.
4. This entire issue is about IDS mode only.

ENIP has a classic request response model, so, a call to trigger raw
stream reassembly is added on completion of each request and response.

Optimization 7026
Bug 7004
2 months ago
Philippe Antoine 622476c47d http1: use a tx iterator
Better performance than the defaut iterator as we do not need to
read all first elements every time
2 months ago
Philippe Antoine c2756dec75 htp: move transactions list from BTree to VecDeque
As it is more efficient in our case of pipelining requests
2 months ago
Philippe Antoine 756f28d086 http: simplify tx removal interface 2 months ago
Philippe Antoine ba186f5d25 htp: remove unused code 2 months ago
Jeff Lucovsky 07205ab057 detect/xform: Support transform identity data
Transforms that support optional strings, like from_base64 and
pcrexform, should also support identity-strings to treat transforms with
like transform options as the same.

This commit adds transform identity data handling:
- When computing a hash, include identity data from the transform
- When comparing, include the identity data from the transforms
- Omitting the "options" ptr from the transform hash/compare
- Modify xor, pcrexform and from_base64 to supply identification data for
  disambiguation in the compare/hash logic.
2 months ago
Jeff Lucovsky 0b53a19c81 detect/ftp: Set buffer desc for ftp.dynamic_port
Ensure that the buffer description is set for the ftp.dynamic_port
keyword.
2 months ago
Jeff Lucovsky 0b02b1d2d1 doc/ftp: Document ftp.mode keyword
Document the ftp.mode keyword
Fixup a typo in the ftp.reply keyword section.

Issue: 7505
2 months ago
Jeff Lucovsky ddb4361432 detect/ftp: Add ftp.mode command
Issue: 7505

This commit adds support for the ftp.mode keyword.

ftp.mode: active|passive
2 months ago
Jeff Lucovsky c6e0ba2b85 detect/ftp: Add parser for ftp.mode keyword
Issue: 7505

Add a parser for the ftp.mode command that returns the state struct. Add
a function to free the object it allocates.
2 months ago
Jeff Lucovsky 34d7f0c65d output/ftp: Log mode for passive connections
This commit will now output the FTP mode -- active or passive -- when
the command is
- (previous): PORT, EPRT
- (new): PASV, EPSV

Issue: 7505
2 months ago
Philippe Antoine 4af3bd9c91 rust: bindgen SCDetectHelperKeywordSetCleanCString
Ticket: 7667
2 months ago
Philippe Antoine 13449344e8 rust: bindgen SCDetectHelperKeywordRegister
Ticket: 7667
2 months ago
Philippe Antoine f909bbba68 rust: bindgen SCDetectHelperKeywordAliasRegister
Ticket: 7667
2 months ago
Philippe Antoine a7448a0c16 rust: bindgen SCDetectHelperBufferRegister
Ticket: 7667
2 months ago
Philippe Antoine 0d82e905bc rust: bindgen DetectHelperMultiBufferMpmRegister
Ticket: 7667
2 months ago
Philippe Antoine dfa4df9f54 rust: bindgen SCDetectHelperMultiBufferProgressMpmRegister
Ticket: 7667
2 months ago
Philippe Antoine 7805103f68 detect: fix -Wshorten-64-to-32 warnings for content inspection
Ticket: 6186
2 months ago
Jeff Lucovsky dc994effd6 detect/entropy: Correct slot for url initialization
Correct the sigmatch slot for url initialization.
2 months ago
Victor Julien 26f7632324 htp: fix unittests shutdown order
Make sure to free flow/packets before cleaning up global structs.

Solves an crash sometimes seen on OpenBSD in test HTPParserTest26.
2 months ago
Philippe Antoine 030493c4a8 lua: better doc for ja3 lib
Completes commit 7e78ad944c

Tickt: 7605
2 months ago
Jason Ish c13f85f18d lua: convert file functions to lib suricata.file
This also breaks out the fileinfo function into a method per file info
item. And likewise for state, just return the state and add a new method
for checking if the file is stored.

Ticket: #7491
2 months ago
Victor Julien 3b5a99d239 github-ci: use all profraw files in coverage test 2 months ago
Victor Julien 7b46ec8c24 github-ci: add --list-app-layer-hooks checks 2 months ago
Victor Julien acc7bd0c6f app-layer: add --list-app-layer-hooks options
This provides a list of available hooks.

Ticket: #7702.
2 months ago
Victor Julien 019c746d79 github-ci: add -h option to coverage test 2 months ago
Juliana Fajardini 62949b3815 pgsql: remove unused "password_message" code
``Password message`` is actually logged just as ``Password``.
Remove related dead code.
2 months ago
Juliana Fajardini 6f81caf8d4 pgsql: clearly indicate redacted password message
If a password message was seen while logging passwords was disabled
for pgsql, this would lead to an empty request being logged.
Instead of simply not logging anything when there is a password message
and this is disabled, however, log instead that said password is
redacted.

Bug #7647
2 months ago
Jason Ish 11cef2980b github-ci: use current directory for unit test logging
/tmp appears to exist when you make it, but doesn't appear to actually
exist after msys translation, so just use "."
2 months ago
Shivani Bhardwaj c1b932747c schema: add rule keyword mapping for flow 2 months ago
Victor Julien b14734c113 eve/verdict: log 'accept' for firewall accept rules
Ticket: #7698.
2 months ago
Shivani Bhardwaj e2f0370f4c decode: remove unnecessary PKT_STREAM_EOF flag
PKT_STREAM_EOF flag is set only when a pseudo packet is created. In all
the users of this flag, it suffices to use PKT_PSEUDO_STREAM_END
instead. PKT_PSEUDO_STREAM_END is the more widely used flag as well so
keep it and remove this unneeded flag creating a vacancy.
2 months ago
Shivani Bhardwaj 9f6d52f569 stream/reassemble: remove unused min_inspect_depth
Whether minimum inspection depth should be respected is an important
parameter but it is only used by one caller of StreamReassembleRawDo fn
to calculate progress value. This progress value is already passed as a
parameter to the said fn.
2 months ago
Shivani Bhardwaj edbba842e8 src: apply clang formatting changes 2 months ago
Shivani Bhardwaj 623229728d signature: rename num to iid
to signify that it refers to the internal ID of the signature and
distinguish it from the "id" param that signifies the "sid" specified in
a rule.
2 months ago
Philippe Antoine fd7bd9c200 src: new file detect-engine-inspect-buffer.h
For InspectionBuffer structure and related functions

Bindgen it for rust, especially transforms

Ticket: 7667
2 months ago
Philippe Antoine df0dc2e8ea rust: bindgen SCDetectSignatureAddTransform
by moving it to detect-engine-buffer.h and prefixing it

Ticket: 7667
2 months ago
Philippe Antoine 09664df8ef build: reorder files in Makefile.am 2 months ago
Philippe Antoine 1f871cdd64 rust: reuse/reexport suricata_sys DetectEngineThreadCtx
Use the bindgen'd version instead of our own recoded one
2 months ago
Philippe Antoine cf336396c3 rust: bindgen detect-engine-helper.h
Ticket: 7667

Begin by focusing on transforms
2 months ago