Commit Graph

18811 Commits (898446aee88bcba4dcf2aa1cba2da7ce2cae7018)
 

Author SHA1 Message Date
Philippe Antoine 64d29fcd1c rust: move AppLayerStateData definition to C
and bindgen it to rust, and use default trait instead of new

Will make easier the bindgen of RustParser structure which uses
a callback which uses AppLayerStateData
5 months ago
Philippe Antoine 76efb8af4d rust: move AppLayerGetFileState definition to C
and bindgen it to rust.

Will make easier the bindgen of RustParser structure which uses
a callback which uses AppLayerGetFileState
5 months ago
Philippe Antoine fc23e54c6d rust: bindgen with derive eq
As will be needed such as AppLayerTxData
5 months ago
Andreas Dolp 69eb567dea doc/userguide: fix within-distance pointer graphics in payload-keywords doc
Redmine ticket: #8261

According to [1], the within pointer (if combined with distance)
includes the distance pointer, which is not clearly visible in the
graphic.

Fixed this in a new graphic by some GIMP arts.

PS: Special thanks to one of our team members Annika C. for initially
spotting this!

[1] https://forum.suricata.io/t/is-within-affected-by-distance/1688
5 months ago
Philippe Antoine 0d714b9624 doc/jsonschema: remove non-existing email fields 5 months ago
Philippe Antoine 81cc007a11 doc/jsonschema: remove non-existent ldap field
Probably a duplicate typo
5 months ago
Philippe Antoine 750ae52eac doc/jsonschema: remove obsolete insert_list_fail field
Ticket: 5267
5 months ago
Shivani Bhardwaj d4008a6508 dcerpc: remove bad tests
Unittests test_parse_bind_pdu_infinite_loop and
test_parse_bindack_pdu_infinite_loop seem to have artificially made up
header which does not hold up to the strict calculations enforced by the
parser now. Their headers mark the fraglens as 64 and 72 respectively
which are not enough to hold the kind of bind(ack) items that are expected.
It worked so far as the parser passed the entire input slice around but
with the bugfix for issue 7546, the input passed around is strictly
restricted to the fraglen parsed in the header.

Bug 7546
5 months ago
Shivani Bhardwaj ed1bd8624e dcerpc: use fraglen from header
So far, the fraglen defined in the header was used inconsistently in
certain places to define bounds on input length. Make it consistent by
making sure that only a slice up until fraglen is passed around as that
is the maximum length the fragment should have.
With the help of Applayer::incomplete API, the case when the
stream_slice passed to the parser is smaller than the header defined
fraglen is already handled.

Bug 7546
5 months ago
Shivani Bhardwaj fc9da1c7a1 dcerpc: fix consumed bytes post gap handling
The parser could receive an input that consists of arbitrary data post
gap. This is handled in the beginning of the fn handle_input_data.
However, the rest of the calculation does not take into account the
bytes that were consumed at this stage. Fix the indices and calculations
to consider a new DCERPC fragment beginning post these consumed bytes.
5 months ago
Mingyu Jeon c98112eb67 doc: update tls_cert_notafter/before
refs #3065

* add explanation on omitted values
5 months ago
Mingyu Jeon 940f691245 parse/tls: fix date format parsing
refs #3065

* Fix to support the date format below
=> tls_cert_notafter:[<|>]YYYY
5 months ago
Philippe Antoine 9a52bbcbe9 detect/quic: move quic.cyu.hash to rust
Ticket: 8255
5 months ago
Philippe Antoine 01610aabe7 detect/quic: move quic.cyu.string to rust
Ticket: 8255
5 months ago
Philippe Antoine ee4d4a14d5 detect/quic: move quic.ua to rust
Ticket: 8255
5 months ago
Philippe Antoine 9ba9fdcb87 detect/quic: move quic.sni to rust
Ticket: 8255
5 months ago
Philippe Antoine 907e71a984 detect/quic: move quic.version to rust
Ticket: 8255
5 months ago
Jeff Lucovsky 45a36e961f doc/byte_jump: Clarify bitmask operation
Issue: 6693

Clarify how the bitmask value is used for byte_jump

Snort compatibility says:
- The bitmask value is applied to the extracted value before the
  multiplier is applied.
- The result of the bitmask operation is to be right shifted by the
  number of trailing 0's in the bitmask value.
5 months ago
Jeff Lucovsky b872850f16 detect/byte_jump: Support bitmask value
Issue: 6693

Add bitmask support to byte_jump
- Parse
- Calculate shift count
- Apply to value before applying multiplier
- Order items in DetectBytejumpData to reduce holes.

Snort:
See https://github.com/chenkc/snort2.9/blob/master/snort-2.9.11.1/src/detection-plugins/sp_byte_jump.c#L780
5 months ago
Victor Julien c1d6958940 github-ci: relax block check 6 months ago
Victor Julien 0a86b0f79d github-ci: remove too verbose namespace test output 6 months ago
Victor Julien f3e3795f48 github-ci: relax namespace shutdown error checks
Caddy would sometimes return an error during shutdown. Add a warning if
this happens, but don't fail the job.
6 months ago
Philippe Antoine 4b1b13960d detect/flowbits: align all pointers in the struct
Makes leak sanitizer work without adding LSAN_OPTIONS=use_unaligned=1

Otherwise, leak sanitizer may report the read and black tree pointers
as leaked when it is still owned by some global variable.

Follow-up on d046e82db6
6 months ago
dependabot[bot] 3a9682fd48 github-actions: bump actions/checkout from 6.0.1 to 6.0.2
Bumps [actions/checkout](https://github.com/actions/checkout) from 6.0.1 to 6.0.2.
- [Release notes](https://github.com/actions/checkout/releases)
- [Commits](https://github.com/actions/checkout/compare/v6.0.1...v6.0.2)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: 6.0.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
6 months ago
dependabot[bot] c533ec0a1d github-actions: bump github/codeql-action from 4.31.6 to 4.32.0
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 4.31.6 to 4.32.0.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Commits](https://github.com/github/codeql-action/compare/v4.31.6...v4.32.0)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-version: 4.32.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
6 months ago
Fupeng Zhao f7ec1c8813 redis: Add authentication support
Add authentication support to the Redis logging output.
It introduces `username` and `password` configuration options for Redis,
allowing Suricata to authenticate with Redis servers that require it.

Ticket: 7062
6 months ago
Philippe Antoine e982dd50d0 smtp: use configured FTP ports in probing parser
To allow users to configure more than port 21

Ticket: 6591
6 months ago
Philippe Antoine 4feacfb889 smtp: refine QUIT protocol detection
with probing parser, failing when it is on port 21 as FTP QUIT

Ticket: 6591
6 months ago
Philippe Antoine 7dbe033ae0 app-layer: function to register ci pattern + probe
Ticket: 6591
6 months ago
Philippe Antoine 81572cb457 detect: simplify code for http.header
reuse generic DetectEngineInspectBufferGeneric
6 months ago
Philippe Antoine cd0e4288c4 detect/http: remove unused function parameter
Will help to use generic DetectEngineInspectBufferGeneric
6 months ago
Victor Julien 49783fa9f7 github-ci: add namespace based IPS tests
Implementing a bridge test with af-packet IPS and a routing test
with iptables + nfqueue.

Very helpful explanation and guidance on network namespaces can be
found here: https://www.redhat.com/en/blog/net-namespaces

Sets up 3 network namespaces:
1. client - for running client tools like ping, curl, wget
2. server - for running a server, currently only Caddy
3. dut - for running Suricata, this namespace connects the client and
   server namespaces

Validate IPS operations in 3 ways:
1. check return codes of the client tools
2. check Suricata's IPS stats
3. use tshark to validate expected drops

Run Suricata in AF_PACKET IPS mode for both autofp and workers mode. Do
the same for NFQUEUE.

Tshark's JSON output is used with JQ to validate that pings are dropped.

All tests are codecov enabled.
6 months ago
Philippe Antoine 2cf9a327d5 detect/ssh: move ssh.hassh to rust
Introduces helper SCDetectRegisterBufferLowerMd5Callbacks
6 months ago
Philippe Antoine 14c78d8405 detect: generic setup callback DetectLowerSetupCallback 6 months ago
Philippe Antoine 24e33944aa detect: SetupCallback uses DetectBufferType argument
Will allow for generic callbacks, that can be used from rust
6 months ago
Philippe Antoine 83360cfce0 detect/ssh: move ssh.hassh string to rust
bindgen needed SCSigMatchSilentErrorEnabled on the way
6 months ago
Lukas Sismis 27f398b5f2 dpdk: collect port stats before device stop
Some drivers (e.g. BNXT) fail to report stats after the device is
stopped. Move stats collection (DPDKDumpCounters and PrintDPDKPortXstats)
to run before rte_eth_dev_stop() in HandleShutdown.

Also change PrintDPDKPortXstats error handling from FatalError to
graceful return since stats collection failures during shutdown
should not crash the application.

The commit removes ThreadExitPrintStats callback as the function had no
useful features after the stats were moved.

Ticket: 8251
6 months ago
Lukas Sismis 0fe0390a2f hs: suppress TOCTOU stat use
To explain a bit more the TOCTOU issue found, we can consider
a case where Suricata starts to prune, yet externally somebody also
starts erasing cache files.
Right after Suricata checks the file age with the stat function,
somebody may delete or update the file of our interest.

Suricata aging decision doesn't reflect the actual state of the file.
This commit additionally adds a check for noent failure of the unlink operation
(considered as a success). The code can still delete a file that is recently
updated but was considered stale.

In the documentation-following deployments this should not happen anyway as
one cache folder should only be used by a single Suricata instance (and within
Suricata instance only one thread handles cache eviction).
Additionally, the `stat` and `unlink` command are immediatelly followed, making
this scenario extra unlikely.

Additional comment in the code explains problems of using fstat and potential
issues on Windows.

Ticket: 8243
6 months ago
Ofer Dagan 7627756360 detect/detection_filter: add unique_on option
Add optional unique_on {src_port|dst_port} to detection_filter for
exact distinct port counting within the seconds window.

Features:
- Runtime uses a single 64k-bit (8192 bytes) union bitmap per
  threshold entry with O(1) updates.
- Follows detection_filter semantics: alerting starts after the
  threshold (> count), not at it.
- On window expiry, the window is reset and the current packet's
  port is recorded as the first distinct of the new window.

Validation:
- unique_on requires a ported transport protocol; reject rules
  that are not tcp/udp/sctp or that use ip (protocol any).

Memory management:
- Bitmap memory is bounded by detect.thresholds.memcap.
- New counters: bitmap_memuse and bitmap_alloc_fail.

Tests:
- C unit tests for parsing, distinct counting, window reset, and
  allocation failure fallback.
- suricata-verify tests for distinct src/dst port counting.

Task #7928
6 months ago
Ofer Dagan 2371829bf1 schema: add threshold stats counters
Add schema definitions for new threshold-related statistics:
- bitmap_alloc_fail: Count of bitmap allocation failures
- bitmap_memuse: Memory usage by detection_filter bitmaps
- memcap: Memory cap for threshold hash table
- memuse: Memory usage by threshold hash table

Task #7928
6 months ago
Philippe Antoine d046e82db6 detect/flowbits: align pointer in the struct
Makes leak sanitizer work without adding LSAN_OPTIONS=use_unaligned=1

Otherwise, leak sanitizer may report rule_id as leaked
when it is still owned by some global variable
6 months ago
Jason Ish 5d61f5253d lua: don't attempt to garbage collect a null value
When not sandboxed, a script can get access to the metatable and call
`.__gc` with an invalid value like nil, causing a NULL pointer dereference
in Suricata.

Ticket: #8248
6 months ago
Philippe Antoine b944e3b1ed ci: update rust version to 1.93 6 months ago
Philippe Antoine 02cb0f2ac2 rust: fix unnecessary_unwrap warnings
warning: called `unwrap` on `rd.pipe` after checking its variant with `is_some`
   --> src/smb/smb1.rs:858:28
    |
857 |             if rd.pipe.is_some() {
    |             -------------------- help: try: `if let Some(<item>) = rd.pipe`
858 |                 let pipe = rd.pipe.unwrap();
    |                            ^^^^^^^^^^^^^^^^
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.93.0/index.html#unnecessary_unwrap
    = note: `#[warn(clippy::unnecessary_unwrap)]` on by default
6 months ago
Shivani Bhardwaj 9df5fd180e flowbits: add a validation callback during setup
This should make it possible to catch invalid combinations in the same
signature early. This patch covers checking and erroring on the following
invalid cmd combinations:
- set + isset
- unset + isnotset
- set + toggle
- set + unset
- isset + isnotset
- unset + toggle

the same flowbit in the same signature which is basically an unnecessary
operation at runtime.

This also helps bring down the difficulty of handling of actual complex
flowbit chains.

Bug 7772
Bug 7773
Bug 7774
Bug 7817
Bug 7818
Bug 8166
6 months ago
Lukas Sismis b575ae3fd1 pcap-file: move packet counter to PCAP packet structure
Code refactor to gather all PCAP-related structure members
under one structure.

New pcap_v structure guards protect the union variables from
other capture modes trying to access the packet number incorrectly.

Ticket: 7835
6 months ago
Lukas Sismis 400328c3c3 pcap-file: prep codebase for pcap_cnt move refactor
For an easier review process, this is a two-step change process,
in which pcap_cnt is first accessed by functions-to-be, implemented
as simple macros.

In the follow-up commit, the actual refactor is implemented with the new
function. The old macros are deleted.

Ticket: 7835
6 months ago
Lukas Sismis f33f6e7ee5 stream: remove dead commented out code 6 months ago
Lukas Sismis c7058c1a35 stream: guard auxiliary packet counter to debug
Use of t_pcapcnt is only relevant when compiled in debug mode only.
This patch adds additional debug guard to also shield the declaration
and assignment.
6 months ago
Philippe Antoine cafc398355 dcerpc: consume bytes after gap resync
Ticket: 7567

Re-applies commit 8c3bd3e8a0
which was reverted in f64aec9d11
6 months ago