To explain a bit more the TOCTOU issue found, we can consider
a case where Suricata starts to prune, yet externally somebody also
starts erasing cache files.
Right after Suricata checks the file age with the stat function,
somebody may delete or update the file of our interest.
Suricata aging decision doesn't reflect the actual state of the file.
This commit additionally adds a check for noent failure of the unlink operation
(considered as a success). The code can still delete a file that is recently
updated but was considered stale.
In the documentation-following deployments this should not happen anyway as
one cache folder should only be used by a single Suricata instance (and within
Suricata instance only one thread handles cache eviction).
Additionally, the `stat` and `unlink` command are immediatelly followed, making
this scenario extra unlikely.
Additional comment in the code explains problems of using fstat and potential
issues on Windows.
Ticket: 8243
Add optional unique_on {src_port|dst_port} to detection_filter for
exact distinct port counting within the seconds window.
Features:
- Runtime uses a single 64k-bit (8192 bytes) union bitmap per
threshold entry with O(1) updates.
- Follows detection_filter semantics: alerting starts after the
threshold (> count), not at it.
- On window expiry, the window is reset and the current packet's
port is recorded as the first distinct of the new window.
Validation:
- unique_on requires a ported transport protocol; reject rules
that are not tcp/udp/sctp or that use ip (protocol any).
Memory management:
- Bitmap memory is bounded by detect.thresholds.memcap.
- New counters: bitmap_memuse and bitmap_alloc_fail.
Tests:
- C unit tests for parsing, distinct counting, window reset, and
allocation failure fallback.
- suricata-verify tests for distinct src/dst port counting.
Task #7928
Makes leak sanitizer work without adding LSAN_OPTIONS=use_unaligned=1
Otherwise, leak sanitizer may report rule_id as leaked
when it is still owned by some global variable
When not sandboxed, a script can get access to the metatable and call
`.__gc` with an invalid value like nil, causing a NULL pointer dereference
in Suricata.
Ticket: #8248
warning: called `unwrap` on `rd.pipe` after checking its variant with `is_some`
--> src/smb/smb1.rs:858:28
|
857 | if rd.pipe.is_some() {
| -------------------- help: try: `if let Some(<item>) = rd.pipe`
858 | let pipe = rd.pipe.unwrap();
| ^^^^^^^^^^^^^^^^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.93.0/index.html#unnecessary_unwrap
= note: `#[warn(clippy::unnecessary_unwrap)]` on by default
This should make it possible to catch invalid combinations in the same
signature early. This patch covers checking and erroring on the following
invalid cmd combinations:
- set + isset
- unset + isnotset
- set + toggle
- set + unset
- isset + isnotset
- unset + toggle
the same flowbit in the same signature which is basically an unnecessary
operation at runtime.
This also helps bring down the difficulty of handling of actual complex
flowbit chains.
Bug 7772
Bug 7773
Bug 7774
Bug 7817
Bug 7818
Bug 8166
Code refactor to gather all PCAP-related structure members
under one structure.
New pcap_v structure guards protect the union variables from
other capture modes trying to access the packet number incorrectly.
Ticket: 7835
For an easier review process, this is a two-step change process,
in which pcap_cnt is first accessed by functions-to-be, implemented
as simple macros.
In the follow-up commit, the actual refactor is implemented with the new
function. The old macros are deleted.
Ticket: 7835
Use of t_pcapcnt is only relevant when compiled in debug mode only.
This patch adds additional debug guard to also shield the declaration
and assignment.
It does not return anything.
The only known implementation is in dns where it does not return.
And the C code does not bother to check a return value anyways.
Add basic derive counter support, where a counters value is derived from
2 other counters.
Add DERIVE_DIV that divides the first counter value by the second
counters value.
Convert `decoder.avg_pkt_size` to be derived from `decoder.bytes` and
`decoder.pkts`.
Ticket: #5615.
When in a `base64_decode`-`base64_data` pair the decode was depending
on another match through the relative option, the `buffer_offset` would
be updated to the relative position of the previous match. During the
`base64_data` phase, a relative match would use that offset even though
the match happened in a new buffer.
Example::
http.request_body; content:"|27|"; \
base64_decode:relative; \
base64_data; content:"|ff ff ff ff|"; within:16;
This use of the `buffer_offset` is incorrect as that value is relative
to a buffer and the `base64_data` points to a new buffer.
This patch addresses this by resetting DetectEngineThreadCtx::buffer_offset
before inspecting `base64_data`.
Bug: #7842.
Adds a chapter indicating what are the main steps when adding exception
policies, how is it possible to extend them, as well as main aspects and
files to consider when doing so.
Task #5612
Hyperscan MPM can cache the compiled contexts to files.
This however grows as rulesets change and leads to bloating
the system. This addition prunes the stale cache files based
on their modified file timestamp.
Part of this work incorporates new model for MPM cache stats
to split it out from the cache save function and aggregate
cache-related stats in one place (newly added pruning).
Ticket: 7830
As a intermediary step for Hyperscan (MPM) caching,
the MPM config initialization should be part of the default
detect engine context for later dynamic retrieval.
Ticket: 7830
To have a system-level overview of when was the last time the file was
used, update the file modification timestamp to to the current time.
This is needed to remove stale cache files of the system.
Access time is not used as it may be, on the system level, disabled.
Ticket: 7830
Lukas Sismis
Ofer Dagan
Philippe Antoine
Jason Ish
Shivani Bhardwaj
Jeff Lucovsky
Juliana Fajardini
Victor Julien
Abhijeet Singh
Sergey Zhidkih