Ticket: 8413
RFC 6455 Section 5.4 states
Control frames (see Section 5.5) MAY be injected in the middle of
a fragmented message.
Control frames are identified by opcodes where the most significant
bit of the opcode is 1.
(cherry picked from commit 2fa10052cf)
In some timezones, LogCustomFormatTest01 was failing as we were
specifying the wrong setting for is_dst. Instead set it to -1 so mktime
can make the decision based on the environment. Appears to fix this test
for "Brazil/East" in the summer.
Add GitHub CI tests to test this specific test with a few different
timezones.
(cherry picked from commit 468a13e052)
So that we can log and detect on interface uuids, even if
AlterContext packets change one of the contexts, but other contexts
are still used after it in request/responses
Ticket: 8378
(cherry picked from commit 05a11e2897)
Ticket: 8333
Allows to pass on pseudo packets
So, that a ruleset like
pass ssh any any -> any any (geoip:any,US,IN; sid: 1;)
drop ip any any -> any any (flow:established, to_server; sid:2;)
Does not lead to a drop on the pseudo-packet when we flush
because ssh traffic gets encrypted
(cherry picked from commit d4c036718a)
Unittests test_parse_bind_pdu_infinite_loop and
test_parse_bindack_pdu_infinite_loop seem to have artificially made up
header which does not hold up to the strict calculations enforced by the
parser now. Their headers mark the fraglens as 64 and 72 respectively
which are not enough to hold the kind of bind(ack) items that are expected.
It worked so far as the parser passed the entire input slice around but
with the bugfix for issue 7546, the input passed around is strictly
restricted to the fraglen parsed in the header.
Bug 7546
(cherry picked from commit d4008a6508)
Ticket: 8375
logger already read this variable but it was never set, and
the rust compiler did not warn about this
Allows to log the interface uuid when ctx_id > 0
Wireshark filter for related pcaps
dcerpc.cn_ctx_id > 0 && dcerpc.pkt_type == 0
(cherry picked from commit 6587e363a3)
Update the AlmaLinux recommended minimal build to use the pre built
distribution archive as this is the type of build the documentation is
targetting.
(cherry picked from commit 551f6ded69)
Update the Ubuntu/Debian minimal recommended build to use the
pre-built dist archive instead of building from git, as that is the
type of build this documentation targets.
Also use the ubuntu:22.04 container. The GitHub provided Ubuntu 22.04
VM appears to contain some new additions like a newer Rust that is not
found on a typical Ubuntu 22.04 installation.
(cherry picked from commit 87dba92bdd)
Due to incorrect mempool size calculation, in some cases
(e.g. when rx-descriptors=32k, tx-descriptors=64) the mempool
size was calculated insufficiently in the auto mode (in the example
case it would result to 32767 which wouldn't even hold RX descriptors).
Even with a fix, the mempool size would increase (and in turn hugepage
consumption), as current mempools are just a sum of rx + tx descriptors.
To avoid introducing behavior-change this commit is reverted.
Ticket: 8371
This reverts commit 0c54932a4b.
When during raw reassembly it is detected that last ack is moved beyond
the progress and also beyond the data retrieved, update progress to the
last ack value.
Bug: #8272.
(cherry picked from commit ac1a514c7b)
Ticket: 8289
If stream.reassembly.depth is unlimited,
an attacker controlling the 2 sides of a communication going through Suricata
can send a transition with an infinite number of headers, until suricata OOMs
Solution is to offer a configuration option to bound the number
of HTTP2 frames we store in a HTTP2 transaction, and produce an
anomaly if this bound is crossed
(cherry picked from commit 784e173278)
Ticket: 8294
Fixes: 6c1238b7bd ("tls: Integrate ALPNs into HandshakeParams object")
May happen if we analyze TLS without seeing the hello
(cherry picked from commit cb76301581)
We used to look for a full line, but as we look fot the last eol
we need to use the right index in the not-reversed list
(cherry picked from commit 49fd7001ff)
Ticket: 8292
When we have buffered something in ctx.decoded_line,
we already looked for '\n' in it, so we do not need to run it again
Otherwise, callers that supply mime_smtp_find_url_strings with
a few bytes at a time without "\n", have a quadratic
complexity
(cherry picked from commit 8bba47aa09)
Simple libpcap example for live capture. Allows listening on multiple
interfaces to show how multiple threads (workers) can be used.
Ticket: #8096
(cherry picked from commit f711e57e8e)
To prep for the removal of the lib runmode, simplify this C++ example
to match our simple example. We don't yet have the C++ compatible
headers to allow for a C++ app to register its own custom runmode.
(cherry picked from commit 7dd23392cc)
Debug validation revealed that library ThreadVars were being
created *after* the threads were sealed. And the only way to create
your ThreadVars that fits within the current application life-cycle is
to create them in your own custom run mode.
This is likely a better model for users who are bringing their own
packets and threads anyways, as they are essentially providing their
own capture method, and all capture methods provide their own run
mode. They're also using their own threads, which means adapting to
their own threading model.
This is suitable for a backport to 8.0. But for 9.0 we can go further
and remove the built-in library run mode, which will be done in a
follow-up commit.
Ticket: #8259
(cherry picked from commit 445de77c71)
After the distcheck, this build doesn't need unittests, so add
debug-validation, as this can trigger assertions in our lib examples
that are tested in this job
This triggers a debug-validation error in the custom library example.
Ticket: #8259
(cherry picked from commit 02e4399059)
Hyperscan MPM can cache the compiled contexts to files.
This however grows as rulesets change and leads to bloating
the system. This addition prunes the stale cache files based
on their modified file timestamp.
Part of this work incorporates new model for MPM cache stats
to split it out from the cache save function and aggregate
cache-related stats in one place (newly added pruning).
Ticket: 7893
(cherry picked from commit 15c83be61a)
hs: suppress TOCTOU stat use
To explain a bit more the TOCTOU issue found, we can consider
a case where Suricata starts to prune, yet externally somebody also
starts erasing cache files.
Right after Suricata checks the file age with the stat function,
somebody may delete or update the file of our interest.
Suricata aging decision doesn't reflect the actual state of the file.
This commit additionally adds a check for noent failure of the unlink operation
(considered as a success). The code can still delete a file that is recently
updated but was considered stale.
In the documentation-following deployments this should not happen anyway as
one cache folder should only be used by a single Suricata instance (and within
Suricata instance only one thread handles cache eviction).
Additionally, the `stat` and `unlink` command are immediatelly followed, making
this scenario extra unlikely.
Additional comment in the code explains problems of using fstat and potential
issues on Windows.
Ticket: 8244
(cherry picked from commit 0fe0390a2f)
hs/cache: cleaner and more detailed output
Reduce logging level of a minor informational message.
Split tracking of pruning by age and by version and log those
separately, where the logging only appears if something has been
removed.
Ticket: 8323
(cherry picked from commit 569ba3d26f)
hs: remove redundant file handle in HSLoadCache
HSLoadCache opened the cache file but never used the resulting handle
for reading. The actual read was done by HSReadStream which opened
the same file independently.
Removed the unused fopen/fclose pair and flattened the control flow.
Ticket: 8326
(cherry picked from commit d754b28717)
hs: use binary mode for cache file I/O
HSSaveCache wrote serialized Hyperscan databases using text mode ("w")
while HSReadStream already read them with binary mode ("rb").
Matched file reading modes to the binary format and simplified
write-size check.
Ticket: 8326
(cherry picked from commit 0cdc77b707)
hs: warn about the same cache directory
This is especially relevant for multi-instance simultaneous setups
as we might risk read/write races.
(cherry picked from commit 56c1552c3e)
hs: validate cached database against current HS installation
After deserializing a cached Hyperscan database, verify that its
version, CPU features, and mode match the current Hyperscan
installation by comparing hs_database_info output against a
reference database. Reject loading incompatible caches.
Ticket: 8326
(cherry picked from commit 2e7b12dda4)
hs: include HS platform info in cache file hash
Hash Hyperscan installation info (version, CPU features, mode)
into the cache filename. A Hyperscan upgrade or platform change
would now produce a different filename, so stale caches from an
older installation are never opened.
Ticket: 8326
(cherry picked from commit d640719413)
hs: address coverity warning in a reference string
Move the locking mechanism outside of the getter function and hold the
lock until the reference string is no longer reused.
** CID 1682023: Concurrent data access violations (MISSING_LOCK)
/src/util-mpm-hs-cache.c: 139 in HSGetReferenceDbInfo()
(cherry picked from commit 6ec9e5c957)
As a intermediary step for Hyperscan (MPM) caching,
the MPM config initialization should be part of the default
detect engine context for later dynamic retrieval.
Ticket: 7893
(cherry picked from commit 08f5abe5e9)
To have a system-level overview of when was the last time the file was
used, update the file modification timestamp to to the current time.
This is needed to remove stale cache files of the system.
Access time is not used as it may be, on the system level, disabled.
Ticket: 7893
(cherry picked from commit fd3847db72)
Philippe Antoine
Jason Ish
Juliana Fajardini
Philippe Antoine
Shivani Bhardwaj
Victor Julien
Lukas Sismis
Lukas Sismis
Giuseppe Longo