When during raw reassembly it is detected that last ack is moved beyond
the progress and also beyond the data retrieved, update progress to the
last ack value.
Bug: #8272.
(cherry picked from commit ac1a514c7b)
Ticket: 8289
If stream.reassembly.depth is unlimited,
an attacker controlling the 2 sides of a communication going through Suricata
can send a transition with an infinite number of headers, until suricata OOMs
Solution is to offer a configuration option to bound the number
of HTTP2 frames we store in a HTTP2 transaction, and produce an
anomaly if this bound is crossed
(cherry picked from commit 784e173278)
Ticket: 8294
Fixes: 6c1238b7bd ("tls: Integrate ALPNs into HandshakeParams object")
May happen if we analyze TLS without seeing the hello
(cherry picked from commit cb76301581)
We used to look for a full line, but as we look fot the last eol
we need to use the right index in the not-reversed list
(cherry picked from commit 49fd7001ff)
Ticket: 8292
When we have buffered something in ctx.decoded_line,
we already looked for '\n' in it, so we do not need to run it again
Otherwise, callers that supply mime_smtp_find_url_strings with
a few bytes at a time without "\n", have a quadratic
complexity
(cherry picked from commit 8bba47aa09)
Simple libpcap example for live capture. Allows listening on multiple
interfaces to show how multiple threads (workers) can be used.
Ticket: #8096
(cherry picked from commit f711e57e8e)
To prep for the removal of the lib runmode, simplify this C++ example
to match our simple example. We don't yet have the C++ compatible
headers to allow for a C++ app to register its own custom runmode.
(cherry picked from commit 7dd23392cc)
Debug validation revealed that library ThreadVars were being
created *after* the threads were sealed. And the only way to create
your ThreadVars that fits within the current application life-cycle is
to create them in your own custom run mode.
This is likely a better model for users who are bringing their own
packets and threads anyways, as they are essentially providing their
own capture method, and all capture methods provide their own run
mode. They're also using their own threads, which means adapting to
their own threading model.
This is suitable for a backport to 8.0. But for 9.0 we can go further
and remove the built-in library run mode, which will be done in a
follow-up commit.
Ticket: #8259
(cherry picked from commit 445de77c71)
After the distcheck, this build doesn't need unittests, so add
debug-validation, as this can trigger assertions in our lib examples
that are tested in this job
This triggers a debug-validation error in the custom library example.
Ticket: #8259
(cherry picked from commit 02e4399059)
Hyperscan MPM can cache the compiled contexts to files.
This however grows as rulesets change and leads to bloating
the system. This addition prunes the stale cache files based
on their modified file timestamp.
Part of this work incorporates new model for MPM cache stats
to split it out from the cache save function and aggregate
cache-related stats in one place (newly added pruning).
Ticket: 7893
(cherry picked from commit 15c83be61a)
hs: suppress TOCTOU stat use
To explain a bit more the TOCTOU issue found, we can consider
a case where Suricata starts to prune, yet externally somebody also
starts erasing cache files.
Right after Suricata checks the file age with the stat function,
somebody may delete or update the file of our interest.
Suricata aging decision doesn't reflect the actual state of the file.
This commit additionally adds a check for noent failure of the unlink operation
(considered as a success). The code can still delete a file that is recently
updated but was considered stale.
In the documentation-following deployments this should not happen anyway as
one cache folder should only be used by a single Suricata instance (and within
Suricata instance only one thread handles cache eviction).
Additionally, the `stat` and `unlink` command are immediatelly followed, making
this scenario extra unlikely.
Additional comment in the code explains problems of using fstat and potential
issues on Windows.
Ticket: 8244
(cherry picked from commit 0fe0390a2f)
hs/cache: cleaner and more detailed output
Reduce logging level of a minor informational message.
Split tracking of pruning by age and by version and log those
separately, where the logging only appears if something has been
removed.
Ticket: 8323
(cherry picked from commit 569ba3d26f)
hs: remove redundant file handle in HSLoadCache
HSLoadCache opened the cache file but never used the resulting handle
for reading. The actual read was done by HSReadStream which opened
the same file independently.
Removed the unused fopen/fclose pair and flattened the control flow.
Ticket: 8326
(cherry picked from commit d754b28717)
hs: use binary mode for cache file I/O
HSSaveCache wrote serialized Hyperscan databases using text mode ("w")
while HSReadStream already read them with binary mode ("rb").
Matched file reading modes to the binary format and simplified
write-size check.
Ticket: 8326
(cherry picked from commit 0cdc77b707)
hs: warn about the same cache directory
This is especially relevant for multi-instance simultaneous setups
as we might risk read/write races.
(cherry picked from commit 56c1552c3e)
hs: validate cached database against current HS installation
After deserializing a cached Hyperscan database, verify that its
version, CPU features, and mode match the current Hyperscan
installation by comparing hs_database_info output against a
reference database. Reject loading incompatible caches.
Ticket: 8326
(cherry picked from commit 2e7b12dda4)
hs: include HS platform info in cache file hash
Hash Hyperscan installation info (version, CPU features, mode)
into the cache filename. A Hyperscan upgrade or platform change
would now produce a different filename, so stale caches from an
older installation are never opened.
Ticket: 8326
(cherry picked from commit d640719413)
hs: address coverity warning in a reference string
Move the locking mechanism outside of the getter function and hold the
lock until the reference string is no longer reused.
** CID 1682023: Concurrent data access violations (MISSING_LOCK)
/src/util-mpm-hs-cache.c: 139 in HSGetReferenceDbInfo()
(cherry picked from commit 6ec9e5c957)
As a intermediary step for Hyperscan (MPM) caching,
the MPM config initialization should be part of the default
detect engine context for later dynamic retrieval.
Ticket: 7893
(cherry picked from commit 08f5abe5e9)
To have a system-level overview of when was the last time the file was
used, update the file modification timestamp to to the current time.
This is needed to remove stale cache files of the system.
Access time is not used as it may be, on the system level, disabled.
Ticket: 7893
(cherry picked from commit fd3847db72)
When mempool size was configured really low (<32), Suricata exhausted
the mempool with the rx_burst call, which led to undefined behavior.
The current fix ensures the burst size is at most the size of the mempool,
if the mempool is smaller than BURST_SIZE macro.
(cherry picked from commit 1d06103e08)
Add live tests to verify that mempool, mempool cache sizes, and
potentially other DPDK features are initialized correctly during the
startup.
Follows up on bugs found in mempool settings on real/bonded devices.
The tests are placed in the new folder qa/live/ after a comment
in https://github.com/OISF/suricata/pull/14896 which indicated other
live tests are going to be moved there as well.
Ticket: 8321
(cherry picked from commit d4434ef4e8)
DPDK interface which were added to bond and had auto-calculated
mempool size didn't have enough space as the configured
descriptors and the derived mempool only considered one interface
per YAML node entry.
DPDK Bonding PMD allows for multiple interface to cooperate and
therefore, needs more space.
Ticket: 8216
(cherry picked from commit ecd15a6c7e)
Following DPDK's recommendation to size mempool to 2^n - 1,
the current implementation was flawed for certain combos,
e.g. mp_size 1023 and 1 queue, which resulted to 1022.
(cherry picked from commit ea853e92a3)
Previously, the interface initialization always recalculated
cache size according to the per-queue computed mempool size.
Ticket: 8214
(cherry picked from commit 109742932a)
Currently in parse_request function LdapEvent::InvalidData is not set when a
request is not parsed correctly.
Ticket #8258
(cherry picked from commit de46f4ba9d)
Docs were claiming greater than, less than, etc. support even though this was never supported.
Fixes: 51be8f0238 ("doc/dcerpc: add proto keywords")
Ticket: #8330.
(cherry picked from commit 2480f8c038)
CID 1667318: (#1 of 1): Structurally dead code (UNREACHABLE)
unreachable: This code cannot be reached: hwloc_obj_t non_io_ancestor....
(cherry picked from commit b84ae80146)
Philippe Antoine
Victor Julien
Giuseppe Longo
Jason Ish
Lukas Sismis
Antoine Abou Faysal
Juliana Fajardini
Sven Cuyt