In multi instance flow manager setups, each flow manager gets a slice
of the hash table to manage. Due to a logic error in the chunked
scanning of the hash slice, instances beyond the first would always
rescan the same (first) subslice of their slice.
The `pos` variable that is used to keep the state of what the starting
position for the next scan was supposed to be, was treated as if it held
a relative value. Relative to the bounds of the slice. It was however,
holding an absolute position. This meant that when doing it's bounds
check it was always considered out of bounds. This would reset the sub-
slice to be scanned to the first part of the instances slice.
This patch addresses the issue by correctly handling the fact that the
value is absolute.
Bug: #7365.
Fixes: e9d2417e0f ("flow/manager: adaptive hash eviction timing")
(cherry picked from commit ae072d5c07)
Uncomment in default config. This will make the policy "inline",
which is the same behavior as prior to the urgent policy support.
Add line to docs that this is an experimental feature.
If TCP urgent handling is set to the OOB processing, the number of OOB
bytes is tracked for SEQ offset calculations. If this offset is
non-zero, add the field to the flow record.
Ticket: #7411.
(cherry picked from commit 779f9d8ba3)
TCP urgent handling is a complex topic due to conflicting RFCs and
implementations.
Until now the URG flag and urgent pointer values were simply ignored,
leading to an effective "inline" processing of urgent data. Many
implementations however, do not default to this behavior.
Many actual implementations use the urgent mechanism to send 1 byte of
data out of band to the application.
Complicating the matter is that the way the urgent logic is handled is
generally configurable both of the OS and the app level. So from the
network it is impossible to know with confidence what the settings are.
This patch adds the following policies:
`stream.reassembly.urgent.policy`:
- drop: drop URG packets before they affect the stream engine
- inline: ignore the urgent pointer and process all data inline
- oob (out of band): treat the last byte as out of band
- gap: skip the last byte, but do no adjust sequence offsets, leading to
gaps in the data
For the `oob` option, tracking of a sequence number offset is required,
as the OOB data does "consume" sequence number space. This is limited to
64k. For this reason, there is a second policy:
`stream.reassembly.urgent.oob-limit-policy`:
- drop: drop URG packets before they affect the stream engine
- inline: ignore the urgent pointer and process all data inline
- gap: skip the last byte, but do no adjust sequence offsets, leading to
gaps in the data
Bug: #7411.
(cherry picked from commit 6882bcb3e5)
Add events for the following resource name parsing issues:
- name truncated as its too long
- maximum number of labels reached
- infinite loop
Currently these events are only registered when encountered, but
recoverable. That is where we are able to return some of the name,
usually in a truncated state.
As name parsing has many code paths, we pass in a pointer to a flag
field that can be updated by the name parser, this is done in
addition to the flags being set on a specific name as when logging we
want to designate which fields are truncated, etc. But for alerts, we
just care that something happened during the parse. It also reduces
errors as it won't be forgotten to check for the flags and set the
event if some new parser is written that also parses names.
Ticket: #7280
(cherry picked from commit 19cf0f8133)
If rrname, rdata or mname are truncated, set a flag field like
'rrname_truncated: true' to indicate that the name is truncated.
Ticket: #7280
(cherry picked from commit 37f4c52b22)
Once a name has gone over 1025 chars it will be truncated to 1025
chars and no more labels will be added to it, however the name will
continue to be parsed up to the label limit in attempt to find the end
so parsing can continue.
This introduces a new struct, DNSName which contains the name and any
flags which indicate any name parsing errors which should not error
out parsing the complete message, for example, infinite recursion
after some labels are parsed can continue, or truncation of name where
compression was used so we know the start of the next data to be
parsed.
This limits the logged DNS messages from being over our maximum size
of 10Mb in the case of really long names.
Ticket: #7280
(cherry picked from commit 3a5671739f)
Ticket: 7393
Check if GrowRegionToSize is called with an argument
trying to shrink the region size, and if so do nothing,
ie do not try to shrink, and just return ok.
This way, we avoid a buffer overflow from memeset using an
unsigned having underflowed.
(cherry picked from commit 9a53ec43b1)
Ticket: 7393
As it was possible before earlier patches to get here
with mem_size lesser than start->buf_size,
which caused then an unsigned underflow and a buffer overflow.
(cherry picked from commit 8900041405)
This was not a problem for current callers in Suricata,
as RegionsIntersect is only called through StreamingBufferInsertAt
which is only used by TCP...
And TCP uses default region gap = 256kb, and only calls
StreamingBufferInsertAt with a u16, so TCP never inserts a new
data that will strictly contain an existing region augmented
with region gap, which was the only case where RegionsIntersect
returned the wrong result, which could later lead to a
buffer overflow.
Ticket: 7393
(cherry picked from commit 282509f70c)
The `guess-applayer-tx` work also removed the stream match condition
for adding app-layer metadata to alerts. This is a behavior change that
is not desired at this point, so this commit reverts that part of the
changes.
We keep the exising logging of app-layer metadata if the match was in
the stream.
Ticket: 7199
Uses a config parameter detect.guess-applayer-tx to enable
this behavior (off by default)
This feature is requested for use cases with signatures not
using app-layer keywords but still targetting application
layer transactions, such as pass/drop rule combination,
or lua usage.
This overrides the previous behavior of checking if the signature
has a content match, by checking if there is only one live
transaction, in addition to the config parameter being set.
(cherry picked from commit f2c3776314)
Last packet from the TLS TCP session moves TCP state to CLOSED.
This flags the app-layer with APP_LAYER_PARSER_EOF_TS or
APP_LAYER_PARSER_EOF_TC depending on the direction of the final packet.
This flag will just have been set in a single direction.
This leads to the last packet updating the inspect id in that packets
direction.
At the end of the TLS session a pseudo packet is created, because:
- flow has ended
- inspected tx id == 0, for at least one direction
- total txs is 1
Then a packet rule matches:
```
alert tcp any any -> any 443 (flow: to_server; \
flowbits:isset,tls_error; \
sid:09901033; rev:1; \
msg:"Allow TLS error handling (outgoing packet)"; )
```
The `SIG_MASK_REQUIRE_REAL_PKT` is not preventing the match, as the
`flowbits` keyword doesn't set it.
To avoid this match. This patch skips signatures of the `SIG_TYPE_PKT`
for flow end packets.
Ticket: #7318.
(cherry picked from commit 0e4faba79a)
ICE driver (Intel E810 NIC) requires/supports 52-byte long RSS key.
The 52 byte key length was mandatory from DPDK 23.11 when Suricata
was starting with independently configured ice PMD.
However, Suricata failed to start when ice PMD was part of
net_bonding PMD, requiring 52 byte RSS key even in DPDK versions
lower than 23.11. Since the support for the longer key is present
since DPDK 19.11 the key is set to 52 bytes for all versions.
Ticket: 7445
(cherry picked from commit 18ab9a6ccd)
Similar keywords use `isnotset`, while `flowint` only accepted `notset`
Opted to change the code, not only the regex, to keep the underlying
code also following the same patterns.
Task #7426
(cherry picked from commit 6e4a501e7c)
Default decoder event alert was very sparse, not even logging packet
type and pcap_cnt. Expand support for this record type. It will be more
useful with the ethernet headers and packet field, but these are still
disabled by default.
Ticket: #7433.
(cherry picked from commit 2fe2cf8553)
The new behavior in 8, and backported is to treat unknown requirements
as unsatisfied requirements.
For 7.0.8, add a configuration option, "ignore-unknown-requirements"
to completely ignore unknown requirements, effectively treating them
as available.
Ticket: #7434
For example, "requires: foo bar" is an unknown requirement, however
its not tracked, nor an error as it follows the syntax. Instead,
record these unknown keywords, and fail the requirements check if any
are present.
A future version of Suricata may have new requires keywords, for
example a check for keywords.
Ticket: #7418
(cherry picked from commit 820a3e51b7)
Includes Cargo.lock.in generated for just this single crate update
(minimal atomic update to keep Cargo.lock in sync with Cargo.toml).
This prevents the clippy warning:
508 | #[derive(FromPrimitive, Debug)]
| ^------------
| |
| `FromPrimitive` is not local
| move the `impl` block outside of this constant `_IMPL_NUM_FromPrimitive_FOR_IsakmpPayloadType`
509 | pub enum IsakmpPayloadType {
| ----------------- `IsakmpPayloadType` is not local
|
= note: the derive macro `FromPrimitive` defines the non-local `impl`, and may need to be changed
= note: the derive macro `FromPrimitive` may come from an old version of the `num_derive` crate, try updating your dependency with `cargo update -p num_derive`
= note: an `impl` is never scoped, even when it is nested inside an item, as it may impact type checking outside of that item, which can be the case if neither the trait or the self type are at the same nesting level as the `impl`
= note: items in an anonymous const item (`const _: () = { ... }`) are treated as in the same scope as the anonymous const's declaration for the purpose of this lint
= note: this warning originates in the derive macro `FromPrimitive` (in Nightly builds, run with -Z macro-backtrace for more info)
Backport of 8e408d3730.
In the situation where the mem buffer cannot be expanded to the
requested size, drop the log message.
For each JSON log context, a warning will be emitted once with a partial
bit of the log record being dropped to identify what event types may be
leading to large log records.
This also fixes the call to MemBufferExpand which is supposed be
passed the amount to expand by, not the new size required.
Ticket: #7300
(cherry picked from commit d39e42728a)
If there is a transform before dotprefix, it operates in place
in a single buffer, and must therefore use memmove instead of memcpy
to avoid UB.
Ticket: 7229
instead of writing to a temporary buffer and then copying,
to save the cost of copying.
Ticket: 7229
Not a cherry-pick as we do not put the transforms in rust,
but just do this optimization in C
Rename InvalidHTTP1Settings to InvalidHttp1Settings so it gets the
expected name transformation of "invalid_http1_settings".
Ticket: #7361
(cherry picked from commit b1c26dccf3)
Ticket: 7326
Having a lower progress than one where we actually can get
occurences of the multibuffer made prefilter
bail out too early, not having found a buffer in the multi-buffer
that matiched the prefilter.
For example, we registered http_request_header with progress 0
instad of progress HTP_REQUEST_HEADERS==2, and if the first
packet had only the request line, we would consider
that signatures with http_request_header as prefilter/fast_pattern
could not match for this transaction, even if they in fact
could have a later packet with matching headers.
Hence, we got false negatives, if http.request_header or
http.response_header was used as fast pattern, and if the request
or response came in multiple packets, and the first of these packets
did not have enough data (like only http request line),
and the next packets did have the matching data.
(cherry picked from commit cca59cdaa9)
Rust was using i8 as the return type, while C uses int. As of Rust
1.82, the return value is turned to garbage over the FFI boundary.
Ticket: #7338
(cherry picked from commit 45384ef969)
Victor Julien
Shivani Bhardwaj
Jason Ish
Philippe Antoine
Lukas Sismis
Juliana Fajardini
Sascha Steinbiss
Jeff Lucovsky