Ticket: 6481
Instead of just setting the old transactions to a drop state so
that they get later cleaned up by Suricata, fail creating new ones.
This is because one call to app-layer parsing can create many
transactions, and quadratic complexity could happen in one
single app-layer parsing because of find_or_create_tx
(cherry picked from commit 80abc22f64)
A next PDU may already be in the slice to parse.
Do not skip its parsing, ie do not use rest, but take just
the length of the pdu
(cherry picked from commit 86de7cffa7)
If the next PDU is already in the slice next, do not use it and
restrict ourselves to the length of this PDU.
Avoids overconsumption of memory by quadratic complexity, when
having many small PDUS in one big chunk being parsed
Ticket: #6411
(cherry picked from commit f52c033e56)
Ticket: #6441
This keyword and the response one use a multiple inspection buffer.
But the different instances point to the same memory address
that comes from HttpHeaderGetBufferSpace and is not owned
by the transaction, and is rebuilt, which is a functional
bug in itself.
As it gets crafted, it can get reallocated if one header
is over 1024 bytes, while the previous freed pointer will still get
used for the previous headers.
(cherry picked from commit bc422c17d6)
Ticket: 6477
So as to avoid ending up with too many empty transactions.
This happens when Suricata sees a DATA command in the current
transaction but did not have a confirmation response for it.
Then, if Suricata receives another DATA command, it will
create another new transaction, even if the previous one
is empty. And so, a malicious client can create many empty
transactions by just sending a repeated amount of DATA commands
without having a confirmation code for them.
Suricata cannot use state->current_command == SMTP_COMMAND_DATA
to prevent this attack and needs to resort to a new boolean
is_data because the malicious client may send another dummy command
after each DATA command.
This patch leaves only one call to SMTPTransactionCreate
(cherry picked from commit 61f2e4e1e5)
Ticket: 5926
HTTP2 continuation frames are defined in RFC 9113.
They allow header blocks to be split over multiple HTTP2 frames.
For Suricata to process correctly these header blocks, it
must do the reassembly of the payload of these HTTP2 frames.
Otherwise, we get incomplete decoding for headers names and/or
values while decoding a single frame.
Design is to add a field to the HTTP2 state, as the RFC states that
these continuation frames form a discrete unit :
> Field blocks MUST be transmitted as a contiguous sequence of frames,
> with no interleaved frames of any other type or from any other stream.
So, we do not have to duplicate this reassembly field per stream id.
Another design choice is to wait for the reassembly to be complete
before doing any decoding, to avoid quadratic complexity on partially
decoding of the data.
(cherry picked from commit aff54f29f8)
Especially sets transactions to complete when we get a response
without having seen the request, so that the transactions
end up getting cleaned (instead of living/leaking in the state).
Also try to set the event on the relevant transaction, instead
of creating a new transaction just for the purpose of having
the event.
Ticket: #6299
(cherry picked from commit 89936b6530)
When a TCP flow packet has not led to app-layer updates,
it is useless to run DetectRunTx, as there cannot be new
matches.
This happens for instance, when one side sends in a row multiple
packets which are not acked (and thus not parsed in IDS mode).
Doing so requires to move up the call to
AppLayerParserSetTransactionInspectId
so that it is run the same times DetectRunTx is run, and not in the
case where the transaction was not updated.
Ticket: 6299
(cherry picked from commit 9240ae250c)
"sigerror_ok" and "sigerror_requires" were not being reset after each
rule which could lead to a rule load error being incorrectly tracked
as skipped rather than failed.
Also initialize "skippedsigs" to 0 along with "goodsigs" and
"badsigs", while not directly related to this issue, could also throw
off some stats.
Ticket: #6710
(cherry picked from commit de3cbe4c90)
Remove references that are mentioning Suricata 3 or less
As a note - only one Suricata 4 reference found:
(suricata-yaml.rst:"In 4.1.x")
Fast pattern selection criteria can be internally found by inspecting
SupportFastPatternForSigMatchList and SigTableSetup functions.
Ticket: #6699
(cherry picked from commit 6e4cc79b39)
Previous integration of hugepage analysis only fetched data
from /proc/meminfo. However this proved to be often
deceiving mainly for providing only global information and
not taking into account different hugepage sizes (e.g. 1GB
hugepages) and different NUMA nodes.
Ticket: #6697
(cherry picked from commit ca6f7c2d00)
When running Suricata in XDP bypass mode (bypass: yes),
Suricata started up with error:
Error: threads: thread "FB" failed to start in time: flags 0003
"FB" thread does not transition from THV_INIT_DONE to THV_RUNNING.
Set "FB" thread THV_RUNNING state in BypassedFlowManager().
Bug: #6254
Signed-off-by: Vincent Li <vincent.mc.li@gmail.com>
(cherry picked from commit f80d26db0b)
The shutdown(2) syscall would always return ENOTCONN for FreeBSD 11,
FreeBSD 12, FreeBSD 13 and FreeBSD 14. It could do some action on the
socket in the kernel in FreeBSD 10 and before, did not test.
(cherry picked from commit b239e88c93)
During the pre-scan for "requires", also parse the SID if possible. If
the rule fails high level parsing (syntax), the SID will not be
parsed.
But every keyword other than "sid" and "requires" should expect to be
provided with a parsed sid.
(cherry picked from commit 71bbba9248)
Add a "pre-scan" rule parse that will check for requires statement. It
will return a special error code (-4) if the requires fails due to
missing requirements.
Syntactic errors will also abort parsing here.
Feature: #5972
(cherry picked from commit 435c03172e)
Add a new rule keyword "requires" that allows a rule to require specific
Suricata versions and/or Suricata features to be enabled.
Example:
requires: feature geoip, version >= 7.0.0, version < 8;
requires: version >= 7.0.3 < 8
requires: version >= 7.0.3 < 8 | >= 8.0.3
Feature: #5972
Co-authored-by: Philippe Antoine <pantoine@oisf.net>
(cherry picked from commit 5d5b0509a5)
In our conf.py we reference some ReadTheDocs stylesheets that appear to
be old and break formatting of some items like bulletted lists.
Bug: #6589
(cherry picked from commit cc0adaaf4a)
Parallel builds caused issues during `cargo vendor`. So do just a single
thread build.
make[4]: Entering directory '/__w/suricata/suricata/rust'
cbindgen --config /__w/suricata/suricata/rust/cbindgen.toml \
--quiet --output /__w/suricata/suricata/rust/dist/rust-bindings.h
CARGO_HOME="/github/home/.cargo" /usr/bin/cargo vendor
Blocking waiting for file lock on package cache
Blocking waiting for file lock on package cache
ERROR: Couldn't execute `cargo metadata` with manifest "/__w/suricata/suricata/rust/Cargo.toml": Metadata(Output { status: ExitStatus(unix_wait_status(25856)), stdout: "", stderr: " Blocking waiting for file lock on package cache\n Blocking waiting for file lock on package cache\nerror: failed to download `adler v1.0.2`\n\nCaused by:\n unable to get packages from source\n\nCaused by:\n failed to parse manifest at `/github/home/.cargo/registry/src/github.com-1ecc6299db9ec823/adler-1.0.2/Cargo.toml`\n\nCaused by:\n no targets specified in the manifest\n either src/lib.rs, src/main.rs, a [lib] section, or [[bin]] section must be present\n" })
ERROR: Couldn't generate bindings for /__w/suricata/suricata/rust.
make[4]: *** [Makefile:597: dist/rust-bindings.h] Error 1
make[4]: *** Waiting for unfinished jobs....
(cherry picked from commit c82d93490c)
To avoid future problems with overlapping flag values, give bytejump
its own DETECT_BYTEJUMP_OFFSET_VAR flag.
The values are currently not overlapping, so this patch should have
no side effects.
(cherry picked from commit 101452056d)
When reading a pcap, packet time can move much faster than wall
clock time. This would trigger many more profile syncs than before.
As the sync is using a lock to synchronize with other threads, this
is an expensive operation.
Bug: #6619.
Fixes: b591813b86 ("profiling/rules: reduce sync logic scope")
(cherry picked from commit bcb2b50cfc)
Use jb_append_string_from_bytes() as it works better than
BytesToString+jb_append_string when logging binary data.
Bug: #6664.
(cherry picked from commit f5565f42e7)
Issue: 6439
Clarify the transform validation step. When a transform indicates that
the content/byte-array is not compatible, validation will stop.
Content is incompatible is some cases -- e.g., following the
to_lowercase transform with content containing uppercase characters.
An alert is not possible since the content contains uppercase and the
transform has converted the buffer into all lowercase.
(cherry picked from commit a46779d866)
Shivani Bhardwaj
Philippe Antoine
Jason Ish
Lukas Sismis
Vincent Li
Gleb Smirnoff
Stephen Donnelly
Jeff Lucovsky
Eric Leblond
Juliana Fajardini
Victor Julien