If a port point is single but later on also a part of a range, it ends
up only creating the port groups for single points and not the range.
Fix it by adding the port next to current single one to unique points
and marking it a range port.
Bug 6843
(cherry picked from commit 632ca75dd3)
Fix Coverity warning
** CID 1592992: Incorrect expression (COPY_PASTE_ERROR)
/src/util-port-interval-tree.c: 255 in SCPortIntervalFindOverlaps()
________________________________________________________________________________________________________
*** CID 1592992: Incorrect expression (COPY_PASTE_ERROR)
/src/util-port-interval-tree.c: 255 in SCPortIntervalFindOverlaps()
249 * will be sorted, insert any new ports to the end of the list
250 * and avoid walking the entire list */
251 if (*list == NULL) {
252 *list = new_port;
253 (*list)->last = new_port;
254 } else if (((*list)->last->port != new_port->port) &&
>>> CID 1592992: Incorrect expression (COPY_PASTE_ERROR)
>>> "port" in "(*list)->last->port2 != new_port->port" looks like a copy-paste error.
255 ((*list)->last->port2 != new_port->port)) {
256 DEBUG_VALIDATE_BUG_ON(new_port->port < (*list)->last->port);
257 (*list)->last->next = new_port;
258 new_port->prev = (*list)->last;
259 (*list)->last = new_port;
260 } else {
The code does not generate two port ranges that are same other than the
cases where port == port2 which is why it worked so far. Fix it.
Bug 6839
(cherry picked from commit 2d6708f1ff)
During startup large rulesets use a lot of large bitarrays, that
are frequently merged (OR'd).
Optimize this using SSE2 _mm_or_si128.
(cherry picked from commit 94b4619)
Make rule group head bitarray 16 bytes aligned and padded to 16 bytes
boundaries to assist SIMD operations in follow up commits.
(cherry picked from commit 4ba1f44e0d)
Instead of using in place insertion sort on linked list based on two
keys, convert the linked list to an array, perform sorting on it using
qsort and convert it back to a linked list. This turns out to be much
faster.
Ticket #6795
(cherry picked from commit e7e4305d91)
To avoid getting multiple entries in the final port list and to also
make the next step more efficient by reducing the size of the items to
traverse over.
Ticket 6792
Bug 6414
(cherry picked from commit 643ae85b5f)
As this is already taken care of and a list of ports is available for
use by the next stage.
Ticket 6792
Bug 6414
(cherry picked from commit 83aba93f40)
Using the unique port points, create a list of small port ranges which
contain the DetectPort objects and the designated SGHs found by finding
the overlaps with the existing ports and copying the SGHs accordingly.
Ticket 6792
Bug 6414
(cherry picked from commit 4ac2382f26)
After all the SGHs have been appropriately copied to the designated
ports, create an interval tree out of it for a faster lookup when later
a search for overlaps is made.
Ticket 6792
Bug 6414
(cherry picked from commit a02c44a3a4)
In order to create the smallest possible port ranges, it is convenient
to first have a list of unique ports. Then, the work becomes simple. See
below:
Given, a port range P1 = [1, 8]; SGH1
and another, P2 = [3, 94]; SGH2
right now, the code will follow a logic of recursively cutting port
ranges until we create the small ranges. But, with the help of unique
port points, we get, unique_port_points = [1, 3, 8, 94]
So, now, in a later stage, we can create the ranges as
[1, 2], [3, 7], [8, 8], [9, 94] and copy the designated SGHs where they
belong. Note that the intervals are closed which means that the range
is inclusive of both the points.
The final result becomes:
1. [1, 2]; SGH1
2. [3, 7]; SGH1 + SGH2
3. [8, 8]; SGH1 + SGH2
4. [9, 94]; SGH2
There would be 3 unique rule groups made for the case above.
Group 1: [1, 2]
Group 2: [3, 7], [8, 8]
Group 3: [9, 94]
Ticket 6792
Bug 6414
(cherry picked from commit c9a911b6f8)
Warning was:
src/util-port-interval-tree.c:50:1: warning: Either the condition 'tmp!=NULL' is redundant or there is possible null pointer dereference: tmp. [nullPointerRedundantCheck]
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Assuming that condition 'tmp!=NULL' is not redundant
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Null pointer dereference
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: warning: Either the condition 'oleft!=NULL' is redundant or there is possible null pointer dereference: oleft. [nullPointerRedundantCheck]
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Assuming that condition 'oleft!=NULL' is not redundant
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Null pointer dereference
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: warning: Either the condition 'oright!=NULL' is redundant or there is possible null pointer dereference: oright. [nullPointerRedundantCheck]
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Assuming that condition 'oright!=NULL' is not redundant
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Null pointer dereference
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: warning: Either the condition 'left!=NULL' is redundant or there is possible null pointer dereference: left. [nullPointerRedundantCheck]
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Assuming that condition 'left!=NULL' is not redundant
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Null pointer dereference
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
(cherry picked from commit 86f89e0966)
Add new utility files to deal with the interval trees. These cover the
basic ops:
1. Creation/Destruction of the tree
2. Creation/Destruction of the nodes
It also adds the support for finding overlaps for a given set of ports.
This function is used by the detection engine is the Stage 2 of
signature preparation.
Ticket 6792
Bug 6414
Co-authored-by: Victor Julien <vjulien@oisf.net>
(cherry picked from commit 54558f1b4a)
An interval tree uses red-black tree as its base data structure and
follows all the properties of a usual red-black tree. The additional
params are:
1. An interval such as [low, high] per node.
2. A max attribute per node. This attribute stores the maximum high
value of any subtree rooted at this node.
At any point in time, an inorder traversal of an interval tree should
give the port ranges sorted by the low key in ascending order.
This commit modifies the IRB_AUGMENT macro and it's call sites to make
sure that on every insertion, the max attribute of the tree is properly
updated.
Ticket 6792
Bug 6414
(cherry picked from commit d36d03a428)
In the commit 4a00ae607, the whitelisting check was updated in a quest
to make use of the conditional better but it made things worse as every
range would be whitelisted as long as it had any of the default
whitelisted port which is very common.
(cherry picked from commit fb9680bb7b)
When we only have stream matches.
Ticket: 6846
This solves the case where another transaction was created
by parsing data in the other direction, before running the
detection.
Like
1. get data in direction 1
2. acked data: parse it, but do not run detection in dir 1
3. other data in direction 2
4. other data acked : parse it and create new tx,
then run detection for direction 1 with data from first packet
(cherry picked from commit 7274ad58aa)
Recognize PPP_CCP, PPP_CBCP and PPP_COMP_DGRAM.
Does not implement decoders for these record types, so these
are logged as unsupported types. Was "wrong_type" before.
(cherry picked from commit 516441b600)
Commands would leave use_cnt incremented, never decrementing them. This
would lead to a asserting triggering at shutdown.
Bug: #7020.
(cherry picked from commit d02c57bd1f)
In the default config iface bypass support is not enabled,
and storage API not initialized for it. Using it will lead to a crash.
This commit first checks if the device storage API is initialized.
Bug: #7022.
(cherry picked from commit bc2dfe4c17)
Bug: https://redmine.openinfosecfoundation.org/issues/6782
Callers to these allocators often use ``sc_errno`` to provide context of
the error. And in the case of the above bug, they return ``sc_errno``,
but as it has not been set ``sc_errno = 0; == SC_OK``.
This patch simply sets this variable to ensure there is context provided
upon error.
(cherry picked from commit fc2e49f84a)
The on-disk pcap pkthdr is 16 bytes. This was calculated using
`sizeof(struct pcap_pkthdr)`, which is 24 bytes on 64 bit Linux. On
Macos, it's even worse, as a comment field grows the struct to 280
bytes.
Address this by hardcoding the value of 16.
Bug: #7037.
(cherry picked from commit 6c937a9243)
In offline mode, a timestamp is kept per thread, and the lowest
timestamp of the active threads is used. This was also considering the
non-packet threads, which could lead to the used timestamp being further
behind that needed. This would happen at the start of the program, as
the non-packet threads were set up the same way as the packet threads.
This patch both no longer sets up the timestamp for non-packet threads
as well as not considering non-packet threads during timestamp
retrieval.
Fixes: 6f560144c1 ("time: improve offline time handling")
Bug: #7034.
(cherry picked from commit 5455799795)
Issue: 6861
Without this commit, disabling rule profiling via suricatasc's command
'ruleset-profile-stop' may crash because profiling_rules_entered becomes
negative.
This can happen because
- There can be multiple rules evaluated for a single packet
- Each rule is profiled individually.
- Starting profiling is gated by a configuration setting and rule
profiling being active
- Ending profiling is gated by the same configuration setting and
whether the packet was marked as profiling.
The crash can occur when a rule is being profiled and rule profiling
is then disabled after one at least one rule was profiled for the packet
(which marks the packet as being profiled).
In this scenario, the value of profiling_rules_entered was
not incremented so the BUG_ON in the end profiling macro trips
because it is 0.
The changes to fix the problem are:
- In the profiling end macro, gate the actions taken there by the same
configuration setting and use the profiling_rues_entered (instead of
the per-packet profiling flag). Since the start and end macros are
tightly coupled, this will permit profiling to "finish" if started.
- Modify SCProfileRuleStart to only check the sampling values if the
packet hasn't been marked for profiling already. This change makes all
rules for a packet (once selected) to be profiled (without this change
sampling is applied to each *rule* that applies to the packet.
(cherry picked from commit bf5cfd6ab7)
HttpRangeOpenFileAux may return NULL in different cases, including
when memcap is reached.
But is only caller did not check it before calling HttpRangeAppendData
which would dereference the NULL value.
Ticket: 7029
(cherry picked from commit fd262df457)
Ticket: 7013
Done consistently for all protocols
This may change some protocols behaviors which failed early
if they found there was not enough data...
(cherry picked from commit 37a9003736)
As this triggers rustc 1.78
unsafe precondition(s) violated: slice::from_raw_parts requires
the pointer to be aligned and non-null,
and the total size of the slice not to exceed `isize::MAX`
Ticket: 7013
(cherry picked from commit 5dc8dea869)
Rules would allow checking against value 0, but internally the value
was used to indicate "no value". To address this, the internals now
return negative values for not found. This way value 0 can be fully
supported.
Bug: #6834.
(cherry picked from commit 64dc217f9f)
use brew instead of pip
limit the number of jobs for make
set a prefix where we can install
use brew flags for library finding
(cherry picked from commit 47a1502dbb)
Update the formatting CI job to Ubuntu 22.04 to get a newer version of
clang-format, in this case clang-format-14.
(cherry picked from commit 93071501b5)
The connp objects were incorrectly set per direction leading to
incorrect matches on respective directions.
Bug 6989
(cherry picked from commit 14e2c579f6)
Ticket: 6889
To avoid regexp dos with too much backtracking.
This is already done on pcre keyword, and pcrexform transform.
We use the same default limits for rules parsing.
(cherry picked from commit 316cc528f7)
Ticket: 6900
And thus avoid DOS by logging a request using a compressed
header block repeated many times and having a long value...
(cherry picked from commit 03442c9071)
Ticket: 6892
As HTTP hpack header compression allows one single byte to
express a previously seen arbitrary-size header block (name+value)
we should avoid to copy the vectors data, but just point
to the same data, while reamining memory safe, even in the case
of later headers eviction from the dybnamic table.
Rust std solution is Rc, and the use of clone, so long as the
data is accessed by only one thread.
(cherry picked from commit 390f09692e)
Shivani Bhardwaj
Victor Julien
Philippe Antoine
Richard McConnell
Jeff Lucovsky
Jason Ish