Commit Graph

15500 Commits (2d6708f1ff1e57f83f2dd9994173e8ee62e00616)
 

Author SHA1 Message Date
Shivani Bhardwaj 2d6708f1ff util/interval-tree: fix coverity warning
Fix Coverity warning

** CID 1592992:  Incorrect expression  (COPY_PASTE_ERROR)
/src/util-port-interval-tree.c: 255 in SCPortIntervalFindOverlaps()

________________________________________________________________________________________________________
*** CID 1592992:  Incorrect expression  (COPY_PASTE_ERROR)
/src/util-port-interval-tree.c: 255 in SCPortIntervalFindOverlaps()
249                      * will be sorted, insert any new ports to the end of the list
250                      * and avoid walking the entire list */
251                     if (*list == NULL) {
252                         *list = new_port;
253                         (*list)->last = new_port;
254                     } else if (((*list)->last->port != new_port->port) &&
>>>     CID 1592992:  Incorrect expression  (COPY_PASTE_ERROR)
>>>     "port" in "(*list)->last->port2 != new_port->port" looks like a copy-paste error.
255                                ((*list)->last->port2 != new_port->port)) {
256                         DEBUG_VALIDATE_BUG_ON(new_port->port < (*list)->last->port);
257                         (*list)->last->next = new_port;
258                         new_port->prev = (*list)->last;
259                         (*list)->last = new_port;
260                     } else {

The code does not generate two port ranges that are same other than the
cases where port == port2 which is why it worked so far. Fix it.

Bug 6839
2 years ago
Jeff Lucovsky af529a56a9 netmap: Avoid HW errors when using pipes
Issue: 6837

When using netmap pipes (with lb, for example), avoid direct hardware
related IOCTLs that will fail (not supported with pipes).
2 years ago
Arne Welzel f9cf87a003 schema: Add stats.capture and in_iface properties
New suricata-verify test listens on loopback interface, resulting
in the capture and in_iface fields in the stats and event objects.
2 years ago
Arne Welzel f17204191d stats: Fix non-worker stats missing
Commit b8b8aa69b4 used tm_name of the
first StatsRecord of a thread block as key for the "threads" object.
However, depending on the type of thread, tm_name can be NULL and would
result in no entry being included for that thread at all. This caused
non-worker metrics to vanish from the "threads" object in the
dump-counters output.

This patch fixes this by remembering the first occurrence of a valid
tm_name within the per-thread block and adds another unittest to
cover this scenario.
2 years ago
Victor Julien 1d3a156179 rust: update parser dependencies
Time locked to 0.3.20 to guarantee MSRV of 1.63.
Update snmp-parser to 0.10.0.
Update asn1-rs to 0.6.1.
Update kerberos-parser to 0.8.0.
Update x509-parser 0.16.0.
Update der-parser to 9.0.0.
Remove specific use of der-parser 6.

Ticket: #6817.
Ticket: #6818.
2 years ago
Shivani Bhardwaj 4afaadc0ce detect/port: cleanup address artifacts
A lot of code uses variable names and comments derived from the code
about addresses, make them about port.
2 years ago
Shivani Bhardwaj e14fefe24a detect/port: remove SigGroupHead* ops
The functions in detect-engine-port.c are only being used at the time of
parsing the ports from rules initially. Since there are no SGHs at that
point, remove the ops related to them too.
2 years ago
Victor Julien c4ac6cdc9a detect: optimize sig_cnt setting
Utilize _popcnt64 where available.
2 years ago
Victor Julien 94b4619bb9 detect: optimize group head bitarray handling
During startup large rulesets use a lot of large bitarrays, that
are frequently merged (OR'd).

Optimize this using SSE2 _mm_or_si128.
2 years ago
Victor Julien 4ba1f44e0d detect: prepare for SIMD optimizations
Make rule group head bitarray 16 bytes aligned and padded to 16 bytes
boundaries to assist SIMD operations in follow up commits.
2 years ago
Victor Julien e7e4305d91 detect/port: use qsort instead of insert sort
Instead of using in place insertion sort on linked list based on two
keys, convert the linked list to an array, perform sorting on it using
qsort and convert it back to a linked list. This turns out to be much
faster.

Ticket #6795
2 years ago
Shivani Bhardwaj 643ae85b5f detect/port: merge port ranges for same signatures
To avoid getting multiple entries in the final port list and to also
make the next step more efficient by reducing the size of the items to
traverse over.

Ticket 6792
Bug 6414
2 years ago
Shivani Bhardwaj 83aba93f40 detect/port: remove the port cut/insertion stage
As this is already taken care of and a list of ports is available for
use by the next stage.

Ticket 6792
Bug 6414
2 years ago
Shivani Bhardwaj 4ac2382f26 detect/port: create list of small port ranges
Using the unique port points, create a list of small port ranges which
contain the DetectPort objects and the designated SGHs found by finding
the overlaps with the existing ports and copying the SGHs accordingly.

Ticket 6792
Bug 6414
2 years ago
Shivani Bhardwaj a02c44a3a4 detect/port: create a tree of given ports
After all the SGHs have been appropriately copied to the designated
ports, create an interval tree out of it for a faster lookup when later
a search for overlaps is made.

Ticket 6792
Bug 6414
2 years ago
Shivani Bhardwaj c9a911b6f8 detect/port: find unique port points
In order to create the smallest possible port ranges, it is convenient
to first have a list of unique ports. Then, the work becomes simple. See
below:

Given, a port range P1 = [1, 8]; SGH1
and another, P2 = [3, 94]; SGH2

right now, the code will follow a logic of recursively cutting port
ranges until we create the small ranges. But, with the help of unique
port points, we get, unique_port_points = [1, 3, 8, 94]

So, now, in a later stage, we can create the ranges as
[1, 2], [3, 7], [8, 8], [9, 94] and copy the designated SGHs where they
belong. Note that the intervals are closed which means that the range
is inclusive of both the points.

The final result becomes:
1. [1, 2]; SGH1
2. [3, 7]; SGH1 + SGH2
3. [8, 8]; SGH1 + SGH2
4. [9, 94]; SGH2

There would be 3 unique rule groups made for the case above.
Group 1: [1, 2]
Group 2: [3, 7], [8, 8]
Group 3: [9, 94]

Ticket 6792
Bug 6414
2 years ago
Shivani Bhardwaj 86f89e0966 util/interval-tree: suppress cppcheck warnings
Warning was:
src/util-port-interval-tree.c:50:1: warning: Either the condition 'tmp!=NULL' is redundant or there is possible null pointer dereference: tmp. [nullPointerRedundantCheck]
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Assuming that condition 'tmp!=NULL' is not redundant
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Null pointer dereference
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: warning: Either the condition 'oleft!=NULL' is redundant or there is possible null pointer dereference: oleft. [nullPointerRedundantCheck]
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Assuming that condition 'oleft!=NULL' is not redundant
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Null pointer dereference
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: warning: Either the condition 'oright!=NULL' is redundant or there is possible null pointer dereference: oright. [nullPointerRedundantCheck]
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Assuming that condition 'oright!=NULL' is not redundant
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Null pointer dereference
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: warning: Either the condition 'left!=NULL' is redundant or there is possible null pointer dereference: left. [nullPointerRedundantCheck]
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Assuming that condition 'left!=NULL' is not redundant
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
src/util-port-interval-tree.c:50:1: note: Null pointer dereference
IRB_GENERATE(PI, SCPortIntervalNode, irb, SCPortIntervalCompareAndUpdate);
^
2 years ago
Shivani Bhardwaj 54558f1b4a util/interval-tree: add utility fns
Add new utility files to deal with the interval trees. These cover the
basic ops:
1. Creation/Destruction of the tree
2. Creation/Destruction of the nodes

It also adds the support for finding overlaps for a given set of ports.
This function is used by the detection engine is the Stage 2 of
signature preparation.

Ticket 6792
Bug 6414

Co-authored-by: Victor Julien <vjulien@oisf.net>
2 years ago
Shivani Bhardwaj a08e065c7c detect/port: make DetectPortInit non static
as this fn will be called upon and further used by other files later on.

Ticket 6792
Bug 6414
2 years ago
Shivani Bhardwaj d36d03a428 interval-tree: add augmentation fns to the tree
An interval tree uses red-black tree as its base data structure and
follows all the properties of a usual red-black tree. The additional
params are:
1. An interval such as [low, high] per node.
2. A max attribute per node. This attribute stores the maximum high
   value of any subtree rooted at this node.

At any point in time, an inorder traversal of an interval tree should
give the port ranges sorted by the low key in ascending order.

This commit modifies the IRB_AUGMENT macro and it's call sites to make
sure that on every insertion, the max attribute of the tree is properly
updated.

Ticket 6792
Bug 6414
2 years ago
Shivani Bhardwaj 30b6e4d368 interval-tree: remove splay tree implementation
Ticket 6792
Bug 6414
2 years ago
Shivani Bhardwaj fde4ca5608 interval-tree: add base data structure
Ticket 6792
Bug 6414
2 years ago
Victor Julien fb9680bb7b detect/engine: fix whitelisting check
In the commit 4a00ae607, the whitelisting check was updated in a quest
to make use of the conditional better but it made things worse as every
range would be whitelisted as long as it had any of the default
whitelisted port which is very common.
2 years ago
Lukas Sismis 566e89a0d6 dpdk: output unknown NIC's NUMA node message only on multi-node systems
Ticket: #6715
2 years ago
dependabot[bot] c6c1eac301 github-actions: bump actions/download-artifact from 4.1.3 to 4.1.4
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 4.1.3 to 4.1.4.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](87c55149d9...c850b930e6)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2 years ago
dependabot[bot] f1b0f7c46b github-actions: bump github/codeql-action from 3.24.5 to 3.24.6
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.24.5 to 3.24.6.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Commits](https://github.com/github/codeql-action/compare/v3.24.5...v3.24.6)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2 years ago
Jason Ish 0dc3de332a examples: minimal example capture plugin for ci
Create a mininal capture plugin that injects one packet. While it can
also be a template, we should be able to run this in CI to test the
loading and registration of the capture plugin mechanisms.
2 years ago
Jason Ish acfc1c9395 threads: tm-modules.h depends on Packet so include decode.h 2 years ago
Jason Ish 25d0e0b68c plugins: initialize plugins earlier
Capture plugins need to be registered before LiveDeviceFinalize,
otherwise Suricata errors before the plugin gets a chance to load.

Bug: #6811
2 years ago
Jason Ish 96e61b043c thread modules: separate initialization from registration
Move the zero'ing to the thread module array InitGlobal in an effort
to fix capture modules.

At some point device validation moved to a point in startup before
plugins are loaded meaning that capture plugins could not be
used. Moving plugin registration early enough caused some of their
registration to be wiped out as clearing the array was done after.

Bug: #6811
2 years ago
Jason Ish c2ecae9b82 schema: add flow.wrong_thread 2 years ago
Victor Julien 6d0e11e76c dependabot: reduce to monthly update 2 years ago
Hadiqa Alamdar Bukhari 4b81851097 dns: add dns.rrtype keyword
It matches the rrtype field in DNS
It's an unsigned integer match
valid ranges = [0-65535]
Does not support prefilter
Supports flow in both directions
Feature #6666
2 years ago
Jason Ish 5f0853599b pfring: fix leak of configuration data and in single mode
Fix leak of configuration data on exit. Also, in single mode set
thread count to one instead of the CPU count.

Bug: #4734
2 years ago
Jason Ish 358f1e35ee pfring: fix leak of packet on exit
Bug: #4734
2 years ago
Philippe Antoine d255a5c7a3 output/filestore: delay snprintf until needed
Perf optimization so that we do not call snprintf in the
common code path.

Ticket: 6796
2 years ago
Philippe Antoine 7641c07af9 output/filestore: remove duplicate snprintf
Ticket: 6796
2 years ago
Philippe Antoine c41540f839 output: log tx only when there is a fresh app update
Ticket: 6796

Similar to commit for detection
9240ae250c

We only have more logging to do if the app update was fresh,
ie if p->app_update_direction != 0

If we have data acknowledged in one direction,
and then many packets in the other direction,
the APP_UPDATED flow flags did not get reset because we did not
run detection yet in this direction,
but there is nothing more to do after the first packet in the
other direction.
2 years ago
dependabot[bot] c283e8565a github-actions: bump codecov/codecov-action from 4.0.1 to 4.1.0
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 4.0.1 to 4.1.0.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](e0b68c6749...54bcd8715e)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2 years ago
dependabot[bot] 13da6498b5 github-actions: bump actions/download-artifact from 4.1.2 to 4.1.3
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 4.1.2 to 4.1.3.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](eaceaf801f...87c55149d9)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2 years ago
Giuseppe Longo 3dc24a967a doc: add upgrade section for 8 2 years ago
Giuseppe Longo fe77def816 rust/sip: register pattern matching
This permits to detect the SIP protocol using pattern matching instead of
probing parser.

Since it is no longer used, the respective probing functions have been removed.
2 years ago
Giuseppe Longo add95002b9 suricata.yaml: define SIP_PORTS 2 years ago
Giuseppe Longo 9c9b1a4230 rust/sip: add direction to transaction
This patch permits to set a direction when a new transaction is created in order
to avoid 'signature shadowing' as reported by Eric Leblond in commit
5aaf50760f
2 years ago
Giuseppe Longo c9d309219e rust/sip: register parser for tcp
This patch lets the parser to work over tcp protocol, taking care of handling
data before calling the request/response parsers.

Ticket #3351.
2 years ago
Giuseppe Longo 69f841c998 sip/parser: enforce valid chars for sip version
The `is_version_char` function incorrectly allowed characters that are not
part of the valid SIP version "SIP/2.0".

For instance, 'HTTP/1.1' was mistakenly accepted as a valid SIP version,
although it's not.

This commit fixes the issue by updating the condition to strictly
check for the correct version string.
2 years ago
Giuseppe Longo 7e993d5081 sip/parser: accept valid chars
Accepts valid characters as defined in RFC3261.
2 years ago
Giuseppe Longo 8ff80cb84d rust/sip: rustfmt sip module 2 years ago
Philippe Antoine 497394eec6 detect: do not run tx detection on non established packets
Follows commit 2fb5059

Ticket: 6775
2 years ago
Philippe Antoine e22217bda8 doc: there is no right shift for integer bitmasks
Ticket: 6628
2 years ago