Commit Graph

15587 Commits (suricata-7.0.11)
 

Author SHA1 Message Date
Jason Ish 1e110d0a71 defrag: fix subsequent overlap of start of original (bsd)
Fix the BSD policy case where a subsequent fragment starts before an
original fragment and overlaps the beginning of the original
fragment. In this case the overlapping data from the new fragment is
preferred.

Suricata was preferring the data from the original fragment, but it
should only do that when the original fragment has an offset <= to the
new fragment.

- Adds test for this case

Bug: #6669
(cherry picked from commit f1709ea551)
1 year ago
Jason Ish 2f39ba75f1 defrag: check next fragment for overlap before stopping re-assembly
Instead of breaking the loop when the current fragment does not have
any more fragments, set a flag and continue to the next fragment as
the next fragment may have data that occurs before this fragment, but
overlaps it.

Then break if the next fragment does not overlap the previous.

Bug: #6668
(cherry picked from commit d0fd078250)
1 year ago
Jason Ish 6578f74a95 defrag: use uint8_t in unit tests
(cherry picked from commit bdd17de73d)
1 year ago
Jason Ish 7137d5e7ab defrag: consistent unit test naming
Use a more consistent naming scheme between ipv4 and ipv6.

(cherry picked from commit 2f00b5870a)
1 year ago
Jason Ish 54ffe12f06 defrag: make tests more readable
Make tests more readable for comparing to the paper "Target-Based
Fragmentation Reassembly".

(cherry picked from commit 6339deadce)
1 year ago
Jason Ish bcc7343c0c defrag: minor cleanups
- typo in comment
- remove debug function that is not used and no longer valid

(cherry picked from commit 276d3d6541)
1 year ago
Victor Julien 04b8a7abbe ips: check for livedev.use-for-tracking
For the capture methods that support livedev and IPS,
livedev.use-for-tracking is not supported.

This setting causes major flow tracking issues, as both sides of
a flow would be tracked in different flows.

This patch disables the livedev.use-for-tracking setting if it
is set to true. A warning will be issued.

Ticket: #6726.
(cherry picked from commit 08841f27ff)
1 year ago
Victor Julien b8739ea13f capture: improve IDS + IPS check
Improve it for af-packet, dpdk, netmap. Check would not consider
an interface IDS if the `default` section contained a copy-mode
field.

(cherry picked from commit 58bff9b855)
1 year ago
Victor Julien 8b1dcbd5e3 streaming/buffer: improve integer handling safety
Unsafe handling of buffer offset and to be inserted data's length
could lead to a integer overflow. This in turn would skip growing
the target buffer, which then would be memcpy'd into, leading to
an out of bounds write.

This issue shouldn't be reachable through any of the consumers of
the API, but to be sure some debug validation checks have been
added.

Bug: #6903.
(cherry picked from commit cf6278f95a)
1 year ago
Philippe Antoine 93ac371fe8 detect/http-server-body: avoid FP on toserver direction
Ticket: 6948

http.response_body keyword did not enforce a direction, and thus
could match on files sent with POST requests

(cherry picked from commit e6895b835a)
1 year ago
Lukas Sismis 5c180d81bf dpdk: support 52 byte long key on ice (E810) cards
Ticket: 6872

(cherry picked from commit 10590e6d94)
1 year ago
Lukas Sismis 438f855e2d dpdk: enlarge key length to 52 bytes
(cherry picked from commit d4085fceb6)
1 year ago
Jeff Lucovsky f3a7b0ff28 detect/ipopts: Handle multiple ip options
Issue: 6864

Multiple IP options were not handled properly as the value being OR'd
into the packet's ip option variable were enum values instead of bit
values.

(cherry picked from commit d7026b7b11)
1 year ago
Jeff Lucovsky 2100c346d1 detect/ipopts: Remove unneeded PCRE logic
Issue: 6864

Reduce complexity by eliminating the PCRE logic and adding a unittest to
validate null/empty string handling

(cherry picked from commit ee942391f7)
1 year ago
Jeff Lucovsky b59e1dccaa detect/ipopt: Misc. cleanup
Minor changes to improve readability, remove extraneous include files.

(cherry picked from commit c27dee7a9b)
1 year ago
Andreas Herz 59800113b8 dataset: cleanup datasets that hit the memcap while loading
Datasets that hit the memcap limit need to be discarded if the memcap is
hit or otherwise the datasets are still loaded with partial data while
the signature is not loaded due to the memcap error.

Ticket: #6678
(cherry picked from commit 1f9600e487)
1 year ago
Philippe Antoine 5267479134 ci: clean some disk space to run CIFuzz again
(cherry picked from commit 365a66ac1c)
1 year ago
Philippe Antoine 26a5357e76 conf: avoid quadratic complexity
Ticket: 6878

Follow up on 15649424a7

When adding many sequence nodes, either from start or scalar event

We add "sequence nodes" whose name is an integer cf sequence_node_name
and then run ConfNodeLookupChild to see if it had been already set
(from the command line cf comment in the code)
And ConfNodeLookupChild iterates the whole linked list...

1. We add node 1
2. To add node 2, we check if node 1 equals this new node
3. To add node 3, we check if nodes 1, or 2 equals this new node's name
And so on...

This commits avoids these checks ig the list is empty at the beginning

(cherry picked from commit 240e068b81)
1 year ago
Jeff Lucovsky 6be6d53bc6 flow/inject: Select thread_id by flow flag
Issue: 6957

Rather than selecting the thread_id index by packets traveling to the
server, use the flow flags. If the flow has been reversed, the second
slot is represents the thread id to be used.

(cherry picked from commit c305ed149c)
1 year ago
Arne Welzel ba46f2ff3e schema: Add stats.capture and in_iface properties
New suricata-verify test listens on loopback interface, resulting
in the capture and in_iface fields in the stats and event objects.

(cherry picked from commit f9cf87a003)
1 year ago
Arne Welzel fc8a36051a stats: Fix non-worker stats missing
Commit b8b8aa69b4 used tm_name of the
first StatsRecord of a thread block as key for the "threads" object.
However, depending on the type of thread, tm_name can be NULL and would
result in no entry being included for that thread at all. This caused
non-worker metrics to vanish from the "threads" object in the
dump-counters output.

This patch fixes this by remembering the first occurrence of a valid
tm_name within the per-thread block and adds another unittest to
cover this scenario.

(cherry picked from commit f17204191d)
1 year ago
Victor Julien b146068134 pcap: support LINKTYPE_IPV6 (229)
This is just another variant of DLT_RAW.

Ticket: #6943.
(cherry picked from commit 76322368ed)
1 year ago
Victor Julien 6210b82972 defrag: fix wrong datalink being logged
Eve's packet_info.linktype should correctly indicated what the `packet`
field contains. Until now it was using DLT_RAW even if Ethernet or other
L2+ headers were present.

This commit records the datalink of the packet creating the first
fragment, which can include the L2+ header data.

Bug: #6887.
(cherry picked from commit 49c67b2bb1)
1 year ago
Victor Julien 8404a7fe12 defrag: match up v4 and v6 packet setup
v4 was doing redundant recursion level setup.

v6 was missing PKT_REBUILT_FRAGMENT flag.

(cherry picked from commit af97316f42)
1 year ago
Shivani Bhardwaj f0b856214c doc: add note about fast_pattern w base64_data
Bug 6859
1 year ago
Shivani Bhardwaj d4e64e3f24 base64_data: inform of use w fast_pattern
Bug 6859
1 year ago
Jason Ish b83d808994 jsonbuilder: fix serialization of nan and inf
When outputting a float, check if its infinity, or not a number and
output a null instead.

Using a null was chosen as this is what serde_yaml, Firefox, Chrome,
Node, etc. do.

Ticket: #6921
(cherry picked from commit 71f59e529c)
1 year ago
Victor Julien 65e1c37913 detect/http: fix compile warning in body tests
When --enable-unittests w/o --enable-debug is used.

(cherry picked from commit e651cf922a)
1 year ago
Victor Julien e024c7ad01 host-info: remove pcre2_substring_list_free use
Function prototype has changed in a recent release. Rather than dealing
with detecting that, fall back to our regular pattern of using
pcre2_substring_copy_bynumber().

Bug: #6918.
(cherry picked from commit b224209f45)
1 year ago
Jason Ish 41926c51cd configure: .git can be a file as well
In worktree scenarios, .git is a file. Assuming its a directory causes
the release date to check the ChangeLog instead of the last commit,
while not a big issue, can be confusing.
1 year ago
Jason Ish e0e6ad2cef configure: export release date for documentation
Sphinx embeds a date in the generated man pages, and to provide
reproducible builds this date needs to be provided to Sphinx,
otherwise it will use the current date.

If building from Git, extract the date from the most recent commit. In
a release, this commit would be the commit that sets the version so is
accurate.

If .git does not exist, use the most recent data found in the
ChangeLog.

The ChangeLog is not used when building from git, as the main/master
branch may not have recent enough timestamps.

This should provide a consistent date when re-building the
distribution from the same non-git archive, or from the same git
commit.

Ticket: #6911
(cherry picked from commit b58dd5e585)
1 year ago
Jason Ish 363ca13a9c docs/userguide: use a consistent date for reproducible builds
By default, when Sphinx generates the man pages, the current date will
be embedded in them. This can be set to a specific date with the
"today" variable. Typically the date embedded in manpages in the
release date.

To achieve this, attempt to use the environment variable, RELEASE_DATE
to set the "today" variable, reverting back to the empty string if not
set. It is up to our build system to properly set this date.

Ticket: #6911
(cherry picked from commit 51bf1c3510)
1 year ago
Jason Ish e6dd68ffbb configure: don't check ./revision, it never exists
Stop checking the ./revision file for the git revision info, its never
created.

(cherry picked from commit c00c2b116f)
1 year ago
Jason Ish 3b8ddf395a docs/conf.py: fix python escape warning
/home/jason/oisf/dev/suricata/master/doc/userguide/conf.py:74: SyntaxWarning: invalid escape sequence '\('
  "AC_INIT\(\[suricata\],\s*\[(.*)?\]\)",

(cherry picked from commit 4c16032f63)
1 year ago
Lukas Sismis ac67861fcf dpdk: add interrupt (power-saving) mode
When the packet load is low, Suricata can run in interrupt
mode. This more resembles the classic approach of processing
packets - CPU cores run low and only fetch packets
on interrupt.

Ticket: #6696
(cherry picked from commit 2a2898053c)
1 year ago
Jeff Lucovsky 158018fc1c flow: Swap thread_ids
Issue: 6835

When swapping the flow's direction, also swap the thread_ids.

This should help with the issues identified in
https://redmine.openinfosecfoundation.org/issues/2725

(cherry picked from commit 3c5745978f)
1 year ago
Jeff Lucovsky 4331da4b67 flow/inject: Ensure initialized thread value used
Issue: 6835

When injecting a flow, ensure that the selected thread_id has been
initialized. When a flow is picked up midstream, the initialized thread
can be the second thread element.

(cherry picked from commit 9ad73faa0a)
1 year ago
Philippe Antoine 80d2c6e0a1 rust/mqtt: fix clippy 1.77 warning
error: creating a mutable reference to mutable static is discouraged
   --> src/mqtt/mqtt.rs:752:23
    |
752 |     let max_msg_len = &mut MAX_MSG_LEN;
    |                       ^^^^^^^^^^^^^^^^ mutable reference to mutable static
    |
    = note: for more information, see issue #114447 <https://github.com/rust-lang/rust/issues/114447>
    = note: this will be a hard error in the 2024 edition
    = note: this mutable reference has lifetime `'static`, but if the static gets accessed (read or written) by any other means, or any other reference is created, then any further use of this mutable reference is Undefined Behavior
1 year ago
Philippe Antoine fce01dad77 rust/smb: fix clippy nightly warning
error: unnecessary use of `to_vec`
    --> src/smb/smb.rs:1048:62
     |
1048 |         let (name, is_dcerpc) = match self.guid2name_map.get(&guid.to_vec()) {
     |                                                              ^^^^^^^^^^^^^^ help: replace it with: `guid`
     |
     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_to_owned
     = note: `#[deny(clippy::unnecessary_to_owned)]` implied by `#[deny(warnings)]`

And also other uses of to_vec() on already Vec

(cherry picked from commit f7cde8f00e)
1 year ago
Philippe Antoine 5a05e025ed rust: fix clippy 1.77 warning
Ticket: 6883

error: field `0` is never read
  --> src/asn1/mod.rs:36:14
   |
36 |     BerError(Err<der_parser::error::BerError>),
   |     -------- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |     |
   |     field in this variant
   |

(cherry picked from commit 02f2fb8833)
1 year ago
Victor Julien 58c61f1fe4 rust: add MSRV as rust-version
Update github-actions to use it for the MSRV check.
1 year ago
Shivani Bhardwaj acc898f3e0 version: start development towards 7.0.5 1 year ago
Shivani Bhardwaj d8bad3b1aa release: 7.0.4; update changelog 1 year ago
Philippe Antoine c0b9ddb32e ssh: limit length for banner logs
Ticket: 6770
(cherry picked from commit c4b8fb7aca)
1 year ago
Philippe Antoine f9ef96af30 ssh: avoid quadratic complexity from long banner
Ticket: 6799

When we find an overlong banner, we get into the state just
waiting for end of line, and we just want to skip the bytes
until then.
Returning AppLayerResult::incomplete made TCP engine retain
the bytes and grow the buffer that we parsed again and again...

(cherry picked from commit 271ed2008b)
1 year ago
Philippe Antoine d4acd0fb44 ci: update ubuntu22.04 builds with clang14+asan
using a workround about ASLR

(cherry picked from commit 632e52ca2b)
1 year ago
Philippe Antoine 8b78f5e6dc detect: flush when setting no_inspection
Ticket: 6578

When a protocol such as SSH sets no_inspection, we still have to
flush the current streams and packets that contain clear-text
for detection.

(cherry picked from commit cc3b4b01ec)
1 year ago
Philippe Antoine 14d4221095 detect: update packet action on protocol change
Ticket: #6305

When running FlowWorkerStreamTCPUpdate, one of the dequeued packet
may set the flow action to drop, without updating the not-pseudo
packet action, as is done usually with a previous call to
FlowHandlePacketUpdate

(cherry picked from commit 4c4f7ff1a2)
1 year ago
Victor Julien c8c72030bf yaml: remove newline from error message
(cherry picked from commit f53c4ab149)
1 year ago
Jonathan Perkins 05640d84dc napatech: emit HBA deprecation only once
Issue: 6313
This commit removes duplicate HBA deprecation messages from being
emitted.
1 year ago