Commit Graph

16286 Commits (b3bd57246fbbb80da5a76223d35f57589725cc48)
 

Author SHA1 Message Date
Victor Julien a404fd26af tcp: fix 'broken ack' on flow timeout
Don't set an ACK value if ACK flag is no longer set. This avoids a bogus
`pkt_broken_ack` event set.

Fixes: ebf465a11b ("tcp: do not assign TCP flags to pseudopackets")

Ticket: #7158.
8 months ago
Shivani Bhardwaj f2de3e01cb src: remove truncate fn and glue code
truncate fn is only active and used by dcerpc and smb parsers. In case
stream depth is reached for any side, truncate fn is supposed to set the
tx entity (request/response) in the same direction as complete so the
other side is not forever waiting for data.

However, whether the stream depth is reached is already checked by
AppLayerParserGetStateProgress fn which is called by:
- DetectTx
- DetectEngineInspectBufferGeneric
- AppLayerParserSetTransactionInspectId
- OutputTxLog
- AppLayerParserTransactionsCleanup

and, in such a case, StateGetProgressCompletionStatus is returned for
the respective direction. This fn following efc9a7a, always returns 1
as long as the direction is valid meaning that the progress for the
current direction is marked complete. So, there is no need for the additional
callback to mark the entities as done in case of depth or a gap.
Remove all such glue code and callbacks for truncate fns.

Bug 7044
8 months ago
Shivani Bhardwaj 80159eb519 applayer: remove truncation logic
as its functionality is already covered by the generic code.
This removes APP_LAYER_PARSER_TRUNC_TC and APP_LAYER_PARSER_TRUNC_TS
flags as well as FlowGetDisruptionFlags sets STREAM_DEPTH flag in case
the respective stream depth was reached. This flag tells that whether
all the open files should be truncated or not.

Bug 7044
8 months ago
Philippe Antoine 090079cdd8 decode: fix -Wshorten-64-to-32 warnings
Ticket: #6186
8 months ago
Philippe Antoine eeb290384a flow: fix -Wshorten-64-to-32 warnings
Ticket: #6186
8 months ago
Philippe Antoine 9c0875b2a4 features: fix -Wshorten-64-to-32 warnings
Ticket: #6186
8 months ago
Philippe Antoine b5140c43ca counters: fix -Wshorten-64-to-32 warnings
Ticket: #6186
8 months ago
Philippe Antoine e0fd59a20d doc: state that payload-length includes the gaps 8 months ago
Philippe Antoine d28c646662 output/dcerpc: call jb_get_mark just before jb_open_object 8 months ago
Jason Ish 4d3d57249a doc: update dns section of the eve format documentation 8 months ago
Jason Ish d3c08b9643 doc: upgrade guide for dns logging changes
Bug: #6281
8 months ago
Jason Ish b32f6bf381 eve/dns: allow version to be set with environment variable
There is no sane way to set override the DNS eve version in Suricata
tests without using a copy of the configuration file, and many of the
tests by design use the configuration file of the Suricata under test,
so making a copy would break this assumption.

To get around this, respect the SURICATA_EVE_DNS_VERSION environment
variable as a way to set the version if not explicitly set in the
configuration file.
8 months ago
Jason Ish 575e5b471f dns: add v3 dns logging
DNS v3 logging fixes the discrepancies between request and response
logging with the main difference being queries always being placed in an
array.

Bug: #6281
8 months ago
Jason Ish df656324ba dns: new v3 style logging for alerts
V3 style DNS logging fixes the discrepancies between request and
response logging better dns records and alert records.

The main change is that queries and answers are always logged as
arrays, and header fields are not logged in array items.

For alerts this means that answers are now logged as arrays, queries
already were.

DNS records will get this new format as well, but with a configuration
parameter.

Bug: #6281
8 months ago
Nathan Scrivens 9ecc3573a7 dns: parse and populate OPT rdata struct
Feature: 7017
Add DNSRDataOPT struct and DNSRData enum type OPT.
Add OPT parsing function and test function.
Add DNSRData OPT type to lua.rs match.
Log OPT rdata.
8 months ago
Nathan Scrivens 4598ca164d dns log: add additional section
Feature: 7011
dns_log_json_answer: log additional section records.
update schema.json with new "additionals" section.
8 months ago
Nathan Scrivens 1cd89640ef dns parsing: add additional section
Feature: 7011
Add additionals to DNSMessage struct.
Add parsing logic to populate additional section data.
Patch dns tests to account for additional section parsing.
8 months ago
Sascha Steinbiss 53c62432c6 doc: update MQTT configuration 8 months ago
Sascha Steinbiss e047ad25e2 mqtt: run rustfmt 8 months ago
Sascha Steinbiss ad02040860 mqtt: enable limiting of logged message length
Ticket: #6984
8 months ago
Sascha Steinbiss dd972f72dd rust: add JsonBuilder::set_string_limited() 8 months ago
dependabot[bot] a28666db4e github-actions: bump actions/download-artifact from 4.1.4 to 4.1.8
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 4.1.4 to 4.1.8.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v4.1.4...fa0a91b85d4f404e444e00e005971372dc801d16)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
8 months ago
dependabot[bot] e83a1cf34c github-actions: bump actions/upload-artifact from 4.3.3 to 4.3.4
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.3.3 to 4.3.4.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](65462800fd...0b2256b8c0)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
8 months ago
Jeff Lucovsky 99f9451be3 detect: Use Option where appropriate
This commit uses Option instead of Result.

Issue: 6873
8 months ago
Jeff Lucovsky 70bdc37f96 detect/byte_extract: Move keyword parser to Rust
Implement the keyword parser in Rust.

Issue: 6873
8 months ago
Jeff Lucovsky 73dfc58772 detect/byte: Refactor endian, base
Issue: 6873

Refactor the enums for endian and base handling for broader use.
8 months ago
Philippe Antoine eeec609ac8 util/thash: decrease memuse if array was allocated
THashInitConfig may not allocate array and increase memuse.
Such a failure leads to THashShutdown which should not decrease
the memuse.

Ticket: 7135
8 months ago
Lukas Sismis 35dffc6b32 dpdk: replace TSC clock with GetTime (gettimeofday) function
Getting clock through Time Stamp Counter (TSC) can be precise and fast,
however only for a short duration of time.
The implementation across CPUs seems to vary. The original idea is to
increment the counter with every tick. Then dividing the delta of CPU ticks
by the CPU frequency can return the time that passed.
However, the CPU clock/frequency can change over time, resulting in uneven
incrementation of TSC. On some CPUs this is handled by extra logic.
As a result, obtaining time through this method might drift from the real
time.

This commit therefore substitues TSC time retrieval by the standard system
call wrapped in GetTime function - on Linux it is gettimeofday.

Ticket: 7115
8 months ago
Shivani Bhardwaj a7af371843 applayer/htp-range: fix off by one in expiry check 8 months ago
Shivani Bhardwaj c66f1f4488 doc: add note about datasets string memcaps
Bug 3910
8 months ago
Shivani Bhardwaj f1b44ca8c7 datasets: fix memuse to include string len
So far, when the data size was passed to the THash API, it was sent as
a sizeof(Struct) which works fine for the other data types as they have
a fixed length but not for the StringType.
However, because of the sizeof construct, the length of a string type
dataset was always taken to be 16 Bytes which is only the size of the struct
itself. It did not accomodate the actual size of the string that the
StringType holds. Fix this so that the memuse that is used to determine
whether memcap was reached also takes into consideration the size of the
actual string.

Bug 3910
8 months ago
Shivani Bhardwaj 00f7038beb util/thash: add a length getter fn
In order to have access to the length of datatypes with variable lengths
to correctly update memuse to calculate memcaps.

Bug 3910
8 months ago
Philippe Antoine df5dcfef5f bypass: really bypass udp flow from first packet
Ticket: 7053

As flow state would be overwritten by established...
8 months ago
dependabot[bot] efbf627dbd
github-actions: bump actions/checkout from 3.6.0 to 4.1.7
Bumps [actions/checkout](https://github.com/actions/checkout) from 3.6.0 to 4.1.7.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v3.6.0...692973e3d937129bcbf40652eb9f2f61becf3332)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
8 months ago
Jason Ish 5530fd34b8 github-ci: add non-bundled libhtp build 8 months ago
Jason Ish a0cb7b21f4 github-ci/builds: don't run if only doc changes
A new workflow, "docs" has been created for only doc changes.

Factor out prepare-deps into a reusable action.
8 months ago
Philippe Antoine 7dfddab9ed detect: parse units for integer for every cases
Ticket: #6423

Not just equality, but also >3MB should work
For example flow.bytes_toserver>3MB
8 months ago
Victor Julien f59c43b1c7 smb/ntlmssp: improve version check
Don't assume the ntlmssp version field is always present if the flag is
set. Instead keep track of the offsets of the data of the various blobs
and see if there is space for the version.

Inspired by how Wireshark does the parsing.

Bug: #7121.
8 months ago
dependabot[bot] 653e702aa7 github-actions: bump github/codeql-action from 3.25.7 to 3.25.11
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.25.7 to 3.25.11.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Commits](https://github.com/github/codeql-action/compare/v3.25.7...v3.25.11)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
8 months ago
dependabot[bot] 54b63f896d github-actions: bump codecov/codecov-action from 4.4.1 to 4.5.0
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 4.4.1 to 4.5.0.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](125fc84a9a...e28ff129e5)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
8 months ago
Victor Julien 6767694b39 github-actions: remove centos:7 build due to EOL 8 months ago
Philippe Antoine 1854503465 qa: remove depcrecated files
lgtm has been superseded by codeql github action
docker and travis are obsolete

Preparatory work for ticket 2696 libhtp rust conversion
8 months ago
Philippe Antoine 6b56d5971a output/tx: use dynamic number of app-layer protos
OutputTxLoggerThreadData gets allocated after the number of app-layer
protos is definite
8 months ago
Philippe Antoine 6a942f589c detect/profiling: use dynamic number of app-layer protos 8 months ago
Philippe Antoine f74997f5c7 app-layer: use already defined constant
FLOW_PROTO_APPLAYER_MAX
8 months ago
Philippe Antoine deb4a5a8cc detect/file-data: use dynamic number of app-layer protos 8 months ago
Philippe Antoine 647e878f7c detect: helper function for multibuffer 8 months ago
Victor Julien afc318737a doc/userguide: document threshold backoff type 8 months ago
Victor Julien 9e735fd6bd stream: enable backoff on event rules
Enable backoff for most rules. The rules looking at the session start up
use a count of 1 and a multiplier of 2.

Post-3whs rules use a count of 1 and a multiplier of 10.
8 months ago
Victor Julien 12130df21c detect/threshold: implement backoff type
Implement new `type backoff` for thresholding. This allows alerts to be
limited.

A count of 1 with a multiplier of 10 would generate alerts for matching packets:
1, 10, 100, 1000, 10000, 100000, etc.

A count of 1 with a multiplier of 2 would generate alerts for matching packets:
1, 2, 4, 8, 16, 32, etc.

Like with other thresholds, rule actions like drop and setting of
flowbits will still be performed for each matching packet.

Current implementation is only for the by_flow tracker and for per rule
threshold statements.

Tracking is done using uint32_t. When it reaches this value, the rest of
the packets in the tracker will use the silent match.

Ticket: #7120.
8 months ago