Commit Graph

1455 Commits (6d5022cd1edde09aa84042b71c5860c0f923ff92)

Author SHA1 Message Date
Shivani Bhardwaj f2de3e01cb src: remove truncate fn and glue code
truncate fn is only active and used by dcerpc and smb parsers. In case
stream depth is reached for any side, truncate fn is supposed to set the
tx entity (request/response) in the same direction as complete so the
other side is not forever waiting for data.

However, whether the stream depth is reached is already checked by
AppLayerParserGetStateProgress fn which is called by:
- DetectTx
- DetectEngineInspectBufferGeneric
- AppLayerParserSetTransactionInspectId
- OutputTxLog
- AppLayerParserTransactionsCleanup

and, in such a case, StateGetProgressCompletionStatus is returned for
the respective direction. This fn following efc9a7a, always returns 1
as long as the direction is valid meaning that the progress for the
current direction is marked complete. So, there is no need for the additional
callback to mark the entities as done in case of depth or a gap.
Remove all such glue code and callbacks for truncate fns.

Bug 7044
3 months ago
Shivani Bhardwaj 80159eb519 applayer: remove truncation logic
as its functionality is already covered by the generic code.
This removes APP_LAYER_PARSER_TRUNC_TC and APP_LAYER_PARSER_TRUNC_TS
flags as well as FlowGetDisruptionFlags sets STREAM_DEPTH flag in case
the respective stream depth was reached. This flag tells that whether
all the open files should be truncated or not.

Bug 7044
3 months ago
Jason Ish 575e5b471f dns: add v3 dns logging
DNS v3 logging fixes the discrepancies between request and response
logging with the main difference being queries always being placed in an
array.

Bug: #6281
3 months ago
Jason Ish df656324ba dns: new v3 style logging for alerts
V3 style DNS logging fixes the discrepancies between request and
response logging better dns records and alert records.

The main change is that queries and answers are always logged as
arrays, and header fields are not logged in array items.

For alerts this means that answers are now logged as arrays, queries
already were.

DNS records will get this new format as well, but with a configuration
parameter.

Bug: #6281
3 months ago
Nathan Scrivens 9ecc3573a7 dns: parse and populate OPT rdata struct
Feature: 7017
Add DNSRDataOPT struct and DNSRData enum type OPT.
Add OPT parsing function and test function.
Add DNSRData OPT type to lua.rs match.
Log OPT rdata.
3 months ago
Nathan Scrivens 4598ca164d dns log: add additional section
Feature: 7011
dns_log_json_answer: log additional section records.
update schema.json with new "additionals" section.
3 months ago
Nathan Scrivens 1cd89640ef dns parsing: add additional section
Feature: 7011
Add additionals to DNSMessage struct.
Add parsing logic to populate additional section data.
Patch dns tests to account for additional section parsing.
3 months ago
Sascha Steinbiss e047ad25e2 mqtt: run rustfmt 3 months ago
Sascha Steinbiss ad02040860 mqtt: enable limiting of logged message length
Ticket: #6984
3 months ago
Sascha Steinbiss dd972f72dd rust: add JsonBuilder::set_string_limited() 3 months ago
Jeff Lucovsky 99f9451be3 detect: Use Option where appropriate
This commit uses Option instead of Result.

Issue: 6873
3 months ago
Jeff Lucovsky 70bdc37f96 detect/byte_extract: Move keyword parser to Rust
Implement the keyword parser in Rust.

Issue: 6873
3 months ago
Jeff Lucovsky 73dfc58772 detect/byte: Refactor endian, base
Issue: 6873

Refactor the enums for endian and base handling for broader use.
3 months ago
Philippe Antoine 7dfddab9ed detect: parse units for integer for every cases
Ticket: #6423

Not just equality, but also >3MB should work
For example flow.bytes_toserver>3MB
4 months ago
Victor Julien f59c43b1c7 smb/ntlmssp: improve version check
Don't assume the ntlmssp version field is always present if the flag is
set. Instead keep track of the offsets of the data of the various blobs
and see if there is space for the version.

Inspired by how Wireshark does the parsing.

Bug: #7121.
4 months ago
Philippe Antoine 647e878f7c detect: helper function for multibuffer 4 months ago
Philippe Antoine 5bd17934df http2: do not expand duplicate headers
Ticket: 7104

As this can cause a big mamory allocation due to the quadratic
nature of the HPACK compression.
4 months ago
Philippe Antoine 37509e8e0e modbus: abort flow parsing on flood
Ticket: 6987

Let's not spend more resources for a flow which is trying to
make us do it...
4 months ago
Jeff Lucovsky 5b97f4040c detect/base64: Use Rust defined modes everywhere
Issue: 6487

To avoid ambiguity, a single definition for base 64 decoding modes will
be used. The Rust base64 transform contains the definitions for the
existing mode types: Strict, RFC2045, RFC4648
4 months ago
Jeff Lucovsky 1823681709 detect/transform: from_base64 option parsing
Issue: 6487

Implement from_base64 option parsing in Rust. The Rust module also
contains unit tests.
4 months ago
Jeff Lucovsky ab0cb960a1 detect/parser: Refactor utility routines
Refactor utility functions/definitions from the byte_math module into
the parser module. This includes parse_var and ResultValue

Issue: 6487
4 months ago
Philippe Antoine 5f35035928 filestore: do not try to store a file set to nostore
Ticket: 6390

This can happen with keyword filestore:both,flow
If one direction does not have a signature group with a filestore,
the file is set to nostore on opening, until a signature in
the other direction tries to set it to store.
Subsequent files will be stored in both directions as flow flags
are now set.
4 months ago
Philippe Antoine 5bb5b4f46f rust: remove unnecessary nested unsafe 4 months ago
Philippe Antoine 4ccbcc4684 sip: use right slice to take line from
We iterate over input, but we are now at start.
Avois quadratic complexity turning to OOM.

Ticket: 7093
4 months ago
Jason Ish 49ecf37126 rust/ike: prefix never read field names with _
New warning from rustc.

The other option is to allow dead code, however this is more explicit,
and when they are read, its obvious they should be renamed.
4 months ago
Jason Ish 29d7ff026a rust: simply matches with unwrap_or_default
New default clippy warning:
https://rust-lang.github.io/rust-clippy/master/index.html#manual_unwrap_or_default
4 months ago
Jason Ish ee2175cdb6 rust: fix clippy lint for legacy_numeric_constants
https://rust-lang.github.io/rust-clippy/master/index.html#legacy_numeric_constants
4 months ago
Jason Ish a1bb62c059 cargo: use default-features instead of default_features
"default_features" is being deprecated in Rust 2024.
4 months ago
Philippe Antoine 4fe3f04fa3 detect/enip: move keywords to rust
Ticket: 4863
4 months ago
Philippe Antoine ce1eea4ad6 detect/websocket: move keywords to rust
Ticket: 4863
4 months ago
Philippe Antoine 16952d67e7 detect/dhcp: move keywords to rust
Ticket: 4863
4 months ago
Philippe Antoine ae72376ebe detect/snmp: move keywords to rust
Ticket: 4863

On the way, convert unit test DetectSNMPCommunityTest to a SV test.

And also, make snmp.pdu_type use a generic uint32 for detection,
allowing operators, instead of just equality.
4 months ago
Philippe Antoine 4bbe7d92dc detect: helper to have pure rust keywords
detect: make number of keywords dynamic

Ticket: 4683
4 months ago
Philippe Antoine 08c511f1bf enip: remove unnecessary unsafe
As the function SCEnipRegisterParsers is already marked as unsafe
4 months ago
Victor Julien 37be66eef9 detect/iprep: update function naming
Bring in line with new Rust code naming for FFI functions.
4 months ago
Victor Julien 83976a4cd4 detect/iprep: implement isset and isnotset
Implement special "isset" and "isnotset" modes.

"isset" matches if an IP address is part of an iprep category with any
value.

It is internally implemented as ">=,0", which should always be true if
there is a value to evaluate, as valid reputation values are 0-127.

"isnotset" matches if an IP address is not part of an iprep category.

Internally it is implemented outside the uint support.

Ticket: #6857.
4 months ago
Victor Julien 539ab3a404 detect/iprep: update keyword parser for extendibility 4 months ago
Jason Ish f0dbfe863d misc: prefix functions with SC not Sc 4 months ago
Philippe Antoine d9d5170ec0 websocket: add data frame
Ticket: 7051
4 months ago
Juliana Fajardini bb45ac71ef dns: allow triggering raw stream reassembly
For TCP streams, app proto stream reassembly can start earlier, instead
of waiting and queueing up data before doing so.

Task #7018
Related to
Bug #7004
4 months ago
Philippe Antoine 82c03f72c3 enip: convert to rust
Ticket: 3958

- transactions are now bidirectional
- there is a logger
- gap support is improved with probing for resync
- frames support
- app-layer events
- enip_command keyword accepts now string enumeration as values.
- add enip.status keyword
- add keywords :
    enip.product_name, enip.protocol_version, enip.revision,
    enip.identity_status, enip.state, enip.serial, enip.product_code,
    enip.device_type, enip.vendor_id, enip.capabilities,
    enip.cip_attribute, enip.cip_class, enip.cip_instance,
    enip.cip_status, enip.cip_extendedstatus
4 months ago
Philippe Antoine 0d267e29a5 files: remove the need for state in callbacks
As files now belong to transactions
4 months ago
Philippe Antoine 5167ff6411 smtp/mime: look for urls in base64 message
Ticket: 5185

Previously, it was looked for message in plain text, and base64
encoding was only handled for attachments.

This commit also fixes the buffering got such base64 data streamed
into urls finding, by buffering a beginning non-empty line,
and by ensuring that we run extraction on the last line,
even if it had no EOL.
4 months ago
Juliana Fajardini 0946c213cd pgsql: trigger raw stream reassembly
Expose the raw stream earlier to the detection engine, as Pgsql can have
multiple messages per transaction and usually will have a message
complete within one TCP packet.

Bug #7000

Related to
Bug #7026
4 months ago
Juliana Fajardini 69e26de197 pgsql/logger: open json object from logger function
Before, the JsonBuilder object for the pgsql event was being created
from the C-side function that actually called the Rust logger.

This resulted that if another module - such as the Json Alert called the
PGSQL logger, we wouldn't have the `pgsql` key present in the log output
- only its inner fields.

Bug #6983
4 months ago
Philippe Antoine 7fb10676e7 dns: remove unneeded mut in logger 4 months ago
Philippe Antoine a10c1f1dde smtp: use rust for mime parsing
Ticket: #3487
4 months ago
Philippe Antoine 5f75b9a6e3 http: use rust for mime parsing
Ticket: #3487
4 months ago
Philippe Antoine 5555aa6788 mime: improved token parsing
Accepts escaped quote in escaped string
4 months ago
Jason Ish 936930778c rust/Makefile: cleanup "clean" targets
Remove maintainer-clean-local, this is not needed.

In distclean-local, remove "rust/dist" and "rust/vendor" as they are
created during "make dist".

In "clean-local", remove "rust/target" and "rust/gen" as they are
created during a normal "make".
5 months ago