Commit Graph

1376 Commits (a0bf28296376422ffa1a7803b3563065c536b315)

Author SHA1 Message Date
Philippe Antoine 08c511f1bf enip: remove unnecessary unsafe
As the function SCEnipRegisterParsers is already marked as unsafe
1 year ago
Victor Julien 37be66eef9 detect/iprep: update function naming
Bring in line with new Rust code naming for FFI functions.
1 year ago
Victor Julien 83976a4cd4 detect/iprep: implement isset and isnotset
Implement special "isset" and "isnotset" modes.

"isset" matches if an IP address is part of an iprep category with any
value.

It is internally implemented as ">=,0", which should always be true if
there is a value to evaluate, as valid reputation values are 0-127.

"isnotset" matches if an IP address is not part of an iprep category.

Internally it is implemented outside the uint support.

Ticket: #6857.
1 year ago
Victor Julien 539ab3a404 detect/iprep: update keyword parser for extendibility 1 year ago
Jason Ish f0dbfe863d misc: prefix functions with SC not Sc 1 year ago
Philippe Antoine d9d5170ec0 websocket: add data frame
Ticket: 7051
1 year ago
Juliana Fajardini bb45ac71ef dns: allow triggering raw stream reassembly
For TCP streams, app proto stream reassembly can start earlier, instead
of waiting and queueing up data before doing so.

Task #7018
Related to
Bug #7004
1 year ago
Philippe Antoine 82c03f72c3 enip: convert to rust
Ticket: 3958

- transactions are now bidirectional
- there is a logger
- gap support is improved with probing for resync
- frames support
- app-layer events
- enip_command keyword accepts now string enumeration as values.
- add enip.status keyword
- add keywords :
    enip.product_name, enip.protocol_version, enip.revision,
    enip.identity_status, enip.state, enip.serial, enip.product_code,
    enip.device_type, enip.vendor_id, enip.capabilities,
    enip.cip_attribute, enip.cip_class, enip.cip_instance,
    enip.cip_status, enip.cip_extendedstatus
1 year ago
Philippe Antoine 0d267e29a5 files: remove the need for state in callbacks
As files now belong to transactions
1 year ago
Philippe Antoine 5167ff6411 smtp/mime: look for urls in base64 message
Ticket: 5185

Previously, it was looked for message in plain text, and base64
encoding was only handled for attachments.

This commit also fixes the buffering got such base64 data streamed
into urls finding, by buffering a beginning non-empty line,
and by ensuring that we run extraction on the last line,
even if it had no EOL.
1 year ago
Juliana Fajardini 0946c213cd pgsql: trigger raw stream reassembly
Expose the raw stream earlier to the detection engine, as Pgsql can have
multiple messages per transaction and usually will have a message
complete within one TCP packet.

Bug #7000

Related to
Bug #7026
1 year ago
Juliana Fajardini 69e26de197 pgsql/logger: open json object from logger function
Before, the JsonBuilder object for the pgsql event was being created
from the C-side function that actually called the Rust logger.

This resulted that if another module - such as the Json Alert called the
PGSQL logger, we wouldn't have the `pgsql` key present in the log output
- only its inner fields.

Bug #6983
1 year ago
Philippe Antoine 7fb10676e7 dns: remove unneeded mut in logger 1 year ago
Philippe Antoine a10c1f1dde smtp: use rust for mime parsing
Ticket: #3487
1 year ago
Philippe Antoine 5f75b9a6e3 http: use rust for mime parsing
Ticket: #3487
1 year ago
Philippe Antoine 5555aa6788 mime: improved token parsing
Accepts escaped quote in escaped string
1 year ago
Jason Ish 2e440169d6 lua: remove lua as a compile time feature
Its always built-in. However, can be disabled at runtime.
1 year ago
Jason Ish 1fd2c1a379 rust/lua: remove lua_int8 feature
Now that we're fixed to Lua 5.4, the integer size is always 8.
1 year ago
Jason Ish bc011f2205 lua: use rust crate to vendor (bundle) lua
Remove lua-dev(el) from all CI tests.
1 year ago
Shivani Bhardwaj 3a1c12414a tls: store list of subject alternative names
So far, the SANs were available as a part of IssuerDN via x509_parser
crate but SANs were not available to the SSLState* to be directly used
to setup and match against a sticky buffer.
Expose it to SSLStateConnp.

Feature 5234
2 years ago
Philippe Antoine 37a9003736 rust/probing: safety check for null input
Ticket: 7013

Done consistently for all protocols

This may change some protocols behaviors which failed early
if they found there was not enough data...
2 years ago
Philippe Antoine 5dc8dea869 rust: return empty slice without using from_raw_parts
As this triggers rustc 1.78
unsafe precondition(s) violated: slice::from_raw_parts requires
the pointer to be aligned and non-null,
and the total size of the slice not to exceed `isize::MAX`

Ticket: 7013
2 years ago
Philippe Antoine 806052d762 websocket: fix opcodes values for ping/pong
And also set close

Ticket: 7025
2 years ago
Philippe Antoine 8b103ae755 dns: set tx id for frames 2 years ago
Philippe Antoine 715bf048ee frames: rust API makes tx_id explicit
And set it right for SIP and websocket,
so that relevant tx app-layer metadata gets logged.

Ticket: 6973
2 years ago
Jeff Lucovsky cb56752bf7 config/ja3: Eliminate warnings when JA3 is disabled
This commit eliminates warnings when either ja3, ja4 or both are
disabled.
2 years ago
Philippe Antoine c53e9ac0dd sdp: fix logging medias
As introduced by bff790b6ac

Also handles errors in the caller

Ticket: 6994
2 years ago
Jason Ish df8568ee30 rust/dns: visibility cleanups
Remove pub from functions that don't require it.
2 years ago
Jason Ish 556cfe56bf rust/dns: ffi naming and visibility cleanups
- Remove no_mangle and pub from FFI functions that are only accessed
  with a function pointer.
- Rename all no_mangle FFI functions to our C naming scheme.
2 years ago
Giuseppe Longo 868493529b rust/sip: parse and log sdp
If SDP payload is found within a SIP message, it will be parsed and then
logged.

Ticket #6627
2 years ago
Giuseppe Longo bff790b6ac rust/sdp: implement logger
This implements a logger for the SDP protocol.
Given that SDP is encapsulated within other protocols (such as SIP),
enabling it separately is not necessary.

Ticket #6627
2 years ago
Giuseppe Longo 1ccfc35214 rust/sdp: implement protocol parser
This implements a parser for the SDP protocol.
Given that SDP is encapsulated within other protocols (such as SIP),
enabling it separately is not necessary.

Ticket #6627.
2 years ago
Philippe Antoine 03442c9071 http2: do not log duplicate headers
Ticket: 6900

And thus avoid DOS by logging a request using a compressed
header block repeated many times and having a long value...
2 years ago
Philippe Antoine 390f09692e http2: use a reference counter for headers
Ticket: 6892

As HTTP hpack header compression allows one single byte to
express a previously seen arbitrary-size header block (name+value)
we should avoid to copy the vectors data, but just point
to the same data, while reamining memory safe, even in the case
of later headers eviction from the dybnamic table.

Rust std solution is Rc, and the use of clone, so long as the
data is accessed by only one thread.
2 years ago
Philippe Antoine 0291d37009 websocket: configurable logging of payload in alerts 2 years ago
Philippe Antoine 44b6aa5e4b app-layer: websockets protocol support
Ticket: 2695
2 years ago
Philippe Antoine f83ec543e3 http2: add settings from newer RFCs
Including the one for websocket over HTTP/2
2 years ago
Juliana Fajardini ce1556cefd pgsql: check for eol when parsing response
It was brought to my attention by GLongo that Pgsql parser handled eof
diffrently for requests and responses, and apparently there isn't a good
reason for such a difference therefore, apply same logic used for
rs_pgsql_parse_request for checking for eof when parsing a response.
2 years ago
Sascha Steinbiss 120313f4da ja4: implement for TLS and QUIC
Ticket: OISF#6379
2 years ago
Sascha Steinbiss 9d0db71ebf ja3: make feature compile time configurable 2 years ago
Philippe Antoine 49caf005a4 detect/analyzer: create tojson function for generic integers
As will be needed for tcp.mss
2 years ago
Philippe Antoine 3643b6ed4b output: generic simple tx json logger
Ticket: 3827
2 years ago
Jason Ish 71f59e529c jsonbuilder: fix serialization of nan and inf
When outputting a float, check if its infinity, or not a number and
output a null instead.

Using a null was chosen as this is what serde_yaml, Firefox, Chrome,
Node, etc. do.

Ticket: #6921
2 years ago
Philippe Antoine ee50fe4c30 sip: convert transaction list to vecdeque
Ticket: 6891

So as to avoid quadratic complexity on tx cleanup with SIP/TCP
that can create many transactions in one go.
2 years ago
Philippe Antoine f7cde8f00e rust/smb: fix clippy nightly warning
error: unnecessary use of `to_vec`
    --> src/smb/smb.rs:1048:62
     |
1048 |         let (name, is_dcerpc) = match self.guid2name_map.get(&guid.to_vec()) {
     |                                                              ^^^^^^^^^^^^^^ help: replace it with: `guid`
     |
     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_to_owned
     = note: `#[deny(clippy::unnecessary_to_owned)]` implied by `#[deny(warnings)]`

And also other uses of to_vec() on already Vec
2 years ago
Philippe Antoine 02f2fb8833 rust: fix clippy 1.77 warning
Ticket: 6883

error: field `0` is never read
  --> src/asn1/mod.rs:36:14
   |
36 |     BerError(Err<der_parser::error::BerError>),
   |     -------- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |     |
   |     field in this variant
   |
2 years ago
Philippe Antoine c4b8fb7aca ssh: limit length for banner logs
Ticket: 6770
2 years ago
Philippe Antoine 271ed2008b ssh: avoid quadratic complexity from long banner
Ticket: 6799

When we find an overlong banner, we get into the state just
waiting for end of line, and we just want to skip the bytes
until then.
Returning AppLayerResult::incomplete made TCP engine retain
the bytes and grow the buffer that we parsed again and again...
2 years ago
Hadiqa Alamdar Bukhari 3aa313d0c5 dns: add dns.rcode keyword
dns.rcode matches the rcode header field in DNS messages
It's an unsigned integer
valid ranges = [0-15]
Does not support prefilter
Supports matches in both flow directions

Task #6621
2 years ago
Victor Julien 1d3a156179 rust: update parser dependencies
Time locked to 0.3.20 to guarantee MSRV of 1.63.
Update snmp-parser to 0.10.0.
Update asn1-rs to 0.6.1.
Update kerberos-parser to 0.8.0.
Update x509-parser 0.16.0.
Update der-parser to 9.0.0.
Remove specific use of der-parser 6.

Ticket: #6817.
Ticket: #6818.
2 years ago
Hadiqa Alamdar Bukhari 4b81851097 dns: add dns.rrtype keyword
It matches the rrtype field in DNS
It's an unsigned integer match
valid ranges = [0-65535]
Does not support prefilter
Supports flow in both directions
Feature #6666
2 years ago
Giuseppe Longo fe77def816 rust/sip: register pattern matching
This permits to detect the SIP protocol using pattern matching instead of
probing parser.

Since it is no longer used, the respective probing functions have been removed.
2 years ago
Giuseppe Longo 9c9b1a4230 rust/sip: add direction to transaction
This patch permits to set a direction when a new transaction is created in order
to avoid 'signature shadowing' as reported by Eric Leblond in commit
5aaf50760f
2 years ago
Giuseppe Longo c9d309219e rust/sip: register parser for tcp
This patch lets the parser to work over tcp protocol, taking care of handling
data before calling the request/response parsers.

Ticket #3351.
2 years ago
Giuseppe Longo 69f841c998 sip/parser: enforce valid chars for sip version
The `is_version_char` function incorrectly allowed characters that are not
part of the valid SIP version "SIP/2.0".

For instance, 'HTTP/1.1' was mistakenly accepted as a valid SIP version,
although it's not.

This commit fixes the issue by updating the condition to strictly
check for the correct version string.
2 years ago
Giuseppe Longo 7e993d5081 sip/parser: accept valid chars
Accepts valid characters as defined in RFC3261.
2 years ago
Giuseppe Longo 8ff80cb84d rust/sip: rustfmt sip module 2 years ago
Jeff Lucovsky f9a20dafc6 mqtt: Improve frame parsing w/mult. PDUs
This commit improves the mqtt parsing of frames to handle multiple PDUs.

Issue: 6592
2 years ago
Philippe Antoine c99d93c257 app-layer/template: use a max number of txs
Ticket: 6773
2 years ago
Philippe Antoine 68b0052018 rust: fix clippy ptr_arg warnings
error: writing `&Vec` instead of `&[_]` involves a new object where a slice will do
   --> src/dns/log.rs:371:29
    |
371 | pub fn dns_print_addr(addr: &Vec<u8>) -> std::string::String {
    |                             ^^^^^^^^ help: change this to: `&[u8]`
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#ptr_arg
2 years ago
Philippe Antoine 80abc22f64 http2: limit number of concurrent transactions
Ticket: 6481

Instead of just setting the old transactions to a drop state so
that they get later cleaned up by Suricata, fail creating new ones.

This is because one call to app-layer parsing can create many
transactions, and quadratic complexity could happen in one
single app-layer parsing because of find_or_create_tx
2 years ago
Philippe Antoine 86de7cffa7 pgsql: parse only PDU when type is unknown
A next PDU may already be in the slice to parse.
Do not skip its parsing, ie do not use rest, but take just
the length of the pdu
2 years ago
Philippe Antoine f52c033e56 pgsql: parse auth message within its bound
If the next PDU is already in the slice next, do not use it and
restrict ourselves to the length of this PDU.
Avoids overconsumption of memory by quadratic complexity, when
having many small PDUS in one big chunk being parsed

Ticket: #6411
2 years ago
Philippe Antoine aff54f29f8 http2: handle reassembly for continuation frames
Ticket: 5926

HTTP2 continuation frames are defined in RFC 9113.
They allow header blocks to be split over multiple HTTP2 frames.
For Suricata to process correctly these header blocks, it
must do the reassembly of the payload of these HTTP2 frames.
Otherwise, we get incomplete decoding for headers names and/or
values while decoding a single frame.

Design is to add a field to the HTTP2 state, as the RFC states that
these continuation frames form a discrete unit :
> Field blocks MUST be transmitted as a contiguous sequence of frames,
> with no interleaved frames of any other type or from any other stream.
So, we do not have to duplicate this reassembly field per stream id.

Another design choice is to wait for the reassembly to be complete
before doing any decoding, to avoid quadratic complexity on partially
decoding of the data.
2 years ago
Philippe Antoine f6e1a20215 detect: dns.opcode as first-class integer
Ticket: 5446

That means it can accept ranges
2 years ago
Philippe Antoine d05f3ac791 detect: integer keywords now accept bitmasks
Ticket: 6648

Like &0x40=0x40 to test for a specific bit set
2 years ago
Philippe Antoine 370ac05419 detect/integer: rust derive for enumerations
Ticket: 6647

Allows keywords using integers to use strings in signature
parsing based on a rust enumeration with a derive.
2 years ago
Philippe Antoine 06c5dd3133 detect: integer keywords now accept negated ranges
Ticket: 6646
2 years ago
Philippe Antoine 3b65a2bb61 detect: integer keywords now support hexadecimal
So that we can write enip.revision: 0x203

Ticket: 6645
2 years ago
Philippe Antoine 38db51b878 rust: make cargo clippy clean
Fixing single_match and manual_find intertwined with SCLogDebug
2 years ago
Philippe Antoine 89936b6530 mqtt: fix logic when setting event
Especially sets transactions to complete when we get a response
without having seen the request, so that the transactions
end up getting cleaned (instead of living/leaking in the state).

Also try to set the event on the relevant transaction, instead
of creating a new transaction just for the purpose of having
the event.

Ticket: #6299
2 years ago
jason taylor 3cb7112aa5 detect: update smb.version keyword
Signed-off-by: jason taylor <jtfas90@gmail.com>
2 years ago
jason taylor bfc0790d87 rust: fix rustfmt warnings for smb detect
Signed-off-by: jason taylor <jtfas90@gmail.com>
2 years ago
Eloy Pérez González 415722dab2 smb: add smb.version keyword
Ticket: #5075

Signed-off-by: jason taylor <jtfas90@gmail.com>
2 years ago
Philippe Antoine 259cdf169e rust: fix single_binding
error: this match could be written as a `let` statement
   --> src/nfs/nfs3_records.rs:747:9
    |
747 | /         match result {
748 | |             (r, request) => {
749 | |                 assert_eq!(r.len(), 0);
750 | |                 assert_eq!(request.handle, expected_handle);
751 | |                 assert_eq!(request.name_vec, br#"bln"#);
752 | |             }
753 | |         }
    | |_________^
2 years ago
Philippe Antoine b141eb9f11 rust: fix single_match
warning: you seem to be trying to use `match` for destructuring a single pattern. Consider using `if let`
   --> src/http2/parser.rs:882:17
    |
882 | /                 match ctx.value {
883 | |                     Some(_) => {
884 | |                         panic!("Unexpected value");
885 | |                     }
886 | |                     None => {}
887 | |                 }
    | |_________________^
2 years ago
Philippe Antoine 9a84681bd9 rust: fix vec_init_then_push
warning: calls to `push` immediately after creation
    --> src/pgsql/parser.rs:1179:9
     |
1179 | /         let mut database_param: Vec<PgsqlParameter> = Vec::new();
1180 | |         database_param.push(database);
     | |______________________________________^
help: consider using the `vec![]` macro: `let database_param: Vec<PgsqlParameter> = vec![..];`
2 years ago
Philippe Antoine 85329f5351 rust: fix zero_prefixed_literal
warning: this is a decimal constant
   --> src/mqtt/parser.rs:888:19
    |
888 |             0x00, 06, /* Topic Length: 6 */
    |                   ^^
    |
2 years ago
Philippe Antoine a8199bf2ca rust: fix assertions_on_constants for assert!(false)
using panic! instead with a string message
2 years ago
Philippe Antoine c49463c86f rust: fix assertions_on_constants for assert!(true)
Which will be optimized away by the compiler
2 years ago
Juliana Fajardini 8d3de85edd pgsql: fix u16 overflow in query data_row
Found by oss-fuzz with quadfuzz.

Cf https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=63113

According to PostgreSQL documentation the maximum number of rows can be
the maximum of tuples that can fit onto max u32 pages - 4,294,967,295 (cf
https://www.postgresql.org/docs/current/limits.html). Some rough
calculations for that indicate that this could go over max u32, so
updating the data_row data type to u64.

Bug #6389
2 years ago
Philippe Antoine 673d13d445 rust: allow clippy::items_after_test_module
As clippy began to complain about jsonbuilder.rs
2 years ago
Jeff Lucovsky f12e026696 mqtt: Move conf code to rust
Issue: 6387

This commit moves the configuration logic to Rust.
2 years ago
Jason Ish 5d5b0509a5 requires: add requires keyword
Add a new rule keyword "requires" that allows a rule to require specific
Suricata versions and/or Suricata features to be enabled.

Example:

  requires: feature geoip, version >= 7.0.0, version < 8;
  requires: version >= 7.0.3 < 8
  requires: version >= 7.0.3 < 8 | >= 8.0.3

Feature: #5972

Co-authored-by: Philippe Antoine <pantoine@oisf.net>
2 years ago
Jason Ish 15ed51f9b8 feature: provide a Rust binding to the feature API
As the feature module is not available for Rust unit tests, a mock
version is also provided.
2 years ago
Juliana Fajardini 1afb485dfa pgsql: remove unused msg field
The `ConsolidatedDataRow` struct had a `length` field that wasn't truly
used.

Related to
Bug #6389
2 years ago
Juliana Fajardini 30ac77ce65 pgsql: add cancel request message
A CanceldRequest can occur after any query request, and is sent over a
new connection, leading to a new flow. It won't take any reply, but, if
processed by the backend, will lead to an ErrorResponse.

Task #6577
2 years ago
Juliana Fajardini 7fa8bbfe43 pgsql: extract length validation into function
This is called so many times that it seems to make sense that we use a
function for this.
2 years ago
Victor Julien b8440a0917 jsonbuilder: add set_int for signed ints
Bug: #6615
2 years ago
Jason Ish f91122e0e8 dns: replace usage of rs_dns_tx_get_query_name with SCDnsTxGetQueryName
SCDnsTxGetQueryName was introduced to allow for getting the query name
in responses as well as requests, so covers the functionality of
rs_dns_tx_get_query_name.
2 years ago
Jason Ish 482325e28b dns: add dns.query.name sticky buffer
This buffer is much like dns.query_name but allows for detection in both
directions.

Feature: #6497
2 years ago
Jason Ish 5f99abb0cb dns: add dns.answer.name keyword
This sticky buffer will allow content matching on the answer names.
While ansers typically only occur in DNS responses, we allow the buffer
to be used in request context as well as the request message format
allows it.

Feature: #6496
2 years ago
Jason Ish 9464d0b14a dns: consolidate DNSRequest and DNSResponse to DNSMessage
DNS request and response messages follow the same format so there is
no reason not to use the same data structure for each. While its
unlikely to see fields like answers in a request, the message format
does not disallow them, so it might be interesting data to have the
ability to log.
2 years ago
Jason Ish e2d7a7f877 dns: rustfmt with latest stable 2 years ago
Philippe Antoine 1b5e04bee3 http2: do not have leading space for response line
Ticket: 6547
2 years ago
Juliana Fajardini bdec2d8ea8 pgsql: don't log password msg if password disabled
If the logging of the password is disabled, there isn't much point in
logging the password message itself.
2 years ago
Juliana Fajardini 9aeeac532e pgsql: remove probe_ts function
With the changes in the probing_ts function, this other one could become
obsolete. Remove it, and directly call `parser::parse_request` when
checking for gaps, instead.
2 years ago
Juliana Fajardini 53d29f652a pgsql: remove unused error handling call 2 years ago
Juliana Fajardini afd6e4dc41 pgsql: don't log unknown message type 2 years ago
Juliana Fajardini 4f85d06192 pgsql: fix probing functions
Some non-pgsql traffic seen by Suricata is mistankenly identified as
pgsql, as the probing function is too generic. Now, if the parser sees
an unknown message type, even if it looks like pgsql, it will fail.

Bug #6080
2 years ago