Commit Graph

542 Commits (49ca070446cdca49afc42d71aaedaa9a1b458897)

Author SHA1 Message Date
Philippe Antoine 9d1b030ff0 http2: first connection upgrade from http1 5 years ago
Philippe Antoine 547d6c2d78 applayer: pass parameter to StateAlloc
This parameter is NULL or the pointer to the previous state
for the previous protocol in the case of a protocol change,
for instance from HTTP1 to HTTP2

This way, the new protocol can use the old protocol context.
For instance, HTTP2 mimicks the HTTP1 request, to have a HTTP2
transaction with both request and response
5 years ago
Sascha Steinbiss ed9fed4958 mqtt: add some extra tests for varint parsing 5 years ago
Philippe Antoine 1a88df7e88 http2: handles incomplete frames after banner
To signal incomplete data, we must return the number of
consumed bytes. When we get a banner and some records, we have
to take into account the number of bytes already consumed by
the banner parsing before reaching an incomplete record.
5 years ago
Philippe Antoine 7ab9a01db2 mqtt: limit size of variable integer 5 years ago
Philippe Antoine e3b28bcf2a http2: returns error in case of index 0
As is documented in RFC 7541, section 6.1
The index value of 0 is not used.  It MUST be treated as a decoding
error if found in an indexed header field representation.
5 years ago
Jason Ish e9fec043b5 rust/util: expose function to test strings for valid UTF-8
rs_check_utf8 will check that the provided string is valid
UTF-8 by converting it to a Rust string and returning true
or false.
5 years ago
Simon Dugas 7f26246ce1 dns: parse and log fields for SOA record type
Added `dns_parse_rdata_soa` to parse SOA fields into an `DNSRDataSOA`
struct.

Added logging for answer and authority SOA records in both version
1 & 2, as well as grouped formats.
5 years ago
Simon Dugas af498fd840 dns: use nom's rest to take all remaining rdata
Using nom's `rest` combinator eliminates the need to call the do_parse
macro for parsing a single element.
5 years ago
Simon Dugas 8005f50647 dns: refactor to handle more rdata formats
Represent rdata as `DNSRData` enum variants instead of `Vec<u8>`.
This will allow parsing/logging of more complex formats like SOA.
5 years ago
Jason Ish 647e304f4b rust/log: minor cleanup
Group functions for setting and getting the log level
together.
5 years ago
Jason Ish ea1338b464 rust: function macro now returns the function name
Borrow a macro from https://github.com/popzxc/stdext-rs that
will give us the Rust function name in SCLog messages in Rust.

As this trick only works on Rust 1.38 and newer, keep the old
macro around and set a feature based on a Rust version test
done during ./configure.
5 years ago
Jason Ish bac8016d17 rust: plugin bootstrap function
Functions written in Rust will need to suricata::plugin::init()
to bootstrap themselves. This bootstrap process sets the log level
within the Rust address space, and hooks up function pointers
that are expected to be set during normal runs of Suricata.
5 years ago
Jason Ish 3de98b3595 rust/log: set the log level with a pure Rust function
Make sure the log level is setup with a pure Rust function, so
when it is set, its set within the address space of the caller.

This is important for Rust plugins where the Rust modules are not
in the address space of the Suricata main process.
5 years ago
Jason Ish ac93ab281d rust/logging: allow log macros to be used by plugins
Fix plugin macros so they can be used by external Rust crates
such as plugins.
5 years ago
Philippe Antoine b20a8de646 dns: simply skips zero-sized dns requests/responses 5 years ago
Ilya Bakhtin c903441fe7 mqtt: fixed wrong slice access
Slice access esults in rust panic when mqtt ping is processed
5 years ago
Victor Julien 4aa80ac7f6 detect/http2: fix header inspection
Header inspection was overwriting data that was still being
referenced by the detect engine, leading to ASAN issues.
5 years ago
Jason Ish f26d6eaf98 http2: log as http to abstract http and http2 a little
This commit logs http2 as an http event. The idea is to somewhat
normalize http/http2 so common info can be version agnostic.

This puts the http2 specific fields in an "http2" object inside
the "http" object.

HTTP2 headers/values that are in common with HTTP1 are logged
under the "http" object to be compatible with HTTP1 logging.
5 years ago
Jason Ish 6a55606adb http2: log headers in the same format as http (1)
Log the headers in request_headers, and response_headers like
http1 to remain compatible.
5 years ago
Philippe Antoine 1422b18a99 http2: initial support 5 years ago
Jeff Lucovsky be57f6d3cc rust/rdp: Fix use of incorrect buffer
This commit updates the connection confirmation parsing function to use
the correct buffer when determining the packet type.
5 years ago
Shivani Bhardwaj 2529eb35e1 smb: use uuid crate 5 years ago
Shivani Bhardwaj 4c7f55e636 dcerpc: handle gap for TCP streams 5 years ago
Jason Ish 6cff558663 dns: add tailing data to split tcp unit test
Add trailing data to the complete payload to test the case
where data is consumed, but still incomplete.
5 years ago
Jason Ish cc154ce152 dns: fix subtraction overflow in debug message
The math for the return value wasn't updated in the debug log
to match the logic and could lead to a subtraction overflow
5 years ago
Philippe Antoine 80ccbff28a dns: return status from parsing tcp request 5 years ago
Philippe Antoine 1296740471 dns: fix incomplete results
and messages parsing whose length is >= 32768
5 years ago
Philippe Antoine 83567e8732 files: rust closes files even on 0 length chunk 5 years ago
Sascha Steinbiss c31360070b rust/mqtt: add MQTT parser 5 years ago
Victor Julien b99ffd9ece eve: remove unused jansson code 5 years ago
Jeff Lucovsky 961b314b86 dns: Remove parser buffering code 5 years ago
Victor Julien 00cc3c7374 eve/ssh: change hassh logging format
Elastic search didn't accept the 'hassh' and 'hassh.string'. It would
see the first 'hassh' as a string and split the second key into a
object 'hassh' with a string member 'string'. So two different types
for 'hassh', so it rejected it.

This patch mimics the ja3(s) logging by creating a 'hassh' object
with 2 members: 'hash', which holds the md5 representation, and
'string' which holds the string representation.
5 years ago
Philippe Antoine eb5b927787 ssh: fixing incomplete kex parsing
We use the record length from the ssh record header,
and not the size of the parsed data, as is done in other places.
5 years ago
Victor Julien bcd416e6ba nfs: fix 'dangling' files in lossy sessions
In case of lossy connections the NFS state would properly clean up
transactions, including file transactions. However for files the
state was never set to 'truncated', leading to files to stay 'active'.

This would lead these files staying in the NFS's state. In long running
sessions with lots of files this would lead to performance and memory
use issues.

This patch cleans truncates the file that was being transmitted when
a file transaction is being closed.

Based on 65e9a7c31c
5 years ago
Victor Julien fc7cecb4ae nfs: check post-gap timeouts once a second at most
Based on 25f2efe977
5 years ago
Victor Julien 5d65e5a748 nfs: update ts only if it changed
Based on 8aa380600d
5 years ago
Zach Kelly b25de4d99a rdp: remove parser buffering code 5 years ago
Zach Kelly 22a2bee614 rdp/eve: convert to jsonbuilder 5 years ago
Zach Kelly ef397daba3 rdp: rustfmt (update) 5 years ago
Shivani Bhardwaj 35362b7bfa jsonbuilder: run test if not debug-validate 5 years ago
Shivani Bhardwaj 2ce7d98af1 dcerpc: fix tests to have a valid header 5 years ago
Shivani Bhardwaj 9f9670ebdc logging: Add DCERPC logger 5 years ago
Shivani Bhardwaj bab497ab2c dcerpc: Add multi transaction support
DCERPC parser so far provided support for single transactions only.
Extend that to support multiple transactions.

In order for multiple transactions to work, there is always a
transaction identifier for any protocol in its header that lets a
response match the request. In DCERPC, for TCP, that param is call_id in
the header which is a 32 bit field. For UDP, however since it uses
different version of RPC (4.x), this is defined by serial number field
defined in the header. This field however is not contiguous and needs to
be assembled by the provided serial_low and serial_hi fields.
5 years ago
Shivani Bhardwaj 67e7be633c krb: convert to jsonbuilder
Closes redmine ticket 3754.
5 years ago
Shivani Bhardwaj 72dab0a8b7 snmp: convert to jsonbuilder
Closes redmine ticket 3756.
5 years ago
Jason Ish abc71dc4a5 applayer template (rust): better gap handling example
In the request parser, show checking if a gap was received
and what one example of trying to continue might look like.
5 years ago
Jason Ish b91bb92b7d applayer template (rust): incomplete support
Show how to use the incomplete AppLayerResult type within the
limits of what the template protocol parser can provide.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/3541
5 years ago
Jason Ish 3f615f751b rust app-layer template: add stubs for gap handling 5 years ago
Jason Ish d60671d855 rust/dns: use new flags field to set parser option flags 5 years ago
Jason Ish 53aa967e0b applayer: add flags to parser registration struct
This will allow Rust parsers to register for gap handing from
Rust (some Rust parsers do handle gaps, but they set the flag
from C).
5 years ago
Jason Ish 9d9a901b68 dns: conditional logging
Apply config to newly created response TX.
5 years ago
Victor Julien ac3cf6ff75 detect/config: set config for special cases
Allow app-layer to declare the txs are uni-directional and special
care is needed for applying config.
5 years ago
Victor Julien 5dd4d948d9 app-layer: remove unused detect flags API 5 years ago
Victor Julien c94a5e6392 app-layer/rust: don't use option for GetTxDataFn anymore 5 years ago
Victor Julien 8fe9faecb2 app-layer: remove DetectFlags API. Replaced by AppLayerTxData 5 years ago
Victor Julien 88dd0abb38 rdp: support AppLayerTxData 5 years ago
Victor Julien 9664f73f75 app-layer: remove logged API calls 5 years ago
Victor Julien e0debed0b4 tftp: support AppLayerTxData 5 years ago
Victor Julien 4ff51a0e07 sip: support AppLayerTxData 5 years ago
Victor Julien cc1210c956 ntp: support AppLayerTxData 5 years ago
Victor Julien e0f75157a0 ikev2: support AppLayerTxData 5 years ago
Victor Julien 64e2a27512 applayer/template: support AppLayerTxData 5 years ago
Victor Julien a484bbbe1b dhcp: support AppLayerTxData 5 years ago
Victor Julien 11e2434526 snmp: support AppLayerTxData 5 years ago
Victor Julien 5afe4835ad rfb: support AppLayerTxData 5 years ago
Victor Julien 9f29366c7c krb5: support AppLayerTxData 5 years ago
Victor Julien 2aab1938d7 ssh: support AppLayerTxData 5 years ago
Victor Julien a1e06247a6 dcerpc/udp: support AppLayerTxData 5 years ago
Victor Julien 3202d29325 dcerpc: support AppLayerTxData 5 years ago
Victor Julien 77a95eddd9 smb: support AppLayerTxData 5 years ago
Victor Julien 7a7805cde6 nfs: support AppLayerTxData 5 years ago
Victor Julien fb3bdd8cf3 dns: remove detect_flags and logged now that we use AppLayerTxData 5 years ago
Victor Julien 5665fc8301 app-layer: add ApplyTxConfig API
Optional callback a parser can register for applying configuration
to the 'transaction'. Most parsers have a bidirectional tx. For those
parsers that have different types of transaction handling, this new
callback can be used to properly apply the config.
5 years ago
Victor Julien e15995e2d2 detect: store detect flags in AppLayerTxData 5 years ago
Victor Julien c797c9f09c app-layer: add logger flags to AppLayerTxData 5 years ago
Victor Julien 411f428a38 app-layer: define AppLayerTxData and AppLayerTxConfig
AppLayerTxData is a structure each tx should include that will contain
the common fields the engine needs for tracking logging, detection and
possibly other things.

AppLayerTxConfig will be used by the detection engine to configure
the transaction.
5 years ago
Jason Ish 03efbccfe6 jsonbuilder: set_float, append_float methods
New methods for setting and appending float values.
5 years ago
Philippe Antoine ece29c4210 ssh: fix incomplete return for ssh kex
In the case where we already parsed some records
5 years ago
Philippe Antoine ca6d072297 dcerpc: detect right parsing of empty op version 5 years ago
Emmanuel Thompson 6e5d64f102 detect/asn1: Simplify errors and checks 5 years ago
Emmanuel Thompson 4fc45b5c60 detect/asn1: Update ASN1 struct lifetime
- 'static is only realistic when allocating and leaking it over the
FFI boundary
5 years ago
Emmanuel Thompson 627e90a4bd detect/asn1: Log out errors
- Failure to parse asn1-max-frames
- Failure on asn1 detection checks
5 years ago
Emmanuel Thompson 88601b1993 detect/asn1: Update relative_offset keyword
- To be consistent with recent C version changes
- Add checks for over/underflows
5 years ago
Emmanuel Thompson 7af6cdb7ec detect/asn1: Update asn1 C files to use rust code
Mark rust extern "C" functions as pub in asn1 module to expose via cbindgen
Update detect-asn1.c/h to use rust functions
5 years ago
Emmanuel Thompson 63704fdf13 rust/asn1: Introduce ASN1 rust module
This module uses the `der-parser` crate to parse ASN1 objects in order to replace src/util-decode-asn1.c
It also handles the parsing of the asn1 keyword rules and detection checks performed in src/detect-asn1.c
5 years ago
Emmanuel Thompson 6b8517dc12 rust: Update der, kerberos and snmp parser dependencies
- The update to der-parser allows us to use the latest API changes
5 years ago
Jason Ish 43b9bfaed4 applayer template (rust): convert to JsonBuilder 5 years ago
Vadym Malakhatko 126597144c eve: add Hassh fields to SSH JSON logger and add ssh log condition 5 years ago
Vadym Malakhatko 536cee3ba9 rust/ssh: add hassh generation
Add generation of hassh fingerprints based on fields in the kexinit record
5 years ago
Jeff Lucovsky d5bb41011c output/ikev2: Convert to JsonBuilder
Convert the IKEV2 Json logging to use JsonBuilder.
5 years ago
Victor Julien 65e9a7c31c smb: fix 'dangling' files in lossy sessions
In case of lossy connections the SMB state would properly clean up
transactions, including file transactions. However for files the
state was never set to 'truncated', leading to files to stay 'active'.

This would lead these files staying in the SMB's state. In long running
sessions with lots of files this would lead to performance and memory
use issues.

This patch cleans truncates the file that was being transmitted when
a file transaction is being closed.
5 years ago
Victor Julien 25f2efe977 smb: check post-gap timeouts once a second at most 5 years ago
Victor Julien 8aa380600d smb: update ts only if it changed 5 years ago
Jeff Lucovsky 8c5c949cfa output/tftp: Convert to JsonBuilder
This commit converts the TFTP logging mechanisms to JsonBuilder.
5 years ago
Jason Ish 07e88a7479 jsonbuilder: add debug_validate to state
If debug validation is enabled, panic on invalid state errors.

For example, calling close on an already closed jsonbuilder
object.
5 years ago
Jason Ish ca6b70ea1b rust: macro debug_validate_fail to fail with message
Add a new debug_validate macro that unconditionally panics
with a message. Useful in Rust pattern matching.
5 years ago
Shivani Bhardwaj a7535099b4 smb/eve: convert to jsonbuilder
Closes redmine ticket 3712.
5 years ago
Victor Julien 79681bf655 app-layer: remove old MPMId API calls
Had been deprecated and non-functional since 2017.
6 years ago
Jason Ish a545cdef6a jsonbuilder: setter for formatted data
Create a method to set preformatted data that contains the key
and the value already formatted.

This is an optimization for static data.
6 years ago