Commit Graph

607 Commits (ee6d792b02ee09bfb436a2e7bd3a1f87e4aebf9c)

Author SHA1 Message Date
Philippe Antoine c300a859a0 http2: keep track of dynamic headers table size
And evict entries accordingly to maximum size
4 years ago
Philippe Antoine 1a21eea0e9 http2: variable size integers decoded everywhere 4 years ago
Philippe Antoine b21acfbf21 http2: StreamIdReuse frame types exceptions
Also handles better the state so as not to revert from
HTTP2StateHalfClosedClient to HTTP2StateDataServer and not
go to final HTTP2StateClosed
4 years ago
Philippe Antoine 89573060d9 http2: use variable integer for headers lengths 4 years ago
Shivani Bhardwaj 97c67cd5ce dcerpc: fix gap handling
This patch addresses issues discovered by redmine ticket 3896. With the
approach of finding latest record, there was a chance that no record was
found at all and consumed + needed became input length.

e.g.
input_len = 1000
input = 01 05 00 02 00 03 a5 56 00 00 .....

There exists no |05 00| identifier in the rest of the record. After
having parsed |05 00|, there was a search for another record with the
leftover data. Current data length at this point would be 997. Since the
identifier was not found in the data, we calculate the consumed bytes at
this point i.e. consumed = current_data.len() - 1 which would be 996.
Needed bytes still stay at a constant of 2. So, consumed + needed = 996
+ 2 = 998 which is lesser than initial input length of 1000 and hence
the assertion fails.

There could be two fixes to this problem.
1. Finding the latest record but making use of the last found record in
   case no new record was found.
2. Always use the earliest record.

This patch takes the approach (2). It also makes sure that the gap and
current direction are the same.
4 years ago
Philippe Antoine 4f963717f8 fuzz: better configure checks for MSAN building
More compatible check for rust nightly
Checks for CARGO_BUILD_TARGET
Builds release or debug mode independently
4 years ago
Victor Julien 5d985c4271 dcerpc: implement tx free function 5 years ago
Victor Julien 8b2886635f dcerpc/tcp: implement trunc logic
When one side of the connection reaches the STREAM_DEPTH condition the
parser should be aware of this. Otherwise transactions will forever be
waiting for data in that direction.
5 years ago
Victor Julien 4da0d9bdea applayer/rust: expose truncate callback 5 years ago
Shivani Bhardwaj 301454e9e4 dcerpc: fix datatypes while handling stub data 5 years ago
Shivani Bhardwaj 3fd9a3d420 dcerpc: fix datatype for stub data len 5 years ago
Philippe Antoine 222b386102 rust: rebuilds std when building fuzzers
so as to have MSAN working
5 years ago
Philippe Antoine 15447cc672 dceprc: signature rust check with is_char_boundary
before calling split_at which would panic
5 years ago
Philippe Antoine 6694737fcf http2: settings from http1 upgrade 5 years ago
Philippe Antoine 7011bddf84 http2: mimic HTTP1 request from upgrade 5 years ago
Philippe Antoine 9d1b030ff0 http2: first connection upgrade from http1 5 years ago
Philippe Antoine 547d6c2d78 applayer: pass parameter to StateAlloc
This parameter is NULL or the pointer to the previous state
for the previous protocol in the case of a protocol change,
for instance from HTTP1 to HTTP2

This way, the new protocol can use the old protocol context.
For instance, HTTP2 mimicks the HTTP1 request, to have a HTTP2
transaction with both request and response
5 years ago
Sascha Steinbiss ed9fed4958 mqtt: add some extra tests for varint parsing 5 years ago
Philippe Antoine 1a88df7e88 http2: handles incomplete frames after banner
To signal incomplete data, we must return the number of
consumed bytes. When we get a banner and some records, we have
to take into account the number of bytes already consumed by
the banner parsing before reaching an incomplete record.
5 years ago
Philippe Antoine 7ab9a01db2 mqtt: limit size of variable integer 5 years ago
Philippe Antoine e3b28bcf2a http2: returns error in case of index 0
As is documented in RFC 7541, section 6.1
The index value of 0 is not used.  It MUST be treated as a decoding
error if found in an indexed header field representation.
5 years ago
Jason Ish e9fec043b5 rust/util: expose function to test strings for valid UTF-8
rs_check_utf8 will check that the provided string is valid
UTF-8 by converting it to a Rust string and returning true
or false.
5 years ago
Simon Dugas 7f26246ce1 dns: parse and log fields for SOA record type
Added `dns_parse_rdata_soa` to parse SOA fields into an `DNSRDataSOA`
struct.

Added logging for answer and authority SOA records in both version
1 & 2, as well as grouped formats.
5 years ago
Simon Dugas af498fd840 dns: use nom's rest to take all remaining rdata
Using nom's `rest` combinator eliminates the need to call the do_parse
macro for parsing a single element.
5 years ago
Simon Dugas 8005f50647 dns: refactor to handle more rdata formats
Represent rdata as `DNSRData` enum variants instead of `Vec<u8>`.
This will allow parsing/logging of more complex formats like SOA.
5 years ago
Jason Ish 647e304f4b rust/log: minor cleanup
Group functions for setting and getting the log level
together.
5 years ago
Jason Ish ea1338b464 rust: function macro now returns the function name
Borrow a macro from https://github.com/popzxc/stdext-rs that
will give us the Rust function name in SCLog messages in Rust.

As this trick only works on Rust 1.38 and newer, keep the old
macro around and set a feature based on a Rust version test
done during ./configure.
5 years ago
Jason Ish bac8016d17 rust: plugin bootstrap function
Functions written in Rust will need to suricata::plugin::init()
to bootstrap themselves. This bootstrap process sets the log level
within the Rust address space, and hooks up function pointers
that are expected to be set during normal runs of Suricata.
5 years ago
Jason Ish 3de98b3595 rust/log: set the log level with a pure Rust function
Make sure the log level is setup with a pure Rust function, so
when it is set, its set within the address space of the caller.

This is important for Rust plugins where the Rust modules are not
in the address space of the Suricata main process.
5 years ago
Jason Ish ac93ab281d rust/logging: allow log macros to be used by plugins
Fix plugin macros so they can be used by external Rust crates
such as plugins.
5 years ago
Jason Ish 3672cea876 rust/Cargo: build as rlib for plugin linkage
Build Rust code as an rlib, in addition to a staticlib so plugins
can link with the Rust code.
5 years ago
Philippe Antoine b20a8de646 dns: simply skips zero-sized dns requests/responses 5 years ago
Ilya Bakhtin c903441fe7 mqtt: fixed wrong slice access
Slice access esults in rust panic when mqtt ping is processed
5 years ago
Victor Julien 4aa80ac7f6 detect/http2: fix header inspection
Header inspection was overwriting data that was still being
referenced by the detect engine, leading to ASAN issues.
5 years ago
Jason Ish f26d6eaf98 http2: log as http to abstract http and http2 a little
This commit logs http2 as an http event. The idea is to somewhat
normalize http/http2 so common info can be version agnostic.

This puts the http2 specific fields in an "http2" object inside
the "http" object.

HTTP2 headers/values that are in common with HTTP1 are logged
under the "http" object to be compatible with HTTP1 logging.
5 years ago
Jason Ish 6a55606adb http2: log headers in the same format as http (1)
Log the headers in request_headers, and response_headers like
http1 to remain compatible.
5 years ago
Philippe Antoine 1422b18a99 http2: initial support 5 years ago
Jeff Lucovsky be57f6d3cc rust/rdp: Fix use of incorrect buffer
This commit updates the connection confirmation parsing function to use
the correct buffer when determining the packet type.
5 years ago
Shivani Bhardwaj 2529eb35e1 smb: use uuid crate 5 years ago
Jason Ish e6668560e0 rust: only run cbindgen if needed
Only run cbindgen when necessary. This is a bit tricky. When
building a dist we want to unconditionally build the headers.

When going through a "make; sudo make install" type process,
cbindgen should not be run as the headers already exist, are
valid, and the environment under sudo is more often than
not suitable to pick up the Rust toolchains when installed
with rustup.

For the normal "make" case we have the gen/rust-bindings.h file
depend on library file, this will cause it to only be rebuilt
if the code was modified.

For "make dist" we unconditionally create "dist/rust-bindings.h".
This means the generated file could be in 2 locations, so update
configure.ac, and the library search find to find it.

The "gen/rust-bindings.h" should be picked up first if it exists,
for those who develop from a dist archive where "dist/rust-bindings.h"
also exists.

Not completely happy having the same file in 2 locations, but not
sure how else to get the dependency tracking correct.
5 years ago
Shivani Bhardwaj 4c7f55e636 dcerpc: handle gap for TCP streams 5 years ago
Jason Ish 6cff558663 dns: add tailing data to split tcp unit test
Add trailing data to the complete payload to test the case
where data is consumed, but still incomplete.
5 years ago
Jason Ish cc154ce152 dns: fix subtraction overflow in debug message
The math for the return value wasn't updated in the debug log
to match the logic and could lead to a subtraction overflow
5 years ago
Philippe Antoine 80ccbff28a dns: return status from parsing tcp request 5 years ago
Philippe Antoine 1296740471 dns: fix incomplete results
and messages parsing whose length is >= 32768
5 years ago
Philippe Antoine 83567e8732 files: rust closes files even on 0 length chunk 5 years ago
Sascha Steinbiss c31360070b rust/mqtt: add MQTT parser 5 years ago
Victor Julien b99ffd9ece eve: remove unused jansson code 5 years ago
Jeff Lucovsky 961b314b86 dns: Remove parser buffering code 5 years ago
Victor Julien 00cc3c7374 eve/ssh: change hassh logging format
Elastic search didn't accept the 'hassh' and 'hassh.string'. It would
see the first 'hassh' as a string and split the second key into a
object 'hassh' with a string member 'string'. So two different types
for 'hassh', so it rejected it.

This patch mimics the ja3(s) logging by creating a 'hassh' object
with 2 members: 'hash', which holds the md5 representation, and
'string' which holds the string representation.
5 years ago
Philippe Antoine eb5b927787 ssh: fixing incomplete kex parsing
We use the record length from the ssh record header,
and not the size of the parsed data, as is done in other places.
5 years ago
Victor Julien bcd416e6ba nfs: fix 'dangling' files in lossy sessions
In case of lossy connections the NFS state would properly clean up
transactions, including file transactions. However for files the
state was never set to 'truncated', leading to files to stay 'active'.

This would lead these files staying in the NFS's state. In long running
sessions with lots of files this would lead to performance and memory
use issues.

This patch cleans truncates the file that was being transmitted when
a file transaction is being closed.

Based on 65e9a7c31c
5 years ago
Victor Julien fc7cecb4ae nfs: check post-gap timeouts once a second at most
Based on 25f2efe977
5 years ago
Victor Julien 5d65e5a748 nfs: update ts only if it changed
Based on 8aa380600d
5 years ago
Zach Kelly b25de4d99a rdp: remove parser buffering code 5 years ago
Zach Kelly 22a2bee614 rdp/eve: convert to jsonbuilder 5 years ago
Zach Kelly ef397daba3 rdp: rustfmt (update) 5 years ago
Shivani Bhardwaj 35362b7bfa jsonbuilder: run test if not debug-validate 5 years ago
Shivani Bhardwaj 2ce7d98af1 dcerpc: fix tests to have a valid header 5 years ago
Shivani Bhardwaj 9f9670ebdc logging: Add DCERPC logger 5 years ago
Shivani Bhardwaj bab497ab2c dcerpc: Add multi transaction support
DCERPC parser so far provided support for single transactions only.
Extend that to support multiple transactions.

In order for multiple transactions to work, there is always a
transaction identifier for any protocol in its header that lets a
response match the request. In DCERPC, for TCP, that param is call_id in
the header which is a 32 bit field. For UDP, however since it uses
different version of RPC (4.x), this is defined by serial number field
defined in the header. This field however is not contiguous and needs to
be assembled by the provided serial_low and serial_hi fields.
5 years ago
Shivani Bhardwaj 67e7be633c krb: convert to jsonbuilder
Closes redmine ticket 3754.
5 years ago
Shivani Bhardwaj 72dab0a8b7 snmp: convert to jsonbuilder
Closes redmine ticket 3756.
5 years ago
Jason Ish 7ca24041b6 rust: add doc target to build rust docs
Uses "cargo doc --no-deps" to build the documentation just for
our Suricata package. Without --no-deps, documentation will be
build for all our dependencies as well.

The generated documentation will end up in target/doc as HTML.
5 years ago
Jason Ish abc71dc4a5 applayer template (rust): better gap handling example
In the request parser, show checking if a gap was received
and what one example of trying to continue might look like.
5 years ago
Jason Ish b91bb92b7d applayer template (rust): incomplete support
Show how to use the incomplete AppLayerResult type within the
limits of what the template protocol parser can provide.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/3541
5 years ago
Jason Ish 3f615f751b rust app-layer template: add stubs for gap handling 5 years ago
Jason Ish d60671d855 rust/dns: use new flags field to set parser option flags 5 years ago
Jason Ish 53aa967e0b applayer: add flags to parser registration struct
This will allow Rust parsers to register for gap handing from
Rust (some Rust parsers do handle gaps, but they set the flag
from C).
5 years ago
Jason Ish 9d9a901b68 dns: conditional logging
Apply config to newly created response TX.
5 years ago
Victor Julien ac3cf6ff75 detect/config: set config for special cases
Allow app-layer to declare the txs are uni-directional and special
care is needed for applying config.
5 years ago
Victor Julien 5dd4d948d9 app-layer: remove unused detect flags API 5 years ago
Victor Julien c94a5e6392 app-layer/rust: don't use option for GetTxDataFn anymore 5 years ago
Victor Julien 8fe9faecb2 app-layer: remove DetectFlags API. Replaced by AppLayerTxData 5 years ago
Victor Julien 88dd0abb38 rdp: support AppLayerTxData 5 years ago
Victor Julien 9664f73f75 app-layer: remove logged API calls 5 years ago
Victor Julien e0debed0b4 tftp: support AppLayerTxData 5 years ago
Victor Julien 4ff51a0e07 sip: support AppLayerTxData 5 years ago
Victor Julien cc1210c956 ntp: support AppLayerTxData 5 years ago
Victor Julien e0f75157a0 ikev2: support AppLayerTxData 5 years ago
Victor Julien 64e2a27512 applayer/template: support AppLayerTxData 5 years ago
Victor Julien a484bbbe1b dhcp: support AppLayerTxData 5 years ago
Victor Julien 11e2434526 snmp: support AppLayerTxData 5 years ago
Victor Julien 5afe4835ad rfb: support AppLayerTxData 5 years ago
Victor Julien 9f29366c7c krb5: support AppLayerTxData 5 years ago
Victor Julien 2aab1938d7 ssh: support AppLayerTxData 5 years ago
Victor Julien a1e06247a6 dcerpc/udp: support AppLayerTxData 5 years ago
Victor Julien 3202d29325 dcerpc: support AppLayerTxData 5 years ago
Victor Julien 77a95eddd9 smb: support AppLayerTxData 5 years ago
Victor Julien 7a7805cde6 nfs: support AppLayerTxData 5 years ago
Victor Julien fb3bdd8cf3 dns: remove detect_flags and logged now that we use AppLayerTxData 5 years ago
Victor Julien 5665fc8301 app-layer: add ApplyTxConfig API
Optional callback a parser can register for applying configuration
to the 'transaction'. Most parsers have a bidirectional tx. For those
parsers that have different types of transaction handling, this new
callback can be used to properly apply the config.
5 years ago
Victor Julien e15995e2d2 detect: store detect flags in AppLayerTxData 5 years ago
Victor Julien c797c9f09c app-layer: add logger flags to AppLayerTxData 5 years ago
Victor Julien 411f428a38 app-layer: define AppLayerTxData and AppLayerTxConfig
AppLayerTxData is a structure each tx should include that will contain
the common fields the engine needs for tracking logging, detection and
possibly other things.

AppLayerTxConfig will be used by the detection engine to configure
the transaction.
5 years ago
Jason Ish 03efbccfe6 jsonbuilder: set_float, append_float methods
New methods for setting and appending float values.
5 years ago
Philippe Antoine ece29c4210 ssh: fix incomplete return for ssh kex
In the case where we already parsed some records
5 years ago
Philippe Antoine ca6d072297 dcerpc: detect right parsing of empty op version 5 years ago
Emmanuel Thompson 6e5d64f102 detect/asn1: Simplify errors and checks 5 years ago
Emmanuel Thompson 4fc45b5c60 detect/asn1: Update ASN1 struct lifetime
- 'static is only realistic when allocating and leaking it over the
FFI boundary
5 years ago