aiden
/
suricata
mirror of https://github.com/OISF/suricata
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
12,411 Commits
7 Branches
155 Tags
192 MiB
master
main-7.0.x
master-6.0.x
master-5.0.x
master-4.1.x
master-4.0.x
master-3.2.x
suricata-7.0.8
suricata-7.0.7
suricata-7.0.6
suricata-6.0.20
suricata-7.0.5
suricata-6.0.19
suricata-6.0.18
suricata-7.0.4
suricata-6.0.17
suricata-7.0.3
suricata-6.0.16
suricata-7.0.2
suricata-6.0.15
suricata-7.0.1
suricata-6.0.14
suricata-7.0.0
suricata-7.0.0-rc2
suricata-6.0.13
suricata-6.0.12
suricata-6.0.11
suricata-7.0.0-rc1
suricata-6.0.10
suricata-6.0.9
suricata-7.0.0-beta1
suricata-6.0.8
suricata-6.0.7
suricata-6.0.6
suricata-5.0.10
suricata-6.0.5
suricata-5.0.9
suricata-6.0.4
suricata-5.0.8
suricata-6.0.3
suricata-5.0.7
suricata-6.0.2
suricata-5.0.6
suricata-6.0.1
suricata-5.0.5
suricata-4.1.10
suricata-4.1.9
suricata-5.0.4
suricata-6.0.0
suricata-6.0.0-rc1
suricata-6.0.0-beta1
suricata-5.0.3
suricata-4.1.8
suricata-4.1.7
suricata-5.0.2
suricata-4.1.6
suricata-5.0.1
suricata-5.0.0
suricata-5.0.0-rc1
suricata-4.1.5
suricata-5.0.0-beta1
suricata-4.1.4
suricata-4.1.3
suricata-4.0.7
suricata-4.1.2
suricata-4.1.1
suricata-4.1.0
suricata-4.0.6
suricata-4.1.0-rc2
suricata-4.1.0-rc1
suricata-4.0.5
suricata-4.1.0-beta1
suricata-4.0.4
suricata-4.0.3
suricata-4.0.2
suricata-3.2.5
suricata-4.0.1
suricata-3.2.4
suricata-4.0.0
suricata-4.0.0-rc2
suricata-3.2.3
suricata-4.0.0-rc1
suricata-4.0.0-beta1
suricata-3.2.2
suricata-3.1.4
suricata-3.2.1
suricata-3.2
suricata-3.2RC1
suricata-3.1.3
suricata-3.2beta1
suricata-3.1.2
suricata-3.1.1
suricata-3.1
suricata-3.0.2
suricata-3.1RC1
suricata-3.0.1
suricata-3.0.1RC1
suricata-3.0
suricata-2.0.11
suricata-3.0RC3
suricata-3.0RC2
suricata-3.0RC1
suricata-2.0.10
suricata-2.0.9
suricata-2.1beta4
suricata-2.0.8
suricata-2.0.7
suricata-2.1beta3
suricata-2.0.6
suricata-2.0.5
suricata-2.1beta2
suricata-2.0.4
suricata-2.1beta1
suricata-2.0.3
suricata-2.0.2
suricata-2.0.1
suricata-2.0.1rc1
suricata-2.0
suricata-2.0rc3
suricata-2.0rc2
suricata-2.0rc1
suricata-2.0beta2
suricata-1.4.7
suricata-1.4.6
suricata-1.4.5
suricata-2.0beta1
suricata-1.4.4
suricata-1.4.3
suricata-1.4.2
suricata-1.4.1
suricata-1.3.6
suricata-1.4
suricata-1.3.5
suricata-1.4rc1
suricata-1.3.4
suricata-1.4beta3
suricata-1.3.3
suricata-1.4beta2
suricata-1.3.2
suricata-1.4beta1
suricata-1.3.1
suricata-1.3
suricata-1.3rc1
suricata-1.3beta2
suricata-1.3beta1
suricata-1.2.1
suricata-1.2
suricata-1.2rc1
suricata-1.2beta1
suricata-1.1.1
suricata-0.8.2
suricata-1.0.0
suricata-1.0.1
suricata-1.0.2
suricata-1.0.3
suricata-1.0.4
suricata-1.0.5
suricata-1.1
suricata-1.1beta1
suricata-1.1beta2
suricata-1.1beta3
suricata-1.1rc1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '23768c7181'
${ noResults }
Commit Graph

771 Commits (23768c71814aa267672de772d43394727b3428a4)

Author SHA1 Message Date
Victor Julien 67b5295bbc dcerpc/tcp: add missing detect state cleanup 4 years ago
Philippe Antoine 1fd6f5bc61 http2: asymetric sizes for headers tables 
The headers table from client to server
and the one from server to client
may have different maximum sizes
(even if both endpoints have to keep both tables)
 4 years ago
Philippe Antoine aaa69fe3c5 smb: resistance against padding evasions 
Scenario is use of dummy padding in write AndX request
or other similar commands using a data offset.

Parsing skips now these dummy bytes, and generates one event
 4 years ago
Philippe Antoine caa7946888 smb: adds file overlap event against evasions 
Evasion scenario is
- a first dummy write of one byte at offset 0 is done
- the second full write of EICAR at offset 0 is then done
and does not trigger detection

The last write had the final value, and as we cannot "cancel"
the previous write, we set an event which is then transformed into
an app-layer decoder alert
 4 years ago
Philippe Antoine 3e96f96153 rdp: fix incomplete result 
Aggregating the consumed bytes
 4 years ago
Jason Ish 0529a00ffd dhcp: set unidirection transaction flag 4 years ago
Jason Ish 3036ec4db0 ikev2: set unidirection transaction flag 4 years ago
Jason Ish ff674d0cd0 mqtt: set unidirection transaction flag 4 years ago
Jason Ish 54d5f336d4 sip: set unidirection transaction flag 4 years ago
Jason Ish 1d40d0c5f9 rdp: set unidirection transaction flag 4 years ago
Jason Ish f7dee602e9 krb5: set unidirection transaction flag 4 years ago
Jason Ish 984d3c7f20 ntp: set unidirection transaction flag 4 years ago
Jason Ish fc7d59d92f snmp: set unidirectional transaction flag 4 years ago
Philippe Antoine 64fcba228b http2: complete parsing of priority frames 4 years ago
Philippe Antoine c300a859a0 http2: keep track of dynamic headers table size 
And evict entries accordingly to maximum size
 4 years ago
Philippe Antoine 1a21eea0e9 http2: variable size integers decoded everywhere 4 years ago
Philippe Antoine b21acfbf21 http2: StreamIdReuse frame types exceptions 
Also handles better the state so as not to revert from
HTTP2StateHalfClosedClient to HTTP2StateDataServer and not
go to final HTTP2StateClosed
 4 years ago
Philippe Antoine 89573060d9 http2: use variable integer for headers lengths 4 years ago
Shivani Bhardwaj 97c67cd5ce dcerpc: fix gap handling 
This patch addresses issues discovered by redmine ticket 3896. With the
approach of finding latest record, there was a chance that no record was
found at all and consumed + needed became input length.

e.g.
input_len = 1000
input = 01 05 00 02 00 03 a5 56 00 00 .....

There exists no |05 00| identifier in the rest of the record. After
having parsed |05 00|, there was a search for another record with the
leftover data. Current data length at this point would be 997. Since the
identifier was not found in the data, we calculate the consumed bytes at
this point i.e. consumed = current_data.len() - 1 which would be 996.
Needed bytes still stay at a constant of 2. So, consumed + needed = 996
+ 2 = 998 which is lesser than initial input length of 1000 and hence
the assertion fails.

There could be two fixes to this problem.
1. Finding the latest record but making use of the last found record in
   case no new record was found.
2. Always use the earliest record.

This patch takes the approach (2). It also makes sure that the gap and
current direction are the same.
 4 years ago
Philippe Antoine 4f963717f8 fuzz: better configure checks for MSAN building 
More compatible check for rust nightly
Checks for CARGO_BUILD_TARGET
Builds release or debug mode independently
 4 years ago
Victor Julien 5d985c4271 dcerpc: implement tx free function 4 years ago
Victor Julien 8b2886635f dcerpc/tcp: implement trunc logic 
When one side of the connection reaches the STREAM_DEPTH condition the
parser should be aware of this. Otherwise transactions will forever be
waiting for data in that direction.
 4 years ago
Victor Julien 4da0d9bdea applayer/rust: expose truncate callback 4 years ago
Shivani Bhardwaj 301454e9e4 dcerpc: fix datatypes while handling stub data 4 years ago
Shivani Bhardwaj 3fd9a3d420 dcerpc: fix datatype for stub data len 4 years ago
Philippe Antoine 222b386102 rust: rebuilds std when building fuzzers 
so as to have MSAN working
 5 years ago
Philippe Antoine 15447cc672 dceprc: signature rust check with is_char_boundary 
before calling split_at which would panic
 5 years ago
Philippe Antoine 6694737fcf http2: settings from http1 upgrade 5 years ago
Philippe Antoine 7011bddf84 http2: mimic HTTP1 request from upgrade 5 years ago
Philippe Antoine 9d1b030ff0 http2: first connection upgrade from http1 5 years ago
Philippe Antoine 547d6c2d78 applayer: pass parameter to StateAlloc 
This parameter is NULL or the pointer to the previous state
for the previous protocol in the case of a protocol change,
for instance from HTTP1 to HTTP2

This way, the new protocol can use the old protocol context.
For instance, HTTP2 mimicks the HTTP1 request, to have a HTTP2
transaction with both request and response
 5 years ago
Sascha Steinbiss ed9fed4958 mqtt: add some extra tests for varint parsing 5 years ago
Philippe Antoine 1a88df7e88 http2: handles incomplete frames after banner 
To signal incomplete data, we must return the number of
consumed bytes. When we get a banner and some records, we have
to take into account the number of bytes already consumed by
the banner parsing before reaching an incomplete record.
 5 years ago
Philippe Antoine 7ab9a01db2 mqtt: limit size of variable integer 5 years ago
Philippe Antoine e3b28bcf2a http2: returns error in case of index 0 
As is documented in RFC 7541, section 6.1
The index value of 0 is not used.  It MUST be treated as a decoding
error if found in an indexed header field representation.
 5 years ago
Jason Ish e9fec043b5 rust/util: expose function to test strings for valid UTF-8 
rs_check_utf8 will check that the provided string is valid
UTF-8 by converting it to a Rust string and returning true
or false.
 5 years ago
Simon Dugas 7f26246ce1 dns: parse and log fields for SOA record type 
Added `dns_parse_rdata_soa` to parse SOA fields into an `DNSRDataSOA`
struct.

Added logging for answer and authority SOA records in both version
1 & 2, as well as grouped formats.
 5 years ago
Simon Dugas af498fd840 dns: use nom's rest to take all remaining rdata 
Using nom's `rest` combinator eliminates the need to call the do_parse
macro for parsing a single element.
 5 years ago
Simon Dugas 8005f50647 dns: refactor to handle more rdata formats 
Represent rdata as `DNSRData` enum variants instead of `Vec<u8>`.
This will allow parsing/logging of more complex formats like SOA.
 5 years ago
Jason Ish 647e304f4b rust/log: minor cleanup 
Group functions for setting and getting the log level
together.
 5 years ago
Jason Ish ea1338b464 rust: function macro now returns the function name 
Borrow a macro from https://github.com/popzxc/stdext-rs that
will give us the Rust function name in SCLog messages in Rust.

As this trick only works on Rust 1.38 and newer, keep the old
macro around and set a feature based on a Rust version test
done during ./configure.
 5 years ago
Jason Ish bac8016d17 rust: plugin bootstrap function 
Functions written in Rust will need to suricata::plugin::init()
to bootstrap themselves. This bootstrap process sets the log level
within the Rust address space, and hooks up function pointers
that are expected to be set during normal runs of Suricata.
 5 years ago
Jason Ish 3de98b3595 rust/log: set the log level with a pure Rust function 
Make sure the log level is setup with a pure Rust function, so
when it is set, its set within the address space of the caller.

This is important for Rust plugins where the Rust modules are not
in the address space of the Suricata main process.
 5 years ago
Jason Ish ac93ab281d rust/logging: allow log macros to be used by plugins 
Fix plugin macros so they can be used by external Rust crates
such as plugins.
 5 years ago
Jason Ish 3672cea876 rust/Cargo: build as rlib for plugin linkage 
Build Rust code as an rlib, in addition to a staticlib so plugins
can link with the Rust code.
 5 years ago
Philippe Antoine b20a8de646 dns: simply skips zero-sized dns requests/responses 5 years ago
Ilya Bakhtin c903441fe7 mqtt: fixed wrong slice access 
Slice access esults in rust panic when mqtt ping is processed
 5 years ago
Victor Julien 4aa80ac7f6 detect/http2: fix header inspection 
Header inspection was overwriting data that was still being
referenced by the detect engine, leading to ASAN issues.
 5 years ago
Jason Ish f26d6eaf98 http2: log as http to abstract http and http2 a little 
This commit logs http2 as an http event. The idea is to somewhat
normalize http/http2 so common info can be version agnostic.

This puts the http2 specific fields in an "http2" object inside
the "http" object.

HTTP2 headers/values that are in common with HTTP1 are logged
under the "http" object to be compatible with HTTP1 logging.
 5 years ago
Jason Ish 6a55606adb http2: log headers in the same format as http (1) 
Log the headers in request_headers, and response_headers like
http1 to remain compatible.
 5 years ago
Philippe Antoine 1422b18a99 http2: initial support 5 years ago
Jeff Lucovsky be57f6d3cc rust/rdp: Fix use of incorrect buffer 
This commit updates the connection confirmation parsing function to use
the correct buffer when determining the packet type.
 5 years ago
Shivani Bhardwaj 2529eb35e1 smb: use uuid crate 5 years ago
Jason Ish e6668560e0 rust: only run cbindgen if needed 
Only run cbindgen when necessary. This is a bit tricky. When
building a dist we want to unconditionally build the headers.

When going through a "make; sudo make install" type process,
cbindgen should not be run as the headers already exist, are
valid, and the environment under sudo is more often than
not suitable to pick up the Rust toolchains when installed
with rustup.

For the normal "make" case we have the gen/rust-bindings.h file
depend on library file, this will cause it to only be rebuilt
if the code was modified.

For "make dist" we unconditionally create "dist/rust-bindings.h".
This means the generated file could be in 2 locations, so update
configure.ac, and the library search find to find it.

The "gen/rust-bindings.h" should be picked up first if it exists,
for those who develop from a dist archive where "dist/rust-bindings.h"
also exists.

Not completely happy having the same file in 2 locations, but not
sure how else to get the dependency tracking correct.
 5 years ago
Shivani Bhardwaj 4c7f55e636 dcerpc: handle gap for TCP streams 5 years ago
Jason Ish 6cff558663 dns: add tailing data to split tcp unit test 
Add trailing data to the complete payload to test the case
where data is consumed, but still incomplete.
 5 years ago
Jason Ish cc154ce152 dns: fix subtraction overflow in debug message 
The math for the return value wasn't updated in the debug log
to match the logic and could lead to a subtraction overflow
 5 years ago
Philippe Antoine 80ccbff28a dns: return status from parsing tcp request 5 years ago
Philippe Antoine 1296740471 dns: fix incomplete results 
and messages parsing whose length is >= 32768
 5 years ago
Philippe Antoine 83567e8732 files: rust closes files even on 0 length chunk 5 years ago
Sascha Steinbiss c31360070b rust/mqtt: add MQTT parser 5 years ago
Victor Julien b99ffd9ece eve: remove unused jansson code 5 years ago
Jeff Lucovsky 961b314b86 dns: Remove parser buffering code 5 years ago
Victor Julien 00cc3c7374 eve/ssh: change hassh logging format 
Elastic search didn't accept the 'hassh' and 'hassh.string'. It would
see the first 'hassh' as a string and split the second key into a
object 'hassh' with a string member 'string'. So two different types
for 'hassh', so it rejected it.

This patch mimics the ja3(s) logging by creating a 'hassh' object
with 2 members: 'hash', which holds the md5 representation, and
'string' which holds the string representation.
 5 years ago
Philippe Antoine eb5b927787 ssh: fixing incomplete kex parsing 
We use the record length from the ssh record header,
and not the size of the parsed data, as is done in other places.
 5 years ago
Victor Julien bcd416e6ba nfs: fix 'dangling' files in lossy sessions 
In case of lossy connections the NFS state would properly clean up
transactions, including file transactions. However for files the
state was never set to 'truncated', leading to files to stay 'active'.

This would lead these files staying in the NFS's state. In long running
sessions with lots of files this would lead to performance and memory
use issues.

This patch cleans truncates the file that was being transmitted when
a file transaction is being closed.

Based on 65e9a7c31c
 5 years ago
Victor Julien fc7cecb4ae nfs: check post-gap timeouts once a second at most 
Based on 25f2efe977
 5 years ago
Victor Julien 5d65e5a748 nfs: update ts only if it changed 
Based on 8aa380600d
 5 years ago
Zach Kelly b25de4d99a rdp: remove parser buffering code 5 years ago
Zach Kelly 22a2bee614 rdp/eve: convert to jsonbuilder 5 years ago
Zach Kelly ef397daba3 rdp: rustfmt (update) 5 years ago
Shivani Bhardwaj 35362b7bfa jsonbuilder: run test if not debug-validate 5 years ago
Shivani Bhardwaj 2ce7d98af1 dcerpc: fix tests to have a valid header 5 years ago
Shivani Bhardwaj 9f9670ebdc logging: Add DCERPC logger 5 years ago
Shivani Bhardwaj bab497ab2c dcerpc: Add multi transaction support 
DCERPC parser so far provided support for single transactions only.
Extend that to support multiple transactions.

In order for multiple transactions to work, there is always a
transaction identifier for any protocol in its header that lets a
response match the request. In DCERPC, for TCP, that param is call_id in
the header which is a 32 bit field. For UDP, however since it uses
different version of RPC (4.x), this is defined by serial number field
defined in the header. This field however is not contiguous and needs to
be assembled by the provided serial_low and serial_hi fields.
 5 years ago
Shivani Bhardwaj 67e7be633c krb: convert to jsonbuilder 
Closes redmine ticket 3754.
 5 years ago
Shivani Bhardwaj 72dab0a8b7 snmp: convert to jsonbuilder 
Closes redmine ticket 3756.
 5 years ago
Jason Ish 7ca24041b6 rust: add doc target to build rust docs 
Uses "cargo doc --no-deps" to build the documentation just for
our Suricata package. Without --no-deps, documentation will be
build for all our dependencies as well.

The generated documentation will end up in target/doc as HTML.
 5 years ago
Jason Ish abc71dc4a5 applayer template (rust): better gap handling example 
In the request parser, show checking if a gap was received
and what one example of trying to continue might look like.
 5 years ago
Jason Ish b91bb92b7d applayer template (rust): incomplete support 
Show how to use the incomplete AppLayerResult type within the
limits of what the template protocol parser can provide.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/3541
 5 years ago
Jason Ish 3f615f751b rust app-layer template: add stubs for gap handling 5 years ago
Jason Ish d60671d855 rust/dns: use new flags field to set parser option flags 5 years ago
Jason Ish 53aa967e0b applayer: add flags to parser registration struct 
This will allow Rust parsers to register for gap handing from
Rust (some Rust parsers do handle gaps, but they set the flag
from C).
 5 years ago
Jason Ish 9d9a901b68 dns: conditional logging 
Apply config to newly created response TX.
 5 years ago
Victor Julien ac3cf6ff75 detect/config: set config for special cases 
Allow app-layer to declare the txs are uni-directional and special
care is needed for applying config.
 5 years ago
Victor Julien 5dd4d948d9 app-layer: remove unused detect flags API 5 years ago
Victor Julien c94a5e6392 app-layer/rust: don't use option for GetTxDataFn anymore 5 years ago
Victor Julien 8fe9faecb2 app-layer: remove DetectFlags API. Replaced by AppLayerTxData 5 years ago
Victor Julien 88dd0abb38 rdp: support AppLayerTxData 5 years ago
Victor Julien 9664f73f75 app-layer: remove logged API calls 5 years ago
Victor Julien e0debed0b4 tftp: support AppLayerTxData 5 years ago
Victor Julien 4ff51a0e07 sip: support AppLayerTxData 5 years ago
Victor Julien cc1210c956 ntp: support AppLayerTxData 5 years ago
Victor Julien e0f75157a0 ikev2: support AppLayerTxData 5 years ago
Victor Julien 64e2a27512 applayer/template: support AppLayerTxData 5 years ago
Victor Julien a484bbbe1b dhcp: support AppLayerTxData 5 years ago
Victor Julien 11e2434526 snmp: support AppLayerTxData 5 years ago
Victor Julien 5afe4835ad rfb: support AppLayerTxData 5 years ago
Victor Julien 9f29366c7c krb5: support AppLayerTxData 5 years ago
Victor Julien 2aab1938d7 ssh: support AppLayerTxData 5 years ago
Victor Julien a1e06247a6 dcerpc/udp: support AppLayerTxData 5 years ago
Victor Julien 3202d29325 dcerpc: support AppLayerTxData 5 years ago
Victor Julien 77a95eddd9 smb: support AppLayerTxData 5 years ago
Victor Julien 7a7805cde6 nfs: support AppLayerTxData 5 years ago
Victor Julien fb3bdd8cf3 dns: remove detect_flags and logged now that we use AppLayerTxData 5 years ago
Victor Julien 5665fc8301 app-layer: add ApplyTxConfig API 
Optional callback a parser can register for applying configuration
to the 'transaction'. Most parsers have a bidirectional tx. For those
parsers that have different types of transaction handling, this new
callback can be used to properly apply the config.
 5 years ago
Victor Julien e15995e2d2 detect: store detect flags in AppLayerTxData 5 years ago
Victor Julien c797c9f09c app-layer: add logger flags to AppLayerTxData 5 years ago
Victor Julien 411f428a38 app-layer: define AppLayerTxData and AppLayerTxConfig 
AppLayerTxData is a structure each tx should include that will contain
the common fields the engine needs for tracking logging, detection and
possibly other things.

AppLayerTxConfig will be used by the detection engine to configure
the transaction.
 5 years ago
Jason Ish 03efbccfe6 jsonbuilder: set_float, append_float methods 
New methods for setting and appending float values.
 5 years ago
Philippe Antoine ece29c4210 ssh: fix incomplete return for ssh kex 
In the case where we already parsed some records
 5 years ago
Philippe Antoine ca6d072297 dcerpc: detect right parsing of empty op version 5 years ago
Emmanuel Thompson 6e5d64f102 detect/asn1: Simplify errors and checks 5 years ago
Emmanuel Thompson 4fc45b5c60 detect/asn1: Update ASN1 struct lifetime 
- 'static is only realistic when allocating and leaking it over the
FFI boundary
 5 years ago
Emmanuel Thompson 627e90a4bd detect/asn1: Log out errors 
- Failure to parse asn1-max-frames
- Failure on asn1 detection checks
 5 years ago
Emmanuel Thompson 88601b1993 detect/asn1: Update relative_offset keyword 
- To be consistent with recent C version changes
- Add checks for over/underflows
 5 years ago
Emmanuel Thompson 7af6cdb7ec detect/asn1: Update asn1 C files to use rust code 
Mark rust extern "C" functions as pub in asn1 module to expose via cbindgen
Update detect-asn1.c/h to use rust functions
 5 years ago
Emmanuel Thompson 63704fdf13 rust/asn1: Introduce ASN1 rust module 
This module uses the `der-parser` crate to parse ASN1 objects in order to replace src/util-decode-asn1.c
It also handles the parsing of the asn1 keyword rules and detection checks performed in src/detect-asn1.c
 5 years ago
Emmanuel Thompson 6b8517dc12 rust: Update der, kerberos and snmp parser dependencies 
- The update to der-parser allows us to use the latest API changes
 5 years ago
Jason Ish 43b9bfaed4 applayer template (rust): convert to JsonBuilder 5 years ago
Vadym Malakhatko 126597144c eve: add Hassh fields to SSH JSON logger and add ssh log condition 5 years ago
Vadym Malakhatko 536cee3ba9 rust/ssh: add hassh generation 
Add generation of hassh fingerprints based on fields in the kexinit record
 5 years ago
Jeff Lucovsky d5bb41011c output/ikev2: Convert to JsonBuilder 
Convert the IKEV2 Json logging to use JsonBuilder.
 5 years ago
Victor Julien 65e9a7c31c smb: fix 'dangling' files in lossy sessions 
In case of lossy connections the SMB state would properly clean up
transactions, including file transactions. However for files the
state was never set to 'truncated', leading to files to stay 'active'.

This would lead these files staying in the SMB's state. In long running
sessions with lots of files this would lead to performance and memory
use issues.

This patch cleans truncates the file that was being transmitted when
a file transaction is being closed.
 5 years ago
Victor Julien 25f2efe977 smb: check post-gap timeouts once a second at most 5 years ago
Victor Julien 8aa380600d smb: update ts only if it changed 5 years ago
Jeff Lucovsky 8c5c949cfa output/tftp: Convert to JsonBuilder 
This commit converts the TFTP logging mechanisms to JsonBuilder.
 5 years ago
Jason Ish 07e88a7479 jsonbuilder: add debug_validate to state 
If debug validation is enabled, panic on invalid state errors.

For example, calling close on an already closed jsonbuilder
object.
 5 years ago
Jason Ish ca6b70ea1b rust: macro debug_validate_fail to fail with message 
Add a new debug_validate macro that unconditionally panics
with a message. Useful in Rust pattern matching.
 5 years ago
Shivani Bhardwaj a7535099b4 smb/eve: convert to jsonbuilder 
Closes redmine ticket 3712.
 5 years ago
Jason Ish 639f3d265e rust: lock to nom 5.1.1 
5.1.2 pulls in dependencies that don't build on Rust 1.34.
 5 years ago
Victor Julien 79681bf655 app-layer: remove old MPMId API calls 
Had been deprecated and non-functional since 2017.
 5 years ago
Jason Ish a545cdef6a jsonbuilder: setter for formatted data 
Create a method to set preformatted data that contains the key
and the value already formatted.

This is an optimization for static data.
 5 years ago
Jason Ish e3b7c58218 jsonbuilder: export {set,append}_string_from_bytes to C 5 years ago
Jason Ish f184bcc10e jsonbuilder: use Box::from_raw instead of transmute to free 
I think this is a bad use of transmute, while the end result
is the same, Box::from_raw is more correct as we created this
pointer with Box::into_raw.
 5 years ago
Philippe Antoine baf5f52f22 ssh/eve: convert to jsonbuilder 5 years ago
Victor Julien b3b5802c85 eve/nfs: switch output to jsonbuilder 5 years ago
Jason Ish 6ce9b2972b rdp: enable by default 
Redmine issue:
https://redmine.openinfosecfoundation.org/issues/3255
 5 years ago
Jason Ish 5a7ba62493 sip: enable by default 
Redmine issue:
https://redmine.openinfosecfoundation.org/issues/3256
 5 years ago
Jason Ish 36d687580a rfb/eve: convert to jsonbuilder 5 years ago
Jason Ish 60bfbd43fd jsonbuilder: add reset marks 
Add methods to get the state of a JsonBuilder (called a mark),
then allow restoring to the mark.
 5 years ago
Jason Ish ddb22549be rust: allow some clippy lints without warning 
Suppresses some clippy lints that have more to do with style
than anything else, to reduce the amount of noise in the
clippy output.
 5 years ago
Jason Ish 03cf3dcd6d dns/eve: convert to jsonbuilder 5 years ago
Jason Ish 6a70d6bb6e sip/eve: convert to jsonbuilder 5 years ago
Jason Ish deed0541bb dhcp/eve: convert to jsonbuilder 5 years ago
Jason Ish 942dd08654 jsonbuilder: new module for generating json 
JsonBuilder is a Rust module for creating JSON output. Unlike
Jansson, the final JSON string is built up as items are added,
instead of building up an object tree and rendering it when
done.

The idea is to create a more efficient JSON serializer instead
of a flexible one.
 5 years ago
Jason Ish c1f4edc434 rust: bring back libc as a dependency 
Its already pulled in by some of other dependencies so adds zero
extra weight, and provides handy definitions for basic functions
like free().
 5 years ago
Jason Ish 5513b4ed0b rust/json: expose libjansson json_dumps 
This will be temporarily used by JsonBuilder to add the ability
to extend JsonBuilder with Jansson's json_t types.
 5 years ago
Victor Julien 9fd326b6c0 ssh: minor cleanups in incomplete handling 5 years ago
Victor Julien 3a2434ed4d app-layer: support Copy and Clone traits in AppLayerResult 5 years ago
Victor Julien b0288da686 app-layer: add methods to get status from AppLayerResult 5 years ago
Philippe Antoine 6373071aa3 ssh: handles incomplete record after banner 
To signal incomplete data, we must return the number of
consumed bytes. When we get a banner and some records, we have
to take into account the number of bytes already consumed by
the banner parsing before reaching an incomplete record.
 5 years ago
Philippe Antoine 69b4fffdae parse: move SSH parser from C to Rust 5 years ago
Philippe Antoine cb3c478525 style: compressed function headers for rustfmt 5 years ago
Philippe Antoine 6b2e7dde7d rust: export enums definition to C 5 years ago
Shivani Bhardwaj 80adf7d1cf smb: Import constants from DCERPC 
Remove DCERPC constants to avoid duplicate name errors. Import the
required constants from DCERPC implementation.
 5 years ago
Shivani Bhardwaj 6457754fd6 dcerpc: Replace C function calls with Rust 
All the dead code in C after the Rust implementation is hereby removed.
Invalid/migrated tests have also been deleted.
All the function calls in C have been replaced with appropriate calls to
Rust functions. Same has been done for smb/detect.rs as a part of this
migration.
 5 years ago
Shivani Bhardwaj 8036202c7b rust: Add DCERPC parser 
This parser rewrites the DCE/RPC protocol implementation of Suricata
in Rust. More tests have been added to improve the coverage and some
fixes have been made to the tests already written in C. Most of the
valid tests from C have been imported to Rust.

File anatomy

src/dcerpc.rs
This file contains the implementation of single transactions in DCE/RPC
over TCP. It takes care of REQUEST, RESPONSE, BIND and BINDACK business
logic before and after the data parsing. DCERPCState holds the state
corresponding to a particular transaction and handles all important
aspects. It also defines any common structures and constants required
for DCE/RPC parsing irrespective of the carrier protocol.

src/dcerpc_udp.rs
This file contains the implementation of single transactions in DCE/RPC
over UDP. It takes care of REQUEST and RESPONSE parsing. It borrows the
Request and Response structs from src/dcerpc.rs.

src/detect.rs
This file contains the implementation of dce_iface and opnum detect
keywords. Both the parsing and the matching is taken care of by
functions in this file. Tests have been rewritten with the test data
from C.

src/parser.rs
This file contains all the nom parsers written for DCERPCRequest,
DCERPCResponse, DCERPCBind, DCERPCBindAck, DCERPCHeader, DCERPCHdrUdp.
It also implements functions to assemble and convert UUIDs. All the
fields have their endianness defined unless its an 8bit field or an
unusable one, then it's little endian but it won't make any difference.

src/mod.rs
This file contains all the modules of dcerpc folder which should be
taken into account during compilation.

Function calls

This is a State-wise implementation of the protocol for single
transaction only i.e. a valid state object is required to parse any
record. Function calls start with the app layer parser in C which
detects the application layer protocol to be DCE/RPC and calls the
appropriate functions in C which in turn make a call to these functions
in Rust using FFI. All the necessary information is passed from C to the
parsers and handlers in Rust.

Implementation

When a batch of input comes in, there is an analysis of whether the
input header and the direction is appropriate. Next check is about the
size of fragment. If it is as defined by the header, process goes
through else the data is buffered and more data is awaited. After this,
type of record as indicated by the header is checked. A call to the
appropriate handler is made. After the handling, State is updated with
the latest information about whatever record came in.
AppLayerResult::ok() is returned in case all went well else
AppLayerResult::err() is returned indicating something went wrong.
 5 years ago
Shivani Bhardwaj 6db1f19d62 rust: Add debug_validate_bug_on macro 
This macro allows to check if certain parts of the code are reachable
during fuzzing.
 5 years ago
Shivani Bhardwaj 57ad609a97 rust: Add new crate uuid 5 years ago
Shivani Bhardwaj ab6b4986ce rust: Add Debug and PartialEq to AppLayerResult 5 years ago
frank honza bbe9137f20 rfb: Update incomplete handling in parser. 
This commit adds an updated incomplete handling for the RFB-Parser. If
incomplete data is processed, the successfully consumed position and
length of remainder + 1 is returned. If the next packet is not empty
suricata will call the parser again.

This commit is a result of discussion on https://github.com/OISF/suricata/pull/4792.
 5 years ago
Philippe Antoine edcb784f1a dns: improve probe_tcp handling of incomplete data 5 years ago
Sascha Steinbiss 713c379427 rfb: make sure size calculations do not overflow 
Addresses #3570 by extra checking of calculated size requests.

With the given input, the parser eventually arrived at
parser::parse_failure_reason() which parsed from the remaining four
bytes (describing the string length) that the failure string to follow
would be 4294967295 bytes long. While calculating the total size of the
data to request via AppLayerResult::incomplete(), adding the four bytes
for the parsed but not consumed string length caused the u32 length to
overflow, resulting in a much smaller value triggering the bug condition.

This problem was addressed by more careful checking of values in each step
that could overflow: one subtraction, one addition (which could overflow
the usize length values), and a final check to determine whether the result
still fit into the u32 values required by AppLayerResult::incomplete().
If so, we would safely convert the values and pass them to the result type.
If not, we simply return AppLayerResult::err() but do not erroneously and
silently request the wrong amount.
 5 years ago
Pierre Chifflier 01aef49cbd rust/x509: map decoding errors to decoder events 5 years ago
Pierre Chifflier 36d2e257c6 rust/x509: use the raw serial number so leading zeros are not removed 5 years ago
Pierre Chifflier d92321d8b1 ssl/tls: use the rust decoder to decode X.509 certificates 5 years ago
Pierre Chifflier 10d9deec9f rust: add common function to exchange CString objects from/to C 5 years ago
Sascha Steinbiss 26123e05f2 rfb: use more idiomatic Rust code 
Using 'if let Some()...' makes the code in these many checks more
concise and readable.
 5 years ago
Frank Honza 1c8943dedd add RFB parser 
This commit adds support for the Remote Framebuffer Protocol (RFB) as
used, for example, by various VNC implementations. It targets the
official versions 3.3, 3.7 and 3.8 of the protocol and provides logging
for the RFB handshake communication for now. Logged events include
endpoint versions, details of the security (i.e. authentication)
exchange as well as metadata about the image transfer parameters.
Detection is enabled using keywords for:

 - rfb.name: Session name as sticky buffer
 - rfb.sectype: Security type, e.g. VNC-style challenge-response
 - rfb.secresult: Result of the security exchange, e.g. OK, FAIL, ...

The latter could be used, for example, to detect brute-force attempts
on open VNC servers, while the name could be used to map unwanted VNC
sessions to the desktop owners or machines.

We also ship example EVE-JSON output and keyword docs as part of the
Sphinx source for Suricata's RTD documentation.
 5 years ago
Victor Julien acef21b759 app-layer: extend AppLayerResult to add convenience 5 years ago
Victor Julien 2f5834cdfa rust: merge parser.rs into applayer.rs 
Both were serving the same purpose.
 5 years ago
Victor Julien 21e6f1f063 app-layer: document return macros 5 years ago
Victor Julien 5b9b0b7226 nfs: switch to new 'incomplete' logic 
Remove buffering code in favor of using incomplete API.
 5 years ago
Victor Julien 66598f9ce7 nfs: switch to AppLayerResult 5 years ago
Victor Julien 5cd9cfb5a0 smb: update return type of GAP handling 5 years ago
Victor Julien 4bf87d30e4 smb: convert to return AppLayerResult 
Support returning 'incomplete' and remove the buffering
code from the parser.
 5 years ago
Victor Julien 674b8dc0fb app-layer: add 'incomplete' return logic 
Allow app-layer parsers to indicate how much data they need
before being called again.
 5 years ago
Victor Julien 44d3f264bf app-layer: update API to return more details 
Add AppLayerResult struct as the Parser return type in
preparation of allowing returning 'Incomplete(size)' similar
to what nom in Rust allows.
 5 years ago
Victor Julien 3bcf948a75 app-layer: change return codes 
This patch simplifies the return codes app-layer parsers use,
in preparation of a patch set for overhauling the return type.

Introduce two macros:

APP_LAYER_OK (value 0)
APP_LAYER_ERROR (value -1)

Update all parsers to use this.
 5 years ago
Victor Julien 6ae66cb2bb nfs: code cleanups 
Use 'if let' to replace simple 'match' statements.

Use explicit returns to easy code review.
 5 years ago
Victor Julien a729d266c3 smb: fix rustc 1.42 warnings 5 years ago
Jason Ish a0e3e2d7b4 dns: register parsers from Rust 
And port the C DNS tests to Rust.
 5 years ago
Jason Ish 0af9a3a5f7 dns: remove C wrapper functions to Rust 
Remove registration of C wrapper functions and register
the Rust functions directly for UDP.
 5 years ago
Jason Ish 1b44f839e6 dns: cleanup: move event callbacks into Rust 
Remove app-layer-dns-common.c as its no longer needed.
 5 years ago
Jason Ish ca5a3f0f04 dns: cleanup: remove unused events 
Removed events that are no longer used since the Rust
implementation of DNS:
- UnsolicitedResponse
- StateMemCapReached
- Flooded
 5 years ago
Jason Ish d809b0959b dns: cleanup: move DnsGetRcode (Lua) to rust 
Move the implementation of Lua DnsGetRcode to Rust.
 5 years ago
Philippe Antoine 23f796a021 kerberos: fix against packet split in record size 5 years ago
Victor Julien 76dd951523 krb5/tcp: remove notice logging on failed records 5 years ago
Sascha Steinbiss 11912bd715 sip: address trailing space parsing 5 years ago
Victor Julien 74305c0486 rust/nfs: minor code cleanups 5 years ago
Victor Julien 576e92983e rust/rpc: add partial data tests 5 years ago
Pierre Chifflier 442500678b rust: use the streaming version of combinators to fix incomplete reads 5 years ago
Pierre Chifflier df9a4fd635 rust/rdp: use the streaming version of combinators to fix incomplete reads 5 years ago
Pierre Chifflier 2561da89c0 rust/rdp: fix regression introduced during nom 5 upgrade 5 years ago
Pierre Chifflier f3ddd7127c rust: Add types annotation when required 
Unfortunately, the transition to nom 5 (and functions instead of macros)
has side-effects, one of them being requiring lots of types annotations
when using a parsing, for ex in a match instruction.
 5 years ago
Pierre Chifflier 62e31396ae rust/ftp: upgrade to nom 5 
CompleteByteSlice type has been removed, and replaced by combinators
under the nom::character::complete namespace.
 5 years ago
Pierre Chifflier 1a505ccd11 rust/rdp: add custom error handling 5 years ago
Pierre Chifflier d1bf34a427 rust/smb: add custom error handling 5 years ago
Pierre Chifflier 1ab8c5763c rust: add SecBlobError custom error type for the Kerberos parser 5 years ago
Pierre Chifflier 030c9a3d86 rust: add take_until_and_consume replacement function 5 years ago
Pierre Chifflier 8664a55ee7 rust/dns: remove unneeded calls to closure! 5 years ago
Pierre Chifflier 5b809f77f1 rust: upgrade all parsers to nom 5 5 years ago
Philippe Antoine 6663246563 parser: make rust probing parsers optional 5 years ago
Victor Julien f68c255f09 nfs: implement post-GAP transaction cleanup 
Close all prior transactions in the direction of the GAP, except the
file xfers. Those use their own logic described below.

After a GAP all normal transactions are closed. File transactions
are left open as they can handle GAPs in principle. However, the
GAP might have contained the closing of a file and therefore it
may remain active until the end of the flow.

This patch introduces a time based heuristic for these transactions.
After the GAP all file transactions are stamped with the current
timestamp. If 60 seconds later a file has seen no update, its marked
as closed.

This is meant to fix resource starvation issues observed in long
running SMB sessions where packet loss was causing GAPs. Due to the
similarity of the NFS and SMB parsers, this issue is fixed for NFS
as well in this patch.

Bug #3424.
Bug #3425.
 5 years ago
Victor Julien d41aeccea4 smb: handle file transactions post-GAP 
After a GAP all normal transactions are closed. File transactions
are left open as they can handle GAPs in principle. However, the
GAP might have contained the closing of a file and therefore it
may remain active until the end of the flow.

This patch introduces a time based heuristic for these transactions.
After the GAP all file transactions are stamped with the current
timestamp. If 60 seconds later a file has seen no update, its marked
as closed.

This is meant to fix resource starvation issues observed in long
running SMB sessions where packet loss was causing GAPs.
 5 years ago
Jason Ish 80cafb2979 flow: expose last time as a function 
This function returns the individual components
of the timeval in output pointers making it suitable
for use over Rust FFI.
 5 years ago
Jason Ish 1e741641a6 rust: make clean fixups 
For make clean, only remove gen/ if cbindgen is available.
This prevents make clean from remove gen when the headers
were bundled, but cbindgen is not available to remove them.

Unconditionally remove gen and vendor in maintainerclean.
 5 years ago
Jason Ish 53aaa82deb rust: set edition to 2018 5 years ago
Jason Ish e68dfa46a6 rust: cargo fix for Rust 2018 edition 5 years ago
Jason Ish 5fbe020585 rust/cbindgen: Revert Makefile to a more pre-cbindgen state 
The modifications as part of the cbindgen commit caused issues
with distcheck, revert the Makefile to how it was with the Python
generator, but still using cbindgen.

Also always assume we'll include the generated headers in the
distribution archive to fix make distcheck from distribution
archives with headers included, but no cbindgen.
 5 years ago
Danny Browning b573c16dd5 build: cbindgen 
Rust headers are now generated using cbindgen. If cbindgen is present, they can
be generated during dist, otherwise they will be available for builds.
 5 years ago
Victor Julien ce0ae81d95 rust: fix vendor use on MinGW 5 years ago
Jason Ish 57b683233d rust: Don't use --frozen during build. 
If sources are vendored, we get the same effect of using frozen
with a lock file, and the Cargo.lock is generated based
on the vendored sources.

This also removes the need to ship a Cargo.lock.

Fixed out of source builds with vendored sources.
 5 years ago
Jason Ish c6f168eb98 rust/Makefile: Don't include Cargo.toml 
There is no need to include Cargo.toml in the distribution,
it is always generated from Cargo.toml.in during
./configure.
 5 years ago
Jason Ish 5ee8323028 rust: remove unnecessary parentheses (Rust 1.40 fixup) 
Rust 1.40 in strict mode will now fail the build on the
presence of unnecessary parentheses.

warning: unnecessary parentheses around type
  --> src/smb/smb2_ioctl.rs:41:12
   |
41 |         -> (&mut SMBTransaction)
   |            ^^^^^^^^^^^^^^^^^^^^^ help: remove these parentheses
   |
   = note: `#[warn(unused_parens)]` on by default
 5 years ago
Victor Julien f5b33a070a smb1: allow empty trans records 5 years ago
Victor Julien 40fe29de96 smb1: fix 'event' txs not getting closed 
If the only reason we created a request side TX was to set an event,
we would not close it.

This patch always looks up the TX from the response side.
 5 years ago
Victor Julien 129cd28058 smb/dcerpc: close request tx sooner 5 years ago
Victor Julien 44ac3e30dd smb: post-GAP handling update 
Close all prior transactions in the direction of the GAP, except the
file xfers. Those use their own logic.
 5 years ago
Victor Julien a7ee2ffbde smb: winreg is a DCERPC facility 5 years ago
Ciprian c9cd7559fd configure: fixing rust/cargo cross compile command 
adding --target argument to cargo command line when cross compiling
 5 years ago
Jason Ish e2c846d01f snmp: add tx detect flags 5 years ago
Jason Ish fa4b9d37c2 krb5: register tx detect flags 
Related ticker #3345:
https://redmine.openinfosecfoundation.org/issues/3345
 5 years ago
Jason Ish 8a232be77e rust: define TxDetectFlag struct and binding macros 
Define a TxDetectFlag type and macros to generating C
bindings for getting and settings the tx detect
flags.
 5 years ago
Jason Ish cde49ec246 rust: add tx detect flags function to registration struct 5 years ago
Victor Julien f9155aa121 files: simplify pruning logic 
Since ebcc4db84a the flow worker runs
file pruning after parsing, detection and loging. This means we can
simplify the pruning logic. If a file is in state >= CLOSED, we can
prune it. Detection and outputs will have had a final chance to
process it.

Remove the calls to the pruning code from Rust. They are no longer
needed.
 5 years ago
Victor Julien b4318a11e3 rust: remove build system HAVE_RUST guards 5 years ago
Jason Ish a3cdef2b4d rust: run tests with same features as build 
Cargo check wasn't being passed --features so could have a different
configuration than the build.
 5 years ago
Jason Ish 389272f4c7 rustup: handle rustup for sudo and su 
If rustup is in use, and a user uses sudo or su for the make
install, the install may fail with a "no default toolchain"
error.

To prevent this, detect at configure if rustup is being used,
then set RUSTUP_HOME for all calls to cargo.
 5 years ago
Jason Ish 6e981fd15a rust: fix build when source directory has spaces in it 
Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2668
 5 years ago
Jason Ish f9c9548b74 configure: detect lua integer size 
Lua 5.1 and 5.3 use a different integer size. Run a test program
to set the integer size used in the Rust FFI layer to Rust.
 5 years ago
Jeff Lucovsky bd691778eb rust/ftp: add parser for active mode port handling 6 years ago
Jason Ish 517ecd68a9 sip: rustfmt 
As this is new Rust code, format with rustfmt using default
styling.
 6 years ago
Jason Ish a45a2fa1fc sip: disable by default in 5.0 6 years ago
Jason Ish fdbc2fe49c sip rust fixup: remove unused import in tests 6 years ago
Giuseppe Longo e06291922f detect/sip.response_line: add sticky buffer 
Matches on response line field in SIP.
 6 years ago
Giuseppe Longo 17de4a8023 detect/sip.request_line: add sticky buffer 
Matches on request line field in SIP.
 6 years ago
Giuseppe Longo 8939ece538 detect/sip.stat_msg: add sticky buffer 
Matches on status msg field in SIP.
 6 years ago
Giuseppe Longo bd2219cac6 detect/sip.stat_code: add sticky buffer 
Matches on status code field in SIP.
 6 years ago
Giuseppe Longo 8454122eb2 detect/sip.protocol: add sticky buffer 
Matches on protocol field in SIP.
 6 years ago
Giuseppe Longo 2661c5b298 detect/sip.uri: add sticky buffer 
Matches on uri field in SIP.
 6 years ago
Giuseppe Longo 424eead8c0 detect/sip.method: add sticky buffer 
Matches on uri field in SIP.
 6 years ago
Giuseppe Longo edc2a583a9 rust/sip: add SIP logger 6 years ago
Giuseppe Longo 2e975a0481 rust/sip: add parser for SIP protocol 6 years ago
Jason Ish d79c23baa3 dns/detect: dns.opcode keyword 
Add a rule keyword, dns.opcode to match on the opcode flag
found in the DNS request and response headers.

Only exact matches are allowed with negation.

Examples:
  - dns.opcode:4;
  - dns.opcode:!1;
 6 years ago
Victor Julien 85ba2e16ba rust/conf: don't print failed conf lookups at info level 6 years ago
Jason Ish 664605b5f1 rdp: disable rdp by default for 5.0 6 years ago
Jason Ish 0f10298990 rdp: address comments in pull request 
Pull request:
https://github.com/OISF/suricata/pull/4174

- fix commit: range -> set
- OUTPUT_BUFFER_SIZE -> JSON_OUTPUT_BUFFER_SIZE
- output: check for initdata first
 6 years ago
Zach Kelly caef8b5b38 protocol parser: rdp 
Initial implementation of feature 2314:
1. Add protocol parser for RDP
2. Add transactions for RDP negotiation
3. Add eve logging of transactions
 6 years ago