Commit Graph

856 Commits (334b1382e0c647dfad71e9320350e87dbba90e1f)

Author SHA1 Message Date
Shivani Bhardwaj 13741540ce rfb: remove futile default port setting 4 years ago
Shivani Bhardwaj 7c9d573800 nfs: remove futile default port setting 4 years ago
Shivani Bhardwaj f4f6387a00 dcerpc: use null for default ports 4 years ago
Philippe Antoine 596a4a9d6e http2: better rust style 4 years ago
Philippe Antoine 48ed874dda http2: concatenate one headers multiple values
For detection, as is done with HTTP1
4 years ago
Philippe Antoine e3ff0e7731 http2: generic http2_header_blocks
so as not to forget continuation and push promise
when iterating over headers
4 years ago
Philippe Antoine 0b0649d98e http2: http.header keyword now works for HTTP2
As well as http.header.raw
4 years ago
Philippe Antoine 9b9f909d7d http2: http.header_names keyword now works for HTTP2 4 years ago
Philippe Antoine 547e9f4ab4 http2: http.host normalized keyword now works for HTTP2 4 years ago
Philippe Antoine 75f75e1eb0 http2: turn Host header into authority during upgrade
HTTP1 uses Host, but HTTP2 uses rather :authority cf HPACK
4 years ago
Philippe Antoine bb98a18b3d http2: better file tracking
If an HTTP2 file was within only ont DATA frame, the filetracker
would open it and close it in the same call, preventing the
firther call to incr_files_opened

Also includes rustfmt again for all HTTP2 files
4 years ago
Philippe Antoine 1378b2f451 http2: support deflate decompression
cf #4556
4 years ago
Victor Julien c9cee7af49 smb: add debug validation on file counts 4 years ago
Victor Julien 114d3ba730 smb: count files in tx 4 years ago
Victor Julien c1dfb619c4 http2: support per-tx file accounting 4 years ago
Victor Julien 1b3c3225cd nfs: add debug validation on file counts 4 years ago
Victor Julien 1d48601c25 nfs: support per-tx file accounting 4 years ago
Victor Julien 67759795c6 nfs: don't reuse file transactions
After a file has been closed (CLOSE, COMMIT command or EOF/SYNC part of
READ/WRITE data block) mark it as such so that new file commands on that
file do not reuse the transaction.

When a file transfer is completed it will be flagged as such and not be
found anymore by the NFSState::get_file_tx_by_handle() method. This forces
a new transaction to be created.
4 years ago
Victor Julien 56d3e28a3a filestore: track files getting stored per tx
Avoid evicting a tx before the filedata logger has decided it is
done.
4 years ago
Victor Julien c78f5ac316 app-layer/transactions: track files opens and logs
To make sure a transaction is not evicted before all file logging is complete.
4 years ago
Philippe Antoine 9b8be5a650 smb: get file name in case of chained commands 4 years ago
Philippe Antoine 3e5f59e2cb smb: fix parsing of file deletion over SMB1 4 years ago
Philippe Antoine fde753d9d2 smb: recognizes file deletion over SMB2
using set_info_level == SMB2_FILE_DISPOSITION_INFO
4 years ago
Jason Ish 71679c6ad0 ike: use derive macro from app-layer events 4 years ago
Jason Ish eb55297876 modbus: use derive macro from app-layer events 4 years ago
Jason Ish d3bd008e33 app-layer template: use derived app-layer event 4 years ago
Jason Ish cef2832dcf http2: use derived app-layer event 4 years ago
Jason Ish e92cb36bb8 krb5: use derived app-layer event 4 years ago
Jason Ish 92561837f8 ntp: use derived app-layer event 4 years ago
Jason Ish 1f71fb2cde rfb: register None for get_event_info/get_event_info_by_id
Implementations are not required if they're just going to return
-1. We allow None to be registered for that.
4 years ago
Jason Ish 4fd6aa866f sip: use derived app-layer event 4 years ago
Jason Ish 18448f6ed6 snmp: use derived app-layer event 4 years ago
Jason Ish bb094b17db ssh: use derived app-layer event 4 years ago
Jason Ish 9c3f06d9b5 dhcp: use derived app-layer event 4 years ago
Jason Ish b9f10ba22f smb: use derived get_event_info/get_event_info_by_id 4 years ago
Jason Ish 8eac5fc221 mqtt: derive AppLayerEvent for MQTTEvent 4 years ago
Jason Ish 6ed827a4ef dns: use derive macro for DNSEvent 4 years ago
Jason Ish 9221f1d9d5 applayerevent: derive get_event_info and get_event_info_by_id
Add generation of wrapper functions for get_event_info
and get_event_info_by_id to the derive macro. Eliminates
the need for the wrapper method to be created by the parser
author.
4 years ago
Jason Ish 0fa7b5c2a2 rust/applayer: provide generic event info functions
Provide generic functions for get_event_info and
get_event_info_by_id. These functions can be used by any app-layer
event enum that implements AppLayerEvent.

Unfortunately the parser registration cannot use these functions
directly as generic functions cannot be #[no_mangle]. So they
do need small extern "C" wrappers around them.
4 years ago
Jason Ish 27d1ee98ce rust: derive crate: for custom derives
Currently has one derive, AppLayerEvent to be used like:

  #[derive(AppLayerEvent)]
  pub enum DNSEvent {
      MalformedData,
      NotRequest,
      NotResponse,
      ZFlagSet,
  }

Code will be generated to:
- Convert enum to a c type string
- Convert string to enum variant
- Convert id to enum variant
4 years ago
Jason Ish dbea7d636f rust/applayer: define AppLayerEvent trait
The derive macro will implement this trait for app-layer
event enums.
4 years ago
Jason Ish cf21694ba6 rust(lint): suppress clippy lints that we should fix
Suppress all remaining clippy lints that we trip. This can be
fixed on a per-lint basis.
4 years ago
Jason Ish 91402f9fba rust(lint): remove manual implement of map method
Using `if let` expressions in these cases is better expressed
by the map method, and considered idiomatic Rust for this usage.
4 years ago
Jason Ish b021726a0d rust(lint): map the error instead of using or_else
This is the preffered style and easier to understand the meaning
of the code.
4 years ago
Jason Ish dcf57ecd96 rust(lint): replace push_str of single char to push(<char>) 4 years ago
Jason Ish d5c0962299 rust(lint): fix some usages of references
- ref is discouraged for top level variables
- the other borrow is not required
4 years ago
Jason Ish d0772e04b1 rust(lint): replace checked_mul with saturating_mul
When defaulting checked_mul to u64::max, Rust has a method
that does the same thing called saturating_mul.
4 years ago
Jason Ish d0be7541e9 rust(lint): removed unused unit () return
This is code that is not needed and is a bit confusing to see.
4 years ago
Jason Ish 4abbfd0d97 rust(lint): remove extra parens around bitwise or
This is a readability fix, as on first look they almost look
like a Rust tuple.
4 years ago
Jason Ish ac3a20b6e0 rust(lint): remove useless conversions and clones
These add complexity and may not be optimized out by the compiler.
4 years ago
Jason Ish 8bb6dab69d rust(lint): remove useless format calls
In these simple cases to_string() is recommended and likely
performs better as the formatter is not called.
4 years ago
Jason Ish 5bf5de3350 rust(lint): don't use unwrap_or for function calls
Calling a function in unwrap_or causes that function to always
be called even when not needed. Instead use unwrap_or_else with
a closure which will only be called when needed.
4 years ago
Jason Ish 602bb05e75 rust(lint): fix redundant closures
This lint checks for a closure where a function can be directly
supplied.  Runtime performance is unchanged, but this makes
less work for the compiler.
4 years ago
Jason Ish 69cf5c9eea rust(lint): remove needless borrows
These are needless borrows (references) as the item is already
a reference.
4 years ago
Jason Ish 363b5f99c3 rust: functions that reference raw pointers are unsafe
Based on the Rust clippy lint that recommends that any public
function that dereferences a raw pointer, mark all FFI functions
that reference raw pointers with build_slice and cast_pointer
as unsafe.

This commits starts by removing the unsafe wrapper inside
the build_slice and cast_pointer macros then marks all
functions that use these macros as unsafe.

Then fix all not_unsafe_ptr_arg_deref warnings from clippy.

Fixes clippy lint:
https://rust-lang.github.io/rust-clippy/master/index.html#not_unsafe_ptr_arg_deref
4 years ago
Jason Ish 53413f2d7a rust: remove all usage of transmute
All cases of our transmute can be replaced with more idiomatic
solutions and do no require the power of transmute.

When returning an object to C for life-time management, use
Box::into_raw to convert the boxed object to pointer and use
Box::from_raw to convert back.

For cases where we're just returning a pointer to Rust managed
data, use a cast.
4 years ago
Victor Julien 9d24a53c53 nfs: minor code cleanup 4 years ago
Victor Julien aa9d8658ef smb: minor formatting fixup 4 years ago
Victor Julien 094208823b smb: minor code cleanup 4 years ago
Shivani Bhardwaj 0a1747c1ba nfs: fix comment 4 years ago
Shivani Bhardwaj 58ac9b0f38 nfs: Add rust registration function
Get rid of the C glue code and move registration completely to Rust.
4 years ago
Shivani Bhardwaj 61fca4e9db nfs: add missing code from rust impl of fns 4 years ago
Shivani Bhardwaj de50ac631e nfs: Change fn sign as per rust registration requirement
Registering parsers in Rust requires signatures to be a certain way and
compatible with C. Change signatures of all the functions.
Probe fn has also been changed to return AppProto as required by the new
fn signature.
4 years ago
Shivani Bhardwaj e5c948df87 smb: Add rust registration function
Get rid of the C glue code and move registration completely to Rust.
4 years ago
Shivani Bhardwaj 27af4bb002 smb: add missing code from rust impl of fns 4 years ago
Shivani Bhardwaj 6420df84b7 smb: Change fn sign as per rust registration requirement
Registering parsers in Rust requires signatures to be a certain way and
compatible with C. Change signatures of all the functions.
4 years ago
Shivani Bhardwaj 4d6b6b5dfe smb: add constants 4 years ago
Shivani Bhardwaj d1ea00521b rust/core: Add flow flags 4 years ago
Jason Ish 222e55847c flow: provide flags accessor function
Add an accessor function for flow flags. To be used by Rust where
the flow struct is an opaque data type.
4 years ago
Shivani Bhardwaj cb8bd8c669 rust/applayer: add more externs 4 years ago
Philippe Antoine 31dccd1171 modbus: do not claim to handle gaps 4 years ago
Sascha Steinbiss d541b3d4a8 rust: fix warnings with nightly 4 years ago
Philippe Antoine 9e7ea631b2 dns: improve probing parser
Checks opcode is valid
Checks additional_rr do not exceed message length
Better logic for incomplete cases
4 years ago
Philippe Antoine 6f03ee2e47 dcerpc: handles bigger inputs than 2^16
By comparing integers with the largest size
4 years ago
Philippe Antoine 7d0a39412b detect: use u32 for InspectionBufferMultipleForList
So that we do not have an endless loop casting index to
u16 and having more than 65536 buffers in one transaction

Changes for all protocols, even ones where it is impossible
to have such a pattern, so as to avoid bad pattern copy/paste
in the future
4 years ago
Philippe Antoine b3c1f2ab48 nfs: improve probing parser
Checks credentials flavor is known
4 years ago
Philippe Antoine 39575e2cc9 modbus: use ascii character classes while parsin rule
As the rust regex crate is unicode aware, which was
not the case of the C version
4 years ago
Philippe Antoine ef5755338f rust: SCLogDebug is real nop when built as release
Before, even if there were no outputs, all the arguments
were evaluated, which could turn expensive

All variables which are used only in certain build configurations
are now prefixed by underscore to avoid warnings
4 years ago
Victor Julien 20e8f90981 http2: set Debug on structs 4 years ago
Victor Julien 3587033d9e files: construct with default, free on drop
Update protocols.
4 years ago
Victor Julien d757545f03 files: implement default support 4 years ago
Philippe Antoine fdab22d924 rust: fix app-layer parser flags
This especially allows for SSH bypass to work
4 years ago
Jeff Lucovsky aa8871a5be rust/default: Enable Default usage 4 years ago
Jeff Lucovsky 6028ca7827 nfs: Rework constructs to use Self/Default 4 years ago
Jeff Lucovsky aafb0a60b7 dhcp: Rework constructs to use Self/Default 4 years ago
Jeff Lucovsky 1ef0bd580b dcerpc: Rework constructs to use Self/Default 4 years ago
Jeff Lucovsky 00167121dc dns: Rework constructs to use Self/Default 4 years ago
Jeff Lucovsky 02dccb1529 smb: Rework constructs to use Self/Default
This commit modifies the constructors to use Self and/or
Default::default() when feasible
4 years ago
Jeff Lucovsky f502f21f9e rust/default: Enable Default usage in SMB 4 years ago
Shivani Bhardwaj 581cb6223d dcerpc/udp: Add rust registration function
Get rid of the C glue code and move registration completely to Rust.
4 years ago
Shivani Bhardwaj d7007424bd dcerpc/udp: Change fn sign as per rust registration requirement
Registering parsers in Rust requires signatures to be a certain way and
compatible with C. Change signatures of all the functions.
4 years ago
Shivani Bhardwaj bac69af7e4 dcerpc: Add rust registration function
Get rid of the C glue code and move registration completely to Rust.
4 years ago
Shivani Bhardwaj a0a09a102b dcerpc: Change fn sign as per rust registration requirement
Registering parsers in Rust requires signatures to be a certain way and
compatible with C. Change signatures of all the functions.
4 years ago
Shivani Bhardwaj dee972b863 rust/core: Make AppProto type u16 4 years ago
Shivani Bhardwaj d66ad96f0d applayer/rust: add extern AppLayerProtoDetectPMRegisterPatternCSwPP 4 years ago
Victor Julien d7c3ecb6f9 http2: remove dead code 4 years ago
Philippe Antoine b3c502d572 http2: remove assertion which can be wrong
Brotli decoder stops consuming input it it reaches the
end of its input
4 years ago
Simon Dugas c2720fc2fb modbus: fix quantity and count calculation
The [Modbus Spec S6.11](https://modbus.org/docs/Modbus_Application_Protocol_V1_1b.pdf)
clearly states that the `count = quantity / 8` and not the other way
around. This is fixed in sawp-0.5.0.
4 years ago
Philippe Antoine 999327ba1f http2: http.cookie keyword now works for HTTP2 4 years ago
Philippe Antoine df039555bc http2: http.host.raw keyword now works for HTTP2 4 years ago
Philippe Antoine 1e82d0b3c8 http2: http.method keyword now works for HTTP2 4 years ago
Philippe Antoine 017e39d8fd http2: makes all HTTP1 header keywords work 4 years ago
Simon Dugas a8a51dc004 modbus: add eve logging 4 years ago
Simon Dugas 8342641477 modbus: move tests from c to rust
Move tests in a seperate commit so that we can use the previous one for
regression testing. This also gets rid of the temporary glue that made
the C tests work with the rust implementation.
4 years ago
Simon Dugas a458a94dca modbus: move from C to rust
Adds a new rust modbus app layer parser and detection module.

Moves the C module to rust but leaves the test cases in place to
regression test the new rust module.
4 years ago
frank honza f83d51d0cb ike: set event for multiple server proposals 4 years ago
Philippe Antoine ca6e434e0b ftp: completely parses pasv and epsv responses 4 years ago
Philippe Antoine 6fe8bce3b0 http2: have filecontainer for both directions 4 years ago
Jason Ish c7f44447c9 dns: remove flood protection purging
It doesn't look like flood protection is required with the
stateless parser anymore. It actually can get in the way of TCP
DNS when a large number of requests end-up in the same segment
where a TX can get purged before it has a chance to go through
the normal TX life-cycle.
4 years ago
Shivani Bhardwaj c663ac6ddd dcerpc/tcp: improve detection
Lately, some of the TLS data was misdetected as DCERPC/TCP because of
the pattern |05 00|. Add more checks in DCERPC probe function to ensure
that it is in fact DCERPC/TCP.
4 years ago
Philippe Antoine d2d0e0adc9 rust: remove exported unused functions 4 years ago
Philippe Antoine 6da9a37285 rdp: correctly returns incomplete in parse_tc
Adding the already consumed bytes
In case an incomplete tls handshake is handled with/after
a refular rdp t123_tpkt
4 years ago
Philippe Antoine 3de0123ffb http2: adds check about dynamic headers table size 4 years ago
Philippe Antoine 1ca4f041bb http2: pass data through when decompression fails
as is done for HTTP1
4 years ago
Juliana Fajardini c6a35d09b7 templates: fix typos
- *template*files[ch][rs]: fix typos
- scripts/setup-app-layer: fix typos
4 years ago
Jason Ish 6853bf98fb dns: only register a single logger
DNS no longer requires a logger to be registered for to-client and
to-server directions. This has not been required with the stateless
design of the Rust DNS parser.
4 years ago
Philippe Antoine 0105d4f017 rust: bump bitflags dependency version
So that lexical-core, needed by nom, and using bitflags
is used with version 0.7.5 instead of version 0.7.0
which fixed the fact that BITS is now a reserved keyword
in nightly version
4 years ago
Philippe Antoine cb150e97d0 kerberos: fix probing parser tag condition
according to the comment
4 years ago
Sascha Steinbiss e2dbdd7fd5 ikev1: add ikev1 parser 4 years ago
frank honza ecdf9f6b0b ikev1: rename ikev2 to common ike
Renaming was done with shell commands, git mv for moving the files and content like
find -iname '*.c' | xargs sed -i 's/ikev1/ike/g' respecting the different mixes of upper/lower case.
4 years ago
frank honza ab6171c429 detect: added support for protocol-aliases 4 years ago
frank honza e9494ddd8f util: add function converting u8-array into a hex-String 4 years ago
Philippe Antoine 8307010255 smb: relax probing parser to handle first NBSS message
cf dcerpc-udp S-V test :
First message is Message Type: Session request (0x81)
Second message is SMB
4 years ago
Philippe Antoine 1b6e81cd72 smb: probing parser for start and midstream
The probing parser is more strict at the start of the stream
4 years ago
Philippe Antoine 9dc5258a21 smb: split probing function for code style
Introduces rs_smb_probe_tcp_midstream
4 years ago
Shivani Bhardwaj 0ac5c5376a dcerpc: trigger raw assembly on record completion 4 years ago
Shivani Bhardwaj c77c8e7005 rust/context: add AppLayerParserTriggerRawStreamReassembly 4 years ago
Philippe Antoine 2d14606224 smb: andx support
Add AndX support for SMB1. Finishes #3475.

[Updated by Victor Julien to split functions]
4 years ago
Shivani Bhardwaj f967a49104 dcerpc/udp: improve detection
Lately, Wireguard proto starting w pattern |04 00| is misdetected as
DCERPC/UDP which also starts with the same pattern, add more checks
to make sure that it is the best guess for packet to be dcerpc/udp.
4 years ago
Shivani Bhardwaj 3641f1b522 dcerpc: add probe function 4 years ago
Shivani Bhardwaj d7a3523b12 rust/applayer: split EOF flag per direction 4 years ago
Shivani Bhardwaj 0ca8591994 dcerpc/udp: remove transmute
The book defines transmute as "This is really, truly, the most horribly unsafe
thing you can do in Rust. The guardrails here are dental floss."
Transmute can result into mind boggling undefined behaviors. Get rid of
it wherever possible.
4 years ago
Philippe Antoine c6aadf0dfa protodetect: rename direction to flags
And use whole flags in AppLayerProtoDetectPPGetProto
4 years ago
Philippe Antoine 5465e0b154 http2: http.stat_msg keyword now works for HTTP2 4 years ago
Philippe Antoine 47928babfc http2: http.user_agent keyword now works for HTTP2 4 years ago
Philippe Antoine a98d0fe6ed http2: http.uri keyword now works for HTTP2
cf #4067
4 years ago
Philippe Antoine 707f027231 protos: renaming ALPROTO_HTTP* constants
Having now ALPROTO_HTTP1, ALPROTO_HTTP2 and ALPROTO_HTTP

Run with 3 sed commands
git grep ALPROTO_HTTP | cut -d: -f1 | uniq |
 xargs sed -i -e 's/ALPROTO_HTTP/ALPROTO_HTTP1/g'
git grep ALPROTO_HTTP12 | cut -d: -f1 | uniq |
 xargs sed -i -e 's/ALPROTO_HTTP12/ALPROTO_HTTP2/g'
git grep ALPROTO_HTTP1_ANY | cut -d: -f1 | uniq |
 xargs sed -i -e 's/ALPROTO_HTTP1_ANY/ALPROTO_HTTP/g'

and then running clang-format
4 years ago
Jason Ish dbae17dbc0 install: makefile target to install libraries
As we don't install the libraries by default, provide a make target,
"install-library" to install the libsuricata library files.

If shared library support exists, both the static and shared
libraries will be installed, otherwise only the static libraries
will be installed.
4 years ago
Jason Ish a178ec6bef rust: rename lib to libsuricata_rust
Previously it was libsuricata.a, but eventually we want to get
to a place where libsuricata.a is a combination of the Rust
and C code.
4 years ago
Jason Ish 9f20297cb3 rust/Makefile: add Cargo.toml as make dependency
This will force Cargo.toml to be recreated if Cargo.toml.in
is modified.
4 years ago
Jason Ish 3ada5e1480 rust/ffi: provide AppLayerRegisterParser in context
AppLayerRegisterParser was creating a link error when attempting
to use a convenience library for the Suricata C code, then linking
the library of C code with the library of Rust code into a final
Suricata executable, or use with fuzz targets.

By moving AppLayerRegisterParser to the context structure and
calling it like a callback the circular reference is removed
allowing the convenience libraries to work again.

This is also a stepping block to proving a Suricata library
as a single .a or .so file.
4 years ago
Philippe Antoine 32b604e8c7 template: use response_gap in rust parser 4 years ago
Victor Julien 3f807f3bf6 rust: update dependencies 4 years ago
Victor Julien ebde15f0e2 rust: lock all major crate versions
To avoid surprises with dependencies bumping MSRV.
4 years ago
Victor Julien 4b5af36061 rust: relax nom version to any >=5.1.1 4 years ago
Philippe Antoine d861228214 http2: decompression for files
gzip and brotli decompression for files
4 years ago
Philippe Antoine 2e46b5d100 rust: BIT_U16 macro utility 4 years ago
Philippe Antoine aee8e60149 rust: better panic message for missing file config 4 years ago
Philippe Antoine 8ac363c34d rust: fix warning about unused values in smb tests 4 years ago
Jason Ish 0a3b9e0220 rust/hashing: add function to finalize md5 to hex string
New function, SCMd5FinalizeToHex to finalize an md5 hash
to a hex string.
4 years ago