Commit Graph

58 Commits (9b275d3878643fa27ac4f54d74ba66b51e115459)

Author SHA1 Message Date
Jason Ish 7b11b4d3a1 app-layer: more generic state trait
Instead of a method that is required to return a slice of transactions,
use 2 methods, one to return the number of transactions in the
collection, and another to get a transaction by its index in the
collection.

This allows for the transaction collection to not be a contiguous array
and instead can be a VecDeque, or possibly another collection type that
supports retrieval by index.

Ticket #5278
3 years ago
Victor Julien e6f49e5a05 app/frames: implement name to id API for frames 4 years ago
Victor Julien c073d5cfbf app-layer: use StreamSlice as input to parsers
Remove input, input_len and flags in favor of stream slice.
4 years ago
Victor Julien 6466296b32 app-layer: add StreamSlice to pass data to parsers
Since object to contain relevant pointer, length, offset, flags to make
it easy to pass these to the parsers.
4 years ago
Jason Ish 7732efbec2 app-layer: include decoder events in app-layer tx data
As most parsers use an events structure we can include it in the
tx_data structure to reduce some boilerplate/housekeeping code
in app-layer parsers.
4 years ago
Jason Ish eb6cc62937 dhcp: fix url in comment
rustdoc was complaining about the format of the URL in a comment
while trying to generate documentation. Convert the comment to a
non-rustdoc comment for now to satisfy rustdoc.
4 years ago
Jason Ish 9c67c634c1 app-layer: include DetectEngineState in AppLayerTxData
Every transaction has an existing mandatory field, tx_data. As
DetectEngineState is also mandatory, include it in tx_data.

This allows us to remove the boilerplate every app-layer has
for managing detect engine state.
4 years ago
Jason Ish ac4c5ada2f dhcp: use generic tx iterator 4 years ago
Pierre Chifflier ebd5883da8 rust/dhcp: convert parser to nom7 functions 4 years ago
Modupe Falodun a87c7e5c08 rust: remove unnecessary nested match
Bug: #4605
4 years ago
Sam Muhammed 922a453da5 rust(lint): use is_null() instead of ptr::null_mut()
Bug: #4594
4 years ago
Jason Ish 9c3f06d9b5 dhcp: use derived app-layer event 4 years ago
Jason Ish 363b5f99c3 rust: functions that reference raw pointers are unsafe
Based on the Rust clippy lint that recommends that any public
function that dereferences a raw pointer, mark all FFI functions
that reference raw pointers with build_slice and cast_pointer
as unsafe.

This commits starts by removing the unsafe wrapper inside
the build_slice and cast_pointer macros then marks all
functions that use these macros as unsafe.

Then fix all not_unsafe_ptr_arg_deref warnings from clippy.

Fixes clippy lint:
https://rust-lang.github.io/rust-clippy/master/index.html#not_unsafe_ptr_arg_deref
4 years ago
Jason Ish 53413f2d7a rust: remove all usage of transmute
All cases of our transmute can be replaced with more idiomatic
solutions and do no require the power of transmute.

When returning an object to C for life-time management, use
Box::into_raw to convert the boxed object to pointer and use
Box::from_raw to convert back.

For cases where we're just returning a pointer to Rust managed
data, use a cast.
4 years ago
Jeff Lucovsky aafb0a60b7 dhcp: Rework constructs to use Self/Default 4 years ago
Victor Julien efc9a7a398 app-layer: remove callback for completion status
Since the completion status was a constant for all parsers, remove the
callback logic and instead register the values themselves. This should
avoid a lot of unnecessary callback calls.

Update all parsers to take advantage of this.
5 years ago
Jason Ish 0529a00ffd dhcp: set unidirection transaction flag 5 years ago
Victor Julien 4da0d9bdea applayer/rust: expose truncate callback 5 years ago
Philippe Antoine 547d6c2d78 applayer: pass parameter to StateAlloc
This parameter is NULL or the pointer to the previous state
for the previous protocol in the case of a protocol change,
for instance from HTTP1 to HTTP2

This way, the new protocol can use the old protocol context.
For instance, HTTP2 mimicks the HTTP1 request, to have a HTTP2
transaction with both request and response
5 years ago
Jason Ish ac93ab281d rust/logging: allow log macros to be used by plugins
Fix plugin macros so they can be used by external Rust crates
such as plugins.
5 years ago
Jason Ish 53aa967e0b applayer: add flags to parser registration struct
This will allow Rust parsers to register for gap handing from
Rust (some Rust parsers do handle gaps, but they set the flag
from C).
5 years ago
Victor Julien 5dd4d948d9 app-layer: remove unused detect flags API 5 years ago
Victor Julien c94a5e6392 app-layer/rust: don't use option for GetTxDataFn anymore 5 years ago
Victor Julien 9664f73f75 app-layer: remove logged API calls 5 years ago
Victor Julien a484bbbe1b dhcp: support AppLayerTxData 5 years ago
Victor Julien 5665fc8301 app-layer: add ApplyTxConfig API
Optional callback a parser can register for applying configuration
to the 'transaction'. Most parsers have a bidirectional tx. For those
parsers that have different types of transaction handling, this new
callback can be used to properly apply the config.
5 years ago
Victor Julien 411f428a38 app-layer: define AppLayerTxData and AppLayerTxConfig
AppLayerTxData is a structure each tx should include that will contain
the common fields the engine needs for tracking logging, detection and
possibly other things.

AppLayerTxConfig will be used by the detection engine to configure
the transaction.
5 years ago
Victor Julien 79681bf655 app-layer: remove old MPMId API calls
Had been deprecated and non-functional since 2017.
5 years ago
Jason Ish deed0541bb dhcp/eve: convert to jsonbuilder 5 years ago
Victor Julien 2f5834cdfa rust: merge parser.rs into applayer.rs
Both were serving the same purpose.
5 years ago
Victor Julien 44d3f264bf app-layer: update API to return more details
Add AppLayerResult struct as the Parser return type in
preparation of allowing returning 'Incomplete(size)' similar
to what nom in Rust allows.
5 years ago
Victor Julien 3bcf948a75 app-layer: change return codes
This patch simplifies the return codes app-layer parsers use,
in preparation of a patch set for overhauling the return type.

Introduce two macros:

APP_LAYER_OK (value 0)
APP_LAYER_ERROR (value -1)

Update all parsers to use this.
5 years ago
Pierre Chifflier 442500678b rust: use the streaming version of combinators to fix incomplete reads 5 years ago
Pierre Chifflier 5b809f77f1 rust: upgrade all parsers to nom 5 5 years ago
Philippe Antoine 6663246563 parser: make rust probing parsers optional 5 years ago
Jason Ish cde49ec246 rust: add tx detect flags function to registration struct 6 years ago
Jason Ish 42e5065ab8 rust: update to Rust 2018 with cargo fix
Migrate to Rust 2018 edition.

Credit to Danny Browning for first demontrating this:
https://github.com/OISF/suricata/pull/3604/commits
6 years ago
Shivani Bhardwaj bbfd706e1f rust: fix compiler warning
rustc 1.36 introduced:

error: variable does not need to be mutable
   --> src/dhcp/parser.rs:202:17
    |
202 |             let mut malformed_options = false;
    |                 ----^^^^^^^^^^^^^^^^^
    |                 |
    |                 help: remove this `mut`
    |
note: lint level defined here
   --> src/lib.rs:18:38
    |
18  | #![cfg_attr(feature = "strict", deny(warnings))]
    |                                      ^^^^^^^^
    = note: #[deny(unused_mut)] implied by #[deny(warnings)]

error: aborting due to previous error

error: Could not compile `suricata`.

Ticket #3072.
6 years ago
Jeff Lucovsky e3ca6b43fc rust/dhcp: Support get-info-by-id 6 years ago
Victor Julien 3f6624bf16 rust: remove libc crate dependency
Use std::os::raw instead.
6 years ago
Jeff Lucovsky d568e7fadd eve/logging: 2991 Optimize logging by TX
This changeset makes changes to the TX logging path. Since the txn
is passed to the TX logger, the TX can be used directly instead of
through the TX id.
6 years ago
Victor Julien b1d4931842 rust: fix warnings about wrong type of comments
"rustdoc does not generate documentation for macro expansions"
6 years ago
Victor Julien bf1bd407dd rust: fix libc deprecation warnings for int types 6 years ago
Jason Ish 8be4142aaf dhcp: verify client id len before parsing data
Verify that the client id length is at least 2 per the DHCP
protocol rfc before parsing the data.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2902
6 years ago
Victor Julien 0301ceab13 rust/mingw: fix missing IPPROTO_* declarations
The libc crate doesn't provide these on MinGW, so define them in
our 'core' instead. We only use IPPROTO_TCP and IPPROTO_UDP.

Bug #2733
6 years ago
Victor Julien 422e4892cc proto-detect: improve midstream support
When Suricata picks up a flow it assumes the first packet is
toserver. In a perfect world without packet loss and where all
sessions neatly start after Suricata itself started, this would be
true. However, in reality we have to account for packet loss and
Suricata starting to get packets for flows already active be for
Suricata is (re)started.

The protocol records on the wire would often be able to tell us more
though. For example in SMB1 and SMB2 records there is a flag that
indicates whether the record is a request or a response. This patch
is enabling the procotol detection engine to utilize this information
to 'reverse' the flow.

There are three ways in which this is supported in this patch:

1. patterns for detection are registered per direction. If the proto
   was not recognized in the traffic direction, and midstream is
   enabled, the pattern set for the opposing direction is also
   evaluated. If that matches, the flow is considered to be in the
   wrong direction and is reversed.

2. probing parsers now have a way to feed back their understanding
   of the flow direction. They are now passed the direction as
   Suricata sees the traffic when calling the probing parsers. The
   parser can then see if its own observation matches that, and
   pass back it's own view to the caller.

3. a new pattern + probing parser set up: probing parsers can now
   be registered with a pattern, so that when the pattern matches
   the probing parser is called as well. The probing parser can
   then provide the protocol detection engine with the direction
   of the traffic.

The process of reversing takes a multi step approach as well:

a. reverse the current packets direction
b. reverse most of the flows direction sensitive flags
c. tag the flow as 'reversed'. This is because the 5 tuple is
   *not* reversed, since it is immutable after the flows creation.

Most of the currently registered parsers benefit already:

- HTTP/SMTP/FTP/TLS patterns are registered per direction already
  so they will benefit from the pattern midstream logic in (1)
  above.

- the Rust based SMB parser uses a mix of pattern + probing parser
  as described in (3) above.

- the NFS detection is purely done by probing parser and is updated
  to consider the direction in that parser.

Other protocols, such as DNS, are still to do.

Ticket: #2572
6 years ago
Pierre Chifflier f22695130b rust: nom4 requires to add complete!() when using many! combinators 7 years ago
Pierre Chifflier 8c0cde36c6 rust: fix warnings for unused variables (add _) 7 years ago
Pierre Chifflier 13b7399790 rust: upgrade all parsers to nom4 7 years ago
Victor Julien 0e40231189 app-layer: improve transaction cleanup handling
The app layers with a custom iterator would skip a tx if during
the ..Cleanup() pass a transaction was removed.

Address this by storing the current index instead of the next
index. Also pass in the next "min_tx_id" to be incremented from
the last TX. Update loops to do this increment.

Also make sure that the min_id is properly updated if the last
TX is removed when out of order.

Finally add a SMB unittest to test this.

Reported by: Ilya Bakhtin
7 years ago