Commit Graph

427 Commits (e9a3415fc2ad0ad73b574f0c84c2325a1145c993)

Author SHA1 Message Date
Pierre Chifflier 2561da89c0 rust/rdp: fix regression introduced during nom 5 upgrade 6 years ago
Pierre Chifflier f3ddd7127c rust: Add types annotation when required
Unfortunately, the transition to nom 5 (and functions instead of macros)
has side-effects, one of them being requiring lots of types annotations
when using a parsing, for ex in a match instruction.
6 years ago
Pierre Chifflier 62e31396ae rust/ftp: upgrade to nom 5
CompleteByteSlice type has been removed, and replaced by combinators
under the nom::character::complete namespace.
6 years ago
Pierre Chifflier 1a505ccd11 rust/rdp: add custom error handling 6 years ago
Pierre Chifflier d1bf34a427 rust/smb: add custom error handling 6 years ago
Pierre Chifflier 1ab8c5763c rust: add SecBlobError custom error type for the Kerberos parser 6 years ago
Pierre Chifflier 030c9a3d86 rust: add take_until_and_consume replacement function 6 years ago
Pierre Chifflier 8664a55ee7 rust/dns: remove unneeded calls to closure! 6 years ago
Pierre Chifflier 5b809f77f1 rust: upgrade all parsers to nom 5 6 years ago
Philippe Antoine 6663246563 parser: make rust probing parsers optional 6 years ago
Victor Julien f68c255f09 nfs: implement post-GAP transaction cleanup
Close all prior transactions in the direction of the GAP, except the
file xfers. Those use their own logic described below.

After a GAP all normal transactions are closed. File transactions
are left open as they can handle GAPs in principle. However, the
GAP might have contained the closing of a file and therefore it
may remain active until the end of the flow.

This patch introduces a time based heuristic for these transactions.
After the GAP all file transactions are stamped with the current
timestamp. If 60 seconds later a file has seen no update, its marked
as closed.

This is meant to fix resource starvation issues observed in long
running SMB sessions where packet loss was causing GAPs. Due to the
similarity of the NFS and SMB parsers, this issue is fixed for NFS
as well in this patch.

Bug #3424.
Bug #3425.
6 years ago
Victor Julien d41aeccea4 smb: handle file transactions post-GAP
After a GAP all normal transactions are closed. File transactions
are left open as they can handle GAPs in principle. However, the
GAP might have contained the closing of a file and therefore it
may remain active until the end of the flow.

This patch introduces a time based heuristic for these transactions.
After the GAP all file transactions are stamped with the current
timestamp. If 60 seconds later a file has seen no update, its marked
as closed.

This is meant to fix resource starvation issues observed in long
running SMB sessions where packet loss was causing GAPs.
6 years ago
Jason Ish 80cafb2979 flow: expose last time as a function
This function returns the individual components
of the timeval in output pointers making it suitable
for use over Rust FFI.
6 years ago
Jason Ish 1e741641a6 rust: make clean fixups
For make clean, only remove gen/ if cbindgen is available.
This prevents make clean from remove gen when the headers
were bundled, but cbindgen is not available to remove them.

Unconditionally remove gen and vendor in maintainerclean.
6 years ago
Jason Ish 53aaa82deb rust: set edition to 2018 6 years ago
Jason Ish e68dfa46a6 rust: cargo fix for Rust 2018 edition 6 years ago
Jason Ish 5fbe020585 rust/cbindgen: Revert Makefile to a more pre-cbindgen state
The modifications as part of the cbindgen commit caused issues
with distcheck, revert the Makefile to how it was with the Python
generator, but still using cbindgen.

Also always assume we'll include the generated headers in the
distribution archive to fix make distcheck from distribution
archives with headers included, but no cbindgen.
6 years ago
Danny Browning b573c16dd5 build: cbindgen
Rust headers are now generated using cbindgen. If cbindgen is present, they can
be generated during dist, otherwise they will be available for builds.
6 years ago
Victor Julien ce0ae81d95 rust: fix vendor use on MinGW 6 years ago
Jason Ish 57b683233d rust: Don't use --frozen during build.
If sources are vendored, we get the same effect of using frozen
with a lock file, and the Cargo.lock is generated based
on the vendored sources.

This also removes the need to ship a Cargo.lock.

Fixed out of source builds with vendored sources.
6 years ago
Jason Ish c6f168eb98 rust/Makefile: Don't include Cargo.toml
There is no need to include Cargo.toml in the distribution,
it is always generated from Cargo.toml.in during
./configure.
6 years ago
Jason Ish 5ee8323028 rust: remove unnecessary parentheses (Rust 1.40 fixup)
Rust 1.40 in strict mode will now fail the build on the
presence of unnecessary parentheses.

warning: unnecessary parentheses around type
  --> src/smb/smb2_ioctl.rs:41:12
   |
41 |         -> (&mut SMBTransaction)
   |            ^^^^^^^^^^^^^^^^^^^^^ help: remove these parentheses
   |
   = note: `#[warn(unused_parens)]` on by default
6 years ago
Victor Julien f5b33a070a smb1: allow empty trans records 6 years ago
Victor Julien 40fe29de96 smb1: fix 'event' txs not getting closed
If the only reason we created a request side TX was to set an event,
we would not close it.

This patch always looks up the TX from the response side.
6 years ago
Victor Julien 129cd28058 smb/dcerpc: close request tx sooner 6 years ago
Victor Julien 44ac3e30dd smb: post-GAP handling update
Close all prior transactions in the direction of the GAP, except the
file xfers. Those use their own logic.
6 years ago
Victor Julien a7ee2ffbde smb: winreg is a DCERPC facility 6 years ago
Ciprian c9cd7559fd configure: fixing rust/cargo cross compile command
adding --target argument to cargo command line when cross compiling
6 years ago
Jason Ish e2c846d01f snmp: add tx detect flags 6 years ago
Jason Ish fa4b9d37c2 krb5: register tx detect flags
Related ticker #3345:
https://redmine.openinfosecfoundation.org/issues/3345
6 years ago
Jason Ish 8a232be77e rust: define TxDetectFlag struct and binding macros
Define a TxDetectFlag type and macros to generating C
bindings for getting and settings the tx detect
flags.
6 years ago
Jason Ish cde49ec246 rust: add tx detect flags function to registration struct 6 years ago
Victor Julien f9155aa121 files: simplify pruning logic
Since ebcc4db84a the flow worker runs
file pruning after parsing, detection and loging. This means we can
simplify the pruning logic. If a file is in state >= CLOSED, we can
prune it. Detection and outputs will have had a final chance to
process it.

Remove the calls to the pruning code from Rust. They are no longer
needed.
6 years ago
Victor Julien b4318a11e3 rust: remove build system HAVE_RUST guards 6 years ago
Jason Ish a3cdef2b4d rust: run tests with same features as build
Cargo check wasn't being passed --features so could have a different
configuration than the build.
6 years ago
Jason Ish 389272f4c7 rustup: handle rustup for sudo and su
If rustup is in use, and a user uses sudo or su for the make
install, the install may fail with a "no default toolchain"
error.

To prevent this, detect at configure if rustup is being used,
then set RUSTUP_HOME for all calls to cargo.
6 years ago
Jason Ish 6e981fd15a rust: fix build when source directory has spaces in it
Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2668
6 years ago
Jason Ish f9c9548b74 configure: detect lua integer size
Lua 5.1 and 5.3 use a different integer size. Run a test program
to set the integer size used in the Rust FFI layer to Rust.
6 years ago
Jeff Lucovsky bd691778eb rust/ftp: add parser for active mode port handling 6 years ago
Jason Ish 517ecd68a9 sip: rustfmt
As this is new Rust code, format with rustfmt using default
styling.
6 years ago
Jason Ish a45a2fa1fc sip: disable by default in 5.0 6 years ago
Jason Ish fdbc2fe49c sip rust fixup: remove unused import in tests 6 years ago
Giuseppe Longo e06291922f detect/sip.response_line: add sticky buffer
Matches on response line field in SIP.
6 years ago
Giuseppe Longo 17de4a8023 detect/sip.request_line: add sticky buffer
Matches on request line field in SIP.
6 years ago
Giuseppe Longo 8939ece538 detect/sip.stat_msg: add sticky buffer
Matches on status msg field in SIP.
6 years ago
Giuseppe Longo bd2219cac6 detect/sip.stat_code: add sticky buffer
Matches on status code field in SIP.
6 years ago
Giuseppe Longo 8454122eb2 detect/sip.protocol: add sticky buffer
Matches on protocol field in SIP.
6 years ago
Giuseppe Longo 2661c5b298 detect/sip.uri: add sticky buffer
Matches on uri field in SIP.
6 years ago
Giuseppe Longo 424eead8c0 detect/sip.method: add sticky buffer
Matches on uri field in SIP.
6 years ago
Giuseppe Longo edc2a583a9 rust/sip: add SIP logger 6 years ago
Giuseppe Longo 2e975a0481 rust/sip: add parser for SIP protocol 6 years ago
Jason Ish d79c23baa3 dns/detect: dns.opcode keyword
Add a rule keyword, dns.opcode to match on the opcode flag
found in the DNS request and response headers.

Only exact matches are allowed with negation.

Examples:
  - dns.opcode:4;
  - dns.opcode:!1;
6 years ago
Victor Julien 85ba2e16ba rust/conf: don't print failed conf lookups at info level 6 years ago
Jason Ish 664605b5f1 rdp: disable rdp by default for 5.0 6 years ago
Jason Ish 0f10298990 rdp: address comments in pull request
Pull request:
https://github.com/OISF/suricata/pull/4174

- fix commit: range -> set
- OUTPUT_BUFFER_SIZE -> JSON_OUTPUT_BUFFER_SIZE
- output: check for initdata first
6 years ago
Zach Kelly caef8b5b38 protocol parser: rdp
Initial implementation of feature 2314:
1. Add protocol parser for RDP
2. Add transactions for RDP negotiation
3. Add eve logging of transactions
6 years ago
Jason Ish 5f1d21f247 dns: handle mid stream pickup on response packet
Related Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2146
6 years ago
Jason Ish 42e5065ab8 rust: update to Rust 2018 with cargo fix
Migrate to Rust 2018 edition.

Credit to Danny Browning for first demontrating this:
https://github.com/OISF/suricata/pull/3604/commits
6 years ago
Shivani Bhardwaj 8bebea5d4c rust: Get rid of unneeded macros, fix warnings 6 years ago
Victor Julien 579cc9f02b const: constify decoder, app-layer, detect funcs 6 years ago
Jason Ish 8f66f39b29 rustfmt: use default rustfmt configuration
Remove our config that limited line length to 80 columns.
6 years ago
Shivani Bhardwaj f750e4ca40 configure: Remove enable-rust-debug
Get rid of enable-rust-debug flag and use enable-debug for acheiving the
desired functionality. From now, adding `--enable-debug` to `configure`
shall create an [unoptimitized + debuginfo] target. Rest behavior stays
the same.

Closes redmine ticket #3054
6 years ago
Shivani Bhardwaj 6d39f6fd7d rust: Fix deprecation warnings
Fix the following warnings by compiler,
(1) warning: use of deprecated item 'take_until_s': Please use `take_until` instead
(2) warning: `...` range patterns are deprecated

For the second warning, the builtin lint
"ellipsis_inclusive_range_pattern" has been added which causes the
following warning to show up with rustc 1.24.

warning: unknown lint: `ellipsis_inclusive_range_patterns`
  --> /home/travis/build/OISF/suricata/suricata-5.0.0-dev/rust/src/lib.rs:18:10
   |
18 | #![allow(ellipsis_inclusive_range_patterns)]
   |          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: #[warn(unknown_lints)] on by default

Since there is no other way to fix this, the above warning shall stay.
We need to take care of modifying this if and when the support for 1.24
as MSRV is dropped.
6 years ago
Shivani Bhardwaj bbfd706e1f rust: fix compiler warning
rustc 1.36 introduced:

error: variable does not need to be mutable
   --> src/dhcp/parser.rs:202:17
    |
202 |             let mut malformed_options = false;
    |                 ----^^^^^^^^^^^^^^^^^
    |                 |
    |                 help: remove this `mut`
    |
note: lint level defined here
   --> src/lib.rs:18:38
    |
18  | #![cfg_attr(feature = "strict", deny(warnings))]
    |                                      ^^^^^^^^
    = note: #[deny(unused_mut)] implied by #[deny(warnings)]

error: aborting due to previous error

error: Could not compile `suricata`.

Ticket #3072.
6 years ago
Pierre Chifflier af7d245a31 rust/snmp: add event when expected/received PDU versions mismatch 6 years ago
Pierre Chifflier 1880f6945c rust/snmp: use generic parsing function, for all SNMP versions
Do no restrict parsing to the version seen in the first packet, but
use a generic function, independent of the version.
6 years ago
Jeff Lucovsky 6911cc01ad rust/snmp: Support get-info-by-id 6 years ago
Jeff Lucovsky 7560b75591 rust/ntp: Support get-info-by-id 6 years ago
Jeff Lucovsky 12c2d18c8b rust/krb: Support get-info-by-id 6 years ago
Jeff Lucovsky fb01641629 rust/ikev2: Support get-info-by-id 6 years ago
Jeff Lucovsky e3ca6b43fc rust/dhcp: Support get-info-by-id 6 years ago
Jeff Lucovsky a5d9d37c34 rust/parser: Extend Rust parser for event-by-id
Extend the Rust parsing infrastructure with the "get event info by id"
calls. This changeset extends the parser structure, the C-based
registration handlers and the template parser.
6 years ago
Jeff Lucovsky 9ccc28baeb rust/smb: Implement get event by id 6 years ago
Jeff Lucovsky 643864a8f5 rust/snmp: fix libc deprecation warnings 6 years ago
Victor Julien 3f6624bf16 rust: remove libc crate dependency
Use std::os::raw instead.
6 years ago
Victor Julien 28ed0d3a18 nfs: implement get_event_info_by_id callback 6 years ago
Victor Julien 429ca858dc rust/gen: turn *mut*const T into const T ** 6 years ago
Jeff Lucovsky d568e7fadd eve/logging: 2991 Optimize logging by TX
This changeset makes changes to the TX logging path. Since the txn
is passed to the TX logger, the TX can be used directly instead of
through the TX id.
6 years ago
Jeff Lucovsky 1a1d32c6b2 make: Remove rust generated headers during clean 7 years ago
Pierre Chifflier c1b30fe9fd rust/snmp: fix libc deprecation warnings for int types 7 years ago
Pierre Chifflier bc07656ce7 rust/snmp: use snake_case when logging PDU types 7 years ago
Pierre Chifflier c60f2028e5 rust/snmp: fix missing IPPROTO_* declarations (use core) 7 years ago
Pierre Chifflier 031cbbe868 rust/snmp: fix selection of v1/v2c parser 7 years ago
Pierre Chifflier 9dfec7e734 SNMP: add the "snmp.pdu_type" detection keyword 7 years ago
Pierre Chifflier e1dd19a0eb SNMP: add the "snmp.community" detection keyword 7 years ago
Pierre Chifflier aa608e0ca2 SNMP: add the "snmp.version" detection keyword 7 years ago
Pierre Chifflier 60324740e6 SNMP: use explicit references to support build with old rust compiler 7 years ago
Pierre Chifflier 57b233f462 SNMP: start looking for transactions from end of list 7 years ago
Pierre Chifflier 6fc7fc74cb SNMP: add logger 7 years ago
Pierre Chifflier 2df840a8b8 Add SNMP (v1/v2c/v3) application layer 7 years ago
Pierre Chifflier b65896c0de Rust: expose function AppLayerParserRegisterGetTxIterator 7 years ago
Victor Julien b1d4931842 rust: fix warnings about wrong type of comments
"rustdoc does not generate documentation for macro expansions"
7 years ago
Victor Julien bf1bd407dd rust: fix libc deprecation warnings for int types 7 years ago
Victor Julien 723f1586ca ikev2: remove excess new lines 7 years ago
Victor Julien adcbac1c77 tftp: properly implement tx handling 7 years ago
Victor Julien 63ab296cca nfs: fix integer underflow
Fix int underflow that leads to Rust panic in NFS3 readdirplus
parsing.

Reported-by: Sirko Höer -- Code Intelligence for DCSO.
7 years ago
Jason Ish 8be4142aaf dhcp: verify client id len before parsing data
Verify that the client id length is at least 2 per the DHCP
protocol rfc before parsing the data.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2902
7 years ago
Jason Ish 9d75fdc6ea rust/ftp: validate port components in passive reponse
Make sure they are valid 8 bit integers before combining the
two parts into a u16 to prevent an overflow of the u16
return value.

Add unit tests to check parsing of invalid ports.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2904
7 years ago
Victor Julien 24d6a16459 rust/mingw: build fixes
Fix path passed to cargo by using 'cygpath' if available.
7 years ago
Victor Julien f84667ceb7 nfs: small cleanups 7 years ago