aiden
/
suricata
mirror of https://github.com/OISF/suricata
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
17,728 Commits
7 Branches
161 Tags
206 MiB
main-7.0.x
master
master-6.0.x
master-5.0.x
master-4.1.x
master-4.0.x
master-3.2.x
suricata-8.0.0
suricata-7.0.11
suricata-8.0.0-rc1
suricata-8.0.0-beta1
suricata-7.0.10
suricata-7.0.9
suricata-7.0.8
suricata-7.0.7
suricata-7.0.6
suricata-6.0.20
suricata-7.0.5
suricata-6.0.19
suricata-6.0.18
suricata-7.0.4
suricata-6.0.17
suricata-7.0.3
suricata-6.0.16
suricata-7.0.2
suricata-6.0.15
suricata-7.0.1
suricata-6.0.14
suricata-7.0.0
suricata-7.0.0-rc2
suricata-6.0.13
suricata-6.0.12
suricata-6.0.11
suricata-7.0.0-rc1
suricata-6.0.10
suricata-6.0.9
suricata-7.0.0-beta1
suricata-6.0.8
suricata-6.0.7
suricata-6.0.6
suricata-5.0.10
suricata-6.0.5
suricata-5.0.9
suricata-6.0.4
suricata-5.0.8
suricata-6.0.3
suricata-5.0.7
suricata-6.0.2
suricata-5.0.6
suricata-6.0.1
suricata-5.0.5
suricata-4.1.10
suricata-4.1.9
suricata-5.0.4
suricata-6.0.0
suricata-6.0.0-rc1
suricata-6.0.0-beta1
suricata-5.0.3
suricata-4.1.8
suricata-4.1.7
suricata-5.0.2
suricata-4.1.6
suricata-5.0.1
suricata-5.0.0
suricata-5.0.0-rc1
suricata-4.1.5
suricata-5.0.0-beta1
suricata-4.1.4
suricata-4.1.3
suricata-4.0.7
suricata-4.1.2
suricata-4.1.1
suricata-4.1.0
suricata-4.0.6
suricata-4.1.0-rc2
suricata-4.1.0-rc1
suricata-4.0.5
suricata-4.1.0-beta1
suricata-4.0.4
suricata-4.0.3
suricata-4.0.2
suricata-3.2.5
suricata-4.0.1
suricata-3.2.4
suricata-4.0.0
suricata-4.0.0-rc2
suricata-3.2.3
suricata-4.0.0-rc1
suricata-4.0.0-beta1
suricata-3.2.2
suricata-3.1.4
suricata-3.2.1
suricata-3.2
suricata-3.2RC1
suricata-3.1.3
suricata-3.2beta1
suricata-3.1.2
suricata-3.1.1
suricata-3.1
suricata-3.0.2
suricata-3.1RC1
suricata-3.0.1
suricata-3.0.1RC1
suricata-3.0
suricata-2.0.11
suricata-3.0RC3
suricata-3.0RC2
suricata-3.0RC1
suricata-2.0.10
suricata-2.0.9
suricata-2.1beta4
suricata-2.0.8
suricata-2.0.7
suricata-2.1beta3
suricata-2.0.6
suricata-2.0.5
suricata-2.1beta2
suricata-2.0.4
suricata-2.1beta1
suricata-2.0.3
suricata-2.0.2
suricata-2.0.1
suricata-2.0.1rc1
suricata-2.0
suricata-2.0rc3
suricata-2.0rc2
suricata-2.0rc1
suricata-2.0beta2
suricata-1.4.7
suricata-1.4.6
suricata-1.4.5
suricata-2.0beta1
suricata-1.4.4
suricata-1.4.3
suricata-1.4.2
suricata-1.4.1
suricata-1.3.6
suricata-1.4
suricata-1.3.5
suricata-1.4rc1
suricata-1.3.4
suricata-1.4beta3
suricata-1.3.3
suricata-1.4beta2
suricata-1.3.2
suricata-1.4beta1
suricata-1.3.1
suricata-1.3
suricata-1.3rc1
suricata-1.3beta2
suricata-1.3beta1
suricata-1.2.1
suricata-1.2
suricata-1.2rc1
suricata-1.2beta1
suricata-1.1.1
suricata-0.8.2
suricata-1.0.0
suricata-1.0.1
suricata-1.0.2
suricata-1.0.3
suricata-1.0.4
suricata-1.0.5
suricata-1.1
suricata-1.1beta1
suricata-1.1beta2
suricata-1.1beta3
suricata-1.1rc1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'master'
${ noResults }
Commit Graph

1929 Commits (master)

Author SHA1 Message Date
Eric Leblond 44482a68b0 ntp: add TX orientation 
Set no inspection in the opposite side of the transaction.

Ticket: #5799
 2 years ago
Eric Leblond c64e4526cd krb: add TX orientation 
Set no inspection in the opposite side of the transaction.

Ticket: #5799
 2 years ago
Eric Leblond 322f3896ba mqtt: add TX orientation 
Set no inspection in the opposite side of the transaction.

Ticket: #5799
 2 years ago
Eric Leblond 7ce557a44c ike: add TX orientation 
Set no inspection in the opposite side of the transaction.

Ticket: #5799
 2 years ago
Eric Leblond 8926d82465 dns: add TX orientation 
Set no inspection in the opposite side of the transaction.

Ticket: #5799
 2 years ago
Eric Leblond 744fccee17 bittorrent_dht: add TX orientation 
Set no inspection in the opposite side of the transaction.

Ticket: #5799
 2 years ago
Eric Leblond a82a5aa84b snmp: add TX orientation 
Set no inspection in the opposite side of the transaction.

Ticket: #5799
 2 years ago
Eric Leblond 5aaf50760f app-layer: add flag to skip detection on TX 
Stamus team did discover a problem were a signature can shadow
other signatures.

For example, on a PCAP only containing Kerberos protocol and where the
following signature is matching:

alert krb5 $HOME_NET any -> any any (msg:"krb match"; krb5_cname; content:"marlo"; sid:3; rev:1;)

If we add the following signature to the list of signature

alert ssh $HOME_NET any -> any any (msg:"rr"; content:"rr"; flow:established,to_server; sid:4; rev:2;)

Then the Kerberos signature is not matching anymore.

To understand this case, we need some information:

- The krb5_cname is a to_client keyword
- The signal on ssh is to_server
- Kerberos has unidirectional transaction
- kerberos application state progress is a function always returning 1

As the two signatures are in opposite side, they end up in separate
sig group head.

Another fact is that, in the PCAP, the to_server side of the session
is sent first to the detection. It thus hit the sig group head of
the SSH signature. When Suricata runs detection in this direction
the Kerberos application layer send the transaction as it is existing
and because the alstate progress function just return 1 if the transaction
exists. So Suricata runs DetectRunTx() and stops when it sees that
sgh->tx_engines is NULL.

But the transaction is consumed by the engine as it has been evaluated
in one direction and the kerberos transaction are unidirectional so
there is no need to continue looking at it.

This results in no matching of the kerberos signature as the match
should occur in the evaluation of the other side but the transaction
with the data is already seen has been handled.

This problem was discovered on this Kerberos signature but all
the application layer with unidirectional transaction are impacted.

This patch introduces a flag that can be used by application layer
to signal that the TX should not be inspected. By using this flag
on the directional detect_flags_[ts|tc] the application layer can
prevent the TX to be consumed in the wrong direction.

Application layers with unidirectional TX will be updated
in separate commits to set the flag on the direction opposite
to the one they are.

Ticket: #5799
 2 years ago
Eric Leblond 236869bc58 detect: remove STREAM_FLUSH 
It is unused in the code so can be removed.

Ticket: #5799
 2 years ago
Jason Ish 60e67db452 rust: don't suppress vendor output 
There appears to be some errors happening in CI and this may be hiding
the source of the error.
 2 years ago
Jason Ish 6f14aed0e6 rust: bundle Cargo.lock 
Cargo.lock has to be provided as template, Cargo.lock.in so it can
live beside Cargo.lock in out of tree automake builds, like distcheck.

This will pin Rust dependencies even for git builds, updating
Cargo.lock will now be a manual process that we'll have to take care
of periodically.
 2 years ago
Jason Ish d4418034d1 rust: update aes and aes-gcm crates 
Addresses RUSTSEC-2021-0059, RUSTSEC-2021-0060.
 2 years ago
Jason Ish 159b72c101 rust/clippy: allow derivable impls 
The latest Rust will automatically "fix" derivable default
implementation, which is nice, but makes changes that don't meet our
current MSRV, so allow derivable impls for now.
 2 years ago
Lancer Cheng 9207012e4b smb: fix parser of ntlmssp negotiateflags 
Fix endian-conversion bug in function parse_ntlm_auth_nego_flags

Bug OISF#5783
 2 years ago
Haleema Khan 6b55e53ff5 rfb: add unittests to rfb.rs 
Task: #5741
 2 years ago
Philippe Antoine 233ab11148 smb: handles records with trailing nbss data 
If a file (read/write) SMB record has padding/trailing data
after the buffer being read or written, and that Suricata falls
in one case where it skips the data, it should skip until
the very end of the NBSS record, meaning it should also skip the
padding/trailing data.

Otherwise, an attacker may smuggle some NBSS/SMB record in this
trailing data, that will be interpreted by Suricata, but not
by the SMB client/server, leading to evasions.

Ticket: #5786
 2 years ago
Philippe Antoine c1b7befb18 smb: checks against nbss records length 
When Suricata handles files over SMB, it does not wait for the
NBSS record to be complete, and can stream the payload to the
file... But it did not check the consistency of the SMB record
length being read or written against the NBSS record length.

This could lead to an evasion where an attacker crafts a SMB
write with a too big Length field, and then sends its evil
payload, even if the server returned an error for the write request.

Ticket: #5770
 2 years ago
Jason Ish f15f092a69 rfb: remove duplicate logging of depth 
The "depth" field in the "pixel_format" object was being logged twice.

Issue: 5813
 3 years ago
Jason Ish 717e2b0248 smb: fix duplicate interface logging 
An array of interfaces was being logged without creating an array,
resulting in duplicate "interface" objects being logged. Instead put
these interfaces into an array like already done elsewhere.

Issue: 5814
 3 years ago
Jason Ish 67baab573b smb: remove duplicate tree_id logging 
Remove the second occurrence of tree_id logging which appears to
always be a duplicate of the first tree_id logged, even though they
come from different data structures.

Issue: 5811
 3 years ago
Jason Ish e21ae88e05 rust: utility function to copy Rust strings to C strings 
As there are a few places where a Rust string is copied into a provided
C string buffer, create a utility function to take care of these
details.
 3 years ago
Jason Ish 6344501dba tls: fix date logging for dates before 1970 
The Rust time crate used by the x509-parser crate represents dates
before 1970 as negative numbers which do not survive the conversion to
SCTime_t and formatting with the current time formatting functions.

Instead of fixing our formatting functions to handle such dates,
create a Rust function for logging TLS dates directly to JSON using
the time crate that handles such dates properly.

Also add a FFI function for formatting to a provided C buffer for the
legacy tls-log.

Issue: 5817
 3 years ago
Jason Ish 64cb687a65 rust: suppress specific manual_flatten list 
In this case of debug code, the explicit iterator seems to make more
sense.
 3 years ago
Jason Ish 7080ecbb76 rust: remove explicit lifetimes where not needed 3 years ago
Jason Ish e7f5bd047d rust: fix needless borrows of references 
Fixed automatically by cargo clippy --fix.
 3 years ago
Jason Ish 29f345af1a rust: allow uninlined_format_args 
Newer versions of Rust/clippy are getting picky about format strings.
We should allow and use the new style, but also not prevent the old
style.
 3 years ago
Jason Ish 3f4dad8676 ftp: add events for command too long 
Issue: 5235
 3 years ago
Jason Ish 48920bd784 rust/derive: allow event name to be set as attribute 
When deriving AppLayerEvent, allow the event name to be set with the
"name" attribute in cases where the transformed name is not suitable.

This allows us to use enum variant names like
"FtpEventRequestCommandTooLong" for direct use in C, but is also a
name that doesn't transform well to an event name in rules, where we
want to see "request_command_too_long".
 3 years ago
Philippe Antoine b52293b609 dcerpc: config limit maximum number of live transactions 
As is done for other protocols

Ticket: #5779
 3 years ago
Philippe Antoine ba99241957 http2: fix leak with range files 
Ticket: #5808

May have been introduced by a24d7dc45c

Function http2_range_open expects to be called only when
tx.file_range is nil. One condition to ensure this is to check
that we are beginning the files contents. The filetracker field
file_open is not fit for this, as it may be reset to false.
 3 years ago
Victor Julien 37f13a4fc7 smb: set defaults for file transfer limits 
Ticket: #5782.
 3 years ago
Jason Ish fab3f36b8c dns: never return error on UDP DNS 
UDP parsers should never return error as it should indicate to Suricata
that an unrecoverable error has occurred.  UDP being record based for
the most part is almost always recoverable, at least for protocols like
DNS.
 3 years ago
Jason Ish d720ead470 dns: split header and body parsing 
As part of extra header validation, split out DNS body parsing to
avoid the overhead of parsing the header twice.
 3 years ago
Jason Ish 595700ab7e dns: validate header on every incoming message 
As UDP streams getting probed, a stream that does not appear to be DNS
at first, may have a single packet that does look close enough to DNS
to be picked up as DNS causing every subsequent packet to result in a
parser error.

To mitigate this, probe every incoming DNS message header for validity
before continuing onto the body.  If the header doesn't validate as
DNS, just ignore the packet so no parse error is registered.
 3 years ago
Jason Ish c98c49d4ba dns: parse and alert on invalid opcodes 
Accept DNS messages with an invalid opcode that are otherwise
valid. Such DNS message will create a parser event.

This is a change of behavior, previously an invalid opcode would cause
the DNS message to not be detected or parsed as DNS.

Issue: #5444
 3 years ago
Jason Ish 7afc2e3aed dns: rustfmt 3 years ago
Jason Ish 39d2524bf6 dns: mark test buffers with rustfmt::skip 3 years ago
Victor Julien 6cc9811edd files: move FileContainer into FileTransferTracker 
Update SMB, NFS, HTTP2.
 3 years ago
Victor Julien e3e55406a7 files: update API and callers to take stream config 
This is to allow not storing the stream buffer config in each file.
 3 years ago
Victor Julien 71bc9e75f5 app-layer: get sbconfg with files 3 years ago
Victor Julien a1a221066f files: remove filecontainer drop trait 
In preparation of it becoming impossible to use due to the free
function getting an cfg argument.
 3 years ago
Victor Julien 0320c03f8c http2: explicity free files 
In preparation of adding an argument to the free functions which
means the drop trait can't be used anymore.
 3 years ago
Victor Julien 4b1e9f7c21 smb: explicity free files 
In preparation of adding an argument to the free functions which
means the drop trait can't be used anymore.
 3 years ago
Victor Julien 3a24cce289 nfs: explicity free files 
In preparation of adding an argument to the free functions which
means the drop trait can't be used anymore.
 3 years ago
Victor Julien 4bfeac6591 nfs: file handling cleanups 3 years ago
Victor Julien 33f6a16290 smb: file handling cleanups 3 years ago
Victor Julien d57510a10f files: remove unused Rust binding for file pruning 3 years ago
Victor Julien a24d7dc45c smb: fix post-trunc chunk behavior 
After a gap in a file transaction, the file tracker is truncated. However
this did not clear any stored out of order chunks from memory or stop more
chunks to be stored, leading to accumulation of a large number of chunks.

This patches fixes this be clearing the stored chunks on trunc. It also
makes sure no more chunks are stored in the tracker after the trunc.

Bug: #5781.
 3 years ago
Philippe Antoine 55c4834e4e smb: configurable max number of transactions per flow 
Ticket: #5753
 3 years ago
Philippe Antoine 1d9183638f smb: convert transaction list to vecdeque 
Allows for more efficient removal from front of the list.

Ticket: #5753
 3 years ago
Philippe Antoine cb89192ec3 smb: fix typo in comment 3 years ago
Haleema Khan cfcb7df9dc mqtt: rustfmt parser.rs 3 years ago
Haleema Khan 23acb89653 mqtt: add unittests for nom7 parsers 
Ticket: #5742
 3 years ago
Haleema Khan cdc5ccd7f7 rfb: rustfmt parser.rs 3 years ago
Haleema Khan b95d7efbd0 rfb: add unittests for nom7 parsers 
Task: #5741
 3 years ago
Philippe Antoine 3979acb5ed smb: set event for ntlmssp unusual order 3 years ago
Philippe Antoine e41c01a483 smb: rustfmt ntlmssp_records.rs 3 years ago
Philippe Antoine 1db8685848 smb/ntlmssp: parse fields independently of order 
Instead of relying on the usual ordering...

Ticket: #5258
 3 years ago
Jason Ish ae192ebae7 rust: sync log levels with C 3 years ago
Jeff Lucovsky f8474344cd log: Add module and subsystem identifiers to log 
Issue: 2497

This changeset provides subsystem and module identifiers in the log when
the log format string contains "%S". By convention, the log format
surrounds "%S" with brackets.

The subsystem name is generally the same as the thread name. The module
name is derived from the source code module name and usually consists of
the first one or 2 segments of the name using the dash character as the
segment delimiter.
 3 years ago
Victor Julien b31ffde6f4 output: remove error codes from output 3 years ago
Jason Ish bd9adac3ac rust/clippy: comments on why we have specific allows 3 years ago
Jason Ish dfd7abe185 rust/clippy: fix lint: type_complexity 
Convert a DNS sub-parser to use a return type rather than a large
tuple. For mqtt, allow the lint for now, but remove the global allow.
 3 years ago
Jason Ish e49ce49471 rust/clippy: allow result_unit_err in http2 only 
Its the only module making use of this pattern, but we shouldn't let
new modules use this pattern.
 3 years ago
Jason Ish 7ba2dadc7f rust/clippy: fix lint: upper_case_acronyms 3 years ago
Jason Ish 029ac650d7 rust/clippy: fix lint: manual_find 
These get_tx methods look like ideal candidates for generic and/or
derived methods.
 3 years ago
Jason Ish 4940dfb3bd rust/clippy: fix lint: len_without_is_empty 3 years ago
Jason Ish e1cffd348f rust/clippy: fix lint: field_reassign_with_default 3 years ago
Jason Ish 9df7c326b9 rust/clippy: remove allow: collapsible_else_if 3 years ago
Jason Ish 30ee5fc835 rust/clippy: remove allow: collapsible_if 
Already clean.
 3 years ago
Jason Ish da12b77f18 rust/clippy: fix lint: new_without_default 3 years ago
Jason Ish c4cf062a6f rust/clippy: fix lint: redundant_pattern_matching 3 years ago
Jason Ish 7c293ff68f rust/clippy: fix lint: never_loop 3 years ago
Jason Ish e8823644ec rust/clippy: fix lint: nonminimal_bool 3 years ago
Jason Ish 53ae0c8a06 rust/clippy: fix lint: derive_partial_eq_without_eq 3 years ago
Jason Ish 5d62995e26 rust/clippy: fix lint: explicit_counter_loop 3 years ago
Jason Ish f250b92180 rust/clippy: fix lint: extra_unused_lifetimes 3 years ago
Jason Ish 3044565cf4 rust/clippy: fix lint: needless_range_loop 3 years ago
Jason Ish 2ac52d0610 rust/clippy: remove lint: for_loops_over_fallibles 
Already clean.
 3 years ago
Jason Ish c026d8531b rust/clippy: fix lint: match_ref_pats 3 years ago
Jason Ish 359d5fcb7e rust/clippy: fix lint: needless_lifetimes 3 years ago
Jason Ish 4e001688de rust/clippy: remove lint: bool_comparison 
Already clean.
 3 years ago
Jason Ish f15ffbc869 rust/clippy: fix lint: single_match 
Allow this lint in some cases where a match statement adds clarity.
 3 years ago
Jason Ish 925bc74c1f rust/clippy: fix lint: while_let_loop 3 years ago
Jason Ish cf20fa1e67 template: import c_void, c_char, c_int 
These are ffi types that are commonly used, import them so they can be
used by their short names instead of a fully qualified name.
 3 years ago
Jason Ish 4220f18258 template: remove no_mangle and pub where not needed 
Extern functions that are only used as a function pointer do not
require "pub" or "no_mangle".
 3 years ago
Jason Ish 4a7567b3f0 template: rename template-rust to template 
Remove the distinction between the C template protocol "template" and
the Rust template protocol "template-rust" and make the Rust parser
simply template now that we no longer have support to generate a C
protocol template.
 3 years ago
Jason Ish 38321a213f rust/app-layer-template: rustfmt 3 years ago
Jason Ish 50a787a9a3 app-layer-template-rust: remove C app-layer stub 
Remove the app-layer-PROTO stub for Rust based parsers.  It is no longer
needed as Rust parsers now contain the registration function in Rust.

Ticket: 4939
 3 years ago
Jason Ish baa7021ee6 rust/conf: add fn conf_get_node 
A wrapper around ConfGetNode to get a configuration node by name.
 3 years ago
Victor Julien 64c0459d2d rust/lzma: clippy fixup 3 years ago
Jason Ish 35f99d1af7 rust/http2: fix clippy lint for is_empty() 
This snuck through as "cargo clippy" check wasn't finding lints that
were fixed by the previous test for fixable lints.
 3 years ago
Todd Mortimer 7d1a8cc335 file/swf: Use lzma-rs decompression instead of libhtp. 
Use the lzma-rs crate for decompressing swf/lzma files instead of
the lzma decompressor in libhtp. This decouples suricata from libhtp
except for actual http parsing, and means libhtp no longer has to
export a lzma decompression interface.

Ticket: #5638
 3 years ago
Victor Julien 45eb038e63 smb: fix file reopening issue 
Fuzzing highlighted an issue where a command sequence on the same file
id triggered a logging issue:

file data for id N
close id N
file data for id N

If this happened in a single blob of data passed to the parser, the
existing file tx would be reused, the file "reopened", confusing the
file logging logic. This would trigger a debug assert.

This patch makes sure a new file tx is created for the file data
coming in after the first file tx is closed.

Bug: #5567.
 3 years ago
Philippe Antoine 29f40c9e07 dcerpc: fix integer underflow 
as input.len() can be 65536, it cannot be directly cast to u16

Ticket: #5557
 3 years ago
Jason Ish 91617f479a rust: sha-1 is now sha1 
This is the same crate, but renamed to be more consistent with the
RustCrypto project naming. Some recent discussion is available here:

    https://github.com/RustCrypto/hashes/issues/438
 3 years ago
Philippe Antoine af44504550 smb: do not use tree id to match request and response 
Completes commit e94920b49f

This must be true for access to state ssn2vecoffset_map

Ticket: #5161
 3 years ago
Victor Julien cade6046c5 rust/files: open file without trackid as pointer 3 years ago
Victor Julien ad869e1c52 rust/filecontainer: remove unused declaration 3 years ago
Philippe Antoine 086b28da3d http2: fix decompression buffering 
It was not enough to set Cursor position to 0,
also its inner Vec should be cleared.

This way, a new input gets written at the beginning of the
Cursor and its inner Vec...

Ticket: #5691
 3 years ago
Philippe Antoine c6349d3cfc http2: support padded data frames 
Ticket: #5691
 3 years ago
Philippe Antoine e1ee401a12 quic: use VecDeque 
Ticket: #5637
 3 years ago
Philippe Antoine 286bd2a7ed rust: fix cargo clippy --all-features 3 years ago
Philippe Antoine bc287018e5 rust: cargo clippy --all-features --fix --allow-no-vcs 3 years ago
Philippe Antoine cd4bf518f3 rust: fix warnings on rustc 1.67.0-nightly 
warning: for loop over an `Option`. This is more readably written
as an `if let` statement
 3 years ago
Juliana Fajardini a654ef50de pgsql: add test for parameter status parser 
Since we've done some changes to how the parameters are parsed, add one
more test case to check that.

Bug #5579
 3 years ago
Juliana Fajardini c4fbd78770 pgsql: move database into opt parameters list 
For StartupMessages, the database parameter is optional. This moves the
parameter into the optional_parameters list.

Bug #5579
 3 years ago
Philippe Antoine cc68898532 pgsql: support empty parameter values 
Bug #5579
 3 years ago
Philippe Antoine 1e0190bc6b pgsql: support out of order parameters for startup message 
As user can be not the first parameter

Bug #5579
 3 years ago
Jason Ish 1f056f9974 bittorrent-dht: parse and log node6 lists 
Node6 lists are just like node lists, but for IPv6 addresses.
 3 years ago
Jason Ish 86d5ab8644 bittorrent-dht: remove tests that are no longer valid 3 years ago
Jason Ish 2f9eb5d1dd bittorrent-dht: fix values decoding, as a list of peers 
The "values" field is not a string, but instead peer information in
compact format. Decode this properly and then properly format in the
log.
 3 years ago
Jason Ish 4a0859beeb jsonbuilder: add append_hex - add hex to array 
New method, append_hex to add a byte array to a JSON array in hex
encoding.
 3 years ago
Jason Ish 4bc9cf3986 bittorrent-dht: parse token and target as byte values 3 years ago
Jason Ish 5a30ee77a1 bittorrent-dht: only attempt to parse dht messages 
The bittorrent flow is shared with transport messages as well as dht
messages. Only attempt to parse dht message as dht, ignore the rest.
 3 years ago
Jason Ish 98a9391210 bittorrent-dht: decode node data structures 
Instead of decoding the nodes field into a blog of bytes, decode it into
an array of node info objects, each with a node id, IP address and port.
 3 years ago
Jason Ish 3cb50592ed bittorrent-dht: convert some fields to byte arrays 
Some fields that were previously strings are not always value UTF-8
data, instead the protocol specification refers to them as strings of
bytes, so in other words byte arrays.

Currently fields converted are:
- client_version
- info_hash
- response.id
- request.id
- nodes
- token
 3 years ago
Jason Ish 78ba17caa8 bittorrent-dht: register a pattern for protocol detection 
Removes the port based probing which takes a long time to register for
ports 1024-65535 and instead use pattern based protocol detection.
 3 years ago
Jason Ish 350c0723d7 bittorrent-dht: set parser to unidirectional 
This parser does not match up responses with requests so flag it as
unidirectional.
 3 years ago
Jason Ish 06eaec67ac bittorrent: updates for new event handling 
Fixes anomaly events.
 3 years ago
Aaron Bungay 86037885a9 bittorrent-dht: add bittorrent-dht app layer 
Parses and logs the bittorrent-dht protocol.

Note: Includes some compilation fixups after rebase by Jason Ish.

Feature: #3086
 3 years ago
Haleema Khan 8d5c5f24a1 dns/eve: add 'HTTPS' type logging 
Add a new DNS record type to represent HTTPS
Ticket: #4751
 3 years ago
Alice Akaki ccdc992a71 rust: fix lint warnings about mixed case hex literals 
Ticket: #4593
 3 years ago
Gabriel Lima Luz 4e90d17fd9 rust: fix lint warnings about if same then else 
Ticket: 4609
 3 years ago
Kristina Jefferson 9cd00424c3 rust: fix lint warnings about ptr_arg 
Ticket: #4599
 3 years ago
Jason Ish 21bb697bc9 rust: fix clippy lint for unneeded late initialization 3 years ago
Jason Ish 7cca238128 rust: fix clippy lint for cmp_null is debug code 
Ticket: 5577
 3 years ago
Jason Ish f0952aef0d rust: fix clippy lints for unneeded reference in debug code 3 years ago
Jason Ish 6a7439a26b rust: fix clippy lints for is_empty in debug code 3 years ago
Jason Ish 560c4ea125 rust: don't allow fixed up clippy lints 3 years ago
Jason Ish 6db85d6f89 rust: clippy fix for bitwise or with 0 3 years ago
Jason Ish 04f0ee0151 rust: fix clippy lints for clippy::unnecessary_cast 3 years ago
Jason Ish b6cc0e25b1 rust: fix clippy lints for clippy::redundant_static_lifetimes 3 years ago
Jason Ish 13db83274b rust: fix clippy lints for clippy::redundant_pattern_matching 3 years ago
Jason Ish 7ba1d3e300 rust: fix clippy lints for clippy::nonminimal_bool 3 years ago
Jason Ish 6ba0a67143 rust: fix clippy lints for clippy::map_flatten 3 years ago
Jason Ish 7ebdfa539a rust: fix clippy lints for clippy::manual_find 3 years ago
Jason Ish 119e02cf81 rust: fix clippy lints for clippy::collapsible_if 3 years ago
Jason Ish 572505870a rust: fix clippy lints for clippy::collapsible_else_if 3 years ago
Jason Ish 6b71d69356 rust: fix clippy lints for clippy::bool_comparison 3 years ago
Jason Ish e373d9f5e0 rust: fix clippy lints for clippy::crate_in_macro_def 3 years ago
Jason Ish 565da0d0af rust: fix clippy lints for clippy::redundant_field_names 3 years ago
Jason Ish 5f7ba03e63 rust: fix clippy lints for clippy::needless_bool 3 years ago
Jason Ish 3ec435a703 rust: fix clippy lints for clippy::manual_range_contains 3 years ago
Jason Ish f342d4aacd rust: fix clippy lints for clippy::len_zero 3 years ago
Jason Ish 5e5401d3e9 rust: fix clippy lints for clippy::char_lit_as_u8 3 years ago
Jason Ish 29a4a7fddc rust: fix clippy lints for clippy::assign_op_pattern 3 years ago
Jason Ish c4034dafa1 rust: fix clippy lints for clippy::derive_partial_eq_without_eq 3 years ago
Jason Ish 5a10fcd2d8 rust: suppress large enum variant lint at location 
And disable the global lint.
 3 years ago
Jason Ish 74b7522b6a rust/http2: box decompressor variants 
These variants, in particular the Brotli one can be large at over 2500
bytes which is allocated no matter which decompressor is being used.

Gzip comes in at over 500 bytes.  Box deflate for consistency.
 3 years ago
Jason Ish 36f8ada305 rust: remove clippy lints that no longer warn 3 years ago
Jason Ish e8c00dd980 rust: sort clippy allow statements 3 years ago
Haleema Khan 6c922e0b98 rust: fix lint warning for clippy::enum's name 
Ticket: #4597
 3 years ago
Jason Ish 2a42386c28 rust: fix clippy lint for null comparison 
Use .is_null() instead of checking for equality against
std::ptr::null().
 3 years ago
Jason Ish 45dfea2497 rust/modbus: derive default instead of manual impl 
Cleans up a clippy lint for a trivial default impl that can be derived.
 3 years ago
Jason Ish 9218da0eb8 rust/frames: cleanup clippy lint for unsafe 
Where possible mark the relevant functions unsafe.  Otherwise suppress
the warning for now as this pattern is supposed to be a safe API around
an unsafe one. Might need some further investigation, but in general the
"guarantee" here is provided from the C side.
 3 years ago
Jason Ish 105d9a5f02 rust: fix clippy lint for unnecessary_unwrap 
Avoid check if not none followed by unwrap.
 3 years ago
Jason Ish 85cfa7254b rust: fix clippy lint for single_char_add_str 
Idiomatic cleanup and a fix automatically done by `cargo clippy --fix`.
 3 years ago
Jason Ish f3e4bcfe23 rust: fix clippy lint for bool_assert_comparison 
Checking for is_empty is faster than checking for equality.
 3 years ago
Jason Ish f60e1b30f6 rust: fix clippy lint for partialeq_to_none 
Use .is_some() and .is_none() instead of comparing against None.
Comparing against None requires a value to impl PartialEq, is_none() and
is_some() do not and are more idiomatic.
 3 years ago
Jason Ish 7d623f0854 rust: fix clippy lint for explicit_auto_deref 
This adds unnecessary complexity to code.
 3 years ago
Jason Ish c503ca62e2 rust: fix clippy lint for needless_late_init 3 years ago
Jason Ish 94dd85baed rust: fix clippy lint for borrow_deref_ref 
This type of borrow then reference has no effect.
 3 years ago
Jason Ish e9597f3d0c rust: fix clippy lint for redundant_closure 
Removes a closure where the function can be directly provided.
 3 years ago
Jason Ish c5b26e2043 rust: fix clippy ling for needless borrows 
Cleanup needless borrows found by clippy. This fix done automatically by
`cargo clippy --fix`.
 3 years ago
Jason Ish 63b3d73ccc rust: allow some more clippy lints 
Allow these lints for now until some more investigation can be done, as
--fix attempts to fix these.
 3 years ago
Eric Leblond a9519778de rust/smb: avoid allocation in smb status function 
Avoid an allocation by returning a static string.
 3 years ago
Eric Leblond 9cb06d4376 detect/smb: add smb.ntlmssp_domain keyword 
Feature #5411.
 3 years ago
Eric Leblond 5debb86cd5 rust/smb1: add a missing command 3 years ago
Eric Leblond 69ef1bc194 detect/smb: add smb.ntlmssp_user keyword 
Feature #5411.
 3 years ago
Eric Leblond f46f895e8d rust/smb: import NT status code for Microsoft doc 
This patch updates the NT status code definition to use the status
definition used on Microsoft documentation website. A first python
script is building JSON object with code definition.

```
import json
from bs4 import BeautifulSoup
import requests

ntstatus = requests.get('https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-erref/596a1078-e883-4972-9bbc-49e60bebca55')

ntstatus_parsed = BeautifulSoup(ntstatus.text, 'html.parser')

ntstatus_parsed = ntstatus_parsed.find('tbody')

ntstatus_dict = {}

for item in ntstatus_parsed.find_all('tr'):
    cell = item.find_all('td')
    if len(cell) == 0:
        continue
    code = cell[0].find_all('p')
    description_ps = cell[1].find_all('p')
    description_list = []
    if len(description_ps):
        for desc in description_ps:
            if not desc.string is None:
                description_list.append(desc.string.replace('\n ', ''))
    else:
        description_list = ['Description not available']
    if not code[0].string.lower() in ntstatus_dict:
        ntstatus_dict[code[0].string.lower()] = {"text": code[1].string, "desc": ' '.join(description_list)}

print(json.dumps(ntstatus_dict))
```

The second one is generating the code that is ready to be inserted into the
source file:

```
import json

ntstatus_file = open('ntstatus.json', 'r')

ntstatus = json.loads(ntstatus_file.read())

declaration_format = 'pub const SMB_NT%s:%su32 = %s;\n'
resolution_format = '        SMB_NT%s%s=> "%s",\n'

declaration = ""
resolution = ""

text_max = len(max([ntstatus[x]['text'] for x in ntstatus.keys()], key=len))

for code in ntstatus.keys():
    text = ntstatus[code]['text']
    text_spaces = ' ' * (4 + text_max - len(text))
    declaration += declaration_format % (text, text_spaces, code)
    resolution += resolution_format % (text, text_spaces, text)

print(declaration)
print('\n')
print('''
pub fn smb_ntstatus_string(c: u32) -> String {
    match c {
''')
print(resolution)
print('''
        _ => { return (c).to_string(); },
    }.to_string()
}
''')
```

Bug #5412.
 3 years ago
Victor Julien db0f9ddc69 files/tx: inspection, logging and loop optimizations 
Introduce AppLayerTxData::file_tx as direction(s) indicator for transactions.
When set to 0, its not a file tx and it will not be considered for file
inspection, logging and housekeeping tasks.

Various tx loop optimizations in housekeeping and output.

Update the "file capable" app-layers to set the fields based on their
directional file support as well as on the traffic.
 3 years ago
Victor Julien 79499e4769 app-layer: move files into transactions 
Update APIs to store files in transactions instead of the per flow state.

Goal is to avoid the overhead of matching up files and transactions in
cases where there are many of both.

Update all protocol implementations to support this.

Update file logging logic to account for having files in transactions. Instead
of it acting separately on file containers, it is now tied into the
transaction logging.

Update the filestore keyword to consider a match if filestore output not
enabled.
 3 years ago
Victor Julien 01e64d80da app-layer: trunc parser per direction 3 years ago
Victor Julien ff9d1807f9 app-layer: parser flags to u16 3 years ago
Victor Julien c27df6304d app-layer: introduce common AppLayerStateData API 
Add per state structure for storing flags and other variables.
 3 years ago
Victor Julien 879a46f085 rust: lock to time 0.3.13 to avoid MSRV bump to 1.59 
Indirect dependency through x509-parser.
 3 years ago
Pierre Chifflier 16db04c1a7 rust: remove nom 5 dependency 3 years ago
Pierre Chifflier 0acf75bff7 rust/applayertemplate: convert to nom7 3 years ago
Pierre Chifflier 378e915846 rust/asn1: convert parsers to nom7 3 years ago
Pierre Chifflier 0ba0572c4a rust/x509: finish transition to nom7 3 years ago
Pierre Chifflier 3ef5121ab0 rust/telnet: convert parsers to nom7 3 years ago
Pierre Chifflier d98b386f36 rust/conf: convert parser to nom7 3 years ago
Pierre Chifflier db9a1e17b6 rust/ssh: finish transition to nom7 3 years ago
Pierre Chifflier b31c72c06a rust/rdp: convert parsers to nom7 3 years ago
Pierre Chifflier 49520b2143 rust/rdp: upgrade dependency on tls-parser 3 years ago
Pierre Chifflier beadd090b8 rust: upgrade versions of BER/DER, Kerberos and SNMP parsers 3 years ago
Jason Ish baab1de735 rust: update x509-parser to 0.14.0 
Resolves RustSec issues in time and chrono:
- https://rustsec.org/advisories/RUSTSEC-2020-0071
- https://rustsec.org/advisories/RUSTSEC-2020-0159

Ticket: #5259.

Ammended by Victor Julien to bump to 0.14 instead of 0.13.
 3 years ago
Pierre Chifflier 3aace49649 rust/x509: update dependency on x509-parser 3 years ago
Jeff Lucovsky ab4d0f7f4a detect/stream_size: Rename detect.rs to stream_size.rs 
This commit renames detect.rs to stream_size.rs to reflect its content.
 3 years ago
Jeff Lucovsky 2386f2614f detect/iprep: Move iprep logic into a separate module 3 years ago
Jeff Lucovsky c957882d1c detect/uri: Move uri logic into a separate module 3 years ago
Jeff Lucovsky 484c34bc60 detect/uint: Move uint logic into a separate module 
This commit moves the uint logic into its own module.
 3 years ago
Jeff Lucovsky 8bfe427a74 rust/detect: Create detect module for rule parsing 
This commit creates a module named "detect" for rule parsing logic. As
part of this commit, detect.rs is moved from its toplevel position into
the new module. Thus, use crate::detect::detect to refer to items within
detect.rs (instead of create::detect).

Ticket: 5077
 3 years ago
Jeff Lucovsky ccd1063e43 detect/bytemath: convert parser to Rust 
Issue: 5077

This commit
- Converts the PCRE based parser to Rust.
- Adds unit tests to the new Rust modules
- Removes the PCRE parser from detect-bytemath.c
- Adjusts the C source modules to refer to the Rust definitions
- Includes the multiply operator (missing from the C parser)
 3 years ago
Philippe Antoine af40873127 pgsql: config limit maximum number of live transactions 
As is done for other protocols

Ticket: #5527
 3 years ago
Philippe Antoine e160917bcf mqtt: remove quadratic time complexity 
When having many transactions in a single parsing call...

Fix has overhead of having one more field in the mqtt state.

Completes commit a8079dc978

Ticket: #5399
 3 years ago
Philippe Antoine 5ef259722b dhcp: adds renewal-time keyword 
Ticket: #5507
 3 years ago
Philippe Antoine 6faf6299e0 dhcp: adds rebinding-time keyword 
Ticket: #5506
 3 years ago
Philippe Antoine 95f0424423 nfs4: fix write record handling 
Ticket: #5280
 3 years ago
Philippe Antoine bf43011a43 dcerpc: convert transaction list to vecdeque for UDP 
As was done for TCP in dfe76bb90 and d745d28d4

Ticket: #5518
 3 years ago
Eric Leblond 2cc9152fc9 rust/smb: log uuid of interface in dcerpc 
When doing a DCERPC request, we can use the context id to log the
interface that is used. Doing that we can see in one single event
what is the DCERPC interface and opnum that are used. This allows
to have all the information needed to resolve the request to a
function call.

Feature #5413.
 3 years ago
Eric Leblond b6f1cf255c rust/smb/dcerpc: parse context id 
As context id is used to know to which variant of the endpoint the
request is done, it is interesting to parse it.

Feature #5413.
 3 years ago
Philippe Antoine 11f849c3ee protocol-change: sets event in case of failure 
Protocol change can fail if one protocol change is already
occuring.

Ticket: #5509
 3 years ago
Philippe Antoine e94920b49f smb: do not use tree id to match create request and response 
As an SMB2 async response does not have a tree id, even if
the request has it.

Per spec, MessageId should be enough to identifiy a message request
and response uniquely across all messages that are sent on the same
SMB2 Protocol transport connection.
So, the tree id is redundant anyways.

Ticket: #5508
 3 years ago
Philippe Antoine 461725a9bf dhcp: adds leasetime keyword 
As it is logged

Ticket: #5435
 3 years ago
Philippe Antoine 9b4a133777 http2: remove to_vec for comparisons 
Ticket: #5454
 3 years ago
Philippe Antoine d011b468da http2: fix clippy warning about &Vec<u8> 
Using &[u8] instead in function prototype
 3 years ago
Philippe Antoine 3de735ae70 ike: log ikev1 tx fields instead of state ones 
As state fields can grow abitrarily, and this can lead to DOS
by quadratic complexity (CPU time and disk space)

Adds a direction field to retain all the information in the
transaction.

Also checks array vendor_ids had at least one element before
logging it.

Ticket: #5455
 3 years ago
Philippe Antoine d0171d7418 ike: rustfmt 3 years ago
Philippe Antoine 5c7b5c5fb5 krb: detection for ticket encryption 
As is done for logging.

Ticket: #5442
 3 years ago
Philippe Antoine 64b2385c64 krb: log for ticket encryption 
Also logs if the ticket encryption is weak.
It is different from the encryption used for the rest of the
packet, and this allows to detect kerberoasting attack.

Ticket: #5442
 3 years ago
Philippe Antoine 7fcc6696cb krb: rustfmt kerberos.rs 3 years ago
Philippe Antoine 675de33405 krb: bump up crate version 
kerberos parser crate is also used by other procotols : nfs and
smb. These protocols use an older der_parser crate version.
Upgrading der_parser will simplify the code further.
 3 years ago
Philippe Antoine 783dff2c38 krb: rustfmt detect.rs 3 years ago
Jason Ish c862e84c01 rust/frames: cleanups 
- Implement the Display trait on Direction to print "toserver" or
  "toclient" which used in a format string.

- Use Direction struct inside Frame instead of a u32.  Requires a helper
  method as there are two representation in C for direction, and the C
  methods for frames don't use the internal representation of the
  Direction enum (some sweeping changes could help here)
 3 years ago
Jason Ish f92708b8ca rust/frames: derive direction from StreamSlice 
On the Rust side, a Frame requires a StreamSlice to be created. We can
derive the direction from the StreamSlice removing the need for callers
to provide the direction when operating on the frame.
 3 years ago
Jason Ish b39d7f46e7 dns/tests: fix StreamSlice to satisfy debug validation 3 years ago
Philippe Antoine f3b6fd3329 quic: update to nom7 3 years ago
Philippe Antoine 95125811b8 quic: reassemble crypto frames and parse it 3 years ago
Philippe Antoine f242fb7f22 quic: events and rules on them 3 years ago
Philippe Antoine b9c1d9e86b quic: parse gquic version Q039 
Ticket: #5166
 3 years ago
Philippe Antoine 018fef5ef8 quic: ja3 computation and logging and detection 
Logging as is done in TLS.

Detection using the generic generic ja3.string keyword

Ticket: #5143
 3 years ago
Philippe Antoine c6cf61a39b quic: complete parsing of initial for non gquic 
The format of initial packet for quic ietf, ie quic v1,
is described in rfc 9000, section 17.2.2

Parse more frames and logs interesting extensions from crypto frame

Do not try to parse encrypted data, ie after we have seen
a crypto frame in each direction.

Use sni from crypto frame with tls for detection already implemented

Ticket: #4967
 3 years ago
Philippe Antoine 7044131c39 quic: rustfmt 3 years ago
Philippe Antoine 0c346af4a9 rust: bump up digest crates 
so that we can use hkdf crate for quic
 3 years ago
Philippe Antoine 2294e9cdbc rdp: bump up tls-parser crate version 
so that we can use new functions in quic parser
 3 years ago
Philippe Antoine 11e0eb9c89 quic: do not log empty cyu array 
Ticket: #5167
 3 years ago
Philippe Antoine 632581ac95 ike: do not log empty notify array 
Ticket: #5167
 3 years ago
Philippe Antoine 262a93ce18 mqtt: do not log reason_codes if there is none 
Ticket: #5167
 3 years ago
Philippe Antoine 1621f5e453 detect/nfs: use inclusive ranges 3 years ago
Philippe Antoine ed6955ee98 detect: use generic integer functions for iprep 
Ticket: #4112
 3 years ago
Philippe Antoine cfb60d0fce detect: use generic integer functions for urilen 
Ticket: #4112
 3 years ago
Philippe Antoine c57052181c snmp: rustfmt detect.rs 3 years ago
Philippe Antoine c7214be99b snmp: adds usm keyword 
as is logged

Ticker: #5416
 3 years ago
Philippe Antoine eb1c2a6083 smb: use default stream-depth 0 by default 
As broken by commit e5c948df87

Ticket: #5390
 3 years ago
Philippe Antoine c585be338c nfs: fix arbitrary allocation 
Bug introduced by https://github.com/OISF/suricata/pull/7111

Nom's count begins by allocating a Vector, which leads to arbitrary
allocation due to flavors_cnt coming from network, and not even
being checked against i.len()

Ticket: #5237
 3 years ago
Philippe Antoine 26dc70648c dns: remove unused events field from state 
found overflowing by oss-fuzz
 3 years ago
Philippe Antoine d1a4dae36b detect: use generic integer functions for streamsize 
By the way, adds the prefilter feature

Ticket: #2697
Ticket: #4112
 3 years ago
Philippe Antoine 35b6dcec7e detect: use generic integer functions for filesize 
Ticket: #4112
 3 years ago
Philippe Antoine f29b43defd detect: rust generic functions for integers 
Move it away from http2 to generic core crate.
And use it for DCERPC (and SMB)

And remove the C version.
Main change in API is the free function is not free itself, but
a rust wrapper around unbox.

Ticket: #4112
 3 years ago
Philippe Antoine c4d9cb02ec util: better hex print function 
Without dangerous snprintf pattern identified by CodeQL
even if this pattern is not a problem in those precise cases,
it may easily get copy pasted in a dangerous place, so better
get rid of it and make CodeQL happy
 3 years ago
Philippe Antoine 6058792bee rust: make suricata context const 
So that it is read only and its pointers do not get modified
 3 years ago
Philippe Antoine 6224e283fa modbus: bump up rust crate version 
So that probing parser is more strict and does not accept unknown
function code as valid modbus.

Ticket: #5377
 3 years ago
Philippe Antoine 2d761810db rust: cbindgen first verifies existing bindings 
So as not to recompile every C file inclusing rust.h
 3 years ago
Juliana Fajardini 6ccc01a79c rust: fix doc comments that trigger rust warnings 
Rust generates warnings that are treated as errors for documentation
blocks before `extern` blocks.
 3 years ago
Philippe Antoine d745d28d4a dcerpc: use vecdeque tx iterator 
Ticket: #5321
 3 years ago
Jason Ish dfe76bb905 dcerpc: convert transaction list to vecdeque 
Allows for more efficient removal from front of the list.

Ticket: #5271
 3 years ago
Jason Ish 8790968281 mqtt, rdp: fix copyright dates 3 years ago
Philippe Antoine c78722a671 rust: RustParser same fields as AppLayerParser 
So that there is no problem when crossing FFI
 3 years ago
Sam Muhammed 323fe1c1ac nfs3/records: Fix typo 
Fix response_lookup unittest name
 3 years ago
Jason Ish b8b6a17a5b dns: add pdu frame 
Adds a PDU frame to the DNS parser. For UDP this is the DNS payload
portion of the DNS packet, for TCP this is the payload minus the leading
legth field.

Ticket: 4984
 3 years ago
Jason Ish 8d1840f595 frames(rust): don't call into C if running Rust unit tests 
Wrap the calls behind frames to C code if a `cfg!(not(test))` so they
don't get compiled when running Rust unit tests.  Linkage to C functions
is not yet available for Rust unit tests, and this will keep the check
out of individual parsers.

Ticket: 4984
 3 years ago
Jason Ish c74ea3840d frames (rust): method to create StreamSlice from slice 
Useful in unit test for function that require a StreamSlice.
 3 years ago
Jason Ish d712a8b29d eve/dns: remove dns v1 logging 
Removal of DNS v1 logging was scheduled to be removed in May 2022.

Ticket: #4157
 3 years ago
Jason Ish e319d31c14 template(rust): convert transaction list to vecdeque 
Allows for more efficient removal from front of the list.

Ticket: #5298
 3 years ago
Jason Ish 9b0b2beac1 pgsql: convert transaction list to vecdeque 
Allows for more efficient removal from front of the list.

Ticket: #5297
 3 years ago
Jason Ish 2db84726ad http2: convert transaction list to vecdeque 
Allows for more efficient removal from front of the list.

Ticket: #5296
 3 years ago
Jason Ish 4e0ad5e0bd rdp: convert transaction list to vecdeque 
Allows for more efficient removal from front of the list.

Ticket: #5295
 3 years ago
Jason Ish 3b422e25f3 mqtt: convert transaction list to vecdeque 
Allows for more efficient removal from front.

Ticket: #5294
 3 years ago
Jason Ish 3189414788 dns: convert transaction list to vecdeque 
Allows for more efficient removal from front of the list.

Ticket: #5277
 3 years ago
Jason Ish 7b11b4d3a1 app-layer: more generic state trait 
Instead of a method that is required to return a slice of transactions,
use 2 methods, one to return the number of transactions in the
collection, and another to get a transaction by its index in the
collection.

This allows for the transaction collection to not be a contiguous array
and instead can be a VecDeque, or possibly another collection type that
supports retrieval by index.

Ticket #5278
 3 years ago
Juliana Fajardini bbd9a2ff1a pgsql: apply clippy fixes 3 years ago
Juliana Fajardini 0fc9ade7a9 pgsql: fix uint overflow and optimizations 
Fuzzers found a possible integer overflow bug when parsing response
messages. To fix that, removed the case where we incremented the parsed
field length and created a new message type for situations where Suri
parsers an Unknown message. This is good because there may happen that
an unknown message to Suri is valid, and in this case, we would still be
able to log it.

Philippe Antoine found the bug while fuzzing with rust debug assertions.

Bug #5016
 3 years ago
Juliana Fajardini 8daa79513a pgsql: fix clippy is_null usage warning 3 years ago
Philippe Antoine d2f00ac824 dcerpc: use wrappingadd for padding parsing 
As we compute a modulo, we can safely wrap around even if there
is an overflow

Ticket: #5301
 3 years ago
Philippe Antoine 464ff80c6a smb: protocol detection on pattern without midstream 
To recognize a protocol, Suricata first looks for
patterns, which can be confirmed by a probing parser.
If this does not work, Suricata can try to run
some probing parsers on some ports.

This is the case for SMB.

This commit makes handling the confirming and the probing
paser differently even if they share much code.

The confirmation parser knows that a pattern has been found.
So, it must not do the midstream case of looking for this
pattern in the whole buffer, but only check it at the beginning.
But it must reverse direction if needed.
 3 years ago
Victor Julien dc57460427 smb: fix event types for limit exceeded rules 3 years ago
Jason Ish 0623ada24d dns: better error handling when parsing names 
The DNS name parser will error out with an error even if the
error is incomplete. Instead of manually generating errors,
use '?' to let the nom error ripple up the error handling chain.

The reason this wasn't done in the first place is this code
predates the ? operator, or we were not aware of it at the time.

This prevents the case where probing fails when there is enough data to
parse the header, but not enough to complete name parser. In such a case
a parse error is returned (instead of incomplete) resulting in the
payload not being detected as DNS.

Ticket #5034
 3 years ago
Jason Ish 27679a12aa dns: don't parse a full request during probe if not enough data 
If there is more data than a header, but not enough for a complete DNS
message, the hostname parser could return an error causing the probe to
fail on valid DNS messages.

So only parse the complete message if we have enough input data. This is
reliable for TCP as DNS messages are prefixed, but for UDP its just
going to be the size of the input buffer presented to the parser, so
incomplete could still happen.

Ticket #5034
 3 years ago
Victor Julien fc9b65d8d3 smb2: validate negotiate read/write max sizes 
Raise event if they exceed the configured limit.
 3 years ago
Victor Julien 4be8334c9e smb2: allow limiting in-flight data size/cnt 
Allow limiting in-flight out or order data chunks per size or count.

Implemented for read and writes separately:

app-layer.protocols.smb.max-write-queue-size
app-layer.protocols.smb.max-write-queue-cnt
app-layer.protocols.smb.max-read-queue-size
app-layer.protocols.smb.max-read-queue-cnt
 3 years ago
Victor Julien 2c5ad8858e filetracker: track total queued data (in_flight) 
As well as expose number of chunks.
 3 years ago
Victor Julien 90d4b8e438 smb: log max read/write sizes 3 years ago
Victor Julien 5bcc4162f7 smb2: add options for max read/write size 
Add options for the max read/write size accepted by the parser.
 3 years ago
Victor Julien f28888513a smb2: track max read/write size and enforce its values 3 years ago
Victor Julien 594acec5dc smb: minor function cleanup 
Remove used argument from `filetracker_newchunk()`. We're not
using fill_bytes with smb.
 3 years ago
Victor Julien c7a474c725 filetracker: make FileChunk private 3 years ago
Philippe Antoine 3b13008c1b mqtt: fix consumed bytes computation for truncated msg 
Ticket: 5268
 3 years ago
Philippe Antoine 704bc878ea dcerpc: store consumed_bytes as i32 
As it can grow bigger than u16
 3 years ago
Philippe Antoine dfd17e9acc ike: fix integer underflow in parse_proposal 
By not restricting a usize to i16
 3 years ago
Victor Julien 93d5bce0aa rust: update regex & memchr dependencies 
Bug: #5260.
 3 years ago
Victor Julien 053a9d2e68 smb/ntlmssp: add stricter len/offset validation 3 years ago
Philippe Antoine 3e48881b78 smb: prevents integer underflow 
Ticket: 5246

If msg_id is 0, we cannot find the previous request
 3 years ago
Philippe Antoine e72036f12f smb: ntlmssp domain_blob_offset underflow check 
Ticket: 5246
 3 years ago
Philippe Antoine 817a5001a5 smb: check on param parsing 
Ticket: 5246

so as not to overflow u16
 3 years ago
Sascha Steinbiss 7eb279ac53 mqtt: remove redundant "where" keyword 3 years ago
Sascha Steinbiss d63e5b8c51 mqtt: make some functions non-public 3 years ago
Sascha Steinbiss 2a3ed9a6ae mqtt: rustfmt 3 years ago
Sascha Steinbiss 1ba62993d5 mqtt: raise event on parse error 3 years ago
Sascha Steinbiss 5618273ef4 mqtt: ensure we do not request extra data after buffering 
This addresses Redmine bug #5018 by ensuring that the parser
never requests additional data via the Incomplete error, but to
raise an actual parse error, since it is supposed to have all
the data as specified by the message length in the header already.
 3 years ago
Victor Julien 6d30f4442c http2: fix file accounting for ranged files 
Increment files_opened for tx that 'gets' reassembled ranged file
 3 years ago
Victor Julien b336882008 smb1: apply close to direction 
Instead of closing files in both direction when receiving a close request,
close only toserver files for the request and close toclient on receiving
a response.
 3 years ago
Victor Julien b9cd502249 smb: convert 'close' parser to function 3 years ago
Sam Muhammed 3a490fb16c nfs: Implement frames 
Feature #4872

Frames:
  - RPC Frames: Generic over TCP/UDP
     - rpc.pdu
     - rpc.hdr
     - rpc.data
     - rpc.creds -- for rpc calls

  - NFSv2, NFSv3
     - nfs.pdu
     - nfs.status -- for nfs responses

  - NFSv4 Only Frames
     - nfs4.pdu
     - nfs4.hdr
     - nfs4.ops -- for compound request/response operations
     - nfs4.status -- for nfs4 responses

RPC tcp/udp frames created with separate registeration functions e.g:
add_rpc_tcp_tc_frames()
add_rpc_udp_tc_frames()
 3 years ago
Sam Muhammed d090dcbce9 rpc: Improve rpc_record struct 
Add creds_len field to rpc_record
needed for rpc.creds frame length calculation
 3 years ago
Sam Muhammed 8064a5348d rust/nfs4: Add NFSPROC4_DESTROY_CLIENTID op parsers 3 years ago
Sam Muhammed 9d1fad28a7 rust/nfs4: Add NFSPROC4_DESTROY_SESSION op parsers 
Also add respective request unittest
test_nfs4_request_destroy_session()
 3 years ago
Sam Muhammed ff81cad4f1 rust/nfs4: Add NFSPROC4_LAYOUTRETURN op parsers 
Also add respective request unittest
test_nfs4_request_layoutreturn()
 3 years ago
Sam Muhammed 073244a0b8 rust/nfs4: Add NFSPROC4_GETDEVINFO op parsers 
Also add respective response/request unittests
test_nfs4_response_getdevinfo()
test_nfs4_request_getdevinfo()
 3 years ago
Sam Muhammed ff54a6d9d5 rust/nfs4: Add NFSPROC4_LAYOUTGET op parsers 
Also add respective response/request unittests
test_nfs4_response_layoutget()
test_nfs4_request_layoutget()
 3 years ago
Sam Muhammed 3d542fcc67 rust/nfs4: Add NFSPROC4_SECINFO_NO_NAME op parsers 3 years ago
Sam Muhammed b35d635ac7 rust/nfs4: Add NFSPROC4_RECLAIM_COMPLETE op parsers 3 years ago
Sam Muhammed 2a41b46eca rust/nfs4: Add NFSPROC4_CREATE_SESSION op parsers 
Also add respective response/request unittests
test_nfs4_request_create_session()
test_nfs4_response_create_session()
 3 years ago
Sam Muhammed 0a69c66153 rust/nfs4: Add NFSPROC4_EXCHANGEID response parser 
Also add test_nfs4_response_exchangeid() unittest
 3 years ago
Sam Muhammed fe7a49b737 rust/nfs4: improve NFSPROC4_OPEN op parser 
Improve nfs4_res_open() parser to reflect other file-delegation types
Reflect the changes on test_nfs4_response_open() unittest
 3 years ago
Jason Ish 2341f47755 smb: handle records in the wrong direction 
If an SMB record is seen in the wrong direction, set an event on the PDU
frame and don't process the record in the state.

No error is returned, so the next record will be processed.
 3 years ago
Jason Ish 09e2d3b216 smb: expose smb1 request/reply flags with a method 
Adds `.is_request()` and `.is_reply()` to check if a SMB record flags
say the message is a request or a reply.
 3 years ago
Jason Ish 7b659489c8 smb: fix smb2 header flag parsing 
The bits were being parsed in the order they're displayed in Wireshark,
rather than the order they were being seen on the wire, resulting in
direction and async being 0 more often than they should be.

Instead of bits, take the 4 bytes as an le_u32 and just use bit masks to
extract what we need into a struct, I think its easier to reason about
this way when comparing to the Microsoft documentation.
 3 years ago
Philippe Antoine bfcd6cb46a range: validity check when end is bigger than size 
Ticket: 5132

Down the line, HttpRangeOpenFileAux assumes the range has a
valid value when doing buflen = end - start + 1;
 3 years ago
Victor Julien 07b1100713 nfs: clean up partial record handling 
There should be no remaining data after parsing the partial
RPC record, so don't handle it but instead add a debug validation
bug on.

Successful processing for NFSv3 read/write records returns
AppLayerResult::ok() directly as all data is consumed.
 3 years ago
Victor Julien d85b77cad0 nfs3: improve read validation; fix partial handling 3 years ago
Victor Julien 4418fc1b02 nfs3: fix partial write record handling 3 years ago
Victor Julien 5baf94e40d nfs3: enforce more values 
Enforce values of a number of u32's that are used as bools or for
really low values.
 3 years ago
Victor Julien 1c57e3c18d rpc: enforce various field values 
Minimal frag_len. Correct msgtype and others.
 3 years ago
Victor Julien 64d8a1e16e nfs/rpc: update full record parsers to be more exact 
Instead of 'take'ing all data for the RPC prog_data and then
letting the higher level parsers figure out which part to use
take the exact amount.
 3 years ago
Victor Julien bfb5ae867e nfs: break out partial record handling 3 years ago
Victor Julien fe76ab1803 nfs/rpc: enforce length field limits 
Limits based on the Linux kernel limits. Then multiplied a few times
to allow for other implementations to have higher limits.
 3 years ago
Victor Julien 5ecb626e50 nfs4: verify bool fields 3 years ago
Jason Ish b1c09369af rust/derive: pin proc-macro-crate to v1.1.0. 
The just released proc-macro-crate v1.1.2 requires at least Rust 1.53.
Pin to the previous release for now.
 3 years ago
Pierre Chifflier b8f767d84c rust/mime: convert parser to nom7 3 years ago
Juliana Fajardini 5a7645fac1 rust: add comment tags to support documentation 
With these, the portion of code within the tags should be included
in the related code-snippets (for frame support documentation) w/o
errors, even if the code within changes. The tags can also work as
a reminder that the existing code is being shown elsewhere, so folks
know documentation might need updates, in case of major changes.
 3 years ago
Juliana Fajardini e0dd1820c2 sip: apply rustfmt to a few functions 
Our current rust code isn't always documentation friendly when it
comes to using code snippets. Used rustfmt to apply rust default
formatting on functions that we wanted to show in our documentation
for Frame support
 3 years ago
Juliana Fajardini 71cbd2bf0e telnet: apply rustfmt to parse_request 
When we want to share our code in our documentation pages, the current
rust formatting isn't so nice to read. Formatted just the portion of
the code that will be shown, for now.
 3 years ago
Philippe Antoine 8adf172ab8 nfs: limits the number of active transactions per flow 
Ticket: 4530
 3 years ago
Philippe Antoine 0e85dea3ff nfs: remove unused events variable 3 years ago
Philippe Antoine e4f2f8f78d nfs: derive AppLayerEvent for NFSEvent 3 years ago
Jason Ish 2ebb525f7e build: remove configure check for cargo vendor 
cargo vendor has been part of the core cargo command since Rust 1.37,
and are minimum Rust version is not 1.41, so remove the check. Its
always available now.
 3 years ago
Jason Ish 62cc813f88 rust/make: fix maintainer-clean-local target 
Was using the wrong name, so vendored Rust crates were not being
cleaned up on make maintainer-clean.
 3 years ago
Pierre Chifflier 8dc3431d86 rust/dcerpc: convert parser to nom7 functions 3 years ago
Pierre Chifflier b5166bdb93 rust/ntp: upgrade dependency on ntp-parser 4 years ago
Pierre Chifflier fa63945bdc rust/ike: convert parser to nom7 functions and upgrade dependency 4 years ago
Pierre Chifflier 3493537ec3 rust/rfb: convert parser to nom7 functions 4 years ago
Victor Julien 474e0e3644 sip: enable for 5061/udp 4 years ago
Victor Julien 1203750388 sip: add frames support 
Frames:
- sip.pdu
- sip.request_line
- sip.response_line
- sip.request_headers
- sip.response_headers
- sip.request_body
- sip.response_body

The `sip.pdu` frame is always created, the rest only if the record
parser succeeded.

Ticket: #5036.
 4 years ago
Philippe Antoine e42094f238 mqtt: make max transactions configurable 
Allows users to find balance between completeness of decoding
and increases resource consumption, which can DOS suricata.
 4 years ago
Philippe Antoine 4f90d4254e http2: makes some settings configurable 
max-streams and max-table-size

Allows users to find balance between completeness of decoding
and increases resource consumption, which can DOS suricata.
 4 years ago
Philippe Antoine a8079dc978 mqtt: limits the number of active transactions per flow 
Ticket: 4530

So, that we do not get DOS by quadratic complexity, while
looking for a new pkt_id over the ever growing list
of active transactions
 4 years ago
Philippe Antoine 5475212f21 http2: limits the number of active transactions per flow 
Ticket: 4530

So, that we do not get DOS by quadratic complexity, while
looking for a new stream id over the ever growing list
of active streams
 4 years ago
Philippe Antoine f0e869b26b mqtt: parse properties with the right buffer's length 4 years ago
Philippe Antoine df2cbd6517 http2: event for variable-length integer overflow 
http2_parse_var_uint can overflow the variable-length
integer it is decoding. In this case, it now returns an error
of kind LengthValue.

The new function http2_parse_headers_blocks, which factorizes
the code loop for headers, push promise, and continuation, will
check for this specific error, and instead of erroring itself,
will return the list of so far parsed headers, plus another one
with HTTP2HeaderDecodeStatus::HTTP2HeaderDecodeIntegerOverflow

This status is then checked by process_headers to create an
app-layer event.
 4 years ago
Philippe Antoine b86beb9b68 http2: check overflow before it happens 
instead of checking afterwards if value got smaller
 4 years ago
Eloy Pérez González bff0774767 smb/dce_iface: avoid deleting current ifaces from state 
The smb dce_iface keyword must match for all those dcerpc requests
and responses sent in the context of the given interface. They are
not matching as the current bind interfaces are deleted by any
non bind message.

Ticket: 4767
 4 years ago
Eloy Pérez González 1ae22fd5de smb/dce_iface: use DCERPC_TYPE_REQUEST 
The smb dce_iface keyword must match for all those dcerpc requests and
responses sent in the context of the given interface. They are not
matching because in rs_smb_tx_get_dce_iface, x.req_cmd is erroneously
compared with 1. Fix this by comparing with DCERPC_TYPE_REQUEST instead.

Ticket: 4767
 4 years ago
Eloy Pérez González 333db3b385 smb/dce_opnum: move range if to outer context 
The smb dce_opnum matches all the opnums that are higher that the
indicated opnum. This is due the range comparison if was put in the
exact comparison context, and in case the opnum doesn't match exactly,
then the range comparison is triggered (the upper limit is always true).

Move the erroneus if to the outer context, as else option of the block
checks if comparison should be exact or range.

Ticket: 4767
 4 years ago
Eloy Pérez González 8dca3d0416 smb/dce_opnum: use DCERPC_TYPE_REQUEST 
The smb dce_opnum keyword doesn't match the dcerpc requests/responses.
This occurs because in the rs_smb_tx_match_dce_opnum function, the
x.req_cmd is matched against the erroneous code 1. Fix this by using
DCERPC_TYPE_REQUEST for the comparison instead.

Ticket: 4767
 4 years ago
Eloy Pérez González 15f493f516 dcerpc: remove prev_tx_call_id and clear_bind_cache from DCERPCState 
remove those fields since are not used because of the removal of
handle_bind_cache.
 4 years ago
Eloy Pérez González 1aca2676a6 dcerpc: avoid delete the rpc state interface context 
The bug:
The dcerpc dce_iface keyword just match the packet following the bind. Only the
next request after the rpc is sent will match. However the expected behaviour it
that all the rpc requests/responses sent under the context of the given
interface would match.

In the Open Group c706 the following is indicated:

In 2.2.1 Binding-related Operations, indicates that one category of binding
operations are those that "operations that establish internal call routing
information for the server." (The other are to establish the protocol which is
not relevant here). And the following statement can be found:

Operations in the second category establish a set of mappings that the server
can use to route calls internally to the appropriate manager routine. This
routing is based on the interface and version, operation and any object
requested by the call.

It indicates that server routes (to call methods) are based on the operation,
interface and object.

- Operation: To indicate the method to call, and operation number is
             specified as indicated in the second step of 2.3.3.2 (Client
             Binding Steps).
- Interface: An interface is a set of remotely callable operations offered by a
             server and invokable by clients. (2.1.1.1)
- Object: Is the manager that implements the interface, as stated in section
          Interface and Manager Selection of 2.3.3.3. It is not mandatory, can
          be nil.

To call a method, a client must send a request message as defined in 2.6.4.9,
that contains these identifiers:

- opnum: The opnum field identifies the operation being invoked within the
         interface.
- p_cont_id (Context ID in Wireshark): The p_cont_id field holds a presentation
                                       context identifier that identifies the
                                       data representation and interface, as
                                       defined in 12.6.3.4 (Context Identifiers).
- object: The object field is contained if the PFC_OBJECT_UUID is set. (Could be
          interesting to create a keyword dce_object for matching this UUID)

Therefore, to get the correct method to invoke, the server must map the context
to the correct interface. This is negotiated by the bind request

Interfaces are first negotiated using the bind message (12.6.4.3), contained in
the p_context_elem array. Then they are accepted or rejected using the bind_ack
message (12.6.4.4).

Once these contexts are established, both client and server can use the context
id, which is the index of the p_context_elem array, to refer the interface they
are using.

Moreover, in the middle of the connection, the context can be changed with the
alter_context message.

This is way suricata shouldn't delete the bindack attribute, that contains
the contexts, used by match_backuuid. This is the only way to know the interface
a request message is referring to.

ticket: 4769
https://redmine.openinfosecfoundation.org/issues/4769
 4 years ago
Juliana Fajardini 0bf1227f0f pgsql: fix defect found by coverity 
Pgsql was using bitwise operations to assign password output config to
its context flags, but mixing that with logic negation of the default
value, resulting in the expressions having a constant value as result.

Bug: #5007
 4 years ago
Pierre Chifflier ce9efc34c7 rust/pgsql: convert parsers to nom7 functions 4 years ago
Juliana Fajardini 579d7dcc01 pgsql: add initial support 
- add nom parsers for decoding most messages from StartupPhase and
SimpleQuery subprotocols
- add unittests
- tests/fuzz: add pgsql to confyaml

Feature: #4241
 4 years ago
Juliana Fajardini 4c743b809c rust/applayer: add function for upgrading to TLS 4 years ago
Victor Julien e02b52c895 quic: add quic.ua for matching user agent 4 years ago
Victor Julien 4c13b73c4d quic: log user agent when available 4 years ago
Victor Julien da8b024b99 detect/quic: add quic.sni sticky buffer 4 years ago
Victor Julien 7b836af1b2 quic: log sni; reduce number of transactions 
Only create transactions for long headers.

Store SNI in tx, log it.
 4 years ago
Victor Julien ccab28aad3 quic: log version as string 
Log as Q043, Q044, Q045, Q046. If the version is not supported/recognized,
log the 4 bytes as hex.

Only log for txs based on long headers.
 4 years ago
Victor Julien 24a21af4ab quic: redo quic.version; parser cleanups 
Reimplement quic.version as sticky buffer.

Removed unused parts of the parser.

Set unidirectional tx flag to fix double matching.
 4 years ago
Emmanuel Thompson 7e51987263 quic: Add QUIC App Layer 
Parses quic and logs a CYU hash for gquic frames
 4 years ago
Sam Muhammed c4bd3cd70e nfs4_records: add unittests 
Add unittests for setclientid, readdir records
Task #4866
 4 years ago
Pierre Chifflier c61cbd9b35 rust: simplify bits parser annotations 4 years ago
Pierre Chifflier 1b25bcbb13 rust/smb: simplify bits parser annotations 4 years ago
Pierre Chifflier 4d6aa6d532 rust: add 'bits' combinator to simplify nom bits parsers 
Add a specialized version of the 'bits' nom combinator so adding
bits-level parsers does not require type annotations.
 4 years ago
Victor Julien 44c9241b6a telnet: initial support with frames 
Bootstrapped using setup script. Basic option parsing for purpose
of tagging frames.
 4 years ago
Jason Ish 3cdefd5f8b smb: use derive AppLayerFrameType 4 years ago
Victor Julien 0c9fdf8f4f smb: implement frames 
SMB1 record parsing code simplification.

Frames:

    nbss.pdu
    nbss.hdr
    nbss.data

    smb1.pdu
    smb1.hdr
    smb1.data

    smb2.pdu
    smb2.hdr
    smb2.data

    smb3.pdu
    smb3.hdr
    smb3.data

The smb* frames are created for valid SMB records.
 4 years ago
Jason Ish 8a40b7b42e cbindgen: ignore frames module 4 years ago
Jason Ish de870e2fbf rust: derive macro for app-layer frame type 4 years ago
Jason Ish 0ece208074 rust/applayer: create trait for app-layer frame types 4 years ago
Jason Ish cb7f7a7e08 app/frames: implement rust API 4 years ago
Victor Julien e6f49e5a05 app/frames: implement name to id API for frames 4 years ago
Pierre Chifflier 3e19ccdc0c rust/http2: convert parser to nom7 functions (HTTP2 ranges) 4 years ago
Pierre Chifflier f8647b0ffb rust/http2: convert parser to nom7 functions (HTTP2 core functions) 4 years ago
Philippe Antoine e1c0725e05 doc: fix typo lenght/length 4 years ago
Sam Muhammed 9bea850d53 nfs4_records: add unittests for nom7 parsers 
Task #4866
 4 years ago
Sam Muhammed 463fbdc36d nfs4_records: add missing field to res_sequence_ok() 
Missing _seqid in sequence op struct left a trailing four zeros
that are parsed by nfs4_res_compound_command() as a cmd
causing a Switch Error Code
 4 years ago
Sam Muhammed 4e2edd44aa nfs3-records: add unittests to nom7 parsers 
Task #4866
 4 years ago
Sam Muhammed 03906010a2 nfs3-records: add missing fields and update parsers 
Add missing fields to some record structures and
update their respective parsers
 4 years ago
Sam Muhammed 86c273dadc nfs2-records: add unittests for nom7 parsers 
Task #4866
 4 years ago
Victor Julien ddf14e51dc nfs2: improve READ parsing 
Take fill_bytes into account.
 4 years ago
Jason Ish 2011a5579c rust/app-layer: expose AppLayerEvent derive macro 
Export the AppLayerEvent derive macro so plugin (or library code) can
use it as expected, for example:

use suricata::applayer::AppLayerEvent;

enum MyEvent {
    EventOne,
    EventTwo,
}
 4 years ago
Jason Ish ba310440a6 rust/derive: make usable from a plugin or lib user 
The macro was generating code that references names use the "crate"
prefix which will fail if the macro is used by a library user or plugin.
Dynamically check where we are running an use the correct import paths
as needed.
 4 years ago
Jason Ish bbd5e6402b rust: rename to suricata (from suricata_rust) 
Rename the Rust lib to simply "suricata" instead of "suricata_rust".
This allows Rust plugin/library code to use it under the name "suricata"
which is what should be expected.

The name was only "suricata_rust" to prevent on-disk conflict with the C
code, so just rename the file on disk, which doesn't affect how the code
is interacted with from an API layer.
 4 years ago
Jason Ish 2ffe88c1f0 rust: remove feature function-macro 
The function macro existed so it would only be enabled on Rust
versions that supported.  Now that our MSRV is 1.41, which is
greater than 1.38 we can assume we always have support for
this macro.
 4 years ago
Jason Ish 50fdcd098c rust/http2: use base64 crate for base64 decode 4 years ago
Jason Ish 6392216f6b base64: use the Rust base64 encode implementation 
Replace our internal base64 implementation with a ffi wrapper
around the Rust implementation provided by an external crate.
 4 years ago
Jason Ish 8181030f72 jsonbuilder: add methods to encode values as base64 
Add new methods to set a value as a base64 encoded string of
a byte array. This uses the Rust base64 crate and encodes
directly into the JsonBuilder buffer with no intermediate
buffer required.

jb_set_base64: set a field on an object
jb_append_base64: append a value to an array
 4 years ago
Victor Julien c073d5cfbf app-layer: use StreamSlice as input to parsers 
Remove input, input_len and flags in favor of stream slice.
 4 years ago
Victor Julien 6466296b32 app-layer: add StreamSlice to pass data to parsers 
Since object to contain relevant pointer, length, offset, flags to make
it easy to pass these to the parsers.
 4 years ago
Jason Ish 0861b66e15 dns: add dns flag to dns request logging 
Ticket #4515
 4 years ago
Odin Jenseg dfb6f105e8 dns: Logging of Z-bit 
[Edit by Jason Ish: fix flag bit value]

Ticket #4515
 4 years ago
Jason Ish fcbdc30426 dns: create transaction even if z-bit was set 
It appears that DNS servers will still process a DNS request even if the
z-bit is set, our parser will fail the transaction. So create the
transaction, but still set the event.

Ticket #4924
 4 years ago
Pierre Chifflier d67f8f9196 rust/smb: convert parser to nom7 functions (SMB1) 4 years ago
Pierre Chifflier 895a54cea4 rust/smb: convert parser to nom7 functions (DCERPC records) 4 years ago
Pierre Chifflier 8d77ce1ffc rust/smb: convert parser to nom7 functions (SMB2) 4 years ago
Pierre Chifflier 5cadb878ff rust/smb: convert parser to nom7 functions (SMB3) 4 years ago
Pierre Chifflier 4c97dfa851 rust/smb: convert parser to nom7 functions (NTLM/SSP records) 4 years ago
Pierre Chifflier 3da816eb23 rust/smb: convert parser to nom7 functions (NBSS records) 4 years ago
Pierre Chifflier 90f9450971 rust: add nom7 combinator take_until_and_consume 4 years ago
Philippe Antoine 87d9c44ec5 rust: export constants via cbindgen 
so that constants are not defined twice in Rust anc C
So that we are sure they have the same value
 4 years ago
Philippe Antoine 784558df2e mime: handles multiple sections for a parameter 
Ticket: 4386

as per RFC2231.
For instance filename can be split between filename*0,
filename*1, etc...
 4 years ago
Philippe Antoine 8feb9c35ae mime: move FindMimeHeaderTokenRestrict to rust 
Also fixes the case where the token name is present
in a value
 4 years ago
Philippe Antoine 1b10848d84 mqtt: fix transaction completion 
Ticket: 4862

A transaction to client is always considered
complete in the direction to server and vice versa.
Otherwise, transactions are never complete for
AppLayerParserTransactionsCleanup
 4 years ago
Jason Ish 7732efbec2 app-layer: include decoder events in app-layer tx data 
As most parsers use an events structure we can include it in the
tx_data structure to reduce some boilerplate/housekeeping code
in app-layer parsers.
 4 years ago
Philippe Antoine 0caaf6bd23 range: prevents memory leak of file from HTTP2 
Ticket: 4811
Completes commit c023116857

state.free should also close files with ranges
as state.free_tx did already

And file_range field should be reset so that there is no
use after free.
 4 years ago
Jeff Lucovsky 23faeaea5c ftp: Remove diagnostic print 
This commit removes a diagnostic message displayed during extraction of
the EPSV port.
 4 years ago
Philippe Antoine c023116857 range: prevents memory leak of file from HTTP2 
If a HTTP2 transaction gets freed before the end of the range
request, we need to have the files container which is in
the state, to transfer owernship of this file to the files
container.

Ticket: 4811
 4 years ago
Pierre Chifflier acb3ec6db1 rust/nfs: convert parser to nom7 functions (NFS v2 records) 4 years ago
Pierre Chifflier ea1d03f8e3 rust/nfs: add a maximum number of operations per compound 
The `count` combinator preallocates a number of bytes. Since the value
is untrusted, this can result in an Out Of Memory allocation.
Use a maximum value, large enough to cover all current implementations.
 4 years ago
Pierre Chifflier 0ffe123330 rust/nfs: convert parser to nom7 functions (NFS v3 and v4 records) 4 years ago
Jason Ish eb6cc62937 dhcp: fix url in comment 
rustdoc was complaining about the format of the URL in a comment
while trying to generate documentation. Convert the comment to a
non-rustdoc comment for now to satisfy rustdoc.
 4 years ago
Jason Ish b57280ff48 rdp: fix transaction id 
By our convention the transaction ID is incremented then applied
to the new transaction. And the generic transaction iterator
requires this behaviour.
 4 years ago
Jason Ish 9c67c634c1 app-layer: include DetectEngineState in AppLayerTxData 
Every transaction has an existing mandatory field, tx_data. As
DetectEngineState is also mandatory, include it in tx_data.

This allows us to remove the boilerplate every app-layer has
for managing detect engine state.
 4 years ago
Jason Ish f4b4d531b0 rdp: add tx iterator 4 years ago
Jason Ish 238ec953b7 krb5: use tx iterator 4 years ago
Jason Ish ef0c351953 ntp: add tx iterator 4 years ago
Jason Ish 871fb035b4 sip: add tx iterator 4 years ago
Jason Ish d6b2d7e16a ike: add tx iterator 
For IKE the manual iterator functions were there, but never
registered. So this commit does add a tx iterator to ike.
 4 years ago
Jason Ish 3f2d2bc12b snmp: use generic tx iterator 4 years ago
Jason Ish ac4c5ada2f dhcp: use generic tx iterator 4 years ago
Jason Ish 54e62ddf71 http2: use generic tx iterator 4 years ago
Jason Ish 6cffecfe3e template: use generic tx iterator 4 years ago
Jason Ish a936755731 nfs: use generic tx iterator 4 years ago
Jason Ish 0188a01daf rfb: use generic tx iterator 4 years ago
Jason Ish b335409690 mqtt: use generic tx iterator 4 years ago
Jason Ish d71bcd82d9 modbus: use generic tx iterator 4 years ago
Jason Ish fcfc9876ce smb: use generic tx iterator 4 years ago
Jason Ish 049d43212e rust/app-layer: provide generic implementation of iterator 
Create traits for app-layer State and Transaction that allow
a generic implementation of a transaction iterator that parser
can use when the follow the common pattern for iterating
transactions.

Also convert DNS to use the generic for testing purposes.
 4 years ago
Shivani Bhardwaj 26c7d3cc35 http2: remove needless borrows 4 years ago
Shivani Bhardwaj f3a1e3b92e core: Remove unneeded consts 4 years ago
Shivani Bhardwaj b5a123adb1 ssh: use Direction enum 4 years ago
Shivani Bhardwaj baf30cfc05 snmp: use Direction enum 4 years ago
Shivani Bhardwaj 89cb337930 smb: use Direction enum 4 years ago
Shivani Bhardwaj 8f9f78c2d0 sip: use Direction enum 4 years ago
Shivani Bhardwaj 11c438a07d nfs: use Direction enum 4 years ago
Shivani Bhardwaj a7ac79bed7 mqtt: use Direction enum 4 years ago
Shivani Bhardwaj 209e2f17fa krb: use Direction enum 4 years ago
Shivani Bhardwaj 243960a511 ike: use Direction enum 4 years ago
Shivani Bhardwaj ee5b300ccf http2: use Direction enum 4 years ago
Shivani Bhardwaj 0c6e9ac931 files: use Direction enum 4 years ago
Shivani Bhardwaj a19d2b4e1e dns: use Direction enum 4 years ago
Shivani Bhardwaj a866499bca dcerpc: use Direction enum 4 years ago
Shivani Bhardwaj 9512bfd729 core: add Direction enum 
Ticket: 3832
 4 years ago
Philippe Antoine 3212fa7d2b ntp: fixes leak of de_state 
Bug: #4752.
 4 years ago
Philippe Antoine 28a3181a2d snmp: fixes leak of de_state 
Bug: #4752.
 4 years ago
Philippe Antoine f37240a3e2 smb: midstream probing checks for netbios message type 
If it is available

Bug: #4620.
 4 years ago
Philippe Antoine 8f8823b6f2 rust: right condition for both uint to be zero 
Theay can overflow leading to their addition to be zero

If a NFS read reply indicates a count of 0xFFFFFFFF

Bug: #4680.
 4 years ago
Pierre Chifflier ce652511bd rust/tftp: convert parser to nom7 functions 4 years ago
Pierre Chifflier c525a1337c rust/dns: convert parser to nom7 functions 4 years ago
Pierre Chifflier 74be8b94ec rust/ssh: convert parser to nom7 functions 4 years ago
Pierre Chifflier 8a584c211e rust/mqtt: convert parser to nom7 functions 4 years ago
Pierre Chifflier d27125d77a rust/sip: convert parser to nom7 functions 4 years ago
Pierre Chifflier 1046a7d1a3 rust/ftp: convert parser to nom7 functions 4 years ago
Pierre Chifflier ebd5883da8 rust/dhcp: convert parser to nom7 functions 4 years ago
Pierre Chifflier 17170c41aa rust: add nom7 dependency 4 years ago
Modupe Falodun a87c7e5c08 rust: remove unnecessary nested match 
Bug: #4605
 4 years ago
Modupe Falodun 74c39500c3 rust: fix inherent to string 
Bug: OISF#4618
 4 years ago
Sam Muhammed 922a453da5 rust(lint): use is_null() instead of ptr::null_mut() 
Bug: #4594
 4 years ago
Sam Muhammed 23768c7181 rust(lint): use is_null() instead of ptr::null() 
Bug: #4594
 4 years ago
Sam Muhammed da0a976e23 rust(lint): use let for binding single value 
`match` is better used with binding to multiple variables,
for binding to a single value, `let` statement is recommended.

Bug: #4616
 4 years ago
Philippe Antoine 5bd065cb3c range: checks that end is after start for HTTP2 
As was done only for HTTP1 in previous commit

The verification part stays separated from the parsing part,
as we want to keep on logging invalid ranges values.
 4 years ago
Philippe Antoine accdad7881 ike: do not keep server transforms in state 
Fixes #4534

Now, only the tx with the transforms will match
with ike.chosen_sa_attribute
 4 years ago
Philippe Antoine 83887510a8 modbus: tx iterator 
When there are a lot of open transactions, as is possible with
modbus, the default tx_iterator will loop for the whole
transacations vector to find each transaction, that means
quadratic complexity.

Reusing the tx_iterator from the template, and keeping as a state
the last index where to start looking avoids this quadratic
complexity.
 4 years ago
Philippe Antoine ea4a509a54 app-layer: disable by default if not in configuration 
DNP3, ENIP, HTTP2 and Modbus are supposed to be disabled
by default. That means the default configuration does it,
but that also means that, if they are not in suricata.yaml,
the protocol should stay disabled.
 4 years ago
Philippe Antoine 8e8899c90c http2: range: check return value when opening 
HttpRangeContainerOpenFile can return NULL
so, http2_range_open can set file_range to NULL
And we should check this before calling http2_range_close
 4 years ago
Philippe Antoine cb30772372 style: remove latest warnings 
about unused variables
 4 years ago
Philippe Antoine 98f84d5a9b http2: follow range requests 
Move the content-range parsing code to rust
 4 years ago
Philippe Antoine 56fae072b2 http2: better rust lifetimes 
so that borrow check gets happy
 4 years ago
Philippe Antoine a1f9e0c97a rust: rename to StreamingBufferConfig as in C 4 years ago
Shivani Bhardwaj 42da0fb5c5 smb: fix broken stream depth setting 
The stream depth setting was broken since it was moved to Rust because
of a missing parser for memory values in configuration.
Use get_memval fn from conf.rs to calculate and fetch the correct
values.
 4 years ago
Shivani Bhardwaj 0cfe512ef0 rust/conf: add getter for memval 
Add a parser for memory values like 50kb, 20mb, etc on the Rust side.
 4 years ago
Shivani Bhardwaj f3fcc39738 ssh: remove futile default port setting 4 years ago
Shivani Bhardwaj 1f48714e75 smb: remove futile default port setting 4 years ago
Shivani Bhardwaj 13741540ce rfb: remove futile default port setting 4 years ago
Shivani Bhardwaj 7c9d573800 nfs: remove futile default port setting 4 years ago
Shivani Bhardwaj f4f6387a00 dcerpc: use null for default ports 4 years ago
Philippe Antoine 596a4a9d6e http2: better rust style 4 years ago
Philippe Antoine 48ed874dda http2: concatenate one headers multiple values 
For detection, as is done with HTTP1
 4 years ago
Philippe Antoine e3ff0e7731 http2: generic http2_header_blocks 
so as not to forget continuation and push promise
when iterating over headers
 4 years ago
Philippe Antoine 0b0649d98e http2: http.header keyword now works for HTTP2 
As well as http.header.raw
 4 years ago
Philippe Antoine 9b9f909d7d http2: http.header_names keyword now works for HTTP2 4 years ago
Philippe Antoine 547e9f4ab4 http2: http.host normalized keyword now works for HTTP2 4 years ago
Philippe Antoine 75f75e1eb0 http2: turn Host header into authority during upgrade 
HTTP1 uses Host, but HTTP2 uses rather :authority cf HPACK
 4 years ago
Philippe Antoine bb98a18b3d http2: better file tracking 
If an HTTP2 file was within only ont DATA frame, the filetracker
would open it and close it in the same call, preventing the
firther call to incr_files_opened

Also includes rustfmt again for all HTTP2 files
 4 years ago
Philippe Antoine 1378b2f451 http2: support deflate decompression 
cf #4556
 4 years ago
Victor Julien c9cee7af49 smb: add debug validation on file counts 4 years ago
Victor Julien 114d3ba730 smb: count files in tx 4 years ago
Victor Julien c1dfb619c4 http2: support per-tx file accounting 4 years ago
Victor Julien 1b3c3225cd nfs: add debug validation on file counts 4 years ago
Victor Julien 1d48601c25 nfs: support per-tx file accounting 4 years ago
Victor Julien 67759795c6 nfs: don't reuse file transactions 
After a file has been closed (CLOSE, COMMIT command or EOF/SYNC part of
READ/WRITE data block) mark it as such so that new file commands on that
file do not reuse the transaction.

When a file transfer is completed it will be flagged as such and not be
found anymore by the NFSState::get_file_tx_by_handle() method. This forces
a new transaction to be created.
 4 years ago
Victor Julien 56d3e28a3a filestore: track files getting stored per tx 
Avoid evicting a tx before the filedata logger has decided it is
done.
 4 years ago
Victor Julien c78f5ac316 app-layer/transactions: track files opens and logs 
To make sure a transaction is not evicted before all file logging is complete.
 4 years ago
Philippe Antoine 9b8be5a650 smb: get file name in case of chained commands 4 years ago
Philippe Antoine 3e5f59e2cb smb: fix parsing of file deletion over SMB1 4 years ago
Philippe Antoine fde753d9d2 smb: recognizes file deletion over SMB2 
using set_info_level == SMB2_FILE_DISPOSITION_INFO
 4 years ago
Jason Ish 71679c6ad0 ike: use derive macro from app-layer events 4 years ago
Jason Ish eb55297876 modbus: use derive macro from app-layer events 4 years ago
Jason Ish d3bd008e33 app-layer template: use derived app-layer event 4 years ago
Jason Ish cef2832dcf http2: use derived app-layer event 4 years ago
Jason Ish e92cb36bb8 krb5: use derived app-layer event 4 years ago
Jason Ish 92561837f8 ntp: use derived app-layer event 4 years ago
Jason Ish 1f71fb2cde rfb: register None for get_event_info/get_event_info_by_id 
Implementations are not required if they're just going to return
-1. We allow None to be registered for that.
 4 years ago
Jason Ish 4fd6aa866f sip: use derived app-layer event 4 years ago
Jason Ish 18448f6ed6 snmp: use derived app-layer event 4 years ago
Jason Ish bb094b17db ssh: use derived app-layer event 4 years ago
Jason Ish 9c3f06d9b5 dhcp: use derived app-layer event 4 years ago
Jason Ish b9f10ba22f smb: use derived get_event_info/get_event_info_by_id 4 years ago
Jason Ish 8eac5fc221 mqtt: derive AppLayerEvent for MQTTEvent 4 years ago
Jason Ish 6ed827a4ef dns: use derive macro for DNSEvent 4 years ago
Jason Ish 9221f1d9d5 applayerevent: derive get_event_info and get_event_info_by_id 
Add generation of wrapper functions for get_event_info
and get_event_info_by_id to the derive macro. Eliminates
the need for the wrapper method to be created by the parser
author.
 4 years ago
Jason Ish 0fa7b5c2a2 rust/applayer: provide generic event info functions 
Provide generic functions for get_event_info and
get_event_info_by_id. These functions can be used by any app-layer
event enum that implements AppLayerEvent.

Unfortunately the parser registration cannot use these functions
directly as generic functions cannot be #[no_mangle]. So they
do need small extern "C" wrappers around them.
 4 years ago
Jason Ish 27d1ee98ce rust: derive crate: for custom derives 
Currently has one derive, AppLayerEvent to be used like:

  #[derive(AppLayerEvent)]
  pub enum DNSEvent {
      MalformedData,
      NotRequest,
      NotResponse,
      ZFlagSet,
  }

Code will be generated to:
- Convert enum to a c type string
- Convert string to enum variant
- Convert id to enum variant
 4 years ago
Jason Ish dbea7d636f rust/applayer: define AppLayerEvent trait 
The derive macro will implement this trait for app-layer
event enums.
 4 years ago
Jason Ish cf21694ba6 rust(lint): suppress clippy lints that we should fix 
Suppress all remaining clippy lints that we trip. This can be
fixed on a per-lint basis.
 4 years ago
Jason Ish 91402f9fba rust(lint): remove manual implement of map method 
Using `if let` expressions in these cases is better expressed
by the map method, and considered idiomatic Rust for this usage.
 4 years ago
Jason Ish b021726a0d rust(lint): map the error instead of using or_else 
This is the preffered style and easier to understand the meaning
of the code.
 4 years ago
Jason Ish dcf57ecd96 rust(lint): replace push_str of single char to push(<char>) 4 years ago
Jason Ish d5c0962299 rust(lint): fix some usages of references 
- ref is discouraged for top level variables
- the other borrow is not required
 4 years ago
Jason Ish d0772e04b1 rust(lint): replace checked_mul with saturating_mul 
When defaulting checked_mul to u64::max, Rust has a method
that does the same thing called saturating_mul.
 4 years ago
Jason Ish d0be7541e9 rust(lint): removed unused unit () return 
This is code that is not needed and is a bit confusing to see.
 4 years ago
Jason Ish 4abbfd0d97 rust(lint): remove extra parens around bitwise or 
This is a readability fix, as on first look they almost look
like a Rust tuple.
 4 years ago
Jason Ish ac3a20b6e0 rust(lint): remove useless conversions and clones 
These add complexity and may not be optimized out by the compiler.
 4 years ago
Jason Ish 8bb6dab69d rust(lint): remove useless format calls 
In these simple cases to_string() is recommended and likely
performs better as the formatter is not called.
 4 years ago
Jason Ish 5bf5de3350 rust(lint): don't use unwrap_or for function calls 
Calling a function in unwrap_or causes that function to always
be called even when not needed. Instead use unwrap_or_else with
a closure which will only be called when needed.
 4 years ago
Jason Ish 602bb05e75 rust(lint): fix redundant closures 
This lint checks for a closure where a function can be directly
supplied.  Runtime performance is unchanged, but this makes
less work for the compiler.
 4 years ago
Jason Ish 69cf5c9eea rust(lint): remove needless borrows 
These are needless borrows (references) as the item is already
a reference.
 4 years ago
Jason Ish 363b5f99c3 rust: functions that reference raw pointers are unsafe 
Based on the Rust clippy lint that recommends that any public
function that dereferences a raw pointer, mark all FFI functions
that reference raw pointers with build_slice and cast_pointer
as unsafe.

This commits starts by removing the unsafe wrapper inside
the build_slice and cast_pointer macros then marks all
functions that use these macros as unsafe.

Then fix all not_unsafe_ptr_arg_deref warnings from clippy.

Fixes clippy lint:
https://rust-lang.github.io/rust-clippy/master/index.html#not_unsafe_ptr_arg_deref
 4 years ago
Jason Ish 53413f2d7a rust: remove all usage of transmute 
All cases of our transmute can be replaced with more idiomatic
solutions and do no require the power of transmute.

When returning an object to C for life-time management, use
Box::into_raw to convert the boxed object to pointer and use
Box::from_raw to convert back.

For cases where we're just returning a pointer to Rust managed
data, use a cast.
 4 years ago
Victor Julien 9d24a53c53 nfs: minor code cleanup 4 years ago
Victor Julien aa9d8658ef smb: minor formatting fixup 4 years ago
Victor Julien 094208823b smb: minor code cleanup 4 years ago
Shivani Bhardwaj 0a1747c1ba nfs: fix comment 4 years ago
Shivani Bhardwaj 58ac9b0f38 nfs: Add rust registration function 
Get rid of the C glue code and move registration completely to Rust.
 4 years ago
Shivani Bhardwaj 61fca4e9db nfs: add missing code from rust impl of fns 4 years ago
Shivani Bhardwaj de50ac631e nfs: Change fn sign as per rust registration requirement 
Registering parsers in Rust requires signatures to be a certain way and
compatible with C. Change signatures of all the functions.
Probe fn has also been changed to return AppProto as required by the new
fn signature.
 4 years ago
Shivani Bhardwaj e5c948df87 smb: Add rust registration function 
Get rid of the C glue code and move registration completely to Rust.
 4 years ago
Shivani Bhardwaj 27af4bb002 smb: add missing code from rust impl of fns 4 years ago
Shivani Bhardwaj 6420df84b7 smb: Change fn sign as per rust registration requirement 
Registering parsers in Rust requires signatures to be a certain way and
compatible with C. Change signatures of all the functions.
 4 years ago
Shivani Bhardwaj 4d6b6b5dfe smb: add constants 4 years ago
Shivani Bhardwaj d1ea00521b rust/core: Add flow flags 4 years ago
Jason Ish 222e55847c flow: provide flags accessor function 
Add an accessor function for flow flags. To be used by Rust where
the flow struct is an opaque data type.
 4 years ago
Shivani Bhardwaj cb8bd8c669 rust/applayer: add more externs 4 years ago
Philippe Antoine 31dccd1171 modbus: do not claim to handle gaps 4 years ago
Sascha Steinbiss d541b3d4a8 rust: fix warnings with nightly 4 years ago
Philippe Antoine 9e7ea631b2 dns: improve probing parser 
Checks opcode is valid
Checks additional_rr do not exceed message length
Better logic for incomplete cases
 4 years ago
Philippe Antoine 6f03ee2e47 dcerpc: handles bigger inputs than 2^16 
By comparing integers with the largest size
 4 years ago
Philippe Antoine 7d0a39412b detect: use u32 for InspectionBufferMultipleForList 
So that we do not have an endless loop casting index to
u16 and having more than 65536 buffers in one transaction

Changes for all protocols, even ones where it is impossible
to have such a pattern, so as to avoid bad pattern copy/paste
in the future
 4 years ago
Philippe Antoine b3c1f2ab48 nfs: improve probing parser 
Checks credentials flavor is known
 4 years ago
Philippe Antoine 39575e2cc9 modbus: use ascii character classes while parsin rule 
As the rust regex crate is unicode aware, which was
not the case of the C version
 4 years ago
Philippe Antoine ef5755338f rust: SCLogDebug is real nop when built as release 
Before, even if there were no outputs, all the arguments
were evaluated, which could turn expensive

All variables which are used only in certain build configurations
are now prefixed by underscore to avoid warnings
 4 years ago
Victor Julien 20e8f90981 http2: set Debug on structs 4 years ago
Victor Julien 3587033d9e files: construct with default, free on drop 
Update protocols.
 4 years ago
Victor Julien d757545f03 files: implement default support 4 years ago
Philippe Antoine fdab22d924 rust: fix app-layer parser flags 
This especially allows for SSH bypass to work
 4 years ago
Jeff Lucovsky aa8871a5be rust/default: Enable Default usage 4 years ago
Jeff Lucovsky 6028ca7827 nfs: Rework constructs to use Self/Default 4 years ago
Jeff Lucovsky aafb0a60b7 dhcp: Rework constructs to use Self/Default 4 years ago
Jeff Lucovsky 1ef0bd580b dcerpc: Rework constructs to use Self/Default 4 years ago
Jeff Lucovsky 00167121dc dns: Rework constructs to use Self/Default 4 years ago
Jeff Lucovsky 02dccb1529 smb: Rework constructs to use Self/Default 
This commit modifies the constructors to use Self and/or
Default::default() when feasible
 4 years ago
Jeff Lucovsky f502f21f9e rust/default: Enable Default usage in SMB 4 years ago
Shivani Bhardwaj 581cb6223d dcerpc/udp: Add rust registration function 
Get rid of the C glue code and move registration completely to Rust.
 4 years ago
Shivani Bhardwaj d7007424bd dcerpc/udp: Change fn sign as per rust registration requirement 
Registering parsers in Rust requires signatures to be a certain way and
compatible with C. Change signatures of all the functions.
 4 years ago
Shivani Bhardwaj bac69af7e4 dcerpc: Add rust registration function 
Get rid of the C glue code and move registration completely to Rust.
 4 years ago
Shivani Bhardwaj a0a09a102b dcerpc: Change fn sign as per rust registration requirement 
Registering parsers in Rust requires signatures to be a certain way and
compatible with C. Change signatures of all the functions.
 4 years ago
Shivani Bhardwaj dee972b863 rust/core: Make AppProto type u16 4 years ago
Shivani Bhardwaj d66ad96f0d applayer/rust: add extern AppLayerProtoDetectPMRegisterPatternCSwPP 4 years ago
Victor Julien d7c3ecb6f9 http2: remove dead code 4 years ago
Philippe Antoine b3c502d572 http2: remove assertion which can be wrong 
Brotli decoder stops consuming input it it reaches the
end of its input
 4 years ago
Simon Dugas c2720fc2fb modbus: fix quantity and count calculation 
The [Modbus Spec S6.11](https://modbus.org/docs/Modbus_Application_Protocol_V1_1b.pdf)
clearly states that the `count = quantity / 8` and not the other way
around. This is fixed in sawp-0.5.0.
 4 years ago
Philippe Antoine 999327ba1f http2: http.cookie keyword now works for HTTP2 4 years ago
Philippe Antoine df039555bc http2: http.host.raw keyword now works for HTTP2 4 years ago
Philippe Antoine 1e82d0b3c8 http2: http.method keyword now works for HTTP2 4 years ago
Philippe Antoine 017e39d8fd http2: makes all HTTP1 header keywords work 4 years ago
Simon Dugas a8a51dc004 modbus: add eve logging 4 years ago
Simon Dugas 8342641477 modbus: move tests from c to rust 
Move tests in a seperate commit so that we can use the previous one for
regression testing. This also gets rid of the temporary glue that made
the C tests work with the rust implementation.
 4 years ago
Simon Dugas a458a94dca modbus: move from C to rust 
Adds a new rust modbus app layer parser and detection module.

Moves the C module to rust but leaves the test cases in place to
regression test the new rust module.
 4 years ago
frank honza f83d51d0cb ike: set event for multiple server proposals 4 years ago
Philippe Antoine ca6e434e0b ftp: completely parses pasv and epsv responses 4 years ago
Philippe Antoine 6fe8bce3b0 http2: have filecontainer for both directions 4 years ago
Jason Ish c7f44447c9 dns: remove flood protection purging 
It doesn't look like flood protection is required with the
stateless parser anymore. It actually can get in the way of TCP
DNS when a large number of requests end-up in the same segment
where a TX can get purged before it has a chance to go through
the normal TX life-cycle.
 4 years ago
Shivani Bhardwaj c663ac6ddd dcerpc/tcp: improve detection 
Lately, some of the TLS data was misdetected as DCERPC/TCP because of
the pattern |05 00|. Add more checks in DCERPC probe function to ensure
that it is in fact DCERPC/TCP.
 4 years ago
Philippe Antoine d2d0e0adc9 rust: remove exported unused functions 4 years ago
Philippe Antoine 6da9a37285 rdp: correctly returns incomplete in parse_tc 
Adding the already consumed bytes
In case an incomplete tls handshake is handled with/after
a refular rdp t123_tpkt
 4 years ago
Philippe Antoine 3de0123ffb http2: adds check about dynamic headers table size 4 years ago
Philippe Antoine 1ca4f041bb http2: pass data through when decompression fails 
as is done for HTTP1
 4 years ago
Juliana Fajardini c6a35d09b7 templates: fix typos 
- *template*files[ch][rs]: fix typos
- scripts/setup-app-layer: fix typos
 4 years ago
Jason Ish 6853bf98fb dns: only register a single logger 
DNS no longer requires a logger to be registered for to-client and
to-server directions. This has not been required with the stateless
design of the Rust DNS parser.
 4 years ago
Philippe Antoine 0105d4f017 rust: bump bitflags dependency version 
So that lexical-core, needed by nom, and using bitflags
is used with version 0.7.5 instead of version 0.7.0
which fixed the fact that BITS is now a reserved keyword
in nightly version
 4 years ago
Philippe Antoine cb150e97d0 kerberos: fix probing parser tag condition 
according to the comment
 4 years ago
Sascha Steinbiss e2dbdd7fd5 ikev1: add ikev1 parser 4 years ago
frank honza ecdf9f6b0b ikev1: rename ikev2 to common ike 
Renaming was done with shell commands, git mv for moving the files and content like
find -iname '*.c' | xargs sed -i 's/ikev1/ike/g' respecting the different mixes of upper/lower case.
 4 years ago
frank honza ab6171c429 detect: added support for protocol-aliases 4 years ago
frank honza e9494ddd8f util: add function converting u8-array into a hex-String 4 years ago
Philippe Antoine 8307010255 smb: relax probing parser to handle first NBSS message 
cf dcerpc-udp S-V test :
First message is Message Type: Session request (0x81)
Second message is SMB
 4 years ago
Philippe Antoine 1b6e81cd72 smb: probing parser for start and midstream 
The probing parser is more strict at the start of the stream
 4 years ago
Philippe Antoine 9dc5258a21 smb: split probing function for code style 
Introduces rs_smb_probe_tcp_midstream
 4 years ago
Shivani Bhardwaj 0ac5c5376a dcerpc: trigger raw assembly on record completion 4 years ago
Shivani Bhardwaj c77c8e7005 rust/context: add AppLayerParserTriggerRawStreamReassembly 4 years ago
Philippe Antoine 2d14606224 smb: andx support 
Add AndX support for SMB1. Finishes #3475.

[Updated by Victor Julien to split functions]
 4 years ago
Shivani Bhardwaj f967a49104 dcerpc/udp: improve detection 
Lately, Wireguard proto starting w pattern |04 00| is misdetected as
DCERPC/UDP which also starts with the same pattern, add more checks
to make sure that it is the best guess for packet to be dcerpc/udp.
 4 years ago
Shivani Bhardwaj 3641f1b522 dcerpc: add probe function 4 years ago
Shivani Bhardwaj d7a3523b12 rust/applayer: split EOF flag per direction 4 years ago
Shivani Bhardwaj 0ca8591994 dcerpc/udp: remove transmute 
The book defines transmute as "This is really, truly, the most horribly unsafe
thing you can do in Rust. The guardrails here are dental floss."
Transmute can result into mind boggling undefined behaviors. Get rid of
it wherever possible.
 4 years ago
Philippe Antoine c6aadf0dfa protodetect: rename direction to flags 
And use whole flags in AppLayerProtoDetectPPGetProto
 4 years ago
Philippe Antoine 5465e0b154 http2: http.stat_msg keyword now works for HTTP2 4 years ago
Philippe Antoine 47928babfc http2: http.user_agent keyword now works for HTTP2 4 years ago
Philippe Antoine a98d0fe6ed http2: http.uri keyword now works for HTTP2 
cf #4067
 4 years ago
Philippe Antoine 707f027231 protos: renaming ALPROTO_HTTP* constants 
Having now ALPROTO_HTTP1, ALPROTO_HTTP2 and ALPROTO_HTTP

Run with 3 sed commands
git grep ALPROTO_HTTP | cut -d: -f1 | uniq |
 xargs sed -i -e 's/ALPROTO_HTTP/ALPROTO_HTTP1/g'
git grep ALPROTO_HTTP12 | cut -d: -f1 | uniq |
 xargs sed -i -e 's/ALPROTO_HTTP12/ALPROTO_HTTP2/g'
git grep ALPROTO_HTTP1_ANY | cut -d: -f1 | uniq |
 xargs sed -i -e 's/ALPROTO_HTTP1_ANY/ALPROTO_HTTP/g'

and then running clang-format
 4 years ago
Jason Ish dbae17dbc0 install: makefile target to install libraries 
As we don't install the libraries by default, provide a make target,
"install-library" to install the libsuricata library files.

If shared library support exists, both the static and shared
libraries will be installed, otherwise only the static libraries
will be installed.
 4 years ago
Jason Ish a178ec6bef rust: rename lib to libsuricata_rust 
Previously it was libsuricata.a, but eventually we want to get
to a place where libsuricata.a is a combination of the Rust
and C code.
 4 years ago
Jason Ish 9f20297cb3 rust/Makefile: add Cargo.toml as make dependency 
This will force Cargo.toml to be recreated if Cargo.toml.in
is modified.
 4 years ago
Jason Ish 3ada5e1480 rust/ffi: provide AppLayerRegisterParser in context 
AppLayerRegisterParser was creating a link error when attempting
to use a convenience library for the Suricata C code, then linking
the library of C code with the library of Rust code into a final
Suricata executable, or use with fuzz targets.

By moving AppLayerRegisterParser to the context structure and
calling it like a callback the circular reference is removed
allowing the convenience libraries to work again.

This is also a stepping block to proving a Suricata library
as a single .a or .so file.
 5 years ago
Philippe Antoine 32b604e8c7 template: use response_gap in rust parser 5 years ago
Victor Julien 3f807f3bf6 rust: update dependencies 5 years ago
Victor Julien ebde15f0e2 rust: lock all major crate versions 
To avoid surprises with dependencies bumping MSRV.
 5 years ago
Victor Julien 4b5af36061 rust: relax nom version to any >=5.1.1 5 years ago
Philippe Antoine d861228214 http2: decompression for files 
gzip and brotli decompression for files
 5 years ago
Philippe Antoine 2e46b5d100 rust: BIT_U16 macro utility 5 years ago
Philippe Antoine aee8e60149 rust: better panic message for missing file config 5 years ago
Philippe Antoine 8ac363c34d rust: fix warning about unused values in smb tests 5 years ago
Jason Ish 0a3b9e0220 rust/hashing: add function to finalize md5 to hex string 
New function, SCMd5FinalizeToHex to finalize an md5 hash
to a hex string.
 5 years ago
Jason Ish 26c1321668 rust/hashing: function to md5 hash buffer to hex 
Add function SCMd5HashBufferToHex to hash a single buffer to an
md5 hex string.
 5 years ago
Jason Ish 0f714be9f3 rust/hashing: method to SHA256 and finalize in one call 
Add SCSha256HashBuffer to hash a single buffer returning the
result.
 5 years ago
Jason Ish eb5cfd9b82 rust/hashing: new function to SHA1 hash a single buffer 
SCSha1HashBuffer will has a single buffer and compute the digest
in one call.
 5 years ago
Jason Ish ff37526c6b rust: remove md5 crate, unalias md-5 
Remove the md5 crate as a dependency as it is no longer removed.
We now use md-5 from RustCrypto.
 5 years ago
Jason Ish 0a2d8509c9 rust/ssh: use md-5 crate instead of md5 
The "md-5" crate is part of the RustCrypto project that also
uses the sha1 and sha256 crates we are using. These all implement
the Digest trait for a common API.
 5 years ago
Jason Ish 1022b217ed rust/hashing: a method to md5 hash a single buffer 
Add SCMd5HashBuffer as a replacement for NSS HASH_HashBuf as
used in ja3 to hash a single buffer.
 5 years ago
Jason Ish 46ceb13c08 rust: add ffi module for sha256, sha1 and md5 
Add a Rust module that exposes Rust implementations of
sha256, sha1 and md5 from the RustCrypto project.

This is an experiment in replacing the libnss hash functions with
pure Rust versions that will allow us to remove nss as a compile
time option.

Initial tests are good, even with a 10% or so performance
improvement when being called from C.

Also trying a module naming scheme where modules under the ffi
modules are purely for exports to C, as it doesn't make any
sense to use this new hashing module directly from Rust.
 5 years ago
Jeff Lucovsky 8ce2078a38 tftp: Add test cases 
This commit adds test cases for the TFTP parser.
 5 years ago
Jeff Lucovsky e900b6d265 tftp: Improve parser 
This commit improves TFTP parsing by ensuring the mode and opcode are
valid.
 5 years ago
Jason Ish f77fd0c0cb rust: include file cleanup 
The cbindgen generated header should not include rust.h as
rust.h already includes the generated binding.

Fixup C source code that only pulled the generated include, it
should instead pull in "rust.h" which includes the generated
binding plus other misc. stuff.
 5 years ago
Juliana Fajardini e33bbee9b7 nbss: add parser tests 
Add tests to parse_nbss_record and parse_nbss_record_partial
 5 years ago
Juliana Fajardini 5226ba1c15 Rust: generic files definition 
Issue: Optimization 3825
- filecontainer: add Files structure, to replace/unify SMBFiles,
NFSFiles and HTTP2Files
- smb/files: delete SMBFiles implementation
- smb/smb: replace SMBFiles with Files
- nfs/nfs: delete NFSFiles implementation, replace its former
 declarations with Files' ones
- http2/http2: replace HTTP2Files with Files
- http2/mod: Delete reference to file files.rs
- http2/files: Delete
 5 years ago
Sascha Steinbiss a4556c9427 dns: use rest() for NULL parsing 5 years ago
Sascha Steinbiss 81b206a5f4 dns: add test for SRV 5 years ago
Sascha Steinbiss 08a6f1441a dns: parse SRV records 5 years ago
Antti Tönkyrä f3ad7f14fa dns: add rdata logging for NS rrtype 5 years ago
Simon Dugas 4336a0e739 dns: log rdata for NULL record type 
Logs the rdata for a NULL record type as a printable string.
 5 years ago
Victor Julien efc9a7a398 app-layer: remove callback for completion status 
Since the completion status was a constant for all parsers, remove the
callback logic and instead register the values themselves. This should
avoid a lot of unnecessary callback calls.

Update all parsers to take advantage of this.
 5 years ago
Philippe Antoine f574663352 http2: files inspection API fixes 
uses right transaction id for file tracker
uses FILE_USE_DETECT for good matches with keyword startswith
 5 years ago
Philippe Antoine a63ee5adbd http2: allow multiple size updates in one headers batch 
cf RFC 7541 section 4.2
 5 years ago
Philippe Antoine 6181459086 smb: do not rely on one valid NBSS byte for probing 
Need to have the SMB header so as to validate
 5 years ago
Jason Ish f3c59ef8a6 rust: handle windows naming change from .lib to .a 
Prior to Rust 1.44, Cargo would name static libs with the .lib
extension. 1.44 changes this extension to .a when running under
a GNU environment on Windows like msys to make it more similar
to other unix environments.

Now assume static library name to be the same on Windows and
unix, but rename the .lib if found to still support older
versions of Rust on Windows.
 5 years ago
Victor Julien ba781265a4 dcerpc/udp: fix transaction handling and logging 
Implement missing transaction handling.

Fix logging wrongly casting 'state' to DCERPCState instead of
DCERPCUDPState leading to crashes and malformed output.

Remove unused fields from DCERPCUDPState.
 5 years ago
Philippe Antoine 8db78208f9 rust: fix warnings found by nightly compiler 
warning: getting the inner pointer of a temporary `CString`
this `CString` is deallocated at the end of the statement,
bind it to a variable to extend its lifetime
 5 years ago
Jason Ish 2f81f3fbe9 rust/log: clarify comment in non-debug mode SCLogDebug 5 years ago
Jason Ish a453d28bc6 rust/log: order log macros in descending order 
Readability cleanup.
- error, notice, ... debug
 5 years ago
Jason Ish 411a5d41c1 rust/log: expand macros after checking log level 
Expand macros in the do_log macro after checking the log level
instead of each log macro (ie: SCLogDebug) expanding the macros
then passing off to do_log to have the log level check.

Will eliminate any expense of expanding macros if this log level
does not permit the given message to be logged.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/4114
 5 years ago
Shivani Bhardwaj 269324e84d dcerpc/log: Log fields particular to an RPC version 
Log fields that only are meant to be in a PDU for a particular RPC
version. Since DCERPC/UDP works on RPC version 4 and DCERPC/TCP works on
RPC version 5, there are certain fields that are particular to each
version.
Remove call_id from the logger for UDP.
Add activityuuid and seqnum fields to the logger for UDP.
call_id and (activityuuid + seqnum) fields are used to uniquely pair a
request with response for RPC versions 5 and 4 respectively.
 5 years ago
Ilya Bakhtin 2033f386f9 rust/dcerpc: Remove redundant fields 5 years ago
Ilya Bakhtin 2840a2e064 rust/dcerpc: Make tx_id u64 5 years ago
Ilya Bakhtin 6916b63f09 dcerpc/udp: Fix pairing of request response 
So far, request and response were paired with serial number fields in
the header. This is incorrect. According to
https://pubs.opengroup.org/onlinepubs/9629399/chap12.htm,
"Together, the activity UUID and the sequence number uniquely identify
a remote procedure call."

Hence, add activity uuid and sequence number to the transaction and pair
the request accordingly. Remove incorrect handling of this and fix
tests.
 5 years ago
Ilya Bakhtin e9b21553cc rust/dcerpc: Add UDP flag definitions 5 years ago
Victor Julien 51f4e4d0b7 dcerpc/udp: add missing tx free logic 5 years ago
Victor Julien 9f9c29a14a dcerpc: fix stream flag handling 
Only hardcoded direction flags were passed to the parser, not the
full range.

Handle receiving an EOF flag w/o data.

Bug: #3856
 5 years ago
Victor Julien f31372ad1d dcerpc/tcp: fix compile warning 
warning: variable does not need to be mutable
    --> src/dcerpc/dcerpc.rs:1036:42
     |
1036 |                     let tx = if let Some(mut tx) = self.get_tx_by_call_id(current_call_id, core::STREAM_TOCLIENT) {
     |                                          ----^^
     |                                          |
     |                                          help: remove this `mut`
     |
     = note: `#[warn(unused_mut)]` on by default

warning: variable does not need to be mutable
    --> src/dcerpc/dcerpc.rs:1061:30
     |
1061 |                         Some(mut tx) => {
     |                              ----^^
     |                              |
     |                              help: remove this `mut`
 5 years ago
Victor Julien 67b5295bbc dcerpc/tcp: add missing detect state cleanup 5 years ago
Philippe Antoine 1fd6f5bc61 http2: asymetric sizes for headers tables 
The headers table from client to server
and the one from server to client
may have different maximum sizes
(even if both endpoints have to keep both tables)
 5 years ago
Philippe Antoine aaa69fe3c5 smb: resistance against padding evasions 
Scenario is use of dummy padding in write AndX request
or other similar commands using a data offset.

Parsing skips now these dummy bytes, and generates one event
 5 years ago
Philippe Antoine caa7946888 smb: adds file overlap event against evasions 
Evasion scenario is
- a first dummy write of one byte at offset 0 is done
- the second full write of EICAR at offset 0 is then done
and does not trigger detection

The last write had the final value, and as we cannot "cancel"
the previous write, we set an event which is then transformed into
an app-layer decoder alert
 5 years ago
Philippe Antoine 3e96f96153 rdp: fix incomplete result 
Aggregating the consumed bytes
 5 years ago
Jason Ish 0529a00ffd dhcp: set unidirection transaction flag 5 years ago
Jason Ish 3036ec4db0 ikev2: set unidirection transaction flag 5 years ago
Jason Ish ff674d0cd0 mqtt: set unidirection transaction flag 5 years ago
Jason Ish 54d5f336d4 sip: set unidirection transaction flag 5 years ago
Jason Ish 1d40d0c5f9 rdp: set unidirection transaction flag 5 years ago
Jason Ish f7dee602e9 krb5: set unidirection transaction flag 5 years ago
Jason Ish 984d3c7f20 ntp: set unidirection transaction flag 5 years ago
Jason Ish fc7d59d92f snmp: set unidirectional transaction flag 5 years ago
Philippe Antoine 64fcba228b http2: complete parsing of priority frames 5 years ago
Philippe Antoine c300a859a0 http2: keep track of dynamic headers table size 
And evict entries accordingly to maximum size
 5 years ago
Philippe Antoine 1a21eea0e9 http2: variable size integers decoded everywhere 5 years ago
Philippe Antoine b21acfbf21 http2: StreamIdReuse frame types exceptions 
Also handles better the state so as not to revert from
HTTP2StateHalfClosedClient to HTTP2StateDataServer and not
go to final HTTP2StateClosed
 5 years ago
Philippe Antoine 89573060d9 http2: use variable integer for headers lengths 5 years ago
Shivani Bhardwaj 97c67cd5ce dcerpc: fix gap handling 
This patch addresses issues discovered by redmine ticket 3896. With the
approach of finding latest record, there was a chance that no record was
found at all and consumed + needed became input length.

e.g.
input_len = 1000
input = 01 05 00 02 00 03 a5 56 00 00 .....

There exists no |05 00| identifier in the rest of the record. After
having parsed |05 00|, there was a search for another record with the
leftover data. Current data length at this point would be 997. Since the
identifier was not found in the data, we calculate the consumed bytes at
this point i.e. consumed = current_data.len() - 1 which would be 996.
Needed bytes still stay at a constant of 2. So, consumed + needed = 996
+ 2 = 998 which is lesser than initial input length of 1000 and hence
the assertion fails.

There could be two fixes to this problem.
1. Finding the latest record but making use of the last found record in
   case no new record was found.
2. Always use the earliest record.

This patch takes the approach (2). It also makes sure that the gap and
current direction are the same.
 5 years ago
Philippe Antoine 4f963717f8 fuzz: better configure checks for MSAN building 
More compatible check for rust nightly
Checks for CARGO_BUILD_TARGET
Builds release or debug mode independently
 5 years ago
Victor Julien 5d985c4271 dcerpc: implement tx free function 5 years ago
Victor Julien 8b2886635f dcerpc/tcp: implement trunc logic 
When one side of the connection reaches the STREAM_DEPTH condition the
parser should be aware of this. Otherwise transactions will forever be
waiting for data in that direction.
 5 years ago
Victor Julien 4da0d9bdea applayer/rust: expose truncate callback 5 years ago
Shivani Bhardwaj 301454e9e4 dcerpc: fix datatypes while handling stub data 5 years ago
Shivani Bhardwaj 3fd9a3d420 dcerpc: fix datatype for stub data len 5 years ago
Philippe Antoine 222b386102 rust: rebuilds std when building fuzzers 
so as to have MSAN working
 5 years ago
Philippe Antoine 15447cc672 dceprc: signature rust check with is_char_boundary 
before calling split_at which would panic
 5 years ago
Philippe Antoine 6694737fcf http2: settings from http1 upgrade 5 years ago
Philippe Antoine 7011bddf84 http2: mimic HTTP1 request from upgrade 5 years ago
Philippe Antoine 9d1b030ff0 http2: first connection upgrade from http1 5 years ago
Philippe Antoine 547d6c2d78 applayer: pass parameter to StateAlloc 
This parameter is NULL or the pointer to the previous state
for the previous protocol in the case of a protocol change,
for instance from HTTP1 to HTTP2

This way, the new protocol can use the old protocol context.
For instance, HTTP2 mimicks the HTTP1 request, to have a HTTP2
transaction with both request and response
 5 years ago
Sascha Steinbiss ed9fed4958 mqtt: add some extra tests for varint parsing 5 years ago
Philippe Antoine 1a88df7e88 http2: handles incomplete frames after banner 
To signal incomplete data, we must return the number of
consumed bytes. When we get a banner and some records, we have
to take into account the number of bytes already consumed by
the banner parsing before reaching an incomplete record.
 5 years ago
Philippe Antoine 7ab9a01db2 mqtt: limit size of variable integer 5 years ago
Philippe Antoine e3b28bcf2a http2: returns error in case of index 0 
As is documented in RFC 7541, section 6.1
The index value of 0 is not used.  It MUST be treated as a decoding
error if found in an indexed header field representation.
 5 years ago
Jason Ish e9fec043b5 rust/util: expose function to test strings for valid UTF-8 
rs_check_utf8 will check that the provided string is valid
UTF-8 by converting it to a Rust string and returning true
or false.
 5 years ago
Simon Dugas 7f26246ce1 dns: parse and log fields for SOA record type 
Added `dns_parse_rdata_soa` to parse SOA fields into an `DNSRDataSOA`
struct.

Added logging for answer and authority SOA records in both version
1 & 2, as well as grouped formats.
 5 years ago
Simon Dugas af498fd840 dns: use nom's rest to take all remaining rdata 
Using nom's `rest` combinator eliminates the need to call the do_parse
macro for parsing a single element.
 5 years ago
Simon Dugas 8005f50647 dns: refactor to handle more rdata formats 
Represent rdata as `DNSRData` enum variants instead of `Vec<u8>`.
This will allow parsing/logging of more complex formats like SOA.
 5 years ago
Jason Ish 647e304f4b rust/log: minor cleanup 
Group functions for setting and getting the log level
together.
 5 years ago
Jason Ish ea1338b464 rust: function macro now returns the function name 
Borrow a macro from https://github.com/popzxc/stdext-rs that
will give us the Rust function name in SCLog messages in Rust.

As this trick only works on Rust 1.38 and newer, keep the old
macro around and set a feature based on a Rust version test
done during ./configure.
 5 years ago
Jason Ish bac8016d17 rust: plugin bootstrap function 
Functions written in Rust will need to suricata::plugin::init()
to bootstrap themselves. This bootstrap process sets the log level
within the Rust address space, and hooks up function pointers
that are expected to be set during normal runs of Suricata.
 5 years ago
Jason Ish 3de98b3595 rust/log: set the log level with a pure Rust function 
Make sure the log level is setup with a pure Rust function, so
when it is set, its set within the address space of the caller.

This is important for Rust plugins where the Rust modules are not
in the address space of the Suricata main process.
 5 years ago
Jason Ish ac93ab281d rust/logging: allow log macros to be used by plugins 
Fix plugin macros so they can be used by external Rust crates
such as plugins.
 5 years ago
Jason Ish 3672cea876 rust/Cargo: build as rlib for plugin linkage 
Build Rust code as an rlib, in addition to a staticlib so plugins
can link with the Rust code.
 5 years ago
Philippe Antoine b20a8de646 dns: simply skips zero-sized dns requests/responses 5 years ago
Ilya Bakhtin c903441fe7 mqtt: fixed wrong slice access 
Slice access esults in rust panic when mqtt ping is processed
 5 years ago
Victor Julien 4aa80ac7f6 detect/http2: fix header inspection 
Header inspection was overwriting data that was still being
referenced by the detect engine, leading to ASAN issues.
 5 years ago
Jason Ish f26d6eaf98 http2: log as http to abstract http and http2 a little 
This commit logs http2 as an http event. The idea is to somewhat
normalize http/http2 so common info can be version agnostic.

This puts the http2 specific fields in an "http2" object inside
the "http" object.

HTTP2 headers/values that are in common with HTTP1 are logged
under the "http" object to be compatible with HTTP1 logging.
 5 years ago
Jason Ish 6a55606adb http2: log headers in the same format as http (1) 
Log the headers in request_headers, and response_headers like
http1 to remain compatible.
 5 years ago
Philippe Antoine 1422b18a99 http2: initial support 5 years ago
Jeff Lucovsky be57f6d3cc rust/rdp: Fix use of incorrect buffer 
This commit updates the connection confirmation parsing function to use
the correct buffer when determining the packet type.
 5 years ago
Shivani Bhardwaj 2529eb35e1 smb: use uuid crate 5 years ago
Jason Ish e6668560e0 rust: only run cbindgen if needed 
Only run cbindgen when necessary. This is a bit tricky. When
building a dist we want to unconditionally build the headers.

When going through a "make; sudo make install" type process,
cbindgen should not be run as the headers already exist, are
valid, and the environment under sudo is more often than
not suitable to pick up the Rust toolchains when installed
with rustup.

For the normal "make" case we have the gen/rust-bindings.h file
depend on library file, this will cause it to only be rebuilt
if the code was modified.

For "make dist" we unconditionally create "dist/rust-bindings.h".
This means the generated file could be in 2 locations, so update
configure.ac, and the library search find to find it.

The "gen/rust-bindings.h" should be picked up first if it exists,
for those who develop from a dist archive where "dist/rust-bindings.h"
also exists.

Not completely happy having the same file in 2 locations, but not
sure how else to get the dependency tracking correct.
 5 years ago
Shivani Bhardwaj 4c7f55e636 dcerpc: handle gap for TCP streams 5 years ago
Jason Ish 6cff558663 dns: add tailing data to split tcp unit test 
Add trailing data to the complete payload to test the case
where data is consumed, but still incomplete.
 5 years ago
Jason Ish cc154ce152 dns: fix subtraction overflow in debug message 
The math for the return value wasn't updated in the debug log
to match the logic and could lead to a subtraction overflow
 5 years ago
Philippe Antoine 80ccbff28a dns: return status from parsing tcp request 5 years ago
Philippe Antoine 1296740471 dns: fix incomplete results 
and messages parsing whose length is >= 32768
 5 years ago
Philippe Antoine 83567e8732 files: rust closes files even on 0 length chunk 5 years ago
Sascha Steinbiss c31360070b rust/mqtt: add MQTT parser 5 years ago
Victor Julien b99ffd9ece eve: remove unused jansson code 5 years ago
Jeff Lucovsky 961b314b86 dns: Remove parser buffering code 5 years ago
Victor Julien 00cc3c7374 eve/ssh: change hassh logging format 
Elastic search didn't accept the 'hassh' and 'hassh.string'. It would
see the first 'hassh' as a string and split the second key into a
object 'hassh' with a string member 'string'. So two different types
for 'hassh', so it rejected it.

This patch mimics the ja3(s) logging by creating a 'hassh' object
with 2 members: 'hash', which holds the md5 representation, and
'string' which holds the string representation.
 5 years ago
Philippe Antoine eb5b927787 ssh: fixing incomplete kex parsing 
We use the record length from the ssh record header,
and not the size of the parsed data, as is done in other places.
 5 years ago
Victor Julien bcd416e6ba nfs: fix 'dangling' files in lossy sessions 
In case of lossy connections the NFS state would properly clean up
transactions, including file transactions. However for files the
state was never set to 'truncated', leading to files to stay 'active'.

This would lead these files staying in the NFS's state. In long running
sessions with lots of files this would lead to performance and memory
use issues.

This patch cleans truncates the file that was being transmitted when
a file transaction is being closed.

Based on 65e9a7c31c
 5 years ago
Victor Julien fc7cecb4ae nfs: check post-gap timeouts once a second at most 
Based on 25f2efe977
 5 years ago
Victor Julien 5d65e5a748 nfs: update ts only if it changed 
Based on 8aa380600d
 5 years ago
Zach Kelly b25de4d99a rdp: remove parser buffering code 5 years ago
Zach Kelly 22a2bee614 rdp/eve: convert to jsonbuilder 5 years ago
Zach Kelly ef397daba3 rdp: rustfmt (update) 5 years ago
Shivani Bhardwaj 35362b7bfa jsonbuilder: run test if not debug-validate 5 years ago
Shivani Bhardwaj 2ce7d98af1 dcerpc: fix tests to have a valid header 5 years ago
Shivani Bhardwaj 9f9670ebdc logging: Add DCERPC logger 5 years ago
Shivani Bhardwaj bab497ab2c dcerpc: Add multi transaction support 
DCERPC parser so far provided support for single transactions only.
Extend that to support multiple transactions.

In order for multiple transactions to work, there is always a
transaction identifier for any protocol in its header that lets a
response match the request. In DCERPC, for TCP, that param is call_id in
the header which is a 32 bit field. For UDP, however since it uses
different version of RPC (4.x), this is defined by serial number field
defined in the header. This field however is not contiguous and needs to
be assembled by the provided serial_low and serial_hi fields.
 5 years ago
Shivani Bhardwaj 67e7be633c krb: convert to jsonbuilder 
Closes redmine ticket 3754.
 5 years ago
Shivani Bhardwaj 72dab0a8b7 snmp: convert to jsonbuilder 
Closes redmine ticket 3756.
 5 years ago
Jason Ish 7ca24041b6 rust: add doc target to build rust docs 
Uses "cargo doc --no-deps" to build the documentation just for
our Suricata package. Without --no-deps, documentation will be
build for all our dependencies as well.

The generated documentation will end up in target/doc as HTML.
 5 years ago
Jason Ish abc71dc4a5 applayer template (rust): better gap handling example 
In the request parser, show checking if a gap was received
and what one example of trying to continue might look like.
 5 years ago
Jason Ish b91bb92b7d applayer template (rust): incomplete support 
Show how to use the incomplete AppLayerResult type within the
limits of what the template protocol parser can provide.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/3541
 5 years ago
Jason Ish 3f615f751b rust app-layer template: add stubs for gap handling 5 years ago
Jason Ish d60671d855 rust/dns: use new flags field to set parser option flags 5 years ago
Jason Ish 53aa967e0b applayer: add flags to parser registration struct 
This will allow Rust parsers to register for gap handing from
Rust (some Rust parsers do handle gaps, but they set the flag
from C).
 5 years ago
Jason Ish 9d9a901b68 dns: conditional logging 
Apply config to newly created response TX.
 5 years ago
Victor Julien ac3cf6ff75 detect/config: set config for special cases 
Allow app-layer to declare the txs are uni-directional and special
care is needed for applying config.
 5 years ago
Victor Julien 5dd4d948d9 app-layer: remove unused detect flags API 5 years ago
Victor Julien c94a5e6392 app-layer/rust: don't use option for GetTxDataFn anymore 5 years ago
Victor Julien 8fe9faecb2 app-layer: remove DetectFlags API. Replaced by AppLayerTxData 5 years ago
Victor Julien 88dd0abb38 rdp: support AppLayerTxData 5 years ago
Victor Julien 9664f73f75 app-layer: remove logged API calls 5 years ago
Victor Julien e0debed0b4 tftp: support AppLayerTxData 5 years ago
Victor Julien 4ff51a0e07 sip: support AppLayerTxData 5 years ago