Commit Graph

10221 Commits (d776d72711800168cda5d62a7cc4669abda379be)

Author SHA1 Message Date
Jeff Lucovsky 02fe026046 output: Fix possible null deref
This commit corrects an issue uncovered by Coverity. See the redmine
issue for details: https://redmine.openinfosecfoundation.org/issues/4495
4 years ago
Philippe Antoine d00b755b64 http2: only mimic http1 request if there is one
That may not be the case in midstream/async configurations
4 years ago
Jason Ish 70b21df756 makefile: don't include the whole test/ directory
Including the whole directory results in .deps files ending up
in the distribution archive which shouldn't be there. Instead
we have to list all the test sources individually.
4 years ago
Shivani Bhardwaj 581cb6223d dcerpc/udp: Add rust registration function
Get rid of the C glue code and move registration completely to Rust.
4 years ago
Shivani Bhardwaj bac69af7e4 dcerpc: Add rust registration function
Get rid of the C glue code and move registration completely to Rust.
4 years ago
Shivani Bhardwaj a0a09a102b dcerpc: Change fn sign as per rust registration requirement
Registering parsers in Rust requires signatures to be a certain way and
compatible with C. Change signatures of all the functions.
4 years ago
Jason Ish 65809be8ec suricata-plugin.h: don't include autoconf.h
It is not required here and just creates double inclusion in some
scenarios.
4 years ago
Philippe Antoine 999327ba1f http2: http.cookie keyword now works for HTTP2 5 years ago
Philippe Antoine df039555bc http2: http.host.raw keyword now works for HTTP2 5 years ago
Philippe Antoine 1e82d0b3c8 http2: http.method keyword now works for HTTP2 5 years ago
Philippe Antoine 017e39d8fd http2: makes all HTTP1 header keywords work 5 years ago
Philippe Antoine 2cadddda89 http2: there is no status msg in HTTP2
so we revert its detection, mistaken with the status code
5 years ago
Philippe Antoine 1e96272576 http2: http.stat_code keyword now works for HTTP2 5 years ago
Jeff Lucovsky e77e8dbe18 proto: Remove dependency on /etc/protocols
This commit eliminates the dependency on /etc/protocols and equivalent
on other platforms by using a static table of IANA assigned protocol
values (names, description).
5 years ago
Jason Ish 587c326d73 yaml: treat some unquoted values as null (per spec)
Per the YAML spec, the following values when present unquoted
should be equivalent to null:
- ~
- NULL
- Null
- null
5 years ago
Jeff Lucovsky 7fa98cde4d output/redis: Redis threaded output changes 5 years ago
Jeff Lucovsky 1defca3c34 output/plugin: Support threaded output plugins 5 years ago
Jeff Lucovsky 05836a4452 output/plugin: API changes for threaded support
This commit extends the interface to better support file output plugins.
5 years ago
Simon Dugas a8a51dc004 modbus: add eve logging 5 years ago
Simon Dugas 8342641477 modbus: move tests from c to rust
Move tests in a seperate commit so that we can use the previous one for
regression testing. This also gets rid of the temporary glue that made
the C tests work with the rust implementation.
5 years ago
Simon Dugas a458a94dca modbus: move from C to rust
Adds a new rust modbus app layer parser and detection module.

Moves the C module to rust but leaves the test cases in place to
regression test the new rust module.
5 years ago
Simon Dugas 7c99fe3689 modbus: fix app-layer test cases
invalidFunctionCode: make protocol id valid since we are only testing
the function code here.

readCoilsErrorRsp: changed to different invalid response code.

ModbusParserTest10: wrong length was passed to AppLayerParserParse.

ModbusParserTest11: allocate the entire buffer.
5 years ago
Jason Ish 488d5fb342 unix-socket: reset to ready state on startup
As part of commit ea15282f47,
some initialization was moved to happen even in unix socket mode,
however, this initialization does setup some loggers that can only have
one instance enabled (anomaly, drop, file-store).

This will cause these loggers to error out on the first pcap, but work
on subsequent runs of the pcap as some deinitialization is done after
each pcap.

This fix just runs the post pcap-file deinitialization routine to
reset some of the initialization done on startup, like is done after
running each pcap in unix socket mode.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/4225

Additionally this prevents alerts from being logged two times
on the first run of a pcap through the unix socket:

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/4434
5 years ago
Jeff Lucovsky 0f0cb5169f decode/vntag: Add VNTag decoder logic 5 years ago
Jeff Lucovsky 596d760833 tests/vntag: VNTAG decoder unittests 5 years ago
Jeff Lucovsky 713bace44f decode/vntag: VNTAG 802.1Qbh decoder 5 years ago
Jeff Lucovsky b944e636a8 decode/stats: VNTAG stats 5 years ago
Jeff Lucovsky 1ddad0a0d6 decode/events: VNTAG decoder events 5 years ago
Jeff Lucovsky 049afde3a2 decode: Add ethertype for VNTAG 5 years ago
Luke Coughlan 7fb56a9075 flow/bypass: Properly set the ICMP emergency-bypassed value
Currently the ICMP emergency-bypassed value defined in suricata.conf is
overwriting the UDP value rather than correctly setting it for ICMP.
This commit corrects this bug so that the ICMP value can be set as
expected.
5 years ago
Jeff Lucovsky 1eeb96696b general: Cleanup bool usage 5 years ago
Jason Ish d4554ec6bb misc: include queue.h before other headers
At least on FreeBSD, some other include is including "sys/queue.h"
which results in FreeBSDs /usr/include/sys/queue.h being picked
up and setting __SYS_QUEUE_H__ so our queue.h is not picked up.

But the FreeBSD queue.h does not have the CIRCLEQ definitions. To
fix just include our queue.h first, which also sets __SYS_QUEUE_H__
preventing the system one from being picked up.
5 years ago
Jason Ish afaa18c5ad tx: fix unidir tx cleanup
A unidirection protocol parser should only have its transactions
marked as "skipped" if it is skipped in both the TS and TC
directions, otherwise unidir transactions are always considered
skipped and the cleanup will never updates its minimum id.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/4437
5 years ago
Jeff Lucovsky fc7a443c3f general: Typo cleanup 5 years ago
Jeff Lucovsky 2c0485ae15 detect/address: Improve support for large addrs
This commit improves support for large address variables. Without this
commit, address size was fixed at 8196 or less. This commit permits
larger sized address variables.
5 years ago
Shivani Bhardwaj 089972fd31 applayer: fix test data for a valid DCERPC pkt 5 years ago
Andreas Herz d62616f805 detect-rawbytes: add rawbytes doc help output 5 years ago
Andreas Herz 37789d9189 detect-rawbytes: update to new clang format 5 years ago
Jason Ish 06f58650d6 eve: refactor OutputJsonBuilderBuffer to take context
All callers of OutputJsonBuilderBuffer are now calling it
using fields from an OutputJsonThreadCtx, so just pass
a pointer to the thread context now.
5 years ago
Jason Ish 08eee26d27 eve: convert many loggers to use generate thread context
- mqtt
- dnp3
- smtp
- ike
- dns
- alert
- tls
- anomaly
- drop
- file
- http
- http2
- templates
- dhcp

The idea is to factor out the commom code for setting
up the output file objects, which is repetitive, and
often done wrong when it comes to threading.
5 years ago
Jason Ish 013becf569 eve: reset buffer in OutputJsonBuilderBuffer
Reset the buffer here so each caller doesn't need to do it.
5 years ago
Jason Ish c890f9db63 eve: factor thread context creation/free for reuse 5 years ago
Jason Ish 702f3b3c73 eve: remove duplicate call to LogFileEnsureExists
Remove duplicate call to LogFileEnsureExists in the generic
eve thread init function.
5 years ago
Eric Leblond d477d3a878 util/ebpf: fix deprecation warning
The function bpf_program__title has been deprecated in favor of
bpf_program__section_name.
5 years ago
Juliana Fajardini eb4c71fdd6 ippair/bit: fix formatting 5 years ago
Juliana Fajardini e7c1c3c374 ebpf/util: change flow storage to new 'id' type 5 years ago
Juliana Fajardini 3b1a653467 device/storage: use dedicated 'id' type
- Wrap the id in a new LiveDevStorageId struct, to avoid id
 confusion with other storage API calls.
- Formatting fixes by clang.
5 years ago
Juliana Fajardini 68b8b3d63e detect/engine-tag: fix typo 5 years ago
Juliana Fajardini b807059c34 host/storage: use dedicated 'id' type
- Wrap the id in a HostStorageId struct to avoid id confusion
with other storage API calls.
- Fix formatting with clang script.
5 years ago
Juliana Fajardini cf516de587 ippair/storage: use dedicated 'id' type
- Wrap the id in a new IPPairStorageId struct, to avoid id
confusion with other storage API calls.
- Formatting fixes by clang.
5 years ago
Jeff Lucovsky aa9ad56a5b output/log: Removed pcie (Tilera) log vestiges
This commit removes the last remnants of the Tilera log output mechanism
(unsupported since 5.0.x).
5 years ago
Jeff Lucovsky 38ae21a196 output/log: Ensure files closed in threaded mode
This commit ensures that file objects are closed in threaded mode.
5 years ago
Victor Julien bc667a4a93 flow/storage: use dedicated 'id' type
Wrap the id in a new FlowStorageId struct to avoid id confusion with other
storage API calls.
5 years ago
Victor Julien 4b3be24506 app-layer/expectation: clean up storage id logic 5 years ago
Philippe Antoine 68d6922e3c ftp: fixes leak with duplicate expectation 5 years ago
Philippe Antoine cd8c2ef994 fuzz: use stream.midstream=true 5 years ago
Philippe Antoine e9b76a0e66 fuzz: specify protocol with fuzz target name
cf https://redmine.openinfosecfoundation.org/issues/4125

This allows fuzz_applayerparser_parse to fuzz one specific
app-layer protocol based on the binary name, as is done
with the environment variable FUZZ_APPLAYER
That is if we rename/copy to fuzz_applayerparser_parse_smb,
it will fuzz only SMB protocol
This way, we can easily produce different fuzz targets for
each protocol in oss-fuzz
5 years ago
Jeff Lucovsky 2893b04ab0 general: Typo cleanup 5 years ago
Jeff Lucovsky 02ceac8b8d detect/threshold: Improve threshold.config perf
This commit improves performance when parsing threshold.config by
removing a loop-invariant to create a one-time object with the parsed
address(es).

Then, as needed, copies of this object are made as the suppression
rule(s) are processed.
5 years ago
Jeff Lucovsky e873632a28 detect/threshold: Function to deep-copy thresh obj
This commit adds a function to make a deep copy of a DetectThresholdData
object.

The function is used when parsing threshold.config items to make a
one-time object and then add copies as needed.
5 years ago
Jeff Lucovsky 11f9cc6524 detect/address: Expose DetectAddressCopy function 5 years ago
Jeff Lucovsky ef62761e8c threshold-config: Improve support for big IP lists 5 years ago
Juliana Fajardini c6a35d09b7 templates: fix typos
- *template*files[ch][rs]: fix typos
- scripts/setup-app-layer: fix typos
5 years ago
Jason Ish 877e5214b8 logging: removed unused logger IDs
- pre-json dns logger
- unified2
- pre-json drop logger
5 years ago
Jason Ish 6853bf98fb dns: only register a single logger
DNS no longer requires a logger to be registered for to-client and
to-server directions. This has not been required with the stateless
design of the Rust DNS parser.
5 years ago
Victor Julien b1fee90392 output/tx: add warning to avoid future bugs 5 years ago
Victor Julien 3cc3df2172 output/tx: move eof checks out of logging loop 5 years ago
Victor Julien b05bd058e9 app-layer: minor code cleanups 5 years ago
Victor Julien 1098e3b7c6 app-layer: remove conditional logic around API calls
Remove logic that suggested some API calls could be conditional,
even though Suricata wouldn't even start up if they weren't
registered.
5 years ago
Jason Ish 4d5d7b4bd3 eve/netflow: use generic json context 5 years ago
Jason Ish a68d50608b eve/flow: use generic json context 5 years ago
Jason Ish 67c4621bdb eve/ftp: use generic json context
The FTP logger contained no extra data in its context so the
generic json context can be used.
5 years ago
Jason Ish 2d78afe4b0 eve: refactor CreateEveHeaderWithTx to include common options 5 years ago
Jason Ish 06ba611667 eve cleanup: remove duplicate/redundant code
The first change was to have CreateEveHeader add the common options
as this was left out in a few loggers. While update all the loggers
that use CreateEveHeader, remove redundant code, in particular
from loggers that don't need to use their own context but
can use the generic one.
5 years ago
Jason Ish 64330498f8 eve/mqtt: fix mqtt logging with threaded eve
Mqtt was not setting up a per-thread file context for logging
in threaded mode, leading a crash when used in threaded mode.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/4404
5 years ago
Jeff Lucovsky dd8eeb6353 general: Correct typos 5 years ago
Jeff Lucovsky 11ec61d0b4 thresholds: Improve validation of threshold.config
This commit improves the handling of threshold.config. When used with
"-T", a non-zero return code occurs when the file cannot be validated.

To maintain legacy behavior, when "-T" is not used and threshold.config
contains one or more invalid lines, Suricata continues execution.
5 years ago
Jeff Lucovsky cb03455c04 error: Add code for threshold config validation
This commit adds a new warning code for threshold config file validation
failures.
5 years ago
Eric Leblond a73b5f0ea5 eve/ike: restore common option logging 5 years ago
Philippe Antoine 2997be6707 sslv2: precise detection pattern with probing parser 5 years ago
Philippe Antoine e8415f249b fuzz: adds structure aware target
so as not to fuzz libpcap
and generate structure aware signatures
5 years ago
Victor Julien 398ebf9345 eve/drop: use highest priority drop
When adding the alert to a drop record make sure the add the highest
priority.

It would until now add all drops from high to low prio, effectively
overwriting the record each time.

Ticket #4397
5 years ago
Victor Julien 6cf44fc839 detect/alert: apply pd only actions to flow
Ticket #4394
5 years ago
Victor Julien 6c594d29db detect/alert: minor code refactor
Use a simpler reject check and move logic into util func.
5 years ago
Victor Julien fbcdd2ec26 detect/iponly: don't check & set flow flags twice
Per flow IP-only flags are checked and set by IP-only engine, so
no need to set/check them per alert.
5 years ago
Victor Julien 55a0e29c8e eve/ike: gracefully handle renamed output config 5 years ago
Sascha Steinbiss 37940180a8 ikev1: add metadata to alerts 5 years ago
Sascha Steinbiss e2dbdd7fd5 ikev1: add ikev1 parser 5 years ago
frank honza ecdf9f6b0b ikev1: rename ikev2 to common ike
Renaming was done with shell commands, git mv for moving the files and content like
find -iname '*.c' | xargs sed -i 's/ikev1/ike/g' respecting the different mixes of upper/lower case.
5 years ago
frank honza ab6171c429 detect: added support for protocol-aliases 5 years ago
frank honza b80cdae1df detect: add comparison-mode LTE/GTE for Detect(U32/u8)Data 5 years ago
Victor Julien c3075cba42 detect/analyzer: fix mpm display on payload only rules 5 years ago
Victor Julien 9dd1444f44 detect: suppress error message for pcre only rules 5 years ago
Victor Julien b55b327db1 detect/analyzer: suggest modern keywords 5 years ago
Victor Julien 57f7612ffd detect/analyzer: fix json output for warnings/notes 5 years ago
Victor Julien 018b9a0a8c detect/asn1: minor cleanups 5 years ago
Victor Julien 8b8cc697d5 detect/http-server-body: clean up test 5 years ago
Victor Julien 68f8b2f40f detect/icmp: reject invalid rules for icode/itype 5 years ago
Victor Julien 7d6835958b detect/prefilter: fix null ptr deref on invalid rule
A bad rule 'icode:<0; prefilter;' would trigger a null ptr deref
in ApplyToU8Hash.

Bug #4375.
5 years ago
Victor Julien e964643088 detect/state: fix reset bug
Fix issue where after a reset the now empty list elements are not
reused and the values may not be valid for the current detect
engine anymore.

Introduce a 'current' (cur) pointer that points to the store element
currently being filled. This way existing stores will be reused.

If 'cur' is NULL and 'head' is not NULL it means we need to use
'tail' to append a new store.
5 years ago
Victor Julien f766139159 detect/state: test to show reset bug 5 years ago
Victor Julien a808474d38 detect/state: minor code cleanup 5 years ago
Jason Ish 0aed5e188b filestore: fix global counter init in unix socket mode
Move initialization of filestore global counter to PreRunInit,
so they get registered during program initialization, or as
required in unix-socket mode, initialized for each file run.

Fixes Redmine issue:
https://redmine.openinfosecfoundation.org/issues/4216
5 years ago
Philippe Antoine 660e9e489b protodetect: only run ProbingParserTc if STREAM_TOCLIENT 5 years ago
Philippe Antoine 52ea3fc7ac fuzz: more precise assertion for protocol detection
Only in the cases of stream start is the assertion valable.
Otherwise, it can only be best effort.
5 years ago
Philippe Antoine 2d765d6c68 detect: fix overflows in SetupU8Hash
For instance ">255" resulted in overflow
5 years ago
Philippe Antoine eb460cf78d ssl: reset state when breaking out of SSLV3_HANDSHAKE_PROTOCOL
So that we cannot resumt it with corrupted values
5 years ago
Philippe Antoine 89030d3e59 modbus: stop allocating transactions when flooded
cf #4224
5 years ago
Philippe Antoine ddb4d289ae icmpv6: bail out for icmpv6.hdr keyword if not ICMPv6 5 years ago
Jeff Lucovsky 538fc58b37 output/http2: Multi-threaded EVE logging support
This commit adds multi-threaded EVE logging support to the HTTP/2
logging path.
5 years ago
Philippe Antoine 35f6c80bbf eve: fix memory leak in metadata
Fixes #4205
5 years ago
Philippe Antoine 7500c29300 decode: limits the number of decoded layers
so as to avoid overrecursion leading to stack exhaustion
5 years ago
Victor Julien 4a1482a1cf detect/http.request_body: fix tracking with xforms
Fix handling of file progress tracking for regular http.request_body
along with transform combinations.

This is done by implementing the 'base id' logic.

Related tickets: #4361 #4199 #3616
5 years ago
Victor Julien ea3fb4a465 detect/file.data: fix mixing transforms (http)
Fix handling of file progress tracking for regular file.data along
with transform combinations for the part of the implementation that
uses the HTTP inspection logic.

This is done by implementing the 'base id' logic.

Related tickets: #4361 #4199 #3616
5 years ago
Victor Julien 54ad7de9ce detect/file.data: fix mixing transforms (file api)
Fix handling of file progress tracking for regular file.data along
with transform combinations for the part of the implementation that
uses the File API.

This is done by implementing the 'base id' logic.

Related tickets: #4361 #4199 #3616
5 years ago
Victor Julien 975062cf40 detect: track base id for xform buffers
Buffers with transforms are based on the non-transformed "base"
buffer, with a new ID assigned and the transform callbacks added.

This patch stores the id of the original buffer in the new buffer
inspect and prefilter structures. This way the buffers with and
without transforms can share some of the logic are progression
of file and body inspection trackers.

Related tickets: #4361 #4199 #3616
5 years ago
Victor Julien 52692da7cf detect/analyzer: fix pkt engine display 5 years ago
Shivani Bhardwaj c77c8e7005 rust/context: add AppLayerParserTriggerRawStreamReassembly 5 years ago
Ilya Bakhtin 1ecea0f44c stream/tcp: fix stream side after direction change 5 years ago
Philippe Antoine a04b5566a6 http: makes decompression time limit configurable 5 years ago
Eric Leblond 6ef28d0a70 util/thash: fix memcap consolidate function
The function THashConsolidateMemcap is used to allow to load a
dataset even when the memcap is not set. But the implementation
was in fact resetting the memcap value to the max of memory
usaga after loading and default memcap. As a result, the
function was resetting memcap to the default memcap even if
a huge memcap was set in the dataset definition. In the case
of dataset where we add to the set it was leading to memcap
limit hitting despite the settings of memcap by the user.

This patch udpates the code to set the final memcap value to
the max of memory usage after loading and set memcap.
5 years ago
Ilya Bakhtin b3b64803e5 stream: TcpStreamCnf.midstream type changed to bool 5 years ago
Ilya Bakhtin 5285163d8f protodetect: improve midstream handling
Set "done flag" only if parsers for both directions are not found in a
case of midstream parsers from other direction are tried if nothing is found
for the initial one. "done flag" must be set if nothing is found in both
directions. Otherwise processing of incomplete data is terminated at the very
first try.
5 years ago
Shivani Bhardwaj 3641f1b522 dcerpc: add probe function 5 years ago
Philippe Antoine c6aadf0dfa protodetect: rename direction to flags
And use whole flags in AppLayerProtoDetectPPGetProto
5 years ago
Philippe Antoine 7264f58f2c tcp: remove debug asserts about large windows
Completes 00d7c9034b
5 years ago
Victor Julien 0dd5921bc9 detect/prefilter: fix handling of prefilter as fast_pattern alias 5 years ago
Philippe Antoine b7fd01c86e detect: forbids unsupported prefilters 5 years ago
Victor Julien e374d5ac15 detect/fast_pattern: add prefilter test 5 years ago
Philippe Antoine 18fcbb20e2 fuzz: fix typo in comment 5 years ago
Philippe Antoine 5465e0b154 http2: http.stat_msg keyword now works for HTTP2 5 years ago
Philippe Antoine 5d676c5998 http2: http.uri.raw keyword now works for HTTP2 5 years ago
Philippe Antoine 47928babfc http2: http.user_agent keyword now works for HTTP2 5 years ago
Philippe Antoine a98d0fe6ed http2: http.uri keyword now works for HTTP2
cf #4067
5 years ago
Philippe Antoine 707f027231 protos: renaming ALPROTO_HTTP* constants
Having now ALPROTO_HTTP1, ALPROTO_HTTP2 and ALPROTO_HTTP

Run with 3 sed commands
git grep ALPROTO_HTTP | cut -d: -f1 | uniq |
 xargs sed -i -e 's/ALPROTO_HTTP/ALPROTO_HTTP1/g'
git grep ALPROTO_HTTP12 | cut -d: -f1 | uniq |
 xargs sed -i -e 's/ALPROTO_HTTP12/ALPROTO_HTTP2/g'
git grep ALPROTO_HTTP1_ANY | cut -d: -f1 | uniq |
 xargs sed -i -e 's/ALPROTO_HTTP1_ANY/ALPROTO_HTTP/g'

and then running clang-format
5 years ago
Philippe Antoine 93e6401ce0 http: introduces ALPROTO_HTTP_ANY
For any versions of HTTP, both ALPROTO_HTTP and ALPROTO_HTTP2
5 years ago
Philippe Antoine c8dbe24fb6 proto: introduce signature protocol, as extension to flow protocol
AppProtoEquals function allows to check if a flow protocol
matches a signature protocol
5 years ago
Jason Ish 02218a8a42 Makefile: break headers and source into 2 vars
Split the headers and source into 2 variables. Headers are
marked noinst so they don't get automatically installed on
"make install". Instead they will be installed by a custom
Makefile target, "make install-headers".
5 years ago
Jason Ish 2c5e1d6a6d rust: separate the rust lib from RUST_LDADD
Fix another issue with library ordering when breaking apart
LDFLAGS from LIBS for outputting usable command lines for
users of a Suricata library.

RUST_LDADD should just contain the extra libs required by
Rust, not the actual Suricata Rust library.
5 years ago
Jason Ish dbae17dbc0 install: makefile target to install libraries
As we don't install the libraries by default, provide a make target,
"install-library" to install the libsuricata library files.

If shared library support exists, both the static and shared
libraries will be installed, otherwise only the static libraries
will be installed.
5 years ago
Jason Ish e227d97e5e lib: build shared library on Linux
Building the shared library on Linux is not something by default.
Instead a user must opt-in to building by running the
"make libsuricata.so" target in the src/ directory.

Currently shared library support is only available on Linux. More
OSs will be supported as we can test them.
5 years ago
Jason Ish e99dde0078 build: use a static convenience library for C code
With the circular reference gone, we can now make use
of a convenience library for the Suricata program
as well as any other programs that depend on the same
source such as the fuzzer.

While its not a libtool convenience library, it serves
the same purpose and is a common idiom in Make and CMake
projects whereas the COMMON_SOURCES approach was more
of a hack we had to resort to until the circular
reference was resolved.
5 years ago
Victor Julien 6bfc5afa23 host: improve compare logic
The old compare macro would compare all bytes of an address, even
when for IPv4 addresses the additional bytes were not in use. This
made the logic vulnerable to mistakes like in issue #4280.
5 years ago
Victor Julien 7b03e6837e detect/iprep: fix loading of mixed ipv4/ipv6 lists
Improper reuse of the address data structure between loading
different lines in the iprep file would lead to the host using
a malformed address.
5 years ago
Jason Ish 3ada5e1480 rust/ffi: provide AppLayerRegisterParser in context
AppLayerRegisterParser was creating a link error when attempting
to use a convenience library for the Suricata C code, then linking
the library of C code with the library of Rust code into a final
Suricata executable, or use with fuzz targets.

By moving AppLayerRegisterParser to the context structure and
calling it like a callback the circular reference is removed
allowing the convenience libraries to work again.

This is also a stepping block to proving a Suricata library
as a single .a or .so file.
5 years ago
Victor Julien 3ce05a3583 fuzz: run OSS-Fuzz corpus and track coverage 5 years ago
Philippe Antoine 2b043150ed detect: initializes memory in bytemath parsing 5 years ago
Philippe Antoine b5d24a9a57 fuzz: driver running directories as well as single files 5 years ago
Eric Leblond 0dba1b09de suricata: improve list keywords
Exit with error if a keyword is not supported or not existing
and display a message.
5 years ago
Eric Leblond 2e4af5a091 suricata: return error value of custom run modes 5 years ago
Eric Leblond 44460f1945 util/running-modes: don't exit in running mode 5 years ago
Eric Leblond 921d44b262 log/pcap: exit on invalid filename
If the filename has to % sign and if pcap logging is using multi
mode, then the pcap capture will fail. So let's exit if ever this
is the case.
5 years ago
Eric Leblond 6a45064d4c suricata: unix-socket mode and -l are compatible
Commit 93642a0d1d did prevent to
specify the logging directory on command line and use the unix
socket.

It looks like the implementation has evolved and the arbitrary
limitation can be removed allowing a user to start unix socket
without editing the configuration file.
5 years ago
Eric Leblond 7304389438 eve: only output ja3 and ja3s if present
This will prevent JSON entries like the following that occur
with the dedault configuration (ja3 deactivated and extended
tls ouput activated):

  "tls": {
    "subject": "C=GB, ST=London, L=London, O=Global Security, OU=IT Department, CN=example.com",
    "issuerdn": "C=GB, ST=London, L=London, O=Global Security, OU=IT Department, CN=example.com",
    "serial": "00:9C:FC:DA:1D:A4:70:87:5D",
    "fingerprint": "b8:18:2d:cb:c9:f8:1a:66:75:13:18:31:24:e0:92:35:42🆎96:89",
    "version": "TLSv1",
    "notbefore": "2020-05-03T11:07:28",
    "notafter": "2021-05-03T11:07:28",
    "ja3": {},
    "ja3s": {}
  }
5 years ago
Jeff Lucovsky cbb03dbb39 detect/pcre: Test capture group/var mismatch 5 years ago
Jeff Lucovsky 469d5bb214 detct/pcre: Correct capture group count check
This commit corrects the validation check between the number of
variables used and the number of specified capture groups.
5 years ago
Victor Julien ed05c51d99 detect/state: optimize state keeping 5 years ago
Victor Julien 13cebb1857 detect: fix heap overflow issue with buffer setup
In some cases, the InspectionBufferGet function would be followed by
a failure to set the buffer up, for example due to a HTTP body limit
not yet being reached. Yet each call to InspectionBufferGet would lead
to the matching list_id to be added to the
DetectEngineThreadCtx::inspect.to_clear_queue. This array is sized to
add each list only once, but in this case the same id could be added
multiple times, potentially overflowing the array.
5 years ago
Victor Julien 17a38f1823 flow/manager: (u)sleep slightly longer
Sleep 250 microseconds instead of 100 as running in KVM cause the
old value to use 100% CPU for these threads.

Perf testing suggests no measurable impact for the non-KVM case.

Ticket: #4096
5 years ago
Victor Julien 8baef60d60 app-layer: fix transaction cleanup
Fix a 'skipped' transaction early in the list leading to all further
transactions getting skipped, even if they were fully processed and
ready to be cleaned up.
5 years ago
Philippe Antoine 62e665c848 fuzz: rightly uses PacketFreeOrRelease in target
instead of PacketFree because packets
may belong to the pool
5 years ago
Philippe Antoine e586d8526b fuzz: use some value for max_pending_packets
so as not to timeout waiting forever for the condition
in PacketPoolWait
5 years ago
Philippe Antoine a6bbb608f7 fuzz: makes target sigpcap more reproducible
By removing the temporary rules file if it existed
before the first run
5 years ago
Philippe Antoine b869ac01ee http: enables request decompression 5 years ago
Eric Leblond 85327890f5 suricata: avoid at exit crash in nfq mode
When Suricata was build with ebpf support and when it was started
in NFQ mode, it was crashing at exit because it was trying to free
the device extension.

This patch fixes the issue by only trigger the eBPF related code
when Suricata is running in AFP_PACKET mode.
5 years ago
Eric Leblond e6cfcb704c storage: fix a variable name 5 years ago
Eric Leblond 628458e7d3 detect: fix link to documentation 5 years ago
Philippe Antoine 43f25f127f ftp: ftp-data recognized by StringToAppProto 5 years ago
Philippe Antoine d861228214 http2: decompression for files
gzip and brotli decompression for files
5 years ago
Philippe Antoine 76db6e34a1 protocol detection: fix failure case
as reached by CIFuzz even if unreachable from Suricata
5 years ago
Philippe Antoine f5d8e953a8 protodetect: debug validation when multiple patterns match 5 years ago
Victor Julien 00d7c9034b stream: remove debug assert
In cases of large windows in the past the check would tigger.
5 years ago
Victor Julien b66d013294 detect/http_client_body: minor test cleanups 5 years ago
Eric Leblond 64f994f753 dataset: fix dataset string lookup
The data was unlocked but the use_cnt was not decreased resulting
in the data entry not being removable.
5 years ago
Victor Julien 191461a028 detect/file_data: cleanup tests 5 years ago
Victor Julien 116c089de0 stream/tests: minor cleanups 5 years ago
Victor Julien ee6d792b02 stream: move tests into tests/ 5 years ago
Victor Julien 226a82bade detect/fast_pattern: redo unittests 5 years ago
Victor Julien 66d7f5941a detect/fast_pattern: remove dead code 5 years ago
Victor Julien bc9e7743f3 detect/http-ua: cleanup tests 5 years ago
Emmanuel Thompson f12daa710f decode/flow/esp: Add ESP decoder & flow
- Adds an ESP (Encapsulating Security Payload) header decoder
- Tracks ESP flows via the SPI field
5 years ago
Victor Julien 9adeae07b1 decode: reformat REINIT macro 5 years ago
Victor Julien 3f4398cc90 decode: minor unittest cleanups 5 years ago
Victor Julien bf00285d0a proto/names: add SCTP if not defined in system
If SCTP is missing from /etc/protocols, add it manually.
5 years ago
Victor Julien c25afbccc1 json: remove unused jansson wrappers 5 years ago
Victor Julien b6b317cae6 http: enable and fix content range tests 5 years ago
Victor Julien a7cd765f20 app-layer/nfs: dead code removal 5 years ago
Juliana Fajardini 97350d9a2c detect/rpc: clean up unittests
- detect-rpc: convert unit tests to new FAIL/PASS API.
- detect-rpc: replace SigInit with DetectEngineAppendSig for more
  concise code.
5 years ago
Jason Ish 512b0350a0 lua: fix coverity issue with out of scope variable
Fix usage of out-of-scope variables. Introduced with the hashing
and adding the guard of g_disable_hashing.

To fix, just remove the guard so all variables are in scope. Hashes
are not initialized here so there is no need for the guard.
5 years ago
Jeff Lucovsky 1c68f4aed6 lua/test: Test cases using SC prefix
This commit adds paired test cases to ensure that the SC variant of the
entry points are tested.
5 years ago
Jeff Lucovsky c845974639 general: Correct typo 5 years ago
Jeff Lucovsky 431018d6f7 lua: Use SC prefix for Lua functions
This commit adds additional Lua API interfaces to bring consistency to
functions such that the `SC` prefix is available consistently across
flow int and flow var functions.
5 years ago
Victor Julien 3a8ba663a9 email/md5: optimize md5 handling 5 years ago
Jason Ish 6299222c4e email/eve: use Rust function to hash buffer to hex
Use SCMd5HashBufferToHex to hash the subject to a hex string.
Removes snprintf loop.
5 years ago
Jason Ish 0a3b9e0220 rust/hashing: add function to finalize md5 to hex string
New function, SCMd5FinalizeToHex to finalize an md5 hash
to a hex string.
5 years ago
Jason Ish 3a82153866 ja3: use SCMd5HashBufferToHex to print hash as hex
Replace snprintf loop with new function that hashes a single
buffer to an MD5 hex string.
5 years ago
Jason Ish e00d21a5cb filestore: respect g_disable_hashing
If g_disable_hashing is set, behave like libnss wasn't compiled
in.
5 years ago
Jason Ish 7525295e63 hashing: remove remaining HAVE_NSS guards
For features, we pretend to HAVE_NSS so scripts, external tests
continue to work.
5 years ago
Jason Ish 9b314bebe0 output-json-email: use Rust md5 bindings instead of libnss 5 years ago
Jason Ish 815396263b util/mime: use Rust md5 bindings instead of libnss
As the new Md5 hashing consumes its context on finalize, an bool
has_md5 flag has been added to let the logger know there is an
md5 hash available.
5 years ago