Commit Graph

10221 Commits (d776d72711800168cda5d62a7cc4669abda379be)

Author SHA1 Message Date
Victor Julien f766139159 detect/state: test to show reset bug 5 years ago
Victor Julien a808474d38 detect/state: minor code cleanup 5 years ago
Jason Ish 0aed5e188b filestore: fix global counter init in unix socket mode
Move initialization of filestore global counter to PreRunInit,
so they get registered during program initialization, or as
required in unix-socket mode, initialized for each file run.

Fixes Redmine issue:
https://redmine.openinfosecfoundation.org/issues/4216
5 years ago
Philippe Antoine 660e9e489b protodetect: only run ProbingParserTc if STREAM_TOCLIENT 5 years ago
Philippe Antoine 52ea3fc7ac fuzz: more precise assertion for protocol detection
Only in the cases of stream start is the assertion valable.
Otherwise, it can only be best effort.
5 years ago
Philippe Antoine 2d765d6c68 detect: fix overflows in SetupU8Hash
For instance ">255" resulted in overflow
5 years ago
Philippe Antoine eb460cf78d ssl: reset state when breaking out of SSLV3_HANDSHAKE_PROTOCOL
So that we cannot resumt it with corrupted values
5 years ago
Philippe Antoine 89030d3e59 modbus: stop allocating transactions when flooded
cf #4224
5 years ago
Philippe Antoine ddb4d289ae icmpv6: bail out for icmpv6.hdr keyword if not ICMPv6 5 years ago
Jeff Lucovsky 538fc58b37 output/http2: Multi-threaded EVE logging support
This commit adds multi-threaded EVE logging support to the HTTP/2
logging path.
5 years ago
Philippe Antoine 35f6c80bbf eve: fix memory leak in metadata
Fixes #4205
5 years ago
Philippe Antoine 7500c29300 decode: limits the number of decoded layers
so as to avoid overrecursion leading to stack exhaustion
5 years ago
Victor Julien 4a1482a1cf detect/http.request_body: fix tracking with xforms
Fix handling of file progress tracking for regular http.request_body
along with transform combinations.

This is done by implementing the 'base id' logic.

Related tickets: #4361 #4199 #3616
5 years ago
Victor Julien ea3fb4a465 detect/file.data: fix mixing transforms (http)
Fix handling of file progress tracking for regular file.data along
with transform combinations for the part of the implementation that
uses the HTTP inspection logic.

This is done by implementing the 'base id' logic.

Related tickets: #4361 #4199 #3616
5 years ago
Victor Julien 54ad7de9ce detect/file.data: fix mixing transforms (file api)
Fix handling of file progress tracking for regular file.data along
with transform combinations for the part of the implementation that
uses the File API.

This is done by implementing the 'base id' logic.

Related tickets: #4361 #4199 #3616
5 years ago
Victor Julien 975062cf40 detect: track base id for xform buffers
Buffers with transforms are based on the non-transformed "base"
buffer, with a new ID assigned and the transform callbacks added.

This patch stores the id of the original buffer in the new buffer
inspect and prefilter structures. This way the buffers with and
without transforms can share some of the logic are progression
of file and body inspection trackers.

Related tickets: #4361 #4199 #3616
5 years ago
Victor Julien 52692da7cf detect/analyzer: fix pkt engine display 5 years ago
Shivani Bhardwaj c77c8e7005 rust/context: add AppLayerParserTriggerRawStreamReassembly 5 years ago
Ilya Bakhtin 1ecea0f44c stream/tcp: fix stream side after direction change 5 years ago
Philippe Antoine a04b5566a6 http: makes decompression time limit configurable 5 years ago
Eric Leblond 6ef28d0a70 util/thash: fix memcap consolidate function
The function THashConsolidateMemcap is used to allow to load a
dataset even when the memcap is not set. But the implementation
was in fact resetting the memcap value to the max of memory
usaga after loading and default memcap. As a result, the
function was resetting memcap to the default memcap even if
a huge memcap was set in the dataset definition. In the case
of dataset where we add to the set it was leading to memcap
limit hitting despite the settings of memcap by the user.

This patch udpates the code to set the final memcap value to
the max of memory usage after loading and set memcap.
5 years ago
Ilya Bakhtin b3b64803e5 stream: TcpStreamCnf.midstream type changed to bool 5 years ago
Ilya Bakhtin 5285163d8f protodetect: improve midstream handling
Set "done flag" only if parsers for both directions are not found in a
case of midstream parsers from other direction are tried if nothing is found
for the initial one. "done flag" must be set if nothing is found in both
directions. Otherwise processing of incomplete data is terminated at the very
first try.
5 years ago
Shivani Bhardwaj 3641f1b522 dcerpc: add probe function 5 years ago
Philippe Antoine c6aadf0dfa protodetect: rename direction to flags
And use whole flags in AppLayerProtoDetectPPGetProto
5 years ago
Philippe Antoine 7264f58f2c tcp: remove debug asserts about large windows
Completes 00d7c9034b
5 years ago
Victor Julien 0dd5921bc9 detect/prefilter: fix handling of prefilter as fast_pattern alias 5 years ago
Philippe Antoine b7fd01c86e detect: forbids unsupported prefilters 5 years ago
Victor Julien e374d5ac15 detect/fast_pattern: add prefilter test 5 years ago
Philippe Antoine 18fcbb20e2 fuzz: fix typo in comment 5 years ago
Philippe Antoine 5465e0b154 http2: http.stat_msg keyword now works for HTTP2 5 years ago
Philippe Antoine 5d676c5998 http2: http.uri.raw keyword now works for HTTP2 5 years ago
Philippe Antoine 47928babfc http2: http.user_agent keyword now works for HTTP2 5 years ago
Philippe Antoine a98d0fe6ed http2: http.uri keyword now works for HTTP2
cf #4067
5 years ago
Philippe Antoine 707f027231 protos: renaming ALPROTO_HTTP* constants
Having now ALPROTO_HTTP1, ALPROTO_HTTP2 and ALPROTO_HTTP

Run with 3 sed commands
git grep ALPROTO_HTTP | cut -d: -f1 | uniq |
 xargs sed -i -e 's/ALPROTO_HTTP/ALPROTO_HTTP1/g'
git grep ALPROTO_HTTP12 | cut -d: -f1 | uniq |
 xargs sed -i -e 's/ALPROTO_HTTP12/ALPROTO_HTTP2/g'
git grep ALPROTO_HTTP1_ANY | cut -d: -f1 | uniq |
 xargs sed -i -e 's/ALPROTO_HTTP1_ANY/ALPROTO_HTTP/g'

and then running clang-format
5 years ago
Philippe Antoine 93e6401ce0 http: introduces ALPROTO_HTTP_ANY
For any versions of HTTP, both ALPROTO_HTTP and ALPROTO_HTTP2
5 years ago
Philippe Antoine c8dbe24fb6 proto: introduce signature protocol, as extension to flow protocol
AppProtoEquals function allows to check if a flow protocol
matches a signature protocol
5 years ago
Jason Ish 02218a8a42 Makefile: break headers and source into 2 vars
Split the headers and source into 2 variables. Headers are
marked noinst so they don't get automatically installed on
"make install". Instead they will be installed by a custom
Makefile target, "make install-headers".
5 years ago
Jason Ish 2c5e1d6a6d rust: separate the rust lib from RUST_LDADD
Fix another issue with library ordering when breaking apart
LDFLAGS from LIBS for outputting usable command lines for
users of a Suricata library.

RUST_LDADD should just contain the extra libs required by
Rust, not the actual Suricata Rust library.
5 years ago
Jason Ish dbae17dbc0 install: makefile target to install libraries
As we don't install the libraries by default, provide a make target,
"install-library" to install the libsuricata library files.

If shared library support exists, both the static and shared
libraries will be installed, otherwise only the static libraries
will be installed.
5 years ago
Jason Ish e227d97e5e lib: build shared library on Linux
Building the shared library on Linux is not something by default.
Instead a user must opt-in to building by running the
"make libsuricata.so" target in the src/ directory.

Currently shared library support is only available on Linux. More
OSs will be supported as we can test them.
5 years ago
Jason Ish e99dde0078 build: use a static convenience library for C code
With the circular reference gone, we can now make use
of a convenience library for the Suricata program
as well as any other programs that depend on the same
source such as the fuzzer.

While its not a libtool convenience library, it serves
the same purpose and is a common idiom in Make and CMake
projects whereas the COMMON_SOURCES approach was more
of a hack we had to resort to until the circular
reference was resolved.
5 years ago
Victor Julien 6bfc5afa23 host: improve compare logic
The old compare macro would compare all bytes of an address, even
when for IPv4 addresses the additional bytes were not in use. This
made the logic vulnerable to mistakes like in issue #4280.
5 years ago
Victor Julien 7b03e6837e detect/iprep: fix loading of mixed ipv4/ipv6 lists
Improper reuse of the address data structure between loading
different lines in the iprep file would lead to the host using
a malformed address.
5 years ago
Jason Ish 3ada5e1480 rust/ffi: provide AppLayerRegisterParser in context
AppLayerRegisterParser was creating a link error when attempting
to use a convenience library for the Suricata C code, then linking
the library of C code with the library of Rust code into a final
Suricata executable, or use with fuzz targets.

By moving AppLayerRegisterParser to the context structure and
calling it like a callback the circular reference is removed
allowing the convenience libraries to work again.

This is also a stepping block to proving a Suricata library
as a single .a or .so file.
5 years ago
Victor Julien 3ce05a3583 fuzz: run OSS-Fuzz corpus and track coverage 5 years ago
Philippe Antoine 2b043150ed detect: initializes memory in bytemath parsing 5 years ago
Philippe Antoine b5d24a9a57 fuzz: driver running directories as well as single files 5 years ago
Eric Leblond 0dba1b09de suricata: improve list keywords
Exit with error if a keyword is not supported or not existing
and display a message.
5 years ago
Eric Leblond 2e4af5a091 suricata: return error value of custom run modes 5 years ago
Eric Leblond 44460f1945 util/running-modes: don't exit in running mode 5 years ago
Eric Leblond 921d44b262 log/pcap: exit on invalid filename
If the filename has to % sign and if pcap logging is using multi
mode, then the pcap capture will fail. So let's exit if ever this
is the case.
5 years ago
Eric Leblond 6a45064d4c suricata: unix-socket mode and -l are compatible
Commit 93642a0d1d did prevent to
specify the logging directory on command line and use the unix
socket.

It looks like the implementation has evolved and the arbitrary
limitation can be removed allowing a user to start unix socket
without editing the configuration file.
5 years ago
Eric Leblond 7304389438 eve: only output ja3 and ja3s if present
This will prevent JSON entries like the following that occur
with the dedault configuration (ja3 deactivated and extended
tls ouput activated):

  "tls": {
    "subject": "C=GB, ST=London, L=London, O=Global Security, OU=IT Department, CN=example.com",
    "issuerdn": "C=GB, ST=London, L=London, O=Global Security, OU=IT Department, CN=example.com",
    "serial": "00:9C:FC:DA:1D:A4:70:87:5D",
    "fingerprint": "b8:18:2d:cb:c9:f8:1a:66:75:13:18:31:24:e0:92:35:42🆎96:89",
    "version": "TLSv1",
    "notbefore": "2020-05-03T11:07:28",
    "notafter": "2021-05-03T11:07:28",
    "ja3": {},
    "ja3s": {}
  }
5 years ago
Jeff Lucovsky cbb03dbb39 detect/pcre: Test capture group/var mismatch 5 years ago
Jeff Lucovsky 469d5bb214 detct/pcre: Correct capture group count check
This commit corrects the validation check between the number of
variables used and the number of specified capture groups.
5 years ago
Victor Julien ed05c51d99 detect/state: optimize state keeping 5 years ago
Victor Julien 13cebb1857 detect: fix heap overflow issue with buffer setup
In some cases, the InspectionBufferGet function would be followed by
a failure to set the buffer up, for example due to a HTTP body limit
not yet being reached. Yet each call to InspectionBufferGet would lead
to the matching list_id to be added to the
DetectEngineThreadCtx::inspect.to_clear_queue. This array is sized to
add each list only once, but in this case the same id could be added
multiple times, potentially overflowing the array.
5 years ago
Victor Julien 17a38f1823 flow/manager: (u)sleep slightly longer
Sleep 250 microseconds instead of 100 as running in KVM cause the
old value to use 100% CPU for these threads.

Perf testing suggests no measurable impact for the non-KVM case.

Ticket: #4096
5 years ago
Victor Julien 8baef60d60 app-layer: fix transaction cleanup
Fix a 'skipped' transaction early in the list leading to all further
transactions getting skipped, even if they were fully processed and
ready to be cleaned up.
5 years ago
Philippe Antoine 62e665c848 fuzz: rightly uses PacketFreeOrRelease in target
instead of PacketFree because packets
may belong to the pool
5 years ago
Philippe Antoine e586d8526b fuzz: use some value for max_pending_packets
so as not to timeout waiting forever for the condition
in PacketPoolWait
5 years ago
Philippe Antoine a6bbb608f7 fuzz: makes target sigpcap more reproducible
By removing the temporary rules file if it existed
before the first run
5 years ago
Philippe Antoine b869ac01ee http: enables request decompression 5 years ago
Eric Leblond 85327890f5 suricata: avoid at exit crash in nfq mode
When Suricata was build with ebpf support and when it was started
in NFQ mode, it was crashing at exit because it was trying to free
the device extension.

This patch fixes the issue by only trigger the eBPF related code
when Suricata is running in AFP_PACKET mode.
5 years ago
Eric Leblond e6cfcb704c storage: fix a variable name 5 years ago
Eric Leblond 628458e7d3 detect: fix link to documentation 5 years ago
Philippe Antoine 43f25f127f ftp: ftp-data recognized by StringToAppProto 5 years ago
Philippe Antoine d861228214 http2: decompression for files
gzip and brotli decompression for files
5 years ago
Philippe Antoine 76db6e34a1 protocol detection: fix failure case
as reached by CIFuzz even if unreachable from Suricata
5 years ago
Philippe Antoine f5d8e953a8 protodetect: debug validation when multiple patterns match 5 years ago
Victor Julien 00d7c9034b stream: remove debug assert
In cases of large windows in the past the check would tigger.
5 years ago
Victor Julien b66d013294 detect/http_client_body: minor test cleanups 5 years ago
Eric Leblond 64f994f753 dataset: fix dataset string lookup
The data was unlocked but the use_cnt was not decreased resulting
in the data entry not being removable.
5 years ago
Victor Julien 191461a028 detect/file_data: cleanup tests 5 years ago
Victor Julien 116c089de0 stream/tests: minor cleanups 5 years ago
Victor Julien ee6d792b02 stream: move tests into tests/ 5 years ago
Victor Julien 226a82bade detect/fast_pattern: redo unittests 5 years ago
Victor Julien 66d7f5941a detect/fast_pattern: remove dead code 5 years ago
Victor Julien bc9e7743f3 detect/http-ua: cleanup tests 5 years ago
Emmanuel Thompson f12daa710f decode/flow/esp: Add ESP decoder & flow
- Adds an ESP (Encapsulating Security Payload) header decoder
- Tracks ESP flows via the SPI field
5 years ago
Victor Julien 9adeae07b1 decode: reformat REINIT macro 5 years ago
Victor Julien 3f4398cc90 decode: minor unittest cleanups 5 years ago
Victor Julien bf00285d0a proto/names: add SCTP if not defined in system
If SCTP is missing from /etc/protocols, add it manually.
5 years ago
Victor Julien c25afbccc1 json: remove unused jansson wrappers 5 years ago
Victor Julien b6b317cae6 http: enable and fix content range tests 5 years ago
Victor Julien a7cd765f20 app-layer/nfs: dead code removal 5 years ago
Juliana Fajardini 97350d9a2c detect/rpc: clean up unittests
- detect-rpc: convert unit tests to new FAIL/PASS API.
- detect-rpc: replace SigInit with DetectEngineAppendSig for more
  concise code.
5 years ago
Jason Ish 512b0350a0 lua: fix coverity issue with out of scope variable
Fix usage of out-of-scope variables. Introduced with the hashing
and adding the guard of g_disable_hashing.

To fix, just remove the guard so all variables are in scope. Hashes
are not initialized here so there is no need for the guard.
5 years ago
Jeff Lucovsky 1c68f4aed6 lua/test: Test cases using SC prefix
This commit adds paired test cases to ensure that the SC variant of the
entry points are tested.
5 years ago
Jeff Lucovsky c845974639 general: Correct typo 5 years ago
Jeff Lucovsky 431018d6f7 lua: Use SC prefix for Lua functions
This commit adds additional Lua API interfaces to bring consistency to
functions such that the `SC` prefix is available consistently across
flow int and flow var functions.
5 years ago
Victor Julien 3a8ba663a9 email/md5: optimize md5 handling 5 years ago
Jason Ish 6299222c4e email/eve: use Rust function to hash buffer to hex
Use SCMd5HashBufferToHex to hash the subject to a hex string.
Removes snprintf loop.
5 years ago
Jason Ish 0a3b9e0220 rust/hashing: add function to finalize md5 to hex string
New function, SCMd5FinalizeToHex to finalize an md5 hash
to a hex string.
5 years ago
Jason Ish 3a82153866 ja3: use SCMd5HashBufferToHex to print hash as hex
Replace snprintf loop with new function that hashes a single
buffer to an MD5 hex string.
5 years ago
Jason Ish e00d21a5cb filestore: respect g_disable_hashing
If g_disable_hashing is set, behave like libnss wasn't compiled
in.
5 years ago
Jason Ish 7525295e63 hashing: remove remaining HAVE_NSS guards
For features, we pretend to HAVE_NSS so scripts, external tests
continue to work.
5 years ago
Jason Ish 9b314bebe0 output-json-email: use Rust md5 bindings instead of libnss 5 years ago
Jason Ish 815396263b util/mime: use Rust md5 bindings instead of libnss
As the new Md5 hashing consumes its context on finalize, an bool
has_md5 flag has been added to let the logger know there is an
md5 hash available.
5 years ago
Jason Ish 9e14c00298 transform-sha256: use Rust sha256 bindings
Use new Rust sha256 bindings insead of libnss.
5 years ago
Jason Ish d98e34045b transform-sha1: use Rust sha1 bindings
Removes dependence on NSS.
5 years ago
Jason Ish 3d1a294bc7 util-crypt: remove ComputeSHA1
There is now a Rust replacement for ComputeSHA1 that is exposed
to C as SCSha1HashBuffer.
5 years ago
Jason Ish 90dcaff41d ssl: replace ComputeSHA1 with Rust SCSha1HashBuffer
Removes the final use of internal ComputeSHA1 function.
5 years ago
Jason Ish e0a2d10e7e transform/md5: use Rust md5 bindings 5 years ago
Jason Ish c2dd2c25b6 error: new disabled hashing error code
Add new error code to be used when something fails due to hashing being
disabled.
5 years ago
Jason Ish f723dc1309 suricata: add --disable-hashing option
The idea with a flag to disable hashing is to provide a way to
get the effect of building Suricata without libnss, which is
sometimes done for performance reasons.
5 years ago
Jason Ish 245a1bd4ce suricata.c: clang-format required after new option added 5 years ago
Jason Ish 7bb917b835 suricata.c: disable clang-format on long_opts init
clang-format will reformat this to have multiple options on a
single line which is hard to read and I don't see a specific
clang-format option to format this better. So for now,
disable clang-format on this section of code.
5 years ago
Jason Ish 0321f60ebf community-id: use Rust SHA1 to compute hash
Removes use of our internal ComputeSHA1 for calculating the
community ID.
5 years ago
Jason Ish d5bf748c90 ja3: remove requirement on NSS 5 years ago
Jason Ish e50ee7eb62 filestore: remove requirement of nss for filestore
Required including NSS header in places that depended on
util-file.h including it.

All filestore suricata-verify tests now pass without libnss.

Also enabled detect-file{md5,sha1,sha256} without NSS support.
5 years ago
Jason Ish e4acbcbb5e filestore: use Rust bindings for sha256/sha1/md5 5 years ago
Jason Ish f77fd0c0cb rust: include file cleanup
The cbindgen generated header should not include rust.h as
rust.h already includes the generated binding.

Fixup C source code that only pulled the generated include, it
should instead pull in "rust.h" which includes the generated
binding plus other misc. stuff.
5 years ago
Philippe Antoine 47dd9a5ebc signature: Fix leak in urilen parsing
cf #4254
5 years ago
Victor Julien 423a5e2f6c detect/file.data: support ftp and ftp-data 5 years ago
Victor Julien 45eddde573 detect/file.name: register inspect engine for ftp-data 5 years ago
Jeff Lucovsky f8fef0dd05 decode/tcp: Improved handling of TFO options
This commit improves handling of TCP fast open options
- Option length must be in [6, 18]
- Option length must be an even value
5 years ago
Gianni Tedesco 10ea60a237 detect: Validate that NOOPT options don't have optvals
Without this, a simple typo between : and ; is able to hide actual bugs
in rules.

I discovered 2 bugs in ET open ruleset this way.
5 years ago
Gianni Tedesco cebe15c23b detect-fast-pattern: Mark as OPTIONAL_OPT, instead of NOOPT
Also update the erroneous comment about it.
5 years ago
Eric Leblond 8d034b4163 eve/dhcp: avoid to call common logging twice 5 years ago
Joshua Lumb 9d432bbab3 threads: More descriptive startup output 5 years ago
Carl Smith 9b840104bd lua: Make the rule action available to output scripts
Useful for those that want to do custom logging from lua
5 years ago
Justin Ossevoort 320de5f43d eve: Log tenant_id for all eve-json messages 5 years ago
Carl Smith 660b68a083 nsh: Parsing of Network Services Header and payload
Support for EtherType 0x894F and basic header
5 years ago
Victor Julien db2dbaaf40 stream/midstream: handle packet loss after SYN/ACK 5 years ago
Victor Julien 895938080f stream/tcp: fix invalid ack events in timewait state 5 years ago
Victor Julien 49bd1f85b9 flow/timeout: fix TCP seq/ack for reversed flows
When a flow is swapped it also swaps the stream trackers, so it does
not make sense to reverse them during pseudo packet creation.
5 years ago
Victor Julien 3c7c361b60 detect/stream: fix async stream inspection
Move raw progress forward only if detect uses stream data, indicated
by the PKT_DETECT_HAS_STREAMDATA flag.
5 years ago
Victor Julien 8aa02c6d15 stream/tcp: fix async mode ACK validation 5 years ago
Philippe Antoine 8d659c6500 fuzz: improves sigpcap target with PacketPoolInit 5 years ago
Shivani Bhardwaj 6c19e7d0d4 counters: convert to FAIL/PASS API 5 years ago
Shivani Bhardwaj 7eef4e83e5 confyaml: convert to FAIL/PASS API 5 years ago
Shivani Bhardwaj fb311e3d9d applayer: convert to FAIL/PASS API 5 years ago
Shivani Bhardwaj bfd956ce67 applayer/htp/xff: convert to FAIL/PASS API 5 years ago
Shivani Bhardwaj b195ffbe18 applayer/ftp: convert to FAIL/PASS API 5 years ago
Shivani Bhardwaj 2e3d408e6f fastlog: convert to FAIL/PASS API 5 years ago
Janani Ramjee 2d3cf11906 detect/mark: convert unittests to use PASS/FAIL API 5 years ago
Juliana Fajardini 2765bff0c5 detect/msg: convert to FAIL/PASS API
Issue 4053. Adjust code formatting style (wrap long lines).
Replace SigInit with DetectEngineAppendSig.
5 years ago
Sumera Priyadarsini d4264748d8 detect: filesha1: convert unittests to use PASS/FAIL API
Currently, unit tests use integer values 1 and 0 to denote pass
and fail status of tests respectively. Modify the unit test
detect-filesha1 to use the PASS/FAIL API instead.
5 years ago
Juliana Fajardini d2144c0dfb detect-sid: fail/pass api
- convert unittests to new FAIL/PASS API.
5 years ago
Jason Ish f2ab5803fb dns: initialize log flags as an unsigned long long
On 64 bit all 64 bits were being initialized, but on 32 bit
only 32 bits were as it was being initialized as a long.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/4206
5 years ago
Victor Julien efc9a7a398 app-layer: remove callback for completion status
Since the completion status was a constant for all parsers, remove the
callback logic and instead register the values themselves. This should
avoid a lot of unnecessary callback calls.

Update all parsers to take advantage of this.
5 years ago
Victor Julien 84385549fe detect: remove unused arg from generic list inspect 5 years ago
Victor Julien d694dac5a8 detect: remove inspect v1 API 5 years ago
Victor Julien 92a8d8b4f4 detect/template: convert to v2 inspect API 5 years ago
Victor Julien 494f8f2700 detect/tls: convert to v2 inspect API 5 years ago
Victor Julien 71a508000b detect/ssh: convert to v2 inspect API 5 years ago
Victor Julien 86686eff17 detect/snmp: convert to v2 inspect API 5 years ago
Victor Julien 962230040a detect/rfb: convert to v2 inspect API 5 years ago
Victor Julien b11cc0fbcd detect/nfs: convert to v2 inspect API 5 years ago
Victor Julien 15c84bf3c6 detect/lua: convert to v2 inspect API 5 years ago
Victor Julien 5062023ffe detect/ftp: convert to v2 inspect API 5 years ago
Victor Julien 2320d62872 detect/mqtt: convert to v2 inspect API 5 years ago
Victor Julien 5611f9ddbd detect/modbus: convert to v2 inspect API 5 years ago
Victor Julien ce6f8605ea detect/enip/cip: convert to v2 inspect API 5 years ago
Victor Julien 35310313de detect/http2: inspect API v2 5 years ago
Victor Julien 67d45f7e3c detect/krb5: use inspect api v2 5 years ago
Victor Julien 197f99a134 detect/ftpbounce: switch to inspect api v2 5 years ago
Victor Julien 0f2c94ffea detect/files: remove unused prototypes 5 years ago
Victor Julien e0e4454db7 detect/files: inspect api v2 5 years ago
Victor Julien 48094b05c8 detect/file: minor cleanup 5 years ago
Victor Julien cd146adef0 detect/dnp3: convert to v2 inspect API 5 years ago
Victor Julien 9f8a8f368a detect/dcerpc: convert to v2 inspect API 5 years ago
Victor Julien db0665bccc detect/dns: convert to v2 inspect API 5 years ago
Victor Julien 8c7423fcb8 detect/app-layer-event: convert to v2 inspect API 5 years ago
Victor Julien c6c93d1d12 detect/stream: convert to inspect API v2 5 years ago
Victor Julien 51f38f6453 detect/payload: minor formatting fixup 5 years ago
Philippe Antoine 8c4e53e803 dnp3: regenerates C code with script 5 years ago
Shivani Bhardwaj ea0936199d output: use BASE64_BUFFER_SIZE macro
Base64Encode function requires the maximum length of the output string
as its last parameter. Use the macro BASE64_BUFFER_SIZE to calculate it
correctly.
5 years ago
Shivani Bhardwaj b0a6ed1e2a datasets/string: fix buffer overflow
The size of encoded_data array and the maximum output length parameter
to Base64Encode function were incorrect leading to buffer overflow for
certain cases. The algorithm requires at least 5 bytes of space to even
convert a string of length 1.

Use BASE64_BUFFER_SIZE macro to correctly calculate this output length.
Set size of encoded_data array to the calculated output length.
5 years ago
Shivani Bhardwaj 02942a123a util/crypt: Add macro for max base64encode len
Maximum length of a base64 encoded string can be 33% over the actual
length of the input string. The formula to best cover all the edge cases
is mathematically
(4 * (input_length + 2) / 3) + 1

Add a macro to calculate this for a given input length.
5 years ago
Philippe Antoine 057c4b34c8 ftp: optimize FTPGetOldestTx by starting from last handled tx
Avoids DOS by quadratic complexity algorithm.
Attack is
1 stack many requests/transactions (like cwd commands on a line)
2 get many answers
5 years ago
Victor Julien a8fc481cd3 http2: avoid null pointer deref in alert output
Bug #4120.
5 years ago
Philippe Antoine 096dce4bba http2: allow filestore to work with HTTP2 5 years ago
Philippe Antoine d5ac77ee8b packet: set length of 0 for too big copy 5 years ago
Philippe Antoine 6d2247391e ssl: improves keyword ssl_version parsing
Removes the use of PCRE for performance
Forbids empty negations after a valid pattern
Forbids mixing negative and positive forms as it is irrelevant
Forbids useless repetition of a version
5 years ago
Philippe Antoine 48f0c612e3 ssl: upgarde to uint32 for bytes_processed
as it can overflow, and is compared with uint32 such as
message_length and message_start and
5 years ago
Philippe Antoine 445e03d752 detect: null sanity checks for pkthdr
Even when the rules are only applied on traffic with the protocol
the structure for the protocol header can be set to NULL if there
was an error parsing the header
5 years ago
Philippe Antoine 33905b0910 fuzz: check PacketCopyData return value before processing packet 5 years ago
Philippe Antoine 18d8164a27 fuzz: limit memcap for datasets when fuzzing 5 years ago
Philippe Antoine 65e232669c dnp3: better limit for tests when fuzzing 5 years ago
Philippe Antoine 80dc6c6f1e fuzz: improves detect proto target
By mimicking assert message so as to clusterfuzz differentiates
between failures
5 years ago
Victor Julien 4664444067 detect: fix inspection order with stateful rules
When stateful detection rules, for which detection has already started
for a previous packet, are added to the candidates array, the array
is sorted to mantain the correct inspection order. However, due to a
trivial error in the sort helper the array was sorted in descending
instead of ascending order.
5 years ago
Victor Julien ba781265a4 dcerpc/udp: fix transaction handling and logging
Implement missing transaction handling.

Fix logging wrongly casting 'state' to DCERPCState instead of
DCERPCUDPState leading to crashes and malformed output.

Remove unused fields from DCERPCUDPState.
5 years ago
Victor Julien e7b5201016 detect/mpm: remove usused cleanup function 5 years ago
Victor Julien e799357d9f detect: optimize prefilter result handling 5 years ago
Victor Julien ffb0945b4c detect/prefilter: small cleanup 5 years ago
Victor Julien 14aacbd067 decode/null: fix type parsing 5 years ago
Jason Ish 76e011a5ba dnp3: set byte order when logging dnp3 src and dst
DNP3 uses little endian on the wire, for the most part this
is handled as the messages are deserialize. However, the link
header is a cast over raw data, so swap these bytes as they
are being logged.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/4173
5 years ago
Philippe Antoine 65711f6bc2 app-layer: do not try to parse gaps during protocol change
As this will leak the flow alstate because AppLayerParserParse
relies on FlowChangeProto to know if it should allocate a new
alstate
5 years ago
Philippe Antoine 89ae1a0036 detect: apply transforms to http body 5 years ago
Philippe Antoine 6b50a71d1a app-layer: lower limit for protocol detection on protocol change
So that protocol detection does not run for too long because
TCPProtoDetectCheckBailConditions somehow relies on its TCP stream
to start from zero, which is not the case on protocol change

Adds also debug validation checks, such as
both sides are known on protocol change

And only sets once alproto_orig
5 years ago
Jeff Lucovsky dabd50eeee detect: Register icmpv4 header 5 years ago
Jeff Lucovsky ac8532966b detect: Add icmpv4.hdr sticky buffer
This commit adds a new sticky buffer to access the ICMPv4 header.
5 years ago
Jeff Lucovsky 7cbe7c6463 detect: New enum for icmpv4 header keyword 5 years ago
Jeff Lucovsky 988bb26828 decode: Improved handling of ICMPv4 messages
This commit improves handling of ICMPv4 messages, especially those with
variable sized headers.

This commit also adds a header length variable for use by the new
sticky buffer for the header.
5 years ago
Jeff Lucovsky 828bf6d1d6 detect: Treat offset as a signed value
This commit updates the detector to treat 'offset' as a signed value to
be compatible with Snort.
5 years ago
Phil Young dc5349a30c napatech: Add Deprecation Warning Message for HBA
Added a message that HBA will be deprecated in the future and removed
hba from default conf file.
5 years ago
Jason Ish 8dbc774dfa dns: eve 1 deprecation warning
Redmine issue:
https://redmine.openinfosecfoundation.org/issues/4137
5 years ago
Jason Ish d8242c5d07 dns: fix leak in dns v1 logging
Intermediate JsonBuilder object was not being freed.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/4086
5 years ago
Phil Young fa77e02cf1 napatech: Removed restriction on use of inline mode
Removed the unnecessary restriction on the use of inline mode only when
bypass is enabled.  Now, Inline can be used independent of bypass
functionality.
5 years ago
Phil Young 789dcbd49d napatech: Fix potential double release of packet
This addresses readmine issue #4018.  There was the potential for a packet
buffer to be released twice in response to an error condition.  This
addresses this by only calling NT_NetRxRelease() when the p->ReleasePacket
is called.
5 years ago
Philippe Antoine 7d594d8710 fuzz: better packet recycling in sigpcap target
need especially to set datalink for every packet
5 years ago
Philippe Antoine c93cbe66c2 fuzz: applayer target uses yaml config
so that every app-layer protocol is enabled
5 years ago
Sascha Steinbiss ea2bc4c962 eve: do not access flow storage in packet context
We must make sure not to access the flow storage (e.g. keeping a
MacSet) before making sure we have a flow to begin with, We can,
for example, run into an alert without a flow with `ip` rules,
in which case the flow might be NULL. See Redmine issue #4109.
5 years ago
Jeff Lucovsky 68418a26db detect/file-data: Improved support for share bufs
This commit improves support for shared buffer usage, i.e., when
multiple rules share the file data (http) buffer and apply different
combinations of transforms and fast_patterns (or none).
5 years ago
Victor Julien 0eb692e500 mingw: fix format string warnings 5 years ago
Victor Julien ba2a9be11a detect/mpm: fix id confusion in mpm_ctx sharing
Mixing of dynamic id's and hardcoded config values could possibly
lead to the settings not getting applied properly.
5 years ago
Victor Julien 82d7f64630 detect/mpm: improve transforms handling
Make sure keywords with transforms get their own mpm ctx, instead of
sharing it with the 'pure' version of the keyword.
5 years ago
Victor Julien ee15bd8076 detect/mpm: rename variable to ease code review 5 years ago
Victor Julien 9f9c29a14a dcerpc: fix stream flag handling
Only hardcoded direction flags were passed to the parser, not the
full range.

Handle receiving an EOF flag w/o data.

Bug: #3856
5 years ago
Angelo Mirabella c72069c3b2 detect/magic: fix crash on rule reloading
This changseset fixes a bug causing a segmentation fault.
When rules are reloaded and a rule using libmagic matches, suricata
crashes due to an improper reinitialization of the thread contexts.

Bug: #3726
5 years ago
Angelo Mirabella ea15282f47 unix-socket: fix alert metadata logging
This changeset fixes a bug that was preventing suricata to dump
alert metadata info when running in unix-socket mode.
When running in unix-socket mode, suricata was skipping the
initialization of the output modules and, as a consequence,
the metadata output module was never invoked.
5 years ago
Victor Julien 62cc0c7acf alert/syslog: fix minor compile warning 5 years ago
Victor Julien 4e925ca260 stream: harden data size handling
Handle edge cases around ACKs and last_ack getting below
'app_progress', which can happen during shutdown of a flow
with multiple GAPs.
5 years ago
Victor Julien 5391746518 stream: bail early if no data after a gap 5 years ago
Victor Julien 27c9280735 stream: harden data size handling 5 years ago
Victor Julien 8a08fe82d4 stream: don't send STREAM_START multiple times 5 years ago
Victor Julien 479078d377 detect/content: add more negation tests 5 years ago
Victor Julien c41e64d637 dnp3: reword warning message 5 years ago
Victor Julien 36a93380a5 defrag: fix pthread warning on OpenBSD 5 years ago
Philippe Antoine 61d0cd1399 signature: checks protocol for file.name keyword
By setting the flags as for the filename keyword (not sticky version)
5 years ago
Philippe Antoine bde0c88984 dnp3: more precise probing for banners 5 years ago
Philippe Antoine 12dfc33e05 dnp3: fix probing test not using final null in string 5 years ago
Philippe Antoine 4706b38866 ssl: do not reuse struct session_id_length
As it can be confused between SSLv2 and TLSv13
In SSLv2, this variable is not used after the function scope, so
we can use a temporary variable.
5 years ago
Philippe Antoine f9c6b60edc detect: fix use of uninit memory in DetectDatarepParse
By initializing to empty string
5 years ago
Philippe Antoine 6b735a2e57 util: fix warning about format string for integer 5 years ago
Philippe Antoine dc7a9d2b46 util: THashInitConfig limits hash size when fuzzing 5 years ago
Philippe Antoine dcd5e4dec9 util: THashInitConfig does not exit but return error 5 years ago
Jason Ish 2b215a45e0 enip: set unidirection transaction flag 5 years ago
Jason Ish 60ebc27c4e app-layer: handle unidirectional transactions 5 years ago
Philippe Antoine 2eacc6a9a8 ssl: adds safety checks
Ensure the client version is valid by checking hello flags

Ensure no integer underflow occurs in SSLv3ParseHandshakeType
5 years ago
Jeff Lucovsky 4624e66cdd detect/transform: Validator for compress-ws
This commit adds a buffer validator for compress whitespace. Buffers
containing two or more consecutive whitespace characters are invalid
with this transform.
5 years ago
Jeff Lucovsky 30b1d7a9c1 detect: Correct typo 5 years ago
Jeff Lucovsky 7ab986250a detect/file-data: Apply transforms on file_data
This commit cause transforms to be applied on the sticky buffer.
5 years ago
Victor Julien e07a4393a9 http: error check htp_list_size
This avoids a potential casting to uint64_t of -1, leading to a very
high upper bound of the tx loop.
5 years ago
Jeff Lucovsky a75e9667ce log/eve: Threaded filename change: eve.N.json
This commit changes the name of the file used with threaded eve logging
to better support log rotation

Instead of using "eve.json.N" and creating potential issues with log
rotation (which also uses a ".N" suffix), the eve logs will be named
"eve.N.json" when threaded.
5 years ago
Jeff Lucovsky 73567272cd path: SCBasename - function to return basename
This commit returns the basename of a file, if it exists
in the same way that `basename(1)` works.
5 years ago
Jeff Lucovsky db9776af64 detect/rules: Fix copy/paste error
This commit corrects a copy/paste error handling the `include-mpm-stats`
configuration setting.
5 years ago
Jeff Lucovsky 41967a53ec detect: Apply clang-format updates 5 years ago
Jeff Lucovsky d911fb87b8 detect/rules: Increase array size to remove SEGV
This commit changes the size of reporting variables to be dynamic based
on the buffer ids in use instead of a fixed value to address a SEGV when
the fixed value was less than the max buffer/type id in use.
5 years ago
Philippe Antoine d57275fa64 protodetect: rerun probing parser if pattern matched
If a pattern matches in the other direction, after
probing parser finished without finding a protocol,
we will rerun the probing parser, which will include
the newly protocol found by its pattern
5 years ago
Philippe Antoine 5f348e9907 protodetect: runs probing parser on protocol found
It a protocol is found in a first direction, we should run the
probing parser, even if it is not in the known ports.

That can happen for HTTP2, where client magic is detected,
then server probe can be run
5 years ago
Philippe Antoine 518e0e66cb applayer: fix a leak in protocol change
TCPProtoDetect can either set f->alproto, change f->alstate
and return error.

When the original alstate gets freed, we shall set the pointer
to NULL, as it can get reused.
5 years ago
Jeff Lucovsky bb71eac9f9 log/anomaly: Move metadata out of anomaly section
This commit moves the metadata from the anomaly object where it was
incorrectly located.
5 years ago
Victor Julien daffcc6e53 app-layer: improve depth handling
Consider txs inspected and done for a direction after depth is reached for
that direction.
5 years ago
Victor Julien a5a46795bd stream: make sure to call app-layer in case of depth reached 5 years ago
Victor Julien 8b2886635f dcerpc/tcp: implement trunc logic
When one side of the connection reaches the STREAM_DEPTH condition the
parser should be aware of this. Otherwise transactions will forever be
waiting for data in that direction.
5 years ago
Victor Julien 4da0d9bdea applayer/rust: expose truncate callback 5 years ago
Victor Julien 2cfa35ccc9 app-layer: don't check tx detect flags if detect is disabled 5 years ago
Victor Julien 34e83b8acf stream: remove GAP flag from stream
This flag was checked in many places, but never set expect in 2 unittests.
5 years ago
Jeff Lucovsky c5ace81a27 log/eve: Rename fileinfo alert object to files
This commit changes the name of the "fileinfo" array in the alert object
to "files" to better support legacy use of "fileinfo" in reporting and
elsewhere.

The "fileinfo" event type is not an array while the alert "fileinfo"
member was.
5 years ago
Shivani Bhardwaj 98285177ba datasets: use default memcap 5 years ago
Philippe Antoine 872de829ea detect: initialize pointer before calling pcre_study
So as not to check an uninitialized value
Found by MSAN
5 years ago
Victor Julien f21a4bc40e datasets: remove experimental warning 5 years ago
Shivani Bhardwaj 1286b0a8f1 datasets: parse defaults section from yaml
Datasets can now have a global defaults setting in suricata.yaml. In
case the settings for memcap and hashsize are not find in the yaml or
rule, this shall be the fallback.

Example:

datasets:
  defaults:
    memcap: 100mb
    hashsize: 2048
  ua-seen:
    type: string
    load: datasets.csv
5 years ago
Shivani Bhardwaj 5ac94fc407 datasets: allow memcap, hashsize be set via yaml or rule
It is now possible to set the memcap and hashsize via suricata.yaml and
rules.

Rule example:

alert http any any -> any any (http.user_agent; dataset:isset,ua-seen,type string,load datasets.csv,memcap 100mb,hashsize 2048; sid:1;)

suricata.yaml example:

datasets:
  ua-seen:
    type: string
    load: datasets.csv
    memcap: 20mb
    hashsize: 2048
5 years ago
Shivani Bhardwaj b2482d6c60 datasets: allow max possible memcap while loading
While using the "load" option of datasets, it should be possible to load
any file from the disk, so set the limit to highest possible.
5 years ago
Jeff Lucovsky ce603d662f log/eve: Ensure eve logs have sequential suffixes
This commit ensures that the eve logs have sequential suffixes without
gaps.
5 years ago
Jeff Lucovsky ad2e18be3e atomics: Add "decl and init with value" function
This commit adds an interface to declare and initialize an atomic with a
specific value. This can help with situations where there's no defined
initialization path to set things up.
5 years ago
Jeff Lucovsky 8395a9201e log: Ensure threaded eve honors SIGHUP
This commit ensures that all logging contexts register for the file
rotation mechanism (SIGHUP and configured).
5 years ago
Philippe Antoine 9b5c923327 http: disables lzma by default for HTTP 5 years ago
Philippe Antoine 7011bddf84 http2: mimic HTTP1 request from upgrade 5 years ago
Philippe Antoine 9d1b030ff0 http2: first connection upgrade from http1 5 years ago
Philippe Antoine 9185a90fc9 dnp3: fix unit tests when fuzzing 5 years ago
Philippe Antoine 82f1758573 applayer: keep running detection on protocol change
ie do not stop on first try if we do not have enough data
5 years ago
Philippe Antoine 21e741795d applayer: on protocol change, use previous state 5 years ago
Philippe Antoine 828ff2dc3c http: removal of connect unit tests
moved to suricata-verify
5 years ago
Philippe Antoine 547d6c2d78 applayer: pass parameter to StateAlloc
This parameter is NULL or the pointer to the previous state
for the previous protocol in the case of a protocol change,
for instance from HTTP1 to HTTP2

This way, the new protocol can use the old protocol context.
For instance, HTTP2 mimicks the HTTP1 request, to have a HTTP2
transaction with both request and response
5 years ago
Jason Ish c4d0a61eca datasets: fix dataset load path construction
Test the full path instead of just the filename provided in the
rule to see if it exists.

Fixes the case where a rule file is loaded from a directory
other than the default-rule-directory.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/3916
5 years ago
Philippe Antoine 9788b2ec8d signature: frees transform options in SigMatchPrepare 5 years ago
Philippe Antoine 1674239442 detect: checks for overflow when comparing signatures priorities 5 years ago
Victor Julien abca451901 flow: suppress Coverity FP 5 years ago
Victor Julien 204302cbac flow: minor code cleanup 5 years ago
Victor Julien 42ce297e0e flow: turn BUG_ON into debug check 5 years ago
Eric Leblond 6494abc6b1 ebpf: fix invalid description in doc string 5 years ago
Victor Julien 6492fe0841 detect/app-layer-events: improve warnings/errors
Improve handling of outdated events that are no longer supported by the engine.
5 years ago
Victor Julien 8cd82486e2 flow/bypass: don't bypass on flow timeout pseudo packets 5 years ago
Jason Ish cc2c7b731a rule parsing: valid that input rule string is UTF8
Before parsing a rule string, validate that it is UTF-8 first.

Related Redmine issue:
https://redmine.openinfosecfoundation.org/issues/3850
5 years ago
Jason Ish 583593aa1b http/eve: use set_string_from_bytes where appropriate
The JsonBuilder set_string_from_bytes will escape unprintable
characters in the output stream. Using BytesToStringBuffer
can generate invalid UTF-8 which prevents the conversion from
a C string to a Rust string.
5 years ago
Victor Julien d6cf4b3335 detect/dataset: error if set couldn't be fully loaded 5 years ago
Victor Julien 017c038bcb datasets: free old data when reusing a hash container 5 years ago
Jeff Lucovsky 00f77f9643 log: Log errors while writing log info
This commit adds logic to log errors during output. Errors are logged
once and the number of errors is maintained.
5 years ago
Jeff Lucovsky b9458adf8a log: Add log output error code 5 years ago
Jeff Lucovsky 6cdd87f5ad log: Use unlocked variants of stdio functions
This commit uses the unlocked variants of additional stdio functions
5 years ago
Jeff Lucovsky 3c91e14352 log: Add more stdio_unlocked macros
This commit adds additional macros for interfaces in stdio_unlocked
according to their local availability.
5 years ago
Victor Julien fa0b91b18f error: reformat enum 5 years ago
Victor Julien fbdc776525 app-layer: handle parser return code issues more gracefully 5 years ago
Victor Julien 5155982ba5 pcap: fix minor scan-build warnings 5 years ago
Victor Julien aac2b9dbb9 commandline: add static analyzer hint for -r parsing 5 years ago
Victor Julien 71297f575f commandline: minor formatting fixes 5 years ago
Jeff Lucovsky 9efb936697 general: Improve grammar in error messages
This commit corrects a minor grammar issue in address/port error
messages.
5 years ago
Ali Jad Khalil 6be08637c3 decode/teredo: Modified/refactored Teredo logic
This is just a slight refactor to make analagous decoding/encapsulation
schemes - Geneve, Teredo, and VXLAN - be implemented as similarly as
possible.
5 years ago
Ali Jad Khalil 66452dd38a decode/vxlan: Modified/refactored VXLAN logic
This is just a slight refactor to make analagous decoding/encapsulation
schemes - Geneve, Teredo, and VXLAN - be implemented as similarly as
possible.
5 years ago
Ali Jad Khalil 5d955c1836 decode/geneve: Add Geneve decoding functionality
These changes are in response to feature request 3063. Geneve is
very similar to VXLAN, but uses a slightly different encapsulation
scheme.
5 years ago
Victor Julien 77bafe13a3 decode: reformat event table 5 years ago
Jason Ish 665328b29e plugins: require registration function SCPluginRegister
Instead of looking for a symbol, "PluginSpec" look for a function
named SCPluginRegister that returns a SCPlugin.

This makes it much easier to create Rust plugins without having
to deal with dlopen constructors and such, which is rather
straight forward in C, but a bit of advanced boilerplate in Rust
that can be eliminated by simply calling a registration function.
5 years ago
Jason Ish 335e4e728f suricata: expose the SuricataContext with a function
Expose the "SuricataContext" required by Rust as a function. During
normal startup we register this context with the Rust code, but
plugins written in Rust will need to get the same registration
done, but to do this in a plugin, the plugin code must
call and set the context within its address space.
5 years ago
Jason Ish 87a91e6aa8 logging: expose the log level with a function
The log level needs to exposed so Rust plugins can bootstrap
themselves with the correct login to SCLogNotice!(), etc work
as expected.
5 years ago