Commit Graph

9309 Commits (fa2a1385eafb1606bd49c1fcff4939f255fd81e6)

Author SHA1 Message Date
Victor Julien 5b9d17b485 atomics: remove unused macros 6 years ago
Victor Julien c83a607b6a atomics: add SC_ATOMIC_INITPTR macro
Until now both atomic ints and pointers were initialized by SC_ATOMIC_INIT
by setting them to 0. However, C11's atomic pointer type cannot be
initialized this way w/o causing compiler warnings.

As a preparation to supporting C11's atomics, this patch introduces a
new macro to initialize atomic pointers and updates the relevant callers
to use it.
6 years ago
Victor Julien 531ff3ddec atomics: change SC_ATOMIC_ADD to 'fetch_add'
Until this point the SC_ATOMIC_ADD macro pointed to a 'add_fetch'
intrinsic. This patch changes it to a 'fetch_add'.

There are 2 reasons for this:

1. C11 stdatomics.h has only 'atomic_fetch_add' and no 'add_fetch'
   So this patch prepares for adding support for C11 atomics.

2. It was not consistent with SC_ATOMIC_SUB, which did use 'fetch_sub'
   and not 'sub_fetch'.

Most callers are not using the return value, so these are unaffected.
The callers that do use the return value are updated.
6 years ago
Victor Julien 109b2ae551 atomics: avoid unnecessary (direct) CAS use 6 years ago
Victor Julien c660757153 atomics: remove useless SC_ATOMIC_DESTROY 6 years ago
Victor Julien 1cb7eec52d atomics: remove spinlocked fallback 6 years ago
Victor Julien 967340e901 fuzz: fix applayer eof check segv 6 years ago
Philippe Antoine 4fda7ed4bd fuzz: stop app layer target as Suricata
Before being overwhelmed by successive errors
6 years ago
Philippe Antoine fe1d36ec7e conf: returns instead of exiting in ConfYamlParse
So that we can keep on fuzzing even on too much recursion
6 years ago
Victor Julien dfdf2eb050 fuzz: add missing debug validation to configure 6 years ago
Victor Julien c76f98073e fuzz: add configure wrapper for oss-fuzz 6 years ago
Victor Julien 5e13816380 includes: don't include sys/types.h twice 6 years ago
Victor Julien df79613fb5 privs: include headers in suricata-common.h 6 years ago
Victor Julien 61c9e01f87 conf/yaml: include yaml.h after suricata-common.h 6 years ago
Victor Julien f6bf86f136 fuzz/sigpcap: enable all of eve 6 years ago
Victor Julien 4d50eb1647 detect/iponly: fix parsing of '0' valued netmask 6 years ago
Victor Julien d4613e5c70 util/mem: reduce scope of win32 specific include 6 years ago
Victor Julien 415c992909 util/mem: cleanup by moving atomic from mem hdr 6 years ago
Victor Julien 3b877929e3 util/mem: move most logic to functions
Reduce macro use and simplify code. Also reduces compiled code
size.
6 years ago
Victor Julien 48bb26abe7 util/mem: remove old debug code for counting allocs 6 years ago
Victor Julien 481a1923b4 logging: turn SCLog and SCLogErr into funcs
Reduces compiled code size.
6 years ago
Victor Julien 64e307936e common: add ATTR_FMT_PRINTF wrapper
Wraps around __attribute__((format(printf, (x), (y))))
6 years ago
Victor Julien a8c8e2d5c9 common: use suricata-common.h in more places 6 years ago
Victor Julien b856caad94 common: use WARN_UNUSED macro 6 years ago
Victor Julien f903766849 detect/mpm: don't process empty store 6 years ago
Victor Julien a95fa3c156 dns/tests: comment typo fixes 6 years ago
Victor Julien d5712efc91 decode: return bool network layer
So that the caller can set the correct event type on error.
6 years ago
Victor Julien 328a94206e decode/hdlc: initial support 6 years ago
Victor Julien 136d351e40 decode: single network layer entrypoint
This way new layers can be added in a single place.
6 years ago
Victor Julien 88bccfb80e decode: create linklayer entry point
Make AF_PACKET and PCAP mode use it.
6 years ago
Victor Julien 685d490d07 decode/ieee8021ah: fix possible packet truncation 6 years ago
Victor Julien 5404dc7f6d fuzz/siginit: cleanup detect engine every 1024 runs 6 years ago
Andreas Herz aaa604b4c6 app-layer-template: fix log typo 6 years ago
Jason Ish 4dc80a6e6f conf/yaml: limit recursion depth while paring YAML
A deeply nested YAML file can cause a stack-overflow while
reading in the configuration to do the recursive parser. Limit
the recursion level to something sane (128) to prevent this
from happening.

The default Suricata configuration has a recursion level of 128
so there is still lots of room to grow (not that we should).

Redmine ticket:
https://redmine.openinfosecfoundation.org/issues/3630
6 years ago
Victor Julien fc6ada8541 detect/parse: properly free bidir sigs in error path 6 years ago
Victor Julien 5abead9325 detect/parse: fix minor memory leak in error path
Only reachable on SCMalloc so should be unlikely to be reached.
6 years ago
Victor Julien 27186778b8 fuzz: allow uninitialized stats api 6 years ago
Victor Julien 794d9eeb83 fuzz: remove UNITTEST dependency
Expose UTH flow builder to new 'FUZZ' define as well. Move UTHbufferToFile
as well and rename it to a more generic 'TestHelperBufferToFile'.

This way UNITTESTS can be disabled. This leads to smaller code size
and more realistic testing as in some parts of the code things
behave slightly differently when UNITTESTS are enabled.
6 years ago
Jason Ish 4639dd7932 source/erf: validate record length before read
Check the ERF record length before attempting to read it as
a record length less than the size of the record header
is invalid.

Redmine ticket:
https://redmine.openinfosecfoundation.org/issues/3593
6 years ago
Victor Julien 960c52d7ff fuzz/sigpcap: initialize empty packet pool
Fixes runs with --enable-debug-validation. The target did not init a
packet pool, so for a tunnel packet would try to get a packet from
an uninitialized pool. In non-debug mode, this silently works by
falling back to a packet from alloc.

    (gdb) bt
    #0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
    #1  0x00007ffff35a6801 in __GI_abort () at abort.c:79
    #2  0x00007ffff359639a in __assert_fail_base (fmt=0x7ffff371d7d8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x555557fe7260 "!(pool->initialized == 0)",
        file=file@entry=0x555557fe7220 "tmqh-packetpool.c", line=line@entry=253, function=function@entry=0x555557fe7500 <__PRETTY_FUNCTION__.21181> "PacketPoolGetPacket") at assert.c:92
    #3  0x00007ffff3596412 in __GI___assert_fail (assertion=0x555557fe7260 "!(pool->initialized == 0)", file=0x555557fe7220 "tmqh-packetpool.c", line=253,
        function=0x555557fe7500 <__PRETTY_FUNCTION__.21181> "PacketPoolGetPacket") at assert.c:101
    #4  0x00005555577e24be in PacketPoolGetPacket () at tmqh-packetpool.c:253
    #5  0x0000555556914ecd in PacketGetFromQueueOrAlloc () at decode.c:183
    #6  0x00005555569161e1 in PacketTunnelPktSetup (tv=0x555559863980 <tv>, dtv=0x614000068e40, parent=0x61e0000fc080, pkt=0x61e0000fc470 "LL", len=72, proto=DECODE_TUNNEL_IPV4) at decode.c:286
    #7  0x00005555569de694 in DecodeIPv4inIPv6 (tv=0x555559863980 <tv>, dtv=0x614000068e40, p=0x61e0000fc080, pkt=0x61e0000fc470 "LL", plen=72) at decode-ipv6.c:59
    #8  0x00005555569e60b5 in DecodeIPV6ExtHdrs (tv=0x555559863980 <tv>, dtv=0x614000068e40, p=0x61e0000fc080, pkt=0x61e0000fc470 "LL", len=112) at decode-ipv6.c:522
    #9  0x00005555569e846f in DecodeIPV6 (tv=0x555559863980 <tv>, dtv=0x614000068e40, p=0x61e0000fc080, pkt=0x61e0000fc420 "cLL", len=255) at decode-ipv6.c:641
    #10 0x0000555556a032f9 in DecodeRaw (tv=0x555559863980 <tv>, dtv=0x614000068e40, p=0x61e0000fc080, pkt=0x61e0000fc420 "cLL", len=255) at decode-raw.c:70
    #11 0x0000555557659ba8 in DecodePcapFile (tv=0x555559863980 <tv>, p=0x61e0000fc080, data=0x614000068e40) at source-pcap-file.c:412
    #12 0x0000555556573401 in LLVMFuzzerTestOneInput (data=0x613000000047 "\241\262\315\064", size=339) at tests/fuzz/fuzz_sigpcap.c:158
    #13 0x0000555557a4dc66 in main (argc=2, argv=0x7fffffffdfa8) at tests/fuzz/onefile.c:51

That line:

    BUG_ON(pool->initialized == 0);
6 years ago
Todd Mortimer 944209592f detect/threshold: Add tests for thresholding by_rule and by_both. 6 years ago
Todd Mortimer 50e5b80463 detect/threshold: Add a common function to (re)allocate the by_rule threshold table.
Ensure that the by_rule threshold table is initialized if a rule
is thresholded by_rule. Replace manual table reallocaton with calls
to the common function.
6 years ago
Todd Mortimer 82dc61f4c3 detect/threshold: Refactor threshold calculation to handle by_rule and by_both.
The only difference between threshold calculations for by_src/by_dst,
by_rule or by_both is which table stores the DetectThresholdEntry.
Refactor the ThresholdHandlePacket* functions to do table lookup and
storage individually, but calculate thresholds in a common function.
6 years ago
Todd Mortimer 9fafc1031c time: Add TIMEVAL_EARLIER and TIMEVAL_DIFF_SEC macros.
Make it easy to compare 'struct timeval's and get their difference.
6 years ago
Todd Mortimer e945dea244 detect/threshold: Parse by_rule and by_both in rules.
Also add tests for parsing them.
6 years ago
Victor Julien ed8f48b053 app-layer/proto-detect: minor cleanup
Make sure the mask calculation is u32.
6 years ago
Victor Julien aba4e19548 detect/pktvar: fix memory leaks 6 years ago
Philippe Antoine 240df05af5 fuzz: limit input size for protocol detection consistency check 6 years ago
Jeff Lucovsky 6bffe0bd35 detect/ssl: Fix memory leak in version parsing
This commit fixes a memory leak in the SSL version handling that
manifests when the version identifier is incomplete or incorrect.
6 years ago
Philippe Antoine 91b2930891 fuzz: build compatibility with oss-fuzz flags
ie C define FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
6 years ago
Victor Julien 09a21545ce flow: cleanup expectations first
Make sure to cleanup expectations for a flow as the first step, before
parts of the flow itself are getting cleaned/freed.

Also indicate use unlikely as flows with expectations should be relatively
rare.
6 years ago
Eric Leblond fcfeeeb694 app-layer-expectation: update copyright date 6 years ago
Eric Leblond 1ddd77fae0 app-layer-expectation: clean expectation at flow end
When a flow timeout, we can have still existing expectations that
are linked to this flow. Given that there is a delay between the
real ending of the flow and its destruction by Suricata, the
expectation should be already honored so we can assume the risk
to clean the expectations that have been triggered by the
to-be-deleted flow.
6 years ago
Eric Leblond 6c9d1c0861 app-layer-expectation: limit number of expectations
This patch introduces a limitation in term of number of
expectations attached to one IPPair. This is done using
a circle list so we have a FIFO approach on expectation
handling.

Circleq list code is copied from BSD code like was pre existing code
in queue.h.
6 years ago
Eric Leblond 03e4bfeb02 app-layer-expectation: remove unused parameter 6 years ago
Jeff Lucovsky 0ae6b0b250 tests/bsize: Fuzzing test case added
This commit adds a test case to validate the issue found during fuzz
testing.
6 years ago
Jeff Lucovsky 5b38bc9894 detect/bsize: Ensure numeric values fit
This commit ensures that the numeric values will not exceed the size of
the containers used to hold them.
6 years ago
Victor Julien 095981cb2a detect/parse: fix crash on 'internal' keyword use
When keyword __flowvar__postmatch__, an internal keyword, is used
in a rule the 'Setup' func ptr will be NULL. This caused a crash.
6 years ago
Victor Julien 1e71eecf47 fuzz/siginit: fix leak in case of bidir sig 6 years ago
Victor Julien 5430141f7a fuzz/siginit: minor improvements
Enable detect engine 'quiet' mode to generate less output.

Set a fake filename so that datarep doesn't hit a reachable assert.
6 years ago
Victor Julien 13c9d0ca7e detect/pkt_data: error on unconsumed transforms
If a rule has transforms w/o consuming them (e.g. a content keyword),
don't consider 'pkt_data' valid.
6 years ago
Victor Julien e1c474a1b0 detect/pkt_data: code and test cleanup 6 years ago
Victor Julien 7f19da1cc0 detect: more robust against transform issues
In case of transform issues (transform not consumed before pkt_data
for example), the code would hit an ugly BUG_ON.

Address this by a more graceful error message, that will still
invalidate the sig but not crash the engine.
6 years ago
Jeff Lucovsky 2823bc5aed detect/tls: Use pcre_copy_substring to avoid leak
This commit eliminates a memory leak while parsing TLS version
information. The leak was identified through fuzzing.
6 years ago
Victor Julien 3d969a1c7d build: wrap fuzz targets in guard to fix 'make tags' 6 years ago
Victor Julien 8cbae1371f fuzz/sigpcap: fix FPs due to missing pkt cleanup 6 years ago
Victor Julien e97cdb48f3 decode/teredo: implement port support
Implement support for limiting Teredo detection and decoding to specific
UDP ports, with 3544 as the default.

If no ports are specified, the old behaviour of detecting/decoding on any
port is still in place. This can also be forced by specifying 'any' as the
port setting.
6 years ago
Shivani Bhardwaj 0e4f261224 Use StringParse* for all parsers and configurations 6 years ago
Shivani Bhardwaj c4c734541a Use appropriate ByteExtractString* functions 6 years ago
Shivani Bhardwaj 6b2c7d5be8 util: Add StringParse* functions
StringParse* functions would perform a stricter check compared to
ByteExtractString* functions. These new functions shall also check if
any extra characters follow the extracted numeric value in addition to
the checks performed by ByteExtractString* and return -1 in that case.
This is particularly important in parser, configuration and setup functions.
6 years ago
Philippe Antoine 293eebd999 fuzz: remove obsolete AFL code 6 years ago
Philippe Antoine bf60959d84 fuzz: simpler way to force usage of CXX linker 6 years ago
Philippe Antoine 440bb4d600 fuzz: remove decodeder fuzz target
As we removed decodeder function
6 years ago
Victor Julien e500c59b99 stream/tcp: fix STREAM_HAS_SEEN_DATA macro
The macro would not return true for smaller TCP streams, leading to
cases where the app-layer was not notified of EOF.
6 years ago
Victor Julien 1618fb1b97 stream/tcp: clean up stream flags 6 years ago
Pierre Chifflier 01aef49cbd rust/x509: map decoding errors to decoder events 6 years ago
Pierre Chifflier 333fcc43e7 ssl/tls: call rs_cstring_free for strings allocated in Rust 6 years ago
Pierre Chifflier 1d9f37a60e DER: remove the C parser for DER 6 years ago
Pierre Chifflier d92321d8b1 ssl/tls: use the rust decoder to decode X.509 certificates 6 years ago
Jeff Lucovsky e0bd79670c detect: byte-test convert neg_op flag to a bool
Only 8 flags are permitted so convert one of them to a struct member. I
choose neg_op
6 years ago
Jeff Lucovsky 313c23a26b detect: Add unittests to exercise bitmask 6 years ago
Jeff Lucovsky d12950c9e4 detect: fixup incorrect comments, indentation 6 years ago
Jeff Lucovsky 31ed9786f6 detect: byte_test impl for bitmask
This commit implements byte_test's bitmask feature.
6 years ago
Victor Julien b85539b2ab stream/tcp: fix fast open off by one
With data on SYN the sequence number used for the first data
was off by one, leading to the next segments to appear to come
after a one byte gap.
6 years ago
Philippe Antoine f51d7d8947 fuzz: check tcp splitting evasions in protocol detection 6 years ago
Philippe Antoine 9eddaa038e fuzz: enable AFLFUZZ_PERSISTANT_MODE for libfuzzer targets 6 years ago
Philippe Antoine ac35118ebe fuzz: use env variable to restrict app layer 6 years ago
Philippe Antoine 600b0d7c55 fuzz: adds eight fuzz targets
And ways to compile them with enable-fuzztargets at configure time
Adds utility function in util-unittest-helper
6 years ago
Frank Honza 1c8943dedd add RFB parser
This commit adds support for the Remote Framebuffer Protocol (RFB) as
used, for example, by various VNC implementations. It targets the
official versions 3.3, 3.7 and 3.8 of the protocol and provides logging
for the RFB handshake communication for now. Logged events include
endpoint versions, details of the security (i.e. authentication)
exchange as well as metadata about the image transfer parameters.
Detection is enabled using keywords for:

 - rfb.name: Session name as sticky buffer
 - rfb.sectype: Security type, e.g. VNC-style challenge-response
 - rfb.secresult: Result of the security exchange, e.g. OK, FAIL, ...

The latter could be used, for example, to detect brute-force attempts
on open VNC servers, while the name could be used to map unwanted VNC
sessions to the desktop owners or machines.

We also ship example EVE-JSON output and keyword docs as part of the
Sphinx source for Suricata's RTD documentation.
6 years ago
Victor Julien b4d75b7448 output/anomaly: minor code cleanups 6 years ago
Victor Julien 4d21b03575 detect/app-layer-event: code cleanups 6 years ago
Jeff Lucovsky f0bd69e7e3 detect/pcre: Correct spelling typos 6 years ago
Jeff Lucovsky 7f6af10fed general: copyright bump 6 years ago
Jeff Lucovsky 4b0085b03c detect: Update to take advantage of PCRE refactor
This commit changes the keyword detectors to use the refactored PCRE
modifications from detect-parse.[ch]
6 years ago
Jeff Lucovsky abe0cdc4ad detect/pcre: Changes to support pcre_jit_exec
This command causes `pcre_jit_exec` to be used when available. If it's
available and there are allocation errors preparing for it, things
fallback to `pcre_exec`.
6 years ago
Jeff Lucovsky aa67a0a236 detect/pcre: Add warning for failed registrations
This commit adds a warning used by the PCRE detect logic when it fails
to register initialization and free functions for per-thread JIT stack
handling.

This error code is only used when the platform has PCRE JIT exec
functionality.
6 years ago
Jeff Lucovsky d19429f7e5 detect/parse: Refactor interfaces/definitions
This commit refactors existing code patterns to reduce code duplication
and to be a base for supporting additional PCRE jit-related actions.
6 years ago
Philippe Antoine 1cd314c500 detect: adds icmpv6.mtu keyword 6 years ago
Philippe Antoine 75ec528384 detect: adds utility file for uint keywords 6 years ago
Philippe Antoine 0355b70f5a detect: define generic PrefilterIsPrefilterableById 6 years ago