aiden
/
suricata
mirror of https://github.com/OISF/suricata
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,968 Commits
8 Branches
165 Tags
229 MiB
main
main-7.0.x
main-8.0.x
master-6.0.x
master-5.0.x
master-4.1.x
master-4.0.x
master-3.2.x
suricata-8.0.2
suricata-7.0.13
suricata-8.0.1
suricata-7.0.12
suricata-8.0.0
suricata-7.0.11
suricata-8.0.0-rc1
suricata-8.0.0-beta1
suricata-7.0.10
suricata-7.0.9
suricata-7.0.8
suricata-7.0.7
suricata-7.0.6
suricata-6.0.20
suricata-7.0.5
suricata-6.0.19
suricata-6.0.18
suricata-7.0.4
suricata-6.0.17
suricata-7.0.3
suricata-6.0.16
suricata-7.0.2
suricata-6.0.15
suricata-7.0.1
suricata-6.0.14
suricata-7.0.0
suricata-7.0.0-rc2
suricata-6.0.13
suricata-6.0.12
suricata-6.0.11
suricata-7.0.0-rc1
suricata-6.0.10
suricata-6.0.9
suricata-7.0.0-beta1
suricata-6.0.8
suricata-6.0.7
suricata-6.0.6
suricata-5.0.10
suricata-6.0.5
suricata-5.0.9
suricata-6.0.4
suricata-5.0.8
suricata-6.0.3
suricata-5.0.7
suricata-6.0.2
suricata-5.0.6
suricata-6.0.1
suricata-5.0.5
suricata-4.1.10
suricata-4.1.9
suricata-5.0.4
suricata-6.0.0
suricata-6.0.0-rc1
suricata-6.0.0-beta1
suricata-5.0.3
suricata-4.1.8
suricata-4.1.7
suricata-5.0.2
suricata-4.1.6
suricata-5.0.1
suricata-5.0.0
suricata-5.0.0-rc1
suricata-4.1.5
suricata-5.0.0-beta1
suricata-4.1.4
suricata-4.1.3
suricata-4.0.7
suricata-4.1.2
suricata-4.1.1
suricata-4.1.0
suricata-4.0.6
suricata-4.1.0-rc2
suricata-4.1.0-rc1
suricata-4.0.5
suricata-4.1.0-beta1
suricata-4.0.4
suricata-4.0.3
suricata-4.0.2
suricata-3.2.5
suricata-4.0.1
suricata-3.2.4
suricata-4.0.0
suricata-4.0.0-rc2
suricata-3.2.3
suricata-4.0.0-rc1
suricata-4.0.0-beta1
suricata-3.2.2
suricata-3.1.4
suricata-3.2.1
suricata-3.2
suricata-3.2RC1
suricata-3.1.3
suricata-3.2beta1
suricata-3.1.2
suricata-3.1.1
suricata-3.1
suricata-3.0.2
suricata-3.1RC1
suricata-3.0.1
suricata-3.0.1RC1
suricata-3.0
suricata-2.0.11
suricata-3.0RC3
suricata-3.0RC2
suricata-3.0RC1
suricata-2.0.10
suricata-2.0.9
suricata-2.1beta4
suricata-2.0.8
suricata-2.0.7
suricata-2.1beta3
suricata-2.0.6
suricata-2.0.5
suricata-2.1beta2
suricata-2.0.4
suricata-2.1beta1
suricata-2.0.3
suricata-2.0.2
suricata-2.0.1
suricata-2.0.1rc1
suricata-2.0
suricata-2.0rc3
suricata-2.0rc2
suricata-2.0rc1
suricata-2.0beta2
suricata-1.4.7
suricata-1.4.6
suricata-1.4.5
suricata-2.0beta1
suricata-1.4.4
suricata-1.4.3
suricata-1.4.2
suricata-1.4.1
suricata-1.3.6
suricata-1.4
suricata-1.3.5
suricata-1.4rc1
suricata-1.3.4
suricata-1.4beta3
suricata-1.3.3
suricata-1.4beta2
suricata-1.3.2
suricata-1.4beta1
suricata-1.3.1
suricata-1.3
suricata-1.3rc1
suricata-1.3beta2
suricata-1.3beta1
suricata-1.2.1
suricata-1.2
suricata-1.2rc1
suricata-1.2beta1
suricata-1.1.1
suricata-0.8.2
suricata-1.0.0
suricata-1.0.1
suricata-1.0.2
suricata-1.0.3
suricata-1.0.4
suricata-1.0.5
suricata-1.1
suricata-1.1beta1
suricata-1.1beta2
suricata-1.1beta3
suricata-1.1rc1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '5b954212f7'
${ noResults }
Commit Graph

7852 Commits (5b954212f73dc4e9f838370667d087f91455ae6a)

Author SHA1 Message Date
Victor Julien cd0fb1ab1a dcerpc: cleanup, remove unused field 8 years ago
Victor Julien 31daf43579 afl: add dcerpc entry points 
Add for requests and mix of request/responses.

Implement storing the files to disk and rereading them.
 8 years ago
Victor Julien e5eb0bbe32 stream/bypass: more liberal policy if no detection 
The reason the stream engine can't easily decide to bypass streams
is that there can be non-stream dependent rules that wouldn't match
if bypassing is done too aggressively.

However, if there is no detection engine, there is no reason to hold
back. In this case we can bypass as soon as the stream engine is done
with a session.
 8 years ago
Eric Leblond cc82ef065c af-packet: optimize BPF 
This patch turn on code optimization on BPF filter building by
libpcap. This allow to reduce the size of the BPF bytecode and
thus increase the size of BPF filter supported by Suricata.

Reported-by: Martijn van Oosterhout
 8 years ago
Eric Leblond 2979a0a2e1 app-layer-ssh: trigger bypass when done 
Trigger bypass when application layer will not inspect anymore.
 8 years ago
Victor Julien dbd2d7c058 detect: more gracefully handle mpm prepare failure 
Exit with error instead of using the detection engine in a broken state.

Bug #2187
 8 years ago
Victor Julien e087d93883 detect: reject dsize rules that can't match 
Rules can contain conflicting statements and lead to a unmatchable rule.

2 examples are rejected by this patch:

1. dsize < content
2. dsize < content@offset

Bug #2187
 8 years ago
Victor Julien c02739e535 mingw: don't try to build unix socket 8 years ago
Victor Julien 8c31cd4bea win32: minor compile warning fixes 8 years ago
Victor Julien d1b6be99de mingw: fix random function 8 years ago
Victor Julien afed6fe4a2 cleanup: remove all uint use 8 years ago
Jason Ish 83c385a98f dnp3: use BasicSearch instead of memmem 
Mingw doesn't support memmem.
 8 years ago
Victor Julien 74f4f6dd63 gcc7: format-truncation fix for lua 8 years ago
Victor Julien df3a3c7857 der/asn1: limit recursion 
Limit the number of recursive calls in the DER/ASN.1 decoder to avoid
stack overflows.

Found using AFL.
 8 years ago
Victor Julien a306ccfd34 rust/nfs: implement events 
Remove lots of panic statements in favor of setting non-fatal events.

Bug #2175.
 8 years ago
Victor Julien 3e9b583d47 radix: fix risky malloc call 
GCC7 said:
  CC       util-radix-tree.o
In file included from util-debug-filters.h:29:0,
                 from util-debug.h:34,
                 from suricata-common.h:421,
                 from util-radix-tree.c:26:
util-radix-tree.c: In function ‘SCRadixAddKey’:
util-mem.h:177:12: error: argument 1 range [18446744071562067968, 18446744073709551615] exceeds maximum object size 9223372036854775807 [-Werror=alloc-size-larger-than=]
     ptrmem = malloc((a)); \
     ~~~~~~~^~~~~~~~~~~~~
util-radix-tree.c:749:42: note: in expansion of macro ‘SCMalloc’
             if ( (inter_node->netmasks = SCMalloc((node->netmask_cnt - i) *
                                          ^~~~~~~~
In file included from suricata-common.h:69:0,
                 from util-radix-tree.c:26:
/usr/include/stdlib.h:443:14: note: in a call to allocation function ‘malloc’ declared here
 extern void *malloc (size_t __size) __THROW __attribute_malloc__ __wur;
              ^~~~~~

scan-build said:
util-radix-tree.c:749:42: warning: Call to 'malloc' has an allocation size of 0 bytes
            if ( (inter_node->netmasks = SCMalloc((node->netmask_cnt - i) *
                                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./util-mem.h:177:14: note: expanded from macro 'SCMalloc'
    ptrmem = malloc((a)); \
             ^~~~~~~~~~~
1 warning generated.
 8 years ago
Victor Julien 5b84c01cd3 gcc7: fix format-truncation warnings in runmodes 
Example:

util-runmodes.c: In function ‘RunModeSetIPSAutoFp’:
util-runmodes.c:496:40: error: ‘snprintf’ output may be truncated before the last format character [-Werror=format-truncation=]
         snprintf(qname, sizeof(qname), "pickup%d", thread+1);
                                        ^~~~~~~~~~
util-runmodes.c:496:9: note: ‘snprintf’ output between 8 and 17 bytes into a destination of size16
         snprintf(qname, sizeof(qname), "pickup%d", thread+1);
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Solved by reducing 'thread' to a uint16_t and limiting the max
thread count to 1024.
 8 years ago
Victor Julien 7e72553ff8 gcc7: fix format truncation warning 
detect-rpc.c: In function ‘DetectRpcParse’:
detect-rpc.c:225:50: error: ‘%s’ directive argument is null [-Werror=format-truncation=]
                 SCLogError(SC_ERR_INVALID_VALUE, "invalid rpc option %s",args[i]);
                                                  ^
util-debug.h:239:77: note: in definition of macro ‘SCLogErr’
             int _sc_log_ret = snprintf(_sc_log_msg, SC_LOG_MAX_LOG_MSG_LEN, __VA_ARGS__);   \
                                                                             ^~~~~~~~~~~
detect-rpc.c:225:17: note: in expansion of macro ‘SCLogError’
                 SCLogError(SC_ERR_INVALID_VALUE, "invalid rpc option %s",args[i]);
                 ^~~~~~~~~~
 8 years ago
Victor Julien 16845d8c92 pcap-log: fix path construct check 8 years ago
Victor Julien 96b2e8afc0 gcc7: fixes for format string warnings 
GCC 7.1.1 on Fedora gave several warnings with -Wimplicit-fallthrough
and -Wformat-truncation

This patch addresses the warnings.
 9 years ago
Victor Julien 1236578a7c proto detect: improve 'failed' handling 
Don't try to call parser for 'failed'. Also don't set one direction
warning if TS is failed and our direction is unknown/complete so failed
as well.
 9 years ago
Victor Julien 8dd077943c ssl: minor code reformatting 9 years ago
Eric Leblond 091290dd1c app-layer: increment flow counter if one sided 
In the case of protocol like SMTP, we detect application layer on
only one side.  Consequence was a missed increment in the flow
counter.
 9 years ago
Victor Julien 5afe1a9814 stream: don't reset state on syn/ack resend 
Bug #1958.

The reset was originally created for issue #523, but that works
well without the reset as well.
 9 years ago
Victor Julien 62b6f9fe25 decode: add config option to disable teredo 
Ticket #744.
 9 years ago
Victor Julien 52b39a41e0 stats: print alert count at shutdown 
Bug #1855.
 9 years ago
Victor Julien 3c05379cbd detect: fix mix of pass and noalert 
Noalert rules did not apply pass logic to the flow.

Bug #1888.
 9 years ago
Victor Julien d459d0b352 lua/alert: expose transaction if available 
Bug #1748.
 9 years ago
Victor Julien 5781c8fc78 pcre: fix \xHH issue for http_host 
The http_host keyword checks if the regex contains uppercase characters.
This check was rejecting valid syntax in the following format:

    content:"|2E|suricata"; http_host; pcre:"/\x2Esuricata$/W";

This patch addresses this case.

Bug #1957.
 9 years ago
Victor Julien c18e950d7d isdataat: fix mem leak 9 years ago
Eric Leblond e9323810ad output-json-alert: fix error handling 
MemBuffer was not freed in case of allocation error.
 9 years ago
Eric Leblond 33d7f7d539 output-json-alert: log http body 
Add support for HTTP body logging as printable or as base64.
 9 years ago
Giuseppe Longo fff5f7f1f9 output-json-http: add functions to log http body 
This patch adds two functions that permits to log the http body
in printable or base64 format.
 9 years ago
Pierre Chifflier 08e4908dbe Add NULL-terminator to app-layer template (fix #1930) 9 years ago
Victor Julien 67b97c7948 log: destroy file mutex 9 years ago
Jason Ish fa742d1d14 log: wrap rotation and write in lock 
The application log is subject to rotation, so the check for
rotation, the actual rotation and write needs to be done under
lock to ensure the file pointer is in a consisten state
at the time of write().

Fixes issue:
https://redmine.openinfosecfoundation.org/issues/2155
 9 years ago
Pierre Chifflier efe11dc37e Add NTP parser (rust-experimental) 9 years ago
Victor Julien fee0fdc595 pcap: fix linktype raw issues 
On OpenBSD 6.0 and 6.1 the following pcap gets a datalink type of
101 instead of our defined DLT_RAW.

    File type:           Wireshark/tcpdump/... - pcap
    File encapsulation:  Raw IP
    File timestamp precision:  microseconds (6)
    Packet size limit:   file hdr: 262144 bytes
    Number of packets:   23
    File size:           11 kB
    Data size:           11 kB
    Capture duration:    7,424945 seconds
    First packet time:   2017-05-25 21:59:31,957953
    Last packet time:    2017-05-25 21:59:39,382898
    Data byte rate:      1536 bytes/s
    Data bit rate:       12 kbps
    Average packet size: 496,00 bytes
    Average packet rate: 3 packets/s
    SHA1:                120cff9878b93ac74b68fb9216027bef3b3c018f
    RIPEMD160:           35fa287bf30d8be8b8654abfe26e8d3883262e8e
    MD5:                 13fe4bc50fe09bdd38f07739bd1ff0f0
    Strict time order:   True
    Number of interfaces in file: 1
    Interface #0 info:
                         Encapsulation = Raw IP (7/101 - rawip)
                         Capture length = 262144
                         Time precision = microseconds (6)
                         Time ticks per second = 1000000
                         Number of stat entries = 0
                         Number of packets = 23

On Linux it is 12.

On the tcpdump/libpcap site the DLT_RAW is defined as 101:
http://www.tcpdump.org/linktypes.html

Strangely, on OpenBSD the DLT_RAW macro is defined as 14 as expected.
So for some reason, libpcap on OpenBSD uses 101 which seems to match
the tcpdump/libpcap documentation.

So this patch adds support for datalink 101 as RAW.
 9 years ago
Victor Julien 885b8cefec detect: fix crash when stream inspect runs on UDP 
Certain rules can apply to both TCP and UDP. For example 'alert dns'
rules are inspected against both TCP and UDP. This lead to the
stream inspect engine being called on a UDP packet.

This patch fixes the issue by exiting early from the stream inspect
engine if a) proto is not TCP or b) ssn is not available

Bug #2158.
 9 years ago
Victor Julien 6226338d5b eve/alert: redo option parsing 
Clean up option parsing. Allow options to be disabled as well as
enabled.

E.g.
    metadata: true
    flow: false

The metadata setting will enable all. Then flow is disabled.
 9 years ago
Eric Leblond 66da9d0ba6 output-json-alert: rename applayer to metadata 9 years ago
Eric Leblond da9005c404 output-json-alert: add app_proto or flow to events 
This patch adds a partial flow entry in the alert event
(if applayer or flow is selected) or simply app_proto if
it is not.

app_proto is useful as filter and aggregation field. And
the partial flow entry contains more information about the
proto as well as some volumetry info.
 9 years ago
Jason Ish 1ebad6bd71 rust/dns: cargo unit test prototype 
Do remove compiler warning when building without unit tests.
 9 years ago
Eric Leblond 4be031394b output-json-alert: fallback to payload if stream is void 
If stream logging results in no data then we fallback to payload
data to get somethingi that could be interesting  instead of
nothing.
 9 years ago
Eric Leblond eb70b1e195 detect-asn1: fix memory leak 9 years ago
Eric Leblond 170591a0b1 util-print: add 0 at end of buffer 
Add a 0 at the end of the printed buffer to be sure we terminate
with a 0 to avoid problem when calling strlen().
 9 years ago
Victor Julien d61fa0c43c tunnel: refactor tunnel verdict handling 
Observed:

STARTTLS creates 2 pseudo packets which are tied to a real packet.
TPR (tunnel packet ref) counter increased to 2.

Pseudo 1: goes through 'verdict', increments 'ready to verdict' to 1.
Packet pool return code frees this packet and decrements TPR in root
to 1. RTV counter not changed. So both are now 1.

Pseudo 2: verdict code sees RTV == TPR, so verdict is set based on
pseudo packet. This is too soon. Packet pool return code frees this
packet and decrements TPR in root to 0.

Real packet: TRP is 0 so set verdict on this packet. As verdict was
already set, NFQ reports an issue.

The decrementing of TPR doesn't seem to make sense as RTV is not
updated.

Solution:

This patch refactors the ref count and verdict count logic. The beef
is now handled in the generic function TmqhOutputPacketpool(). NFQ
and IPFW call a utility function VerdictTunnelPacket to see if they
need to verdict a packet.

Remove some unused macro's for managing these counters.
 9 years ago
Victor Julien ed706583e9 nfs: add nfs to alerts 
Also add a single 'applayer' option for alert augmentation that
applies to all app-layers.
 9 years ago
Victor Julien e8dae2e093 nfs: add to fileinfo events 9 years ago
Victor Julien e0c6565e68 nfs: nfs_version keyword 
Store nfs version in tx and add keyword to match on it.
 9 years ago
Victor Julien aff576b524 eve/nfs: log nfs version 9 years ago
Victor Julien 0d79181d78 nfs: rename nfs3 to nfs 
Since the parser now also does nfs2, the name nfs3 became confusing.
As it's still in beta, we can rename so this patch renames all 'nfs3'
logic to simply 'nfs'.
 9 years ago
Victor Julien 41376da03c nfs: log more rpc 9 years ago
Victor Julien 2a29f79960 nfs: fix rust data type declaration 9 years ago
Victor Julien 5153271b87 nfs2: basic record parsing and tracking 9 years ago
Victor Julien c7e10c73f9 nfs3: support NFS over UDP 9 years ago
Victor Julien d9f87cec3d nfs3: probing parsers in both directions 9 years ago
Victor Julien 58af39131f rust/nfs: handle GAPs 
In normal records it will try to continue parsing.

GAP 'data' will be passed to file api as '0's. New call is used
so that the file API does know it is dealing with a GAP. Such
files are flagged as truncated at the end of the file and no
checksums are calculated.
 9 years ago
Victor Julien 1a2985ed76 app-layer: remove checks 
Now that app-layer parser registrations are validated at startup,
a number of runtime checks are no longer necessary. So remove them.
 9 years ago
Victor Julien e930513125 app-layer: detect state registrations are mandatory 9 years ago
Victor Julien ed172985ca app-layer: validate registration 9 years ago
Victor Julien d090cd2edf dcerpc/udp: add missing tx support 9 years ago
Jason Ish 30be9f0b5d stream: don't do protocol detection on gap 
A gap notification has no data.

Also, break out the gap handling into its own code block to
simplify the conditional statements.
 9 years ago
Victor Julien b582cdef31 hyperscan: unittests compiler warning fixes 9 years ago
Eric Leblond 26eb49d721 bypass: add explicit flag in stream engine 
TCP reassembly is now deactivated more frequently and triggering a
bypass on it is resulting in missing some alerts due forgetting
about packet based signature.

So this patch is introducing a dedicated flag that can be set in
the app layer and transmitted in the streaming to trigger bypass.

It is currently used by the SSL app layer to trigger bypass when
the stream becomes encrypted.
 9 years ago
Jason Ish 70808a4f1d rust/dns: support gaps in TCP DNS 9 years ago
Jason Ish 26914cd59a rust/dns: copy over dns unit tests 
Only the tests that make sense were copied over, those testing
correlation of responses to requests were not.

Also, remove compiler warning when not built with
unit tests.
 9 years ago
Eric Leblond 5be44eb500 output-json-alert: don't decref used object 
In the unlikely case of a allocation error we will still use the
existing object so it should not be decref and freed.
 9 years ago
Eric Leblond 0c3a3101b1 alert-prelude: correctly set Source and Target 
IDMEF alert contains two entities named Source and Target that are
defined using common language:
* "The Source class contains information about the possible source(s) of
   the event(s) that generated an alert."
* "The Target class contains information about the possible target(s) of
   the event(s) that generated an alert."

Previous alerts event were not following that so we can updated the code
when we know the direction thanks to the metadata field.
 9 years ago
Eric Leblond f0e8062b2b alert-prelude: fix warnings on callback type 9 years ago
Eric Leblond 6af529d0c6 output-json-alert: output source and target 
Use metadata provided information to output the Source and Target
in the definition of IDMEF.

The output is now the following:

  "alert": {
    "action": "allowed",
    "gid": 1,
    "signature_id": 1,
    "rev": 1,
    "signature": "connection to home",
    "category": "",
    "severity": 3,
    "source": {
      "ip": "2001:31d0:000a:f68a:0000:0000:0000:0001",
      "port": 80
    },
    "target": {
      "ip": "2a01:0e34:ee97:b130:c685:08ff:dab3:c9c8",
      "port": 48390
    }
 9 years ago
Eric Leblond 97b89c0a54 detect-target: introduce new keyword 
The target keyword allows rules writer to specify information about
target of the attack. Using this keyword in a signature causes
some fields to be added in the EVE output. It also fixes ambiguity
in the Prelude output.
 9 years ago
Victor Julien 8ea9a5a7d6 common: sync PROG_VER version with configure.ac 9 years ago
Victor Julien 889a6de926 rust: add to features 9 years ago
Victor Julien efdbc91687 log: fix mem leak in error path (CID1404888) 9 years ago
Eric Leblond 7a17b4acf5 stream-tcp: use flags field to store inline info 9 years ago
Eric Leblond 85dab65414 stream-tcp: use flags field to store bypass info 9 years ago
Eric Leblond 050d8f788b af-packet: warn when tpacket_v3 is used in IPS 
Update yaml and add an error message.
 9 years ago
Eric Leblond ea9b9b5063 stream-tcp: add option to accept invalid packets 
Suricata was inconditionaly dropping packets that are invalid with
respect to the streaming engine. In some corner case like asymetric
trafic capture, this was leading to dropping some legitimate trafic.

The async-oneside option did help but this was not perfect in some
real life case. So this patch introduces an option that allow the
user to tell Suricata not to drop packet that are invalid with
respect to streaming.
 9 years ago
Eric Leblond 775e67459c filestore: avoid open write close sequence 
Current file storing approach is using a open file, write data,
close file logic. If this technic is fixing the problem of getting
too much open files in Suricata it is not optimal.

Test on a loop shows that open, write, close on a single file is
two time slower than a single open, loop of write, close.

This patch updates the logic by storing the fd in the File structure.
This is done for a certain number of files. If this amount is exceeded
then the previous logic is used.

This patch also adds two counters. First is the number of
currently open files. The second one is the number of time
the open, write, close sequence has been used due to too much
open files.

In EVE, the entries are:
 stats {file_store: {"open_files_max_hit":0,"open_files":5}}
 9 years ago
Eric Leblond 54718b306e filestore: add option to disable meta file writing 
As the fileinfo entry is containing the file_id it is enough to
have this entry to link the extracted file with metadata.
 9 years ago
Victor Julien d6592211d0 rust/nfs: NFSv3 parser, logger and detection 9 years ago
Victor Julien 71ddc43d49 rust/core: add file tx API call 9 years ago
Victor Julien a809f090d3 rust: filecontainer API 
Wrapper around Suricata's File and FileContainer API. Built around
assumption that a rust owned structure will have a
'SuricataFileContainer' member that is managed by the C-side of
things.
 9 years ago
Victor Julien f47fd2c243 rust/json: expose json_boolean 9 years ago
Jason Ish ba1a67e2cb rust: dns: add log filtering on rrtype 
While the filtering is still configured in C, the filtering
flags are passed into Rust so it can determine if a record
should be logged or not.
 9 years ago
Jason Ish b588b49779 rust: lua support for DNS based Rust 
Uses Rust wrappers around Lua to populate Lua
data structures.
 9 years ago
Jason Ish 73388042b2 rust: DNS app-layer. 
A DNS application layer in Rust. This is different than the
C based one, as it is partially stateless by not matching
up responses to replies.
 9 years ago
Jason Ish 94032d3ada rust: wrapper around C logging, and "context" 
Where the context is a struct passed from C with pointers
to all the functions that may be called.

Instead of referencing C functions directly, wrap them
in function pointers so pure Rust unit tests can still run.
 9 years ago
Jason Ish f6f126d53d rust: example of how an app-layer may be initialized 
Also shows basic usage of the configuration API from Rust.
 9 years ago
Jason Ish de5bb1f953 rust: stub out logging from rust 9 years ago
Jason Ish 8f81792da5 rust: hook rust into the build 
Rust is currently optional, use the --enable-rust configure
argument to enable Rust.

By default Rust will be built in release mode. If debug is enabled
then it will be built in debug mode.

On make dist, "cargo vendor" will be run to make a local copy
of Rust dependencies for the distribution archive file.

Add autoconf checks to test for the vendored source, and if it
exists setup the build to use the vendored code instead of
fetching it from the network.

Also, as Cargo requires semantic versioning, the Suricata version
had to change from 4.0dev to 4.0.0-dev.
 9 years ago
Victor Julien b332551dbb stream: suppress GAP notice message 9 years ago
Victor Julien fae822d82e netmap: minor output cleanup 9 years ago
Victor Julien 1fef8569a5 stream: minor output cleanup 9 years ago
Victor Julien f702e6c33d flow-hash: optimize to avoid branch misses 
Profiling showed a lot of branch misses in flow hashing. Restructure
code to avoid branching.
 9 years ago
Jason Ish c8ac479f90 dns: accept gaps in TCP DNS 
On gap notification a flag is set, on the next call the input
data is reprobed to make sure it can be processed.
 9 years ago
Jason Ish c862bbdc4b app-layer: notify parsers of gaps if enabled 
A parser can now set a flag that will tell the application
layer that it is capable of handling gaps. If enabled, and a
gap occurs, the app-layer needs to be prepared to accept
input that is NULL with a length, where the length is the
number of bytes lost. It is up to the app-layer to
determine if it can sync up with the input data again.
 9 years ago
Victor Julien dfff228f75 stream/async: more liberal RST acceptance 9 years ago
Victor Julien 5ea1d3bd53 stream/async: improvements for IPS 
Initialize midstream with async if enabled. Unset async on seeing
bidirectional traffic.

If only async-oneside is enabled, set ASYNC flag on session creation
when receiving a SYN packet.

Let last_ack stay in sync with next_seq so that various checks work
better.
 9 years ago
Victor Julien b767f1d306 output/streaming: fixes and redo tcp logging 
Fix HTTP body logging.

Redo TCP data logging based on the streaming API.
 9 years ago
Victor Julien f4c4ef12c0 stream: introduce optional 'log' progress tracker 
For logging streaming TCP data so far the individual segments where
used. However since the last big stream changes, the segments are
no longer the proper place for this. Segments can now have overlaps
etc.

This patch introduces a new tracker. Next to the existing 'app' and
'raw' trackers, the new tracker is 'log'. When the TCP logging is
used, a flag in the config is set and the log tracker is used to
determine how much of the stream window can be moved.
 9 years ago
Victor Julien 5b1d8c7e94 streaming/lua: add direction indication to streamer 
Call SCStreamingBuffer as follows:

    data, sb_open, sb_close, sb_ts, sb_tc = SCStreamingBuffer()

sb_ts and sb_tc are bools indicating the direction of the data.
 9 years ago
Victor Julien 35edc5264d luajit: cleanup states before return to pool 9 years ago
Victor Julien 888ed9db0d streaming: small code cleanup 9 years ago
Victor Julien 30a8b2def0 lua/streaming: fix http body logging 9 years ago
Victor Julien 0f4a9d740e flow-worker: improve no-flow case 9 years ago
Jason Ish 922a27ed97 detect-dns-query: use unit test macros 9 years ago
Victor Julien a636d96b15 detect/file: cleanups 
TX id is enfored in the engine, so the keywords don't need to.

Unify detect file engines.
 9 years ago
Victor Julien 9492a43060 output-tx: small cleanups and scan-build suppression 9 years ago
Jason Ish bb10bd7fca profiling: fix const compiler warnings 9 years ago
Victor Julien 43b0bc8c8e unix socket: improve runmode handling 
Improve output of unix mode in --list-runmodes

Honor the runmode commandline setting. Supported are 'single'
and 'autofp', with the latter still the default.
 9 years ago
Victor Julien 75edfcc2cd unix-socket: fix minor memleak 9 years ago
Eric Leblond 1655a67654 log-tlsstore: fix error handling 
In case of realloc error, the length of the encoding buffer was not
reset and this could result in trying to write to NULL pointer.
 9 years ago
Victor Julien 41e6c6dc36 flow: enforce 51 bits id globally 9 years ago
Victor Julien 71c6df1655 lua: add SCFlowId for getting the flow id 9 years ago
Victor Julien 8127730f00 bug 2113: fix live modes 9 years ago
Jason Ish c7ddbbc586 dns: fix log filtering 
Previously only a subset of the records could be selected
in custom. Now allow any to be selected.
 9 years ago
Victor Julien 4217c6839a stream: single GAP check 
Move all GAP checks into CheckGap. Remove seg_list based check.
Also remove seg_list == NULL check to make sure the Gap check is
done on an empty list as well.

Improve next_seq < last_ack check, but add data beyond gap check.
 9 years ago
Victor Julien 39183f7a8e doc: fix doc links for http keywords 9 years ago
Victor Julien c513896786 bug 2113: unix-socket start up race 9 years ago
Victor Julien 5b56d324c4 app-layer: optimize many-tx case 9 years ago
Victor Julien 4459b88782 output: tx logging optimizations 9 years ago
Victor Julien 5c01b40931 tests: update tests for app-layer changes 9 years ago
Victor Julien 3148ff34b6 app-layer API optimizations and cleanups 9 years ago
Victor Julien cd97fa80f1 file: fix pruning for parallel files 
Allow pruning of random files, not just list head.
 9 years ago
Victor Julien afedd5c6df file: fix storing parallel files 
When looping available files 'flags' misuse would lead to all files
being closed after the first close.

This patch separates per file and per call flags.
 9 years ago
Victor Julien ae99e08396 file: update loops to account for parallel files 9 years ago
Victor Julien c4c93872f8 file: introduce per file 'track id' 
Some protocols transfer multiple files in parallel. To support this add
a 'track id' to the API. This track id is set by the protocol parser. It
will use this id to indicate what file in the FileContainer it wants to
act on.
 9 years ago
Victor Julien 1062a9213b file-store: small cleanup 9 years ago
Victor Julien 944ab48b20 file: clarify file store id name 9 years ago
Victor Julien f18c976a8e flow: counters for total number of flows 
flow.tcp
flow.udp
flow.icmpv4
flow.icmpv6
 9 years ago
Victor Julien 312ad9e3ad pfring: compiler warning fixes 9 years ago
Victor Julien f6e3755b5c lua: extend SCFlowAppLayerProto 
Change SCFlowAppLayerProto to return 5 values:
<alproto> <alproto_ts> <alproto_tc> <alproto_orig> <alproto_expect>:

alproto: detected protocol
alproto_ts: detected protocol in toserver direction
alproto_tc: detected protocol in toclient direction
alproto_orig: pre-change/upgrade protocol
alproto_expected: expected protocol in change/upgrade

Orig and expect are used when changing and upgrading protocols. In a
SMTP STARTTLS case, orig would normally be set to "smtp" and expect
to "tls".
 9 years ago
Victor Julien 9c071d1724 eve.flow: log original and expected app_protocols 
Log protocols if they are available.
 9 years ago
Victor Julien 88177694fd nfq: don't try to verdict detect/log flush pkts 9 years ago
Victor Julien d9908216d8 connect/starttls: handle detection corner cases 
When switching protocol from http to tls the following corner case
was observed:

 pkt 6, TC "200 connection established"
 pkt 7, TS acks pkt 6 + adds "client hello"
 pkt 8 TC, acks pkt 7
 pkt 8 is where normally the detect on the 200 connection established
       would run however before detection runs the app-layer is called
       and it resets the state

So the issue is missed detection on the last data in the original
protocol before the switch.

Another case was:

TS ->    STARTTLS
TC ->    Ack "STARTTLS data"
         220
TS ->    Ack "220 data"
         Client Hello

In IDS mode, this made a rule that wanted to look at content:"STARTTLS"
in combination with the protocol SMTP 'alert smtp ... content:"STARTTLS";'
impossible. By the time the content would match, the protocol was already
switched.

This patch fixes this case by creating a 'Detect/Log Flush' packet in
both directions. This will force final inspection and logging of the
pre-upgrade protocol (SMTP in this example) before doing the final
switch.
 9 years ago
Victor Julien 6f42ae91c7 app-layer: protocol change API 
Add API calls to upgrade to TLS or to request a protocol change
without a specific protocol expectation.

If the HTTP CONNECT session includes a port on the url, use that to
look up the probing parser during protocol detection. Solves a
missed detection of a SSLv2 session that upgrades to TLSv1. SSLv2
relies on the probing parser which is limited to certain ports.

In case of STARTTLS in SMTP and FTP, the port is hardcoded to 443.

A new event APPLAYER_UNEXPECTED_PROTOCOL is set if there was a
mismatch.
 9 years ago
Mats Klepsland 72c757433a app-layer: add decoder event for missing TLS after STARTTLS 9 years ago
Mats Klepsland 11b9e6fdab app-layer-ftp: add STARTTLS support 9 years ago
Mats Klepsland 8125f78f5f app-layer-ftp: detect FTP alproto when using AUTH TLS 
Try to detect FTP using the patterns '220 (' and 'FEAT', since 'USER '
and 'PASS ' are not sent in cleartext when using AUTH TLS.
 9 years ago
Mats Klepsland 74aa65073b output-json-tls: log 'from_proto' field 
Log the original application level protocol when protocol have been
changed because of STARTTLS, HTTP CONNECT or similar.
 9 years ago
Mats Klepsland e8800b1893 app-layer-smtp: add STARTTLS support 9 years ago
Mats Klepsland b6c2b7052b app-layer-htp: add HTTP CONNECT support 9 years ago
Victor Julien 893f868b42 proto-detect: add debug output 9 years ago
Mats Klepsland b8d13f354b app-layer: support changing flow alproto 
Support changing the application level protocol for a flow. This is
needed by STARTTLS and HTTP CONNECT to switch from the original
alproto to tls.

This commit allows a flag to be set 'FLOW_CHANGE_PROTO', which
triggers a new protocol detection on the next packet for a flow.
 9 years ago
Victor Julien 9b1f74409b magic: fix compile warnings 9 years ago
Victor Julien 3ff5dc3653 nfq: remove obsolete and broken netfilterforwin support 9 years ago
Victor Julien ea99099c64 isdataat: add test for leading space 9 years ago
Victor Julien 6142e88ed5 nflog: compiler warning fix 9 years ago
Victor Julien ab1200fbd7 compiler: more strict compiler warnings 
Set flags by default:

    -Wmissing-prototypes
    -Wmissing-declarations
    -Wstrict-prototypes
    -Wwrite-strings
    -Wcast-align
    -Wbad-function-cast
    -Wformat-security
    -Wno-format-nonliteral
    -Wmissing-format-attribute
    -funsigned-char

Fix minor compiler warnings for these new flags on gcc and clang.
 9 years ago
Victor Julien 342059835f detect-parse: improve common parser 
In preparation of turning input to keyword parsers to const add
options to the common rule parser to enforce and strip double
quotes and parse negation support.

At registration, the keyword can register 3 extra flags:

    SIGMATCH_QUOTES_MANDATORY: value to keyword must be quoted

    SIGMATCH_QUOTES_OPTIONAL: value to keyword may be quoted

    SIGMATCH_HANDLE_NEGATION: leading ! is parsed

In all cases leading spaces are removed. If the 'quote' flags are
set, the quotes are removed from the input as well.
 9 years ago
Victor Julien 842dfbc3f8 detect: enforce isdataat:!1,relative earlier 
The expression 'isdataat:!1,relative' is used to make sure a match
is at the end of a buffer quite often. This patch optimizes this case
for 'content' followed by the expression. It enforces it by setting
and 'ends with' flag on the content and then taking that flag into
account while doing the pattern match.
 9 years ago
Victor Julien c0275c2b29 detect: more content inspection tests 9 years ago
Victor Julien 7eda6beade detect: don't rescan when just distance is used 
Content inspection optimization: when just distance is used without
within we don't need to search recursively.

E.g. content:"a"; content:"b"; distance:1; will scan the buffer for
'a' and when it finds 'a' it will scan the remainder for 'b'. Until
now, the failure to find 'b' would lead to looking for the next 'a'
and then for 'b' after that. However, we already inspected the
entire buffer for 'b', so we know this will fail.
 9 years ago
Victor Julien 84b97ca155 detect: content-inspection tests 
Add tests for the content inspection engine that count the number
of steps it takes to eval a rule.
 9 years ago
Victor Julien b9579fbe7d detect: avoid needless recursive scanning 
Don't recursively inspect a detect list if the recursion
doesn't increase chance of success.
 9 years ago
Victor Julien bc7c01ecbc detect: use BIT_U32 macro for content flags 9 years ago
Victor Julien c65a119cc0 debug: suppress notice message 9 years ago
Victor Julien 276125c1ef cleanup: remove unused ringbuffer code 9 years ago
Victor Julien cda6e0291f cleanup: remove libpcap < 1 support 9 years ago
Victor Julien d31cb083e9 detect: update tests that mix state/stream inspect 9 years ago
Victor Julien eb5857b68a unittests: add/improve helpers for stream/flow 9 years ago
Victor Julien 15dcac92f2 http_header: enable trailer prefilter engines 
Now that the mpm engines run only for the proper 'progress'
value, the trailing headers need their own engine registration.
 9 years ago
Victor Julien cf7f819888 state: check progress before calling engine 
Make sure progress of an inspect engine is available.
 9 years ago
Victor Julien 1bbf555318 detect: improve stateful detection 
Now that MPM runs when the TX progress is right, stateful detection
operates differently.

Changes:

1. raw stream inspection is now also an inspect engine

   Since this engine doesn't take the transactions into account, it
   could potentially run multiple times on the same data. To avoid
   this, basic result caching is in place.

2. the engines are sorted by progress, but the 'MPM' engine is first
   even if the progress is higher

   If MPM flags a rule to be inspected, the inspect engine for that
   buffer runs first. If this step fails, the rule is no longer
   evaluated. No state is stored.
 9 years ago
Victor Julien d1b7a83905 detect: change mask logic 
Previously the MPM/Prefilter engines would suggest the same rule
candidates multiple times.

For example, while processing the request body, the http headers
would be inspected by MPM multiple times.

The mask check was one way to quickly decide which rules could be
skipped.

Now that the MPM engines normally return a rule just once, this
mask check no longer makes sense. If the rule meets the ip/port/
direction based conditions, it needs to be evaluated if the MPM
said so. Even if not all conditions are yet true.

WIP disable mask as it no longer makes sense

WIP redo mask match
 9 years ago
Victor Julien a0fad6bb7f mpm: run engines as few times as possible 
In various scenarios buffers would be checked my MPM more than
once. This was because the buffers would be inspected for a
certain progress value or higher.

For example, for each packet in a file upload, the engine would
not just rerun the 'http client body' MPM on the new data, it
would also rerun the method, uri, headers, cookie, etc MPMs.

This was obviously inefficent, so this patch changes the logic.

The patch only runs the MPM engines when the progress is exactly
the intended progress. If the progress is beyond the desired
value, it is run once. A tracker is added to the app layer API,
where the completed MPMs are tracked.

Implemented for HTTP, TLS and SSH.
 9 years ago
Victor Julien d304be5bc3 detect: register progress in inspect engines 
Register required progress so we can stop inspecting as soon
as the progress isn't far enough yet.
 9 years ago
Victor Julien bc1698cfbe detect-state: don't use casts to uint 9 years ago
Victor Julien 53b21e5ee1 http_uri: unittest cleanup 9 years ago
Victor Julien 8d2f3b46e6 http_header: add another trailer test 9 years ago
Victor Julien 8d18be1fdb http_header (trailer) test cleanup 9 years ago
Victor Julien 1c46af477e ssh: fix test 9 years ago
Victor Julien a744d00f45 ssh: fix banner state setting 9 years ago
Victor Julien e3bd5f371d detect: more detailed state profiling 9 years ago
Victor Julien 6d562f3b5e app-layer: set stream-depth after stream init 9 years ago
Victor Julien 358e41b935 detect: clean up stateful detect 9 years ago
Victor Julien 9f4884a132 stream: reduce scope of new ssn func 9 years ago
Victor Julien 5c31f22e09 autotools: add src/tests to extra dist 9 years ago
Victor Julien 5a210984d5 stream: move inline tests 9 years ago
Victor Julien bea2b2c00c stream: list management cleanups 9 years ago
Victor Julien 34f7cb2b55 stream: debug improvements 9 years ago
Victor Julien aba9cd7d02 stream inspection: add debug counters 9 years ago
Victor Julien 2b433fab53 stream: pack config struct 9 years ago
Victor Julien 606f515fe9 stream: enforce gap earlier in app reassembly 9 years ago
Victor Julien 314516ffe2 stream: don't call app reassembly if disable flag set 9 years ago
Victor Julien 89af036336 stream: app-layer micro optimizations 9 years ago
Victor Julien 2f77302eeb stream: raw reassembly explicit disable raw handling 9 years ago
Victor Julien d6d7f65050 stream: mpm inspect micro optimizations 9 years ago
Victor Julien 7bddd0e168 stream: improve --disable-detection GAP handling 9 years ago
Victor Julien 6fefe70196 stream: remove unused StreamTcpGetStreamSize function 9 years ago
Victor Julien 422095668e stream: optimize session pruning 9 years ago
Victor Julien ee00a6f2ec stream: validate code 9 years ago
Victor Julien e1aba7d6c2 detect: only do flow dependent cleanup if a flow is present 9 years ago
Victor Julien 61c35d3c39 detect: make SigMatchSignatures void 
None of the callers cared for it's retval, so get rid of it.
 9 years ago
Victor Julien f49150ddb9 detect: turn single detect flag into bool 9 years ago
Victor Julien 6f76cbb870 detect: remove unused detect flag 9 years ago
Victor Julien 04b24cf24e stream: improve needs reassembly code 9 years ago
Victor Julien 55e19bfb89 stream: more aggressive StreamReassembleRawHasDataReady 9 years ago
Victor Julien bf3f3ce6b2 app-layer: change logic of setting 'no reassembly' 
Instead of killing all reassembly instantly do things slightly more
gracefully:
1. disable app-layer reassembly immediately
2. flag raw reassembly not to accept new data

This will allow the current data to be inspected still.

After detect as run the raw reassembly will be fully disabled and
thus all reassembly will be as well.
 9 years ago
Victor Julien de4f4e23a0 stream: new depth / disable raw logic 
Depth reach sets NOREASSEMBLY after detect.

No new raw sets NORAW after detect.
 9 years ago
Victor Julien 7c56c9ada0 stream: allow raw reassembly catch up 
If raw reassembly falls behind, for example because no raw mpm is
active, then we need to sync up to the app progress if that is
available, or to the generic tcp tracking otherwise.
 9 years ago
Victor Julien 89d0267df2 stream: detect stream GAP also during reassembly 9 years ago
Victor Julien 0c1ec17c92 debug-validation: add stream checks 9 years ago
Victor Julien 69519bda48 stream: StreamTcpReassembleRawCheckLimit cleanup 9 years ago
Victor Julien b099008b94 stream: handle no stream scanning case 
Now that detect moves the raw progress forward, it's important
to deal with the case where detect don't consider raw inspection.

If no 'stream' rules are active, disable raw. For this the disable
raw flag is now per stream.
 9 years ago
Victor Julien 0ef46a8fd2 stream: raw content inspection inline mode 
Implement the inline mode for raw content inspection. Packets
are leading, and when a packet's payload has been added to the
stream, the packet is inspected in the context of the stream.

Reassembly will return a buffer with the packet data with older
data in front of it and after it, if available.
 9 years ago
Victor Julien 149e324060 flow/stream: reduce/disable pseudo packet injections 
At flow timeout, we no longer need to first run reassembly in
one dir, then inspection in the other. We can do both in single
packet now.

Disable pseudo packets when receiving stream end packets. Instead
call the app-layer parser in the packet direction for stream end
packets and flow end packets.

These changes in handling of those stream end packets make the
pseudo packets unnecessary.
 9 years ago
Victor Julien 2d223b69cd stream: set 'trigger raw' per direction 9 years ago
Victor Julien 971ab18b95 detect / stream: new 'raw' stream inspection 
Remove the 'StreamMsg' approach from the engine. In this approach the
stream engine would create a list of chunks for inspection by the
detection engine. There were several issues:

1. the messages had a fixed size, so blocks of data bigger than ~4k
   would be cut into multiple messages

2. it lead to lots of data copying and unnecessary memory use

3. the StreamMsgs used a central pool

The Stream engine switched over to the streaming buffer API, which
means that the reassembled data is always available. This made the
StreamMsg approach even clunkier.

The new approach exposes the streaming buffer data to the detection
engine. It has to pay attention to an important issue though: packet
loss. The data may have gaps. The streaming buffer API tracks the
blocks of continuous data.

To access the data for inspection a callback approach is used. The
'StreamReassembleRaw' function is called with a callback and data.
This way it runs the MPM and individual rule inspection code. At
the end of each detection run the stream engine is notified that it
can move forward it's 'progress'.
 9 years ago
Victor Julien 564c0bd2c1 stream: constify StreamTcpReassembleRawCheckLimit 9 years ago
Victor Julien 0bff0de516 unittests: fail if TCP memory still in use 
abort() so test can be analyzed.
 9 years ago
Victor Julien 807312320f stream-tcp: implement thread pool for segments 
Config option:

stream:
  reassembly:
    segment-prealloc: 2048
 9 years ago
Victor Julien bd821f57f2 stream: implement memory handling functions 9 years ago
Victor Julien c2a5b9c393 stream: use static instead of dynamic streaming buffer structure 9 years ago
Victor Julien dd2b8bb298 stream: test cleanups and fixes 9 years ago
Victor Julien 8924653cd4 stream: add insert failure counters 9 years ago
Victor Julien f02dc377ef stream: add tcp.overlap and tcp.overlap_diff_data counters 9 years ago
Victor Julien 8c36e52d93 stream: improve no app and no raw case 9 years ago
Victor Julien 5ee36a0c8b stream: make raw_progress relative to STREAM_BASE_OFFSET 9 years ago
Victor Julien fa2a832022 stream: make app_progress relative to STREAM_BASE_OFFSET 9 years ago
Victor Julien 187e2381c8 stream: reduce space used for progress tracking 
Instead of the explicit base_seq_offset, use a macro instead. The
macro points to the stream buffer offset. The two were always
in sync.
 9 years ago
Victor Julien 9bd11bcda5 stream: small cleanups 9 years ago
Victor Julien b3e9d39771 stream: remove unused zero copy setting 9 years ago
Victor Julien bbb0df14d2 stream: safety check in overlap handling 9 years ago
Victor Julien 8c9f521707 tcp: streaming implementation 
Make stream engine use the streaming buffer API for it's data storage.

This means that the data is stored in a single reassembled sliding
buffer. The subleties of the reassembly, e.g. overlap handling, are
taken care of at segment insertion.

The TcpSegments now have a StreamingBufferSegment that contains an
offset and a length. Using this the segment data can be retrieved
per segment.

Redo segment insertion. The insertion code is moved to it's own file
and is simplified a lot.

A major difference with the previous implementation is that the segment
list now contains overlapping segments if the traffic is that way.
Previously there could be more and smaller segments in the memory list
than what was seen on the wire.

Due to the matching of in memory segments and on the wire segments,
the overlap with different data detection (potential mots attacks)
is much more accurate.

Raw and App reassembly progress is no longer tracked per segment using
flags, but there is now a progress tracker in the TcpStream for each.

When pruning we make sure we don't slide beyond in-use segments. When
both app-layer and raw inspection are beyond the start of the segment
list, the segments might not be freed even though the data in the
streaming buffer is already gone. This is caused by the 'in-use' status
that the segments can implicitly have. This patch accounts for that
when calculating the 'left_edge' of the streaming window.

Raw reassembly still sets up 'StreamMsg' objects for content
inspection. They are set up based on either the full StreamingBuffer,
or based on the StreamingBufferBlocks if there are gaps in the data.

Reworked 'stream needs work' logic. When a flow times out the flow
engine checks whether a TCP flow still needs work. The
StreamNeedsReassembly function is used to test if a stream still has
unreassembled segments or uninspected stream chunks.

This patch updates the function to consider the app and/or raw
progress. It also cleans the function up and adds more meaningful
debug messages. Finally it makes it non-inline.

Unittests have been overhauled, and partly moved into their own files.

Remove lots of dead code.
 9 years ago
Victor Julien 3fa2e8689c streaming: remove BUG_ON and other improvements 
Can be triggered by memory limits.
 9 years ago
Victor Julien d789dc7e6d streaming: add blocklist 
Add list of 'blocks'. This list contains offsets and lengths to
continuous data blocks. This is useful for TCP tracking where we
can have data gaps.

The blocks don't contain any data themselves, instead they contain
lenght and offsets. This way no extra copying is needed.

On inserting new data, existing blocks are expanded instead of
having multiple neighbouring blocks.
 9 years ago
Victor Julien 9e1470d81c stream-tcp: StreamTcpUTAddPayload unittest helper 9 years ago
Victor Julien 3411697106 profile: account flow-worker tcp-prune step 9 years ago
Victor Julien 595f6d1f26 detect: implement http_content_len sticky buffer 
This implements inspection of the Content-Length buffer as a content
sticky buffer.
 9 years ago
Victor Julien d96cbddbe4 detect: implement http_content_type sticky buffer 9 years ago
Victor Julien 46febef7cd detect: implement http_accept_enc sticky buffer 
Inspects Accept-Encoding header.
 9 years ago
Victor Julien 9d5bbc3af6 detect: implement http_accept_lang sticky buffer 
Inspects Accept-Language header
 9 years ago
Victor Julien 1d1176b0d5 detect: implement http_connection sticky buffer 9 years ago
Victor Julien 5714129e32 detect: implement http referer sticky buffer 9 years ago
Victor Julien 88cfb99910 detect: http_accept sticky buffer + common code 
Implement common code to easily add more per HTTP header detection
keywords.

Implement http_accept sticky buffer. It operates on the HTTP Accept
header.
 9 years ago
Ray Ruvinskiy 7539973109 tls: logging for session resumption 
We assume session resumption has occurred if the Client Hello message
included a session id, we have not seen the server certificate, but
we have seen a Change Cipher Spec message from the server.

Previously, these transactions were not logged at all because the
server cert was never seen.

Ticket: https://redmine.openinfosecfoundation.org/issues/1969
 9 years ago
Ray Ruvinskiy 0dc57c937e device: fix warning about NULL device 
Fix '[ERRCODE: SC_ERR_INVALID_VALUE(130)] - Name of device should not be null'
warning on start-up with pfring.

Ticket: https://redmine.openinfosecfoundation.org/issues/2097
 9 years ago
Victor Julien 29c8125758 redis: use SCCalloc to reduce risk of unitialized vars 9 years ago
fooinha a64e5e77c7 eve: async mode for redis output 
eve: detects libevent for async redis at configure
eve: moves redis output code to new file - util-log-redis.{c,h}
eve: redis ECHO and QUIT commands for async mode
eve: redis output defaults if conf is missing
 9 years ago
Victor Julien c3b4dd5a7d qa: add --no-random commandline option 9 years ago
Victor Julien 535d9e3529 random: convert stream and htp to new call 9 years ago
Victor Julien dd70b3fda0 random: improve random logic 
Improve random logic for hash tables.

Implement Windows random API if it is available.
 9 years ago
Victor Julien ec964ebf84 mingw: don't use uint type as mingw doesn't have it 9 years ago
Victor Julien 8f15e30a58 common: improve byte order and wordsize detection 9 years ago
Victor Julien a4dce24151 core dumps: check for sys/resource.h 9 years ago
Victor Julien 618f351210 cleanup: get rid of %llu format specifiers 
Use more explicit types instead.
 9 years ago
Jason Ish 9ff8882cbd detect: don't consider an empty rule file an error 9 years ago
Jason Ish 8436a892f9 logging: only do non-blocking writes if live 
If running against a pcap there is no reason to drop events,
a blocking socket is fine here. So only do non-blocking writes
when running off a live device.
 9 years ago
Jason Ish 673549e5cf eve: log number of events dropped at exit 9 years ago
Jason Ish 59b98649de logging: don't block on socket writes 
Writing to a unix socket can cause Suricata to block in the
packet path. This could happen if the read-endpoint of the
unix socket stays connected, but stops reading, or simply
can't read fast enough as part of its event processing.

To choose packets over events, do non-blocking socket
writes and drop the event if the write would block and
update a dropped counter.
 9 years ago
Mats Klepsland ffbf8cec16 logging: create log directories when needed 
Recursively create new log directories when needed. This makes it
possible to use date modifiers in the file path to create
directories based on date, e.g.:

  /var/log/suricata/2017/02/14/
 9 years ago
Mats Klepsland 47a5b493d7 output-json: rotate log file based on time 
Rotate log file based on time. Support both rotating based on a timer (XXs,
XXm, XXd, XXw) and rotating based on a absolute time, like each minute,
hour or day.
 9 years ago
Mats Klepsland db6c80fd8e logging: support date modifiers in log filenames 
Allow log filenames to contain date modifiers, e.g.:

  - eve-log:
    filename: eve-%Y-%m-%d-%H:%M:%S.json
 9 years ago
Jason Ish 845e18f25a defrag: (windows) detect more overlaps 9 years ago
Jason Ish 7922f9be1b defrag: (linux) fix an error in overlapping fragments 
If a subsequent fragment has a lower offset than a previous
one and overlaps, trim off the beginning of the previous
fragment.

Based on an issue reported privately.
 9 years ago
Jason Ish ca92be896e defrag: use new unit test macros 
Also reformat unit test functions to Suricata style.
 9 years ago
Jason Ish ff30d924db unix-socket: return failure on failure 
UnixManagerThreadInit needs to return a failure code if the socket
fails to initialize to avoid entering the UnixManager loop which
will continuously fail on the call to bind, as no listening
socket was setup.

This can occur when the socket fails to initialize due to a
permissions error and fatal init errors is not on.
 9 years ago
fooinha 20d4d40051 log: tls custom format log 9 years ago
fooinha af174c82bb log: common custom format output 9 years ago
Victor Julien d5a0eb4b2c geoip: fix compiler warning 
detect-geoip.c:78:40: error: incompatible pointer types assigning to 'int (*)(ThreadVars *, DetectEngineThreadCtx *, Packet *, const Signature *, const SigMatchCtx *)' from 'int (ThreadVars *, DetectEngineThreadCtx *, Packet *, Signature *, const SigMatchCtx *)' [-Werror,-Wincompatible-pointer-types]
    sigmatch_table[DETECT_GEOIP].Match = DetectGeoipMatch;
                                       ^ ~~~~~~~~~~~~~~~~
1 error generated.

Bug #2045
 9 years ago
Victor Julien b134dfcce7 modbus: fix compiler warnings about alignment 
app-layer-modbus.c:1226:39: warning: taking address of packed member 'transactionId' of class or structure 'ModbusHeader_' may result in an unaligned pointer value [-Waddress-of-packed-member]
    if (ModbusExtractUint16(modbus, &(header->transactionId), input, input_len, &offset)    ||
                                      ^~~~~~~~~~~~~~~~~~~~~
app-layer-modbus.c:1228:39: warning: taking address of packed member 'protocolId' of class or structure 'ModbusHeader_' may result in an unaligned pointer value [-Waddress-of-packed-member]
        ModbusExtractUint16(modbus, &(header->protocolId), input, input_len, &offset)       ||
                                      ^~~~~~~~~~~~~~~~~~
app-layer-modbus.c:1230:39: warning: taking address of packed member 'length' of class or structure 'ModbusHeader_' may result in an unaligned pointer value [-Waddress-of-packed-member]
        ModbusExtractUint16(modbus, &(header->length), input, input_len, &offset)           ||
                                      ^~~~~~~~~~~~~~
3 warnings generated.

Bug #2088
 9 years ago
Victor Julien e1bf48c0ee pool: fix compiler warning 
clang-4.0 reported:

util-pool.c:242:13: warning: logical not is only applied to the left hand side of this bitwise operator [-Wlogical-not-parentheses]
        if (! pb->flags & POOL_BUCKET_PREALLOCATED) {
            ^           ~
util-pool.c:242:13: note: add parentheses after the '!' to evaluate the bitwise operator first
        if (! pb->flags & POOL_BUCKET_PREALLOCATED) {
            ^
              (                                   )
util-pool.c:242:13: note: add parentheses around left hand side expression to silence this warning
        if (! pb->flags & POOL_BUCKET_PREALLOCATED) {
            ^
            (          )
util-pool.c:261:13: warning: logical not is only applied to the left hand side of this bitwise operator [-Wlogical-not-parentheses]
        if (! pb->flags & POOL_BUCKET_PREALLOCATED) {
            ^           ~
util-pool.c:261:13: note: add parentheses after the '!' to evaluate the bitwise operator first
        if (! pb->flags & POOL_BUCKET_PREALLOCATED) {
            ^
              (                                   )
util-pool.c:261:13: note: add parentheses around left hand side expression to silence this warning
        if (! pb->flags & POOL_BUCKET_PREALLOCATED) {
            ^
            (          )
2 warnings generated.
 9 years ago
Victor Julien 6227d0955f af-packet: fix cppcheck false positive 
[src/source-af-packet.c:1903]: (error) Resource leak: fd
 9 years ago
Victor Julien 3726fd66be http: fix body tracking corner case 
In some cases, observed with inspect limits 0, the body tracking could
get confused. When all chunks were already freed, a new chunk would
be considered to be the start of the body. This would overwrite the
bodies 'content_len_so_far' tracker, instead of adding to it. This in
turn could lead to a assertion abort in the inspection code.

This patch redoes the append code to always add the current lenght. It
cleans up the code to remove redundant logic.

Issue: https://redmine.openinfosecfoundation.org/issues/2078
Reported-By: Jørgen Bøhnsdalen
 9 years ago
Jason Ish 3ca1a29bbd defrag: fix argument used in macro to match signature 
"p" was being used in the macro but was not an argument to
the macro, but it worked due to the context of the macro.

Use the actual macro argument, d2, instead of p.

Results in no change to generated code.
 9 years ago
Mats Klepsland 365aa1f3e5 lua: add SCFlowTimestamps function 
Add SCFlowTimestamps() to return startts and lastts as seconds and
microseconds from flow.

Examples:

  startts, lastts = SCFlowTimestamps()
  startts_s, lastts_s, startts_us, lastts_us = SCFlowTimestamps()
 9 years ago
Mats Klepsland 005a700e54 logging: support custom file permissions 
Support setting file permissions per logger using 'filemode', e.g.:

  outputs:
    - eve-log:
        enabled: yes
        filetype: regular
        filename: eve.json
        filemode: 660
 9 years ago
Jason Ish 63078909d5 ipv4: update checksum function to be like tcp/udp 
Update the IPv4 checksum function to be like the
changed TCP/UDP checksum functions for consistency.
 9 years ago
Jason Ish b79a18ea15 tcp/udp: rename checksum functions for better meaning 
The TCP/UDP checksum functions no longer just calculate
the checksum, they can validate as well as calculate so
use a more generic name.
 9 years ago
Jason Ish f56428d996 tcp/udp: fix checksum validation when 0xffff 
Issue:
https://redmine.openinfosecfoundation.org/issues/2041

One approach to fixing this issue to just validate the
checksum instead of regenerating it and comparing it. This
method is used in some kernels and other network tools.

When validating, the current checksum is passed in as an
initial argument which will cause the final checksum to be 0
if OK. If generating a checksum, 0 is passed and the result
is the generated checksum.
 9 years ago
Victor Julien b58127edcb print: constify input 9 years ago
Victor Julien 44bab885d6 detect: fix ssl_state test 9 years ago
Victor Julien 4f8eacdc69 disable-detect: fix needless file hashing 
When detection is running flags are set on flows to indicate if file
hashing is needed. This is based on global output settings and rules.

In the case of --disable-detection this was not happening, so all
files where hashed with all methods. This has a significant
performance impact.

This patch adds logic to set the flow flags in --disable-detect mode.
 9 years ago
Victor Julien e24eb0f2b1 app-layer: remove version logic 9 years ago
Victor Julien a56b4817ef flow: remove unused alversion fields 9 years ago
Victor Julien c066f043a5 detect: remove unused alversion logic 9 years ago
Victor Julien 6c6802b957 detect: simplify state detect code: remove unused params 9 years ago
Victor Julien c961da2dce detect: clean up test 9 years ago
Victor Julien 3626ecb474 bytejump: don't print errors when matching 
When bytejump was told to convert some payload data to int from a
string it would print an error to the screen if the conversion
failed. This is unwanted as the payload is controlled by an attacker
and printing is expensive.
 9 years ago
Victor Julien 15d26f14e1 file-store: fix force store 9 years ago
Victor Julien dbbf185173 app-layer: fix gap handling in protocol detection 
A GAP during protocol detection would lead to all reassembly
getting disabled, so also the raw reassembly. In addition, it
could prevent the opposing side from doing protocol detection.

This patch remove the 'disable reassembly' logic. Stream engine
will take the stream with GAP and app-layer will make the proto
detection as complete.
 9 years ago
Victor Julien f1e19b5dd4 file store: store multiple files if available 9 years ago
Victor Julien 7c0a3b5582 app-layer: fix memleak on bad traffic 
If state was alloc'd after protocol detection, but then the direction
turned out to be wrong, the state would not be freed.
 9 years ago
Victor Julien 132e0e21f0 ippair: fix xbits unset memleak 9 years ago
Victor Julien 4c597bb9e1 ippair: use both addresses in hash 9 years ago
Victor Julien 1de1413863 stream: validate SACK right edge to be in window 9 years ago
Victor Julien daf39b422b stream: remove unused stream config member 9 years ago
Victor Julien 7512949670 stream: make data pointer in StreamSegmentCallback const 9 years ago
Victor Julien 2a5b9eb5fa stream: remove unused variable 9 years ago
Victor Julien a86b7b7042 app-layer: minor debug improvement 9 years ago
Jason Ish 6585ac4a9f logging: remove unused print stats callbacks 
Remove the ThreadExitPrintStats callback from logging modules
that weren't doing anything with it.
 9 years ago
Jason Ish 0c3f1e2974 logging: move lock into write function 
All loggers were wrapping just the write in a lock with some
updating a counter.  This moves the lock into the write function.

The log_ctx alerts counter was also removed as many modules have
stopped using this and the alert count is available elsewhere.

Should satisfy Coverity CID 1400798:

CID 1400798 (#1 of 1): Data race condition (MISSING_LOCK) 2.
missing_lock: Accessing log_ctx->rotation_flag without holding lock
LogFileCtx_.fp_mutex. Elsewhere, "LogFileCtx_.rotation_flag" is accessed
with LogFileCtx_.fp_mutex held 4 out of 5 times.

Which appears to be a false positive as all calls to SCLogFileWrite
were done under lock, but this will make it more explicit.
 9 years ago
Jason Ish ddf1bf6518 unified2: unlock using same dereference as lock 
Addresses Coverity CIDs:
    1400797
    1400796

Note that the mutex was actually being unlocked, but
from a different variable pointing to the same mutex.
 9 years ago
Mats Klepsland d6508e640a detect: add (mpm) keyword tls_cert_serial 
Match on TLS certificate serial number using tls_cert_serial
keyword, e.g.:

alert tls any any -> any any (msg:"TLS cert serial test";
        tls_cert_serial; content:"5C:19:B7:B1:32:3B:1C:A1";
        sid:12345;)
 9 years ago
Mats Klepsland 95864375f2 lua: add function to print certificate serial number 
Add function LuaGetCertSerial to print serial number from TLS
certificate.

Example:

function log (args)
    serial = TlsGetCertSerial()

    if serial then
        file:write(serial .. "\n");
        file:flush()
    end
end
 9 years ago
Mats Klepsland 2e339aa791 tls-log: log certificate serial number 9 years ago
Mats Klepsland 2b460b8d06 output-json-lua: log certificate serial number 9 years ago
Mats Klepsland 2c1a36dd6e app-layer-tls: decode certificate serial number 9 years ago
Alexander Gozman 310b27a196 af-packet: fix parsing packet in TPACKET_V3 mode 
AFPParsePacketV3() saved tpacket_block_desc structure
instead of tpacket3_hdr. As a result, reconstructed
packets were wrong.

Bug #2044.
 9 years ago
Alexander Gozman e41a9d637b af-packet: get VLAN info for packets in TPACKET_V3 mode 9 years ago
Alexander Gozman ee7e689b54 af-packet: write VLAN info for both TPACKET_V2 and V3 9 years ago
Jason Ish 92db12c349 pidfile: fail if the pid file exists 
Changes the pidfile check function to fail just on the
existence of the file to address issue
https://redmine.openinfosecfoundation.org/issues/1335
but log a message if the pid file appears to be stale.
 9 years ago
Mats Klepsland bbd32c1419 output-json-tls: code cleanup 9 years ago
Mats Klepsland 115b3138cc output-json-tls: log certificate and chain 
Log entire certificate and certificate chain Base64 encoded.
 9 years ago
Mats Klepsland 0716199acb output-json-tls: custom tls logging 9 years ago
Jason Ish 82f6103149 unified2: nostamp and file rotation 
Give unified2 a nostamp option which will create the file
without the timestamp suffix (like Snort's nostamp option).

Also register for rotation notification on SIGHUP so the file
will be recreated if it is removed by an external rotation
program (only when nostamp is used).
 9 years ago
Jason Ish 2c01985e73 autoconf - look for stdbool.h 9 years ago
Jason Ish d8e9e1fd77 unified2: minor cleanups 
- remove unused arguments and make static
 9 years ago
Mats Klepsland 234a3a890b output-json-alert: print 'tunnel' JSON object if tunnel 
Log src_ip, dst_ip and proto for root packet (p->root) if the
packet that triggered is inside a tunnel, as JSON object
'tunnel'. Also log recursion depth to indicate the depth of
the tunnel.
 9 years ago
Mats Klepsland 7293286f2f output-json: move code to get 5-tuple to own function 
Move code to get 5-tuple in JSON object to own function 'JsonFiveTuple'.
This enables this code to be reused when printing 'parent' JSON object in
output-json-alert.
 9 years ago
Mats Klepsland 8c8db1cf0f output-json-alert: fix wrongful comments 9 years ago
Mats Klepsland 65317ba865 output-json: make JSON flags in eve-log user configurable 9 years ago
Victor Julien d445b4b5fa dce: remove commented out code 9 years ago
Victor Julien 9da7be81f0 flow-worker: clean up thread init 9 years ago
Victor Julien c859d39f4f coverity: suppress CID 1400648 9 years ago
Victor Julien 955c227127 detect-ssh: cleanup duplicate code 9 years ago
Victor Julien 2f30adb08a detect-lua: setup cleanup, fixing a potential int issue 9 years ago
Victor Julien 511e804915 detect: fix missing unlock in error path 9 years ago
Victor Julien f74eff9eac threads: address sleep under lock issue 9 years ago
Victor Julien f380871057 threads: don't sleep under lock 9 years ago
Victor Julien cc4010343d detect: add and use util func for alproto sets 9 years ago
Victor Julien 4a49260897 flowvar: shrink flowvar type by using padded space 9 years ago
Victor Julien 99517cbd53 lua: support key/value flowvars in lua 9 years ago
Victor Julien f0af133c5f flowvar: remove unused DETECT_VAR_TYPE_ALWAYS 9 years ago
Victor Julien 88888c3d8b unix-socket: add/list/remove hostbit commands 
add-hostbit adds a named hostbit with an expire time in seconds.
remove-hostbit removes hostbit by name.

add-hostbit, remove-hostbit return success or failure.

list-hostbit returns a json array of hostbits with their name and
expire time:

    {
        "message": {
            "count": 1,
            "hostbits":
                [{
                    "expire": 3222,
                    "name": "firefox-users"
                }]
        },
        "return": "OK"
    }
 9 years ago
Victor Julien b6e4276792 hostbits: add list API 9 years ago
Victor Julien 996112edf5 pktvars: same name pktvars, key-value vars 9 years ago
Victor Julien 5ca4a2e6fe outputs: vars log 
EVE addition called 'vars' that logs pkt/flow vars for each packet/flow.
 9 years ago
Victor Julien 1a2ad059a1 eve: log pktvars/flowvars/bits/ints 
Optionally logs 'vars' into alerts
 9 years ago
Victor Julien 1ba8c2fe3a pcre: new way of specifying var names 
Until now the way to specify a var name in pcre substring capture
into pkt and flow vars was to use the pcre named substring support:
e.g. /(?P<pkt_somename>.*)/

This had 2 drawbacks:

1. limitations of the name. The name could be max 32 chars, only have
   alphanumeric and the underscore characters. This imposed limitations
   that are not present in flowbits/ints.

2. we didn't actually use the named substrings in pcre through the
   API. We parsed the names separately. So putting the names in pcre
   would actually be wasteful.

This patch introduces a new way of mapping captures with names:

  pcre:"/(.*)/, pkt:somename";
  pcre:"/([A-z]+) ([0-9]+)/, pkt:somename,flow:anothername";

The order of the captures and the order of the names are mapped 1 on 1.
This method is no longer limited by the pcre API's naming limits. The
'flow:' and 'pkt:' prefixes indicate what the type of variable is. It's
mandatory to specify one.

The old method is still supported as well.
 9 years ago
Victor Julien 0f708d427b pkt-var: abuse flowvar postmatch logic for pktvars 
Flowvars were already using a temporary store in the detect thread
ctx.

Use the same facility for pktvars. The reasons are:

1. packet is not always available, e.g. when running pcre on http
   buffers.

2. setting of vars should be done post match. Until now it was also
   possible that it is done on a partial match.
 9 years ago
Victor Julien 5e39486399 pkt-var: use id instead of name pointer 9 years ago
Victor Julien a0bd15a1c4 pcre: support multiple captures 
Support up to 8 substring captures into pkt or flow vars.
 9 years ago
Victor Julien 017b16d421 detect-pcre: small cleanups 9 years ago
Victor Julien ac42a44280 alert-debug: print flowvar/int names 9 years ago
Victor Julien e95a0c1344 alert-debug: print flowbit names from VarNameStore 9 years ago
Victor Julien 22f3205664 var-names: expose outside of detect engine 
Until now variable names, such as flowbit names, were local to a detect
engine. This made sense as they were only ever used in that context.

For the purpose of logging these names, this needs a different approach.
The loggers live outside of the detect engine. Also, in the case of
reloads and multi-tenancy, there are even multiple detect engines, so
it would be even more tricky to access them from the outside.

This patch brings a new approach. A any time, there is a single active
hash table mapping the variable names and their id's. For multiple
tenants the table is shared between tenants.

The table is set up in a 'staging' area, where locking makes sure that
multiple loading threads don't mess things up. Then when the preparing
of a detection engine is ready, but before the detect threads are made
aware of the new detect engine, the active varname hash is swapped with
the staging instance.

For this to work, all the mappings from the 'current' or active mapping
are added to the staging table.

After the threads have reloaded and the new detection engine is active,
the old table can be freed.

For multi tenancy things are similar. The staging area is used for
setting up until the new detection engines / tenants are applied to
the system.

This patch also changes the variable 'id'/'idx' field to uint32_t. Due
to data structure padding and alignment, this should have no practical
drawback while allowing for a lot more vars.
 9 years ago
Victor Julien 43cc06eabe detect: use engine version instead of id 
Use engine version based on global detect engine master. This is
incremented between reloads.
 9 years ago
Victor Julien 920709fe6f detect: ssh_software sticky buffer 9 years ago
Victor Julien f1ab6a6153 detect: ssh_proto stickybuffer 9 years ago
Victor Julien dfac5276b8 detect: remove unused SIGMATCH_PAYLOAD flag 9 years ago
Victor Julien 14ced15e36 detect: remove unused state file flag 9 years ago
Victor Julien fa1ef158b2 detect: small API cleanup 9 years ago
Victor Julien 073fcbeb7f detect: move file hash common code 9 years ago
Victor Julien 5bafc64c08 detect: unify FileMatch API with other calls 9 years ago
Victor Julien fe415ae518 detect: remove DMATCH list 9 years ago
Victor Julien 1c02cf4542 flow: remove unused Flow::de_state 9 years ago
Victor Julien ad238121e3 detect: remove the AMATCH list 9 years ago
Victor Julien 775e182531 detect: remove AppLayerMatch API call 9 years ago
Victor Julien f018ae94b0 dce: dynamic lists 9 years ago
Victor Julien 84ba9cf9df smb/dcerpc: use tx api 9 years ago
Victor Julien d318bfc934 dcerpc: simplify common detect code 9 years ago
Victor Julien 402eb645a0 ftp: parser and ftpbounce update 
Convert parser to TX API.

Convert ftpbounce keyword to use that.
 9 years ago
Victor Julien d9a300cd8c detect: move lua smtp support to dynamic list 9 years ago
Victor Julien 96b8100a51 lua: convert lua output to be tx aware 9 years ago
Victor Julien a10b2fdecf detect: make ssh detection use dynamic list 9 years ago
Victor Julien c412352474 ssh: remove single logger limit 9 years ago
Victor Julien 3ee4989ba7 ssh: convert app-layer parser to be tx aware 
Like with SSL, there is only a single 'tx'.
 9 years ago
Victor Julien 4ae4fd0802 lua: use tls_generic list for ssl/tls 9 years ago
Victor Julien a8975c68e0 detect ssl/tls: use dynamic lists 9 years ago
Victor Julien 1cacba435b ssl/tls: clean up keywords 9 years ago
Victor Julien 6ef4712cd6 detect: convert old tls keywords to dynamic list 9 years ago
Victor Julien ef149bed7b tls.store: convert to postmatch 9 years ago
Victor Julien 8b3a71a285 tls.store: cleanup 9 years ago
Victor Julien 8eac1156c6 app-layer-events: remove unused API options 9 years ago
Victor Julien 1087495d6d detect: http_start sticky buffer 
Matches on the start of a HTTP request or response.

Uses a buffer constructed from the request line and normalized request
headers, including the Cookie header.

Or for the response side, it uses the response line plus the
normalized response headers, including the Set-Cookie header.

Both buffers are terminated by an extra \r\n.
 9 years ago
Victor Julien 36535efa04 detect: add http_protocol sticky buffer 
Matches on protocol field in HTTP.
 9 years ago
Victor Julien f2fc5a255f http_header: convert to use common code 9 years ago
Victor Julien 6279ec399e http_header: common detection code 9 years ago
Victor Julien 5ee68ca2b2 http_header: remove old files 9 years ago
Victor Julien 2bb0cae05a http_header: move all code into keyword files 9 years ago
Victor Julien aaeeae0722 detect: http_header_names sticky buffer keyword 
A sticky buffer that allows content inspection on a contructed buffer
of HTTP header names. The buffer starts with \r\n, the names are
separated by \r\n and the end of the buffer contains an extra \r\n.

E.g. \r\nHost\r\nUser-Agent\r\n\r\n

The leading \r\n is to make sure one can match on a full name in all
cases.
 9 years ago
Victor Julien cf9678d926 detect: global registery for keyword thread data 
Some keywords need a scratch space where they can do store the results
of expensive operations that remain valid for the time of a packets
journey through the detection engine.

An example is the reconstructed 'http_header' field, that is needed
in MPM, and then for each rule that manually inspects it. Storing this
data in the flow is a waste, and reconstructing multiple times on
demand as well.

This API allows for registering a keyword with an init and free function.

It it mean to be used an initialization time, when the keyword is
registered.
 9 years ago
Victor Julien 75907fce06 profiling: output all sort options for rules 
Limit the default number of sids to 10.
 9 years ago
Victor Julien 7d8a5a75ef profiling: honor limit in json rule output 9 years ago
Victor Julien a9a228a289 profiling: fix keyword profiling 9 years ago
Victor Julien a1465bc4fa detect-engine-mpm: api cleanup 9 years ago
Victor Julien 8bd1422948 detect: detect engine registration cleanup 9 years ago
Victor Julien 49fbd28ceb detect: cleanup built-in list id's 9 years ago
Victor Julien 40851eecf0 template: dynamic buffer 9 years ago
Victor Julien 815120896b app-layer-events: dynamic list 9 years ago
Victor Julien b68343e372 files: use dynamic list 9 years ago
Victor Julien e4bfdd53c2 cip/enip: dynamic buffer 9 years ago
Victor Julien f2393c1ae0 modbus: dynamic buffer 9 years ago
Victor Julien cfdd934aba dnp3: dynamic buffers/lists 9 years ago
Victor Julien 9ba386a141 tls: dynamic buffers 9 years ago
Victor Julien d9b3ae6cd6 dns: use dynamic buffers 9 years ago
Victor Julien d2f77978ec detect-parse: content modifier cleanup 9 years ago
Victor Julien 6f867c3c73 http_raw_uri: dynamic buffer 9 years ago
Victor Julien ee55aefa1c http_client_body: dynamic buffer 9 years ago
Victor Julien e7d5e845c7 http_header / http_raw_header: dynamic buffers 9 years ago
Victor Julien b694d96e22 http_stat_msg: dynamic buffer 9 years ago
Victor Julien 7e3ab4f5ea http_stat_code: dynamic buffer 9 years ago
Victor Julien 128b59d4f6 http_raw_host: dynamic buffer 9 years ago
Victor Julien 296c275e23 http_host: dynamic buffer 9 years ago
Victor Julien 67b7d9734e http_cookie: dynamic buffer 9 years ago
Victor Julien 54604c7bf2 http_user_agent: dynamic buffer 9 years ago
Victor Julien 9262fa3dcf http_response_line: dynamic buffer 9 years ago
Victor Julien 6346a074a7 http_uri: dynamic buffer 
Clean up tests
 9 years ago
Victor Julien e34102d67a http_method: make list dynamic 9 years ago
Victor Julien 6bd37611ee file_data: dynamic buffer 9 years ago
Victor Julien 7052f9b933 http_request_line: dynamic buffer 9 years ago
Victor Julien 779d40cedf detect: remove hardcoded sm_list logic from setup 
Introduce utility functions to aid this.
 9 years ago
Victor Julien 04592efb76 detect: buffer type API 
To replace the hardcoded SigMatch list id's, use this API to register
and query lists by name.

Also allow for registering descriptions and whether mpm is supported.

Registration is only allowed at startup.
 9 years ago
Victor Julien 58e1180efe detect: inspect engine setup cleanup 9 years ago
Victor Julien debc1a6334 detect: dce test fixes and improvements 9 years ago
Victor Julien a2b521b7fa detect-csum: redo tests 9 years ago
Victor Julien f370e88135 detect: move init only Signature members to init_data 9 years ago
Victor Julien 0a5ae415b8 detect: shrink Signature::sm_arrays 
Signature::sm_arrays now only contains 'built-in' lists, and so is
sized appropriately.
 9 years ago
Victor Julien 4978a7a133 detect: reorganize id's in prep of dynamic lists 9 years ago
Victor Julien 59303d1fbb threshold: fix and redo tests 9 years ago
Victor Julien 6f7e4adbe8 detect: improve memory handling & comments 9 years ago
Victor Julien 8edc954e82 detect: get rid of Signature::sm_lists 
Instead use the lists in init_data during setup and the SigMatchData
arrays during runtime.
 9 years ago
Victor Julien f281481b67 detect: use detect list passed to generic funcs 
Until now the GenericList users used hardcoded list id's.
 9 years ago
Victor Julien bd456076a8 detect: pass SigMatchData to inspect functions 9 years ago
Victor Julien a0fe67a3c0 detect: template list in engine 9 years ago
Victor Julien da7c816c7c detect: enip/cip list in engine 9 years ago
Victor Julien e94a7bddb3 detect: modbus list in engine 9 years ago
Victor Julien 7f7d4296da detect: file list in engine 9 years ago
Victor Julien f5adccba1d detect: app-event list in engine 9 years ago
Victor Julien 747dbf92ce detect: dns & tls lists in engine 9 years ago
Victor Julien 5a2e568385 detect: http lists in engine 9 years ago
Victor Julien 1ee32da2ca detect-engine: memory handling of sm_lists 
For lists that are registered multiple times, like http_header and
http_cookie, making the engines owner of the lists is complicated.
Multiple engines in a sig may be pointing to the same list. To
address this the 'free' code needs to be extra careful about not
double freeing, so it takes an approach to first fill an array
of the to-free pointers before freeing them.
 9 years ago
Victor Julien f81b90dacd detect: when freeing sig also see sm in inspect engine 9 years ago
Victor Julien 2f87c975d4 detect: add SigMatch arg to inspect functions 9 years ago
Victor Julien cf42fbf51f detect: use InspectEngineFuncPtr in inspect engines 
Replace explicit function pointer use by InspectEngineFuncPtr typedef
 9 years ago
Victor Julien 5f7e096be4 detect: shrink inspect engine by using 'id' as state flag 9 years ago
Victor Julien 715ff60087 detect: remove unused SIG_FLAG_INIT_PAYLOAD init_flag 9 years ago
Victor Julien 859cb89c7e detect alert/threshold/tag: sm_list -> sm_array 9 years ago
Victor Julien 99580487e5 detect: fix file_data / http_server_body tests 9 years ago
Victor Julien faadec0d7f detect file_data: improve error messages 9 years ago
Victor Julien e2c6e1be33 detect-parse: set ipprotos earlier 
A high level proto like HTTP implies TCP. However this wasn't set
until after all the parsing was complete which means that keywords
couldn't test if the ipproto matched.

This patch populates the ipprotos right when the higher level proto
is parsed.
 9 years ago
Victor Julien 5e0b0eea4b detect: remove unused flags 9 years ago
Victor Julien 39613778cd detect: make setup/free/match funcs static where possible 9 years ago
Victor Julien bfd4bc8233 detect: constify Signature/SigMatch use at runtime 9 years ago
Victor Julien a44da9f5cb detect: simplify SIG_FLAG_STATE_MATCH set logic 9 years ago
Victor Julien be3ee5330f detect: remove alproto from keyword registration 
It was already marked as depricated and no longer in use anywhere.
 9 years ago
Victor Julien 113a238e90 Open 4.0 development branch 9 years ago
Victor Julien 71710f088e dns: fix outputs with 0-len A/AAAA records 9 years ago
Victor Julien 20990f7a7e dns: fix out of bounds read 
On a zero size A or AAAA record, 4 or 16 bytes would still be
read.

Found with AFL+ASAN.
 9 years ago
Jason Ish 4a04f814b1 defrag - take protocol into account during re-assembly 
The IP protocol was not being used to match fragments with
their packets allowing a carefully constructed packet
with a different protocol to be matched, allowing re-assembly
to complete, creating a packet that would not be re-assembled
by the destination host.
 9 years ago
Victor Julien 292baf0872 afl: add ethernet and erspan entry points 9 years ago
Victor Julien 49c41fc79e afl: clean up commandline parsing 9 years ago
Victor Julien b56b04f84c afl: pass a packet queue to decoder calls 9 years ago
Jason Ish 35488eefda afl: set the packet data so pktlen gets set 9 years ago
Victor Julien fbd69729aa afl: improve packet fuzz testing 
Due to the use of AFL_LOOP and initialization/deinit outside of it,
part of the fuzzing relied on the global 'state' in flow and defrag.
Because of this crashes that were found could not be reproduced. The
saved crash input was only the last in the series.

This patch addresses that. It requires a new output directory 'dump'
where the packet fuzzers will store all their input. If the AFL_LOOP
fails the files will not be removed and this 'serie' can be read
again for reproducing the issue.

e.g.: AFL would work with:
--afl-decoder-ppp=@@

and after a crash is found the produced serie can be read with:
--afl-decoder-ppp-serie=1486656919-514163

The series have a timestamp as name and a suffix that controls the
order in which the files will be 'replayed' in Suricata.
 9 years ago
Victor Julien 923d93f314 afl: add decoder ipv4 option 9 years ago
Sascha Steinbiss 5e96977983 mpm-ac: fix integer overflow on allocation 
The size of a memory buffer to be allocated was kept in a signed int
instead of a size_t, leading to an overflow when large lists of long
and diverse patterns cause the amount of AC states to blow up (>2GB).
Fixes Redmine issues #1827 and #1843.

Signed-off-by: Sascha Steinbiss <sascha@steinbiss.name>
 9 years ago
Sascha Steinbiss b25b067d93 alert: silence compiler type warning 
The `ts_ecr' and `ts_val' struct fields are integer types, not
pointers. This leads GCC 6.3.0 to complain about comparisons to
NULL.

Signed-off-by: Sascha Steinbiss <sascha@steinbiss.name>
 9 years ago
Victor Julien 86222428dd detect: don't run IP inspection on non-IP packets 
The code to get the rule group (sgh) would return the group for
IP proto 0 instead of nothing. This lead to certain types of rules
unintentionally matching (False Positive).

Since the packets weren't actually IP, the logged alert records
were missing the IP header.

Bug #2017.
 9 years ago
Victor Julien 4683b0e662 afl: fix ENIP, switch DNS to UDP and add --afl-dnstcp* 9 years ago
Victor Julien c89ce17017 afl: with -Wshadow issues 9 years ago
Eric Leblond ecf59be413 af-packet: add VLAN header when needed in IPS mode 
When packet is coming from a real ethernet card, the kernel is
stripping the vlan header and delivering a modified packet so
we need to insert the VLAN header back before sending the packet
on the wire.

To do so, we pass an option to the raw socket to add a reserve
before the packet data. It will get Suricata some head room to
to move the ethernet addresses before there actual place and
and insert the VLAN header in the correct place.

We get VLAN info from the ring buffer as the call of AFPWrite is
always done in the release function so we still have access to the
memory.
 9 years ago
Eric Leblond f407d77016 detect-tls-sni: add link to documentation 9 years ago
Eric Leblond 1af713d67d detect-tls: add url field pointing to doc 9 years ago
Eric Leblond 0695ad4bf0 detect-xbits: set documentation URL 9 years ago
Jason Ish 21bbac5648 dns-log: log requests even when there is no response 
The JSON logger had already been updated to handle
transactions without a response. Apply the same logic
to the older dns-log where a logger is registered
for each direction.

Fixes issue 2012.
 9 years ago
Andreas Herz d8b5bf9bc6 app-layer-parsing: detect malformed input 
If the app-layer-parsing has a very long content it exceeds the maximum
defined in "alproto_name". This adds a check for the too long content
before it will be passed to "strlcpy" and logs an error.
 9 years ago
Victor Julien f91d490d25 detect: remove unused flow_locked hint 9 years ago
Victor Julien 31a96d5a79 detect: make tenant loading less verbose 9 years ago
Victor Julien addf64f1f7 profiling: fix memory leaks 9 years ago
Victor Julien 6e876182d7 detect: use TLS_STATE_CERT_READY in cert inspect 9 years ago
Victor Julien 473dae75b5 tls: introduce 'cert ready' state 9 years ago
Victor Julien 15accc86c9 common: add BIT_U8 macro 9 years ago
Sascha Steinbiss e6044aaf1c mpm/spm: check for SSSE3 and enable/disable HS 
The new Hyperscan 4.4 API provides a function to check for SSSE3
presence at runtime. This allows us to fall back to non-Hyperscan
matchers on systems without SSSE3 even when the suricata executable
is built with Hyperscan support. Addresses Redmine issue #2010.

Signed-off-by: Sascha Steinbiss <sascha@steinbiss.name>
Tested-by: Arturo Borrero Gonzalez <arturo@debian.org>
 9 years ago
Victor Julien a0580d8805 stream: initialize stream segment pool from mtu 
If segments section in the yaml is ommitted (default) or when the
pool size is set to 'from_mtu', the size of the pool will be MTU
minus 40. If the MTU couldn't be determined, it's assumed to be
1500, so the segment size for the bool will be 1460.
 9 years ago
Victor Julien 1ba15d3721 mtu: track max mtu for capture devices 9 years ago
Victor Julien 7ca466c598 shutdown: remove pid file last 9 years ago
Victor Julien 816dd7b301 startup: clean up main loop 9 years ago
Victor Julien 2eec07cc3a unittests: clean up registration and startup 9 years ago
Victor Julien f452df761a shutdown: move global shutdown steps into func 9 years ago
Victor Julien babe8a299e startup/shutdown: cleanup and unify with unix mode 9 years ago
Victor Julien 3c64cfb384 threads: fix missed logging at shutdown 
At shutdown, all flows that still need work are handled by the flow
force reassembly logic. This means one or more flow end pseudo packets
are generated and pushed through the engine for final detection and
logging.

In some cases this would not work correctly. This was caused by the
flow timeout logic kicking in before all the 'live' packets were
processed. Before the flow timeout handling runs the receive threads
are disabled, however the engine did not wait for the in-flight
packets to be fully processed. In autofp mode, packets could still
be in the queue between receive thread(s) and flow worker(s).

This patch adds a new function that 'drains' all the packet threads
of any in-progress packets before moving on the flow timeout logic.

Bug #1946.
 9 years ago
Mats Klepsland a2659ed7ec output-json-flow: add has_alerts field 
Add has_alerts field to flow eve-log to indicate if a flow has
any alerts or not.
 9 years ago
Mats Klepsland c531e8f77c lua: add SCFlowHasAlerts function 
Add SCFlowHasAlerts() to check if a flow has alerts. Returns true
on alerts, false otherwise.

Example:

  has_alerts = SCFlowHasAlerts()
  if has_alerts then
    -- do something
  end
 9 years ago
Mats Klepsland d9b87e502d flow: set flag to indicate that a flow has alerts 
Set FLOW_HAS_ALERTS flag on the flow on alerts. Add FlowHasAlerts(..)
and FlowSetHasAlertsFlag(..) to check and set this flag.
 9 years ago
Eric Leblond 569cc5d238 util-file: introduce new functions for file size 
This patch introduces the FileDataSize and FileTrackedSize functions.
The first one is just a renaming of the initial FilSize function
whereas the other one is using the newly introduced size field as
value.
 9 years ago
Eric Leblond a098896b28 output-json-file: use size instead of FileSize 
FileSize is not returning the actual value when file store is not
used.
 9 years ago
Eric Leblond fbc2dbac28 util-file: change file size computation 
The file size returned by FileSize is invalid if file store is not
used so we introduce a new size field in File structure that is used
to store the size.
 9 years ago
Alexander Gozman 187a6f392c Bug #2009: added CAP_NET_ADMIN for PCAP and af-packet modes. 
Without this capability suricata is unable to get network
interface's settings.
 9 years ago
Jason Ish 19e578a740 pcap-log: fix pcre_study error check 
Code was failing on a NULL return value which can be returned
when there was nothing todo instead of an error. Instead
check the errbuf for a non-NULL value to determine error.
 9 years ago
Jason Ish 5c55373679 app-layer-detect-proto.c: fix indent 
A recent commit was outdented by 1 column.
 9 years ago
Jason Ish ec44585dca app-layer - fix secondary probing parser logic 
Apply the same logic to pe2 as pe1 for determining which
probe to call. Missed in previous commit.
 9 years ago
Victor Julien cb36dee477 hyperscan: fix minor coverity issue in error path 
*** CID 1398951:  API usage errors  (LOCK)
/src/util-mpm-hs.c: 722 in SCHSPreparePatterns()
716         SCMutexUnlock(&g_db_table_mutex);
717
718         SCHSFreeCompileData(cd);
719         return 0;
720
721     error:
>>>     CID 1398951:  API usage errors  (LOCK)
>>>     "pthread_mutex_unlock" unlocks "g_db_table_mutex" while it is unlocked.
722         SCMutexUnlock(&g_db_table_mutex);
723         if (pd) {
724             PatternDatabaseFree(pd);
725         }
726         if (cd) {
 9 years ago
Victor Julien fa8cbd8741 smb: detect protocol in both directions 9 years ago
Jason Ish e9fccfa67c tx logging: only update logged tx id if all loggers logged 
Prevents the case where the logged id is incremented if a newer
transaction is complete and an older one is still outstanding.

For example, dns request0, unsolicited dns response, dns response0

would result in the valid response0 never being logged.

Similarily this could happen for:
  request0, request1, response1, response0

which would end up having request0, request1 and response1 logged,
but response0 would not be logged.
 9 years ago
Eric Leblond 0d5fd0f658 util-file: fix error logic in hash computation 
This patch fixes an issue with hash computation resulting in the
invalidity of at least one hash when at least two different hashes
functions were used.

Impact was setting as `force-hash: [md5, sha256]` not to be valid.
Also it could lead to false negative if too different hash functions
had to be used on a single file due to signatures.
 9 years ago
Jason Ish 20111cab23 unix-socket: fix shadowed variable 
ret does not need to be redefined here, the existing
declaration of ret can be used.
 9 years ago
Victor Julien f964cdbc93 address parsing: fix memory leak in error path 9 years ago
Victor Julien a6fccd952e ssl: suppress scan-build warnings 9 years ago
Victor Julien 61b72c6981 output: clean up output function 
Don't allocate memory per call.
 9 years ago
Victor Julien 709d20f8c6 smb/dcerpc: suppress scan-build warnings 9 years ago
Victor Julien 618ab4e177 ac-bs: fix scan-build warnings 9 years ago
Victor Julien bbc02205fb queue: add debug assertions to TAILQ 
To avoid scan-build fp's add assertions that are only active if
built with scan-build.
 9 years ago
Mats Klepsland 03ad9d4ec0 tls-store: fix bug that causes Suricata to crash 
Fix bug that causes Suricata to crash when the tls.store keyword is used.

*** Error in `/usr/bin/suricata': free(): invalid next size (fast):
0x00007fd4b4373180 ***
 9 years ago
Jason Ish 87b5bf9541 proto detect - fix coverity CID 1204325 
CID 1204325 (#1 of 1): Logically dead code (DEADCODE)
dead_error_line: Execution cannot reach this statement: mask = 0U;.
433        mask = 0;

additionally, mask is initialized to 0
 9 years ago
Jason Ish d09cd16c8c template logger - fix coverity CID 1324964 
null: At condition templatejs != NULL, the value of templatejs must be
NULL.
dead_error_condition: The condition templatejs != NULL cannot be true.
113    if (templatejs != NULL) {
CID 1324964 (#1 of 1): Logically dead code (DEADCODE)
dead_error_line: Execution cannot reach this statement:
json_decref(templatejs);.
114        json_decref(templatejs);
115    }
 9 years ago
Jason Ish a10a9220cf dns (tcp) - fix coverity CIDs 1374306, 1374305 
CID 1374306 (#1 of 1): Dereference before null check (REVERSE_INULL)
check_after_deref: Null-checking dns_state suggests that it may be null,
but it has already been dereferenced on all paths leading to the check.
585    if (dns_state != NULL && f != NULL) {
586        dns_state->last_req = f->lastts;
587    }

CID 1374305 (#1 of 1): Dereference before null check (REVERSE_INULL)
check_after_deref: Null-checking dns_state suggests that it may be null,
but it has already been dereferenced on all paths leading to the check.
366    if (dns_state != NULL && f != NULL) {
367        dns_state->last_req = f->lastts;
368    }
 9 years ago
Jason Ish dfbfb50f64 dns (tcp) - fix coverity cid 1374307 
CID 1374307 (#1 of 1): Dereference before null check (REVERSE_INULL)
check_after_deref: Null-checking dns_state suggests that it may be null,
but it has already been dereferenced on all paths leading to the check.
317    if (dns_state != NULL && f != NULL) {
318        dns_state->last_resp = f->lastts;
319    }
 9 years ago
Eric Leblond 5b1de57d73 detect-parse: simplify port prefiltering 
Regular expression was not matching some authorized setting like
"![1234, 1235]". This patch simplify the regexp to match on
possible character and let the port parsing code handle the
complete verification.
 9 years ago
Jason Ish b0de5ad1a8 dns: increment tx id when allocated during response 9 years ago
Victor Julien fe4e119278 common: improve BUG_ON 
When BUG_ON is a wrapper for assert(), we risk getting rid of certain
code lines. Assert is a no-op when NDEBUG is defined.

This patch defines an alternate path for BUG_ON that exits after
printing an error.

Bug #2003.
 9 years ago
Andreas Herz 98e8b13bf0 decode-icmpv6: add missing types 
There have been some ICMPv6 types missing within the DecodeICMPV6 that
are added by this commit and the code check is adjusted to always use
the DEFINE.
 9 years ago
Jason Ish bcdbd12839 dns (tcp): register a to_client (response) probing parser 
Just a minimal parser to make sure the data contains at
least a header.
 9 years ago
Jason Ish c35c18a797 app-layer: support to server and to client probing parsers 
When registering a probing parser allow to_server and
to_client parsers to be registered. Previously the
probing parser may be called for both directions which
in some cases works OK, but in others can cause
the to_client side to be detected as failed.
 9 years ago
Victor Julien 586774203f redis: support for all output types 9 years ago
Victor Julien 2820ed332e redis: use 'binary' notation for output 9 years ago
Victor Julien df28c1ac6e common: add WARN_UNUSED macro 9 years ago
Victor Julien 8c65d45d55 detect: remove dead code 9 years ago
Eric Leblond 63a3b84127 util-magic: fix build when magic is not available 
If HAVE_MAGIC is not defined then we don't have the test functions
so we can't register them.
 9 years ago
Victor Julien cb08f02140 xbits: clean up parsing and tests 9 years ago
Jason Ish 27ec811187 pcap-log: fix memory leak during initialization of ring buffer 
A free was missing when files are removed during initialization
of the ring buffer.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/1985
 9 years ago
Eric Leblond 9f6b58747b smtp: commands and replies are not case sensitive 
RFC states that "Commands and replies are not case sensitive" and
patterns were registered to be case sensitive. So this patch fixes
a trivial evasion of SMTP signatures.
 9 years ago
Mats Klepsland ffcb4ad232 tls: fix tls_cert_subject prefilter bug 
If check in prefilter was checking that issuer was non-NULL, when
it in fact should be checking subject.
 9 years ago
Mats Klepsland 10c93221fa tls: increase max number of tls records per packet 
Tls packets may contain several records. This increase the number
of allowed records per packet from 30 to 255, and adds a new and
more informative decoder event when this limit is reached.
 9 years ago
Mats Klepsland 554065189c tls: don't trigger decoder event on no extensions in CLIENT_HELLO 
No extensions are allowed in <TLSv.1.2, so don't trigger SURICATA
TLS handshake invalid length decoder event when no extensions are
specified in CLIENT HELLO.
 9 years ago
Victor Julien 810e43f373 magic: make optional 
Make libmagic optional. If installed it will be enabled by default in
configure. Use --disable-libmagic to disable.
 9 years ago
Jason Ish 79a3c6c7b1 log-pcap: use a snaplen of 262144 instead of -1. 
Newer versions of libpcap will not open pcap files with a
snaplen of -1, instead use the current maximum value of
262144.

Issue:
https://redmine.openinfosecfoundation.org/issues/1987
 9 years ago
Victor Julien a44b612a41 hostbits/xbits: free hostbit 
Fix memory leak. Hostbits were not actually freed.

Bug #1975.
 9 years ago
Victor Julien 579d6d3ff7 http: allow lower/mixed case in proto detect 
In HTTP detection registered patterns were upper case only. Since the
detection is based on both sides this would still work for sessions
where one of the talkers misbehaved. If both sides misbehave this
would fail however, so this patch introduces case insensive matching.
 9 years ago
Jason Ish 92885d6960 profiling: fix shadow error 
Local variable store was shadowing variable in function
definition.
 9 years ago
Victor Julien abbc0f76eb unix-socket: clean up path handling 
Create/check socket path in a single place. Don't use dynamic
memory allocation.
 9 years ago
Victor Julien c5e550b10d unix-socket: create socket directory if possible 
Create the socket directory in the default case.

Since we're doing stat+mkdir indicate to Coverity not to worry about
the toctou case.
 9 years ago
Victor Julien 3f741e450b unix-socket: be more specific about problems 9 years ago
Victor Julien 9368013645 unix-socket: don't error out on unix socket failure 
If --init-errors-fatal is specified do error out.

Bug https://redmine.openinfosecfoundation.org/issues/1973
 9 years ago
Jason Ish 2ce95babd6 dnp3: use _ in keyword names instead of "." 
dnp3.ind -> dnp3_ind
dnp3.func -> dnp3_func
dnp3.obj -> dnp3_obj

The variations with a "." are now aliases.
 9 years ago
Victor Julien ab8faefd37 af-packet: fix fanout support on Debian Jessie 
Debian Jessie with kernel 3.16 would not accept the 'id' of 99 used
in the test. Id 1 does work.
 9 years ago
Alexander Gozman e492f0dc89 Fix port parsing in config file, added one more corresponding test. 
Some examples from wiki caused parsing errors.
For example, "[1:80,![2,4]]" was treated as a mistake.

Also fixed loop detection in variables declaration. For example,
'A: "HOME_NET, !$HOME_NET"' resulted in parsing error.
 9 years ago
Jason Ish ba6a1aa73e unittest-helper: fix format string compiler warnings 9 years ago
Jason Ish 4aceaf9fcd detect-stream_size: fix format string compiler warnings 9 years ago
Jason Ish c0377a9870 stream-tcp: fix format string compiler warnings 9 years ago
Jason Ish 5c80a3edf7 modbus: fix format string compiler warnings 9 years ago
Jason Ish 53a8c75385 dnp3: rename "index" variables to "point_index" 
Gcc 4.6 will warning with -Wshadow for a local variable
named "index" as <strings.h> has a function named "index".

Newer versions of gcc handle this case.
 9 years ago
Victor Julien b4ac048b41 proto detect: fix -Wshadow warning 9 years ago
Victor Julien 07e1e3e02a dcerpc: fix -Wshadow warnings 9 years ago
Victor Julien 783d2991e5 commandline: fix -Wshadow warnings 9 years ago
Victor Julien 66c213f30c detect-address: fix -Wshadow warnings 9 years ago
Victor Julien 9a20335b6d asn1: fix -Wshadow warnings 9 years ago
Victor Julien dab51144af asn1: modernize test 9 years ago
Victor Julien 613174e9ce yaml: fix tests for -Wshadow 9 years ago
Victor Julien 968813b655 dnp3: fix test for -Wshadow 9 years ago
Victor Julien bb2d8a7133 runmodes: fix -Wshadow warnings 9 years ago
Victor Julien 8c1d157cd2 mpm ac-bs: fix -Wshadow warnings 9 years ago
Victor Julien 0d6d8e01c8 threads: fix -Wshadow warnings 9 years ago
Victor Julien cd04da673b commandline: fix -Wshadow warnings 9 years ago
Victor Julien 69ee2f0eb9 nfq: fix -Wshadow warnings 
Rename globals to make sure it's clear they are globals.
 9 years ago
Victor Julien 65d2443ccd reputation: fix -Wshadow warnings 9 years ago