Commit Graph

8385 Commits (3ab9120821e2b5cbc5925470bcfa5bcfb53f246b)
 

Author SHA1 Message Date
Dana Helwig 3ab9120821 source-pcap-file: Pcap Directory Mode (Feature #2222)
https://redmine.openinfosecfoundation.org/issues/2222

Pcap file mode that when passed a directory will process all files in
that directory. If --pcap-file-continuous or continuous option is passed
in json, the directory will be monitored  until the directory is
moved/deleted, suricata is interrupted, or the pcap-interrupt command
is used with unix command socket. Existing file implementation and new
directory implementation has moved from source-pcap-file into
pcap-file-helper and pcap-directory-helper.

Engine state will not reset between files.

Also satisfies:
 * https://redmine.openinfosecfoundation.org/issues/2299
 * https://redmine.openinfosecfoundation.org/issues/724
 * https://redmine.openinfosecfoundation.org/issues/1476

Co-Authors: Dana Helwig <dana.helwig@protectwise.com> and
Danny Browning <danny.browning@protectwise.com>
7 years ago
Eric Leblond 94e9d13791 doc: add ruleset commands available in unix socket 7 years ago
Giuseppe Longo e7392a0780 unix-manager: block live reload when -s/-S is specified
Currently, when live reload is executed through
unix-socket, suri prints in the console the following
error message:
"Live rule reload not possible if -s or -S option used at runtime."

Instead, prints "done" in unix socket,
when the live reload is not executed.
7 years ago
Giuseppe Longo f1d372a505 unix-manager: add "ruleset-reload-rules" command
This add a new command the new naming convention.

The (old) command "reload-rules" is kept for
backward compatibility
7 years ago
Giuseppe Longo 9131afbe71 unix-manager: print failed rules
This permits to print the invalid rules through
unix socket.

An example output is the following:

>>> show-failed-rules
Success:
[
    {
        "filename": "/home/eric/git/oisf/benches/tls-store.rules",
        "line": 2,
        "rule": "alert ts any any -> any 334 (msg:\"Store TLS\"; tls.store; sid:2; rev:1;)"
    },
    {
        "filename": "/home/eric/git/oisf/benches/tls-store.rules",
        "line": 3,
        "rule": "alert ls any any -> any 334 (msg:\"Store TLS\"; tls.store; sid:3; rev:1;)"
    }
]

The dump is limited to 20 entries to avoid to send a too big
message to the client that don't support it by default.
7 years ago
Giuseppe Longo 1328ecb8f6 detect: save invalid rules
This keeps the invalid rules in string format into a list,
added in DetectEngineCtx.
7 years ago
Eric Leblond e17b9616ab unix-socket: add ruleset-reload-nonblocking command
Add a non blocking function to reload rules. It will be useful
for remote system management to avoid to block them waiting the
reload. And as we now have a last-reload command we can get the
status of the current reload.
7 years ago
Giuseppe Longo 1567f84cd2 detect-engine: remove DONE state
Remove the DONE state to fix a problem with state not being
changed correctly when multiple reload were done. As DONE was
not really useful, we can remove it.
7 years ago
Giuseppe Longo 89c629a79d unix-socket: add commands to print engine stats
This permits to print engine stats through
unix socket.
7 years ago
Giuseppe Longo 720cfcfcca json-stats: print engine stats
This adds the engine stats in stats event.
If multi-tenancy is enabled, it will add
stats for each tenant

The following is a snippet of the generated EVE entry:

"detect":{"engines":[{"last_reload":"2015-10-13T09:59:48.044996+0200","rules_loaded":17184,"rules_failed":0}],"alert":28}

Multi-tenancy enabled:

"detect":{"engines":[{"id":1,"last_reload":"2015-10-13T09:56:46.447153+0200","rules_loaded":17084,"rules_failed":0},
                     {"id":2,"last_reload":"2015-10-13T09:56:36.504877+0200","rules_loaded":3268,"rules_failed":0}],
                     "alert":28}
7 years ago
Giuseppe Longo 56000acefb detect-engine: add reload time/rules stats
This patch adds the following stats for
the detect engine:
- time of the last reload
- number of rules loaded
- number of rules failed
7 years ago
Pascal Delalande 0c99338e07 doc: update docs for DNS flags logging 7 years ago
Pascal Delalande d474c95349 dns: store flags for logging for TCP 7 years ago
Clément Galland 3396747cd6 Dns logger display flags information 7 years ago
Giuseppe Longo 44bf785ecf dns: log flags field
This adds dns header's flags in eve
log.

Signed-off-by: Eric Leblond <eric@regit.org>
7 years ago
Ralph Broenink f6938933d9 doc: Amend the list of accepted protocols
Based on the list in suricata.yaml
7 years ago
Ralph Broenink d830177b7b doc: Add my own name to the acknowledgements 7 years ago
Ralph Broenink 98a1ec490f doc: Move IP reputation keyword to rules section 7 years ago
Ralph Broenink 722cff1862 doc: Restructure ToC
* All sections up to 2 levels deep are now shown regardless of whether they are a separate page
* Rename Xbits and Thresholding for more consistent naming
* Minor adjustment in the Payload Keywords section
7 years ago
Ralph Broenink 196ba1da70 doc: Make the header keywords section separate sections in ToC 7 years ago
Ralph Broenink a55a6cdb62 doc: Move flowint as integral part of flow keywords 7 years ago
Ralph Broenink f6c766112c doc: Minor changes in structuring of HTTP Keywords / Snort differences 7 years ago
Ralph Broenink e9b25988ba doc: Move pcre entirely to Payload Keywords section
(plus remove lingering screenshot of a rule)
7 years ago
Ralph Broenink bb1bf2643d doc: Move fast_pattern and prefilter to dedicated page 7 years ago
Ralph Broenink fea037fda8 doc: Moved explanation of normalized buffers to rules introduction 7 years ago
Ralph Broenink 11990c7117 doc: Move the definition of modifier keywords to the introduction 7 years ago
Ralph Broenink dfae19247d doc: Completely rewrite the rules introduction for more clearity 7 years ago
Ralph Broenink 274c36eb2f doc: Meta-settings -> Meta Keywords plus some textual changes
Most importantly, conventions are now placed in tip boxes
7 years ago
Ralph Broenink 3413793768 doc: Use lowercased keyword names as section titles 7 years ago
Ralph Broenink a52aacb4ea doc: Replace images of tables and rules with text in rules docs
In some chapters of the rules documentation, many sections used examples of rules, but these were inserted into images. These have been replaced by text and HTML emphasis.

Additionally, some tables embedded into images were also replaced by reST tables.
7 years ago
Ralph Broenink 44926e2369 doc: Add suricata.css to allow for some custom styling 7 years ago
Victor Julien 5335d8b877 detect/uri: apply urilen contents as depth 7 years ago
Victor Julien 606eab937c detect/http_uri: remove broken tests 7 years ago
Wolfgang Hotwagner c16509a8b6 conf: stack-based buffer-overflow in ParseFilename
There is a stack-based buffer-overflow in ParseFilename. Since the length of "outputs.pcap-log.filename" is not checked and the destination buffer "str" has a fixed length of 512 bytes, a buffer overflow happens with long filenames. An attacker could exploit this for code execution if the configuration-file is not protected properly. This commit fixes ticket #2335

This is what the asan-output looks like:

~/suricata-1/src# suricata -T -c ./suricata.yaml
[27871] 3/12/2017 -- 20:48:13 - (suricata.c:1876) <Info> (ParseCommandLine) -- Running suricata under test mode
[27871] 3/12/2017 -- 20:48:13 - (suricata.c:1109) <Notice> (LogVersion) -- This is Suricata version 4.0.0-dev (rev f3fea60b)
=================================================================
==27871==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7fffbe9d75e0 at pc 0x55897b5f935f bp 0x7fffbe9d72b0 sp 0x7fffbe9d72a8
WRITE of size 1 at 0x7fffbe9d75e0 thread T0 (Suricata-Main)
    0 0x55897b5f935e in ParseFilename /root/suricata-1/src/log-pcap.c:895
    1 0x55897b5fb173 in PcapLogInitCtx /root/suricata-1/src/log-pcap.c:985
    2 0x55897b6af103 in RunModeInitializeOutputs /root/suricata-1/src/runmodes.c:752
    3 0x55897b72c6b5 in PreRunPostPrivsDropInit /root/suricata-1/src/suricata.c:2263
    4 0x55897b730416 in main /root/suricata-1/src/suricata.c:2898
    5 0x7f947f6db2b0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x202b0)
    6 0x55897b2d4c19 in _start (/usr/local/bin/suricata+0xc4c19)

Address 0x7fffbe9d75e0 is located in stack of thread T0 (Suricata-Main) at offset 672 in frame
    0 0x55897b5f7fcc in ParseFilename /root/suricata-1/src/log-pcap.c:836

  This frame has 3 object(s):
    [32, 104) 'toks'
    [160, 672) 'str' <== Memory access at offset 672 overflows this variable
    [704, 2752) '_sc_log_msg'
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow /root/suricata-1/src/log-pcap.c:895 in ParseFilename
Shadow bytes around the buggy address:
  0x100077d32e60: 00 00 00 00 00 00 00 00 f1 f1 f1 f1 00 00 00 00
  0x100077d32e70: 00 00 00 00 00 f4 f4 f4 f2 f2 f2 f2 00 00 00 00
  0x100077d32e80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x100077d32e90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x100077d32ea0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x100077d32eb0: 00 00 00 00 00 00 00 00 00 00 00 00[f2]f2 f2 f2
  0x100077d32ec0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x100077d32ed0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x100077d32ee0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x100077d32ef0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x100077d32f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==27871==ABORTING
7 years ago
Ruslan Usmanov 1090ee9d8d rate_filter by_both through IPPair storage
Ticket https://redmine.openinfosecfoundation.org/issues/2127
7 years ago
Danny Browning 84b66b7aaa enum: don't printf on util-enum errors
When util-enum encounters an error around enum value it should log the error
rather than losing it to console with printf.

Bug #2268
7 years ago
Victor Julien 999b50476b detect/http_host: add sid to nocase warning 7 years ago
Victor Julien f68067be94 hosts: release packet references to hosts 7 years ago
Gaurav Singh 637a7c8e55 Adds options to mark when a file is final.
This takes the form of an option to add the pid of the process to file
names. Additionally, it adds a suffix to the file name to indicate it is
not finalized.

Adding the pid to the file name reduces the likelihood that a file is
overwritten when suricata is unexpectedly killed. The number in the
waldo file is only written out during a clean shutdown. In the event
of an improper shutdown, extracted files will be written using the old
number and existing files with the same name will be overwritten.

Writes extracted files and their metadata to a temporary file suffixed
with '.tmp'. Renames the files when they are completely done being
written. As-is there is no way to know that a file on disk is still
being written to by suricata.
7 years ago
Victor Julien a1f8cf40e2 detect/http_start: check if 'line' is valid
In certain conditions like low memory the line can be NULL.

Bug #2307.
7 years ago
Victor Julien 9abac08cc7 detect/flowint: harden code
Make sure packet has a flow.

Related to bug #2288.
7 years ago
Victor Julien 40a819d5a6 detect/flowint: only check if packet has flow
Fixed bug #2288.
7 years ago
Victor Julien db24fee16e detect/flowint: improve unittests
In preparation of fixing bug #2288, make sure the unittests setup
the flow in the packet properly.
7 years ago
Victor Julien 7394ee17ec unittest/helpers: add helper to assign flow to packet 7 years ago
Victor Julien 83f220a6b0 detect/depth: reject rules with depth smaller than content 7 years ago
Jason Ish d0846cc561 detect-parse: string copy not required
Without using pcre, copies of the strings are no longer
required.
7 years ago
Jason Ish 73d1e4bc84 detect-parse: don't use pcre for rule parsing
Don't use pcre for the high level rule parsing, instead
using a tokenizing parser for breaking out the rule
into keywords and options.

Much faster, especially on older CPUs. Should also allow
us to provide better context where a rule parse error
occurs.
7 years ago
Victor Julien 93b120e70d runmodes: config test is offline 7 years ago
Victor Julien 71c3141ec6 afl: enable afl dumps by envvar
If SC_AFL_DUMP_FILES is set the inputs are stored to disk.
7 years ago
Victor Julien f1da18ec1a http: allow shinking in HTPRealloc 7 years ago