Commit Graph

8988 Commits (7e06e765f39088c51a700cf232f2c3cc953929f7)
 

Author SHA1 Message Date
Giuseppe Longo d0f92e2a56 app-layer-htp: add swf decompression settings
This adds some settings needed to do swf file decompression
under libhtp section in suricata.yaml
7 years ago
Giuseppe Longo b60065caec configure: check for zlib and liblzma
This checks if zlib and libzma are installed on the system
in order to decompress swf files.
7 years ago
Maurizio Abba 8354f62b19 signal: enable SIGUSR2 after Reload when delayed-detect
Enable SIGUSR2 Handler after the first rule reload when delayed-detect
is enabled
7 years ago
Eric Leblond 3c68a22092 suricatasc: implement autoreconnect
Implement a basic autoreconnect support. It tries to reconnect once
when connection has been lost. If it fails, it discards the command
and try again to connect at next command.
7 years ago
Eric Leblond 3d0ba36ba8 unix socket: protocol v0.2
This patch updates the unix socket protocol. Messages send from
the server and the client have now a '\n' at the end. This allows
both sides to detect easily the end of a command.

As a side effect, this fixes the problem of long answer in
suricatasc. There is now a limit at the arbitrary value of 65536.

Backward compatility is preserved as a client with the older
version of the protocol can still connect to a Suricata with
version 2 of the protocol.
7 years ago
Jason Ish 3fd7256af5 setup-app-layer-detect: update for changes in detect 7 years ago
Jason Ish 74e036d09f doc: update eve/alert/metadata configuration 7 years ago
Jason Ish 45a38c0431 eve/alert: new metadata configuration (sane defaults)
Under eve/alert, introduce a new metadata configuration
section. If no provided, or simply yes defaults will be used.
Otherwise this a map with fields that can be toggled on and
off. The defaults are:

outputs:
  - eve-log:
      types:
        - alert:
            metadata:
              app-layer: true
              flow: true
              rule:
                raw: false
                metadata: true

To enable something that is disabled by default, or to disable
something that is enabled by default, only that key need to
be changed, everything else will keep its default value.
7 years ago
Jason Ish 472cc8ea61 conf: new function: ConfNodeHasChildren
Test if a configuration node has any children, indicating
that it is a non-empty map or sequence.
7 years ago
Martin Natano fe9cac5870 eve/alert: include rule text in alert output
For SIEM analysis it is often useful to refer to the actual rules to
find out why a specific alert has been triggered when the signature
message does not convey enough information.

Turn on the new rule flag to include the rule text in eve alert output.
The feature is turned off by default.

With a rule like this:

    alert dns $HOME_NET any -> 8.8.8.8 any (msg:"Google DNS server contacted"; sid:42;)

The eve alert output might look something like this (pretty-printed for
readability):

    {
      "timestamp": "2017-08-14T12:35:05.830812+0200",
      "flow_id": 1919856770919772,
      "in_iface": "eth0",
      "event_type": "alert",
      "src_ip": "10.20.30.40",
      "src_port": 50968,
      "dest_ip": "8.8.8.8",
      "dest_port": 53,
      "proto": "UDP",
      "alert": {
        "action": "allowed",
        "gid": 1,
        "signature_id": 42,
        "rev": 0,
        "signature": "Google DNS server contacted",
        "category": "",
        "severity": 3,
        "rule": "alert dns $HOME_NET any -> 8.8.8.8 any (msg:\"Google DNS server contacted\"; sid:43;)"
      },
      "app_proto": "dns",
      "flow": {
        "pkts_toserver": 1,
        "pkts_toclient": 0,
        "bytes_toserver": 81,
        "bytes_toclient": 0,
        "start": "2017-08-14T12:35:05.830812+0200"
      }
    }

Feature #2020
7 years ago
Eric Leblond 72c8cd67d5 doc: documentation update on metadata 7 years ago
Eric Leblond 9864552484 detect-metadata: add a string storage to de_ctx
To avoid to have a lot of string allocations, we use a hash table
stored in de_ctx to point to existing string instead of duplicating
them.
7 years ago
Eric Leblond 3a2431a2fb suricata: init output before detection
As we need to know if we should parse the signature metadata, we
have to parse the output configuration before initializing the
detection engine.
7 years ago
Eric Leblond 6bf00ab289 output-json-alert: conditionaly output metadata
Metadata of the signature can now conditionaly put in the alert
events. This will allow user to get more context about the events
generated by the alert.

detect-metadata: conditional parsing

Only parses metadata if an output module will use the information.
Patch also adds a unittest to check metadata is not parsed if not
asked to.

output-json-alert: optional output keys as array

Update rule metadata configuration to have an option to output
value as array. Also adds an option to log only a series of keys
as array. This is useful in the case of some ruleset where from
instance the `tag` key is used multiple time.

(Jason Ish) rule metadata: always log as lists

After review of rule metadata, we can't make assumptions
on what should be a list or not. So log everything as a list.
7 years ago
Eric Leblond 1bd6d1c209 detect-metadata: add unit test 7 years ago
Eric Leblond 474fc60671 detect-metadata: store metadata key value pairs
This patch updates the Signature structure so it contains the
metadata under a key value form.
Later patch will make that dictionary available in the events.
7 years ago
Victor Julien d0ea147263 travis/rust: update rust minimum to 1.21
Ubuntu LTS and CentOS7/EPEL has upgraded to 1.21.

Update highest known working version to 1.23.
7 years ago
Victor Julien 6f7e24d3f2 autogen/rust: remove Cargo.lock
Remove Cargo.lock to avoid issues when updating Cargo.toml
7 years ago
Victor Julien 6e82df274d rust: update dependencies 7 years ago
Victor Julien dfae3297a5 rust: don't gen C headers if Rust isn't enabled 7 years ago
Jason Ish ab939f4aaa doc: breakout eve-log section to a partial file
Both the suricata.yaml and eve configuration sections
included the eve-log section from suricata.yaml. First,
sync these up with the actual suricata.yaml then break
it out into its own file, so only one file needs to
be kept in sync with the actual configuration file.
7 years ago
Jason Ish 93b056d89e eve/alert: log metadata be default
By default log metadata.

Remove toggles for individual protocol types and just use a
single toggle to control including the app-layer with the
alert.

The metadata (currently app-layer and flow) can be disabled
by setting metadata to a falsey value, but its removed
from the default configuration (but wil be in docs)
7 years ago
Jason Ish b659222ea0 eve/metadata: log flowvars as a list of k/v pairs
To match the pktvars output.
7 years ago
Jason Ish 1f47f77bd5 eve/metadata: special handling for traffic-id labels
Give traffic/id and traffic/label flowbits special handling
in the eve output. Instead of just logging them as flowbits,
give them their own top level object.

{
  "traffic": {
    "id": ["id0", "id1"],
    "label": ["label0", "label1"]
  }
}
7 years ago
Jason Ish 0e02684634 doc: update eve-log section for metadata 7 years ago
Jason Ish 572a62f35a output-json-vars: rename to metadata
No functional change, just rename of files and functions
to reflect the metadata event type now used.
7 years ago
Jason Ish 34811cf69e json-vars: rename to metadata and use new metadata format 7 years ago
Jason Ish a23d54ce3e eve: netflow: global metadata config 7 years ago
Jason Ish 3eaca7c239 eve: http: global metadata config 7 years ago
Jason Ish 790ce3743b eve: flow: global metadata config 7 years ago
Jason Ish 23bbbc5818 eve: dns: global metadata config 7 years ago
Jason Ish 4a05160353 eve: alert: global metadata config
Also, remove vars as a subtype. Adding the top level metadata
field is an eve lebel parameter, not alert now.
7 years ago
Jason Ish 5da5fc1f7d eve: drop: global metadata config 7 years ago
Jason Ish 2247b9aad2 eve: email: respect global metadata config 7 years ago
Jason Ish 885452fc22 eve: nfs: respect global metadata config 7 years ago
Jason Ish b577f4a0c9 eve: smtp: respect global metadata config 7 years ago
Jason Ish 7f5439a300 eve: dnp3: respect global metadata config 7 years ago
Jason Ish 32da579239 eve: ssh: respect global metadata config 7 years ago
Jason Ish 88ac0f2b1a eve: tls: respect global metadata config 7 years ago
Jason Ish dd988d9934 eve: metadata setting to enable/disable metadata
This is a top level metadata object containing flowbits,
flowints, pktvars and flowvars.

Enabling it at the top level enables it for all log types.
7 years ago
Jason Ish 5138f99c58 eve: top level metadata object
Contains:
- flowbits (as array)
- flowints (map)
- flowvars (map)
- pktvars (map)
7 years ago
Victor Julien 6f339abdf0 htp: minor debug addition 7 years ago
Victor Julien e86be22737 htp: remove unused field from tx state 7 years ago
Victor Julien c63b1ce2c6 htp: remove used body operation field 7 years ago
Victor Julien 07cbbfb0d1 htp: code cleanups 7 years ago
Victor Julien 9ca71beb03 htp: remove usused file flags 7 years ago
Victor Julien daeba48f77 htp: remove usused flags 7 years ago
Victor Julien c0d26de665 stream: improve overlap detection
Improve detection of overlapping different data. Keep some data around
even if it was already ACK'd to check if packets have overlap.
7 years ago
Victor Julien e64941144e htp: allow HTTP pickup of response data
Now that libhtp can pick up sessions that start with a response
we can enable support for it as well.
7 years ago
Victor Julien 49927024c6 http: add tests for malformed response lines 7 years ago