Commit Graph

95 Commits (c3c20dd04110df753acd02846fd9443c53e5a7dc)

Author SHA1 Message Date
Eric Leblond c3c20dd041 pcap: add new config style 14 years ago
Eric Leblond 78467f1846 Update configuration file to new pfring format. 14 years ago
Anoop Saldanha de635b5866 remove trailing whitespace from conf file 14 years ago
William Metcalf 3b3f5816bf You spin me right round baby, right round like a rotating packet capture right round. Oh, also log file size counters are now uint64_t 14 years ago
Eric Leblond 5bbd8fe910 Add reference to events sig files in suricata YAML config. 14 years ago
Eric Leblond df7dbe36b6 af-packet: Add option to disable promiscuous mode
This patch adds an option to suricata.yaml to be able to disable
the switch of the interface into promiscuous mode.
14 years ago
Eric Leblond 18c6503a08 af-packet: change configuration format for multi interface 14 years ago
Eric Leblond e80b30c082 af-packet: finalize code
This patch handles the end of AF_PACKET socket support work. It
provides conditional compilation, autofp and single runmode.

It also adds a 'defrag' option which is used to activate defrag
support in kernel to avoid rx_hash computation in flow mode to fail
due to fragmentation.

This patch contains some fixes by Anoop Saldanha, and incorporate
change following review by Anoop Saldanha and Victor Julien.

AF_PACKET support is only build if the --enable-af-packet flag is
given to the configure command line. Detection of code availability
is also done: a check of the existence of AF_PACKET in standard
header is done. It seems this variable is Linux specific and it
should be enough to avoid compilation of AF_PACKET support on other
OSes.
Compilation does not depend on up-to-date headers on the system. If
none are present, wemake our own declaration of FANOUT variables. This
will permit compilation of the feature for system where only the kernel
has been updated to a version superior to 3.1.
14 years ago
Eric Leblond c45d898572 af-packet: basic support for AF_PACKET socket
This patch provides basic support for AF_PACKET socket. It is
completed by a subsequent patches prodiding extended features
and bugfixes.
14 years ago
Victor Julien 820b0ded82 Add per packet profiling.
Per packet profiling uses tick based accounting. It has 2 outputs, a summary
and a csv file that contains per packet stats.

Stats per packet include:
 1) total ticks spent
 2) ticks spent per individual thread module
 3) "threading overhead" which is simply calculated by subtracting (2) of (1).

A number of changes were made to integrate the new code in a clean way:
a number of generic enums are now placed in tm-threads-common.h so we can
include them from any part of the engine.

Code depends on --enable-profiling just like the rule profiling code.

New yaml parameters:

profiling:
  # packet profiling
  packets:

    # Profiling can be disabled here, but it will still have a
    # performance impact if compiled in.
    enabled: yes
    filename: packet_stats.log
    append: yes

    # per packet csv output
    csv:

      # Output can be disabled here, but it will still have a
      # performance impact if compiled in.
      enabled: no
      filename: packet_stats.csv

Example output of summary stats:

IP ver   Proto   cnt        min      max          avg
------   -----   ------     ------   ----------   -------
 IPv4       6     19436      11448      5404365     32993
 IPv4     256         4      11511        49968     30575

Per Thread module stats:

Thread Module              IP ver   Proto   cnt        min      max          avg
------------------------   ------   -----   ------     ------   ----------   -------
TMM_DECODEPCAPFILE          IPv4       6     19434       1242        47889      1770
TMM_DETECT                  IPv4       6     19436       1107       137241      1504
TMM_ALERTFASTLOG            IPv4       6     19436         90         1323       155
TMM_ALERTUNIFIED2ALERT      IPv4       6     19436        108         1359       138
TMM_ALERTDEBUGLOG           IPv4       6     19436         90         1134       154
TMM_LOGHTTPLOG              IPv4       6     19436        414      5392089      7944
TMM_STREAMTCP               IPv4       6     19434        828      1299159     19438

The proto 256 is a counter for handling of pseudo/tunnel packets.

Example output of csv:

pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading
1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337

First line of the file contains labels.

2 example gnuplot scripts added to plot the data.
14 years ago
William 6730c3ace1 Actually limit recursion and backtracking and stack usage by PCRE. Logic was broken, no example was provided in suricata.yaml even though it could be set from there. 14 years ago
Eric Leblond fb6494abc1 config file: add missing variable example
The variable threads was not shown in the default configuration file
and this is really needed to optimize the system easily.
14 years ago
William 85643fe780 Convert to logging perf stats to file by default. Add a few columns to output avg ticks per match, avg ticks non match, allow sorting on based on them. 14 years ago
Anoop Saldanha e4d890e186 modify runmode api to accept conf runmode paramter as a char string, instead of an interger id 14 years ago
Anoop Saldanha 229f7281ea list runmodes. Allow specification of runmode id from cof file. Also allow for command line override 14 years ago
Victor Julien a96eae01fa Disable alert-debuglog that was accidently enabled in a previous commit. 14 years ago
Victor Julien 4f5aad1476 Enforce configurable minimum chunk size in raw stream reassembly. Minor stream cleanups, unittest updates. 14 years ago
Victor Julien bc7e21aee6 Add special sguil mode to log-pcap to support logging into date based directory structure and rotate when the day passes. Also do not log packets beyond stream reassembly depth and encrypted traffic. 14 years ago
Victor Julien f1e3d636d9 Add limit option to pcap-log logging config. 14 years ago
William Metcalf 023a0f94a2 first stab at pcap logging no rotating buff etc 14 years ago
Victor Julien da423a59d5 Allow users of the alert-syslog to set the identity. 14 years ago
Victor Julien 91f28afef4 Add option to PF_RING to have multiple reader threads. Improve general performance of the PF_RING module. 14 years ago
Victor Julien a3303fcf9d Rename request-body-limit to request_body_limit to remain consistant with other options. Keep old notation around for compatibility. 14 years ago
Gurvinder Singh 7d0781b349 added support to log dropped packet as netfilter logs while in inline mode 14 years ago
Victor Julien 1c7b7a01a6 Add option to set the syslog level for the alerts. Minor cleanups. 14 years ago
Gurvinder Singh e5edc6e8e3 add the support to log the fast.log alerts type to syslog 14 years ago
Eric Leblond 0b5e5b8772 affinity: change config format and misc fixes
This patch fixes some problem with affinity work and modify the
configuration file format.

For example, the detect cup set can be formatted as follow:
     - detect_cpu_set:
        cpu: [ "all" ]
        mode: "exclusive" # run detect threads in these cpus
        prio:
          low: [ 0 ] # threads on CPU 0 have low prio
          medium: [ "1-2" ] # threads on CPU 1 and 2 have medium prio
          high: [ 3 ] # threads on CPU 3 have high prio
          default: "medium" #default priority is "medium"
14 years ago
Eric Leblond c74116949c source-nfq: improve nfq option system
This patch modifies the NFQ option system to avoid implicit
choice. 'nfq.mode' is now a string which can take a value
in the 'accept', 'repeat' and 'route' set.

Signed-off-by: Eric Leblond <eric@regit.org>
14 years ago
Eric Leblond 1e600c1054 source-nfq: add simulated non-terminal NFQUEUE verdict
This patch adds a new mode for NFQ inline mode. The idea is to
simulate a non final NFQUEUE rules.
This permit to do send all needed packets to suricata via a simple
FORWARD rule:
    iptables -I FORWARD -m mark ! --mark $MARK/$MASK -j NFQUEUE
And below, we have a standard filtering ruleset.

To do so, suricata issues a NF_REPEAT instead of a NF_ACCEPT verdict and
put a mark ($MARK) with respect to a mask ($MASK) on the handled packet.

NF_REPEAT verdict has for effect to have the packet reinjected at start
of the hook after the verdict. As it has been marked by suricata during
the verdict it will not rematch the initial rules and make his way to
the following classical ruleset.

Mode, mark and mask can be configured via suricata.yaml file with the
following syntax:
   nfq:
     repeat_mode: (false|true)
     mark: $MARK
     mask: $MASK
Default is false to preserve backward compatibility.

Signed-off-by: Eric Leblond <eric@regit.org>
14 years ago
Eric Leblond 789d46cc3c Add per-cpu prio handling
This patch updates affinity setting to add a support for per cpu
priority setting. In exclusive mode a thread is dedicated to a CPU.
This patch adds the ability to set the thread prio for all threads
of a family running on a given CPU.

With this patch we can write
    - detect_cpu_set:
        cpu: [ "all" ]
        mode: "exclusive" # run detect threads in these cpus
        low_prio: [ 0 ]
        medium_prio: [ "1-2" ]
        high_prio: [ 3 ]
With this configuration, detect threads assigned to cpu 0 will
have a low priority. Detect threads on cpus 1 and 2 will have
prio medium...

The previous configuration is equivalent to:
    - detect_cpu_set:
        cpu: [ "all" ]
        mode: "exclusive" # run detect threads in these cpus
        low_prio: [ 0 ]
        high_prio: [ 3 ]
        prio: "medium"
because the prio value is used a default.

Signed-off-by: Eric Leblond <eric@regit.org>
14 years ago
Eric Leblond 559b5db7df yaml: add config for cpu_affinity
This patch introduces the configuration part of cpu_affinity
evolution. The idea is to have thread families (receive, stream, ...)
and to be able to specify for each of them:
 - The cpus to use
 - The mode of balancing between the cpus
 - The thread priority to use (value of nice)

Signed-off-by: Eric Leblond <eric@regit.org>
14 years ago
Victor Julien d00c6172c9 Update stream section of example configuration. 14 years ago
Eric Leblond e802e1ed16 Modify Packet structure and prepare accessor.
This patch modifies decode.c and decode.h to avoid the usage
by default of a bigger than 65535 bytes array in Packet structure.
The idea is that the packet are mainly under 1514 bytes size and
a bigger size must be supported but should not be the default.

If the packet length is bigger than DFLT_PACKET_SIZE then the
data are stored in a dynamically allocated part of the memory.

To ease the modification of the rest of the code, functions to
access and set the payload/length in a Packet have been introduced.

The default packet size can be set at runtime via the default-packet-size
configuration variable.
14 years ago
Eric Leblond 8471626916 Fix error message and adds information to config
This patch fixes a typo in an error message and add some
information to the checksum verification option.
14 years ago
Gurvinder Singh f4392e1dcc added support for appending the log files 14 years ago
Pierre Chifflier de41612ea1 Add options to choose if we log header and content in Prelude alert module.
Signed-off-by: Pierre Chifflier <chifflier@edenwall.com>
14 years ago
Victor Julien e66b3e3ee7 Switch mpm-algo in example suricata.yaml to ac 14 years ago
Victor Julien 355f237bfd Fix compiler warnings, cleanup counters config code. 14 years ago
Gurvinder Singh ba18110abd support for stats.log configurable and fixed timezone issue in faslog and debuglog 14 years ago
Martin Beyer b1c577f829 cuda streams support in b2g-cuda MPM 14 years ago
Anoop Saldanha c734cd1bdd make cuda mpm parameters configurable 14 years ago
Victor Julien 0dd07df60d Add reference to suricata.yaml documentation in our redmine wiki. 14 years ago
Victor Julien 6ebe7b7cd3 Change the way the request body limit is enforced. 14 years ago
Anoop Saldanha 778ec0939c make client body buffer limit configurable. Also some minor changes 14 years ago
Jason Ish a4d19e4130 Add new profiling sort option, maxticks. 15 years ago
Anoop Saldanha 5d9a453e0d find an optimal value for detect-engine:inspection-recursion_limit + unittest 15 years ago
Anoop Saldanha 88d94b136d Support for reference.config file 15 years ago
Victor Julien 344ea14695 Change mpm hash_size config setting highest to higher as highest wasn't the... highest. Max was higher. Leaving highest as an alias to higher for backwards compatibility. 15 years ago
Victor Julien 3bd7441ea5 Default to 'single' ctx for ac-gfbs as well. 15 years ago
Anoop Saldanha 62f814a4c3 change default value for detect-engine.sgh_mpm_context to auto 15 years ago