aiden
/
suricata
mirror of https://github.com/OISF/suricata
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,101 Commits
8 Branches
163 Tags
184 MiB
main-7.0.x
main-8.0.x
main
master-6.0.x
master-5.0.x
master-4.1.x
master-4.0.x
master-3.2.x
suricata-8.0.1
suricata-7.0.12
suricata-8.0.0
suricata-7.0.11
suricata-8.0.0-rc1
suricata-8.0.0-beta1
suricata-7.0.10
suricata-7.0.9
suricata-7.0.8
suricata-7.0.7
suricata-7.0.6
suricata-6.0.20
suricata-7.0.5
suricata-6.0.19
suricata-6.0.18
suricata-7.0.4
suricata-6.0.17
suricata-7.0.3
suricata-6.0.16
suricata-7.0.2
suricata-6.0.15
suricata-7.0.1
suricata-6.0.14
suricata-7.0.0
suricata-7.0.0-rc2
suricata-6.0.13
suricata-6.0.12
suricata-6.0.11
suricata-7.0.0-rc1
suricata-6.0.10
suricata-6.0.9
suricata-7.0.0-beta1
suricata-6.0.8
suricata-6.0.7
suricata-6.0.6
suricata-5.0.10
suricata-6.0.5
suricata-5.0.9
suricata-6.0.4
suricata-5.0.8
suricata-6.0.3
suricata-5.0.7
suricata-6.0.2
suricata-5.0.6
suricata-6.0.1
suricata-5.0.5
suricata-4.1.10
suricata-4.1.9
suricata-5.0.4
suricata-6.0.0
suricata-6.0.0-rc1
suricata-6.0.0-beta1
suricata-5.0.3
suricata-4.1.8
suricata-4.1.7
suricata-5.0.2
suricata-4.1.6
suricata-5.0.1
suricata-5.0.0
suricata-5.0.0-rc1
suricata-4.1.5
suricata-5.0.0-beta1
suricata-4.1.4
suricata-4.1.3
suricata-4.0.7
suricata-4.1.2
suricata-4.1.1
suricata-4.1.0
suricata-4.0.6
suricata-4.1.0-rc2
suricata-4.1.0-rc1
suricata-4.0.5
suricata-4.1.0-beta1
suricata-4.0.4
suricata-4.0.3
suricata-4.0.2
suricata-3.2.5
suricata-4.0.1
suricata-3.2.4
suricata-4.0.0
suricata-4.0.0-rc2
suricata-3.2.3
suricata-4.0.0-rc1
suricata-4.0.0-beta1
suricata-3.2.2
suricata-3.1.4
suricata-3.2.1
suricata-3.2
suricata-3.2RC1
suricata-3.1.3
suricata-3.2beta1
suricata-3.1.2
suricata-3.1.1
suricata-3.1
suricata-3.0.2
suricata-3.1RC1
suricata-3.0.1
suricata-3.0.1RC1
suricata-3.0
suricata-2.0.11
suricata-3.0RC3
suricata-3.0RC2
suricata-3.0RC1
suricata-2.0.10
suricata-2.0.9
suricata-2.1beta4
suricata-2.0.8
suricata-2.0.7
suricata-2.1beta3
suricata-2.0.6
suricata-2.0.5
suricata-2.1beta2
suricata-2.0.4
suricata-2.1beta1
suricata-2.0.3
suricata-2.0.2
suricata-2.0.1
suricata-2.0.1rc1
suricata-2.0
suricata-2.0rc3
suricata-2.0rc2
suricata-2.0rc1
suricata-2.0beta2
suricata-1.4.7
suricata-1.4.6
suricata-1.4.5
suricata-2.0beta1
suricata-1.4.4
suricata-1.4.3
suricata-1.4.2
suricata-1.4.1
suricata-1.3.6
suricata-1.4
suricata-1.3.5
suricata-1.4rc1
suricata-1.3.4
suricata-1.4beta3
suricata-1.3.3
suricata-1.4beta2
suricata-1.3.2
suricata-1.4beta1
suricata-1.3.1
suricata-1.3
suricata-1.3rc1
suricata-1.3beta2
suricata-1.3beta1
suricata-1.2.1
suricata-1.2
suricata-1.2rc1
suricata-1.2beta1
suricata-1.1.1
suricata-0.8.2
suricata-1.0.0
suricata-1.0.1
suricata-1.0.2
suricata-1.0.3
suricata-1.0.4
suricata-1.0.5
suricata-1.1
suricata-1.1beta1
suricata-1.1beta2
suricata-1.1beta3
suricata-1.1rc1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'de6dac0043'
${ noResults }
Commit Graph

426 Commits (de6dac004331412b28ab55508b9a55ff5728c859)

Author SHA1 Message Date
Matt Keeler 844e4dba11 Napatech 3GD Support 
For use with Network Cards from Napatech utilizing the 3GD driver/api.

    - Implemented new run modes in runmode-napatech-3gd.*
    - Implemented capture/decode threads in source-napatech-3gd.*
    - Integrated the new run modes and source into the build infrastructure.

    New configure switches
    --enabled-napatech-3gd : Turns on the NT 3GD support
    --with-napatech-3gd-includes : The directory containing the NT 3GD header files
    --with-napatech-3gd-libraries : The directory containing the NT 3GD libraries to link against.

    New CLI switch
    --napatech-3gd : Uses the Napatech 3GD run mode

    Runmodes Supported:
    - auto
    - autofp
    - workers

    Notes:
    - tested with 1 Gbps sustained traffic (no drops)

Signed-off-by: Matt Keeler <mk@npulsetech.com>
 13 years ago
Eric Leblond 00b95c69c0 suricata: list-keywords does not depend on unittest 13 years ago
Anoop Saldanha b99f9fe890 New app inspection engine introduced. Moved existing inspecting engines to use it. 13 years ago
Eric Leblond e176be6fcc Use unlikely for error treatment. 
When handling error case on SCMallog, SCCalloc or SCStrdup
we are in an unlikely case. This patch adds the unlikely()
expression to indicate this to gcc.

This patch has been obtained via coccinelle. The transformation
is the following:

@istested@
identifier x;
statement S1;
identifier func =~ "(SCMalloc|SCStrdup|SCCalloc)";
@@

x = func(...)
... when != x
- if (x == NULL) S1
+ if (unlikely(x == NULL)) S1
 13 years ago
Victor Julien 98484ffdcc luajit: prealloc lua states to increases chances of alloc success. Luajit requires them to be in memory <2GB. 13 years ago
Victor Julien ec7e79c748 Rule profiling update 
- Remove usage of counters api.
- Store stats in detect engine thread ctx to remove locking
- Support rule reloads
 13 years ago
Eric Leblond b0a2aefc78 af-packet: fix build on systems without AF_PACKET 13 years ago
Eric Leblond 662dccd8a5 af-packet: IPS and TAP feature 
This patch adds a new feature to AF_PACKET capture mode. It is now
possible to use AF_PACKET in IPS and TAP mode: all traffic received
on a interface will be forwarded (at the Ethernet level) to an other
interface. To do so, Suricata create a raw socket and sends the receive
packets to a interface designed in the configuration file.

This patch adds two variables to the configuration of af-packet
interface:
 copy-mode: ips or tap
 copy-iface: eth1 #the interface where packet are copied
If copy-mode is set to ips then the packet wth action DROP are not
copied to the destination interface. If copy-mode is set to tap,
all packets are copied to the destination interface.
Any other value of copy-mode results in the feature to be unused.
There is no default interface for copy-iface and the variable has
to be set for the ids or tap mode to work.

For now, this feature depends of the release data system. This
implies you need to activate the ring mode and zero copy. Basically
use-mmap has to be set to yes.

This patch adds a peering of AF_PACKET sockets from the thread on
one interface to the threads on another interface. Peering is
necessary as if we use an other socket the capture socket receives
all emitted packets. This is made using a new AFPPeer structure to
avoid direct interaction between AFPTreadVars.

There is currently a bug in Linux kernel (prior to 3.6) and it is
not possible to use multiple threads.

You need to setup two interfaces with equality on the threads
variable. copy-mode variable must be set on the two interfaces
and use-mmap must be set to activated.

A valid configuration for an IPS using eth0 and vboxnet1 interfaces
will look like:

af-packet:
  - interface: eth0
    threads: 1
    defrag: yes
    cluster-type: cluster_flow
    cluster-id: 98
    copy-mode: ips
    copy-iface: vboxnet1
    buffer-size: 64535
    use-mmap: yes
  - interface: vboxnet1
    threads: 1
    cluster-id: 97
    defrag: yes
    cluster-type: cluster_flow
    copy-mode: ips
    copy-iface: eth0
    buffer-size: 64535
    use-mmap: yes
 13 years ago
Eric Leblond 66a083dafa Get rid of AppLayerHtpRegisterExtraCallbacks 
This patch add a early exit condition to the body handling callback.
This permits to avoid to avoid a complex system to handle htp
object change.
 13 years ago
Eric Leblond 7e09cdc265 Delay Detect threads initialization 
This patch modifies the init of Detect threads. They are now started
with a dummy function and their initialisation is done after the
signatures are loaded. Just after this, the dummy function is switched
to normal one.

In IPS mode, this permit to route packets without waiting for the
signature to start and should fix #488.

Offline mode such as pcap file don't use this mode to be sure to
analyse all packets in the file.

The patch introduces a "delayed-detect" configuration variable
under detect-engine. It can be used to activate the feature
(set to "yes" to have signature loaded after capture is started).
 13 years ago
Jean-Paul Roliers efdf96ccba tls: adding TLS Log support 
Creation of the log-tlslog file in order to log tls message.
Need to add some information into suricata.yaml to work.

  - tls-log:
      enabled: yes	# Log TLS connections.
      filename: tls.log # File to store TLS logs.
 13 years ago
Anoop Saldanha 3eb0fd878d Don't wait for packetpool to be back to full state before continuing with the shutdown process, on received shutdown signal 13 years ago
Anoop Saldanha 5f198e3a1d Suricata shutdown updates + minor cleanup 13 years ago
Victor Julien 4c6fd7ad4c Bug #510. Produce error if max-pending-packets is higher than 65534. 13 years ago
Anoop Saldanha b2f589527a Set thread name Suricata-Main for main thread and LiveRuleSwap for live swap thread 13 years ago
Anoop Saldanha 34f0897163 check if all packets are processed before disabling detect threads + kill all threads <= detect after FFR + other minor fixes 13 years ago
Victor Julien 9f3e079bcf Make live reloads optional and disabled by default. 13 years ago
Victor Julien 728c4f9ea0 Clean up packet pool at shut down. 14 years ago
Eric Leblond 0227a87fcb cleaning: fix warning when building with clang. 
clang was issuing some warnings related to unused return in function.
This patch adds some needed error treatment and ignore the rest of the
warnings by adding a cast to void.
 14 years ago
Anoop Saldanha 2bc7d0792d update clean up of old detection engine contexts for live rule swap 14 years ago
Anoop Saldanha eee33866df DetectEngineCtxFree() cleanup, also in main 14 years ago
Anoop Saldanha 8fb2040eee disable live rule swap when -s or -S option's used at startup 14 years ago
Anoop Saldanha 31eb5fa2f6 Introduce util-signal.[ch]. Move our signal setup functions here 14 years ago
Anoop Saldanha e5edcfaca8 add unittest for atomic operation with void * 14 years ago
Anoop Saldanha ecad4a24fa live rule support added 
To reload ruleset during engine runtime, send the USR2 signal to the engine, and the ruleset would be reloaded from the same yaml file supplied at engine startup
 14 years ago
Anoop Saldanha f4ce9011d2 make mpm ctx container de_ctx specific. Also introduce global variable in mpm_ctx. this is a workaround for cleaning non global mpm_ctx's since we now don't supply the de_ctx around the detection engine API 14 years ago
Anoop Saldanha f5af4c9ceb util action api returns error code if it encounters wrong values parsing wrong action conf 14 years ago
Anoop Saldanha 988c92f71c http user agent keyword + mpm + inspection + fast pattern support added 14 years ago
Victor Julien b210bf1290 Fix commandline supplied yaml path being ignored. 14 years ago
Eric Leblond 2d22f667c2 config: use config file in sysconfdir by default. 14 years ago
marcos 8dfddd0a0f Added -T switch to suricata output. 
Simply added the -T to be printed out when suricata is run without any
arguments.  The capability to test a configuration file has been in
suricata for some time, just doesn't show up as an option right now.
 14 years ago
Victor Julien 5e95524122 Improve error reporting in case of syntax errors in the address and port vars. 14 years ago
Victor Julien 086a934ca9 #449: fix md5 calculation in daemon mode. 14 years ago
Anoop Saldanha 678763c3f4 bug #454 - global check to see if address and port vars are properly configured 14 years ago
Jason Ish 105173939b Implement single, autofp and workers run modes for DAG interfaces. Includes multiple interface support. 
Remove auto mode due to bad performance.
 14 years ago
Xavier Lange fea5e68a7b Include conf_test in special cases for unset RUNMODE 14 years ago
Xavier Lange 1d774dae61 Make conf_test local. Simplify if/else to if. 14 years ago
Xavier Lange 1ae10b9a42 Do not spawn threads for conf test 14 years ago
Xavier Lange eaacc5d0fe Added conf_test flag and behavior 14 years ago
Anoop Saldanha 4d192a6881 update all spm algos to use 16 bit pattern lengths. Should compress a lot of tables 14 years ago
Victor Julien 8d1fe9f2fa Make 'autofp' the default runmode. Increase default max-pending-packets to 1024. Move some advanced and uncommonly changed settings down in the stock suricata.yaml. Closes #433. 14 years ago
Anoop Saldanha 5ffb050ada Adapt flow tmqh counters to be atomic vars. Remove support for active flows q handler. Introduce SC_ATOMIC_SET 14 years ago
Anoop Saldanha 4e417b72b5 support flow q handler schedulers active_flows and active_packets. Support new yaml option autofp_scheduler. Support for printing q handler stats as well 14 years ago
Victor Julien 0b3f6c464a Make list-app-layer-protos option name match the help explanation. Make sure it works w/o passing a config. 14 years ago
Anoop Saldanha 109662450d Add new command line option --list-app-layer-protocols to list supported app layer protocols in sigs 14 years ago
Victor Julien feff6f7705 Clean up error message. 14 years ago
Eileen Donlon 85c364da09 disallow-use-of-configuration-file-with-unittests 14 years ago
Victor Julien d908e707d7 profiling: add per lock location profiling 
Add profiling per lock location in the code. Accounts how often a
lock is requested, how often it was contended, the max number of
ticks spent waiting for it, avg number of ticks waiting for it and
the total ticks for that location.

Added a new configure flag --enable-profiling-locks to enable this
feature.
 14 years ago
Victor Julien a05df345de Introduce host table, make tag use it 
Add a host table similar to the flow table. A hash using fine grained
locking. Flow manager for now takes care of book keeping / garbage
collecting.

Tag subsystem now uses this for host based tagging instead of the
global tag hash table. Because the latter used a global lock and the
new code uses very fine grained locking this patch should improve
scalability.
 14 years ago
Victor Julien 0150e66ede flow engine: improve scalability 
Major redesign of the flow engine. Remove the flow queues that turned
out to be major choke points when using many threads. Flow manager now
walks the hash table directly. Simplify the way we get a new flow in
case of emergency.
 14 years ago
Victor Julien cdba2f50d1 Various fixes and improvements based on feedback by Coverity analyzer. 14 years ago
Nikolay Denev 139768dd58 Do not use underscored config vars internally. 14 years ago
Victor Julien 2197f1a625 file-inspection: split 'file' output module into file-store and file-log. Store stores files. Log logs json records. 14 years ago
Victor Julien 860971eca0 Misc afpacket changes. 14 years ago
Victor Julien 337f7861a4 Make sure that if not built against libnss, we still compile. Only no md5 for you then\! 14 years ago
Victor Julien 69b3df96fb Initial on the fly MD5 calculation for extracted files using libnss. 14 years ago
Anoop Saldanha e682796d03 feature #414 - support listing supported keywords. Remove support for dummy keywords __address__, __proto__, __port__. Remove support for recursive keyword and all references to it 14 years ago
Anoop Saldanha 09313cf9bd Support http stat code detection engine, fast pattern(mpm engine included). Fix http stat code setup function. Fix pcre option for stat msg keyword. With this the pcre options for server_body is Q, for stat_msg is Y and for stat_code is S 14 years ago
Anoop Saldanha 2007c2711c Support http stat msg detection engine, fast pattern(mpm engine included). Fix http stat msg setup function. Fix pcre option for stat msg keyword 14 years ago
Victor Julien 489b8b8bcc Allow other yaml files to be included in the main yaml. 14 years ago
Victor Julien 1d9f6ff8f2 Initial Napatech support by Randy Caldejon / nPulse. 14 years ago
Victor Julien 87e6be610a Issue warning if libhtp version used is not up to date. 14 years ago
Victor Julien 39ef24ccc4 Fix pcap -i mode. 14 years ago
Victor Julien e526525f83 Fix pcap -i <ip>. 14 years ago
Victor Julien 35467db151 Indicate that the Suricata version used is a release or a git checkout. 14 years ago
Victor Julien 28e15be526 Clean up default output. Use simpler output format for releases. 14 years ago
Victor Julien 515d070554 Print elapsed time with millisecond precision. 14 years ago
Victor Julien 1ac6054c23 Clean up configure check for htp_tx_get_response_headers_raw. Misc changes. 14 years ago
Victor Julien effe01ae7b Add Init and DeInit calls to the thread module API. 14 years ago
Victor Julien 08f3ef7685 Reshuffle version printing so -V prints it only once. 14 years ago
deltay 37dc83d411 ignore signal SIGPIPE and SIGSYS 14 years ago
Victor Julien 89f83e714c Introduce http_server_body keyword. 
The http_server_body content modifier modifies the previous content to inspect
the normalized (dechunked, unzipped) http_server_body. The workings are similar
to http_client_body. Additionally, a new pcre flag was introduced "/S".

To facilitate this change the signature flags field was changed to be 64 bit.
 14 years ago
Eric Leblond 6e7a8f38bf ipfw: Add support for autofp and worker runmode 
This patch convert ipfw code to the PcktAcqLoop API and
rework the running mode to use the running mode wrapper
already used by NFQ.
 14 years ago
Eric Leblond 5cfdd7594f util-device: Modify function name. 
This patch modifies LiveBuildQueueList name to LiveBuildDeviceList
to have a consistent naming accross function. It also adds a
doxygen comment to add author and description of util-device.c
file.
 14 years ago
Victor Julien 678213c9f4 Fix ParseSizeString return code and a compiler warning. 14 years ago
Anoop Saldanha 7c9d1b80fd Update size parsing API with new calls for returing u8, u16, u32 and u64 values. Make updates in the codebase to use these new calls 14 years ago
Anoop Saldanha e0c13434ef bug 333 - support new Size Parsing API. Update various conf params inside the engine to use this API to parse sizes in the format xxx <-just the no represents bytes, xxxkb <- kilobytes, xxxmb <- megabytes, xxxgb <- gigabytes, where xxx is a \d+ 14 years ago
Eileen Donlon dbdf2d888f Enable/disable core dump in config (feature 319) 14 years ago
Victor Julien 34450b9b57 Don't parse layers / ext headers above ipv6 frag header. This is taken care of by defrag. 14 years ago
Victor Julien 63c9a3ab85 Remove duplicate include. 14 years ago
Victor Julien f4a6f4b293 Add libmagic detection, linking and a basic API. 14 years ago
Victor Julien 1eef36b011 Initial checkin of a log-file module, that can write files extracted from flows to disk. 14 years ago
Pablo Rincon 6d60b3a747 filename and fileext keywords 14 years ago
deltay 211193b0af Get pidfile from config file if not available in command options 14 years ago
Victor Julien 8cc82c7241 Add -S commandline option that loads a rule file exclusively. Issue #338. 14 years ago
Victor Julien 55da9787a4 Win32 compile fixes. 14 years ago
Eric Leblond 391d813c82 Remove unified1 output module. 14 years ago
Eric Leblond 27f1d88374 Add pcap-info alert format. 
This patch adds a new alert format called pcap-info. It aims at
providing an easy to parse one-line per-alert format containing
the packet id in the parsed pcap for each alert. This permit to
add information inside the pcap parser.

This format is made to be used with suriwire which is a plugin for
wireshark. Its target is to enable the display of suricata results
inside wireshark.

This format doesn't use append mode per default because a clean file
is needed to operate with wireshark.

The format is a list of values separated by ':':
  Packet number:GID of matching signature:SID of signature:REV of signature:Flow:To Server:To Client:0:0:Message of signature
The two zero are not yet used values. Candidate for usage is the
part of the packet that matched the signature.
 14 years ago
Eric Leblond 788fa1e5a1 pfring: Fix typo in help. 14 years ago
Eric Leblond c75fffe92d Improve help message 
Usage of command line has evolved with the introduction of long option.
This patch updates the description of the related options.
 14 years ago
Eric Leblond dc075a74a2 pcap: add --pcap option 
This patch adds a --pcap option which can be used to select or
an interface if an argument is provided or the interfaces defined
in the configuration file.
 14 years ago
Eric Leblond d9d8286671 pfring: restore compatibility with v1.0 config 
Compatibility of pfring module with previous version was broken. This
patch restores backward compatibility.
 14 years ago
Eric Leblond a64dcfeba2 pfring: use factorisation function 
This patch convert pfring to pktacqloop and use the new factorisation
function. This also fixes commmand line parsing of pfring which is now
able to work like af-packet:
 - 'suricata -c s.yaml --pfring' start suricata with all interfaces in
 conf
 - 'suricata -c s.yaml --pfring=eth2' start suricata on eth2
 14 years ago
Anoop Saldanha b6ba944e6d Rearrange flow manager functions into flow-manager.[ch]. Some other minor changes/updates 14 years ago
Anoop Saldanha 727a950e39 Move time elapsed right after we finish all packet processing 14 years ago
Anoop Saldanha c365bafbf6 We now inspect timed out streams + streams not processed as yet, at engine shutdown 14 years ago
Anoop Saldanha a844eecb0e - Updated all runmodes to use synchronization points, right before each thread(slot function) tries to de-init the thread. - Main thread now first disables receive thread(s) before it kills receive and rest of the threads. 14 years ago
Victor Julien 3d396e8b1e Update PCRE JIT code to support official JIT implementation in pcre-8.20-RC1. 14 years ago
Eric Leblond fb4be6199f af-packet: change option name 
This patch changes the option name. af-packet long option is
now used instead of -a to mimic pfring behaviour.

This patch improves the standard parsing of the command line.
Running
 suricata -c suricata.yaml --af-packet
will start a suricata running in AF_PACKET mode listening on all
interfaces defined in the suricata.yaml configuration file. The
traditionnal syntax:
 suricata -c suricata.yaml --af-packet=ppp0
will start a suricata listening on ppp0 only.
 14 years ago
Eric Leblond fbca1a4e6b af-packet: multi interface support 
This patch adds multi interface support to AF_PACKET. A structure
is used at thread creation to give all needed information to the
input module. Parsing of the options is done in runmode preparation
through a dedicated function which return the configuration in a
structure usable by thread creation.
 14 years ago
Eric Leblond e80b30c082 af-packet: finalize code 
This patch handles the end of AF_PACKET socket support work. It
provides conditional compilation, autofp and single runmode.

It also adds a 'defrag' option which is used to activate defrag
support in kernel to avoid rx_hash computation in flow mode to fail
due to fragmentation.

This patch contains some fixes by Anoop Saldanha, and incorporate
change following review by Anoop Saldanha and Victor Julien.

AF_PACKET support is only build if the --enable-af-packet flag is
given to the configure command line. Detection of code availability
is also done: a check of the existence of AF_PACKET in standard
header is done. It seems this variable is Linux specific and it
should be enough to avoid compilation of AF_PACKET support on other
OSes.
Compilation does not depend on up-to-date headers on the system. If
none are present, wemake our own declaration of FANOUT variables. This
will permit compilation of the feature for system where only the kernel
has been updated to a version superior to 3.1.
 14 years ago
Eric Leblond 871b21892a factorize pcap live device function 
They are not specific to pcap and could thus be used in other module.
 14 years ago
Eric Leblond c45d898572 af-packet: basic support for AF_PACKET socket 
This patch provides basic support for AF_PACKET socket. It is
completed by a subsequent patches prodiding extended features
and bugfixes.
 14 years ago
Victor Julien 820b0ded82 Add per packet profiling. 
Per packet profiling uses tick based accounting. It has 2 outputs, a summary
and a csv file that contains per packet stats.

Stats per packet include:
 1) total ticks spent
 2) ticks spent per individual thread module
 3) "threading overhead" which is simply calculated by subtracting (2) of (1).

A number of changes were made to integrate the new code in a clean way:
a number of generic enums are now placed in tm-threads-common.h so we can
include them from any part of the engine.

Code depends on --enable-profiling just like the rule profiling code.

New yaml parameters:

profiling:
  # packet profiling
  packets:

    # Profiling can be disabled here, but it will still have a
    # performance impact if compiled in.
    enabled: yes
    filename: packet_stats.log
    append: yes

    # per packet csv output
    csv:

      # Output can be disabled here, but it will still have a
      # performance impact if compiled in.
      enabled: no
      filename: packet_stats.csv

Example output of summary stats:

IP ver   Proto   cnt        min      max          avg
------   -----   ------     ------   ----------   -------
 IPv4       6     19436      11448      5404365     32993
 IPv4     256         4      11511        49968     30575

Per Thread module stats:

Thread Module              IP ver   Proto   cnt        min      max          avg
------------------------   ------   -----   ------     ------   ----------   -------
TMM_DECODEPCAPFILE          IPv4       6     19434       1242        47889      1770
TMM_DETECT                  IPv4       6     19436       1107       137241      1504
TMM_ALERTFASTLOG            IPv4       6     19436         90         1323       155
TMM_ALERTUNIFIED2ALERT      IPv4       6     19436        108         1359       138
TMM_ALERTDEBUGLOG           IPv4       6     19436         90         1134       154
TMM_LOGHTTPLOG              IPv4       6     19436        414      5392089      7944
TMM_STREAMTCP               IPv4       6     19434        828      1299159     19438

The proto 256 is a counter for handling of pseudo/tunnel packets.

Example output of csv:

pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading
1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337

First line of the file contains labels.

2 example gnuplot scripts added to plot the data.
 14 years ago
Anoop Saldanha 975ebf2e4f Minor changes to move function calls that kills threads + frees resources to the clean up phase right to the end of main thread 14 years ago
Anoop Saldanha 576ec7da66 smtp parser support 15 years ago
Gerardo Iglesias Galvan a9509eea2e Fix very minor mem leak when setting bpf filter 15 years ago
Gerardo Iglesias Galvan 313067f47f Check return code of DetectEngineCtxInit at startup 15 years ago
deltay 2856cf0de5 #277 ignore bpf filter if fread failed. 15 years ago
deltay e3270f20b2 #277 Add -F option to load bpf filter from file 15 years ago
William d74fe520e5 Experimental support for PCRE-sljit enable via --enable-pcre-sljit 15 years ago
William 85643fe780 Convert to logging perf stats to file by default. Add a few columns to output avg ticks per match, avg ticks non match, allow sorting on based on them. 15 years ago
Victor Julien 36917c7d66 Fix not using new htp callback when using the bundled htp. Add indication to --build-info. Fix valgrind warning in test and further improve test. 15 years ago
Anoop Saldanha 000ce98cd1 push all proto detection code into their respective app parser register functions for every alproto 15 years ago
Victor Julien a0799f0ff9 Wait longer at shutdown before concluding it's taking too long. Hopefully enables our slow QA boxes to complete in time. 15 years ago
Victor Julien cd75201dc7 Fix pfring commandline handling. 15 years ago
Victor Julien 5f2a0653b4 If engine shutdown (processing in-engine packets) times out, exit Suricata with EXIT_FAILURE. 15 years ago
Victor Julien 8978266a91 If shutdown doesn't complete processing all packets that are already in the engine within 30 seconds, force quit. 15 years ago
Victor Julien 5d2f633c48 Properly initialize pfring runmode before using it. Fix malformed conf api calls. 15 years ago
Anoop Saldanha 966119b6aa support for http_raw_uri keyword + mpm engine 15 years ago
Anoop Saldanha e4d890e186 modify runmode api to accept conf runmode paramter as a char string, instead of an interger id 15 years ago
Anoop Saldanha 229f7281ea list runmodes. Allow specification of runmode id from cof file. Also allow for command line override 15 years ago
Anoop Saldanha d7c707e656 modify runmodes to take all arguments from the conf API 15 years ago
Anoop Saldanha a165d45da9 naming changes for runmodes 15 years ago
Anoop Saldanha 5c880b04c9 fix ipv4 defrag + fix recursion level in defrag pseudo packet 15 years ago
William Metcalf 023a0f94a2 first stab at pcap logging no rotating buff etc 15 years ago
Anoop Saldanha 013d3aea1c update ssl parser test. Some minor indentation changes 15 years ago
Anoop Saldanha 4e8de99dcd tls/ssl parser modifications/fixes. We now have just one file doing all the ssl parsing stuff, i.e. app-layer-tls.[ch], instead of app-layer-ssl.[ch] and app-layer-tls.[ch] 15 years ago
Victor Julien 07a85427dd Add --build-info command line option to output some basic build settings. 15 years ago
Victor Julien 07776c113b Fix valgrind error on pfring_recv, rename threads from RecvPfring to RxPfring so the name still looks right for 100+ threads. Add --pfring commandline option that just enables pfring, then takes interface from config. 15 years ago
Victor Julien 1c9e48ae98 Fix compilation error on non-pfring systems. 15 years ago
Victor Julien 91f28afef4 Add option to PF_RING to have multiple reader threads. Improve general performance of the PF_RING module. 15 years ago
Gurvinder Singh 7d0781b349 added support to log dropped packet as netfilter logs while in inline mode 15 years ago
Gurvinder Singh 8f8b1212af support for ssl_version keyword 15 years ago
Gurvinder Singh e5edc6e8e3 add the support to log the fast.log alerts type to syslog 15 years ago
Eric Leblond 1e600c1054 source-nfq: add simulated non-terminal NFQUEUE verdict 
This patch adds a new mode for NFQ inline mode. The idea is to
simulate a non final NFQUEUE rules.
This permit to do send all needed packets to suricata via a simple
FORWARD rule:
    iptables -I FORWARD -m mark ! --mark $MARK/$MASK -j NFQUEUE
And below, we have a standard filtering ruleset.

To do so, suricata issues a NF_REPEAT instead of a NF_ACCEPT verdict and
put a mark ($MARK) with respect to a mask ($MASK) on the handled packet.

NF_REPEAT verdict has for effect to have the packet reinjected at start
of the hook after the verdict. As it has been marked by suricata during
the verdict it will not rematch the initial rules and make his way to
the following classical ruleset.

Mode, mark and mask can be configured via suricata.yaml file with the
following syntax:
   nfq:
     repeat_mode: (false|true)
     mark: $MARK
     mask: $MASK
Default is false to preserve backward compatibility.

Signed-off-by: Eric Leblond <eric@regit.org>
 15 years ago
Eric Leblond 8330747234 Add multi queue support to NFQ run mode 
This patch adds support for multiple Netfilter queue
in the NFQ run mode. Suricata can now be started on
multiple queue by using a comma separated list of
queue identifier on the command line. The following syntax:
	suricata -q 0 -q 1 -c /opt/suricata/etc/suricata.yaml
will start a suricata listening to Netfilter queue 0 and 1.

Signed-off-by: Eric Leblond <eric@regit.org>
 15 years ago
Victor Julien 6fca55e068 Add some debug output to app-layer-htp. 15 years ago
Anoop Saldanha c9897a44a4 fast pattern support for http_cookie. Also support relative modifiers 15 years ago
Anoop Saldanha bbbedaf963 fast pattern support for http_method. Also support relative modifiers 15 years ago
Eric Leblond 9c2bdc6d0c Main loop: increase timer. 
Timer in the main loop was of 100 usec. This patch increases it
to 10 ms which should be a reasonnable delay to declare some threads
dead.

Signed-off-by: Eric Leblond <eric@regit.org>
 15 years ago
Eric Leblond 67b95c8c4d Auto discovery of default packet size 
If default-packet-size is not set, it is possible in some case to
guess a correct value.

If PCAP or PF_RING are used we are linked to a "physical" interface.
Thus, it is possible to get information about the link MTU and
hardware header size. This give us the ability to auto discover a
decent default-packet-size.
If suricata is running under a different running-mode, it will
default to 1514.
 15 years ago
Eric Leblond dd038c1906 Modify files to avoid direct pckt payload access 
This patch implements the needed modification of payload access
in a Packet structure to support the abstraction introduced by
the extended data system.
 15 years ago
Eric Leblond e802e1ed16 Modify Packet structure and prepare accessor. 
This patch modifies decode.c and decode.h to avoid the usage
by default of a bigger than 65535 bytes array in Packet structure.
The idea is that the packet are mainly under 1514 bytes size and
a bigger size must be supported but should not be the default.

If the packet length is bigger than DFLT_PACKET_SIZE then the
data are stored in a dynamically allocated part of the memory.

To ease the modification of the rest of the code, functions to
access and set the payload/length in a Packet have been introduced.

The default packet size can be set at runtime via the default-packet-size
configuration variable.
 15 years ago
Victor Julien 878d3d87db Add (experimental) support for using multiple pcap devices to acquire packets from. Just passing multiple -i <dev> options on the commandline will activate this. Windows not yet supported. 15 years ago
Victor Julien e3bde3e95d Add a simple revision based on the git rev to the version number, like a build number. 15 years ago
Victor Julien ffcd512167 Clean up packet pool handler on shutdown. 15 years ago
Victor Julien 3710296057 Cleanup defrag engine on shutdown. 15 years ago
Anoop Saldanha d241e51b32 adapt fast pattern engine analysis to reflect the new changes made to your mpm design 15 years ago
Anoop Saldanha 42c1287028 renintroduce g_u8_lowercase_table for b2g cuda 15 years ago
Anoop Saldanha c734cd1bdd make cuda mpm parameters configurable 15 years ago
Anoop Saldanha 7ec0382774 support fast pattern for http raw header. Also support relative modifiers for http raw header 15 years ago
Anoop Saldanha c61c68fd36 mpm and fast pattern support for http_header. Also support relative modifiers for http_header 15 years ago
Anoop Saldanha 5c6a65dc58 support relative modifiers for http_client_body. Introduce body processing engine in detect-engine-hcbd.[ch] 15 years ago
Anoop Saldanha eade60f0fd make some name changes. break PopulateMpm(). Set the avoid mpm double check flags 15 years ago
Anoop Saldanha 46b4806d8e use a single populatempm() function to add the right content for mpm 15 years ago
Anoop Saldanha bc99328ec8 define a new conf paramter detect-engine:inspection-recursion-limit; Defines a recursion limit for content inspection code 15 years ago
Victor Julien 001dcaae84 Minor reference.config support changes: improve error handling, reduce hash table size. 15 years ago
Anoop Saldanha 88d94b136d Support for reference.config file 15 years ago
Anoop Saldanha 2cdb5be391 Print out file name for fast_pattern engine_analysis. Also add some info logs 15 years ago
Anoop Saldanha 0ef684705c support single mpm context distribution across sghs in staging. Also see to it that ac works fine with this setup 15 years ago
Anoop Saldanha a85fa6b792 support for fast_pattern only and fast_pattern:offset,length. Also support the new option for engine-analysis 15 years ago
Pablo Rincon b3a8f0a90f Fix asn1 decoder frame oob mem. Adding max stack frames to suricata.yaml 15 years ago
Gurvinder Singh 3eab715153 support for printing protocol names for known protocol 15 years ago
Victor Julien 1859ed54c7 Add memcmp api with a plain memcmp function and a SSE3 accelerated memcmp. 15 years ago
Victor Julien 3971bcc83a Switch to faster tolower function for u8_tolower. 15 years ago
Pablo Rincon 5c43db85ce Drop streams on inline mode when a drop rule match from a reassembled stream and/or app layer inspection 15 years ago
Pablo Rincon 76af1b049b Make malloc errors on initialization stage a fatal error, resulting on a exit() call 15 years ago
Pablo Rincon 9d7baa7a9f Adding ssh app layer module with two new keywords: ssh.protoversion and ssh.softwareversion 15 years ago
Anoop Saldanha 07491f8887 add --list-cuda-cards option to list the cuda cards on the system. Add conf parameter to select the cuda device to use. Also change the threshhold limit to 2.4k packets to buffer 16 years ago
Anoop Saldanha 33f4beb0bc batching of packets support for cuda b2g mpm. Supported for both 32 and 64 bit platforms 16 years ago
Victor Julien 8cdd02877f Add unittests for ringbuffer. 16 years ago
Anoop Saldanha 36e4b1830e add pcre with U modifiers to the umatch sigmatch list. fix for bug 155 16 years ago
Pablo Rincon eed0ef6e69 Adding tag keyword support 16 years ago
Kirby Kuehl c3b9305259 dcerpc udp support 16 years ago
Victor Julien 6519a86ec7 Move packet pool to ringbuffer, update packet pool api and ringbuffer api. Remove memset usage from PACKET_RECYCLE, add proper cleanup macros. 16 years ago
Victor Julien 012e602c3f Add a (disabled by default) flow pinned runmode for file pcap. 16 years ago
Anoop Saldanha 45ea0d914e dce stub content keywords support using dcepayload.c support for all dce related content keywords 16 years ago
Jason MacLulich 835630efbd Add initial support for reading packets from a DAG card, we only support reading from a single stream at this time. 
Use the --dag <dagname> cmd line option to specify from which DAG card to read pkts
from.

Issue at the moment with pkts being ejected during shutdown -- at the moment we
ignore any packets that are not of link type Ethernet.
 16 years ago
Ondrej Slanina 6bf7d76005 added possibility to run suricata as WIN32 service 16 years ago
Ondrej Slanina 79443b1991 added INT and TERM signals on WIN32 16 years ago
William Metcalf 9ce1399db8 Add option for setting pcap buffer size if it is available 16 years ago
Victor Julien 4e7df60b2f Make pcap file mode read multiple packets per 'read'. Update threading model to deal with this. 16 years ago
Victor Julien 6f502f0da5 lockfree ringbuffer wip2, including proper shutdown. 16 years ago
Jason Ish a93b2e6b84 Support for reading ERF files. 16 years ago
William Metcalf 2eef905c07 GPL and Copyright header updates. 16 years ago
Victor Julien 2910759943 Rename TranslateIPToPcapDev to PcapTranslateIPToDevice and make the length argument size_t. 16 years ago
Ondrej Slanina 55d0d1e7a1 added support for finding pcap device via it's IP. 16 years ago
Gurvinder Singh 5fe1dc1d24 support for sslv2/sslv3 their unit tests and better stream no reassembly flag handling 16 years ago
Victor Julien b8641f300d Rename asn1 files, fix an invalid free, fix improper init of vars in one unittest. 16 years ago
Pablo Rincon 3fa3229e01 ASN1 decoder and keyword implementation 16 years ago
Victor Julien 70b32f7380 First stab at creating a stateful detection engine. 
Stateful detection for app layer detection keywords, except uricontent. Stores it's partial results in the flow structure. Other modifications:

- Generalize transaction tracking, logging and inspection.
- Adapt http and dcerpc to use the new transaction handling.
- Stream engine now always notifies app layer of a stream eof.

This commit fixes bug #124.
 16 years ago
Jason Ish 18e5ac8cde Basic rule profiling even though the results may be skewed by a bad rule in a grouping of rules. 16 years ago
William Metcalf b629b7c5c1 only show cli opts via help that we have support for 16 years ago
Victor Julien 692eb935ea Fix updated memory api using debug mode by default. Small cleanups. 16 years ago
Gerardo Iglesias Galvan 9f4fae5b1a Fix inconsistent use of dynamic memory allocation 16 years ago
Victor Julien e741bd0202 Cleanup packet recycling code. Fix issues in the packet tunnel/pseudo code. 16 years ago
Victor Julien e1a8c8f76c Switch time api from mutex to spinlock. 16 years ago
Victor Julien 19584d0416 Fix tunneled and defrag reassembled packets with the new pending limits. 16 years ago
Victor Julien ccf22cf563 Move to different way of enforcing max_pending. Should require less locks. 16 years ago