aiden
/
suricata
mirror of https://github.com/OISF/suricata
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
4,472 Commits
8 Branches
165 Tags
231 MiB
main
main-7.0.x
main-8.0.x
master-6.0.x
master-5.0.x
master-4.1.x
master-4.0.x
master-3.2.x
suricata-8.0.2
suricata-7.0.13
suricata-8.0.1
suricata-7.0.12
suricata-8.0.0
suricata-7.0.11
suricata-8.0.0-rc1
suricata-8.0.0-beta1
suricata-7.0.10
suricata-7.0.9
suricata-7.0.8
suricata-7.0.7
suricata-7.0.6
suricata-6.0.20
suricata-7.0.5
suricata-6.0.19
suricata-6.0.18
suricata-7.0.4
suricata-6.0.17
suricata-7.0.3
suricata-6.0.16
suricata-7.0.2
suricata-6.0.15
suricata-7.0.1
suricata-6.0.14
suricata-7.0.0
suricata-7.0.0-rc2
suricata-6.0.13
suricata-6.0.12
suricata-6.0.11
suricata-7.0.0-rc1
suricata-6.0.10
suricata-6.0.9
suricata-7.0.0-beta1
suricata-6.0.8
suricata-6.0.7
suricata-6.0.6
suricata-5.0.10
suricata-6.0.5
suricata-5.0.9
suricata-6.0.4
suricata-5.0.8
suricata-6.0.3
suricata-5.0.7
suricata-6.0.2
suricata-5.0.6
suricata-6.0.1
suricata-5.0.5
suricata-4.1.10
suricata-4.1.9
suricata-5.0.4
suricata-6.0.0
suricata-6.0.0-rc1
suricata-6.0.0-beta1
suricata-5.0.3
suricata-4.1.8
suricata-4.1.7
suricata-5.0.2
suricata-4.1.6
suricata-5.0.1
suricata-5.0.0
suricata-5.0.0-rc1
suricata-4.1.5
suricata-5.0.0-beta1
suricata-4.1.4
suricata-4.1.3
suricata-4.0.7
suricata-4.1.2
suricata-4.1.1
suricata-4.1.0
suricata-4.0.6
suricata-4.1.0-rc2
suricata-4.1.0-rc1
suricata-4.0.5
suricata-4.1.0-beta1
suricata-4.0.4
suricata-4.0.3
suricata-4.0.2
suricata-3.2.5
suricata-4.0.1
suricata-3.2.4
suricata-4.0.0
suricata-4.0.0-rc2
suricata-3.2.3
suricata-4.0.0-rc1
suricata-4.0.0-beta1
suricata-3.2.2
suricata-3.1.4
suricata-3.2.1
suricata-3.2
suricata-3.2RC1
suricata-3.1.3
suricata-3.2beta1
suricata-3.1.2
suricata-3.1.1
suricata-3.1
suricata-3.0.2
suricata-3.1RC1
suricata-3.0.1
suricata-3.0.1RC1
suricata-3.0
suricata-2.0.11
suricata-3.0RC3
suricata-3.0RC2
suricata-3.0RC1
suricata-2.0.10
suricata-2.0.9
suricata-2.1beta4
suricata-2.0.8
suricata-2.0.7
suricata-2.1beta3
suricata-2.0.6
suricata-2.0.5
suricata-2.1beta2
suricata-2.0.4
suricata-2.1beta1
suricata-2.0.3
suricata-2.0.2
suricata-2.0.1
suricata-2.0.1rc1
suricata-2.0
suricata-2.0rc3
suricata-2.0rc2
suricata-2.0rc1
suricata-2.0beta2
suricata-1.4.7
suricata-1.4.6
suricata-1.4.5
suricata-2.0beta1
suricata-1.4.4
suricata-1.4.3
suricata-1.4.2
suricata-1.4.1
suricata-1.3.6
suricata-1.4
suricata-1.3.5
suricata-1.4rc1
suricata-1.3.4
suricata-1.4beta3
suricata-1.3.3
suricata-1.4beta2
suricata-1.3.2
suricata-1.4beta1
suricata-1.3.1
suricata-1.3
suricata-1.3rc1
suricata-1.3beta2
suricata-1.3beta1
suricata-1.2.1
suricata-1.2
suricata-1.2rc1
suricata-1.2beta1
suricata-1.1.1
suricata-0.8.2
suricata-1.0.0
suricata-1.0.1
suricata-1.0.2
suricata-1.0.3
suricata-1.0.4
suricata-1.0.5
suricata-1.1
suricata-1.1beta1
suricata-1.1beta2
suricata-1.1beta3
suricata-1.1rc1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '4dd605ae3b'
${ noResults }
Commit Graph

161 Commits (4dd605ae3b1aa66d16ef98e19fed5aca44269bac)

Author SHA1 Message Date
Victor Julien 404868c28b Get rid of strcasestr call as win32 doesn't have it. 14 years ago
Anoop Saldanha d23e775ae2 fix threading bug. Main thread's restart TV code waiting on a failed TV. Now main thread sets the de_init flag before waiting on the failed thread. Thanks to Eric Leblond for reporting it 14 years ago
Anoop Saldanha d68f182ebd introduce SCPerfSyncCounters/SCPerfSyncCounters macro to synchronize counters 14 years ago
pilcrow ed69eeab14 Safer macro parenthesization and do/while use 14 years ago
Victor Julien 1ab6443e44 Fix compilation when profiling is enabled. 14 years ago
Anoop Saldanha 7c729d2d53 some more code cleanup + comments added 14 years ago
Anoop Saldanha 3f1c4efceb Add new flags var to tm module. TMs can now set flags to identify special properties. Also use these to identify receive TMs 14 years ago
Anoop Saldanha 54f6e4ff4d Merge thread kill functions. Merge slot's tm_id with the one used by packet profiling. Remove some junk unused code from ms sync pts. Timeout setup cleanup as well. packet q dbg_maxlen now u32 var. 14 years ago
Anoop Saldanha 6c95526423 Introduce a new wrapper macro that wait loops till the flag(s) in question have been set 14 years ago
Anoop Saldanha a7acf9ea8f Remove all code introduced earlier concerned with ms sync points 14 years ago
Anoop Saldanha b0a588beeb Introduce another solution to solve stream timeout shutdown issue using thread flags. No more MSSyncPts 14 years ago
Anoop Saldanha f2bcf9ea2c modify post_pq packet handling. 
- Lock the q just once, once we have detected the presence of packet(s)
  in the queue.  Unlock it when we consume all packets from the q.
 14 years ago
Anoop Saldanha 9256c7bf0a always keep queue locked till we exit flowprune. Should prevent potential threading issues 14 years ago
Anoop Saldanha a559bfc165 signal the post pq if possible, whenever pseudo packets are injected into engine flow. Also carry out post pq processing irrespective of packet retrieval from the flow. 14 years ago
Anoop Saldanha 8363533a02 support for forced stream reassembly for to be pruned flows 14 years ago
Anoop Saldanha 15359dc47e Slot structure now holds the TV it belongs to 14 years ago
Anoop Saldanha c365bafbf6 We now inspect timed out streams + streams not processed as yet, at engine shutdown 14 years ago
Anoop Saldanha 56432cee16 Single thread kill also checks if inq is cleared before shutting down 14 years ago
Anoop Saldanha 8fa923c5ac - All threads also check to see if their inq is cleared before they shutdown. 14 years ago
Anoop Saldanha a844eecb0e - Updated all runmodes to use synchronization points, right before each thread(slot function) tries to de-init the thread. - Main thread now first disables receive thread(s) before it kills receive and rest of the threads. 14 years ago
Anoop Saldanha e567c2d002 Introduce master-slave synchronization support for ThreadVars 14 years ago
Anoop Saldanha 94c5ecb069 introduce inline function version of TmThreadsSlotProcessPkt macro. Retain the macro as well 14 years ago
Anoop Saldanha 3fb65f5ec2 fix local var usage for slot in tm-threads.c 14 years ago
Anoop Saldanha acbcee69ff support post pq packet processing in var slot 14 years ago
Victor Julien baddfcaa1a Extend packet profiling to other thread 'slot' functions. 14 years ago
Victor Julien 820b0ded82 Add per packet profiling. 
Per packet profiling uses tick based accounting. It has 2 outputs, a summary
and a csv file that contains per packet stats.

Stats per packet include:
 1) total ticks spent
 2) ticks spent per individual thread module
 3) "threading overhead" which is simply calculated by subtracting (2) of (1).

A number of changes were made to integrate the new code in a clean way:
a number of generic enums are now placed in tm-threads-common.h so we can
include them from any part of the engine.

Code depends on --enable-profiling just like the rule profiling code.

New yaml parameters:

profiling:
  # packet profiling
  packets:

    # Profiling can be disabled here, but it will still have a
    # performance impact if compiled in.
    enabled: yes
    filename: packet_stats.log
    append: yes

    # per packet csv output
    csv:

      # Output can be disabled here, but it will still have a
      # performance impact if compiled in.
      enabled: no
      filename: packet_stats.csv

Example output of summary stats:

IP ver   Proto   cnt        min      max          avg
------   -----   ------     ------   ----------   -------
 IPv4       6     19436      11448      5404365     32993
 IPv4     256         4      11511        49968     30575

Per Thread module stats:

Thread Module              IP ver   Proto   cnt        min      max          avg
------------------------   ------   -----   ------     ------   ----------   -------
TMM_DECODEPCAPFILE          IPv4       6     19434       1242        47889      1770
TMM_DETECT                  IPv4       6     19436       1107       137241      1504
TMM_ALERTFASTLOG            IPv4       6     19436         90         1323       155
TMM_ALERTUNIFIED2ALERT      IPv4       6     19436        108         1359       138
TMM_ALERTDEBUGLOG           IPv4       6     19436         90         1134       154
TMM_LOGHTTPLOG              IPv4       6     19436        414      5392089      7944
TMM_STREAMTCP               IPv4       6     19434        828      1299159     19438

The proto 256 is a counter for handling of pseudo/tunnel packets.

Example output of csv:

pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading
1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337

First line of the file contains labels.

2 example gnuplot scripts added to plot the data.
 14 years ago
Eric Leblond a8b21066df tm-thread: fix documentation string 14 years ago
Victor Julien 7e1d911215 Small optimizations to pkt acq loop code. 14 years ago
Victor Julien b753ecce50 Implement a pkt acq loop infra with support for pcap-file. 14 years ago
Anoop Saldanha ff7284e7b7 Fix code that allows the engine to restart threads that have exited on failure 14 years ago
Anoop Saldanha 524af82b1a code cleanup in tm-threads.c 14 years ago
Anoop Saldanha 4f7df1029d Unify the use of slots to a single struct for threading API. Remove separate slot append functions for 1slot and varslot 14 years ago
Gerardo Iglesias Galvan f545df3ea7 Fix potential issue in TmThreadsSlot1NoIn 15 years ago
Victor Julien 92ea1f68d4 Exit on thread restart limit reached. 15 years ago
Eric Leblond 4e9231266a Compilation fix for OpenBSD and win32. 
This patch fixes compilation on OpenBSD platform. It is running
fine on a pcap file. The patch should also fix compilation on
WIN32 platform but this is not tested.
 15 years ago
Pablo Rincon 35c168ab03 Fix CPU_* macros for Mac OS X 15 years ago
Eric Leblond f9e453e14c affinity: Use configured 'threads' value if set 
This patch modifies runmodes to make them use the new 'threads'
variable.

Signed-off-by: Eric Leblond <eric@regit.org>
 15 years ago
Eric Leblond 789d46cc3c Add per-cpu prio handling 
This patch updates affinity setting to add a support for per cpu
priority setting. In exclusive mode a thread is dedicated to a CPU.
This patch adds the ability to set the thread prio for all threads
of a family running on a given CPU.

With this patch we can write
    - detect_cpu_set:
        cpu: [ "all" ]
        mode: "exclusive" # run detect threads in these cpus
        low_prio: [ 0 ]
        medium_prio: [ "1-2" ]
        high_prio: [ 3 ]
With this configuration, detect threads assigned to cpu 0 will
have a low priority. Detect threads on cpus 1 and 2 will have
prio medium...

The previous configuration is equivalent to:
    - detect_cpu_set:
        cpu: [ "all" ]
        mode: "exclusive" # run detect threads in these cpus
        low_prio: [ 0 ]
        high_prio: [ 3 ]
        prio: "medium"
because the prio value is used a default.

Signed-off-by: Eric Leblond <eric@regit.org>
 15 years ago
Eric Leblond ea566d6601 Handle management thread with corresponding affinity 
This patch implement the setting of each management threads in
the corresponding thread affinity. This is done by modifiying
thread creation function.

Signed-off-by: Eric Leblond <eric@regit.org>
 15 years ago
Eric Leblond 0809deafc4 Implement function needed for affinity in tm-threads 
This patch features the implementation of affinity related
changes in tm-threads. In place code has been used but some
refactoring has been done to avoid code duplication.

Signed-off-by: Eric Leblond <eric@regit.org>
 15 years ago
Victor Julien bfb6aac495 Sleep after checking for a thread flag in TmThreadWaitOnThreadInit now that the check is so much cheaper. 15 years ago
Victor Julien 9634389b9e Fix TmThreadsUnsetFlag not unsetting flag if __sync_fetch_and_nand was used. 15 years ago
Pablo Rincon fcff1ce7f4 Converting threadvar flags to atomic vars to avoid using the old spinlock 15 years ago
Anoop Saldanha 33f4beb0bc batching of packets support for cuda b2g mpm. Supported for both 32 and 64 bit platforms 16 years ago
Victor Julien 1d74797b17 Attempt to work around NULL packets we're seeing ending up in queues when the compiler has optimized our code. 16 years ago
Victor Julien 37eb2290b0 Add some checks for 'impossible' conditions that become possible after enabling optimizations :-/ 16 years ago
Victor Julien ba12f3c109 Applayer to flow fixes and cleanups. 16 years ago
Victor Julien 3005297af2 Check for being properly setup before activating a thread. Fixes a potential although unlikely null-dereference. 16 years ago
Victor Julien f80016530c Remove unnecessary locking for thread-local packet-queues breaking on Win32. 16 years ago
Victor Julien 53acf08996 Add multi packet reading for pcap live mode. Add a partly lock free multi writer, multi reader ringbuffer. 16 years ago
Victor Julien 4e7df60b2f Make pcap file mode read multiple packets per 'read'. Update threading model to deal with this. 16 years ago
Victor Julien a48a767efc Lockfree ringbuffer wip. 16 years ago
Gerardo Iglesias Galvan 9f4fae5b1a Fix inconsistent use of dynamic memory allocation 16 years ago
Victor Julien 19584d0416 Fix tunneled and defrag reassembled packets with the new pending limits. 16 years ago
William Metcalf ce01927515 Import of GPLv2 Header 050410 16 years ago
Victor Julien 070ed778b8 Libcap-ng support by Gurvinder Singh and myself. Basic support for per thread caps is added, but not activated as it doesn't seem to work yet. Work around for incompatibility between libnet 1.1 and libcap-ng added. 16 years ago
Victor Julien 23cb020493 Fix thread flag code to compile with gnu99 16 years ago
Victor Julien 63cb1818ed Move TmThread*Flag functions to the header so they can be inline with gnu99 as well. 16 years ago
Victor Julien 79a780ffe4 Improve a number of error and info messages. 16 years ago
Victor Julien eeb98c6900 Move SCSetThreadName to proper functions. 16 years ago
Pablo Rincon 25a3a5c6d8 Adding mem wrapper to debug runtime alloc()/free() functions. Fixing some memory leaks. 16 years ago
Jan Jezek 4e72ccf987 Fixed Win32 compilation, unit tests now compile. 16 years ago
Pablo Rincon 7719216575 Setting thread priorities with nice 16 years ago
Victor Julien aa1564791d Disable thread priority code until we understand it better. 16 years ago
Jan Jezek fe6a72befc Code is now compilable on the Win32 platform 16 years ago
Victor Julien b28488508b Cleanup threading cpu affinity and prio output. 16 years ago
Anoop Saldanha 41e6735b92 mpm b2g cuda support added 16 years ago
Pablo Rincon 985d565cfd Change priorities only if we are EUID == 0 16 years ago
Pablo Rincon b482471a7d Adding auto runmodes based on available core/cpu's. Setting thread priorities 16 years ago
Pablo Rincon d0404d8447 Renaming errors with naming conventions 16 years ago
Pablo Rincon ad2c136e8f Renaming errors (naming conventions) 16 years ago
Nick Rogness 2b7b78f1bf Intial IPFW support FreeBSD and OSX 16 years ago
Victor Julien ecf86f9c23 Rename to Suricata. 16 years ago
Victor Julien ba7e8012af Add some debugging and simplify locking for app layer slightly. 16 years ago
Pablo Rincon e26833be3f Changing mutex/spinlocks/conditions naming types 16 years ago
Pablo Rincon 769022f4be Adding support for Mac OS X, FreeBSD, centrailizing mutex/spins/conditions in a macro API, and some unittests 16 years ago
Anoop Saldanha ceb7e495ae refactoring perf stats code 16 years ago
Victor Julien 4fb4dd59e4 Detect errors in the spin locks which somehow seems to fix some deadlocking withing valgrind. 16 years ago
Gurvinder Singh 4edcc530ec fixed unified alert2 ecode and removed printfs 16 years ago
Gurvinder Singh e7952b1331 some changes in threading constants 16 years ago
Gurvinder Singh 40b8afdd56 support for thread exit constants 16 years ago
Gurvinder Singh 48c94bbf03 initial thread code support 16 years ago
Victor Julien 91bc83e5c6 More logging API usage changes. 16 years ago
Anoop Saldanha 157d5e8113 Implementation of the logging module 16 years ago
Victor Julien 597d0e9a20 Fix detection of failed thread startup. Cleanup startup output a bit. 16 years ago
Victor Julien 0d0ffb9963 Reorganize header inclusions. 16 years ago
Victor Julien b6deadd2b4 Fixed a few missing places for tv->flags access. Changed mutex init for packets slightly. 16 years ago
Victor Julien 1858be7a2f Lock threadvars flags using spinlocks. 16 years ago
Anoop Saldanha 3a45b2711c Fixed the Perf API startup issue 16 years ago
Victor Julien a39108843e Small tm module API rename to reflect that Init/Deinit/ExitPrintStats are per thread calls. 16 years ago
Victor Julien 7c5f95db01 Small threading update. 16 years ago
Anoop Saldanha f35d9f0437 threading improvements. Replaced the use of slot(2/3) with varslot. Improve error handling in slot functions. Additional helper functions for thread creation 16 years ago
Victor Julien 3636ca9703 Adding a "flow" queue handler. This queue handler passes packets of the same flow to the same queue. Changed the default IDS mode to use this. 
Some output cleanups, shutdown should be cleaner now.
 16 years ago
Brian Rectanus fa5939ca91 64 bit cleanup part2 17 years ago
Anoop Saldanha 53c21410b6 improve the threading api for the ids 17 years ago
Anoop Saldanha a5fb240a4a Changes added for the Performance Counter API 17 years ago
Victor Julien 7c36b315fd Kill the engine if one of the threads fails to initialize. 17 years ago
Victor Julien c53dfea379 Improve the threading code to enable a single pcap file processing thread. 17 years ago
Victor Julien 689bbfdc45 Rename all structure definitions in the "typedef struct _SomeStruct" format to "typedef struct SomeStruct_" to make the Doxygen output more useful. 
Remove the Trie multi pattern matcher code. It wasn't used anymore.
 17 years ago
Victor Julien 1c2240cfeb Stream reassembly update and WIP code for L7 modules. 17 years ago