Commit Graph

1878 Commits (b8cf50678fc3bdc932bff13886bbfd1b8ebb0643)

Author SHA1 Message Date
Eric Leblond 789d46cc3c Add per-cpu prio handling
This patch updates affinity setting to add a support for per cpu
priority setting. In exclusive mode a thread is dedicated to a CPU.
This patch adds the ability to set the thread prio for all threads
of a family running on a given CPU.

With this patch we can write
    - detect_cpu_set:
        cpu: [ "all" ]
        mode: "exclusive" # run detect threads in these cpus
        low_prio: [ 0 ]
        medium_prio: [ "1-2" ]
        high_prio: [ 3 ]
With this configuration, detect threads assigned to cpu 0 will
have a low priority. Detect threads on cpus 1 and 2 will have
prio medium...

The previous configuration is equivalent to:
    - detect_cpu_set:
        cpu: [ "all" ]
        mode: "exclusive" # run detect threads in these cpus
        low_prio: [ 0 ]
        high_prio: [ 3 ]
        prio: "medium"
because the prio value is used a default.

Signed-off-by: Eric Leblond <eric@regit.org>
14 years ago
Eric Leblond a11e40dedf Pcapfile mode: support for cpu affinity settings
This patch adds support for cpu affinity setting in the pcapfile
runmode.

Signed-off-by: Eric Leblond <eric@regit.org>
14 years ago
Eric Leblond 9d5f08e0d9 Pcap mode: use CPU affinity setting
This patch adds support for CPU affinity settings
in pcap mode.

Signed-off-by: Eric Leblond <eric@regit.org>
14 years ago
Eric Leblond 51df6beb26 Convert RunModeIpsNFQAuto to new affinity mode.
The default NFQ run mode is now using the new affinity system. It
thus can be configured via suricata.yaml.

Signed-off-by: Eric Leblond <eric@regit.org>
14 years ago
Eric Leblond ea566d6601 Handle management thread with corresponding affinity
This patch implement the setting of each management threads in
the corresponding thread affinity. This is done by modifiying
thread creation function.

Signed-off-by: Eric Leblond <eric@regit.org>
14 years ago
Eric Leblond 0809deafc4 Implement function needed for affinity in tm-threads
This patch features the implementation of affinity related
changes in tm-threads. In place code has been used but some
refactoring has been done to avoid code duplication.

Signed-off-by: Eric Leblond <eric@regit.org>
14 years ago
Eric Leblond 77f2b6a7a9 Make runmode parse affinity settings.
This patch modifies runmode to parse configuration file related
to affinity settings. It also prepare the export of the
set_cpu_affinity which was previously local. It is now used
in the affinity and tm-threads files.

Signed-off-by: Eric Leblond <eric@regit.org>
14 years ago
Eric Leblond 2011366429 Include affinity in runmodes and threadvars.
This small patch add inclusion of util-affinity.h in the
files that will have to use affinity related features.

Signed-off-by: Eric Leblond <eric@regit.org>
14 years ago
Eric Leblond 37ee483b75 Add affinity util function and related files
This patch adds two new files which implement advanced affinity
settings.

Signed-off-by: Eric Leblond <eric@regit.org>
14 years ago
Eric Leblond 96e3852191 source-nfq: add define of SOL_NETLINK
Signed-off-by: Eric Leblond <eric@regit.org>
14 years ago
Martin Beyer 396b750414 Fixed optional args in SCCudaModuleGetGlobal 14 years ago
Martin Beyer 5dc5d73a66 fixed NULL checks in util-cuda 14 years ago
Martin Beyer 0d4ac48aa0 added texture reference api to util-cuda 14 years ago
Victor Julien d10cf5b4e3 Increase stream msg size. 14 years ago
Victor Julien ec5b622553 Inspect all stream msgs at any time when running in stream-inline mode. Skip detection for packets flagged for dropping before detect. 14 years ago
Victor Julien 48c7f18453 Fix bug in the segment insert code causing an inconsistent segment list in some overlap conditions. 14 years ago
Victor Julien 929ce0bb9b Add a counter to NFQ for modified packets. 14 years ago
Victor Julien 05539d7357 Fix a reassembly overlap issue. Fix a inline reassembly gap handling issue. 14 years ago
Victor Julien 00e4dde6a6 Fix PKT_STREAM_EOF never being set, resulting in some raw stream chunks never being inspected. Improve debug output. 14 years ago
Victor Julien e92ab40d39 Fix compilation for non-DEBUG case. 14 years ago
Victor Julien 1dca88fe69 Do the actual checksum recalculation and packet replacement on modifing a packet in the stream engine. 14 years ago
Victor Julien 2db06cc79e Improve Inline reassembly wrt to GAP handling. Add more tests. 14 years ago
Victor Julien 29e02abc94 Expand and fix stream unittest helpers. 14 years ago
Victor Julien 121e9c72aa Add more debug printing of reassembled data into the app layer api. 14 years ago
Victor Julien 4c82c0e750 Improve RawInline reassembly: remove unnecessary segments from the stream in an earlier stage. Test this properly. 14 years ago
Victor Julien 668bd46c1c Add flow prune debug counters (disabled by default). 14 years ago
Victor Julien 4bf4382354 Make sure tunnel packets (and pseudo packets) properly decrement the flow use counter in all cases. 14 years ago
Victor Julien 37587c0b7d Add missing stream inline files. 14 years ago
Victor Julien 3a774165fa Initial version of a inline raw reassembly function that reassembles in a sliding window. Introduce new unittest helpers for stream reassembly. 14 years ago
Victor Julien abdffadc1c Add a new app layer reassembly function that is for inline use, and use it when the stream engine is in inline mode. 14 years ago
Victor Julien 8cacd5fe50 Fix the stream.inline config option. Set PKT_STREAM_EST flag also for packets that are part of a session in a state beyond TCP_ESTABLISHED. 14 years ago
Victor Julien a8bb98836b Don't handle and validate the TCP timestamp at the same time. Instead validate first, then later when all other validation has been done as well, handle. 14 years ago
Victor Julien 8d3f9c53a9 Minor cleanups. 14 years ago
Victor Julien bff70eed6d Update to depth code. Get segment from the correct pool when a payload is truncated. 14 years ago
Victor Julien 66c40f782c Have reassembly errors also set a stream event. 14 years ago
Victor Julien 0f072648e6 Another iteration of the reassembly depth enforcement, now considering retransmissions. 14 years ago
Victor Julien 935958219d Rename RST validation function to match convention 14 years ago
Victor Julien 94fe0d5fa2 Add ACK validation to Reset/RST validation code. 14 years ago
Victor Julien 16cd31a408 Remove unused pseudo packet reassembly code. 14 years ago
Victor Julien bf88a6de09 Add depth comment. 14 years ago
Victor Julien a26768ce7a Change the way the reassembly depth is enforced. Ignore retransmissions, get rid of per session counter. 14 years ago
Victor Julien 7af9c58af7 Improve ACK value validation, timestamp checking code. Overall layout. 14 years ago
Victor Julien 0f5b6a8bd7 Fix minor comment typo. 14 years ago
Victor Julien aa04d9eefb Improve stream gap handling. Instead of giving up as soon as we see a gap we now wait much longer before we decide it's a gap. 14 years ago
Victor Julien 6ffb9da9be Better support ack/psh data packets on several states. Updates to ack validation code. 14 years ago
Victor Julien 6fca55e068 Add some debug output to app-layer-htp. 14 years ago
Victor Julien 25f5589078 First round of adding 'stream events'. Basic stream tracking events added. 14 years ago
Victor Julien 2849d2b1d3 Initial code for stream 'inline' mode: packets that are (partly) overlapping with already accepted packets (meaning in the streams seg list) are rewritten to make sure they contain the exact same data. 14 years ago
Victor Julien 3857154f4b Fix the pseudo packet having the wrong proto set, causing massive fp's. Flag packets to be part of the established phase of a tcp session, so we won't prematurely inspect the app layer state. 14 years ago
Victor Julien 8b5f553a35 Inspect a pseudo packet upon receiving a RST so that we are sure both sides of the TCP session are inspected. 14 years ago
Gurvinder Singh 55a863359c support for pseudo packet creation from reassembled stream segments 14 years ago
Victor Julien cc116d71ef Fix unittests after merge. 14 years ago
Gurvinder Singh 2beb7af7f8 support for validating the ACK before updating the last_ack field and also update next_seq if we missed the last packet 14 years ago
Victor Julien acc38c9ebf Make sure we don't try to 'verdict' the fake PKT_PSEUDO_STREAM_END packets. 14 years ago
Victor Julien c955254b4e Adapt stream code to packet memory allocation changes. 14 years ago
Victor Julien 44e678b86b Comment out disabled unittests. 14 years ago
Victor Julien a622ad5047 Fix new unittests introduced by rebase with next branch. 14 years ago
Victor Julien 1d971b53a6 Update all unittests 14 years ago
Victor Julien fadd6d6361 Add pseudo packet counter. 14 years ago
Victor Julien f606621e8c Fix the pseudo packet having the wrong proto set, causing massive fp's. Flag packets to be part of the established phase of a tcp session, so we won't prematurely inspect the app layer state. 14 years ago
Victor Julien b0901ab30d Fix compilation with --enable-debug 14 years ago
Victor Julien 6482c34909 Increment flow use cnt for pseudo packets as the flow is not supposed to disappear while dealing with those packets. 14 years ago
Victor Julien 2072ad80af Never create a pseudo packet based on a pseudo packet. 14 years ago
Victor Julien 61a9936d55 Inspect a pseudo packet upon receiving a RST so that we are sure both sides of the TCP session are inspected. 14 years ago
Gurvinder Singh 00f21f34e8 support for pseudo packet creation from reassembled stream segments 14 years ago
Victor Julien 8fa5a2c025 Split applayer and raw stream reassembly
Split stream reassembly in 2 parts: a part that sends ack'd data to the app
layer parsers as soon as it's available, and another part that queues up
data into larger chunks for raw inspection.
14 years ago
Victor Julien dda6d3e07b Add error counters. 14 years ago
Victor Julien 3b239b3e48 Cleanup and document AppLayerHandleTCPData 14 years ago
Victor Julien fe6bf728d3 Create a AppLayerHandleTCPData function to directly feed data from the reassembly engine to the app layer parsing. 14 years ago
Victor Julien b5a5ef14b9 Make sure we reuse a TCP session if we receive a valid 3WHS on a closed TCP session, can happen if a new session has the same tuple. 14 years ago
Victor Julien 24f071cabb Make sure http_cookie inspects all HTTP transactions. Clean up error messages. Get rid of unused code and dead comments. 14 years ago
Anoop Saldanha c9897a44a4 fast pattern support for http_cookie. Also support relative modifiers 14 years ago
Anoop Saldanha bbbedaf963 fast pattern support for http_method. Also support relative modifiers 14 years ago
Anoop Saldanha 2321a4dd58 support isdataat negation. Also fix addiing isdataat to appropriate lists 14 years ago
Victor Julien 4ae7144876 Fix 2 cases where overlapping data in the stream engine wouldn't be properly handled potentially causing the wrong data being used in stream reassembly. 14 years ago
Eric Leblond 9c2bdc6d0c Main loop: increase timer.
Timer in the main loop was of 100 usec. This patch increases it
to 10 ms which should be a reasonnable delay to declare some threads
dead.

Signed-off-by: Eric Leblond <eric@regit.org>
14 years ago
Eric Leblond 89558ab9a4 RFC: modify error treatment in PacketCopyData
Hello Victor
This patch modifies error treatment following our discussion on IRC.
It tries to follow the error treatment guideline I've been able to
read in the different files.
I will merge this patch in the original commit if the error treatment
seems ok for you.

BR,
Eric
14 years ago
Eric Leblond 49adc264bc Don't print message after SCMalloc failure.
This patch generated via coccinelle is getting rid of logging
message after a SCMalloc failure. They were useless as SCMalloc
already displays a message.
14 years ago
Eric Leblond 67b95c8c4d Auto discovery of default packet size
If default-packet-size is not set, it is possible in some case to
guess a correct value.

If PCAP or PF_RING are used we are linked to a "physical" interface.
Thus, it is possible to get information about the link MTU and
hardware header size. This give us the ability to auto discover a
decent default-packet-size.
If suricata is running under a different running-mode, it will
default to 1514.
14 years ago
Eric Leblond 3eada85ff8 Add interface setting discovery via ioctl
This patch adds support for MTU discovery of link following idea
of go.ph1g. It also adds some function to give a approximation of
link header length.
14 years ago
Eric Leblond 1db4aadd16 Supress usage of Packet declaration in tests.
For convenience, a massive usage of 'Packet p;' declaration has
been done in the tests function. Although this was completely
legal, this is not possible anymore because of the new Packet
allocation structure. This massive patch modifies all suricata
files to use a SCMalloc allocated pointer to Packet instead.

This patch has been done using coccinelle (http://coccinelle.lip6.fr)
which is a semantic patching tool. This ensures that things like call
to SCFree() should have not been forget because the semantic patch
explicitly forces the call to SCFree(p) before each return. With this
patch all unittests are running fine with a small and a big default
packet size.
14 years ago
Eric Leblond 156b202597 Fix decode part of source-nfq 14 years ago
Eric Leblond dd038c1906 Modify files to avoid direct pckt payload access
This patch implements the needed modification of payload access
in a Packet structure to support the abstraction introduced by
the extended data system.
14 years ago
Eric Leblond e802e1ed16 Modify Packet structure and prepare accessor.
This patch modifies decode.c and decode.h to avoid the usage
by default of a bigger than 65535 bytes array in Packet structure.
The idea is that the packet are mainly under 1514 bytes size and
a bigger size must be supported but should not be the default.

If the packet length is bigger than DFLT_PACKET_SIZE then the
data are stored in a dynamically allocated part of the memory.

To ease the modification of the rest of the code, functions to
access and set the payload/length in a Packet have been introduced.

The default packet size can be set at runtime via the default-packet-size
configuration variable.
14 years ago
Eric Leblond 8471626916 Fix error message and adds information to config
This patch fixes a typo in an error message and add some
information to the checksum verification option.
14 years ago
Anoop Saldanha 6fc5dae2f9 fix leak for accepted uuid list in dcerpc state 14 years ago
Victor Julien 3409513a44 Fix FlowTest* unittests to fail sometimes. 14 years ago
Eric Leblond a69bb94335 Checksum match: fix logic problem
This patch fixes a logic error in the checksum matches. In
case the protocol is not the one tested, the test must return
0 and not 1 (test matched).

Signed-off-by: Eric Leblond <eric@regit.org>
14 years ago
Victor Julien 878d3d87db Add (experimental) support for using multiple pcap devices to acquire packets from. Just passing multiple -i <dev> options on the commandline will activate this. Windows not yet supported. 14 years ago
Victor Julien 18b4e3380f Make mpm-algo use the mpm_table that has the actual mpm's registered. Clean up dead code. 14 years ago
Victor Julien 6131dec8a1 Fix a compiler warning due to a broken prototype declaration. 14 years ago
Victor Julien e3bde3e95d Add a simple revision based on the git rev to the version number, like a build number. 14 years ago
Eric Leblond 56c95bf622 Convert thread PRIO to a enum
This patch converts thread prio value to an enum. This
can add some useful check by gcc in switch.

Signed-off-by: Eric Leblond <eric@regit.org>
14 years ago
Eric Leblond 62cf7eea4c util-cpu: fix trivial typo in documentation
This patch fixes a trivial typo in a documentation message.

Signed-off-by: Eric Leblond <eric@regit.org>
14 years ago
Victor Julien ffcd512167 Clean up packet pool handler on shutdown. 14 years ago
Victor Julien b24ccf8c80 Clean up stream pmqs in the detect thread ctx. 14 years ago
Victor Julien 3710296057 Cleanup defrag engine on shutdown. 14 years ago
Anoop Saldanha 9c9f3ec963 fix mem leak in http_ engines 14 years ago
Victor Julien 0057a7c15e Suppress a AC debug message. 14 years ago
Victor Julien d48ff8f6aa Extend 'append' option to stats.log as well. Small cleanups. 14 years ago
Gurvinder Singh f4392e1dcc added support for appending the log files 14 years ago
Pierre Chifflier de41612ea1 Add options to choose if we log header and content in Prelude alert module.
Signed-off-by: Pierre Chifflier <chifflier@edenwall.com>
14 years ago
Pierre Chifflier 9a53a09c58 Log verdict in Prelude alert module
Signed-off-by: Pierre Chifflier <chifflier@edenwall.com>
14 years ago
Eric Leblond f73c60b4ff nfq: set some options on netlink socket
This patch modify nfq system to set some options on the netlink socket.
This should improve performances by handling more correctly capacity
overrun.
14 years ago
Victor Julien 96ec2a76c6 Update version to 1.1beta1 14 years ago
Victor Julien 014f62247a Another batch of clang fixes. Nothing really serious. Includes a couple of fixes for broken fixes from yesterday. 14 years ago
Victor Julien cac95010f3 Minor unified1-log changes to work around a clang issue, but also to clean up the logic slightly. 14 years ago
Victor Julien 3f47eade6b Fix couple of cases where incorrect handling of keyword parsing errors would lead to access of uninitialized memory. Found by clang. 14 years ago
Victor Julien 7dc4b164a8 Fix a clang warning in unittest DetectUriSigTest12. 14 years ago
Victor Julien 2c6f9abaff Work around a suspected fp in clang. 14 years ago
Victor Julien ac97bb7799 Fix a number of small clang issues. Clang doesn't know we exit on malloc errors during init. 14 years ago
Victor Julien b600c9ac09 Fix a clang issue 'Assigned value is garbage or undefined' in the threshold code in case a packet was neither ipv4 or ipv6. 14 years ago
Victor Julien 1112e103a8 Disable DBG_PERF by default except for when DEBUG is enabled. 14 years ago
Victor Julien 6af30e5b2e Handle a clang warning that says dstq can be null referenced. In no call of FlowRequeue dstq can be null so not a real issue. Added a BUG_ON just in case, but only in DEBUG mode to prevent the extra overhead. If the code changes we will run it in DEBUG mode and catch the error. 14 years ago
Victor Julien 743ed7626c Fix potential null deref (introduced a few commits ago) found by clang. 14 years ago
Victor Julien 3d60e9bfeb Clean up output. 14 years ago
Victor Julien 3fcfaef9f7 Fix compiler warning in log-httplog.c & change stats.log to log as mm/dd/yyyy as well. 14 years ago
Gurvinder Singh 791d177c7f fixed the timestamp issue in http.log 14 years ago
Victor Julien 355f237bfd Fix compiler warnings, cleanup counters config code. 14 years ago
Gurvinder Singh ba18110abd support for stats.log configurable and fixed timezone issue in faslog and debuglog 14 years ago
Anoop Saldanha 58c228a56b fix sig ordering bugs. Flowvars and pktvars user type retrieval should be from pmatch list, as well as from match list. Also fix lousy unit tests 14 years ago
Victor Julien c64b9362cf Remove unused stream flag. 14 years ago
Victor Julien 4cacb1e970 Disable adding to unregistered mbit/s counter. 14 years ago
Anoop Saldanha dc2c8be583 always read config.h header file first 14 years ago
Victor Julien 40de0b0270 Have each output use the global log format if none is specified for that specific output. 14 years ago
Anoop Saldanha d241e51b32 adapt fast pattern engine analysis to reflect the new changes made to your mpm design 14 years ago
Anoop Saldanha 54854d74c1 add some header files that we missed while rebasing 14 years ago
Anoop Saldanha 05adf2de41 fix live runmode decode TM for cuda 14 years ago
Anoop Saldanha 42c1287028 renintroduce g_u8_lowercase_table for b2g cuda 14 years ago
Anoop Saldanha 1fb121d0ba wrap cuda based util-mpm.c tests in __SC_CUDA_SUPPORT__ ifdef 14 years ago
Martin Beyer b1c577f829 cuda streams support in b2g-cuda MPM 14 years ago
Martin Beyer 0ce86efe40 cuda handlers support multiple CUmodules per context 14 years ago
Martin Beyer 621815ded0 cuda-packet-batcher timeout supports float values 14 years ago
Anoop Saldanha 7dd2392dea updated cuda todos. Please look at cuda-packet-batcher.c to have a look at the new todos 14 years ago
Anoop Saldanha c734cd1bdd make cuda mpm parameters configurable 14 years ago
Anoop Saldanha 2c08aebec1 enable write combined memory for cuda mpm. Some other minor cleanup 14 years ago
Martin Beyer 8adff3c63c use of pagelocked memory for CUDA 14 years ago
Victor Julien 0e8e8e3728 Don't stop stream reassembly if protocol detection failed, only stop/prevent app layer parsing. 14 years ago
Victor Julien b7a5f16b69 Fix FlowBitsTestSig06 test no longer working properly. 14 years ago
Victor Julien 39dea56a84 Remove flowbits as a mask prefilter as they are dynamic. Add a dynamic check. 14 years ago
Victor Julien 6943a7eb8c Move updating the time from the pcap callback to the decoding stage in file mode. 14 years ago
Anoop Saldanha ceb7fd748e support /D option for pcre - http raw header. Also support relative pcre for http raw header. All pcre processing for http header moved to hrhd engine 14 years ago
Victor Julien 16e4e3fe50 Fix request-body-limit option for libhtp config. 14 years ago
Victor Julien 39a5348d2b Remove dead pcre code. 14 years ago
Victor Julien 6ebe7b7cd3 Change the way the request body limit is enforced. 14 years ago
Victor Julien 0cd2bce7da Manually add unittest by Pablo Rincon from bug #210. 14 years ago
Anoop Saldanha 8bd6a38318 support relative pcre for http header. All pcre processing for http header moved to hhd engine 14 years ago
Anoop Saldanha 2b781f00d7 support relative pcre for client body. All pcre processing for client body moved to hcbd engine 14 years ago
Victor Julien 0a58f0728a Remove redundant checks in http header and http client body code. 14 years ago
Anoop Saldanha 8c21511c99 don't buffer raw headers. Retrieve them individually from htp_state during mpm stage and content valiadation stage 14 years ago