aiden
/
suricata
mirror of https://github.com/OISF/suricata
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
4,472 Commits
8 Branches
173 Tags
192 MiB
main
main-7.0.x
main-8.0.x
master-6.0.x
master-5.0.x
master-4.1.x
master-4.0.x
master-3.2.x
suricata-8.0.6
suricata-7.0.17
suricata-8.0.5
suricata-7.0.16
suricata-8.0.4
suricata-7.0.15
suricata-8.0.3
suricata-7.0.14
suricata-8.0.2
suricata-7.0.13
suricata-8.0.1
suricata-7.0.12
suricata-8.0.0
suricata-7.0.11
suricata-8.0.0-rc1
suricata-8.0.0-beta1
suricata-7.0.10
suricata-7.0.9
suricata-7.0.8
suricata-7.0.7
suricata-7.0.6
suricata-6.0.20
suricata-7.0.5
suricata-6.0.19
suricata-6.0.18
suricata-7.0.4
suricata-6.0.17
suricata-7.0.3
suricata-6.0.16
suricata-7.0.2
suricata-6.0.15
suricata-7.0.1
suricata-6.0.14
suricata-7.0.0
suricata-7.0.0-rc2
suricata-6.0.13
suricata-6.0.12
suricata-6.0.11
suricata-7.0.0-rc1
suricata-6.0.10
suricata-6.0.9
suricata-7.0.0-beta1
suricata-6.0.8
suricata-6.0.7
suricata-6.0.6
suricata-5.0.10
suricata-6.0.5
suricata-5.0.9
suricata-6.0.4
suricata-5.0.8
suricata-6.0.3
suricata-5.0.7
suricata-6.0.2
suricata-5.0.6
suricata-6.0.1
suricata-5.0.5
suricata-4.1.10
suricata-4.1.9
suricata-5.0.4
suricata-6.0.0
suricata-6.0.0-rc1
suricata-6.0.0-beta1
suricata-5.0.3
suricata-4.1.8
suricata-4.1.7
suricata-5.0.2
suricata-4.1.6
suricata-5.0.1
suricata-5.0.0
suricata-5.0.0-rc1
suricata-4.1.5
suricata-5.0.0-beta1
suricata-4.1.4
suricata-4.1.3
suricata-4.0.7
suricata-4.1.2
suricata-4.1.1
suricata-4.1.0
suricata-4.0.6
suricata-4.1.0-rc2
suricata-4.1.0-rc1
suricata-4.0.5
suricata-4.1.0-beta1
suricata-4.0.4
suricata-4.0.3
suricata-4.0.2
suricata-3.2.5
suricata-4.0.1
suricata-3.2.4
suricata-4.0.0
suricata-4.0.0-rc2
suricata-3.2.3
suricata-4.0.0-rc1
suricata-4.0.0-beta1
suricata-3.2.2
suricata-3.1.4
suricata-3.2.1
suricata-3.2
suricata-3.2RC1
suricata-3.1.3
suricata-3.2beta1
suricata-3.1.2
suricata-3.1.1
suricata-3.1
suricata-3.0.2
suricata-3.1RC1
suricata-3.0.1
suricata-3.0.1RC1
suricata-3.0
suricata-2.0.11
suricata-3.0RC3
suricata-3.0RC2
suricata-3.0RC1
suricata-2.0.10
suricata-2.0.9
suricata-2.1beta4
suricata-2.0.8
suricata-2.0.7
suricata-2.1beta3
suricata-2.0.6
suricata-2.0.5
suricata-2.1beta2
suricata-2.0.4
suricata-2.1beta1
suricata-2.0.3
suricata-2.0.2
suricata-2.0.1
suricata-2.0.1rc1
suricata-2.0
suricata-2.0rc3
suricata-2.0rc2
suricata-2.0rc1
suricata-2.0beta2
suricata-1.4.7
suricata-1.4.6
suricata-1.4.5
suricata-2.0beta1
suricata-1.4.4
suricata-1.4.3
suricata-1.4.2
suricata-1.4.1
suricata-1.3.6
suricata-1.4
suricata-1.3.5
suricata-1.4rc1
suricata-1.3.4
suricata-1.4beta3
suricata-1.3.3
suricata-1.4beta2
suricata-1.3.2
suricata-1.4beta1
suricata-1.3.1
suricata-1.3
suricata-1.3rc1
suricata-1.3beta2
suricata-1.3beta1
suricata-1.2.1
suricata-1.2
suricata-1.2rc1
suricata-1.2beta1
suricata-1.1.1
suricata-0.8.2
suricata-1.0.0
suricata-1.0.1
suricata-1.0.2
suricata-1.0.3
suricata-1.0.4
suricata-1.0.5
suricata-1.1
suricata-1.1beta1
suricata-1.1beta2
suricata-1.1beta3
suricata-1.1rc1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '4dd605ae3b'
${ noResults }
Commit Graph

4133 Commits (4dd605ae3b1aa66d16ef98e19fed5aca44269bac)

Author SHA1 Message Date
Victor Julien 8237bbf18a Coverity 1038101: remove dead code from host hash timeout code 13 years ago
Victor Julien 440124a4b9 Coverity 1038100: remove dead code from flow hash timeout code(2) 13 years ago
Victor Julien 243060a6b7 Coverity 1038099: remove dead code from flow hash timeout code 13 years ago
Victor Julien 2e82772a0a Coverity 1038098: remove dead code from flow hash 13 years ago
Victor Julien aecefd00bd Coverity 1038095: remove dead code from defrag hash timeout code 13 years ago
Victor Julien 16056d51f2 Coverity 1038094: remove dead code from defrag hash 13 years ago
Victor Julien 32503bafaa Coverity 1038089: error check fseek call 13 years ago
Victor Julien 4827a4dcef Coverity 400477: pcre_get_substring retval 
Add missing return code check to pcre_get_substring call.
 13 years ago
Victor Julien 790866656b Coverity 1038129 fix 
Don't leak memory on malloc error in b2gm mpm implementation.
 13 years ago
Victor Julien 33919559d0 Fix memory leak on invalid luajit signature. Coverity 1038520. 13 years ago
Victor Julien 51c6a333d9 geoip: never try to store more locations than possible (Coverity 1038517) 13 years ago
Victor Julien 3cf3b485f2 Coverity 1038138 fix 
Clean up parsing code to suppress Coverity:
Dereference before null check (REVERSE_INULL)

Proper checking was already done.
 13 years ago
Victor Julien 27ea4232fe Coverity 1038134 fix 
Cleaned up error check. "ipdup" can only be non-NULL there, so remove check
that confused coverity.
 13 years ago
Victor Julien ecd5c7573b Coverity 1038135 fix 
Small cleanup in the error handling. The extra null check confused
Coverity.
 13 years ago
Victor Julien 38b6103ff5 Coverity 1038133 fix 
Clean up parsing code to suppress Coverity:
Dereference before null check (REVERSE_INULL)

Proper checking was already done.
 13 years ago
Ken Steele 50f859e9f2 Move SIMD implementations out of detect.c 
Move SIMD the implementations of SigMatchSignaturesBuildMatchArray()
for SSE3 and Tile out of detect.c to reduce the size of the file.

Also moved SIMD unit tests to detect-simd.c
 13 years ago
Victor Julien 7f140f6726 Coverity 1038111: fix local overrun of a string in app layer proto detect setup code. 13 years ago
Ken Steele eb4f0da97f Change one more atomic size in detect.h 
Change uint16_t to int for better tile atomic performance. Checked with
pahole that it doesn't increase the size of the structure.
 13 years ago
Ken Steele b08ddfa7f1 Support for Tile Gx atomic instructions 
Tilera's GCC supports the GCC __sync_ intrinsics.

Increase the size of some atomic variables for better performance on
Tile.  The Tile-Gx architecture has native support for 32-bit and
64-bit atomic operations, but not 8-bit and 16-bit, which are emulated
using 32-bit atomics, so changing some 16-bit and 8-bit atomic into
ints improves performance.

Increasing the size of the atomic variables modified in this change
does not increase the total size of the structures in which they
reside because of existing padding requirements. The one case that
would increase the size of the structure (Flow_) was confitionalized
to only change the size on Tile.
 13 years ago
Anoop Saldanha 54847e396f unittests for gzip, deflate http compression, multiple stacked 
compressions, cunning compression that's not what it says it is, etc.

These unittests are tweaked to pass.  When libhtp fixes these issues
we will have to reenable them.
 13 years ago
Anoop Saldanha 94e2527606 Introduce a saner way to validate the completion of request and 
response bodies.

Also don't change app state for http from inside inspection.
 13 years ago
Anoop Saldanha dcdcbd9721 Fix creating a backup of htp config. This is used by unittests that 
changed htp config.
 13 years ago
Ken Steele 62540eff3e Align some structures to cacheline 
Align strucutres with pthread mutex locks to start on cachelines to keep
the lock within one cacheline.
 13 years ago
Ken Steele d84079ba7d Move FlowIncrUsecnt to header file to allow for inlining. 
Move FlowIncrUsecnt() and FlowDecrUsecnt() from flow.c to flow.h to
allow for inlining.
 13 years ago
Ken Steele e05034f5dd New Multi-pattern matcher, ac-tile, optimized for Tile architecture. 
Aho-Corasick mpm optimized for Tilera Tile-Gx architecture. Based on the
util-mpm-ac.c code base. The primary optimizations are:
1) Matching function used Tilera specific instructions.
2) Alphabet compression to reduce delta table size to increase cache
   utilization  and performance.

The basic observation is that not all 256 ASCII characters are used by
the set of multiple patterns in a group for which a DFA is
created. The first reason is that Suricata's pattern matching is
case-insensitive, so all uppercase characters are converted to
lowercase, leaving a hole of 26 characters in the
alphabet. Previously, this hole was simply left in the middle of the
alphabet and thus in the generated Next State (delta) tables.

A new, smaller, alphabet is created using a translation table of 256
bytes per mpm group. Previously, there was one global translation
table for converting upper case to lowercase.

Additional, unused characters are found by creating a histogram of all
the characters in all the patterns. Then all the characters with zero
counts are mapped to one character (0) in the new alphabet. Since
These characters appear in no pattern, they can all be mapped to a
single character and still result in the same matches being
found. Zero was chosen for the value in the new alphabet since this
"character" is more likely to appear in the input. The unused
character always results in the next state being state zero, but that
fact is not currently used by the code, since special casing takes
additional instructions.

The characters that do appear in some pattern are mapped to
consecutive characters in the new alphabet, starting at 1. This
results in a dense packing of next state values in the delta tables
and additionally can allow for a smaller number of columns in that
table, thus using less memory and better packing into the cache. The
size of the new alphabet is the number of used characters plus 1 for
the unused catch-all character.

The alphabet size is rounded up to the next larger power-of-2 so that
multiplication by the alphabet size can be done with a shift.  It
might be possible to use a multiply instruction, so that the exact
alphabet size could be used, which would further reduce the size of
the delta tables, increase cache density and not require the
specialized search functions. The multiply would likely add 1 cycle to
the inner search loop.

Since the multiply by alphabet-size is cleverly merged with a mask
instruction (in the SINDEX macro), specialized versions of the
SCACSearch function are generated for alphabet sizes 256, 128, 64, 32
and 16.  This is done by including the file util-mpm-ac-small.c
multiple times with a redefined SINDEX macro. A function pointer is
then stored in the mpm context for the search function. For alpha bit
sizes of 8 or smaller, the number of states usually small, so the DFA
is already very small, so there is little difference using the 16
state search function.

The SCACSearch function is also specialized by the size of the value
stored in the next state (delta) tables, either 16-bits or 32-bits.
This removes a conditional inside the Search function. That
conditional is only called once, but doesn't hurt to remove
it. 16-bits are used for up to 32K states, with the sign bit set for
states with matches.

Future optimization:

The state-has-match values is only needed per state, not per next
state, so checking the next-state sign bit could be replaced with
reading a different value, at the cost of an additional load, but
increasing the 16-bit next state span to 64K.

Since the order of the characters in the new alphabet doesn't matter,
the new alphabet could be sorted by the frequency of the characters in
the expected input stream for that multi-pattern matcher. This would
group more frequent characters into the same cache lines, thus
increasing the probability of reusing a cache-line.

All the next state values for each state live in their own set of
cache-lines. With power-of-two sizes alphabets, these don't overlap.
So either 32 or 16 character's next states are loaded in each cache
line load. If the alphabet size is not an exact power-of-2, then the
last cache-line is not completely full and up to 31*2 bytes of that
line could be wasted per state.

The next state table could be transposed, so that all the next states
for a specific character are stored sequentially, this could be better
if some characters, for example the unused character, are much more
frequent.
 13 years ago
Victor Julien 77b429c402 xff: fix unittest crashes 13 years ago
Victor Julien 05d68ce394 xff: don't do xff check if there are no alerts anyway. 13 years ago
Duarte Silva 7dbb305255 Adds X-Forwarded-For support to the Unified2 output format 
- Added the Unified2 file format related constants
 - Added IPv6 support
 - Two modes of operation with a fall-back to "extra-data" mode if
   "overwrite" mode is not applicable
 - Changed the configuration loading code to handle the new
   configuration structure
 - When creating the packet that fakes the one that generated the alert
   the flow direction wasn't taken into account in overwrite mode
 - Fixed BUG_ON condition
 13 years ago
Victor Julien 900918a5d1 Bug #948: detect thread local storage support 13 years ago
Ken Steele 0861d3a2a3 Minor optimization in time caching code. 
Reduced the size of the cached string buffer from 128 to 32, which is
still larger than the largest possible time string, which is 26
characters.

Added a check for the user passing in an output buffer that is smaller
than the cached string. Previously, the code would have copied past
the end of the users buffer.
 13 years ago
Anoop Saldanha 49dcb0ca84 fix for #925. 
Log sensible error message when the user doesn't supply a value for
stream.prealloc-sessions or when the values supplied in invalid and
the engine resorts to using a default.
 13 years ago
Anoop Saldanha db6ef81fb0 fix for #926. 
Supply meaningful error message when user supplies invalid value for
host.prealloc.
 13 years ago
Anoop Saldanha b90a56b626 fix for #927. 
Print an error message when the user supplies an invalid value for
detect-thread-ratio in the conf file.
 13 years ago
Anoop Saldanha bed3f605fa Fix for #922. 
Add more relevant error message when we supply invalid value for
defrag.trackers and defrag.hash-size
 13 years ago
Anoop Saldanha 6608e7f523 Introduce generic utility API to log message on invalid config entry. 13 years ago
Victor Julien 6d34834623 Runmode fixes and cleanups 
Bug #939: thread name buffers are sized inconsistently
These buffers are now all fixed at 16 bytes.

Bug #914: Having a high number of pickup queues (216+) makes suricata crash
Fixed so that we can now have 256 pickup queues, which is the current built-in
maximum. Improved the error reporting.

Bug #928: Max number of threads
Error reporting improved. Issue was the same as #914.
 13 years ago
Anoop Saldanha 56143131da Fix unittests that use chunked encoding. 13 years ago
Nelson Escobar ef4d11aeb5 Use the Async versions of SCCudaMemcpy* to improve gpu performance. 13 years ago
Eric Leblond 77f2b9968e autotools: use builddir instead of srcdir 
srcdir is supposed to be read-only when running distcheck so it is better to
create the log directory in builddir.
 13 years ago
Ignacio Sanchez 1b2f251866 Various custom http logging improvements 
Cookie is parsed now using uint8_t pointers (inliniac PR comments)
Changed buffer size to a power of 2 (8192) and cookie value extraction function to static (inliniac PR comments)
Added %b for request size (vinfang patch)
Writing "-" if an unknown % directive is used (vinfang patch)
Fixed bug in cookie parser
Fixed format string issue logging literal values
Improve error handling (Victor Julien comments)

(patchset rebased and reworded by Victor Julien)
 13 years ago
Ignacio Sanchez 8051dc8a6a Added modifications suggested by Charles Smutz (https://redmine.openinfosecfoundation.org/issues/602) 13 years ago
Ignacio Sanchez 796bfab231 Added support for %{cookiename}C 
Added support for the definition of maximun length. ie: %[50]{user-agent}i
Some small bugfixes
 13 years ago
Eric Leblond 3dbf6c6fee solaris: fix compilation failure 
This patch fixes a compilation failure on Solaris. Compiler does
not support when a function returning void is used in return of
an other function returning void.
 13 years ago
Ken Steele a2b502a30c Formatting change for function call. 
Put open brace { for function on a new line to match coding standard.

Changed:

int foo(int x) {
}

to:

int foo(int x)
{
}
 13 years ago
Ken Steele d4dd18eb85 Clean up SCLocalTime() usage 
Remove cast of return type from SCLocalTime() as it is not needed.
Replace last use of localtime_r() with SCLocalTime().
 13 years ago
Ken Steele 77fae5313d On Open BSD systems don't cache time. 
Open BSD doesn't support __thread, which is used for time caching, so
don't do time chaching for BSD systems.
 13 years ago
Ken Steele 2feb37c155 Cache time conversions for localtime() and CreateTimeString() 
When converting a time in seconds (64-bit seconds since 1970) to
Month/Day/Year hours minutes, Suricata calls localtime_r(), which
always aquires a lock and then does complex comutation based on the
current time zone. The time zone can be specified in the TZ
environment variable, which is only parsed the first time it is used,
or from a file. The default file is /etc/localtime. The file is
checked each time to see if it might have changed and is reparsed if
it has changed.

The GLIBC library has a lock inside localtime_r(), which limits
parallelism, which is a problem when the rate of generating alerts is
high, since Suricata generates a new ascii time string for each alert
into fast.log.

This change caches the value returned by localtime_t() and then sets
the seconds within the minute based on the cached start-of-minute
time. All of the values return, expect for the seconds, is constant
within the same minute. Switching to a new seconds could change all
the other values, year, month, day, hour. The cache stores the current
and previous minute values.

The same trick is used in CreateTimeString() for generated time
string. The string, up to the minutes, is cached and then copied into
the result string, followed by printing the new seconds into the
result string.

The seconds within a minute are calculated as the difference in
seconds from the start of the current minute.
 13 years ago
Ken Steele 68d26dcec7 Merge multiple copies of CreateTimeString() to one copy. 
There were 8 identical copies of CreateTimeString() in 8 files.
Most used SCLocalTime, to replace localtime_r(), but some did not.
Created one copy in util-time.c.
 13 years ago
Ken Steele 5532af4621 Create SCMUTEX_INITIALIZER to abstract out PTHREAD_MUTEX_INITIALIZER 
This allows replacing pthread mutexes with other types of mutex.
 13 years ago
Ken Steele 784843b146 Use Tilera SIMD for Signature matching ala SSE3 
Makes use of 8-wide byte compare instructions in signature matching.

For allocating aligned memory, _mm_malloc() is SSE only, so added
check for __tile__ to use memalign() instead.

Shows a 13% speed up.
 13 years ago
Ken Steele 22225a7e99 Tile SIMD implementation of SCMemcmp and SCMemcmpLowercase 
Based on the SSE3 implementation, it checks 8 bytes at a time.
 13 years ago
Anoop Saldanha e68d44b051 fix for #932. 
ipv6 tunnel decoder wrongly treats the tunneled ipv6 packets as an ipv4
packet.
 13 years ago
Anoop Saldanha e2f4144d99 fix for #920. 
Cull the space before the address specified in address var variables.
 13 years ago
Duarte Silva ab215c72f6 Now using the common functions 13 years ago
Duarte Silva 0a5c798729 Now using the common functions 
- Removed some non printable ANSI characters
- Removed unecessary include
 13 years ago
Duarte Silva 8ce95af09c Added the new files containing the repeated functions 
- Renamed the functions to something more generic
- Added the source and include files to the Makefile
 13 years ago
Anoop Saldanha a44d42b124 Fixes segv inside rule swap under low mem conditions. 
We now gracefully exit rule swap on any allocation or other failures.
 13 years ago
Anoop Saldanha 8516ba24c9 Rearrange ac state. 
Notice a minor speed bump of around 2% on runs.  More updates to follow.
 13 years ago
Ken Steele 4b8bb11454 Enable using Tile cycle counter. 
The Tile processors all have a cycle counter with a simple interface. Use
that for UtilCpuGetTicks.
 13 years ago
Victor Julien 38aaae1fd7 IsRuleReloadSet() shouldn't return an uninitialized value 13 years ago
Eric Leblond 189327981a unittests: fix stream-tcp.c 
Lock and recycle fixes for stream-tcp.c
 13 years ago
Eric Leblond cd3e32ce19 unittests: some functions needs a flow lock. 
In debug validation mode, it is required to call application layer
parsing and other functions with a lock on flow. This patch updates
the code to do so.
 13 years ago
Eric Leblond c5bd04f102 unittest: recycle packet before exit 
To avoid an issue with flow validation, we need to recycle the packet
before cleaning the flow.
 13 years ago
Anoop Saldanha d292f1a529 fix for #915. Fix segv when we send NULL to snprintf. 13 years ago
Eric Leblond c6e8c5bf1f pf_ring: avoid to ask for extended header. 
This patch update pf_ring capture to avoid to ask for extended
header. They are only needed when rxonly checksum checks is used
and this is only possible when interface is not a DNA interface.
 13 years ago
Victor Julien ff668c2030 Fix Tile compile 13 years ago
Eric Leblond 20ca270dc3 fix pf_ring build 13 years ago
Eric Leblond 2a46f0dae4 suricata: rename SuriInstance to SCInstance. 13 years ago
Eric Leblond 9b422f3a8c suricata: suppress Suri prefix 
Suppress Suri prefix in internal function name.
 13 years ago
Eric Leblond 18ced653c3 Use a typedef for SuriInstance. 13 years ago
Eric Leblond 2d77e53f2c Add offline flag to SuriInstance and some refactoring 13 years ago
Eric Leblond 34abd818dd Prefix util-conf function with Config 13 years ago
Eric Leblond 7242cb30e7 Move CreateLowercaseTable to GLobalInits 13 years ago
Eric Leblond 02e9851315 Generic code don't need ifdef 13 years ago
Eric Leblond 8c00a963aa Use function for delayed detect setup. 13 years ago
Eric Leblond 4296e5f29e Add functions for elapsed time computation. 13 years ago
Eric Leblond 9d1d08c7a4 Factorize Signature loading 13 years ago
Eric Leblond 20c5683b60 Use function for daemonification and signal handler 13 years ago
Eric Leblond 90aaf55201 set rule_reload as part of SuriInstance 13 years ago
Eric Leblond bb19ce1847 SetBPfString is part of command line parsing 13 years ago
Eric Leblond 1a6983ee19 suricata: use function to print version 13 years ago
Eric Leblond 4f789dbe84 Add function for internal running mode 13 years ago
Eric Leblond d3cb043001 suricata: windows specific in one function 13 years ago
Eric Leblond 4401c048ba Running mode is set earlier so out earlier 13 years ago
Eric Leblond 40a25112a0 kill remaining run_mode usage 13 years ago
Eric Leblond 75fa1e20d7 engine analysis is a running mode 13 years ago
Eric Leblond c0d5ee77f9 get (almost) rid of run_mode variable. 13 years ago
Eric Leblond 80542816cd add internal running mode 13 years ago
Eric Leblond e07fdb20a8 Add SuriInstance structure 
To be able to split code in functions in main, we need to pass
information about the current running Suricata to functions.
For that we create a structure to store suricata run parameters.

In this patch it allows to separate command line parsing and to
treat internal running mode in a switch just after command line
parsing.
 13 years ago
Eric Leblond 325462d396 Export IsRuleReloadSet and use it. 13 years ago
Eric Leblond 6d9a66d522 unittest: make check use a qa/log dir for logging 
This patch is using the qa/log directory to store the output
of the check. In case of success, the directory is deleted.
In case of failure, the directory remains in place.

This should fixes #910.
 13 years ago
Eric Leblond 4424f5a231 af-packet: add sanity check in free function 13 years ago
Eric Leblond 8e68b357c7 Suppress Suri prefix. 13 years ago
Eric Leblond 42011e2d32 suricata: function for lowercase table creation 13 years ago
Eric Leblond 132bebb2b2 Simplify code by removing comment 13 years ago
Eric Leblond 07ef1f9837 suricata: add wrapper for interface listing 13 years ago
Eric Leblond 54006de40c Use new function GetLogDirectory() 13 years ago
Eric Leblond 2be7c8aea8 Add util-conf for config util 13 years ago
Eric Leblond 27752818c2 suricata: add some wrapper for config file handling 13 years ago
Eric Leblond b2fa4edd36 move unittest out of suricata.c 13 years ago
Eric Leblond 9a0bf0956b suricata: list cuda cards in separate function 13 years ago
Eric Leblond bed48e3a54 suricata: separate keyword and app layer listing code 
The list-keyword and app-layer listing code was spread over all the
init code. This patch introduces a separate file to store non standard
running mode like these ones.
 13 years ago
Eric Leblond 135ef0186b runmodes: fix comment 13 years ago
Victor Julien 5a7bf53a6b Storage: rename Init to Alloc to reflect actual functioning. Comment updates. 13 years ago
Victor Julien f06694d0c1 Storage API: add safety check for cases when there is no storage used. 13 years ago
Eric Leblond caf730d988 engine-tag: rename var and add sanity check 13 years ago
Eric Leblond fb55931c30 flow tag: conversion to flow storage API 
This patch is updating the flow tag system to use the flow
storage API. The tag_list member of Flow structure is suppressed
and its cleaning operation are suppressed too as this is handled
transparently by the flow storage API.
 13 years ago
Eric Leblond 4db2fc2cbb Add per-flow generic storage 
This patch adds a per-flow storage that can be created via the functions
available in flow-storage.c.
 13 years ago
Eric Leblond 6d08807b2d Host: use global free storage function 
This patch is here to avoid that all modules using a local storage
have to update host code to add their free function. It modifies
previous behavior by calling HostFreeStorage in any case.
 13 years ago
Victor Julien 27023872de Use Host Storage API for per host thresholding 13 years ago
Victor Julien c08b395c2c Init storage api at start up 13 years ago
Victor Julien 5919901675 Storage API: add registration check closed test in debug mode. 13 years ago
Victor Julien 3447324c36 Move Host Tag storage to Host Storage API. 13 years ago
Victor Julien 0d2a6e515e Add Host specific wrapper to StorageRegister() 13 years ago
Victor Julien b5ccf0b9c7 storage: allow preallocated storage 13 years ago
Victor Julien e2b006f523 host: use storage api 13 years ago
Victor Julien 022c0e466e Initial storage api work 13 years ago
Victor Julien 1c06d52208 Misc fixes after make check feedback 13 years ago
Ken Steele 316190c6b9 Add TILE-Gx mPIPE packet processing support. 
The TILE-Gx processor includes a packet processing engine, called
mPIPE, that can deliver packets directly into user space memory. It
handles buffer allocation and load balancing (either static 5-tuple
hashing, or dynamic flow affinity hashing are used here). The new
packet source code is in source-mpipe.c and source-mpipe.h

A new Tile runmode is added that configures the Suricata pipelines in
worker mode, where each thread does the entire packet processing
pipeline.  It scales across all the Gx chips sizes of 9, 16, 36 or 72
cores. The new runmode is in runmode-tile.c and runmode-tile.h

The configure script detects the TILE-Gx architecture and defines
HAVE_MPIPE, which is then used to conditionally enable the code to
support mPIPE packet processing. Suricata runs on TILE-Gx even without
mPIPE support enabled.

The Suricata Packet structures are allocated by the mPIPE hardware by
allocating the Suricata Packet structure immediatley before the mPIPE
packet buffer and then pushing the mPIPE packet buffer pointer onto
the mPIPE buffer stack.  This way, mPIPE writes the packet data into
the buffer, returns the mPIPE packet buffer pointer, which is then
converted into a Suricata Packet pointer for processing inside
Suricata. When the Packet is freed, the buffer is returned to mPIPE's
buffer stack, by setting ReleasePacket to an mPIPE release specific
function.

The code checks for the largest Huge page available in Linux when
Suricata is started. TILE-Gx supports Huge pages sizes of 16MB, 64MB,
256MB, 1GB and 4GB. Suricata then divides one of those page into
packet buffers for mPIPE.

The code is not yet optimized for high performance. Performance
improvements will follow shortly.

The code was originally written by Tom Decanio and then further
modified by Tilera.

This code has been tested with Tilera's Multicore Developement
Environment (MDE) version 4.1.5. The TILEncore-Gx36 (PCIe card) and
TILEmpower-Gx (1U Rack mount).
 13 years ago
Victor Julien 04f3f14541 ipv6: fix parsing of malformed ext hdr. Bug #908. 13 years ago
Victor Julien 4b4111e9e2 icmpv6: fix icmp_id and icmp_seq keywords 
Bug #907
 13 years ago
Victor Julien d82ce3f50c Fix compiler warning due to missing include 
decode.c: In function 'DecodeThreadVarsAlloc':
decode.c:437: error: implicit declaration of function 'ConfGetBool'
 13 years ago
Victor Julien 16c3487444 Add yaml option to disable vlan ids hashing 
In some cases using the vlan id(s) in flow hashing is problematic. Cases
of broken routers have been reported. So this option allows for disabling
the use of vlan id(s) while calculating the flow hash, and in the future
other hashes.

Vlan tracking for flow is enabled by default.
 13 years ago
Victor Julien 58ed1f2411 flow: take vlan_id's into account in the flow hash 
In VLAN we can have 2 layers of encapsulation. In this patch both
layers are used in the flow hash to distinguish between encapsulated
traffic.
 13 years ago
Victor Julien 055b422c28 Remove obsolete code: flow alert sid storage 13 years ago
Victor Julien 9faa4b740d Add --unittests-coverage option to list how many code modules have tests 13 years ago
Victor Julien fc7879322e Rename GetIfaceMaxPayloadSize to GetIfaceMaxPacketSize to reflect the actual function. 13 years ago
Victor Julien bd21b5ed9c Pcap: fix snaplen autodetection, GetIfaceMTU doesn't include link layer length 13 years ago
Anoop Saldanha ee0b21652b fix bug where we were not printing http hostname(printing <unknown> 
previously) in httplog, filestore meta and file log.
 13 years ago
Victor Julien 7edcc13514 NFQ: fix packets not getting freed 13 years ago
Anoop Saldanha cdaa13012a fix for #882. 
Refactor the code that initializes the cuda mpm environment.
 13 years ago
Victor Julien 9f3e2f7a92 NFQ: adapt to ReleasePacket API 13 years ago
Ken Steele b076a26cdc Replace ReleaseData function on Packet Structure with ReleasePacket. 
This commit allows handling Packets allocated by different methods.
The ReleaseData function pointer in the Packet structure is replaced
with ReleasePacket function pointer, which is then always called to
release the memory associated with a Packet.

Currently, the only usage of ReleaseData is in AF Packet. Previously
ReleaseData was only called when it was not NULL. To implement the
same functionality as before in AF Packet, a new function is defined
in AF Packet to first call the AFP specific ReleaseData function and
then releases the Packet structure.

Three new general functions are defined for releasing packets in the
default case:
    1) PacketFree() - To release a packet alloced with SCMalloc()
    2) PacketPoolReturnPacket() - For packets allocated from the Packet Pool.
                                  Calls RECYCLE_PACKET(p)
    3) PacketFreeOrRelease() - Calls PacketFree() or PacketPoolReturnPacket()
                                 based on the PKT_ALLOC flag.

Having these functions removes the need to check the PKT_ALLOC flag
when releasing a packet in most cases, since the ReleasePacket
function encodes how the Packet was allocated. The PKT_ALLOC flag is
still set and is needed when AF Packet releases a packet, since it
replaces the ReleasePacket function pointer with its own function and
then calls PacketFreeOfRelease(), which uses the PKT_ALLOC flag.
 13 years ago
Anoop Saldanha f85a2dc84b fix for #875. 
Update configure.ac to check for either 0.5.5 and 0.5.x version of libhtp.
 13 years ago
Anoop Saldanha 9698a5d78c Code to enable cuda support for pfring live mode. 13 years ago
Victor Julien 91fb47475b DNS: break out of DNSResponseGetNameByOffset if we're in there too long. Can happen on bad data. 13 years ago
Victor Julien aa26dae5a1 Stream: don't inject stream end pseudo pkt on FinWait2 state. Bug #883. 13 years ago
Victor Julien 2f3f577fb6 DNS: convert info logs to debugs 13 years ago
Victor Julien 97f51c1011 Fix ac-bs and ac-gfbs mpm-algo settings leading to fatal error if CUDA is enabled. Workaround for #882. 13 years ago
Eric Leblond e2334fbfe8 unix socket: fix typo in error message 13 years ago
Eric Leblond c2cbb43776 autotool: INCLUDES usage is deprecated 13 years ago
Eric Leblond 281d2f27f8 Fix compilation warning 
A goto could lead to the use de_ctx without declaring it.
 13 years ago
Victor Julien f4dcba6de3 In case of fragments, don't consider ports. Bug #847. 13 years ago
Anoop Saldanha e7f09f24c8 Code to enable cuda support for live mode pcap and af-packet. Keep an eye 
out on the mailing list and http://planet.suricata-ids.org for performance
and other profiling data.
 13 years ago
Victor Julien 51d6c63860 Luajit: fix compilation and tests after libhtp upgrade 13 years ago
Anoop Saldanha 48cf0585fb Suricata upgrade to libhtp 0.5.x. 
Remove the support for now unsupported personalities from libhtp -
TOMCAT_6_0, APACHE and APACHE_2_2.  We instead use the APACHE_2
personality.
 13 years ago
Victor Julien 080c15b3fc Enable libhtp 0.3.0 compilation and crash free UT run. Still see 5 failed tests. 13 years ago
Victor Julien 538da26812 Fix sgh mpm flags assignment 13 years ago
Eric Leblond 150cd39c6e detect-engine: do a direct update of flag 
There is no reason not to update the flag directly. So do it
to avoid to crash the test.
 13 years ago
Eric Leblond 2f2916d9ec ccccinelle: add formatted comment for flag test 13 years ago
Victor Julien aafc65c757 Autotools: move libhtp conditionals to configure 
In preparation of the libhtp upgrade, move all libhtp related conditionals
to configure. This allows for one set of build scripts that works regardless
of the presence of a local libhtp dir.
 13 years ago
Victor Julien 73e27c1fb7 Generate proper errors if sid,gid,rev values are out of range. Bug #779. 13 years ago
Victor Julien 164d60e8cd Yaml: give a more detailed error if the user supplies a directory instead of a yaml file. Bug #803. 13 years ago
Victor Julien a4e838c1d3 TLS: create certs directory during startup if it doesn't exist yet. Bug #710. 13 years ago
Eric Leblond 0e92469222 nfq: be sure to always verdict packets 
To be sure to always verdict packets (bug #769), this patch adds
a ReleaseData function to NFQ packets. The release function simply
drop the packet if it has not been verdicted before.
 13 years ago
Victor Julien 4a0050b9ea Print pkt src to alert-debug log 13 years ago
Victor Julien 1c371da46d DNS: better handle TX' with lost replies 13 years ago
Victor Julien 0fd9b0c4fa HTP: free TX from transaction free API call 13 years ago
Victor Julien 1367074c75 App layer: clean up TX before lowest active one 
Update DNS to handle cleaning up this way.
 13 years ago
Victor Julien 0b229ec8b9 DNS: suppress log-dns registration message 13 years ago
Victor Julien f59f90331d Applayer: remove obsolete StateUpdateTransactionId 
Also, update StateTransactionFree to take an u64 tx id, so it's
consistant with the rest of the engine.

To reflect these changes, AppLayerRegisterTransactionIdFuncs has
been renamed to AppLayerRegisterTxFreeFunc.

HTP, DNS, SMB, DCERPC parsers updated.
 13 years ago
Victor Julien ebab9aee83 DNS: move internal tx id tracking to u64 13 years ago
Victor Julien e8ad876b48 App layer: add 'StateHasEvents' API call 
Per TX decoder events resulted in significant overhead to the
detection engine, as it walked all TX' all the time to check
if decoder events were available.

This commit introduces a new API call StateHasEvents, which speeds
up this process, at the expense of keeping a counter in the state.

Implement this for DNS as well.
 13 years ago
Anoop Saldanha cd7b4fac40 remove unused pattern id assignment functions. Goodbye 13 years ago
Victor Julien f353fb630c DNS: convert dns_query to sticky buffer 13 years ago
Victor Julien 7292998a58 Content: set up sticky buffers like file_data and dce_stub_data w/o flags, but with a list variable 13 years ago
Victor Julien d476e4e50d Coverity 1040312, 1040313, 1040314 1040315: improve pool thread error handling. 13 years ago
Victor Julien 1373a20e8a Thread: remove thread id 13 years ago
Victor Julien d7aaa9464c Stream: use per thread ssn_pool_id instead of thread id. 13 years ago
Victor Julien 92b7ffad69 Improve memory cleanup in some unittests 13 years ago
Victor Julien fd7899cc8b Stream: fix unittests after ssn pool changes. 13 years ago
Victor Julien aa449d51ca Stream: use per thread ssn pool 
Use per thread pools to store and retrieve SSN's from. Uses PoolThread
API.

Remove max-sessions setting. Pools are set to unlimited, but TCP memcap
limits the amount of sessions.

The prealloc_session settings now applies to each thread, so lowered the
default from 32k to 2k.
 13 years ago
Victor Julien b6af6cb241 pool: add error msgs and improve memory layout 13 years ago
Victor Julien 5b9ef94f34 pool: add api for per thread pools 
This API is a wrapper around the regular pools where the thread pools
are arrays of locks+pools.
 13 years ago
Victor Julien 016d03bdaf pool: add error msgs and improve memory layout 13 years ago
Victor Julien 46af6b7e0f Add a per threadvars thread local thread id, that starts at 0 and increments for each thread. 13 years ago
Victor Julien b3b554c269 Coverity 1038959: DNS mpm might use initialized variable 13 years ago
Anoop Saldanha fba95e9125 Remove mpm ctxs in the wrong direction. 
A lot of http mpm ctxs have now been removed as a result of this.
 13 years ago
Anoop Saldanha 3c2ddf04c1 Update mpm init ctx to not accept the final cuda_rc_module argument. 
It was a part of our older architecture and is no longer used.
 13 years ago
Victor Julien 33818c0272 DNS: fix CUDA build 13 years ago
Victor Julien be7e6cdd7a DNS: fix warning when debug is not enabled 13 years ago
Victor Julien 571b8ac186 DNS: add support for per TX decoder events. 13 years ago
Victor Julien 9dc04d9fab app layer: add support for per TX decoder events 13 years ago
Victor Julien 72e35efbc6 Reset app layer events when we start inspecting a new TX 13 years ago
Victor Julien 28a6c1d9f8 DNS: add test for app layer event match 13 years ago
Victor Julien 6645620c03 Merge SIG_FLAG_MPM_HTTP and SIG_FLAG_MPM_DNS into SIG_FLAG_MPM_APPLAYER, do the same for the _NEG variant. 13 years ago
Victor Julien 43ba5a677e DNS: enable mpm/fast_pattern support for dns_query 13 years ago
Victor Julien 4817e1305f DNS: add /F modifier to pcre to inspect DNS query name 13 years ago
Victor Julien e567e12230 DNS: add unittests for UDP and TCP for dns_query keyword 13 years ago
Victor Julien f10dd603ff DNS: adding dns_request content modifier 13 years ago
Victor Julien 6674f4892c DNS: add per tx internal id 
Add per TX id. Rename transaction_cnt to transaction_max (id) and increment it on tx creation.
 13 years ago
Victor Julien 59780ca770 Hacks to enable alert dns even though we have dnstcp and dnsudp parsers. Needs proper solution later. 13 years ago
Victor Julien 8e01cba85d DNS TCP and UDP parser and DNS response logger 13 years ago
Eric Leblond 4521de2dfd Use PACKET_* macro instead of UPDATE 
Setting the ACTION_DROP flag can be done via PACKET_DROP instead
of using PACKET_UPDATE_ACTION.
 13 years ago
Eric Leblond c0c59fbd17 decode: factorize macro code 
PACKET_* are now wrapper to the newly introduced PACKET_SET_ACTION
macro.
 13 years ago
Eric Leblond 3f107fa130 decode: Packet action start with PACKET 
Rename all Packet action macro to have them prefixed by PACKET.
 13 years ago
Anoop Saldanha 3304c91c91 Don't let geoip match on pseudo packets. 13 years ago
Anoop Saldanha 1fb4aae993 Coverity 1038523: Fix using cuda buffer slice that has been returned to the pool. 13 years ago
Victor Julien 51cdd464a6 stream: detect keep-alive and keep-alive ACK 13 years ago
Victor Julien 03c3ff5632 stream: fix typo in function name 13 years ago
Eric Leblond 4c6595f437 Coverity 1038106: fix FP out-of-bond access 
A cast during the reading of a configuration variable was invalid
because a 16 bit integer was cast to a 32 bit integer. The called
function is only setting the pointer value to 1 or 0 so there is
no real issue there.
 13 years ago
Eric Leblond 7df156ef50 Coverity: 1038139 suppress sanity check 
The sanity check was really useless as the NULL value is checked in
the code flow.
 13 years ago
Eric Leblond 75cd1f6096 Coverity 1038515: check function return 
This is more cosmetic than useful but it is cleaner anyway.
 13 years ago
Anoop Saldanha b4e750068f Cuda make distcheck fix for cuda-ptxdump.h 13 years ago
Anoop Saldanha 7d46d59cdd Coverity 1038522: fix memset inside cuda code. Wrong size specified to memset. 13 years ago
Victor Julien d6fcd07a31 Coverity 1038085: remove 'default' statement in SCErrorToString. This way a warning will be given if an error is defined w/o updating this function. 13 years ago
Victor Julien e2444f0ed5 Coverity 1038092 & 1038093: remove dead code 13 years ago
Victor Julien 0902c7f3aa Coverity 1038518: fix wrong error check 13 years ago
Victor Julien db1dad8cc6 Coverity 1038124: memory leak on 'seq' keyword parsing failure 13 years ago
Victor Julien b2e962da03 Coverity 1038123: memory leak on 'flowint' keyword parsing failure 13 years ago
Victor Julien 5c5b2f98dd Coverity 1038116 & 1038117: memory leaks on 'app-layer-event' keyword parsing failure 13 years ago
Victor Julien 778851626c Coverity 1038115: memory leak on 'ack' keyword parsing failure 13 years ago
Victor Julien 98dbf3e62c Coverity 1038113: possibly out of bounds read 13 years ago
Anoop Saldanha 602c91ed41 Minor cosmetic changes to the cuda code. 
Moved a couple of functions to more cuda relevant files;
Re-structured some data types.
 13 years ago
Anoop Saldanha c9f076def3 Modified CudaBufferCullCompletedSlices. 
Allow readers specify max size of data they want to read.
 13 years ago
Anoop Saldanha 70cb4d30eb Add a usleep to CudaBuffer culling process. Would lead to a situation where the thread wouldn't care to yield to others." 13 years ago
Anoop Saldanha 17c763f855 Version 1 of AC Cuda. 13 years ago
Anoop Saldanha 2de59fc235 Version 1 of CudaBuffer API. Introduced to buffer data to the gpu. 
This version allows async writes to a buffer by threads.  Allows only
sequential reads though.
 13 years ago
Anoop Saldanha 557cab3dc9 We call packet and stream mpm as late as possible now. Won't affect the working of the engine. 
The rationale behind this is, if we have pkt buffered to the gpu, we'd want
to delay processing the pkt as much as possible.
 13 years ago
Anoop Saldanha d2063d98ad pool now uses a queue kinda behaviour when getting/inserting data through poolbuckets. 13 years ago
Anoop Saldanha f4c719b83a code refactoring. Call mpmprefilter slightly later than where it's called atm 13 years ago
Anoop Saldanha b787da5643 Remove all cuda related code in the engine except for the cuda api wrappers 13 years ago
Anoop Saldanha e2a6cfb6a6 update cuda API wrappers 13 years ago
Eric Leblond d8ce2b1ca4 unix-socket: fix OSX build 
MSG_NOSIGNAL is not defined on macOSX and SO_NOSIGPIPE is used
instead.
 13 years ago
Eric Leblond a35c367942 action handling: use macro for test. 
Use test macro instead of direct access to action field.

This patch has been obtained by using the following
spatch file:

  @@
  Packet *p;
  expression E;
  @@

  - p->action & E
  + TEST_PACKET_ACTION(p, E)
 13 years ago
Eric Leblond efaa9a7302 action handling: define and use macros 
The action field in Packet structure should not be accessed
directly as the tunneled packet needs to update the root packet
and not the initial packet.

This patch is fixing issue #819 where suricata was not able to
drop fragmented packets in AF_PACKET IPS mode. It also fixes
drop capability for tunneled packets.
 13 years ago
Anoop Saldanha 429b5cec10 Fix magic unittets. 
Fix segv, when magic_load() fails due to the non-availability of default
magic files.
 13 years ago
Anoop Saldanha 058e9278c5 Fix wrong casting of htp pointer. Fixed it back to (HTPState *) inside 
htp utility functions.
 13 years ago
Anoop Saldanha 21f9cc3a39 discontinue matching on buffer if urilen returns a match failure. 13 years ago
Victor Julien 56c6dd9bb2 bytetest: add unittest showing missed detection 
Tests recursive and relative negative byte_test matching.
 13 years ago
Anoop Saldanha c3d98f9640 Fix the bug specified in the previous commit. 
Bug emanates from byte_test, byte_jump and byte_extract keyword being
unable to handle negative offsets when the inspection pointer is at the
end of the buffer.
 13 years ago
Anoop Saldanha bd6896bee1 Unit-tests exposing a bug in byte_test, byte_jump and byte_extract. 
Bug emanates from all the keywords being unable to handle negative offsets
when the inspection pointer is at the end of the buffer.
 13 years ago
Victor Julien 0fbfaadd53 bytetest: fix debug messages not printing negative offset correctly 13 years ago
Anoop Saldanha ff222b51e7 Http trailer headers unittests added. 13 years ago
Anoop Saldanha ab4b15c2e7 fix for #788. 
Now depth is kept in mind when we inspect chunks in client/server body.
This takes care of FPs originating from inspecting subsequent chunks that
match with depth, but shouldn't.
 13 years ago
Victor Julien f29e5459e6 luajit/flowint: add ScFlowintIncr & ScFlowintDecr 
Add flowint lua functions for incrementing and decrementing flowints.

First use creates the var and inits to 0. So a call:

    a = ScFlowintIncr(0)

Results in a == 1.

If the var reached UINT_MAX (2^32), it's not further incremented. If the
var reaches 0 it's not decremented further.

Calling ScFlowintDecr on a uninitialized var will init it to 0.

Example script:

    function init (args)
        local needs = {}
        needs["http.request_headers"] = tostring(true)
        needs["flowint"] = {"cnt_incr"}
        return needs
    end

    function match(args)
        a = ScFlowintIncr(0);
        if a == 23 then
            return 1
        end

        return 0
    end
    return 0

This script matches the 23rd time it's invoked on a flow.
 13 years ago
Victor Julien f312486c6e flowvar/luajit: make 'sets' real time. Needed for cross HTTP-header matching. 13 years ago
Victor Julien 72f6bc2aed luajit: add flowint support 
Expose ScFlowintGet and ScFlowintSet functions to luajit. These set
flowints in real time, regardless of rule and/or script match.

Example:

function init (args)
    local needs = {}
    needs["http.request_headers"] = tostring(true)
    needs["flowint"] = {"cnt"}
    return needs
end

function match(args)
    a = ScFlowintGet(0);
    if a then
        ScFlowintSet(0, a + 1)
    else
        ScFlowintSet(0, 1)
    end

    a = ScFlowintGet(0);
    if a == 23 then
        return 1
    end

    return 0
end

return 0

Script's init call first registers "cnt" at id 0, then 0 is used to use
this var.
 13 years ago
Victor Julien c3c3cd76e5 flowvar/flowint: split set functions into normal and NoLock version, where the latter won't lock the flow. 13 years ago
Victor Julien 57d3cd97f3 flowvar/flowint: make local function static 13 years ago
Victor Julien 6e18ed0489 luajit flowvar support 
This patch adds flowvar support to luajit. It does so by exposing two special
C functions to the luajit scripts: ScFlowvarGet and ScFlowvarSet.
 13 years ago
Ken Steele 9d677ea006 Clear the PKT_ALLOC flag when storing Packets into the Packet pool. 
The PKT_ALLOC flag is set by PacketGetFromAlloc(), which needs to be
cleared for Packets in the Packet Pool, so clear the flag here.
 13 years ago
Ken Steele 9c7b411a5d More PacketGetFromMalloc() to allocate packets. 13 years ago
Ken Steele 394f99e32c Use PacketGetfromAlloc() for packet allocation instead of SCMalloc. 
Only changed in one file for testing.
 13 years ago
Anoop Saldanha ab1f8afbc3 Removed Signature->order_id and replaced it with Signature->num. 13 years ago
Anoop Saldanha 43d1229dfa 1. Fix assignment of signums, which affected how we used read 
sigs(priority wise) inside staging.

   Previously we would assign signums before sig ordering, and hence the
   order didn't actually reflect the order of the sig in the
   sig_list(assuming sig reordering changed the sig_list).  Staging would
   use the old sig_nums to decide the priority of sigs.
2. Fix sig ordering for flowvar, flowbits, flowint, pktvar sigs.   We have
   introduced a new priority to treat sigs with set + read as lower
   priority compared to set only sigs.
3. Previously we treated sigs with a "priority(keyword)" > another sig's
   priority, as a sig with greater priority than the later.  We have
   reversed it.  Now the sig priority ordering is 1,2,.etc.  Updated
   sigordering unittests to reflect the same.
 13 years ago
Anoop Saldanha 9219079e1a Allow protocols to have both app layer keywords, as well as transaction 
based ones.

Our general logic and assumption is protocols either support one of the
above and not have both.
 13 years ago
Anoop Saldanha a490176c8a More lock fixes for the transaction update. Issues reported by Coverity. 13 years ago
Anoop Saldanha 7cf4042337 Fix luajit compilation failure introduced by the transaction update. 
Fix coverity lock issues reported by transaction update as well.
 13 years ago
Anoop Saldanha d4d18e3136 Transaction engine redesigned. 
Improved accuracy, improved performance.  Performance improvement
noticeable with http heavy traffic and ruleset.

A lot of other cosmetic changes carried out as well.  Wrappers introduced
for a lot of app layer functions.

Failing dce unittests disabled.  Will be reintroduced in the updated dce
engine.

Cross transaction matching taken care of.  FPs emanating from these
matches have now disappeared.  Double inspection of transactions taken
care of as well.
 13 years ago
Anoop Saldanha 6dcde9d7e9 hsbd mpm and packet mpm share same mpm ctx id. 
This is a bug emanating from we having a var reference for hsbd mpm,
but failing to initialize it, and we default to using the packet mpm.
 13 years ago
Ken Steele 93e7304117 Preserve PKT_ALLOC flag inside PACKET_RECYCLE(). 
The PKT_ALLOC flag was being cleared by PACKET_RECYCLE(), which could
then result in a packet being pushed back to the Packet ring buffer
incorrectly.
 13 years ago
Ken Steele 699d9e01f1 Move memset() out of PACKET_INITIALIZE() 
The memset() inside PACKET_INITIALIZE() is redundant in some cases and
it is cleaner to do as part of the memory allocation. This simplifies
changes for integrating Tilera mPIPE support because the size of memory
cleared in that case is different from SIZE_OF_PACKET.

For the cases where Packets are directly allocated and then call
PACKET_INITIALIZE() without memset() first, this patch adds memset() calls.

A further change would use GetPacketFromAlloc() directly.
 13 years ago
Victor Julien 724ad9e8e7 Detect L1 cache line size at build time. Fall back to 64 bytes if detection failed. 13 years ago
Victor Julien 53fe756798 NFQ: convert batchcount related yaml errors to warnings. 13 years ago
Eric Leblond 703e5848e4 nfq: add errno display when verdict fail 
In case of error, errno is set by sendmsg which is called by
nfnetlink and which is called by libnetfilter_queue. This patch
displays the string expression of errno if verdict has failed.
 13 years ago
Florian Westphal 8da02115c9 nfq: add support for batch verdicts 
Normally, there is one verdict per packet, i.e., we receive a packet,
process it, and then tell the kernel what to do with that packet (eg.
DROP or ACCEPT).

recv(), packet id x
send verdict v, packet id x
recv(), packet id x+1
send verdict v, packet id x+1
[..]
recv(), packet id x+n
send verdict v, packet id x+n

An alternative is to process several packets from the queue, and then send
a batch-verdict.

recv(), packet id x
recv(), packet id x+1
[..]
recv(), packet id x+n
send batch verdict v, packet id x+n

A batch verdict affects all previous packets (packet_id <= x+n),
we thus only need to remember the last packet_id seen.

Caveats:
- can't modify payload
- verdict is applied to all packets
- nfmark (if set) will be set for all packets
- increases latency (packets remain queued by the kernel
  until batch verdict is sent).

To solve this, we only defer verdict for up to 20 packets and
send pending batch-verdict immediately if:
- no packets are currently queue
- current packet should be dropped
- current packet has different nfmark
- payload of packet was modified

This patch adds a configurable batch verdict support for workers runmode.
The batch verdicts are turned off by default.

Problem is that batch verdicts only work with kernels >= 3.1, i.e.
using newer libnetfilter_queue with an old kernel means non-working
suricata. So the functionnality has to be disabled by default.
 13 years ago
Florian Westphal 6678c9feb9 nfq: avoid extra copy when running in workers mode 
currently, the packet payload recv()d from the nfqueue netlink
socket is copied into a new packet buffer.

This is required because the recv-buffer space used is tied
to the current thread, but a packet may be handed off to other
threads, and the recv-buffer can be re-used while the packet
is handled by another thread.

However, in worker runmode, the packet will always be handled
by the current thread, and the recv-buffer will only be reused
after the entire packet processing stack is done with the packet.

Thus, in worker runmode, we can avoid the copy and assign
the packet data area directly.
 13 years ago
Victor Julien b68d566c44 alert-debuglog: cleanup TCP check 13 years ago
Victor Julien 4b3166b193 unified2: more udp fixes 13 years ago
Victor Julien bc3f941acb profiling: enabled app layer profiling for UDP app layer modules 13 years ago
Victor Julien 782aa5adae prelude: only call stream callback for TCP 13 years ago
Victor Julien b54a19937f unified2: only call stream callback for TCP 13 years ago
Victor Julien 00948882e7 Suppress warnings when StreamSegmentForEach is called for UDP or SCTP, unless debug is compiled in. 13 years ago
Victor Julien 3b68a9d1c6 UDP: inspection app layer state as soon as we have it. 13 years ago
Victor Julien f15d97b916 Bug 780 unittests, showing no problem. 13 years ago
Victor Julien b6995f7664 Bug 794: stream SACK list needs to respect memcap 13 years ago
Victor Julien a4fca88ba7 stream: default 'random' setting when running unittests is disabled, so that test results are predictable. 13 years ago
Eric Leblond 9b235b3d9e streaming: randomize chunk size 
By randomizing chunk size around the choosen value, it is possible
to escape some evasion technics that are using the fact they know
chunk size to split the attack at the correct place.
This patch activates randomization by default and set the random
interval to chunk size value +- 10%.
 13 years ago
Victor Julien 6ba52230ed Update DetectContentDataParse to reflect the actual data types content uses. 13 years ago
Victor Julien 3ad497e74f Remove filemagic debug statement 13 years ago
Victor Julien 19511cda97 Remove obsolete DetectParseContentString function, it has been replaced by DetectContentDataParse 13 years ago
Victor Julien 4d4f8fd358 file: make fileext, filename and filemagic use the same rule parsing function as others. This has as a side effect that we enforce doubly qouted values now. 13 years ago
Victor Julien 8023007fbd flowvar: cleanup keyword argument parsing. Should also address Coverity 400655. 13 years ago
Victor Julien 07b751b0df Coverity 1005134: fix minor memory leak on flowvar rule setup errors. 13 years ago
Victor Julien e45f683c19 Coverity 1005133: fix unlikely case where malformed pcre statement in rule would lead to null-deref. 13 years ago
Victor Julien 4c6463f378 stream: handle extra different SYN/ACK 
Until now, when processing the TCP 3 way handshake (3whs), retransmissions
of SYN/ACKs are silently accepted, unless they are different somehow. If
the SEQ or ACK values are different they are considered wrong and events
are set. The stream events rules will match on this.

In some cases, this is wrong. If the client missed the SYN/ACK, the server
may send a different one with a different SEQ. This commit deals with this.

As it is impossible to predict which one the client will accept, each is
added to a list. Then on receiving the final ACK from the 3whs, the list
is checked and the state is updated according to the queued SYN/ACK.
 13 years ago
Victor Julien 00a691fc1b flowvar: clean up properly on signature clean up. 14 years ago
Victor Julien 70e2adeb01 flowvar: add unittests for #802. 14 years ago
Victor Julien 4cd736fcc9 flowvar: fix deadlock with http buffers 
Bug #802

Flowvars are set from pcre, and lock the flow when being set. However
when HTTP buffers were inspected, flow was already locked: deadlock.

This patch introduces a post-match list in the detection engine thread
ctx, where store candidates are kept. Then a post-match function is used
to finalize the storing if the rule matches.

Solves the deadlock and brings the handling of flowvars more in line
with flowbits and flowints.
 14 years ago
Victor Julien 4c2e6a8402 flowvars: update funcs to accept u16 id 
All id's are u16, but flowvar functions would only accept u8.

Minor cleanups.
 14 years ago
Victor Julien ffffe6c10e profiling: add formatted totals, percents to packet stats 14 years ago
Victor Julien 4165de4771 Minor SigValidate cleanup 14 years ago
Anoop Saldanha 0d7305dfc7 Update the way we handle http_host keywords. 
Previously we would have forced all users to use nocase with http_host
keywords(since the hostname buffer is lowercase).

We now error out on sigs that has nocase set with http_host set.  Also if
the http_host pattern or http_host pcre has an uppercase character set, we
invalidate such sigs.  Unittests also updated to reflect the above change.
 14 years ago
Victor Julien 9ea4d36f7a Minor reshuffling of Signature struct. 14 years ago
Victor Julien eb11280888 Use define instead of magic number for pmq's per detect thread 14 years ago
Victor Julien 0fa38c13d1 detection engine: consolidate thread setup 
DetectEngineThreadCtxInit and DetectEngineThreadCtxInitForLiveRuleSwap did
pretty much the same thing, except for a counters registration. As can be
predicted with code duplication like this, things got out of sync. To make
sure this doesn't happen again, I created a helper function that does the
heavy lifting in this function.
 14 years ago
Victor Julien 73158fea33 Fix PmqSetup calls in Liveswap thread init. Func was out of sync with normal thread init. 14 years ago
Victor Julien b8078742c3 stream: intro function for SYN/ACK state update 
As the TCP SSN state can be updated from several points in the state
machine on accepting a SYN/ACK, move the update logic into a separate
function.
 14 years ago
Victor Julien 28ea129d9b stream: remove unused 'pause' feature 14 years ago
Victor Julien ea8b6078d8 stream: zero ts is a per stream flag 
Ssn flag STREAMTCP_FLAG_ZERO_TIMESTAMP was used in stream only. Due to
it's value it did not conflict with a real stream flag. Renamed it to
STREAMTCP_STREAM_FLAG_ZERO_TIMESTAMP.
 14 years ago
Victor Julien 374187bf65 stream: don't use ssn timestamp flag in stream 
The STREAMTCP_FLAG_TIMESTAMP flag is a ssn flag, however it was used in
the stream flag field. As it has the same value as
STREAMTCP_STREAM_FLAG_DEPTH_REACHED it's possible that stream reassembly
got confused by the timestamp.
 14 years ago
Victor Julien 40a5ce8f5f Change logic of SCErrorToString causing any missing entries to result in a compiler warning. 14 years ago
Anoop Saldanha 71ffed5128 Handle the case of pcre combined with a relative content, where pcre has the 
set to match from start of line and we discontinue matching on not finding
match.
 14 years ago
Anoop Saldanha aa363a8144 unittest to display #784. 14 years ago
Eric Leblond 26b7af1483 Don't try to sniff 'default' interface 
Whan running suricata via 'suricata --af-packet', the list of interfaces
was containing the 'default' interface and sniffing it was attempted.
This was not wanted.
 14 years ago
Eric Leblond 539de3f5ea bpf filter: use SCLogError instead of fprintf 14 years ago
Eric Leblond b7e78d33b1 af-packet: warn about BPF filter consequence in IPS mode 
This patch add a message to warn user about the impact of using a
BPF filter in IPS mode.
 14 years ago
Eric Leblond dfbb31df8a Exit if bpf is used in IPS mode 14 years ago
Victor Julien ce99a07582 After some discussion we decided that var declarations inside a for statement are not in line with our coding style. So removing a bunch. Decision was not unanimous ^^. 14 years ago