73 Commits (6bccd5aa30c03debc92c1e910d5f0dd1c6a9cc58)

Author	SHA1	Message	Date
Victor Julien	e250ef6402	debug: remove empty header	3 years ago
Juliana Fajardini	e7727c3744	decode: remove unused macros, replace w/ functions ... With the recent changes, these macros weren't being used anymore. Related to Bug #5458	3 years ago
Juliana Fajardini	1f54e8611a	detect/alert: ensure reject action is applied ... Bug 5458 states that the reject action is no longer working. While SV tests that use the reject action still pass, it indeed seems that a regression has happened with commit aa93984, because while the function that applies rule actions to the flow (RuleActionToFlow) does check for the reject action, the newly added function PacketApply SignatureActions only checks for ACTION_DROP or ACTION_PASS when deciding to call RuleActionToFlow. Bug #5458	3 years ago
Juliana Fajardini	1774ff18a6	decode: make PacketDrop use action as parameter ... A Packet may be dropped due to several different reasons. This change adds action as a parameter, so we can update the packet action when we drop it, instead of setting it to drop. Related to Bug #5458	3 years ago
Victor Julien	df2e408d96	detect/threshold: fix offline time handling issue ... Due to the TIMEVAL_DIFF_SEC calculating the delta into an unsigned integer, it would underflow to a high positive value leading to and incorrect result if the packet timestamp was below the timestamp for the threshold entry. In normal conditions this shouldn't happen, but in offline mode each thread has its own concept of time which might differ significantly based on the pcap. In this case the overflow would be very common. Changing it to a signed value calculation triggered fuzz undefined behavior if the packet timeval was very high, so this patch takes a new approach where it no longer calculates a diff but sets up the "seconds" value we compare against as a timeval itself, and uses that to compare. Fixes: 9fafc1031c ("time: Add TIMEVAL_EARLIER and TIMEVAL_DIFF_SEC macros.") Fixes: 82dc61f4c3 ("detect/threshold: Refactor threshold calculation to handle by_rule and by_both.") Uses add `timeradd` specific version where available. Bug: #5386.	3 years ago
Victor Julien	0035673208	eve/drop: log drop reason ... Ticket: #5202.	3 years ago
Philippe Antoine	6fadb97d5d	alert: fixes leak in ThresholdHandlePacketRule ... ThresholdHandlePacketRule may take ownership of an allocated DetectThresholdEntry, and places it in a position of the array th_entry. But it never got released	4 years ago
Mats Klepsland	2a326421aa	thresholds: Fix buffer overflow in threshold context ... th_entry is resized using ThresholdHashRealloc() every time a rule with a threshold using by_rule tracking is added. The problem is that this is done before the rules are reordered, so occasionally a rule with by_rule tracking gets a higher signature number (after reordering) than the number of th_entries allocated, causing Suricata to crash. This commit fixes this by allocating th_entries after all the rules are loaded and reordered. Backtrace from core dump: Program terminated with signal SIGSEGV, Segmentation fault. #0 0x000000000051b381 in ThresholdHandlePacket (p=p@entry=0x7fb0080f3960, lookup_tsh=0x51, new_tsh=new_tsh@entry=0x7fb016c316e0, td=td@entry=0x14adedf0, sid=9800979, gid=1, pa=0x7fb0080f3b18) at detect-engine-threshold.c:415 415>---- if (TIMEVAL_DIFF_SEC(p->ts, lookup_tsh->tv1) < td->seconds) { Bug #4503.	4 years ago
Mats Klepsland	f47e4375b3	thresholds: syntax fixes ... Fix syntax of if statement in SigGetThresholdTypeIter()	4 years ago
Mats Klepsland	b0b4fab794	thresholds: remove unneeded function argument ... Remove packet pointer from SigGetThresholdTypeIter() as it is unused.	4 years ago
Juliana Fajardini	b807059c34	host/storage: use dedicated 'id' type ... - Wrap the id in a HostStorageId struct to avoid id confusion with other storage API calls. - Fix formatting with clang script.	5 years ago
Juliana Fajardini	cf516de587	ippair/storage: use dedicated 'id' type ... - Wrap the id in a new IPPairStorageId struct, to avoid id confusion with other storage API calls. - Formatting fixes by clang.	5 years ago
Shivani Bhardwaj	6f7d8e50c8	src: use FatalError whenever possible ... Replaces all patterns of SCLogError() followed by exit() with FatalError(). Cocci script to do this: @@ constant C; constant char[] msg; @@ - SCLogError(C, + FatalError(SC_ERR_FATAL, msg); - exit(EXIT_FAILURE); Closes redmine ticket 3188.	5 years ago
Todd Mortimer	50e5b80463	detect/threshold: Add a common function to (re)allocate the by_rule threshold table. ... Ensure that the by_rule threshold table is initialized if a rule is thresholded by_rule. Replace manual table reallocaton with calls to the common function.	6 years ago
Todd Mortimer	82dc61f4c3	detect/threshold: Refactor threshold calculation to handle by_rule and by_both. ... The only difference between threshold calculations for by_src/by_dst, by_rule or by_both is which table stores the DetectThresholdEntry. Refactor the ThresholdHandlePacket* functions to do table lookup and storage individually, but calculate thresholds in a common function.	6 years ago
Victor Julien	990e53222e	threshold: minor cleanups	8 years ago
Ruslan Usmanov	fb87d21ec7	rate_filter: by_rule fixed triggering algorithm ... Fixes issue #2258 Program was triggering rate_filter by_rule earlier than needed and generally behaved like a threshold.	8 years ago
Victor Julien	50a762bfd1	thresholds: fix issues with host based thresholds ... The flow manager thread (that also runs the host cleanup code) would sometimes free a host before it's thresholds are timed out. This would lead to misdetection or too many alerts. This was mostly (only?) visible on slower systems. And was caused by a mismatch between time concepts of the async flow manager thread and the packet threads, resulting in the flow manager using a timestamp that was before the threshold entry creation ts. This would lead to an integer underflow in the timeout check, leading to a incorrect conclusion that the threshold entry was timed out. To address this, check if the 'check' timestamp is not before the creation timestamp.	8 years ago
Ruslan Usmanov	1090ee9d8d	rate_filter by_both through IPPair storage ... Ticket https://redmine.openinfosecfoundation.org/issues/2127	8 years ago
Victor Julien	ab1200fbd7	compiler: more strict compiler warnings ... Set flags by default: -Wmissing-prototypes -Wmissing-declarations -Wstrict-prototypes -Wwrite-strings -Wcast-align -Wbad-function-cast -Wformat-security -Wno-format-nonliteral -Wmissing-format-attribute -funsigned-char Fix minor compiler warnings for these new flags on gcc and clang.	9 years ago
Victor Julien	859cb89c7e	detect alert/threshold/tag: sm_list -> sm_array	9 years ago
Victor Julien	e072e70ea6	alert: fix rate_filter issues ... Fix rate_filter issues: if action was modified it wouldn't be logged in EVE. To address this pass the PacketAlert structure to the threshold code so it can flag the PacketAlert as modified. Use this in logging. Update API to use const where possible. Fix a timout issue that this uncovered.	9 years ago
Victor Julien	77302e5d51	threshold: remove debug message from info loglevel	10 years ago
Victor Julien	ebb42f831c	suppress: add track by_either mode ... So far suppress rules would apply to src or dst addresses of a packet. This meant that if a ip would need to suppressed both as src and as dst, 2 suppress rules would be needed. This patch introduces track by_either, which means that the ip(s) in the suppress rule are tested against both the packets source and dest ip's. If either of them is on the suppress list, the alert is suppressed.	11 years ago
Victor Julien	e85a44c383	suppress: support ip-lists ... Ticket: 1137 Support supplying a list of IP's to the suppress keyword. Variables from the address-groups and negation is supported. The same logic (and code) is used that is also used in parting the IP portions of regular detection rules.	11 years ago
Ken Steele	8f1d75039a	Enforce function coding standard ... Functions should be defined as: int foo(void) { } Rather than: int food(void) { } All functions where changed by a script to match this standard.	11 years ago
Victor Julien	8ce38ac8fe	Split Thresholds and Suppression ... Thresholds and suppression can be handled independently. Suppression only suppresses output, and is not related to Threshold state tracking. This simplifies mixing suppression and thresholding rules. Part of the Bug #425 effort.	12 years ago
Victor Julien	5a7bf53a6b	Storage: rename Init to Alloc to reflect actual functioning. Comment updates.	12 years ago
Eric Leblond	6d08807b2d	Host: use global free storage function ... This patch is here to avoid that all modules using a local storage have to update host code to add their free function. It modifies previous behavior by calling HostFreeStorage in any case.	12 years ago
Victor Julien	27023872de	Use Host Storage API for per host thresholding	12 years ago
Eric Leblond	3f107fa130	decode: Packet action start with PACKET ... Rename all Packet action macro to have them prefixed by PACKET.	13 years ago
Victor Julien	80d62b59ec	Fix drop (and other actions) not being applied to thresholded packets. Bug #613.	13 years ago
Eric Leblond	e176be6fcc	Use unlikely for error treatment. ... When handling error case on SCMallog, SCCalloc or SCStrdup we are in an unlikely case. This patch adds the unlikely() expression to indicate this to gcc. This patch has been obtained via coccinelle. The transformation is the following: @istested@ identifier x; statement S1; identifier func =~ "(SCMalloc|SCStrdup|SCCalloc)"; @@ x = func(...) ... when != x - if (x == NULL) S1 + if (unlikely(x == NULL)) S1	13 years ago
Anoop Saldanha	7dec21be4c	fix rate filters that reset the sig ctx data and handled action timeouts wrongly	14 years ago
Anoop Saldanha	85db868a83	indentation fix	14 years ago
Anoop Saldanha	c34713321a	fix rate filter alert suppression. Log error if rate filter has count of 0. Other minor fixes as well	14 years ago
Anoop Saldanha	b48a686d65	considering the tenths of a seconds in a packet, when calculating thresholds	14 years ago
Anoop Saldanha	b899146229	fix detection filter. Had one extra alert than normal previously, now fixed	14 years ago
Victor Julien	0a80e362aa	Fix some minor clang scan-build warnings.	14 years ago
Victor Julien	c0a2cbd478	Move over src and dst thresholding to use host table. Fix a bug in threshold 'both' handling.	14 years ago
Victor Julien	c8c4a76dc6	Move threshold to it's own sig match list.	14 years ago
Eric Leblond	142fe6e4b6	threshold: fix recently introduced function.	14 years ago
Eric Leblond	e5b638e5e8	threshold: introduce SigGetThresholdTypeIter function ... This patch introduces a function called SigGetThresholdTypeIter which iterate on all Threshold for a given signature returning the next DetectThresholdData.	14 years ago
Victor Julien	820b0ded82	Add per packet profiling. ... Per packet profiling uses tick based accounting. It has 2 outputs, a summary and a csv file that contains per packet stats. Stats per packet include: 1) total ticks spent 2) ticks spent per individual thread module 3) "threading overhead" which is simply calculated by subtracting (2) of (1). A number of changes were made to integrate the new code in a clean way: a number of generic enums are now placed in tm-threads-common.h so we can include them from any part of the engine. Code depends on --enable-profiling just like the rule profiling code. New yaml parameters: profiling: # packet profiling packets: # Profiling can be disabled here, but it will still have a # performance impact if compiled in. enabled: yes filename: packet_stats.log append: yes # per packet csv output csv: # Output can be disabled here, but it will still have a # performance impact if compiled in. enabled: no filename: packet_stats.csv Example output of summary stats: IP ver Proto cnt min max avg ------ ----- ------ ------ ---------- ------- IPv4 6 19436 11448 5404365 32993 IPv4 256 4 11511 49968 30575 Per Thread module stats: Thread Module IP ver Proto cnt min max avg ------------------------ ------ ----- ------ ------ ---------- ------- TMM_DECODEPCAPFILE IPv4 6 19434 1242 47889 1770 TMM_DETECT IPv4 6 19436 1107 137241 1504 TMM_ALERTFASTLOG IPv4 6 19436 90 1323 155 TMM_ALERTUNIFIED2ALERT IPv4 6 19436 108 1359 138 TMM_ALERTDEBUGLOG IPv4 6 19436 90 1134 154 TMM_LOGHTTPLOG IPv4 6 19436 414 5392089 7944 TMM_STREAMTCP IPv4 6 19434 828 1299159 19438 The proto 256 is a counter for handling of pseudo/tunnel packets. Example output of csv: pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading 1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337 First line of the file contains labels. 2 example gnuplot scripts added to plot the data.	14 years ago
Eric Leblond	8787e6f6d0	suppress: use DetectAddress instead of DetectAddressHead	14 years ago
Eric Leblond	7938344e1b	threshold: refactoring of parsing code ... This patch factorize the regular expression to be ease the parsing process. It also adds a missing free and factorize exit code.	14 years ago
Eric Leblond	03c185a3ad	threshold: add suppress keyword ... This patch adds the suppress keyword to the threshold.config file. The alerts are suppressed but the other elements like flowbits are maintained.	14 years ago
Eric Leblond	a56f8dd6b2	doc: introduce doxygen group "threshold" ... This patch introduces a doxygen group to put together the documentation relative to threshold. Group appear in a separate page and they can have their own documentation. This is useful when a feature is splitted into different files.	14 years ago
Victor Julien	477bc1d050	Set DROP flag on a packet in addition to the REJECT flags. This makes sure we not only send a reject, but also drop the offending packet. Closes #248.	15 years ago
Victor Julien	b600c9ac09	Fix a clang issue 'Assigned value is garbage or undefined' in the threshold code in case a packet was neither ipv4 or ipv6.	15 years ago