Commit Graph

15514 Commits (e5d3fad5f40acffab937b6f61346346e7e4a28f6)
 

Author SHA1 Message Date
Victor Julien 474a89e098 detect/file.data: modernize test 3 years ago
Victor Julien fa450c0531 detect: modernize unittest 3 years ago
Victor Julien 6cf0e4d604 detect/content-inspect: keyword context as const 3 years ago
Victor Julien b69f4cb5cf detect/pcre: match data is const at match time 3 years ago
Victor Julien 529e02686b detect/http.uri: modernize unittest 3 years ago
Victor Julien 23d15259f5 util/print: minor code cleanups 3 years ago
Victor Julien 132fe57ac6 rust: add copyright header to common.rs 3 years ago
Victor Julien 99c616389e util/prefilter: constify sids 3 years ago
Victor Julien 3b3b0ed30a mpm: free sids in MpmFreePattern as well 3 years ago
Victor Julien 14c452e4c6 mpm: cleanup pattern free function
Avoid redundant pointer checks; instead check once.
3 years ago
Victor Julien 0b21b543a2 mpm/ac-bs: add warning if still used
Fall back to default matcher.

Ticket #6586.
3 years ago
Victor Julien 2be36c0f0c mpm: document Search callback return value 3 years ago
Victor Julien 74ef5fc3d1 mpm/ac-ks: return only unique match count
Bring implementation in line with Hyperscan, which only counts unique matches.

Update test to reflect the new behavior.
3 years ago
Victor Julien 83630ebb9c mpm/ac: return only unique match count
Bring implementation in line with Hyperscan, which only counts unique matches.

Update test to reflect the new behavior.
3 years ago
Victor Julien 6a73b3c90b mpm: remove ac-bs implementation
Ticket: #6586.
3 years ago
Victor Julien ee7c140028 detect: minor cleanup
MPM_HS does not need a guard.
3 years ago
Jason Ish cc0adaaf4a userguide: remove old css files
In our conf.py we reference some ReadTheDocs stylesheets that appear to
be old and break formatting of some items like bulletted lists.

Bug: #6589
3 years ago
Victor Julien 7f42506760 detect: reimplement discontinue matching logic
Previously various steps in the content inspection logic would use
a variable in the DetectEngineThreadCtx to flag that matching should
be discontinued.

This patch reimplements this logic by using a new return code instead.

Split content inspection into public and private version, so that
common initialization can be done in a single place.

Update the callsites.
3 years ago
Victor Julien 368adf4599 detect/file-data: simplify content inspect loop 3 years ago
Victor Julien ee66a7246f detect-engine: minor content inspection cleanup 3 years ago
Victor Julien 4ccc8293b1 packet: minor macro cleanups 3 years ago
Victor Julien 9ae2cd0c59 packet: access packet data through flex array 3 years ago
Victor Julien bc7508e4df log-pcap: constify PcapWrite args
General cleanup, but also needed for packet changes.
3 years ago
Victor Julien f10233fecf device/storage: use flex array instead of calculated ptr 3 years ago
Victor Julien 11d73e284c ippair/storage: use flex array instead of calculated ptr 3 years ago
Victor Julien 3d3a62dfe6 host/storage: use flex array for host storage 3 years ago
Victor Julien d405efd3f6 flow/storage: use flex array instead of calculated ptr 3 years ago
Victor Julien c1b920d056 storage: use proper type instead of void ptr 3 years ago
Victor Julien 69f61c0028 storage: remove unused code
Only used in a unittest; removed that as well.
3 years ago
Philippe Antoine 9c3ab36afc source: fix resource leak
CID: 1426081
3 years ago
Philippe Antoine 5954a91457 detect/filestore: fix memory leak on sig parsing
Ticket: 6574

Introduced by commit c272a646c5
3 years ago
Victor Julien d005fff7b9 stats: improve sync signalling
Make syncs more reliable by using a atomic "sync now" variable and
signalling the conditions under lock.

Ticket: #6569.
3 years ago
Victor Julien 74bc27ecd5 stats: turn sync macros into functions 3 years ago
Victor Julien 3107a4953d flow: fix condition signalling
Signal threads while holding lock. This should make the signalling
more reliable.

From PTHREAD_COND(3):

"Unlocking the mutex and suspending on the condition variable is done
 atomically. Thus, if all threads always acquire the mutex before
 signaling the condition, this guarantees that the condition cannot be
 signaled (and thus ignored) between the time a thread locks the
 mutex and the time it waits on the condition variable."

Ticket: #6569.
3 years ago
Victor Julien 487b78fb3d ippair: fix minor coverity warning
CID 1554232:  Data race undermines locking  (LOCK_EVASION)

Ticket #6565.
3 years ago
Victor Julien 1f7e61ef61 host: fix minor coverity warning
CID 1554240:  Data race undermines locking  (LOCK_EVASION)
3 years ago
Philippe Antoine 3a79984e5e detect/xbits: fix coverity warning
CID 1554237 and CID 1554233

Basically make the code easier to reason with for coverity
without changing the behavior which was fine.
3 years ago
Victor Julien 41c0526fdc packetpool: signal condition within lock
Completes: dc40a139ac ("packetpool: signal waiter within lock")
3 years ago
Victor Julien edc89ce791 packetpool: dynamic return threshold
Problem:

In pcap autofp mode, there is one threads reading packets (RX). These packets
are then passed on to worker threads. When these workers are done with a
packet, they return packets to the pcap reader threads packet pool, which is
the owner of the packets. Since this requires expensive synchronization between
threads, there is logic in place to batch this operation.

When the reader thread depletes its pool, it notifies the other threads that
it is starving and that a sync needs to happen asap. Then the reader enters
a wait state. During this time no new packets are read.

However, there is a problem with this approach. When the reader encountered
an empty pool, it would set an atomic flag that it needed a sync. The first
worker to return a packet to the pool would then set this flag, sync, and
unset the flag. This forced sync could result in just a single packet being
synchronized, or several. So if unlucky, the reader would just get a single
packet before hitting the same condition again.

Solution:

This patch updates the logic to use a new approach. Instead of using a
binary flag approach where the behavior only changes when the reader is
already starved, it uses a dynamic sync threshold that is controlled by
the reader. The reader keeps a running count of packets it its pool,
and calculates the percentage of available packets. This percentage is
then used to set the sync threshold.

When the pool is starved, it sets the threshold to 1 (sync for each packet).
After each successful get/sync the threshold is adjusted.
3 years ago
Jeff Lucovsky ec1482cf48 calloc: Use nmemb with SCCalloc
This commit modifies calls to SCCalloc that had a member count of 1 and
a size count calculated as: element_count * sizeof(element).
3 years ago
Jeff Lucovsky 193e0ea1a9 memory/alloc: Use SCCalloc instead of malloc/memset 3 years ago
Philippe Antoine c272a646c5 detect: SigMatchAppendSMToList can fail
Ticket: #6104

And failures should be handled to say that the rule failed to load

Reverts the fix by 299ee6ed55
that was simple, but not complete (memory leak),
to have this bigger API change which simplifies code.
3 years ago
Philippe Antoine e38b9de6a2 output/krb5: have krb5 properties in alerts
Ticket: 5977
3 years ago
Philippe Antoine 8a09bff0aa output/tftp: have tftp properties in alerts
Ticket: 6501
3 years ago
Philippe Antoine 3b1b163ee6 output/ftp: have ftp properties in alerts
Ticket: 6500
3 years ago
Philippe Antoine 0b6b015e26 output/alert: rewrite code for app-layer properties
Especially fix setup-app-layer script to not forget this part

This allows, for simple loggers, to have a unique definition
of the actual logging function with the jsonbuilder.
This way, alerts, files, and app-layer event can share the code
to output the same data.

Ticket: #3827
3 years ago
Philippe Antoine 90c17652a3 rust: remove unused
Ticket: #4083
3 years ago
Philippe Antoine 32cce122e1 detect: header_lowercase transform
Ticket: 6290
3 years ago
Joseph Reilly 55a4e12884 af-xdp: detach XDP program early
To mitigate a bug with AF_XDP sockets in high traffic scenarios, the XDP program must be detatched before
the sockets are closed. This issue happens when large ammounts of traffic are sent to suricata and
the XDP program is not removed before AF_XDP sockets are closed. I believe this is a race
condition bug as detailed here: https://bugzilla.kernel.org/show_bug.cgi?id=217712

Further investigation shows this may be a bug exclusive to the driver/AMD processor combination.

This commit addresses the bug by ensuring the first thread to run the deinit function
removes the XDP program, which fixes the bug as detailed in the bugzilla link.

Bug #6238
3 years ago
Eric Leblond 180459eeb6 detect/stream_size: fix prefiltering registration
Ticket: #6551
3 years ago