Commit Graph

10259 Commits (d21a252238fd766d3c2ec4c2d165d429dad0d85a)

Author SHA1 Message Date
Philippe Antoine d21a252238 fuzz: target must use the rules it parsed
DetectEngineReloadThreads does not work for the fuzz targets
as there is no_of_detect_tvs = 0 as we did not register
real threads and slots.

So, we force the flow worker module to use the newly detect engine
conetxt with all it needs
5 years ago
Victor Julien fa72a5add8 flow: free spare pool more aggressively
The flows exceeding the spare pools config setting would be freed
per at max 100 flows a second. After a high speed test this would
lead to excessive memory use for a long time.

This patch updates the logic to free 10% of the excess flows per
run, freeing multiple blocks of flows as needed.

Bug: #4731.
5 years ago
Victor Julien ff97d7c15d threading: force break loop on flow inject
Track availability of break loop callback to avoid overhead.
5 years ago
Victor Julien b788d3345c flow: process evicted flows on low/no traffic
In a scenario where there was suddenly no more traffic flowing, flows
in a threads `flow_queue` would not be processed. The easiest way to
see this would be in a traffic replay scenario. After the replay is done
no more packets come in and these evicted flows got stuck.

In workers mode, the capture part handles timeout this was updated to
take the `ThreadVars::flow_queue` into account.

The autofp mode the logic that puts a flow into a threads `flow_queue`
would already wake a thread up, but the `flow_queue` was then ignored.
This has been updated to take the `flow_queue` into account.

In both cases a "capture timeout" packet is pushed through the pipeline
to "flush" the queues.

Bug: #4722.
5 years ago
Victor Julien 31977170a8 threading: minor cleanups 5 years ago
Jeff Lucovsky 314ec77f88 unittests/template: Register template unittests 5 years ago
Jeff Lucovsky 6e149cdec3 unittests/enip: Register ENIP unittests 5 years ago
Philippe Antoine 8a50edbd10 pcre: fixes a memory leak on alloc error 5 years ago
Philippe Antoine 8536048443 http2: do not try to upgrade if http2 is disabled in config 5 years ago
Philippe Antoine 42ba421ca9 http2: flatten code style 5 years ago
Philippe Antoine 527415dba0 protodetect: handle all gaps, even when depth is reached 5 years ago
Philippe Antoine 586522e1e9 pcre: local match data for pcrexform 5 years ago
Philippe Antoine c64a1f6a09 pcre: use thread-storage for matches 5 years ago
Philippe Antoine 3b690e53c8 pcre: using de_ctx in unit tests for free function 5 years ago
Philippe Antoine a049a6c29c pcre: creates a match structure per match run
So that DetectPcrePayloadMatch is thread safe
and does not rewrite a shared parse_regex.match structure
5 years ago
Philippe Antoine 78cf9cfc5f http: range: remove assert that can happen 5 years ago
Philippe Antoine 65a6f61004 http: delete obsolete range log
Commit d776d72711
has been transfering ownership of file container

So, we cannot log it
5 years ago
Philippe Antoine 9b3c355c20 fuzz: adds one target with predefined rules 5 years ago
Jason Ish df0ed6fda4 af-packet: use configured cluster-id when checking for fanout
When testing for fanout support a cluster-id of 1 was always being
used instead of the configured cluster-id. This limited fanout
support to only one Suricata instance.

Instead of hardcoding an ID of 1, use the configured cluster-id.

Also make cluster_id a uint16_t instead of an int in AFPThreadVars.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/3419
5 years ago
Philippe Antoine 3a230c2208 ipv6: simpler generic overlap condition
This also changes the behavior, as the condition is checked in
every case cf ipv6-malformed-fragments-8
5 years ago
Philippe Antoine 066a3d619e fuzz: updates README
Uses main oss-fuzz repository

More complete flags proposition for direct compilation
5 years ago
Philippe Antoine cb30772372 style: remove latest warnings
about unused variables
5 years ago
Philippe Antoine f6ba3699bb flowint: same analysis warnings as flowbits 5 years ago
Philippe Antoine bce3c46874 pcre2: remove PCRE1 as dependency 5 years ago
Philippe Antoine 56f664af6b pcre2: follow code naming style 5 years ago
Philippe Antoine e10d930660 pcre2: only one DetectParseRegex structure 5 years ago
Philippe Antoine 2a1690e110 pcre2: move header include to suricata-common only 5 years ago
Philippe Antoine 48dd0cf804 pcre2: check for PCRE2_ERROR_UNSET
Needs maybe to be generalized
5 years ago
Philippe Antoine 3de99a214c pcre2: migrate keywords parsing 5 years ago
Philippe Antoine 2dea9a1e37 pcre: use pcre2 to parse detect pcre itself 5 years ago
Philippe Antoine f0f3295ba0 pcre: migrate detect-pcre to pcre2
pcre2 substrings need special free...
5 years ago
Philippe Antoine 3633c48e6e pcre2: migrate utility uses of pcre 5 years ago
Philippe Antoine 95a07c8957 pcre2: migrate transform pcrexform 5 years ago
Philippe Antoine 98f84d5a9b http2: follow range requests
Move the content-range parsing code to rust
5 years ago
Philippe Antoine bded2ec92e range: post process out of order blocks
A block is determined out of order on opening.
But on closing, the gap before it may have been filled.
So, we must post-process it, ie iterate over the red and black
tree so see what blocks we can get.
5 years ago
Philippe Antoine 3b9a4f9f19 range: no final zero in key
for future compatibility with rust
5 years ago
Philippe Antoine 7819f3262f http: more consistent return values for HTPFileOpenWithRange 5 years ago
Philippe Antoine bb7cc6f5e3 http: increase tracked files for ranges as well 5 years ago
Philippe Antoine d776d72711 http: range transfering ownership of file container
To make concurrency reasoning clearer
5 years ago
Philippe Antoine 3ed38d2d5d http: avoid one lock for range append data
Better structure design to ensure that one flow maximum
is owning and appending into the file, adding fileOwning field.

Adds also a gap field in a range buffer, so that we can
feed the gap on closing, when we are protected from concurrency
by a lock, (lock which got removed in the append path)

Fixes memcap when encountering a duplicate while inserting
in red and black tree

Adds many comments
5 years ago
Victor Julien 2ef857e533 thash: add debug validation check for use_cnt 5 years ago
Victor Julien 7a797631e0 http/range: cleanup and simplification
Simplify locking by using the THashData lock instead of a separate
range lock.

Avoid size_t in function arguments.

Clean up file handling functions.

Implement handling of alloc errors.

Rename yaml entry to byterange

Unify public api naming
5 years ago
Victor Julien 5ffc9622b3 http/range: optimize struct layout 5 years ago
Victor Julien fc5632664c http/range: fix memory leak on out of order ranges 5 years ago
Philippe Antoine e82416a415 http/range: reassemble files from different flows with range
adds a container, ie a thread safe hash table whose
key is the filename

keep a tree of unordered ranges, up to a memcap limit

adds HTPFileOpenWithRange to handle like HTPFileOpen
if there is a range : open 2 files, one for the whole reassembled,
and one only for the current range
5 years ago
Philippe Antoine 5cb996efcf util: export Djb2 hash string function 5 years ago
Philippe Antoine f0012f97b2 util: adds util function SCBufferCmp
Compares two buffers with their two sizes
5 years ago
Victor Julien 13a6eb3522 detect: suppress scan-build warning 5 years ago
Jason Ish 102f0c74ff lua: move lua includes to util-lua.h
Moves Lua includes to util-lua.h instead of suricata-common
so plugins (or library users) don't need to be aware of the
Lua that Suricata was built with. Instead only source files
that need to be Lua aware can include util-lua.h.
5 years ago
Philippe Antoine ae4e3e95d5 transform: use generic Free function
to free subelements in the context structure
5 years ago