Commit Graph

10777 Commits (75ec528384a5b75c99de110dd41863992b550f1b)
 

Author SHA1 Message Date
Victor Julien dce227ec88 threading/queues: remove 256 queue limit
Convert fixed size array to a dynamic TAILQ so we can
grow as needed.
5 years ago
Victor Julien 0e3f27a87e threading: remove 'trans_q' array of packet queues
Let the queues code set up PacketQueues on demand.
5 years ago
Victor Julien efa8a69923 packet-queue: create alloc and free functions 5 years ago
Victor Julien 550cfdd98d threading: hide 'trans_q' from queue handlers 5 years ago
Victor Julien 45e5e19e6e threading/threadvars: optimize layout
Make sure StatsPublicThreadContext is on its own cache line.
5 years ago
Victor Julien e3fbdf1948 flowworker/stream: use no-lock packet queue
Use smaller structure for temporary packet queues.
5 years ago
Victor Julien f8aed4ce2d threading: change local packet queue logic
Previously each 'TmSlot' had it's own packet queue that was passed
to the registered SlotFunc as an argument. This was used mostly for
tunnel packets by the decoders and by defrag.

This patch removes that in favor of a single queue in the ThreadVars:
decode_pq. This is the non-locked version of the queue as this is
only a temporary store for handling packets within a thread.

This patch removes the PacketQueue pointer argument from the API.
The new queue can be accessed directly through the ThreadVars
pointer.
5 years ago
Victor Julien b8c2b66d33 packet-queue: introduce a non-locked version
Works exactly like PacketQueue, just does not contain a mutex
and cond var, leading to much reduced memory size.
5 years ago
Victor Julien 9ed260c489 threading: more efficient TmSlot layout 5 years ago
Victor Julien 18e652309f threading: remove 'id' field from TmSlot
Field was now unused.
5 years ago
Victor Julien d7cb0774dd detect: cleanup reload thread handling 5 years ago
Victor Julien 786e697590 threading: simplify flow timeout loop 5 years ago
Victor Julien 261b77742e threading: shrink and reorganize TmSlot 5 years ago
Victor Julien 87c9b11d8c threading/threadvars: rearrange for better cache behavior 5 years ago
Victor Julien 071b753e84 threading/threadvars: remove unused 'prev' field 5 years ago
Victor Julien f53f004917 threading: remove unused 'TmThreadRemove' function 5 years ago
Victor Julien 569a5d985b threading: remove handler names to shink struct
Shrink ThreadVars by removing the queue handler names that are only
used at shutdown. Since this is not performance critical, we can use
the id's to look up the queue handler.
5 years ago
Victor Julien 74a6f8d4dd threading/queues: add way to lookup by ID
In preparation of doing runtime operations by ID instead of by name,
add functions to look up by ID and to convert name to ID.
5 years ago
Victor Julien d0218696ba threading: shrink threadvars struct size 5 years ago
Victor Julien c029599515 threading: remove unused threadvars field 5 years ago
Victor Julien f1ee176111 threading: clarify threadvars fields 5 years ago
Victor Julien d50492cb20 threading: cleanup packet thread shutdown loop 5 years ago
Victor Julien 8e762f5190 source-pcap: remove unused function 5 years ago
Victor Julien 3a703c84ad threading/modules: declare prototypes static
Declare registered threading API funcs static where appropriate.
5 years ago
Victor Julien 7c83cb585e sources: fix pipeline failure handling
When TmThreadsSlotProcessPkt fails it will return the packet that was
passed to it to the packetpool.

Some of the packet sources were doing this manually as well. This patch
fixes those sources.
5 years ago
Victor Julien 49599dfe89 threading: use tm_flowworker for pseudo packets
Pseudo packets don't need to be processed by the decoding layer.
5 years ago
Victor Julien 9df8e1c984 threading: add shortcut to flowworker 5 years ago
Victor Julien 02004fa547 threading: remove per slot post_pq
Use a single packet queue per thread for flow timeout packet
injection. The per slot queue was unused except for this use
case. Having a single queue makes the logic and implementation
simpler.

In case of 'autofp', the per thread packet queue will actually
use the threads input queue. For workers/single a dedicated
queue will be set up.

Rename TmThreadsSlotHandlePostPQs to TmThreadsHandleInjectedPackets
to reflect the changed logic.
5 years ago
Victor Julien 15e3bdb7b8 af-packet: prototypes cleanup
Remove unused prototype.

Declare other prototypes static.
5 years ago
Victor Julien 44d7f636f2 threading: remove post_pq argument from 'SlotFunc'
This was not in use anywhere.
5 years ago
Victor Julien f5045af3e3 runmodes: code cleanups 5 years ago
Victor Julien 1a8562b3c6 detect: clean up threads handling
Clean up reload and break loop thread handling.
5 years ago
Victor Julien e5010d7704 detect: inject packet cleanup 5 years ago
Victor Julien abea227cfc flow-manager: code cleanups 5 years ago
Victor Julien 6fd35fb786 flow-manager: avoid doubly signaling threads
Don't try to wake up the threads we just flagged and validated that
they changed their state.
5 years ago
Victor Julien 539c1a275f threading: remove commented out function 5 years ago
Victor Julien b55f617c2f threading: optimize and unify post_pq checks
TmThreadsSlotProcessPkt did not need to look all 'slots' as only the first
slots post_pq can have been used.

Unify post_pq cleanup handling.
5 years ago
Victor Julien 2a1ed3ba1b threading: remove wrong unlikely statement; minor cleanups 5 years ago
Victor Julien e5192ae20a threading: TmSlot::SlotFunc does not need to be atomic 5 years ago
Victor Julien 89048d71ad threading: fix flags handling by using uint32_t everywhere 5 years ago
Victor Julien 85cf341189 threading: optimize error handling in main packet loop 5 years ago
Victor Julien 9d8ea3b4fe threading: minor code style cleanups 5 years ago
Victor Julien b1056b3836 threading: simplify packetpool checks 5 years ago
Victor Julien 603b2ced47 threading/queues: add shortcut for packetpool check
Allows code simplifycation in the threading loops.
5 years ago
Victor Julien 29cb9d1d52 threading/queues: minor code cleanups 5 years ago
Victor Julien f05c12b70f afl: fix compilation 5 years ago
Victor Julien 01862eae78 afl/decoder: make file dumps optional 5 years ago
Victor Julien 42d112e7b6 detect/address: dead code removal and style cleanups 5 years ago
Victor Julien d41aeccea4 smb: handle file transactions post-GAP
After a GAP all normal transactions are closed. File transactions
are left open as they can handle GAPs in principle. However, the
GAP might have contained the closing of a file and therefore it
may remain active until the end of the flow.

This patch introduces a time based heuristic for these transactions.
After the GAP all file transactions are stamped with the current
timestamp. If 60 seconds later a file has seen no update, its marked
as closed.

This is meant to fix resource starvation issues observed in long
running SMB sessions where packet loss was causing GAPs.
5 years ago
Jason Ish 80cafb2979 flow: expose last time as a function
This function returns the individual components
of the timeval in output pointers making it suitable
for use over Rust FFI.
5 years ago