We can loose packets during setup because we are reading nothing.
So it is logical to discard the counter at start of capture to
start from a clean state. This means we don't need to account the
drop at start. But the stats call that will reset the drop counts
will also return and reset the packets count. So we need to know
how many packets we really have. This is in fact the number of
packets coming from the stats call minus the number of discarded
packets and the drop count. All the other packets will have to be
read.
Update the code to use mmap capture by default even in unset in
configuration file. mmap capture is now be turned off by using
explicitely 'use-mmap: no' in configuration.
If TPACKET_V3 is not defined then it is not available and we should
not build anything related to tpacket_v3. This will allow us to
activate it dy default and fallback to v2 if not available.
It is used to set the block size in tpacket_v3. It will allow user
to tune the capture depending on his bandwidth.
Default block size value has been updated to a bigger value to
allow more efficient wlak on block.
Error handling was not done. The implementation is making the
choice to consider we must detroy the socket in case of parsing
error. The same was done for tpacket_v2.
Suppress useless fields in AFPThreadVars. This patch also get rid
of bytes counter as it was only used to display a message at exit.
Information on livedev and on packet counters are enough.
This patch adds a basic implementation of AF_PACKET tpacket v3. It
is basic in the way it is only working for 'workers' runnning mode.
If not in 'workers' mode there is a fallback to tpacket_v2. Feature
is activated via tpacket-v3 option in the af-packet section of
Suricata YAML.
This will handle minimal DetectEngineCtx structures (used in delayed
detect mode) safely, since they don't get SPM global contexts allocated.
Also added BUG_ON checks for valid spm_table entries.
Make the file storage use the streaming buffer API.
As the individual file chunks were not needed by themselves, this
approach uses a chunkless implementation.
Enforce inspect window also in IDS mode. Try always to get at least
'inspect win' worth of data. In case there is more new data, take
some of the old data as well to make sure there is always some overlap.
This unifies IDS and IPS modes, the only difference left is the start
of inspection. IDS waits until min_size is available, IPS starts right
away.
Convert HTTP body handling to use the Streaming Buffer API. This means
the HtpBodyChunks no longer maintain their own data segments, but
instead add their data to the StreamingBuffer instance in the HtpBody
structure.
In case the HtpBodyChunk needs to access it's data it can do so still
through the Streaming Buffer API.
Updates & simplifies the various users of the reassembled bodies:
multipart parsing and the detection engine.
Add a new API to store data from streaming sources, like HTTP body
processing or TCP data.
Currently most of the code uses a pattern of list of data chunks
(e.g. TcpSegment) that is reassembled into a large buffer on-demand.
The Streaming Buffer API changes the logic to store the data in
reassembled form from the start, with the segments/chunks pointing
to the reassembled data.
The main buffer storing the data slides forward, automatically or
manually. The *NoTrack calls allows for a segmentless mode of
operation.
This approach has two main advantages:
1. accessing the reassembled data is virtually cost-free
2. reduction of allocations and memory management
Now that the flow lookup is done in the worker threads the flow
queue handlers running after the capture thread(s) no longer have
access to the flow. This limits the options of how flow balancing
can be done.
This patch removes all code that is now useless. The only 2 methods
that still make sense are 'hash' and 'ippair'.
Initial version of the 'FlowWorker' thread module. This module
combines Flow handling, TCP handling, App layer handling and
Detection in a single module. It does all flow related processing
under a single flow lock.