Configuration option `threads: auto` in DPDK's interface node
overassigns available threads to the interface.
Commit 4dfd44d3 changed the signedness of the remaining threads counter,
which caused surpass of the counter initialization.
The if-clause is switched to first initialize and then use the counter.
Ticket: 7798
Using the new configuration format, it is now possible to set CPU affinity
settings per interface.
The threading.autopin option has been added to automatically use CPUs from the
same NUMA node as the interface. The autopin option requires
hwloc-devel / hwloc-dev to be installed and --enable-hwloc flag in configure
script.
Ticket: 7036
util-device.h exposes some details that are particularly problematic
for C++, even when wrapped in 'extern "C"'. To address this, break the
header into public and private parts. The public part exposes
LiveDevice as an opaque data structure, while the private header has
the actual definition.
The idea is that only Suricata C source files should include the
private header, it should not be re-included in any other header
file. And this is the header library users should use, however we
don't enforce it with tecnical means, a library user could still
include the private header, but the clue there is in the name.
When running in non-forwarding (IDS) mode, it is not required
to create TX queues for the interface.
This can be acheived by setting tx-descriptors configuration
field to 0.
Ticket: 7633
ICE card (Intel E810) was not receiving packets immediatelly
after startup, Suricata workers would act as processing while
it was not. This eliminates the problem by only continuing
in the initialization if the link is already up.
The setting can be turned off manually from the configuraiton
file.
Ticket: 7381
It turned out that having global (interface-specific) mempool
that is shared by the threads of the interface is slower than
having individual mempools per queue for each interface.
The commit brings this change and should be user-invisible,
the config setting remains still as a number of objects of
all mempools summed (of that interface).
Ticket: 7382
Move and adjust the base of RSS configuration from util-dpdk-i40e.c to
a new file that can be later utilized by other cards.
RSS configuration can be configured via rte_flow rules. This is useful
for possible future features such as specific header offload
(vxlan, nvgre) also implemented via rte_flow rules, as rte_flow
rules can be chained via groups and priorities.
i40e uses multiple different rte_flow rules to setup RSS. At first,
function DeviceSetRSSFlowQueues() is used to setup rx queues.
This rule matches all types of traffic, so the equivalent
to dpdk-testpmd pattern would be "pattern end"
This rule can not contain hash types (ipv4, ipv6 etc.) nor hash key.
The hash function used here is RTE_ETH_HASH_FUNCTION_DEFAULT.
The syntax in dpdk-testpmd for this rule with attributes:
port index == 0
used rx queue indices == 0 1 2 3
is as follows:
"flow create 0 ingress pattern end actions rss queues 0 1 2 3 end
func default / end"
The other rules configured by i40eDeviceSetRSSFlowIPv4() and
i40eDeviceSetRSSFlowIPv6() match specific type of traffic by l4 protocol
(none, TCP, UDP, SCTP). For example, pattern to match l3 ipv4 with l4
tcp traffic in dpdk-testpmd syntax would be equivalent of
"pattern eth / ipv4 / tcp / end".
These rules can not have rx queues configured, but have hash types
(l3 src and dst address). This means that the traffic distribution
is affected only by l3 addresses, independent of the l4 specifics.
Also these pattern matching rules have symmetric 6d5a
hash key configured. The length of the key is dependent on DPDK version.
The hash function (either RTE_ETH_HASH_FUNCTION_SYMMETRIC_TOEPLITZ or
RTE_ETH_HASH_FUNCTION_TOEPLITZ, depending on DPKD version) used
in these rules hashes symmetricaly due to the symmetric hash key.
The syntax in dpdk-testpmd for rule to match ipv4-tcp traffic with
attributes:
port index == 0
<hash_key> == 52 bytes long 6d5a symmetric hash key
is as follows:
"flow create 0 ingress pattern eth / ipv4 / tcp / end actions rss types
ipv4-tcp l3-src-only l3-dst-only end queues end key <hash_key>
key_len 52 func toeplitz / end"
(queues need to be set to NULL)
Ticket: 7337
Getting clock through Time Stamp Counter (TSC) can be precise and fast,
however only for a short duration of time.
The implementation across CPUs seems to vary. The original idea is to
increment the counter with every tick. Then dividing the delta of CPU ticks
by the CPU frequency can return the time that passed.
However, the CPU clock/frequency can change over time, resulting in uneven
incrementation of TSC. On some CPUs this is handled by extra logic.
As a result, obtaining time through this method might drift from the real
time.
This commit therefore substitues TSC time retrieval by the standard system
call wrapped in GetTime function - on Linux it is gettimeofday.
Ticket: 7115
When Suricata was running in IPS mode and received a signal to stop,
the first worker of every interface/port stopped the port and
proactively stopped the peered interface as well.
This was done to be as accurate with port stats as possible.
However, in a highly active scenarios (lots of packets moving around)
the peered workers might still be in the process of a packet
release operation. These workers would then attempt to transmit
on a stopped interface - resulting in an errorneous operation.
Instead, this patch proposes a worker synchronization of the given
port. After these workers are synchronized, it is known that no packets
will be sent of the peered interface, therefore the first worker can
stop it. This however cannot be assumed about "its own" port as the
peered workers can still try to send the packets. Therefore, ports
are only stopped by the peered workers.
Ticket: #6790
The function to retrieve port ID from the port name was used multiple times.
This commit removes the redundant usage of the function.
Additionally, in the DeviceConfigureIPS(), the socket ID was wrongly retrieved
for the original interface and not for the out port interface.
When the packet load is low, Suricata can run in interrupt
mode. This more resembles the classic approach of processing
packets - CPU cores run low and only fetch packets
on interrupt.
Ticket: #5839
If a user doesn't allocate/allocates too little hugepages,
Suricata fails to start and outputs a hint to increase
number of hugepages (if enabled).
Ticket: #5966
Multi-tenancy uses loader threads that initialize detection engines. During
this, esp the AC family of MPM implementations, there is significant stack
usage. In most OS' threads have a lower stack size by default. In Linux, when
using the Musl C library, a thread by default gets 128KiB.
This patch does 2 things:
1. it centralizes the handling of the `threading.stack-size`. It it is not
longer handled by the runmodes, but called from the global initialization
logic.
2. it sets a minimum per thread stack size of 512k, unless `threading.stack-size`
is set.
Ticket: #6265.
DPDK apps can specify multiple arguments of the same
type. YAML format only allows unique keys within a single
node. This commit adds support for multiple EAL arguments
of the same type to be used within suricata.yaml.
Ticket: #5964
Lukas Sismis
Jason Ish
Adam Kiripolsky
Victor Julien
liaozhiyuan