Arguments can be specified in either long or short forms. When specifying the arguments, the dashes are omitted.
that are usually passed through command line. These arguments are used to
Among other settings, this configuration node is able to configure available NICs to Suricata, memory settings or other
initialize and configure EAL. Arguments can be specified in either long or
parameters related to EAL.
short forms. When specifying the arguments, the dashes are omitted.
Among other settings, this configuration node is able to configure available
The node `dpdk.interfaces` wraps a list of interface configurations. Items of the list follows the structure that can
NICs to Suricata, memory settings or other parameters related to EAL.
be found in other capture interfaces. The individual items contain the usual configuration options
such as `threads`/`copy-mode`/`checksum-checks` settings. Other capture interfaces, such as AF_PACKET, rely on the user that NICs are appropriately configured.
The node `dpdk.interfaces` wraps a list of interface configurations. Items of
Configuration through kernel does not apply to applications running under DPDK. The application is solely responsible for the
the list follows the structure that can be found in other capture interfaces.
initialization of NICs it is using. So, before the start of Suricata, NICs that Suricata uses, must undergo the process of initialization.
The individual items contain the usual configuration options
As a result, there are extra configuration options (how NICs can be configured) in the items (interfaces) of the `dpdk.interfaces` list.
such as `threads`/`copy-mode`/`checksum-checks` settings. Other capture
At the start of the configuration process, all NIC offloads are disabled to prevent any packet modification.
interfaces, such as AF_PACKET, rely on the user that NICs are appropriately
According to the configuration, checksum validation offload can be enabled to drop invalid packets.
configured.
Other offloads can not be currently enabled.
Configuration through kernel does not apply to applications running under DPDK.
Additionally, the list items of `dpdk.interfaces` contains DPDK specific settings such as `mempool-size` or `rx-descriptors`.
The application is solely responsible for the initialization of NICs it is
These settings adjust individual parameters of EAL. One of the entries of the `dpdk.interfaces` is the `default` interface.
using. So, before the start of Suricata, NICs that Suricata uses, must undergo
When loading interface configuration and some entry is missing, the corresponding value of the `default` interface is used.
the process of initialization.
As a result, there are extra extra configuration options (how NICs can be
The worker threads must be assigned to a specific cores. The configuration module `threading` can be used to set threads affinity.
configured) in the items (interfaces) of the `dpdk.interfaces` list.
Worker threads can be pinned to cores in the array configured in `threading.cpu-affinity["worker-cpu-set"]`.
At the start of the configuration process, all NIC offloads are disabled to
Performance-oriented setups have everything (the NIC, memory and CPU cores interacting with the NIC) based on one NUMA node.
prevent any packet modification. According to the configuration, checksum
It is therefore required to know layout of the server architecture to get the best results.
validation offload can be enabled to drop invalid packets. Other offloads can
The CPU core ids and NUMA locations can be determined for example from the output of `/proc/cpuinfo` where `physical id` described the NUMA number.
not be currently enabled.
The NUMA node to which the NIC is connected to can be determined from the file `/sys/class/net/<KERNEL NAME OF THE NIC>/device/numa_node`.
Additionally, the list items of `dpdk.interfaces` contains DPDK specific
settings such as `mempool-size` or `rx-descriptors`. These settings adjust
individual parameters of EAL. One of the entries of the `dpdk.interfaces` is
the `default` interface. When loading interface configuration and some entry is
missing, the corresponding value of the `default` interface is used.
The worker threads must be assigned to a specific cores. The configuration
module `threading` can be used to set threads affinity.
Worker threads can be pinned to cores in the array configured in
`threading.cpu-affinity["worker-cpu-set"]`. Performance-oriented setups have
everything (the NIC, memory and CPU cores interacting with the NIC) based on
one NUMA node.
It is therefore required to know layout of the server architecture to get the
best results. The CPU core ids and NUMA locations can be determined for example
from the output of `/proc/cpuinfo` where `physical id` described the NUMA
number. The NUMA node to which the NIC is connected to can be determined from
the file `/sys/class/net/<KERNEL NAME OF THE NIC>/device/numa_node`.
::
::
@ -2012,18 +2032,26 @@ The NUMA node to which the NIC is connected to can be determined from the file `
## cat /sys/class/net/<KERNEL NAME OF THE NIC>/device/numa_node e.g.
## cat /sys/class/net/<KERNEL NAME OF THE NIC>/device/numa_node e.g.
cat /sys/class/net/eth1/device/numa_node
cat /sys/class/net/eth1/device/numa_node
If Suricata has enabled at least 2 (or more) workers, the incoming traffic is load balanced across the worker threads
If Suricata has enabled at least 2 (or more) workers, the incoming traffic is
by Receive Side Scaling (RSS). Internally, DPDK runmode uses
load balanced across the worker threads by Receive Side Scaling (RSS).
a `symmetric hash (0x6d5a) <https://www.ran-lifshitz.com/2014/08/28/symmetric-rss-receive-side-scaling/>`_
Internally, DPDK runmode uses a `symmetric hash (0x6d5a)
Before Suricata can be run, it is required to allocate sufficient number of hugepages. Suricata allocates continuous block of memory.
Before Suricata can be run, it is required to allocate sufficient number of
For efficiency, CPU allocates memory in RAM in chunks. These chunks are usually in size of 4096 bytes. DPDK and other memory intensive applications makes use of hugepages.
hugepages. Suricata allocates continuous block of memory.
Hugepages start at the size of 2MB but they can be as large as 1GB. Lower count of pages (memory chunks) allows faster lookup of page entries.
For efficiency, CPU allocates memory in RAM in chunks. These chunks are usually
The hugepages need to be allocated on the NUMA node where the NIC and CPU resides.
in size of 4096 bytes. DPDK and other memory intensive applications makes use
Otherwise, if the hugepages are allocated only on NUMA node 0 and the NIC is connected to NUMA node 1, then the application will fail to start.
of hugepages. Hugepages start at the size of 2MB but they can be as large as
Therefore, it is recommended to first find out to which NUMA node the NIC is connected to and only then allocate hugepages and set CPU cores affinity to the given NUMA node.
1GB. Lower count of pages (memory chunks) allows faster lookup of page entries.
If the Suricata deployment is using multiple NICs on different NUMA nodes then hugepages must be allocated on all of those NUMA nodes.
The hugepages need to be allocated on the NUMA node where the NIC and CPU
resides. Otherwise, if the hugepages are allocated only on NUMA node 0 and
the NIC is connected to NUMA node 1, then the application will fail to start.
Therefore, it is recommended to first find out to which NUMA node the NIC is
connected to and only then allocate hugepages and set CPU cores affinity to
the given NUMA node. If the Suricata deployment is using multiple NICs on
different NUMA nodes then hugepages must be allocated on all of those NUMA
nodes.
::
::
@ -2031,38 +2059,45 @@ If the Suricata deployment is using multiple NICs on different NUMA nodes then h
grep Huge /proc/meminfo
grep Huge /proc/meminfo
## Allocate hugepages on NUMA node 0:
## Allocate hugepages on NUMA node 0:
echo 8192 | sudo tee /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
Lukas Sismis
3 years ago
committed by
Victor Julien