Commit Graph

7730 Commits (2e8678a5ffa740434deb8880de1efb7e59ea0cfb)

Author SHA1 Message Date
Andreas Herz 2e8678a5ff docs: replace redmine links and enforce https on oisf urls 8 years ago
Jason Ish c411519605 app-layer: remove has events callback - not used 8 years ago
Jason Ish 23ceb2cc26 dnp3: regenerate object decoding code 8 years ago
Victor Julien 261f15a146 der: fix recursion depth not being handled correctly
In a mix of sequences the 'depth reached' error would not
be fully propagated.

Found with AFL.
8 years ago
Victor Julien 7ac041b872 der: warn if null passed to decoders
Remove null checks for errcode.
8 years ago
Eric Leblond 356440b380 tm-threads: fix build warning in afl mode 8 years ago
Victor Julien 968ed663a8 detect: fix out of bounds write in detect thread space creation 8 years ago
David DIALLO 6c643d8975 modbus: duplicate alerts unaware of direction
Remove DetectAppLayerInspectEngineRegister for TOCLIENT direction
because Modbus inspection engine is only performing in request (TOSERVER).

Detect Value keyword in read access rule. In read access, match on value
is not possible.

Update Modbus keyword documentation.
8 years ago
Jason Ish 8b74ac6ba0 conf/yaml: don't allow empty key values
When loading an empty file, libyaml will fire a single scalar
event causing us to create a key that contains an empty string.
We're not interested in this, so skip an empty scalar value
when expecting a key.

Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2418
8 years ago
Jason Ish 013c4e0aa0 metadata: fix parsing when not k/v
Allows for parsing metadata with mixed single word and key/val
pairs.
8 years ago
Victor Julien 7548944b49 app-layer: remove unused HasTxDetectState call
Also remove the now useless 'state' argument from the SetTxDetectState
calls. For those app-layer parsers that use a state == tx approach,
the state pointer is passed as tx.

Update app-layer parsers to remove the unused call and update the
modified call.
8 years ago
Victor Julien e96d9c1159 app-layer: add tx iterator API
Until now, the transaction space is assumed to be terse. Transactions
are handled sequentially so the difference between the lowest and highest
active tx id's is small. For this reason the logic of walking every id
between the 'minimum' and max id made sense. The space might look like:

    [..........TTTT]

Here the looping starts at the first T and loops 4 times.

This assumption isn't a great fit though. A protocol like NFS has 2 types
of transactions. Long running file transfer transactions and short lived
request/reply pairs are causing the id space to be sparse. This leads to
a lot of unnecessary looping in various parts of the engine, but most
prominently: detection, tx house keeping and tx logging.

    [.T..T...TTTT.T]

Here the looping starts at the first T and loops for every spot, even
those where no tx exists anymore.

Cases have been observed where the lowest tx id was 2 and the highest
was 50k. This lead to a lot of unnecessary looping.

This patch add an alternative approach. It allows a protocol to register
an iterator function, that simply returns the next transaction until
all transactions are returned. To do this it uses a bit of state the
caller must keep.

The registration is optional. If no iterator is registered the old
behaviour will be used.
8 years ago
Eric Leblond 6f69c65c48 util-ebpf: rename local variable 8 years ago
Eric Leblond 194751654f af-packet: count only CPUs once 8 years ago
Eric Leblond 276b93fb53 unix-socket: add ebpf-bypassed-stats command
This command output the count of element in IPv4 and IPv6 flow
table of interfaces using eBPF/XDP bypass.
8 years ago
Eric Leblond 0356293d84 unix-socket: add bypassed counter to iface-stat 8 years ago
Eric Leblond 93f5b5f1e8 util-ebpf: add bypassed counters
Use LiveDevice bypassed counter and also add hash size counters
for the v4 and v6 flow table.
8 years ago
Eric Leblond f04391031f af-packet: XDP bypass in IPS mode
Implement XDP bypass in IPS mode by using XDP redirect to send
packets from bypassed flow directly to the transmission interface.
8 years ago
Eric Leblond 7bec54158f flow-bypass: introduce update function
Main objective of the function is to be able to bypass a flow on
other interfaces. This is necessary in AF_PACKET case as the flow
table are per interface.
8 years ago
Eric Leblond 2b56b02569 util-ebpf: simplify code cleaning
Avoid to use an unnecessary callback strategy as the purpose of the
function using the callback is hardcoded.
8 years ago
Eric Leblond 94a622cb55 af-packet: add comments to eBPF/XDP code 8 years ago
Eric Leblond 3379311e66 af-packet: fix error handling in bypass case
If the key is already in the hash table then the bypass is
succesful.
8 years ago
Eric Leblond e1515b48ad util-ebpf: fix libbpf error handling 8 years ago
Eric Leblond 4f57008a23 af-packet: add support for XDP cpu redirect map
This patch adds a boolean option "xdp-cpu-redirect" to af-packet
interface configuration. If set, then the XDP filter will load
balance the skb creation on specified CPUs instead of doing the
creation on the CPU handling the packet. In the case of a card
with asymetric hashing this will allow to avoid saturating the
single CPU handling the trafic.

The XDP filter must contains a set of map allowing load balancing.
This is the case of xdp_filter.bpf.

Fixed-by: Jesper Dangaard Brouer <netoptimizer@brouer.com>
8 years ago
Eric Leblond ac5957d461 util-affinity: export CPU set parsing function 8 years ago
Eric Leblond 2598078ee6 af-packet: code cleaning and comments 8 years ago
Eric Leblond cde438f670 af-packet: add missing copyright header
And also fixes the copyright date in some files.
8 years ago
Eric Leblond 3f3a206722 util-ebpf: add error handling in hash value fetch 8 years ago
Eric Leblond 1e729f059f af-packet: improve xdp error handling
Don't try to bypass the flow if the flow table is unknown.

Also continue after error message if ever XDP was not correctly
setup.
8 years ago
Eric Leblond 126488f74d af-packet: add support for multi iface bypass 8 years ago
Eric Leblond 4474889667 util-device: change logic of registration
Device storage requires the devices to be created after storage
is finalized so we need to first get the list of devices then
create them when the storage is finalized.

This patch introduces the LiveDeviceName structure that is a list
of device name used during registration.

Code uses LiveRegisterDeviceName for pre registration and keep
using the LiveRegisterDevice function for part of the code that
create the interface during the runmode creation.
8 years ago
Eric Leblond 0998f37b78 util-device: add an iteration function 8 years ago
Eric Leblond 3ae714d354 device-storage: introduce feature
The capture method may have to store data depending related to the
offloading so having a per interface storage via LiveDevice seems
interesting.
8 years ago
Eric Leblond e2d31e1c57 flow-bypass: fix sleep strategy 8 years ago
Eric Leblond 7a0d53448d tm-threads: fix build warning in afl mode 8 years ago
Eric Leblond 9cb591aa6c af-packet: remove done fixme in XDP 8 years ago
Eric Leblond eff10fce10 af-packet: end of map factoring 8 years ago
Eric Leblond d65f45856c af-packet: cache map fd search 8 years ago
Eric Leblond 17a32bdaa0 af-packet: fix bypassing of IPv6
Also misc fixes.
8 years ago
Eric Leblond b937e1afef util-ebpf: fix ipv6 cleaning and add comments 8 years ago
Eric Leblond 60752d231c util-ebpf: fix XDP delete key
The key was deleted twice so let's remove the local deletion.
8 years ago
Eric Leblond 08eec0833e flow-bypass: add abstraction layer
The flow bypass thread can now be used by any capture method that
register it timeout check function.
8 years ago
Eric Leblond 43ecf0d78d util-ebpf: add call to remove memlock limit
Without that, user has to use ulimit to be able to load the eBPF
file.
8 years ago
Eric Leblond 0654c31397 util-ebpf: suppress call on loop init 8 years ago
Eric Leblond 8c88087948 af-packet: implementation of XDP bypass
This patch adds support for XDP bypass. It provides an XDP
filter that can be loaded to realize the bypass of flows.
8 years ago
Eric Leblond 31c947b4d8 af-packet: use per CPU hash in bypass
eBPF has a data type which is a per CPU array. By adding one element
to the array it is in fact added to all per CPU arrays in the kernel.
This allows to have a lockless structure in the kernel even when doing
counter update.

In userspace, we need to update the flow bypass code to fetch all
elements of the per CPU arrays.
8 years ago
Eric Leblond 8640cc5dcf flow-bypass: only start thread on demand 8 years ago
Eric Leblond 06173267c6 af-packet: kernel bypass implementation
This patch implements bypass capability for af-packet.

The filter only bypass TCP and UDP in IPv4 and IPv6. It don't
don't bypass IPv6 with extended headers.

This patch also introduces a bypassed flow manager that takes
care of timeouting the bypassed flows. It uses a 60 sec
timeout on flow. As they are supposed to be active we can
try that. If they are not active then we don't care to get them
back in Suricata.
8 years ago
Eric Leblond 91e1256b01 af-packet: add support for eBPF cluster and filter
This patch introduces the ebpf cluster mode. This mode is using
an extended BPF function that is loaded into the kernel and
provide the load balancing.

An example of cluster function is provided in the ebpf
subdirectory and provide ippair load balancing function.
This is a function which uses the same method as
the one used in autofp ippair to provide a symetrical
load balancing based on IP addresses.

A simple filter example allowing to drop IPv6 is added to the
source.

This patch also prepares the infrastructure to be able to load
and use map inside eBPF files. This will be used later for flow
bypass.
8 years ago
Giuseppe Longo 884e051671 detect-engine-hsbd: decompress swf files
This checks if a buffer is a swf file and try
to decompress it, if decompression is enabled.
8 years ago