Commit Graph

15638 Commits (3f5d228b9e244f536b7a647e14ff89bb0cef27a1)
 

Author SHA1 Message Date
Philippe Antoine 632e52ca2b ci: update ubuntu22.04 builds with clang14+asan
using a workround about ASLR
2 years ago
Lukas Sismis 16c88f2db7 dpdk: fix typo in the struct member name 2 years ago
Lukas Sismis 5592ec079d dpdk: refactor the main packet loop into smaller functions 2 years ago
Lukas Sismis 0b5966c347 dpdk: only close the port when workers are synchronized
When Suricata was running in IPS mode and received a signal to stop,
the first worker of every interface/port stopped the port and
proactively stopped the peered interface as well.
This was done to be as accurate with port stats as possible.
However, in a highly active scenarios (lots of packets moving around)
the peered workers might still be in the process of a packet
release operation. These workers would then attempt to transmit
on a stopped interface - resulting in an errorneous operation.

Instead, this patch proposes a worker synchronization of the given
port. After these workers are synchronized, it is known that no packets
will be sent of the peered interface, therefore the first worker can
stop it. This however cannot be assumed about "its own" port as the
peered workers can still try to send the packets. Therefore, ports
are only stopped by the peered workers.

Ticket: #6790
2 years ago
Philippe Antoine 45bb936187 http: event on request line missing protocol
Ticket: 6856
2 years ago
Jason Ish 8284df3ed4 devguide: add an upgrade section
Add an upgrade section to the devguide. This should cover any changes
to APIs that users might be using from plugins or as a library user.
2 years ago
Jason Ish b7b16fb481 eve/filetype: reorder fields to match lifecycle
Enhances readability.
2 years ago
Jason Ish a3354e55e6 eve/filetypes: use more const 2 years ago
Jason Ish eee9757dba eve/filetype: ThreadDeinit can return void
Change ThreadDeinit to return void instead of an int, there is nothing
to be done on success or failure.
2 years ago
Jason Ish cd85d89f14 output-eve: doxygen docs for SCEveFileType
Add documentation for the SCEveFileType in Doxygen format.
2 years ago
Jason Ish 500d29f356 doxygen: document the examples directory 2 years ago
Jason Ish bd55cd4c55 eve/filetypes: common init for threaded and non-threaded
In 7.0 if EVE was non-threaded, the ThreadInit for the filetype was
not called meaning that the filetype author had to handle the threaded
and non-threaded cases.

To simplify this, if non-threaded, still call ThreadInit (and
ThreadDeinit) once with a thread_id of 0. This should simplify
authoring EVE filetype plugins.
2 years ago
Jason Ish 3ff72d3efa eve: rename plugin to filetypes
EVE filetypes are not always plugins, for example, null and syslog
that are built-in filetypes.
2 years ago
Jason Ish 7c8c9fff32 plugins: remove conf.h from suricata-plugin.h
Remove "conf.h" from suricata-plugin.h as its not needed by that
header. However, some other files became transitively dependent on
through other includes, so fix those up.
2 years ago
Jason Ish ead09c2497 eve/filetypes: remove from plugin context
Remove EVE filetypes from plugin context as they are not only used
from plugins. Plugins allow user code to register filetypes, but we
also have internal file types that use this api including the null
output and syslog.  Additionally library users can use this API to
register filetypes, and they are not plugins.

Ideally this code would go in "output-json.[ch]" as the "primary" eve
API, however there are currently some include circular include issues
there, so start new cleaned up EVE API in "output-eve.[ch]" which is
"clean" with respect to includes, and as we cleanup existing EVE API for
"public" use, it can be moved here.

Ticket: #6838
2 years ago
Jeff Lucovsky 3bf92bb14f example/plugin: Use ThreadId 2 years ago
Jeff Lucovsky 85d321a689 output/plugin: Use Suri thread-id for plugins
Issue: 6408

Use the Suricata thread id for plugin thread initialization to give the
plugin a better correlating factor to the actual Suricata threads.
2 years ago
Victor Julien 1122fa24b6 mpm/ac: minor test cleanups 2 years ago
Victor Julien d47bfbb14d mpm/ac: add endswith test 2 years ago
Victor Julien c312d670d4 mpm/ac: implement endswith
When a pattern is using endswith, only consider it a match when it
is the end of the data.

Ticket: #6852.
2 years ago
Victor Julien b499239ef5 mpm: register algo features
This is so patterns can reply on mpm match meaning a full match.

Not yet used.
2 years ago
Victor Julien dcfa59bb69 mpm/ac: tidy up main search loop 2 years ago
Victor Julien 4fd0745b78 detect: minor cleanup 2 years ago
Jason Ish ece2029b0b requirements.txt: use suricata-update master 2 years ago
Victor Julien e5d3fad5f4 nfq: stricter thread sync
No longer update `Packet::flags` for tracking packet modifications,
as thread safety was not guaranteed.

Clearly separate between various kinds of `Packet::nfq_v` accesses for:
- mark
- mark_modified
- verdicted
These are either done under lock (Packet::persistent.tunnel_lock) or,
if the Packet is not part of a tunnel, not under lock.

This is safe as in all the related logic the Packet's tunnel state
is fixed and can no longer change.
2 years ago
Victor Julien d2f7e89d11 decode/tunnel: split verdict logic
Allows caller to take their own lock.
2 years ago
Victor Julien 6066c4d6e7 decode/tunnel: improve tunnel handling
Give each packet explicit tunnel type `ttype`: none, root, child.

Assigning happens when a (tunnel) packet is set up and is thread
safe.
2 years ago
Victor Julien 9bc42e3b34 decode/tunnel: move tunnel verdicted logic
In preparation of cleaning up thread safety, move "verdicted"
logic out of Packet::flags. Unsafe writes to "flags" can potentially
have side effects.
2 years ago
Victor Julien c31a6f5649 nfq: remove obsolete comment 2 years ago
Victor Julien 85ddeb6546 nfq: minor code cleanup 2 years ago
Victor Julien 792c87af25 nfq: use bool for verdicted packet var 2 years ago
Victor Julien af97316f42 defrag: match up v4 and v6 packet setup
v4 was doing redundant recursion level setup.

v6 was missing PKT_REBUILT_FRAGMENT flag.
2 years ago
Philippe Antoine 59d18b9a68 decode/pppoe: skip useless loop unless debug
Looping over tags has no effects, except debug logging
2 years ago
Philippe Antoine d2db8bfdca decode/pppoe: pointer cast consistency
Ticket: 6787

To do pointer arithmetic, we need to use uint8_t* pointer :
Pointer arithmetic in C is automatically scaled according
to the size of the data type.

Also simplifies the loop condition
2 years ago
Jason Ish 44388f1b69 src: make include guards more library friendly
Include guards for libraries should use a prefix that is meaningful for
the library to avoid conflicts with other user code. For Suricata, use
SURICATA.

Additionally, remove the pattern of leading and trailing underscores as
these are reserved for the language implementation per the C and C++
standards.
2 years ago
Shivani Bhardwaj 632ca75dd3 detect/port: handle single port that is range too
If a port point is single but later on also a part of a range, it ends
up only creating the port groups for single points and not the range.
Fix it by adding the port next to current single one to unique points
and marking it a range port.

Bug 6843
2 years ago
Hadiqa Alamdar Bukhari 3aa313d0c5 dns: add dns.rcode keyword
dns.rcode matches the rcode header field in DNS messages
It's an unsigned integer
valid ranges = [0-15]
Does not support prefilter
Supports matches in both flow directions

Task #6621
2 years ago
Juliana Fajardini 7b2bef1bc6 devguide: add chapter and short intro to libsuricata
With this, we intend to make more users aware of this use case, and that
we are working towards this.

Related to
Task #2693
2 years ago
Shivani Bhardwaj 2d6708f1ff util/interval-tree: fix coverity warning
Fix Coverity warning

** CID 1592992:  Incorrect expression  (COPY_PASTE_ERROR)
/src/util-port-interval-tree.c: 255 in SCPortIntervalFindOverlaps()

________________________________________________________________________________________________________
*** CID 1592992:  Incorrect expression  (COPY_PASTE_ERROR)
/src/util-port-interval-tree.c: 255 in SCPortIntervalFindOverlaps()
249                      * will be sorted, insert any new ports to the end of the list
250                      * and avoid walking the entire list */
251                     if (*list == NULL) {
252                         *list = new_port;
253                         (*list)->last = new_port;
254                     } else if (((*list)->last->port != new_port->port) &&
>>>     CID 1592992:  Incorrect expression  (COPY_PASTE_ERROR)
>>>     "port" in "(*list)->last->port2 != new_port->port" looks like a copy-paste error.
255                                ((*list)->last->port2 != new_port->port)) {
256                         DEBUG_VALIDATE_BUG_ON(new_port->port < (*list)->last->port);
257                         (*list)->last->next = new_port;
258                         new_port->prev = (*list)->last;
259                         (*list)->last = new_port;
260                     } else {

The code does not generate two port ranges that are same other than the
cases where port == port2 which is why it worked so far. Fix it.

Bug 6839
2 years ago
Jeff Lucovsky af529a56a9 netmap: Avoid HW errors when using pipes
Issue: 6837

When using netmap pipes (with lb, for example), avoid direct hardware
related IOCTLs that will fail (not supported with pipes).
2 years ago
Arne Welzel f9cf87a003 schema: Add stats.capture and in_iface properties
New suricata-verify test listens on loopback interface, resulting
in the capture and in_iface fields in the stats and event objects.
2 years ago
Arne Welzel f17204191d stats: Fix non-worker stats missing
Commit b8b8aa69b4 used tm_name of the
first StatsRecord of a thread block as key for the "threads" object.
However, depending on the type of thread, tm_name can be NULL and would
result in no entry being included for that thread at all. This caused
non-worker metrics to vanish from the "threads" object in the
dump-counters output.

This patch fixes this by remembering the first occurrence of a valid
tm_name within the per-thread block and adds another unittest to
cover this scenario.
2 years ago
Victor Julien 1d3a156179 rust: update parser dependencies
Time locked to 0.3.20 to guarantee MSRV of 1.63.
Update snmp-parser to 0.10.0.
Update asn1-rs to 0.6.1.
Update kerberos-parser to 0.8.0.
Update x509-parser 0.16.0.
Update der-parser to 9.0.0.
Remove specific use of der-parser 6.

Ticket: #6817.
Ticket: #6818.
2 years ago
Shivani Bhardwaj 4afaadc0ce detect/port: cleanup address artifacts
A lot of code uses variable names and comments derived from the code
about addresses, make them about port.
2 years ago
Shivani Bhardwaj e14fefe24a detect/port: remove SigGroupHead* ops
The functions in detect-engine-port.c are only being used at the time of
parsing the ports from rules initially. Since there are no SGHs at that
point, remove the ops related to them too.
2 years ago
Victor Julien c4ac6cdc9a detect: optimize sig_cnt setting
Utilize _popcnt64 where available.
2 years ago
Victor Julien 94b4619bb9 detect: optimize group head bitarray handling
During startup large rulesets use a lot of large bitarrays, that
are frequently merged (OR'd).

Optimize this using SSE2 _mm_or_si128.
2 years ago
Victor Julien 4ba1f44e0d detect: prepare for SIMD optimizations
Make rule group head bitarray 16 bytes aligned and padded to 16 bytes
boundaries to assist SIMD operations in follow up commits.
2 years ago
Victor Julien e7e4305d91 detect/port: use qsort instead of insert sort
Instead of using in place insertion sort on linked list based on two
keys, convert the linked list to an array, perform sorting on it using
qsort and convert it back to a linked list. This turns out to be much
faster.

Ticket #6795
2 years ago
Shivani Bhardwaj 643ae85b5f detect/port: merge port ranges for same signatures
To avoid getting multiple entries in the final port list and to also
make the next step more efficient by reducing the size of the items to
traverse over.

Ticket 6792
Bug 6414
2 years ago