Reduce per tx space for tracking detection/prefilter progress. Instead
of a per direction u64 of flags, where each bit reflected a progress
value, use a simple u8 to track the linear progression through the
progress values. Use an offset to allow 0 to mean no value.
Add flags field as well to track "skip detect" and "inspect complete".
Cache Hyperscan serialized databases to disk to prevent compilation
of the same databases when Suricata is run again with the same
ruleset.
Hyperscan binary files are stored per rulegroup in the designated
folder, by default in the cached library folder.
Since caching is per signature group heads,
some chunk of the ruleset can change and it still can reuse part of
the unchanged signature groups.
Loading *fresh* ET Open ruleset: 19 seconds
Loading *cached* ET Open ruleset: 07 seconds
Ticket: 7170
This variant of hashlittle2() ensures that it avoids
accesses beyond the last byte of the string, which will
cause warnings from tools like Valgrind or Address
Sanitizer.
The buffer-size value that controls file output buffering defaults to
8k. To be consistent with previous logic, the default is being changed
to 0 (e.g., needed if there are old config files that don't specifically
enable the new value).
such as ja4.
Why ?
We do not want to see hard-coded protocol constants such as
ALPROTO_QUIC directly used in generic code in detect-parse.c
How ?
From the keyword point of view, this commit adds the function
DetectSignatureSetMultiAppProto which is similar to
DetectSignatureSetAppProto but takes multiple alprotos.
It restricts the signature alprotos to a set of possible alprotos
and errors out if the interstion gets empty.
The data structure SignatureInitData gets extended with
a fixed-length array, as the use case is a sparse number of protocols
Ticket: 7304
There was an implicit limit of 32 app-layer protocols
used by probing parsers through a mask, meaning that
Suricata should not support more than 32 app-layer protocols
in total.
This limit is relaxed to each flow not being able to
run more than 32 probing parsers, meaning that for each source
and destination port combination, the sum of registered
probing parsers should not exceed 32, even if there are more
than 32 in total.
Also sets probing parsers done sooner in the case the other
side of the connection was detected first.
Ticket: 7437
Methods:
`get` creates the flow object.
`id` returns the flow id.
`has_alerts` returns a boolean indicating if the flow triggered alerts.
`app_layer_proto` returns various app-layer related fields as 5 strings:
alproto, alproto_ts, alproto_tc, alproto_orig, alproto_expect.
`stats` returns cnts for bytes and packets to sever and to client, as 4
numbers.
`tuple` -- returns various fields: srcip, dstip, proto, sp, dp.
`timestamps` returns time as 4 numbers: seconds and microseconds, for
first and last packet of the flow.
`timestring_legacy` returns the first packet from the flow's timestring
as a string (like fastlog).
`timestring_iso8601` returns the first packet from the flow's
timestring as an iso8601 compat timestring (like eve).
Example:
```
name = "lua-scflowstats.log"
local flow = require("suricata.flow")
function init(args)
local needs = {}
needs["type"] = "flow"
return needs
end
function setup(args)
filename = SCLogPath() .. "/" .. name
file = assert(io.open(filename, "a"))
SCLogInfo("lua SCFlowStats Log Filename " .. filename)
end
function log(args)
local f = flow.get()
timestring = f:timestring_legacy()
tscnt, tsbytes, tccnt, tcbytes = f:stats()
file:write ("[**] " .. timestring .. "\nSCFlowStats is\nPacket count to server: " .. tscnt .. "\nByte count to server: " .. tsbytes .. "\nPacket count to client: " .. tccnt .. "\nByte count to client: " .. tcbytes .. "\n[**]")
file:flush()
end
function deinit(args)
file:close(file)
end
```
Task #7489
A DNP3 message can be a request or response, but not both. This is how
the transaction is structured.
Instead of having 2 values, "has_request" and "has_response", just
provide one field, "is_request" as a boolean.
Ticket: #7601
This is an initial 1:1 conversion which is rather simple, as DNP3 only
had one function which converted the whole transaction to a DNP3
table.
Ticket: #7601
In addition to
function init (args)
local needs = {}
needs["dnp3"] = tostring(true)
return needs
end
allow for
function init (args)
return {dnp3=true}
end
with the idea that the former will be completely removed for 8.0.
This works with all existing SV tests and as a string value, any value
is considered "truthy".
Just return NULL if tx->mime_state is NULL or if SCDetectMimeEmailGetData return 0
Fixes:
09db7c7 ("detect: add mime email.subject keyword")
90aab0d ("detect: add email.from")
Alfredo Cardigliano
Alice Akaki
Victor Julien
Lukas Sismis
Philippe Antoine
Shivani Bhardwaj
Jeff Lucovsky
Philippe Antoine
Juliana Fajardini
Jason Ish