Issue: 4082
Move the configuration file handling to Rust.
These changes will no longer terminate Suricata when there's an invalid
value for ftp.memcap. Like earlier Suricata releases, an error message
is logged "Invalid value <value> for ftp.memcap" but Suricata will no
longer terminate execution. It will use a default value of "0" instead.
ldap.responses.dn matches on LDAPDN from responses operations
This keyword maps the following eve fields:
ldap.responses[].search_result_entry.base_object
ldap.responses[].bind_response.matched_dn
ldap.responses[].search_result_done.matched_dn
ldap.responses[].modify_response.matched_dn
ldap.responses[].add_response.matched_dn
ldap.responses[].del_response.matched_dn
ldap.responses[].mod_dn_response.matched_dn
ldap.responses[].compare_response.matched_dn
ldap.responses[].extended_response.matched_dn
It is a sticky buffer
Supports prefiltering
Ticket: #7471
ldap.request.dn matches on LDAPDN from request operations
This keyword maps the following eve fields:
ldap.request.bind_request.name
ldap.request.add_request.entry
ldap.request.search_request.base_object
ldap.request.modify_request.object
ldap.request.del_request.dn
ldap.request.mod_dn_request.entry
ldap.request.compare_request.entry
It is a sticky buffer
Supports prefiltering
Ticket: #7471
If a Suricata inline IPS device is routing traffic over a
non-encrypted tunnel, like IPv6 tunnels, packets in a flow
will be dropped and not be matched. e.g.
The following example is a Suricata inline IPS with an IPv6 tunnel:
request: IPv4]ICMP] -> |IPS| -> IPv6]IPv4]ICMP]
reply: <- |IPS| <- IPv6]IPv4]ICMP]
Both the IPv4 request and IPv6 reply will be seen by Suricata on
ingress. The flows will not be matched due to flow recursion level.
Optionally use pkt recursion level in flow hash. Excluding recursion
level in flow hash allows matching of packet flows and defrag on an
inline IPS Suricata scenario where the IPS device is a tunnel
terminator.
Feature: 6260
ldap.responses.count matches on the number of LDAP responses
This keyword maps to the eve field len(ldap.responses[])
It is an unsigned 32-bit integer
Doesn't support prefiltering
Ticket: #7453
ldap.responses.operation matches on Lightweight Directory Access Protocol response operations
This keyword maps to the eve field ldap.responses[].operation
It is an unsigned 8-bit integer
Doesn't support prefiltering
Ticket: #7453
ldap.request.operation matches on Lightweight Directory Access Protocol request operations
This keyword maps to the eve field ldap.request.operation
It is an unsigned 8-bit integer
Doesn't support prefiltering
Ticket: #7453
Describe how to use the git commit template. The template helps ensure
that the information needed for evaluation and context is included in
the commit message.
Ticket: <Redmine ticket number>
Ticket: 7199
Uses a config parameter detect.guess-applayer-tx to enable
this behavior (off by default)
This feature is requested for use cases with signatures not
using app-layer keywords but still targetting application
layer transactions, such as pass/drop rule combination,
or lua usage.
This overrides the previous behavior of checking if the signature
has a content match, by checking if there is only one live
transaction, in addition to the config parameter being set.
Similar keywords use `isnotset`, while `flowint` only accepted `notset`
Opted to change the code, not only the regex, to keep the underlying
code also following the same patterns.
Task #7426
For example, "requires: foo bar" is an unknown requirement, however
its not tracked, nor an error as it follows the syntax. Instead,
record these unknown keywords, and fail the requirements check if any
are present.
A future version of Suricata may have new requires keywords, for
example a check for keywords.
Ticket: #7418
Ticket: 2224
It takes an argument to match only if the buffer is absent,
or it can still match if the buffer is present, but we test
the absence of some content.
For multi buffers, absent matches if there are 0 buffers.
For file keywords, absent matches if there is no file.
Since enabling custom logging will replace the extended logging, thus
possibly leading to certain fields disappearing from the logs, mention
this aspect.
Related to
Bug #7333
Add EVE documentation for QUIC and Pgsql to their respective sections of
the userguide.
Also add a complete EVE reference as an appendix.
Other protocols can be done, but its a manual process to document in the
schema, then add the glue to pull them into the documentation.
The documentation is generated during "make dist", or if it doesn't
exist, "conf.py" will attempt to generate the eve documentation for
building on Readthedocs.
Ticket: 7191
So as to avoid quadratic complexity in libhtp.
Make the limit configurable from suricata.yaml,
and have an event when network traffic goes over the limit.
Use sed + regex to replace all occurrences of suricata.yaml terms that
used underscore for their up-to-date dash version.
Also search for such terms in the eve-log.yaml partials file, as that
is referenced in the configuration section.
commands used:
sed -i 's/\(^ *[a-z]*\)_\([a-z]*:\)/\1-\2/g'
sed -i 's/\(^ *[a-z]*\)_\([a-z]*\)_\([a-z]*:\)/\1-\2-\3/g'
Some other instances were found manually.
Task #7260
If a stream-only rule matches, and we find a tx where we
want to log the app-layer data, store into the tx data that
we already logged, so that we do not log again the app-layer metadata
Ticket: 7085
Although the `prune-flows` option was removed with a5587fec2e,
when documentation for the suricata.yaml config file was added with
b252b0d, this option was also included - as has remained until now.
They have updated their docs domain, leading to the link we had
returning a 404.
Also checked the other links. Although some seem to only contain old
traffic, they all still work.
This ensures that the correct spelling of the `security_result` EVE
field for RFB (as opposed to `security-result`) is also reflected in the
documentation.
Ticket: #7210
While not incorrect, the previous wording made the sentence almost
paradoxical. While at it, also highlight a side effect that might not be
so clear to users.
Related to
Bug #6976
Ticket: 5734
Adds frames for SSH records, that come after banner, and before
the data is encrypted.
These records may contain cipher lists for instance.
Ticket: 4863
On the way, convert some keywords to use the first-class integer
support.
And helpers for pure rust the support for multi-buffer.
Move the C unit tests about keyword mqtt.protocol_version
to unit tests for generic integer parsing, and test version 5
instead of testing twice version 3.
Also iterate all tx's messages for reason code as is done for other
keywords.
And allow detection on empty topics.
Ticket: 4863
On the way, convert unit test DetectSNMPCommunityTest to a SV test.
And also, make snmp.pdu_type use a generic uint32 for detection,
allowing operators, instead of just equality.
With triggering stream reassembly early, since for certain types of
rules there may be more alerts triggered - even in IPS mode, make this
clear in the upgrading section.
Bug #7026
Ticket: 3958
- transactions are now bidirectional
- there is a logger
- gap support is improved with probing for resync
- frames support
- app-layer events
- enip_command keyword accepts now string enumeration as values.
- add enip.status keyword
- add keywords :
enip.product_name, enip.protocol_version, enip.revision,
enip.identity_status, enip.state, enip.serial, enip.product_code,
enip.device_type, enip.vendor_id, enip.capabilities,
enip.cip_attribute, enip.cip_class, enip.cip_instance,
enip.cip_status, enip.cip_extendedstatus
Although we have the example for a commit message in our Code Submission
Process sub-chapter, seems that people still oversee it a lot. It was
suggested that we put it in a note-box, to make it more visible.
When we added feature #5976 (72146b969), we overlook that we also have
a config stats option for the human-readable stats logs to output
0 counters.
Due to not seeing this before, we now have two different setting names
for basically the same thing, but in different logs:
- zero-valued-counters for EVE
- null-values for stats.log
This ensures we use the same terminology, and change the recently added
one to `null-values`, as this one has been around for longer.
Task #6962
Some stats can be quite verbose if logging all zero valued-counters.
This allows users to disable logging such counters. Default is still
true, as that's the expected behavior for the engine.
Task #5976
Some exception policies can only be applied to entire flows or
individual packets, for some exception scenarios. Make this easier to
read, in the documentation.
Related to
Task #5816
One of the lessons of the XZ backdoor story was that just linking to
libsystemd to call sd_notify is discouraged by the systemd project:
Lennart Poettering:
"PSA: In context of the xzpocalypse we now added an example reimplementation
of sd_notify() to our man page:
https://www.freedesktop.org/software/systemd/man/devel/sd_notify.html#Notes
It's pretty comprehensive (i.e. uses it for reload notification too), but
still relatively short.
In the past, I have been telling anyone who wanted to listen that if all you
want is sd_notify() then don't bother linking to libsystemd, since the
protocol is stable and should be considered the API, not our C wrapper
around it. After all, the protocol is so trivial"
From: https://mastodon.social/@pid_eins/112202687764571433
This commit takes the example code and uses it to reimplement the notify
logic.
The code is enabled if Linux is detected in configure. Since the code
won't do anything if the NOTIFY_SOCKET env var isn't set, this should
also work fine on systems w/o systemd.
Ticket: #6913.
By default, when Sphinx generates the man pages, the current date will
be embedded in them. This can be set to a specific date with the
"today" variable. Typically the date embedded in manpages in the
release date.
To achieve this, attempt to use the environment variable, RELEASE_DATE
to set the "today" variable, reverting back to the empty string if not
set. It is up to our build system to properly set this date.
Ticket: #6911
dns.rcode matches the rcode header field in DNS messages
It's an unsigned integer
valid ranges = [0-15]
Does not support prefilter
Supports matches in both flow directions
Task #6621
It matches the rrtype field in DNS
It's an unsigned integer match
valid ranges = [0-65535]
Does not support prefilter
Supports flow in both directions
Feature #6666
Our code handles Uint ranges as exclusive, but for bsize, our
documentation stated that they're inclusive.
Cf. from uint.rs:
DetectUintMode::DetectUintModeRange => {
if val > x.arg1 && val < x.arg2 {
return true;
}
}
Task #6708
Remove references that are mentioning Suricata 3 or less
As a note - only one Suricata 4 reference found:
(suricata-yaml.rst:"In 4.1.x")
Fast pattern selection criteria can be internally found by inspecting
SupportFastPatternForSigMatchList and SigTableSetup functions.
Ticket: #6570
When the packet load is low, Suricata can run in interrupt
mode. This more resembles the classic approach of processing
packets - CPU cores run low and only fetch packets
on interrupt.
Ticket: #5839
Previous integration of hugepage analysis only fetched data
from /proc/meminfo. However this proved to be often
deceiving mainly for providing only global information and
not taking into account different hugepage sizes (e.g. 1GB
hugepages) and different NUMA nodes.
Ticket: #6419
If a commit introduces code that changes Suricata behavior, the related
documentation changes should go in a separate commit, but refer to the
same ticket number.
This reduces the chances of said changes being lost if there are backports
while still keeping the backporting process a bit less bulky, for each
commit.
Related to
Task #6568
This section seemed to aim both at PR reviewers and PR authors at the
same time, even though some info is probably of low value for
contributors.
Created new section for PR reviewers and maintainers, and kept the info
for PR authors separated. Also highlighted information on requested
changes and stale PRs.