Stamus team did discover a problem were a signature can shadow
other signatures.
For example, on a PCAP only containing Kerberos protocol and where the
following signature is matching:
alert krb5 $HOME_NET any -> any any (msg:"krb match"; krb5_cname; content:"marlo"; sid:3; rev:1;)
If we add the following signature to the list of signature
alert ssh $HOME_NET any -> any any (msg:"rr"; content:"rr"; flow:established,to_server; sid:4; rev:2;)
Then the Kerberos signature is not matching anymore.
To understand this case, we need some information:
- The krb5_cname is a to_client keyword
- The signal on ssh is to_server
- Kerberos has unidirectional transaction
- kerberos application state progress is a function always returning 1
As the two signatures are in opposite side, they end up in separate
sig group head.
Another fact is that, in the PCAP, the to_server side of the session
is sent first to the detection. It thus hit the sig group head of
the SSH signature. When Suricata runs detection in this direction
the Kerberos application layer send the transaction as it is existing
and because the alstate progress function just return 1 if the transaction
exists. So Suricata runs DetectRunTx() and stops when it sees that
sgh->tx_engines is NULL.
But the transaction is consumed by the engine as it has been evaluated
in one direction and the kerberos transaction are unidirectional so
there is no need to continue looking at it.
This results in no matching of the kerberos signature as the match
should occur in the evaluation of the other side but the transaction
with the data is already seen has been handled.
This problem was discovered on this Kerberos signature but all
the application layer with unidirectional transaction are impacted.
This patch introduces a flag that can be used by application layer
to signal that the TX should not be inspected. By using this flag
on the directional detect_flags_[ts|tc] the application layer can
prevent the TX to be consumed in the wrong direction.
Application layers with unidirectional TX will be updated
in separate commits to set the flag on the direction opposite
to the one they are.
Ticket: #5799
The tx progress functions are expecting a direction and were given
a flow flags. As a result, they were not reporting correctly the
status if a DetectRunScratchPad flow_flags was containing some other
bits in the flag.
One case was when a signature was alterating the stream analysis
and triggering the addition of the STREAM_FLUSH flags.
The consequences are quite severe as the transactions are pilling
up waiting to be inspected causing sometimes a 10x performance hit
on pcap parsing. Also as the inspection was not done, Suricata is
missing a part of the alerts.
This was discovered when working on the following set of signatures:
alert ssh $HOME_NET any -> any any (msg:"pcre without content"; pcre:"/rabbit/"; sid:1; rev:1;)
alert smb $HOME_NET any -> any any (msg:"smb share content"; smb.share; content:"C"; sid:2; rev:1;)
When the first one is present the second is not triggering even
though the pcap file had no ssh inside. This is due to the fact
that the ssh signature was triggering the STREAM_FLUSH flag to
be set on the flowflags of the packet. But the application
layer will ask the smb state progress via
r = alp_ctx.ctxs[FlowGetProtoMapping(ipproto)][alproto].
StateGetProgress(alstate, flags);
passing it the flow flags but the smb function is expecting
a direction so we end up in a unplanned case
pub unsafe extern "C" fn rs_smb_tx_get_alstate_progress(tx: *mut ffi::c_void,
direction: u8)
...
if direction == Direction::ToServer as u8 && tx.request_done {
This leads the signature to not be evaluated correctly.
Ticket: #5799
In the case we are truncating a multipart file because of reaching
request.body-limit, we used to not consume the whole buffer, but
keep expected_boundary_len bytes in case a new boundary begins
in these bytes.
Even if we cannot check the complete boundary, we can still check
the first bytes, as will be done in the rust version.
Ticket: #5952
The YAML spec considers duplicate keys to be an error, as do some YAML
implementations, most notably Rust's serde_yaml which would be nice to
use in the future.
Multiple include lines at the same level will still work, but a warning
will be emitted.
These can be fixed by moving to an "include" array:
include:
- file1.yaml
- file2.yaml
Ticket: #5939
If a field named "include" is mapping it is not processed correctly.
Instead return a fatal error.
In our YAML, "include" has always been a reserved word, so this should
not break any known configuration.
Ticket: #5939
As flow.memcap-policy and defrag.memcap-policy do not support flow
actions, clarify that in the documentation. Also fix some typos, and
add missing values in some places where the exception policies were
explained.
Related to
Bug #5940
Defrag memcap and flow memcap do not support flow action for the
exception policies, as there is no flow when the exception condition is
hit. In such cases, the exception policy must be considered for the
packet only, when that makes sense, or should be ignored, in case of
`bypass`.
Bug #5940
Cargo.lock has to be provided as template, Cargo.lock.in so it can
live beside Cargo.lock in out of tree automake builds, like distcheck.
This will pin Rust dependencies even for git builds, updating
Cargo.lock will now be a manual process that we'll have to take care
of periodically.
Updated all cases where flow_config.prealloc was used in a division.
*** CID 1524506: Integer handling issues (DIVIDE_BY_ZERO)
/src/flow-manager.c: 858 in FlowManager()
852 "flow_spare_q status: %" PRIu32 "%% flows at the queue",
853 spare_pool_len, flow_config.prealloc,
854 spare_pool_len * 100 / flow_config.prealloc);
855
856 /* only if we have pruned this "emergency_recovery" percentage
857 * of flows, we will unset the emergency bit */
>>> CID 1524506: Integer handling issues (DIVIDE_BY_ZERO)
>>> In expression "spare_pool_len * 100U / flow_config.prealloc", division by expression "flow_config.prealloc" which may be zero has undefined behavior.
858 if (spare_pool_len * 100 / flow_config.prealloc > flow_config.emergency_recovery) {
859 emerg_over_cnt++;
860 } else {
861 emerg_over_cnt = 0;
862 }
Related to
Bug #5919
libbpf 0.8 deprecates bpf_get_link_xpd_id, and 1.0 removes it. Add
./configure check to see if this method is available and use it if so,
otherwise use the deprecated method which is not available on older
but still supported Linux distributions.
Ticket: #5924
If urilen induced depth was set, later DetectContentPropagateLimits()
would apply a wrong depth setting, leading to a false negative in
some cases.
Bug: #5929.
In pass/drop combinations where the pass rule took precendence over
the drop, a "drop" false positive could still be logged due to the
storing of the drop record in the packet drop alert store.
Bug: #5867.
The rule lang allows for within and distance to act as depth/offset,
but internally this was not handle the same way. This patch converts
within/distance w/o a prior pattern to depth/within.
Eric Leblond
Philippe Antoine
Victor Julien
Jason Ish
Juliana Fajardini
Philippe Antoine
Jeff Lucovsky
jason taylor