Internals
---------
Suricata's stream engine returns data for inspection to the detection
engine from the stream when the chunk size is reached.
Bug
---
Inspection triggered only in the specified chunk sizes may be too late
when it comes to inspection of smaller protocol specific data which
could result in delayed inspection, incorrect data logged with a transaction
and logs misindicating the pkt that triggered an alert.
Fix
---
Fix this by making an explicit call from all respective applayer parsers to
trigger raw stream reassembly which shall make the data available for inspection
in the following call of the stream engine. This needs to happen per direction
on the completion of an entity like a request or a response.
Important notes
---------------
1. The above mentioned behavior with and without this patch is
affected internally by the following conditions.
- inspection depth
- stream depth
In these special cases, the inspection window will be affected and
Suricata may not consider all the data that could be expected to be
inspected.
2. This only applies to applayer protocols running over TCP.
3. The inspection window is only considered up to the ACK'd data.
4. This entire issue is about IDS mode only.
Modbus has a classic request response model, so, a call to trigger raw
stream reassembly is added on completion of each request and response.
Optimization 7026
Bug 7004
Internals
---------
Suricata's stream engine returns data for inspection to the detection
engine from the stream when the chunk size is reached.
Bug
---
Inspection triggered only in the specified chunk sizes may be too late
when it comes to inspection of smaller protocol specific data which
could result in delayed inspection, incorrect data logged with a transaction
and logs misindicating the pkt that triggered an alert.
Fix
---
Fix this by making an explicit call from all respective applayer parsers to
trigger raw stream reassembly which shall make the data available for inspection
in the following call of the stream engine. This needs to happen per direction
on the completion of an entity like a request or a response.
Important notes
---------------
1. The above mentioned behavior with and without this patch is
affected internally by the following conditions.
- inspection depth
- stream depth
In these special cases, the inspection window will be affected and
Suricata may not consider all the data that could be expected to be
inspected.
2. This only applies to applayer protocols running over TCP.
3. The inspection window is only considered up to the ACK'd data.
4. This entire issue is about IDS mode only.
LDAP can have multiple responses corresponding to a request. The call to
trigger raw stream reassembly has been added on common call sites that
see the completion of a request or any of the responses.
Optimization 7026
Bug 7004
Internals
---------
Suricata's stream engine returns data for inspection to the detection
engine from the stream when the chunk size is reached.
Bug
---
Inspection triggered only in the specified chunk sizes may be too late
when it comes to inspection of smaller protocol specific data which
could result in delayed inspection, incorrect data logged with a transaction
and logs misindicating the pkt that triggered an alert.
Fix
---
Fix this by making an explicit call from all respective applayer parsers to
trigger raw stream reassembly which shall make the data available for inspection
in the following call of the stream engine. This needs to happen per direction
on the completion of an entity like a request or a response.
Important notes
---------------
1. The above mentioned behavior with and without this patch is
affected internally by the following conditions.
- inspection depth
- stream depth
In these special cases, the inspection window will be affected and
Suricata may not consider all the data that could be expected to be
inspected.
2. This only applies to applayer protocols running over TCP.
3. The inspection window is only considered up to the ACK'd data.
4. This entire issue is about IDS mode only.
KRB5 creates a transaction based on how each input is parsed. It could
be parsed as a request or response but that is the concern of the
parser. The call to trigger raw stream reassembly has been added after
successful parsing of the respective request/response.
Optimization 7026
Bug 7004
Internals
---------
Suricata's stream engine returns data for inspection to the detection
engine from the stream when the chunk size is reached.
Bug
---
Inspection triggered only in the specified chunk sizes may be too late
when it comes to inspection of smaller protocol specific data which
could result in delayed inspection, incorrect data logged with a transaction
and logs misindicating the pkt that triggered an alert.
Fix
---
Fix this by making an explicit call from all respective applayer parsers to
trigger raw stream reassembly which shall make the data available for inspection
in the following call of the stream engine. This needs to happen per direction
on the completion of an entity like a request or a response.
Important notes
---------------
1. The above mentioned behavior with and without this patch is
affected internally by the following conditions.
- inspection depth
- stream depth
In these special cases, the inspection window will be affected and
Suricata may not consider all the data that could be expected to be
inspected.
2. This only applies to applayer protocols running over TCP.
3. The inspection window is only considered up to the ACK'd data.
4. This entire issue is about IDS mode only.
HTTP2 has a classic request response model, so, a call to trigger raw
stream reassembly is added on completion of each request and response.
HTTP2 parser has its own maximum reassembly setting. The call has been
added irrespective of this setting as it is prudent to make all data so
far available for inspection if maximum was reached until the maximum.
Optimization 7026
Bug 7004
Internals
---------
Suricata's stream engine returns data for inspection to the detection
engine from the stream when the chunk size is reached.
Bug
---
Inspection triggered only in the specified chunk sizes may be too late
when it comes to inspection of smaller protocol specific data which
could result in delayed inspection, incorrect data logged with a transaction
and logs misindicating the pkt that triggered an alert.
Fix
---
Fix this by making an explicit call from all respective applayer parsers to
trigger raw stream reassembly which shall make the data available for inspection
in the following call of the stream engine. This needs to happen per direction
on the completion of an entity like a request or a response.
Important notes
---------------
1. The above mentioned behavior with and without this patch is
affected internally by the following conditions.
- inspection depth
- stream depth
In these special cases, the inspection window will be affected and
Suricata may not consider all the data that could be expected to be
inspected.
2. This only applies to applayer protocols running over TCP.
3. The inspection window is only considered up to the ACK'd data.
4. This entire issue is about IDS mode only.
ENIP has a classic request response model, so, a call to trigger raw
stream reassembly is added on completion of each request and response.
Optimization 7026
Bug 7004
Transforms that support optional strings, like from_base64 and
pcrexform, should also support identity-strings to treat transforms with
like transform options as the same.
This commit adds transform identity data handling:
- When computing a hash, include identity data from the transform
- When comparing, include the identity data from the transforms
- Omitting the "options" ptr from the transform hash/compare
- Modify xor, pcrexform and from_base64 to supply identification data for
disambiguation in the compare/hash logic.
This also breaks out the fileinfo function into a method per file info
item. And likewise for state, just return the state and add a new method
for checking if the file is stored.
Ticket: #7491
If a password message was seen while logging passwords was disabled
for pgsql, this would lead to an empty request being logged.
Instead of simply not logging anything when there is a password message
and this is disabled, however, log instead that said password is
redacted.
Bug #7647
PKT_STREAM_EOF flag is set only when a pseudo packet is created. In all
the users of this flag, it suffices to use PKT_PSEUDO_STREAM_END
instead. PKT_PSEUDO_STREAM_END is the more widely used flag as well so
keep it and remove this unneeded flag creating a vacancy.
Whether minimum inspection depth should be respected is an important
parameter but it is only used by one caller of StreamReassembleRawDo fn
to calculate progress value. This progress value is already passed as a
parameter to the said fn.
CC detect-engine-loader.o
In file included from /usr/include/stdio.h:970,
from suricata-common.h:77,
from detect-engine-loader.c:24:
In function 'fgets',
inlined from 'DetectLoadSigFile' at detect-engine-loader.c:139:11:
/usr/include/x86_64-linux-gnu/bits/stdio2.h:313:12: warning: argument 2 value -1 is negative [-Wstringop-overflow=]
313 | return __fgets_alias (__s, __n, __stream);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/include/features.h:523,
from /usr/include/dirent.h:25,
from suricata-common.h:73:
/usr/include/x86_64-linux-gnu/bits/stdio2-decl.h: In function 'DetectLoadSigFile':
/usr/include/x86_64-linux-gnu/bits/stdio2-decl.h:96:14: note: in a call to function '__fgets_alias' declared with attribute 'access (write_only, 1, 2)'
96 | extern char *__REDIRECT (__fgets_alias,
| ^~~~~~~~~~
Suppress the following warning:
util-pages.c:49:13: warning: Both PROT_WRITE and PROT_EXEC flags are set. This can lead to exploitable memory regions, which could be overwritten with malicious code [security.MmapWriteExec]
49 | if (mprotect(ptr, getpagesize(), PROT_READ|PROT_WRITE|PROT_EXEC) == -1) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
As the check is done to check if the OS allows it or not, for enabling
JIT in pcre.
util-mpm-hs-cache.c:83:25: warning: Value of 'errno' was not checked and may be overwritten by function 'fread' [unix.Errno]
83 | size_t bytes_read = fread(buffer, 1, file_sz, file);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
"After calling 'rewind' reading 'errno' is required to find out if the call has failed".
detect-bytetest.c:523:14: warning: 2nd function call argument is an uninitialized value [core.CallAndMessage]
523 | if (!DetectBytetestValidateNbytes(data, nbytes, optstr)) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
Shivani Bhardwaj
Philippe Antoine
Jeff Lucovsky
Victor Julien
Jason Ish
Juliana Fajardini