Ticket: 2224
It takes an argument to match only if the buffer is absent,
or it can still match if the buffer is present, but we test
the absence of some content.
For multi buffers, absent matches if there are 0 buffers.
For file keywords, absent matches if there is no file.
Previously various steps in the content inspection logic would use
a variable in the DetectEngineThreadCtx to flag that matching should
be discontinued.
This patch reimplements this logic by using a new return code instead.
Split content inspection into public and private version, so that
common initialization can be done in a single place.
Update the callsites.
Issue: 4145
Consolidate file handling for all protocols that use file objects for
file_data.
Make sure http_server_body / http.response_body for HTTP1 continue
to inspect the actual body. For HTTP2, http.response_body acts as
an internal alias for `file_data`.
Replace implementation of the legacy `filemagic` keyword by the
implementation for the `file.magic` variant. This leads to better
performance and hooks the rules into the detection engine better.
Remove unittests that depended on the old logic.
Ticket: #6194.
Protocols supporting files are only defined in one place, which
gets used by all keywords, which can handle some exceptions
(like HTTP2 not having file names)
Update APIs to store files in transactions instead of the per flow state.
Goal is to avoid the overhead of matching up files and transactions in
cases where there are many of both.
Update all protocol implementations to support this.
Update file logging logic to account for having files in transactions. Instead
of it acting separately on file containers, it is now tied into the
transaction logging.
Update the filestore keyword to consider a match if filestore output not
enabled.
Fix multi inspect buffer API causing cleanup logic in the single
inspect buffer paths. This could lead to a buffer overrun in the
"to clear" logic.
Multi buffers now use InspectionBufferSetupMulti instead of
InspectionBuffer. This is enforced by a check in debug validation.
Simplify the multi inspect buffer setup code and update the callers.
In some cases, the InspectionBufferGet function would be followed by
a failure to set the buffer up, for example due to a HTTP body limit
not yet being reached. Yet each call to InspectionBufferGet would lead
to the matching list_id to be added to the
DetectEngineThreadCtx::inspect.to_clear_queue. This array is sized to
add each list only once, but in this case the same id could be added
multiple times, potentially overflowing the array.
This changseset fixes a bug causing a segmentation fault.
When rules are reloaded and a rule using libmagic matches, suricata
crashes due to an improper reinitialization of the thread contexts.
Bug: #3726
Instead of the hardcode L4 matching in MPM that was recently introduced,
add an API similar to the AppLayer MPM and inspect engines.
Share part of the registration code with the AppLayer.
Implement for the tcp.hdr and udp.hdr keywords.