https://redmine.openinfosecfoundation.org/issues/2417
Add option to have pcap files deleted after they have been processed.
This option combines well with pcap file continuous and streaming
files to a directory being processed.
Calls with both START and MIDSTREAM mean the record might be cut and the
start of it could be missing. For this case, enable the same logic as is
used when catching up after a GAP. Search for the start of the record
instead of assuming it sits exactly at the start of the input data.
Last multi-detect changes broken delayed-detect by refusing to reload
a 'stub' detect engine. This patch distinguishes between a stub for
multi-tenancy and for delayed detect.
There are 3 types of detect engine objects:
1. normal
The normal detection engine if no multi-tenancy is in use
2. tenant
A per tenant detection engine
3. stub
A stub (or minimal as it was called before) detect engine
that is needed to have something in place when there are
only tenants.
A stub is also used in case of 'delayed detect', where we
need a minimal detect engine to start up which is replaced
by a full (normal type) detect engine after startup.
This patch adds a new field 'type' to the DetectEngineCtx object
to distinguish between the types. This replaces the boolean 'minimal'.
The global keyword registration and per thread init handling used
the lock from the DetectEngineMasterCtx. This lead to a dead lock
situation at multi-tenancy tenant reloads.
The lock was unnecessary however, as the only time the registration
list is updated is at engine initialization. At that time Suricata
is still running in a single thread. After this, the data structure
doesn't change anymore.
Bug #2516.
Enables IPS functionality on Windows using the open-source
(LGPLv3/GPLv2) WinDivert driver and API.
From https://www.reqrypt.org/windivert-doc.html : "WinDivert is a
user-mode capture/sniffing/modification/blocking/re-injection package
for Windows Vista, Windows Server 2008, Windows 7, and Windows 8.
WinDivert can be used to implement user-mode packet filters, packet
sniffers, firewalls, NAT, VPNs, tunneling applications, etc., without
the need to write kernel-mode code."
- adds `--windivert [filter string]` and `--windivert-forward [filter
string]` command-line options to enable WinDivert IPS mode.
`--windivert[-forward] true` will open a filter for all traffic. See
https://www.reqrypt.org/windivert-doc.html#filter_language for more
information.
Limitation: currently limited to `autofp` runmode.
Additionally:
- `tmm_modules` now zeroed during `RegisterAllModules`
- fixed Windows Vista+ `inet_ntop` call in `PrintInet`
- fixed `GetRandom` bug (nonexistent keys) on fresh Windows installs
- fixed `RandomGetClock` building on Windows builds
- Added WMI queries for MTU
Much of encrypted traffic is uninteresting to Suricata. Once encrypted
communication starts, inspecting the packet payloads is generally
not interesting anymore. The default behavior is to disable the parts
of the detection engine and stream reassembly that relate to raw content
inspection.
The tls app-layer parser also had a crude option to affect this behavior:
set 'no-reassemble' to true went much further than the default behavior.
It disabled the TCP reassembly on the flow completely, disabled all
inspection on the flow and enabled bypass if available.
This patch adds a new option: full inspection. This continues to treat
a TLS session as any other, so without any limits to inspection.
The new option is implemented in a new config option 'encrypt-handling',
that replaces 'no-reassemble'. The new option has 3 values:
'default', 'full' and 'bypass'. Default is the current default behavior,
'bypass' is the current 'no-reassemble = true' behavior and 'full'
is the new full inspection mode.
The use of stream_size in combination with raw content matches is an
indication that the rule needs to be evaluated per packet, not just
per reassembled stream chunk.
This is a DHCP decoder and logger written in Rust. Unlike most
parsers, this one is stateless so responses are not matched
up to requests by Suricata. However, the output does contain
enough fields to match them up in post-processing.
Rules are included to alert of malformed or truncated options.
Before setting up a sub eve-logger, check that it is enabled. This
allows us to set "enabled: no" for loggers that are not registered
with the system without generating an error. An example of this
is loggers that are only available with Rust.
getrandom syscall availability is detected at runtime. So it is
possible that the build is done on a box that supports it but
the run is done on a system with no availability. So a workaround
solution is needed to fix this case.
Also we have seen some issue in docker environment where the build
is detecting getrandom but where it does not work at runtime.
For both reasons, the code is updated to have a call to a fallback
function if ever the getrandom call returns that the syscall is
not available.
Fix Coverity issue:
** CID 1435535: Null pointer dereferences (REVERSE_INULL)
/src/output-json-file.c: 212 in JsonBuildFileInfoRecord()
Where we check a variable for being NULL, when all paths to the
code show that it can't be NULL.
XFF configuration is already set in app-layer-htp-xff, and in
output-json-alert. Extending XFF configuration to files and HTTP allow
to get the same behavior as for alerts.
Extend the configuration of filestore json to let filestore metafile
dump be aware of xff. This is available only if write-fileinfo is set
to yes and file-store version is 2.
When a file is transferred over anything other than HTTP, the previously hard-coded HTTP protocol would trigger a non-existent index into htp_list_array_get(), causing a segfault. This patch mimics the logic in detect-lua-extensions.c.
Validate that the content that follows the 'tls_cert_serial' keyword
is on the correct form. If it's longer than two bytes it should be
separated by colons.
Introduces the option 'outputs.pcap-log.compression' which can be set
to 'none' or 'lz4', plus options to set the compression level and to
enable checksums. SCFmemopen is used to make pcap_dump() write to a
buffer which is then compressed using liblz4.
When traffic is becoming null (mainly seen in tests) we reach the
situation where there is timeouts in the poll on the socket and
only that. Existing code is then just looping on the poll and
the result is that the packet iface counters are not updated.
This patch calls the dump counter function to be sure to get
the counter right faster (and not only right at exit).
The stream depth computation was partly done with the stream_config
depth instead of using the value in the TCP session. As a result,
some configuration were resulting in abnormal behavior.
In particular, when stream depth was 0 and the file store depth was
not 0, Suricata was stopping the streaming on the flow as soon as
the filestore was started.
Reported-by: Pascal Delalande <pdl35@free.fr>
There is a difference in the size of the buffer length as passed from
the content buffers (cfr HttpReassembledBody.buffer_len) and the buflen
variable passed to mpm primitives. This can cause a misdetection
whenever the bufferlen is multiple of 65536 (as uint16(X*65536) == 0).
Increasing the buflen variable type to uint32 solves the issue (this
does not cause any issue with primitives, they all accept uint32).
If a file transfer stops on flow timeout, it won't be closed or
truncated. This patch makes sure that in such cases the files
are indeed truncated. This fixes the filestore-v2 output module,
as that requires a sha256 for storing the partial file correctly.
CID 1428797 (#1 of 1): Unchecked return value (CHECKED_RETURN)
check_return: Calling HashTableAdd without checking return value
(as is done elsewhere 5 out of 6 times).
Don't treat 'external' parsers as more experimental. All parsers
depend on crates to some extend, and all have C glue code. So the
distinction doesn't really make sense.
Victor Julien
Danny Browning
Jason Ish
Eric Leblond
Jacob Masen-Smith
fooinha
Pierre Chifflier
Maurizio Abba
Elazar Broad
Mats Klepsland
Max Fillinger
Giuseppe Longo
Alexander Gozman