Describe Changes
- Added ability to recursively read pcap directories
- src/suricata.c: addition of new command line parameter
--pcap-file-recursive
- src/source-pcap-file.c: parsing of the command line argument
- src/source-pcap-file-directory-helper.h: two thread vars tracking
directory depth and should recurse
- src/util-error.c / src/util-error.h:
Added new warning code "SC_WARN_PATH_READ_ERROR"
- Redmine ticket: https://redmine.openinfosecfoundation.org/issues/2363
Ticket: #2363
This commit provides changes to util-path.c and util-path.h
to support the recursive reading of directories. It adds
4 functions.
- SCIsRegularFile to provide OS independent file info.
- SCIsRegularDirectory to provide OS independent directory info.
- SCRealPath is an OS independent wrapper for realpath.
- PathJoin to manage path resolution logic.
This commit adds MAC address output to the EVE-JSON format. We follow the
remarks made in Redmine ticket #962: for packets, log MAC src/dst as a
scalar field in EVE; for flows, log MAC src/dst as lists in EVE. Field names
are different between flow and packet context to avoid type confusion
(src_mac vs. src_macs). Configuration approach and JSON representation is
taken from previous GitHub PR #2700.
This commit restricts the anomaly logger count. The restriction is
necessary due to state maintenance in the logger that doesn't scale
beyond a single logger.
Until that issue's solved, when multiple anomaly loggers are configured,
an error message will be emitted to highlight the restriction.
This commit modifies the JSON loggers with changes necessary to support
multi-threaded EVE output.
Each "thread-init" function sets up the per-thread log file context for
subsequent calls to the JSON output to buffer function.
This commit changes an internal-only function to remove a parameter
that's invariant in all use cases. This allows an JSON builder
optimization to be used.
Fixes https://redmine.openinfosecfoundation.org/issues/2689
Adds a new source file to handle this keyword.
And modifies documentation, Makefile, and registration accordingly.
url_decode decodes url-encoded data, ie replacing '+' with space
and '%HH' with its value.
When the flow engine enters emergency mode, 3 things happen:
1. a different set of (lower) timeout values are applied
2. the flow manager runs more often
3. worker threads go get a flow directly from the hash table
Testing showed that performance went down significantly due to concurrency
issues:
1. worker threads would fight each other over the hash access
2. flow manager would get in the way of workers
This patch changes the behavior in 2 ways:
1. it makes the flow manager slightly less aggressive. It will still
try to run ~3 times per second, but no longer 10 times.
This should be reducing the contention. At the same time flows
won't time out faster if they are checked many times per second.
2. The 'get a used flow' logic optimizes the use of atomics by only
doing an atomic operation once, and while doing so reserving
a slice of the hash per worker.
The worker will also give up much quicker, to avoid the overhead
of hash walking and taking and releasing locks.
These combined changes show much better 'under stress' behavior, esp
on multi-NUMA systems.