Expose UTH flow builder to new 'FUZZ' define as well. Move UTHbufferToFile
as well and rename it to a more generic 'TestHelperBufferToFile'.
This way UNITTESTS can be disabled. This leads to smaller code size
and more realistic testing as in some parts of the code things
behave slightly differently when UNITTESTS are enabled.
If autodiscovery of libnss was used (default), then the line
libnss support: yes
was never set to no.
Same behavior for libnspr.
Broken by commit 'configure: fix nspr check logic' (7ea269a212)
The modifications as part of the cbindgen commit caused issues
with distcheck, revert the Makefile to how it was with the Python
generator, but still using cbindgen.
Also always assume we'll include the generated headers in the
distribution archive to fix make distcheck from distribution
archives with headers included, but no cbindgen.
The use of AC_CHECK_FILE and AC_CHECK_FILES cause the following error
when cross-compiling:
configure: error: cannot check for file existence when cross compiling
The solution is to check for the file directly instead of using a macro.
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
pcap can depends on nl-3 so use pkg-config to find these dependencies
otherwise all AC_CHECK_LIB calls will fail when building statically
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Don't try to run suricata-update if its not installed.
The 'make install-rules' target would try to run suricata-update
when it was detected that it was bundled, but didn't consider
if suricata-update was actually installed.
If rustup is in use, and a user uses sudo or su for the make
install, the install may fail with a "no default toolchain"
error.
To prevent this, detect at configure if rustup is being used,
then set RUSTUP_HOME for all calls to cargo.
Instead of telling the user what packages to install for missing
Python modules, give generic instructions about what module
needs to be installed.
It is getting tricky to get these package names correct
across distributions.
Don't detect the Python version, it is not needed anyways,
all we need is the Python path.
Also, python2 --version prints to stderr, while python3
prints to stdout, leading to some odd output during
./configure (but fixable).
Replace index by strchr and rindex by strrchr.
index(3) states "POSIX.1-2008 removes the specifications of index() and
rindex(), recommending strchr(3) and strrchr(3) instead."
Add index/rindex to banned function check so they don't get reintroduced.
Bug #1443.
As AC_SUBST doesn't expand to anything in the shell script, this
will generate a bad script on older versions of autoconf.
Change the logic to eliminate the possibility of an empty
if or else block.
Redmine issue:
https://redmine.openinfosecfoundation.org/issues/3124
Get rid of enable-rust-debug flag and use enable-debug for acheiving the
desired functionality. From now, adding `--enable-debug` to `configure`
shall create an [unoptimitized + debuginfo] target. Rest behavior stays
the same.
Closes redmine ticket #3054
This define is used to remove reference to capture bypass in case
no capture method implementing this is active.
This patch also introduces CAPTURE_OFFLOAD_MANAGER that is defined
if we need the flow bypass manager code.
This change makes it possible to generate the eBPF programs even if
Suricata itself is built a different C compiler. It also simplifies
how the correct llc program is detected.
Implements Feature https://redmine.openinfosecfoundation.org/issues/2789
Date makes it even clearer that when was the last commit for the build
that one is running. Add this info alongwith rev. Change inspired by
rustc.
Before
```
$ suricata -V
This is Suricata version 5.0.0-dev (rev 2d217e666)
```
After
```
This is Suricata version 5.0.0-dev (2d217e666 2019-07-12)
```
Closes redmine ticket #3092
* added nss and nspr requirements for appveyor build
* added nss and nspr requirements for travis builds
* added travis build without nss and nspr
Signed-off-by: jason taylor <jtfas90@gmail.com>
- There is now an option to automatically create streams on the
correct NUMA node when using cpu affinity.
- When not using cpu affinity the user can specify streams to be
created in the suricata.yaml file. It is no longer required to
use NTPL to create streams before running suricata.
- The legacy usage model of running NTPL to create streams is still
available. This can be used for legacy configurations and complex
configurations that cannot be satisfied by the auto-config option.
* the lzma check during configure wasn't properly displaying the
additional information on how to install if --enable-lzma was passed
but lzma devel files were not present
* updated additional information blocks to include distribution
package names
* minor formatting updates to add quotes around variables
Signed-off-by: jason taylor <jtfas90@gmail.com>
Require distutils to install the Python tools. Update the logic
to only install suricatactl (and suricatasc) if Python and
distutils are found. Suricata-Update will only be installed if
bundled, and python-distutils and python-yaml are found.
When looking for Python, prefer "python3" over "python2" and
"python".
Also add information about the Python path and version to the
./configure summary.
Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2808
Process multiple packets at nm_dispatch. Use zero copy for workers
recv mode.
Add configure check netmap check for API 11+ and find netmap api version.
Add netmap guide to the userguide.
Currently, if the user provides --enable-libmagic or
--disable-libmagic, libmagic will be disabled because $enableval is not
used to know if the user provided --enable or --disable
Most of the options have this issue so fix them all by using $enableval
Fixes:
- https://redmine.openinfosecfoundation.org/issues/2797
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
This is to prevent suricata-update from being installed if it
would otherwise be installed based on in being bundled, and
its dependencies being available.
Warn the user that Suricata-Update will not be installed if it
is bundled, but python-yaml is missing (this will also cover
the case where Python is missing).
Add "Install suricata-update" to the build summary. For consistency,
relable "Suricatasc install" as "Install suricatasc".
Remove 'experimental' label for Rust, and enable it by default if
rustc and cargo (and libjansson) are available.
Add rustc and cargo versions to the build-info.
If the data file can't be found in the default location, which
normally is 'default-rule-path', try to see if it can be found
in the path of the rule file that references it.
This makes QA much easier.
Move the rule file configuration down near the bottom of the
configuration file under advanced settings. With the bundling
of Suricata-Update, any rule file configuration within
suricata.yaml could be considered advanced.
Add extra comments to the yaml to make it more clear which was
enabled at installation time.
Enables IPS functionality on Windows using the open-source
(LGPLv3/GPLv2) WinDivert driver and API.
From https://www.reqrypt.org/windivert-doc.html : "WinDivert is a
user-mode capture/sniffing/modification/blocking/re-injection package
for Windows Vista, Windows Server 2008, Windows 7, and Windows 8.
WinDivert can be used to implement user-mode packet filters, packet
sniffers, firewalls, NAT, VPNs, tunneling applications, etc., without
the need to write kernel-mode code."
- adds `--windivert [filter string]` and `--windivert-forward [filter
string]` command-line options to enable WinDivert IPS mode.
`--windivert[-forward] true` will open a filter for all traffic. See
https://www.reqrypt.org/windivert-doc.html#filter_language for more
information.
Limitation: currently limited to `autofp` runmode.
Additionally:
- `tmm_modules` now zeroed during `RegisterAllModules`
- fixed Windows Vista+ `inet_ntop` call in `PrintInet`
- fixed `GetRandom` bug (nonexistent keys) on fresh Windows installs
- fixed `RandomGetClock` building on Windows builds
- Added WMI queries for MTU
Autoconf/automake and python setup.py don't play that well
together with out of tree builds.
Makes suricatasc not an autoconf input file, instead use the
defaults module that is already being created.
In the case of an out of tree build, copy the generated defaults.py
to the build directory manually.
Introduces the option 'outputs.pcap-log.compression' which can be set
to 'none' or 'lz4', plus options to set the compression level and to
enable checksums. SCFmemopen is used to make pcap_dump() write to a
buffer which is then compressed using liblz4.
If suricata-update is bundled, set the default-rule-dir
to lib/suricata/rules under the $localstatedir
For now use 2 rule-files section that are renamed depending
on if suricata-update is bundled or not.
Common /usr/share/suricata/rules or /usr/local/share/suricata/rules.
The rules provided by the distribution are installed here as part
of the Suricata install process so will always be installed, even
without the use of install-rules.
Will be built and installed as part of the Python code used
for suricatactl, which is intended to be the generic place
for all Python utility code that gets installed with Suricata.
No change to suricatasc code.
Don't treat 'external' parsers as more experimental. All parsers
depend on crates to some extend, and all have C glue code. So the
distinction doesn't really make sense.
Let user chose to disable libnss and libnspr support even if these
libraries are installed in the system. Default remains to enable when
libraries are found and disable parameter were not used
This patch introduces the ebpf cluster mode. This mode is using
an extended BPF function that is loaded into the kernel and
provide the load balancing.
An example of cluster function is provided in the ebpf
subdirectory and provide ippair load balancing function.
This is a function which uses the same method as
the one used in autofp ippair to provide a symetrical
load balancing based on IP addresses.
A simple filter example allowing to drop IPv6 is added to the
source.
This patch also prepares the infrastructure to be able to load
and use map inside eBPF files. This will be used later for flow
bypass.
Use a new directory, Python to host the Suricata python modules.
One entry point is suricatactl, a control script for
miscalleneous tasks. Currently onl filestore pruning
is implemented.
Add option to put Rust code in non-'--release' mode, preserving
debug symbols.
Until now Suricata would have to be compiled with --enable-debug for
this.
This patch adds support for hw bypass by enabling flow offload in the network
card (when supported) and implementing the BypassPacketsFlow callback.
Hw bypass support is disabled by default, and can be enabled by setting
"bypass: yes" in the pfring interface configuration section in suricata.yaml.
During configure, substitute the path of cargo, as well as the
value of CARGO_HOME as variables. This fixes the case where a
user might do:
make
sudo make install
Which will cause the cargo bits to be rebuilt, including
re-downloading external crates.
By saving these to variables we can be sure that the same
values are used during make install as were used during
make which prevents the Rust artifacts from being rebuild
during "sudo make install".
Rust is currently optional, use the --enable-rust configure
argument to enable Rust.
By default Rust will be built in release mode. If debug is enabled
then it will be built in debug mode.
On make dist, "cargo vendor" will be run to make a local copy
of Rust dependencies for the distribution archive file.
Add autoconf checks to test for the vendored source, and if it
exists setup the build to use the vendored code instead of
fetching it from the network.
Also, as Cargo requires semantic versioning, the Suricata version
had to change from 4.0dev to 4.0.0-dev.
eve: detects libevent for async redis at configure
eve: moves redis output code to new file - util-log-redis.{c,h}
eve: redis ECHO and QUIT commands for async mode
eve: redis output defaults if conf is missing
The new Hyperscan 4.4 API provides a function to check for SSSE3
presence at runtime. This allows us to fall back to non-Hyperscan
matchers on systems without SSSE3 even when the suricata executable
is built with Hyperscan support. Addresses Redmine issue #2010.
Signed-off-by: Sascha Steinbiss <sascha@steinbiss.name>
Tested-by: Arturo Borrero Gonzalez <arturo@debian.org>
On start, look for existing pcap log files and add them to
the ring buffer. This makes pcap-log self maintaining over
restarts removing the need for external tools to clear
orphaned files.
If TPACKET_V3 is not defined then it is not available and we should
not build anything related to tpacket_v3. This will allow us to
activate it dy default and fallback to v2 if not available.
Add support for AFL PERSISTANT_MODE when Suricata is compiled with
a supported compiler (only afl-clang-fast for now).
This gives a ~10x performance boost when fuzzing.
This patch introduces a new set of commandline options meant for
assisting in fuzz testing the app layer implementations.
Per protocol, 2 commandline options are added:
--afl-http-request=<filename>
--afl-http=<filename>
In the former case, the contents of the file are passed directly to
the HTTP parser as request data.
In the latter case, the data is devided between request and responses.
First 64 bytes are request, then next 64 are response, next 64 are
request, etc, etc.
When --with-libpcre-libraries is used we skip the libpcre 8.35 check
since pkg-config might still point to the 8.35 version installed
although newer version was passed with --with-libpcre-libraries.
It is the small things that count. This is an example of the fix
Before
--disable-threading-tls Disable TLS (thread local storage)]
After
--disable-threading-tls Disable TLS (thread local storage)