Work towards making `suricata-common.h` only introduce system headers
and other things that are independent of complex internal Suricata
data structures.
Update files to compile after this.
Remove special DPDK handling for strlcpy and strlcat, as this caused
many compilation failures w/o including DPDK headers for all files.
Remove packet macros from decode.h and move them into their own file,
turn them into functions and rename them to match our function naming
policy.
This patch is adding support for Landlock, a Linux
Security Module available since Linux 5.13.
The concept is to prevent any file operation on directories where
Suricata is not supposed to access.
Landlock support is built by default if the header is present. The
feature is disabled by default and need to be activated in the YAML
to be active.
Landlock documentation: https://docs.kernel.org/userspace-api/landlock.html
Feature: #5479
Instead of using distutils/setuptools for installing the Python code,
just install it into our own Python directory.
Distutils is being removed from Python, and setuptools doesn't work well
when trying to install into your own location. For our usage its just
simpler to install with make.
In addition to removing the configure check for distutils, also remove
the check for pyyaml. This lets the user install pyyaml after Suricata
is installed, and Suricata-Update does handle this case gracefully.
Issue: #5313
The format of initial packet for quic ietf, ie quic v1,
is described in rfc 9000, section 17.2.2
Parse more frames and logs interesting extensions from crypto frame
Do not try to parse encrypted data, ie after we have seen
a crypto frame in each direction.
Use sni from crypto frame with tls for detection already implemented
Ticket: #4967
On some platforms (riscv64, s390x) this value is "undefined" as returned
from getconf. We also need to handle this to avoid using the string
"undefined" blindly in further #defines.
Currently, it seems easier to upload the diagram images to git than to
try to make the image generation script work with out of the tree builds
and other corner cases.
This means, however, that one must activelly remember to update msc
diagram files, run the script and re-add new png files, if those ever
need to be updated. To raise awareness to that, a watermark was added
to the diagram images.
Also removed configuration steps that added mscgen as dependency
(locally and for workflow builds and readthedocs).
Moved devguide dir into userguide dir.
Since the devguide is now incorporated as the last chapter of the
userguide, removed build and configuration files from the devguide
dir, as these are no longer needed.
Task #4909
cargo vendor has been part of the core cargo command since Rust 1.37,
and are minimum Rust version is not 1.41, so remove the check. Its
always available now.
Always use the same path for default-rule-path whether or not
Suricata-Update will be installed as part of the Suricata install or
not.
This provides consistency, and maps better to our recommendation that
Suricata-Update be used to manage rules.
Probably should have been done as part of
55852d0de3.
Ticket #4912.
The function macro existed so it would only be enabled on Rust
versions that supported. Now that our MSRV is 1.41, which is
greater than 1.38 we can assume we always have support for
this macro.
This commit adds support for enabling libunwind -- a library that can be
used to display stack information.
Libunwind is enabled and used by Suricata if present during
configuration. A diagnostic message is displayed if libunwind
cannot be found.
Instead of using local implementations for the queue.h macro,
wrap the system provided queue.h and then adding missing
features as needed.
The idea is that Suricata when integrated with another library
that includes sys/queue.h can look at the same source of truth
for these macros.
But not all operating systems include a queue.h with the same
features, and some don't include it at all, like Windows. So
on Windows this will be a full implementation of all the queue.h
features Suricata needs.
A guide on what is a transaction for Suricata engine, focusing on
developers.
- What's the purpose of a transaction;
- transaction states and API callbacks;
- Examples and sequence diagrams.
- doc/devguide: add transactions.rst
- doc/devguide/extending/app-layer/index.rst: add transactions.rst
Currently has one derive, AppLayerEvent to be used like:
#[derive(AppLayerEvent)]
pub enum DNSEvent {
MalformedData,
NotRequest,
NotResponse,
ZFlagSet,
}
Code will be generated to:
- Convert enum to a c type string
- Convert string to enum variant
- Convert id to enum variant
If suricata-update is not available on "make install-full", don't
exit 1, instead give the reason why its not installed, but still
succeed the install.
Following the pattern of many other libraries, provide a -config
program to output cflags and libs to properly link an application
against the library.
usage: libsuricata-config [--cflags] [--libs] [--static]
--cflags and --libs can be used infividually or together.
--static will link against the static libraries instead of the
shared library. Note that if the shared library is not available,
the static libraries will be provided even without this option.
Fix another issue with library ordering when breaking apart
LDFLAGS from LIBS for outputting usable command lines for
users of a Suricata library.
RUST_LDADD should just contain the extra libs required by
Rust, not the actual Suricata Rust library.
As we don't install the libraries by default, provide a make target,
"install-library" to install the libsuricata library files.
If shared library support exists, both the static and shared
libraries will be installed, otherwise only the static libraries
will be installed.
Building the shared library on Linux is not something by default.
Instead a user must opt-in to building by running the
"make libsuricata.so" target in the src/ directory.
Currently shared library support is only available on Linux. More
OSs will be supported as we can test them.
Where clang --version was returning:
clang version 9.0.1-15+b1
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Newer version like clang-10 on Debian are returning:
Debian clang version 10.0.1-8+b1
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
As a result the parsing was failing to determine which llc was
available on the system.
Prior to Rust 1.44, Cargo would name static libs with the .lib
extension. 1.44 changes this extension to .a when running under
a GNU environment on Windows like msys to make it more similar
to other unix environments.
Now assume static library name to be the same on Windows and
unix, but rename the .lib if found to still support older
versions of Rust on Windows.
Borrow a macro from https://github.com/popzxc/stdext-rs that
will give us the Rust function name in SCLog messages in Rust.
As this trick only works on Rust 1.38 and newer, keep the old
macro around and set a feature based on a Rust version test
done during ./configure.
Only run cbindgen when necessary. This is a bit tricky. When
building a dist we want to unconditionally build the headers.
When going through a "make; sudo make install" type process,
cbindgen should not be run as the headers already exist, are
valid, and the environment under sudo is more often than
not suitable to pick up the Rust toolchains when installed
with rustup.
For the normal "make" case we have the gen/rust-bindings.h file
depend on library file, this will cause it to only be rebuilt
if the code was modified.
For "make dist" we unconditionally create "dist/rust-bindings.h".
This means the generated file could be in 2 locations, so update
configure.ac, and the library search find to find it.
The "gen/rust-bindings.h" should be picked up first if it exists,
for those who develop from a dist archive where "dist/rust-bindings.h"
also exists.
Not completely happy having the same file in 2 locations, but not
sure how else to get the dependency tracking correct.
Support either the __thread GNUism or the C11 _Thread_local.
Use 'thread_local' to point to the one that is used. Convert existing
__thread user to 'thread_local'.
Remove non-thread-local code from the packet pool code.
Expose UTH flow builder to new 'FUZZ' define as well. Move UTHbufferToFile
as well and rename it to a more generic 'TestHelperBufferToFile'.
This way UNITTESTS can be disabled. This leads to smaller code size
and more realistic testing as in some parts of the code things
behave slightly differently when UNITTESTS are enabled.
If autodiscovery of libnss was used (default), then the line
libnss support: yes
was never set to no.
Same behavior for libnspr.
Broken by commit 'configure: fix nspr check logic' (7ea269a212)
The modifications as part of the cbindgen commit caused issues
with distcheck, revert the Makefile to how it was with the Python
generator, but still using cbindgen.
Also always assume we'll include the generated headers in the
distribution archive to fix make distcheck from distribution
archives with headers included, but no cbindgen.
The use of AC_CHECK_FILE and AC_CHECK_FILES cause the following error
when cross-compiling:
configure: error: cannot check for file existence when cross compiling
The solution is to check for the file directly instead of using a macro.
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
pcap can depends on nl-3 so use pkg-config to find these dependencies
otherwise all AC_CHECK_LIB calls will fail when building statically
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Don't try to run suricata-update if its not installed.
The 'make install-rules' target would try to run suricata-update
when it was detected that it was bundled, but didn't consider
if suricata-update was actually installed.
If rustup is in use, and a user uses sudo or su for the make
install, the install may fail with a "no default toolchain"
error.
To prevent this, detect at configure if rustup is being used,
then set RUSTUP_HOME for all calls to cargo.
Instead of telling the user what packages to install for missing
Python modules, give generic instructions about what module
needs to be installed.
It is getting tricky to get these package names correct
across distributions.
Don't detect the Python version, it is not needed anyways,
all we need is the Python path.
Also, python2 --version prints to stderr, while python3
prints to stdout, leading to some odd output during
./configure (but fixable).
Replace index by strchr and rindex by strrchr.
index(3) states "POSIX.1-2008 removes the specifications of index() and
rindex(), recommending strchr(3) and strrchr(3) instead."
Add index/rindex to banned function check so they don't get reintroduced.
Bug #1443.
As AC_SUBST doesn't expand to anything in the shell script, this
will generate a bad script on older versions of autoconf.
Change the logic to eliminate the possibility of an empty
if or else block.
Redmine issue:
https://redmine.openinfosecfoundation.org/issues/3124
Get rid of enable-rust-debug flag and use enable-debug for acheiving the
desired functionality. From now, adding `--enable-debug` to `configure`
shall create an [unoptimitized + debuginfo] target. Rest behavior stays
the same.
Closes redmine ticket #3054
This define is used to remove reference to capture bypass in case
no capture method implementing this is active.
This patch also introduces CAPTURE_OFFLOAD_MANAGER that is defined
if we need the flow bypass manager code.
This change makes it possible to generate the eBPF programs even if
Suricata itself is built a different C compiler. It also simplifies
how the correct llc program is detected.
Implements Feature https://redmine.openinfosecfoundation.org/issues/2789
Date makes it even clearer that when was the last commit for the build
that one is running. Add this info alongwith rev. Change inspired by
rustc.
Before
```
$ suricata -V
This is Suricata version 5.0.0-dev (rev 2d217e666)
```
After
```
This is Suricata version 5.0.0-dev (2d217e666 2019-07-12)
```
Closes redmine ticket #3092
* added nss and nspr requirements for appveyor build
* added nss and nspr requirements for travis builds
* added travis build without nss and nspr
Signed-off-by: jason taylor <jtfas90@gmail.com>
- There is now an option to automatically create streams on the
correct NUMA node when using cpu affinity.
- When not using cpu affinity the user can specify streams to be
created in the suricata.yaml file. It is no longer required to
use NTPL to create streams before running suricata.
- The legacy usage model of running NTPL to create streams is still
available. This can be used for legacy configurations and complex
configurations that cannot be satisfied by the auto-config option.
* the lzma check during configure wasn't properly displaying the
additional information on how to install if --enable-lzma was passed
but lzma devel files were not present
* updated additional information blocks to include distribution
package names
* minor formatting updates to add quotes around variables
Signed-off-by: jason taylor <jtfas90@gmail.com>
Require distutils to install the Python tools. Update the logic
to only install suricatactl (and suricatasc) if Python and
distutils are found. Suricata-Update will only be installed if
bundled, and python-distutils and python-yaml are found.
When looking for Python, prefer "python3" over "python2" and
"python".
Also add information about the Python path and version to the
./configure summary.
Redmine issue:
https://redmine.openinfosecfoundation.org/issues/2808
Victor Julien
Sascha Steinbiss
Eric Leblond
Jason Ish
Philippe Antoine
Juliana Fajardini
Jeff Lucovsky
Shivani Bhardwaj
Lukas Sismis
Eric Leblond
Victor Julien
Pierre Chifflier
Phil Young
Danny Browning
Fabrice Fontaine
Ciprian
Hilko Bengen
Bill Meeks
Andreas Herz
jason taylor
Jason Ish