Commit Graph

423 Commits (suricata-7.0.1)

Author SHA1 Message Date
Victor Julien e93dc24383 mingw: add bcrypt to LDADD for rust compilation 4 years ago
Jeff Lucovsky 1c1c21be9e config/netmap: Check for latest API version
This commit checks for the latest Netmap API version.
4 years ago
Jason Ish 75bc9d9dd8 queue.h: wrap the system sys/queue.h
Instead of using local implementations for the queue.h macro,
wrap the system provided queue.h and then adding missing
features as needed.

The idea is that Suricata when integrated with another library
that includes sys/queue.h can look at the same source of truth
for these macros.

But not all operating systems include a queue.h with the same
features, and some don't include it at all, like Windows. So
on Windows this will be a full implementation of all the queue.h
features Suricata needs.
4 years ago
Juliana Fajardini a5b344e015 doc/devguide: add Transactions documentation page
A guide on what is a transaction for Suricata engine, focusing on
developers.
- What's the purpose of a transaction;
- transaction states and API callbacks;
- Examples and sequence diagrams.
- doc/devguide: add transactions.rst
- doc/devguide/extending/app-layer/index.rst: add transactions.rst
4 years ago
Philippe Antoine bce3c46874 pcre2: remove PCRE1 as dependency 4 years ago
Philippe Antoine acb1ab08a2 pcre2: introduce as a new depdendency 4 years ago
Jason Ish 54be743c48 prelude: remove the prelude output
It was broken in 6 and that didn't cause much issue. Just remove
it for 7.
4 years ago
Jason Ish 27d1ee98ce rust: derive crate: for custom derives
Currently has one derive, AppLayerEvent to be used like:

  #[derive(AppLayerEvent)]
  pub enum DNSEvent {
      MalformedData,
      NotRequest,
      NotResponse,
      ZFlagSet,
  }

Code will be generated to:
- Convert enum to a c type string
- Convert string to enum variant
- Convert id to enum variant
4 years ago
Eric Leblond d477d3a878 util/ebpf: fix deprecation warning
The function bpf_program__title has been deprecated in favor of
bpf_program__section_name.
4 years ago
Philippe Antoine e8415f249b fuzz: adds structure aware target
so as not to fuzz libpcap
and generate structure aware signatures
4 years ago
Jason Ish abb3cc85d5 install: better warning on install-full and don't fail
If suricata-update is not available on "make install-full", don't
exit 1, instead give the reason why its not installed, but still
succeed the install.
4 years ago
Philippe Antoine a04b5566a6 http: makes decompression time limit configurable 5 years ago
Jason Ish dfd930a13e libsuricata-config: program to print build flags
Following the pattern of many other libraries, provide a -config
program to output cflags and libs to properly link an application
against the library.

usage: libsuricata-config [--cflags] [--libs] [--static]

--cflags and --libs can be used infividually or together.

--static will link against the static libraries instead of the
shared library. Note that if the shared library is not available,
the static libraries will be provided even without this option.
5 years ago
Jason Ish 2c5e1d6a6d rust: separate the rust lib from RUST_LDADD
Fix another issue with library ordering when breaking apart
LDFLAGS from LIBS for outputting usable command lines for
users of a Suricata library.

RUST_LDADD should just contain the extra libs required by
Rust, not the actual Suricata Rust library.
5 years ago
Jason Ish d648446c32 configure: put lua libs in LIBS not LDFLAGS
This is required to separate LIBS from LDFLAGS when outputting
a usable LIBS configuration line for users of the shared library.
5 years ago
Jason Ish dbae17dbc0 install: makefile target to install libraries
As we don't install the libraries by default, provide a make target,
"install-library" to install the libsuricata library files.

If shared library support exists, both the static and shared
libraries will be installed, otherwise only the static libraries
will be installed.
5 years ago
Jason Ish e227d97e5e lib: build shared library on Linux
Building the shared library on Linux is not something by default.
Instead a user must opt-in to building by running the
"make libsuricata.so" target in the src/ directory.

Currently shared library support is only available on Linux. More
OSs will be supported as we can test them.
5 years ago
Jason Ish a178ec6bef rust: rename lib to libsuricata_rust
Previously it was libsuricata.a, but eventually we want to get
to a place where libsuricata.a is a combination of the Rust
and C code.
5 years ago
Eric Leblond 37b1595c20 configure: fix llc detection on recent Debian
Where clang --version was returning:

clang version 9.0.1-15+b1
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

Newer version like clang-10 on Debian are returning:

Debian clang version 10.0.1-8+b1
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

As a result the parsing was failing to determine which llc was
available on the system.
5 years ago
Jason Ish 6dfc888966 configure.ac: remove tests for libnspr and libnss 5 years ago
Victor Julien 3fdfec860f version: start 7.0.0 development 5 years ago
Victor Julien e860b9eee9 version: set to 6.0.1 5 years ago
Victor Julien f5af8864f7 configure: require libhtp 0.5.36 5 years ago
Jason Ish f3c59ef8a6 rust: handle windows naming change from .lib to .a
Prior to Rust 1.44, Cargo would name static libs with the .lib
extension. 1.44 changes this extension to .a when running under
a GNU environment on Windows like msys to make it more similar
to other unix environments.

Now assume static library name to be the same on Windows and
unix, but rename the .lib if found to still support older
versions of Rust on Windows.
5 years ago
Jason Ish 10f639e9d1 configure/mingw: move libs in LDFLAGS to LDADD
Moving the libs specified in LDFLAGS to LDADD put them into the
correct placement on in the link command.
5 years ago
Philippe Antoine 466466883b rust: do not rebuild stdlib when coverage is enabled
Because both seem incompatible for now
5 years ago
Victor Julien 8d0b0e8739 atomics: fix compilation on ppc64 5 years ago
Victor Julien 95729e923f configure: remove left over reference to unified2 5 years ago
Victor Julien 8fdee4f685 version: start development towards 6.0.1 5 years ago
Victor Julien 5219691f45 version: update to 6.0.0, require libhtp 0.5.35 5 years ago
Philippe Antoine 4f963717f8 fuzz: better configure checks for MSAN building
More compatible check for rust nightly
Checks for CARGO_BUILD_TARGET
Builds release or debug mode independently
5 years ago
Philippe Antoine 222b386102 rust: rebuilds std when building fuzzers
so as to have MSAN working
5 years ago
Victor Julien 2bef41a630 version: continue 6 development 5 years ago
Victor Julien 914391697a version: update to 6.0.0-rc1 5 years ago
Philippe Antoine 9b5c923327 http: disables lzma by default for HTTP 5 years ago
Jason Ish ea1338b464 rust: function macro now returns the function name
Borrow a macro from https://github.com/popzxc/stdext-rs that
will give us the Rust function name in SCLog messages in Rust.

As this trick only works on Rust 1.38 and newer, keep the old
macro around and set a feature based on a Rust version test
done during ./configure.
5 years ago
Jason Ish da3930e488 configure: fix test for rust headers for cross compile
Use "if test ..." instead of AC_CHECK_FILES which does not work
when cross compiling.
5 years ago
Jeff Lucovsky 5c725d5050 config/lua: Cross-compiling support
This commit guards the run-time check for a Lua integer so that it no
longer attempts execution in a cross-compilation environment.
5 years ago
Jeff Lucovsky 3e8db21ef3 config/pcre: Improved support for cross-compiling
This commit changes the logic used to determine if pcre_jit_exec is
available from a run-time to a compile-time check.
5 years ago
Jason Ish fbc9da450d configure: fix detection of netfilter_queue with older headers
Define _GNU_SOURCE and include sys/types.h so older
netfilter_queue headers can be detected properly, as they are
using u_int_xx style integers.
5 years ago
Victor Julien 22c70f7d66 version: continue 6 development 5 years ago
Victor Julien bb01d8ca31 version: update to 6.0.0-beta1 5 years ago
Jason Ish ff81212d8b configure: check for plugin support
Currently plugin support requires the dlfcn.h header
file and compiler support for -rdynamic.
5 years ago
Jason Ish 900f1522b4 plugins: config.h: move into src and rename to autoconf.h
While fixing files that include config.h, just remove the
include if possible.
5 years ago
Jason Ish e6668560e0 rust: only run cbindgen if needed
Only run cbindgen when necessary. This is a bit tricky. When
building a dist we want to unconditionally build the headers.

When going through a "make; sudo make install" type process,
cbindgen should not be run as the headers already exist, are
valid, and the environment under sudo is more often than
not suitable to pick up the Rust toolchains when installed
with rustup.

For the normal "make" case we have the gen/rust-bindings.h file
depend on library file, this will cause it to only be rebuilt
if the code was modified.

For "make dist" we unconditionally create "dist/rust-bindings.h".
This means the generated file could be in 2 locations, so update
configure.ac, and the library search find to find it.

The "gen/rust-bindings.h" should be picked up first if it exists,
for those who develop from a dist archive where "dist/rust-bindings.h"
also exists.

Not completely happy having the same file in 2 locations, but not
sure how else to get the dependency tracking correct.
5 years ago
Jeff Lucovsky 15b4554ab3 output: Check for fwrite_unlocked
This commit creates a macro for fwrite_unlocked which is probed during
configuration time.
5 years ago
Philippe Antoine 4554ca168a build: allows use of env variable CARGO_BUILD_TARGET
needed by oss-fuzz
5 years ago
Shivani Bhardwaj 6db1f19d62 rust: Add debug_validate_bug_on macro
This macro allows to check if certain parts of the code are reachable
during fuzzing.
5 years ago
Victor Julien f8159bd372 build: default to c11 standard
Rearrange pcap includes to fix builds on MinGW
5 years ago
Victor Julien 1893e40e79 build: don't limit C std to c99 (gnu99)
Now that C11 atomics and thread local storage are supported, the
compiler can figure out what version to use.
5 years ago
Victor Julien 7691fc4f9e configure: check for u_int and friends 5 years ago
Victor Julien 3ba4afd40b threads: make thread local storage manadatory
Support either the __thread GNUism or the C11 _Thread_local.

Use 'thread_local' to point to the one that is used. Convert existing
__thread user to 'thread_local'.

Remove non-thread-local code from the packet pool code.
5 years ago
Victor Julien 32cfd71f1a atomics: stdatomics.h version of SC_ATOMIC_* wrappers 5 years ago
Victor Julien 1cb7eec52d atomics: remove spinlocked fallback 5 years ago
Victor Julien df79613fb5 privs: include headers in suricata-common.h 5 years ago
Victor Julien 794d9eeb83 fuzz: remove UNITTEST dependency
Expose UTH flow builder to new 'FUZZ' define as well. Move UTHbufferToFile
as well and rename it to a more generic 'TestHelperBufferToFile'.

This way UNITTESTS can be disabled. This leads to smaller code size
and more realistic testing as in some parts of the code things
behave slightly differently when UNITTESTS are enabled.
5 years ago
Eric Leblond 752fc77cdc configure: correctly display nss/nspr status
If autodiscovery of libnss was used (default), then the line
 libnss support:                          yes
was never set to no.

Same behavior for libnspr.

Broken by commit 'configure: fix nspr check logic' (7ea269a212)
5 years ago
Philippe Antoine 293eebd999 fuzz: remove obsolete AFL code 5 years ago
Philippe Antoine e15f3db474 configure: right test for AFLFUZZ_PERSISTANT_MODE 5 years ago
Pierre Chifflier 1d9f37a60e DER: remove the C parser for DER 5 years ago
Philippe Antoine 600b0d7c55 fuzz: adds eight fuzz targets
And ways to compile them with enable-fuzztargets at configure time
Adds utility function in util-unittest-helper
5 years ago
Jeff Lucovsky 94df0b08d4 configure: Determine whether pcre_jit_exec exists
This commit adds logic to determine whether pcre_jit_exec is present in
the system's pcre library using AC_RUN_ELSEIF
5 years ago
Shivani Bhardwaj 7b1699c5a8 doc: Add chassis for dev docs
Closes redmine ticket 3344.
6 years ago
Victor Julien 95e7246b75 rust: bump minimum supported version to 1.34.2 6 years ago
Jason Ish d86973b386 unified2: remove deprecated output unified2
Ticket 2385:
https://redmine.openinfosecfoundation.org/issues/2385
6 years ago
Phil Young 1c99536945 napatech: add hardware based bypass support
Napatech hardware bypass support enables Suricata to utilize
capabilities of Napatech SmartNICs to selectively bypass flow-based
traffic.
6 years ago
Victor Julien 99d48cc91f configure: update to match autoscan suggestions 6 years ago
Victor Julien a3ef1b307d configure: clean up func checks 6 years ago
Philippe Antoine f5190da67e util: UTHmemsearch to use memmem if defined 6 years ago
Victor Julien 9ae87e79a2 configure: fix cygpath check 6 years ago
Jason Ish 5fbe020585 rust/cbindgen: Revert Makefile to a more pre-cbindgen state
The modifications as part of the cbindgen commit caused issues
with distcheck, revert the Makefile to how it was with the Python
generator, but still using cbindgen.

Also always assume we'll include the generated headers in the
distribution archive to fix make distcheck from distribution
archives with headers included, but no cbindgen.
6 years ago
Danny Browning b573c16dd5 build: cbindgen
Rust headers are now generated using cbindgen. If cbindgen is present, they can
be generated during dist, otherwise they will be available for builds.
6 years ago
Jason Ish 593da166bb version: starting work on 6.0.0
Bump version to 6.0.0-dev.
6 years ago
Victor Julien ce0ae81d95 rust: fix vendor use on MinGW 6 years ago
Victor Julien 3d9071639b version: starting work on 5.0.2 6 years ago
Victor Julien f9840b513d version: release 5.0.1 6 years ago
Victor Julien 9bcc1118e1 configure: require libhtp 0.5.32 6 years ago
Jason Ish 3ca7dcd8d8 configure: fix test -f for rust/vendor, should be -e
Introduced with commit: c08ec8d8b2
6 years ago
Jason Ish f2117774f5 configure: assume cargo vendor if cargo >= 1.37
Rust/Cargo 1.37 and greater has vendor support built-in.
6 years ago
Fabrice Fontaine c08ec8d8b2 configure.ac: remove AC_CHECK_FILE
The use of AC_CHECK_FILE and AC_CHECK_FILES cause the following error
when cross-compiling:

  configure: error: cannot check for file existence when cross compiling

The solution is to check for the file directly instead of using a macro.

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
6 years ago
Ciprian c9cd7559fd configure: fixing rust/cargo cross compile command
adding --target argument to cargo command line when cross compiling
6 years ago
Victor Julien 51ad701d8e version: starting work on 5.0.1 6 years ago
Jason Ish 412ae11bad automake: use tar-ustar for longer filenames
According to the automake manual it should be considered
portable these days.

https://www.gnu.org/software/automake/manual/html_node/List-of-Automake-options.html

Required for the dist generation with Rust vendoring.
6 years ago
Fabrice Fontaine b026fbb519 configure.ac: fix static build with pcap
pcap can depends on nl-3 so use pkg-config to find these dependencies
otherwise all AC_CHECK_LIB calls will fail when building statically

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
6 years ago
Jason Ish 03da49bfaa suricata-update: don't install if requirements not met
Don't try to run suricata-update if its not installed.

The 'make install-rules' target would try to run suricata-update
when it was detected that it was bundled, but didn't consider
if suricata-update was actually installed.
6 years ago
Victor Julien 728d19eaac configure: don't print ERROR if we don't exit 6 years ago
Jason Ish c44f82cf4c configure: fix python major version check on python 2.6
Python 2.6 doesn't use a named tuple for the version info,
instead use the index of the major version which works
on Python 2.6 upwards.
6 years ago
Jason Ish 389272f4c7 rustup: handle rustup for sudo and su
If rustup is in use, and a user uses sudo or su for the make
install, the install may fail with a "no default toolchain"
error.

To prevent this, detect at configure if rustup is being used,
then set RUSTUP_HOME for all calls to cargo.
6 years ago
Jason Ish a1ee536daa configure: no, followed by reason for python tools
This:
  Install suricatactl:                     no, requires distutils
instead of this:
  Install suricatasc:                      requires distutils
6 years ago
Jason Ish 109cf36866 configure: generic instructions for missing python modules
Instead of telling the user what packages to install for missing
Python modules, give generic instructions about what module
needs to be installed.

It is getting tricky to get these package names correct
across distributions.
6 years ago
Jason Ish c4b856ea99 configure: detect python major version
For informational purposes only when notifying what Python
modules are required during ./configure.
6 years ago
Jason Ish 00ad7a911f configure: don't detect python version
Don't detect the Python version, it is not needed anyways,
all we need is the Python path.

Also, python2 --version prints to stderr, while python3
prints to stdout, leading to some odd output during
./configure (but fixable).
6 years ago
Victor Julien ea3d9c3230 htp: require 0.5.31 6 years ago
Fabrice Fontaine 61becb29bf configure.ac: fix --disable-geoip
$enableval should be used to know if the user has passed --enable-geoip
or --disable-geoip

Fixes:
 - http://autobuild.buildroot.org/results/a7a34f760ae5fe0922fdb720b8234dbcd85ed222

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
6 years ago
Victor Julien 1e50b2e404 lua: fix lua int size detection
Failed to work with non-bundled htp and with some stricter
compile flags.
6 years ago
Jason Ish f9c9548b74 configure: detect lua integer size
Lua 5.1 and 5.3 use a different integer size. Run a test program
to set the integer size used in the Rust FFI layer to Rust.
6 years ago
Jason Ish 5f1c851716 configure: remove unused LUA_PC_NAME.
This variable is no longer used. Instead multiple
lua pkg-config names are checked.
6 years ago
Victor Julien 2da90a1cd8 posix: remove deprecated index/rindex calls
Replace index by strchr and rindex by strrchr.

index(3) states "POSIX.1-2008 removes the specifications of index() and
rindex(), recommending strchr(3) and strrchr(3) instead."

Add index/rindex to banned function check so they don't get reintroduced.

Bug #1443.
6 years ago
Philippe Antoine af4f816204 http: sets compression bomb limit 6 years ago
Philippe Antoine 94aa36df1b lzma: replaces liblzma with own sdk for swf decompression
so as to avoid memory exhaustion
6 years ago