Commit Graph

105 Commits (7bec54158fc09a772457c52448cfc4fc6cc743f6)

Author SHA1 Message Date
Eric Leblond 91e1256b01 af-packet: add support for eBPF cluster and filter
This patch introduces the ebpf cluster mode. This mode is using
an extended BPF function that is loaded into the kernel and
provide the load balancing.

An example of cluster function is provided in the ebpf
subdirectory and provide ippair load balancing function.
This is a function which uses the same method as
the one used in autofp ippair to provide a symetrical
load balancing based on IP addresses.

A simple filter example allowing to drop IPv6 is added to the
source.

This patch also prepares the infrastructure to be able to load
and use map inside eBPF files. This will be used later for flow
bypass.
8 years ago
Jason Ish 34811cf69e json-vars: rename to metadata and use new metadata format 8 years ago
Clement Galland b9cf49e933 rust/tftp: add tftp parsing and logging
TFTP parsing and logging written in Rust.
Log on eve.json the type of request (read or write), the name of the file and
the mode.

Example of output:
    "tftp":{"packet":"read","file":"rfc1350.txt","mode":"octet"}
8 years ago
Victor Julien 11cb84ad35 detect: profiling update for new detect code 8 years ago
Jason Ish f7c3f30186 filestore v2: use fileinfo records as metadata
As fileinfo records are logged to the main eve log, disable
metadata by default. But when enabled, just use the fileinfo
record.

Metadata is stored in a file named:
  <sha256>.<seconds>.<file_id>.json

where the sha256 is the same as the file logged, the seconds
is the unix timestamp in seconds for the fileinfo record,
and the file_id is an atomically incremented integer per
Suricata instance.

This should allow for each occurrence of the same file to have
its own metadata file. But a collision is expected when running
Suricata repeatedly over the same pcap, as that would be the
exact same occurrence of a file.
8 years ago
Jason Ish de2fffca5e util: move SCCreateDirectoryTree to util-path
Renames SCLogCreateDirectoryTree to SCCreateDirectoryTree
and move into a util module for re-use.

Also moves SCMkDir from suricata-common.h to the more
appropriately names util-path.h.

I would have prefered to use util-file for file related options
but that is already used by file store utilities. util-path
is close enough for file related operations.
8 years ago
Victor Julien d634140fa2 logging: unique id's per log direction
For loggers that register once per direction, use unique id's per
direction.

Reshuffle id's to keep tx log id's low so we can use u32 for tracking
logged loggers.
8 years ago
Victor Julien 46cb00ec6c strptime: add implementation from NetBSD
As MinGW doesn't come with strptime take the BSD licensed
implementation from NetBSD. More specifically, the one from

https://github.com/Alexpux/MINGW-packages/blob/master/mingw-w64-libkml/strptime.c

It's slightly modified to get rid on 'uint'.
8 years ago
Victor Julien 11be9bd971 mingw: add SCNtohl and SCNtohs macro's
On MinGW the result of ntohl needs to be casted to uint32_t and
the result of ntohs to uint16_t. To avoid doing this everywhere
add SCNtohl and SCNtohs macros.
8 years ago
Victor Julien d8ddd3b5bc mingw: work around mingw mkdir
mingw doesn't come with a posix compliant mkdir as it only takes
a single argument.
8 years ago
Dana Helwig 3ab9120821 source-pcap-file: Pcap Directory Mode (Feature #2222)
https://redmine.openinfosecfoundation.org/issues/2222

Pcap file mode that when passed a directory will process all files in
that directory. If --pcap-file-continuous or continuous option is passed
in json, the directory will be monitored  until the directory is
moved/deleted, suricata is interrupted, or the pcap-interrupt command
is used with unix command socket. Existing file implementation and new
directory implementation has moved from source-pcap-file into
pcap-file-helper and pcap-directory-helper.

Engine state will not reset between files.

Also satisfies:
 * https://redmine.openinfosecfoundation.org/issues/2299
 * https://redmine.openinfosecfoundation.org/issues/724
 * https://redmine.openinfosecfoundation.org/issues/1476

Co-Authors: Dana Helwig <dana.helwig@protectwise.com> and
Danny Browning <danny.browning@protectwise.com>
8 years ago
Victor Julien 6ce45bcf38 pfring: various build issues
pfring.h brings a different version of likely/unlikely that gives
warnings. So make sure we include our own before.

Make sure pfring.h isn't included globally due to apparent redefinition
of pthread_rwlock_t.
8 years ago
Victor Julien 9b94679fce random: support getrandom(2) if available
Ticket: #2193
8 years ago
Victor Julien 0d79181d78 nfs: rename nfs3 to nfs
Since the parser now also does nfs2, the name nfs3 became confusing.
As it's still in beta, we can rename so this patch renames all 'nfs3'
logic to simply 'nfs'.
8 years ago
Victor Julien d6592211d0 rust/nfs: NFSv3 parser, logger and detection 9 years ago
Victor Julien f47fd2c243 rust/json: expose json_boolean 9 years ago
Victor Julien ab1200fbd7 compiler: more strict compiler warnings
Set flags by default:

    -Wmissing-prototypes
    -Wmissing-declarations
    -Wstrict-prototypes
    -Wwrite-strings
    -Wcast-align
    -Wbad-function-cast
    -Wformat-security
    -Wno-format-nonliteral
    -Wmissing-format-attribute
    -funsigned-char

Fix minor compiler warnings for these new flags on gcc and clang.
9 years ago
Victor Julien e3bd5f371d detect: more detailed state profiling 9 years ago
Victor Julien 8f15e30a58 common: improve byte order and wordsize detection 9 years ago
Jason Ish 2c01985e73 autoconf - look for stdbool.h 9 years ago
Victor Julien 5ca4a2e6fe outputs: vars log
EVE addition called 'vars' that logs pkt/flow vars for each packet/flow.
9 years ago
Victor Julien 15accc86c9 common: add BIT_U8 macro 9 years ago
Victor Julien fe4e119278 common: improve BUG_ON
When BUG_ON is a wrapper for assert(), we risk getting rid of certain
code lines. Assert is a no-op when NDEBUG is defined.

This patch defines an alternate path for BUG_ON that exits after
printing an error.

Bug #2003.
9 years ago
Victor Julien df28c1ac6e common: add WARN_UNUSED macro 9 years ago
Victor Julien 810e43f373 magic: make optional
Make libmagic optional. If installed it will be enabled by default in
configure. Use --disable-libmagic to disable.
9 years ago
Victor Julien 3da7dad514 lua: luajit improvements
Luajit has a strange memory requirement, it's 'states' need to be in the
first 2G of the process' memory.

This patch improves the pool approach by moving it to the front of the
start up.

A new config option 'luajit.states' is added to control how many states
are preallocated. It defaults to 128.

Add a warning when more states are used then preallocated. This may fail
if flow/stream/detect engines use a lot of memory. Add hint at exit that
gives the max states in use if it's higher than the default.
9 years ago
Jason Ish a59f31a99f DNP3: Lua detect support.
Adds support for access the DNP3 transaction in Lua rules.
9 years ago
Jason Ish da40714cb1 common: define json_boolean when not defined
Older versions of jansson in current use don't have this
macro defined.
9 years ago
Victor Julien dba14b676c profiling: more prefilter profiling 9 years ago
Victor Julien 8798bf48b2 profiling: support prefilter engines 9 years ago
Victor Julien 2f5663dfe9 common: introduce BIT_U16 9 years ago
Mats Klepsland 4172c4c8ac tls: add (mpm) keyword tls_cert_subject
This keyword is a replacement for tls.subject.
9 years ago
Mats Klepsland 9b2717799c tls: add (mpm) keyword tls_cert_issuer
This keyword is a replacement for tls.issuerdn.
9 years ago
Jason Ish 04a44a077d logging: convert pcap log to non-thread module 9 years ago
Jason Ish fa27a76462 logging: add profiling back for non-tmm loggers
The loggers moved away from a TMM required a new
profiling support.
9 years ago
Victor Julien ec87123339 configure: check for strings.h: used by SunOS 9 years ago
Victor Julien 5c6ffe5653 common: introduce macro for bit declarations 9 years ago
Mats Klepsland a13df67864 detect: add (mpm) keyword for tls_sni
Match on server name indication (SNI) extension in TLS using tls_sni
keyword, e.g:

alert tls any any -> any any (msg:"SNI test"; tls_sni;
        content:"example.com"; sid:12345;)
10 years ago
Victor Julien ca81c33e14 afl: add --enable-afl configure option 10 years ago
Victor Julien d461837511 afl: add --afl-parse-rules to return 0 on any rule
When fuzzing, AFL will create lots of malformed rules. We don't want
to error out on those. As we're fuzzing the parser any non-crash
should return 0. Crashes (ASAN or not) will return a non-0 code.
10 years ago
Victor Julien e824a8be76 afl: special 'single' runmode
To avoid threading, this 'single' mode doesn't run in it's own thread
but instead runs in the main thread.
10 years ago
Victor Julien a42251d459 afl: add define to disable mgt threads
The inherent non-deterministic nature of the management threads
creates variable test cases.
10 years ago
Victor Julien b2695600ba afl: add define to disable rand_r use
The randomness affects AFL. It creates variable test cases, which
we need to avoid.
10 years ago
Victor Julien 81878c1998 Add global MAX macro. 10 years ago
Victor Julien c0b16fa2bb detect: allow for more than 64k mpm rules 10 years ago
Victor Julien 3f3ed71fe7 jansson: cleanup JSON_ESCAPE_SLASH use 10 years ago
Victor Julien c446abeb47 jansson: include in suricata-common.h 10 years ago
Victor Julien 4086938f1e pool: fix memory leak
Due to pointer size mishandling, the pool code could consider a
block of memory inside the 'preallocated' block. It would then not
free the block.
10 years ago
Victor Julien baadcab1b8 detect: default to u32 for SigIntId 10 years ago
Giuseppe Longo 84dc73d9de mpm: implement prefiltering for smtp 11 years ago