Commit Graph

6995 Commits (96b8100a511b7abeca5a2b801ba26e8a7daef892)

Author SHA1 Message Date
Victor Julien 58e1180efe detect: inspect engine setup cleanup 9 years ago
Victor Julien debc1a6334 detect: dce test fixes and improvements 9 years ago
Victor Julien a2b521b7fa detect-csum: redo tests 9 years ago
Victor Julien f370e88135 detect: move init only Signature members to init_data 9 years ago
Victor Julien 0a5ae415b8 detect: shrink Signature::sm_arrays
Signature::sm_arrays now only contains 'built-in' lists, and so is
sized appropriately.
9 years ago
Victor Julien 4978a7a133 detect: reorganize id's in prep of dynamic lists 9 years ago
Victor Julien 59303d1fbb threshold: fix and redo tests 9 years ago
Victor Julien 6f7e4adbe8 detect: improve memory handling & comments 9 years ago
Victor Julien 8edc954e82 detect: get rid of Signature::sm_lists
Instead use the lists in init_data during setup and the SigMatchData
arrays during runtime.
9 years ago
Victor Julien f281481b67 detect: use detect list passed to generic funcs
Until now the GenericList users used hardcoded list id's.
9 years ago
Victor Julien bd456076a8 detect: pass SigMatchData to inspect functions 9 years ago
Victor Julien a0fe67a3c0 detect: template list in engine 9 years ago
Victor Julien da7c816c7c detect: enip/cip list in engine 9 years ago
Victor Julien e94a7bddb3 detect: modbus list in engine 9 years ago
Victor Julien 7f7d4296da detect: file list in engine 9 years ago
Victor Julien f5adccba1d detect: app-event list in engine 9 years ago
Victor Julien 747dbf92ce detect: dns & tls lists in engine 9 years ago
Victor Julien 5a2e568385 detect: http lists in engine 9 years ago
Victor Julien 1ee32da2ca detect-engine: memory handling of sm_lists
For lists that are registered multiple times, like http_header and
http_cookie, making the engines owner of the lists is complicated.
Multiple engines in a sig may be pointing to the same list. To
address this the 'free' code needs to be extra careful about not
double freeing, so it takes an approach to first fill an array
of the to-free pointers before freeing them.
9 years ago
Victor Julien f81b90dacd detect: when freeing sig also see sm in inspect engine 9 years ago
Victor Julien 2f87c975d4 detect: add SigMatch arg to inspect functions 9 years ago
Victor Julien cf42fbf51f detect: use InspectEngineFuncPtr in inspect engines
Replace explicit function pointer use by InspectEngineFuncPtr typedef
9 years ago
Victor Julien 5f7e096be4 detect: shrink inspect engine by using 'id' as state flag 9 years ago
Victor Julien 715ff60087 detect: remove unused SIG_FLAG_INIT_PAYLOAD init_flag 9 years ago
Victor Julien 859cb89c7e detect alert/threshold/tag: sm_list -> sm_array 9 years ago
Victor Julien 99580487e5 detect: fix file_data / http_server_body tests 9 years ago
Victor Julien faadec0d7f detect file_data: improve error messages 9 years ago
Victor Julien e2c6e1be33 detect-parse: set ipprotos earlier
A high level proto like HTTP implies TCP. However this wasn't set
until after all the parsing was complete which means that keywords
couldn't test if the ipproto matched.

This patch populates the ipprotos right when the higher level proto
is parsed.
9 years ago
Victor Julien 5e0b0eea4b detect: remove unused flags 9 years ago
Victor Julien 39613778cd detect: make setup/free/match funcs static where possible 9 years ago
Victor Julien bfd4bc8233 detect: constify Signature/SigMatch use at runtime 9 years ago
Victor Julien a44da9f5cb detect: simplify SIG_FLAG_STATE_MATCH set logic 9 years ago
Victor Julien be3ee5330f detect: remove alproto from keyword registration
It was already marked as depricated and no longer in use anywhere.
9 years ago
Victor Julien 113a238e90 Open 4.0 development branch 9 years ago
Victor Julien 71710f088e dns: fix outputs with 0-len A/AAAA records 9 years ago
Victor Julien 20990f7a7e dns: fix out of bounds read
On a zero size A or AAAA record, 4 or 16 bytes would still be
read.

Found with AFL+ASAN.
9 years ago
Jason Ish 4a04f814b1 defrag - take protocol into account during re-assembly
The IP protocol was not being used to match fragments with
their packets allowing a carefully constructed packet
with a different protocol to be matched, allowing re-assembly
to complete, creating a packet that would not be re-assembled
by the destination host.
9 years ago
Victor Julien 292baf0872 afl: add ethernet and erspan entry points 9 years ago
Victor Julien 49c41fc79e afl: clean up commandline parsing 9 years ago
Victor Julien b56b04f84c afl: pass a packet queue to decoder calls 9 years ago
Jason Ish 35488eefda afl: set the packet data so pktlen gets set 9 years ago
Victor Julien fbd69729aa afl: improve packet fuzz testing
Due to the use of AFL_LOOP and initialization/deinit outside of it,
part of the fuzzing relied on the global 'state' in flow and defrag.
Because of this crashes that were found could not be reproduced. The
saved crash input was only the last in the series.

This patch addresses that. It requires a new output directory 'dump'
where the packet fuzzers will store all their input. If the AFL_LOOP
fails the files will not be removed and this 'serie' can be read
again for reproducing the issue.

e.g.: AFL would work with:
--afl-decoder-ppp=@@

and after a crash is found the produced serie can be read with:
--afl-decoder-ppp-serie=1486656919-514163

The series have a timestamp as name and a suffix that controls the
order in which the files will be 'replayed' in Suricata.
9 years ago
Victor Julien 923d93f314 afl: add decoder ipv4 option 9 years ago
Sascha Steinbiss 5e96977983 mpm-ac: fix integer overflow on allocation
The size of a memory buffer to be allocated was kept in a signed int
instead of a size_t, leading to an overflow when large lists of long
and diverse patterns cause the amount of AC states to blow up (>2GB).
Fixes Redmine issues #1827 and #1843.

Signed-off-by: Sascha Steinbiss <sascha@steinbiss.name>
9 years ago
Sascha Steinbiss b25b067d93 alert: silence compiler type warning
The `ts_ecr' and `ts_val' struct fields are integer types, not
pointers. This leads GCC 6.3.0 to complain about comparisons to
NULL.

Signed-off-by: Sascha Steinbiss <sascha@steinbiss.name>
9 years ago
Victor Julien 86222428dd detect: don't run IP inspection on non-IP packets
The code to get the rule group (sgh) would return the group for
IP proto 0 instead of nothing. This lead to certain types of rules
unintentionally matching (False Positive).

Since the packets weren't actually IP, the logged alert records
were missing the IP header.

Bug #2017.
9 years ago
Victor Julien 4683b0e662 afl: fix ENIP, switch DNS to UDP and add --afl-dnstcp* 9 years ago
Victor Julien c89ce17017 afl: with -Wshadow issues 9 years ago
Eric Leblond ecf59be413 af-packet: add VLAN header when needed in IPS mode
When packet is coming from a real ethernet card, the kernel is
stripping the vlan header and delivering a modified packet so
we need to insert the VLAN header back before sending the packet
on the wire.

To do so, we pass an option to the raw socket to add a reserve
before the packet data. It will get Suricata some head room to
to move the ethernet addresses before there actual place and
and insert the VLAN header in the correct place.

We get VLAN info from the ring buffer as the call of AFPWrite is
always done in the release function so we still have access to the
memory.
9 years ago
Eric Leblond f407d77016 detect-tls-sni: add link to documentation 9 years ago
Eric Leblond 1af713d67d detect-tls: add url field pointing to doc 9 years ago
Eric Leblond 0695ad4bf0 detect-xbits: set documentation URL 9 years ago
Jason Ish 21bbac5648 dns-log: log requests even when there is no response
The JSON logger had already been updated to handle
transactions without a response. Apply the same logic
to the older dns-log where a logger is registered
for each direction.

Fixes issue 2012.
9 years ago
Andreas Herz d8b5bf9bc6 app-layer-parsing: detect malformed input
If the app-layer-parsing has a very long content it exceeds the maximum
defined in "alproto_name". This adds a check for the too long content
before it will be passed to "strlcpy" and logs an error.
9 years ago
Victor Julien f91d490d25 detect: remove unused flow_locked hint 9 years ago
Victor Julien 31a96d5a79 detect: make tenant loading less verbose 9 years ago
Victor Julien addf64f1f7 profiling: fix memory leaks 9 years ago
Victor Julien 6e876182d7 detect: use TLS_STATE_CERT_READY in cert inspect 9 years ago
Victor Julien 473dae75b5 tls: introduce 'cert ready' state 9 years ago
Victor Julien 15accc86c9 common: add BIT_U8 macro 9 years ago
Sascha Steinbiss e6044aaf1c mpm/spm: check for SSSE3 and enable/disable HS
The new Hyperscan 4.4 API provides a function to check for SSSE3
presence at runtime. This allows us to fall back to non-Hyperscan
matchers on systems without SSSE3 even when the suricata executable
is built with Hyperscan support. Addresses Redmine issue #2010.

Signed-off-by: Sascha Steinbiss <sascha@steinbiss.name>
Tested-by: Arturo Borrero Gonzalez <arturo@debian.org>
9 years ago
Victor Julien a0580d8805 stream: initialize stream segment pool from mtu
If segments section in the yaml is ommitted (default) or when the
pool size is set to 'from_mtu', the size of the pool will be MTU
minus 40. If the MTU couldn't be determined, it's assumed to be
1500, so the segment size for the bool will be 1460.
9 years ago
Victor Julien 1ba15d3721 mtu: track max mtu for capture devices 9 years ago
Victor Julien 7ca466c598 shutdown: remove pid file last 9 years ago
Victor Julien 816dd7b301 startup: clean up main loop 9 years ago
Victor Julien 2eec07cc3a unittests: clean up registration and startup 9 years ago
Victor Julien f452df761a shutdown: move global shutdown steps into func 9 years ago
Victor Julien babe8a299e startup/shutdown: cleanup and unify with unix mode 9 years ago
Victor Julien 3c64cfb384 threads: fix missed logging at shutdown
At shutdown, all flows that still need work are handled by the flow
force reassembly logic. This means one or more flow end pseudo packets
are generated and pushed through the engine for final detection and
logging.

In some cases this would not work correctly. This was caused by the
flow timeout logic kicking in before all the 'live' packets were
processed. Before the flow timeout handling runs the receive threads
are disabled, however the engine did not wait for the in-flight
packets to be fully processed. In autofp mode, packets could still
be in the queue between receive thread(s) and flow worker(s).

This patch adds a new function that 'drains' all the packet threads
of any in-progress packets before moving on the flow timeout logic.

Bug #1946.
9 years ago
Mats Klepsland a2659ed7ec output-json-flow: add has_alerts field
Add has_alerts field to flow eve-log to indicate if a flow has
any alerts or not.
9 years ago
Mats Klepsland c531e8f77c lua: add SCFlowHasAlerts function
Add SCFlowHasAlerts() to check if a flow has alerts. Returns true
on alerts, false otherwise.

Example:

  has_alerts = SCFlowHasAlerts()
  if has_alerts then
    -- do something
  end
9 years ago
Mats Klepsland d9b87e502d flow: set flag to indicate that a flow has alerts
Set FLOW_HAS_ALERTS flag on the flow on alerts. Add FlowHasAlerts(..)
and FlowSetHasAlertsFlag(..) to check and set this flag.
9 years ago
Eric Leblond 569cc5d238 util-file: introduce new functions for file size
This patch introduces the FileDataSize and FileTrackedSize functions.
The first one is just a renaming of the initial FilSize function
whereas the other one is using the newly introduced size field as
value.
9 years ago
Eric Leblond a098896b28 output-json-file: use size instead of FileSize
FileSize is not returning the actual value when file store is not
used.
9 years ago
Eric Leblond fbc2dbac28 util-file: change file size computation
The file size returned by FileSize is invalid if file store is not
used so we introduce a new size field in File structure that is used
to store the size.
9 years ago
Alexander Gozman 187a6f392c Bug #2009: added CAP_NET_ADMIN for PCAP and af-packet modes.
Without this capability suricata is unable to get network
interface's settings.
9 years ago
Jason Ish 19e578a740 pcap-log: fix pcre_study error check
Code was failing on a NULL return value which can be returned
when there was nothing todo instead of an error. Instead
check the errbuf for a non-NULL value to determine error.
9 years ago
Jason Ish 5c55373679 app-layer-detect-proto.c: fix indent
A recent commit was outdented by 1 column.
9 years ago
Jason Ish ec44585dca app-layer - fix secondary probing parser logic
Apply the same logic to pe2 as pe1 for determining which
probe to call. Missed in previous commit.
9 years ago
Victor Julien cb36dee477 hyperscan: fix minor coverity issue in error path
*** CID 1398951:  API usage errors  (LOCK)
/src/util-mpm-hs.c: 722 in SCHSPreparePatterns()
716         SCMutexUnlock(&g_db_table_mutex);
717
718         SCHSFreeCompileData(cd);
719         return 0;
720
721     error:
>>>     CID 1398951:  API usage errors  (LOCK)
>>>     "pthread_mutex_unlock" unlocks "g_db_table_mutex" while it is unlocked.
722         SCMutexUnlock(&g_db_table_mutex);
723         if (pd) {
724             PatternDatabaseFree(pd);
725         }
726         if (cd) {
9 years ago
Victor Julien fa8cbd8741 smb: detect protocol in both directions 9 years ago
Jason Ish e9fccfa67c tx logging: only update logged tx id if all loggers logged
Prevents the case where the logged id is incremented if a newer
transaction is complete and an older one is still outstanding.

For example, dns request0, unsolicited dns response, dns response0

would result in the valid response0 never being logged.

Similarily this could happen for:
  request0, request1, response1, response0

which would end up having request0, request1 and response1 logged,
but response0 would not be logged.
9 years ago
Eric Leblond 0d5fd0f658 util-file: fix error logic in hash computation
This patch fixes an issue with hash computation resulting in the
invalidity of at least one hash when at least two different hashes
functions were used.

Impact was setting as `force-hash: [md5, sha256]` not to be valid.
Also it could lead to false negative if too different hash functions
had to be used on a single file due to signatures.
9 years ago
Jason Ish 20111cab23 unix-socket: fix shadowed variable
ret does not need to be redefined here, the existing
declaration of ret can be used.
9 years ago
Victor Julien f964cdbc93 address parsing: fix memory leak in error path 9 years ago
Victor Julien a6fccd952e ssl: suppress scan-build warnings 9 years ago
Victor Julien 61b72c6981 output: clean up output function
Don't allocate memory per call.
9 years ago
Victor Julien 709d20f8c6 smb/dcerpc: suppress scan-build warnings 9 years ago
Victor Julien 618ab4e177 ac-bs: fix scan-build warnings 9 years ago
Victor Julien bbc02205fb queue: add debug assertions to TAILQ
To avoid scan-build fp's add assertions that are only active if
built with scan-build.
9 years ago
Mats Klepsland 03ad9d4ec0 tls-store: fix bug that causes Suricata to crash
Fix bug that causes Suricata to crash when the tls.store keyword is used.

*** Error in `/usr/bin/suricata': free(): invalid next size (fast):
0x00007fd4b4373180 ***
9 years ago
Jason Ish 87b5bf9541 proto detect - fix coverity CID 1204325
CID 1204325 (#1 of 1): Logically dead code (DEADCODE)
dead_error_line: Execution cannot reach this statement: mask = 0U;.
433        mask = 0;

additionally, mask is initialized to 0
9 years ago
Jason Ish d09cd16c8c template logger - fix coverity CID 1324964
null: At condition templatejs != NULL, the value of templatejs must be
NULL.
dead_error_condition: The condition templatejs != NULL cannot be true.
113    if (templatejs != NULL) {
CID 1324964 (#1 of 1): Logically dead code (DEADCODE)
dead_error_line: Execution cannot reach this statement:
json_decref(templatejs);.
114        json_decref(templatejs);
115    }
9 years ago
Jason Ish a10a9220cf dns (tcp) - fix coverity CIDs 1374306, 1374305
CID 1374306 (#1 of 1): Dereference before null check (REVERSE_INULL)
check_after_deref: Null-checking dns_state suggests that it may be null,
but it has already been dereferenced on all paths leading to the check.
585    if (dns_state != NULL && f != NULL) {
586        dns_state->last_req = f->lastts;
587    }

CID 1374305 (#1 of 1): Dereference before null check (REVERSE_INULL)
check_after_deref: Null-checking dns_state suggests that it may be null,
but it has already been dereferenced on all paths leading to the check.
366    if (dns_state != NULL && f != NULL) {
367        dns_state->last_req = f->lastts;
368    }
9 years ago
Jason Ish dfbfb50f64 dns (tcp) - fix coverity cid 1374307
CID 1374307 (#1 of 1): Dereference before null check (REVERSE_INULL)
check_after_deref: Null-checking dns_state suggests that it may be null,
but it has already been dereferenced on all paths leading to the check.
317    if (dns_state != NULL && f != NULL) {
318        dns_state->last_resp = f->lastts;
319    }
9 years ago
Eric Leblond 5b1de57d73 detect-parse: simplify port prefiltering
Regular expression was not matching some authorized setting like
"![1234, 1235]". This patch simplify the regexp to match on
possible character and let the port parsing code handle the
complete verification.
9 years ago
Jason Ish b0de5ad1a8 dns: increment tx id when allocated during response 9 years ago
Victor Julien fe4e119278 common: improve BUG_ON
When BUG_ON is a wrapper for assert(), we risk getting rid of certain
code lines. Assert is a no-op when NDEBUG is defined.

This patch defines an alternate path for BUG_ON that exits after
printing an error.

Bug #2003.
9 years ago
Andreas Herz 98e8b13bf0 decode-icmpv6: add missing types
There have been some ICMPv6 types missing within the DecodeICMPV6 that
are added by this commit and the code check is adjusted to always use
the DEFINE.
9 years ago
Jason Ish bcdbd12839 dns (tcp): register a to_client (response) probing parser
Just a minimal parser to make sure the data contains at
least a header.
9 years ago