Commit Graph

9515 Commits (57285b54d5cf611cee78cef6f428320555cbe90d)
 

Author SHA1 Message Date
Jason Ish 90dfcf4907 rust/gen-c-headers: don't attempt to split empty lines 7 years ago
Jason Ish ee3aba9008 templates: C stub output for Rust logger 7 years ago
Jason Ish 96dc20abb1 templates: C stub template for Rust parser 7 years ago
Victor Julien 789b1474ed detect/template2: setup script 7 years ago
Victor Julien 486054595a detect/template2: template with prefilter (copy of ttl) 7 years ago
Victor Julien 4d0fc67560 decode/template: minor updates 7 years ago
Victor Julien 5ed7e4fb5f setup-app-layer-detect: update for tests/ dir 7 years ago
Victor Julien 452355bb63 setup-simple-detect: update for new tests location 7 years ago
Jason Ish 9da00bebf0 scripts/setup: remove 'ed' based setup scripts
Removes:
- setup-app-layer.sh
- setup-app-layer-logger.sh

These have been replaced by setup-app-layer.py.
7 years ago
Jason Ish e232fcc415 setup-app-layer: rewrite script in Python
The idea being that it is easier to read and maintain than
wrapping ed commands.

This script also merges the parser and logger setup into a single
script, but still allows just the parser, or just the logger
to be generated with flags, --logger and --parser.
7 years ago
Victor Julien a013cece69 app-layer/template: code cleanups 7 years ago
Victor Julien 33914c2f2f detect/template: clean up packet keyword 7 years ago
Victor Julien d3e5c15995 detect/template: move test to own file in src/tests/ 7 years ago
Victor Julien 1bb8fcecec detect/template: switch to v2 API, add MPM 7 years ago
Victor Julien 234d113838 detect/template: clean up unittest 7 years ago
Jacob Masen-Smith b1b45a54c5 detect/analyzer: disable automatic json output
EngineAnalysisRules2 was in a strange location where it did not respect
the --engine-analysis flag. It has been moved to the same call location
as EngineAnalysisRules.
7 years ago
Victor Julien 64d75496b8 detect/analyzer: add notes (and warnings) 7 years ago
Victor Julien e02b74dee7 http: implement min size stream logic
Update HTTP parser to set the min inspect depth per transaction. This
allows for signatures to have their fast_pattern in the HTTP body,
while still being able to inspect the raw stream reliably with it.

The inspect depth is set per transaction as it:
- depends on the per personality config for min inspect size
- is set to the size of the actual body if it is smaller

After the initial inspection is done, it is set to 0 which disables
the feature for the rest of the transaction.

This removes the rescanning flush logic in commit
7e004f52c6 and provides an alternative
fix for bug #2522. The old approach caused too much rescanning of
HTTP body data leading to a performance degradation.

Bug #2522
7 years ago
Victor Julien 7186ce7b99 stream: introduce min inspect depth logic
Some rules need to inspect both raw stream data and higher level
buffers together. When this higher level buffer is a streaming
buffer itself, the risk of mismatch exists.

This patch allows an app-layer parser to set a 'min inspect depth'.
The value is used by the stream engine to keep at least this
depth worth of data, so that the detection engine can request
all of it for inspection.

For rules that have the SIG_FLAG_FLUSH flag set, data is inspected
not from offset raw_progress, but from raw_progress minus
min_inspect_depth.

At this time this is only used for sigs that have their fast_pattern
in a HTTP body and have raw stream match as well.
7 years ago
Jason Ish 9b86c7c5c0 defrag: break out of loop in linux profile when able to 7 years ago
Jason Ish aa98678662 defrag: remove fragments that have complete overlap
Instead of just marking fragments that have been completely
overlapped and won't be part of the assembled packet, remove
them from the fragment tree when detected.
7 years ago
Jason Ish fe6e96a8c1 defrag: use rb tree to store fragments 7 years ago
Victor Julien 023a2fe9ab unittests: fix format-truncation warning 7 years ago
Victor Julien 269313a53e stream/segments: change packing to reduce size
Change the way fields are ordered to reduce TcpSegment structure
with 8 bytes.
7 years ago
Victor Julien b6b9b56e45 stream/segments: keep track of tree right edge
Use this in places where we need to use the outer right
edge of our sequence space.

This way we can avoid walking the tree to find this, which
is a potentially expensive operation.
7 years ago
Victor Julien ea771c69af streaming/sbb: convert RB_MIN to 'head' 7 years ago
Victor Julien bbf1f78ffe streaming: keep track of tree 'head' 7 years ago
Victor Julien 450500e667 streaming: use rbtree for stream blocks
Switch StreamBufferBlocks implementation to use RBTREE instead of
a list. This makes inserts/removals and lookups a lot cheaper if
the number of data gaps is large.

Use separate compare functions for inserts and regular lookups.
Inserts care about the offset, while lookups care about the blocks
right edge as well.
7 years ago
Victor Julien 9bda558c59 stream/sack: optimize SACK size handling
Optimize by keeping count during insert/remove instead of
walking the tree per check.
7 years ago
Victor Julien 7ec7d234cc stream/sack: turn SACK record list into rbtree
Convert to rbtree from linked list. These ranges, of which there can
be multiple per packet, are fully controlled by an attacked. The
attacker could craft a stream of packet in such a way that the list
would grow very large. This would make inserts/removals very expensive,
as well as the list walk that is done and size calculation and pruning
operations.

The RBTREE makes inserts/removals much cheaper, at a slight overhead
for 'normal' operations and slightly higher per record memory use.
7 years ago
Victor Julien 51ce03e76a stream/segments: speed up inserts
Don't try to do a 'fast path' by checking RB_MAX. RB_MAX walks the
tree which means it can be quite expensive. This cost would be paid
for virtually every data segment. The actual insert that follows would
walk the tree again.

Instead, simply insert it. There is a slight cost of the unnecessary
overlap check, but this is much less than the tree walk in a full
tree.
7 years ago
Victor Julien f4ff33969e stream/segments: remove RB_MIN/RB_MAX 7 years ago
Victor Julien 00e65e3cfa stream/segments: optimize overlap tree operations
Now that with the RBTREE we have a properly sorted Segment tree,
where with exact SEQ matches the tree is sorted by payload_len
smallest to largest, we can avoid walking backwards when checking
for overlaps. Our direct RB_PREV either overlaps or not and that
is a reliable verdict for the rest of the tree.
7 years ago
Victor Julien 26b5e1ed13 stream/segments: turn linked list into rbtree
To improve worst case performance turn the segments list into a rbtree.
This greatly improves inserts, lookups and removals if the number of
segments gets very large.

The tree is sorted by the segment sequence number as its primary key.
If 2 segments have the same seq, the payload_len (segment length) is
used. Then the larger segment will be places after the smaller segment.
Exact matches are not added to the tree.
7 years ago
Victor Julien 6a0cf0dd74 tree: add scan-build assertions to suppress FPs 7 years ago
Victor Julien 555fb15ab4 tree: add 2-clause BSD licensed tree.h 7 years ago
Victor Julien 9266334430 stream: expand GAP detection 7 years ago
Victor Julien 3a0eca9fde detect/mpm: minor code cleanup 7 years ago
Victor Julien 164252e381 detect/file: fix minor scan-build warnings 7 years ago
Victor Julien 11f213fb80 stream/tcp: be more liberal in last_ack
Don't set even if seq is before next_seq, as this could still be
a valid packet that was sent before the state was reached.
7 years ago
Victor Julien d7dae87e8b stream/tcp: add debug statements to state dispatcher 7 years ago
Mats Klepsland eba0d04171 app-layer-ssl: don't decode empty extensions 7 years ago
Mats Klepsland 4d38d0844b doc: add documentation for Lua function 'TlsGetVersion' 7 years ago
Mats Klepsland 04e78ace0a lua: add function 'TlsGetVersion'
Add another function to get TLS version, since 'TlsGetCertInfo' only
works when a TLS session contains a clear text certificate, which is
not the case in TLSv1.3 or when a session is resumed.
7 years ago
Mats Klepsland 10fcc8d2ca doc: update tls.version documentation 7 years ago
Mats Klepsland df9853b75c detect-tls-version: add support for 'raw' matching
Add support for matching a 'raw' TLS version using a hex string, e.g:

  tls.version:0x7f12;

The above example matches TLSv1.3 draft 16.
7 years ago
Mats Klepsland 4323e7840f detect-tls-version: add support for TLSv1.3 7 years ago
Mats Klepsland e813842731 detect-ssl-version: add support for TLSv1.3 7 years ago
Mats Klepsland db2fc9208a lua: use 'SSLVersionToString' in TlsGetCertInfo() 7 years ago
Mats Klepsland 13918be589 tlslog: add support for TLSv1.3 7 years ago