Commit Graph

9202 Commits (1bb8fcecec825de94ed5cd1f85f4a185e98f2169)
 

Author SHA1 Message Date
Victor Julien 1bb8fcecec detect/template: switch to v2 API, add MPM 7 years ago
Victor Julien 234d113838 detect/template: clean up unittest 7 years ago
Jacob Masen-Smith b1b45a54c5 detect/analyzer: disable automatic json output
EngineAnalysisRules2 was in a strange location where it did not respect
the --engine-analysis flag. It has been moved to the same call location
as EngineAnalysisRules.
7 years ago
Victor Julien 64d75496b8 detect/analyzer: add notes (and warnings) 7 years ago
Victor Julien e02b74dee7 http: implement min size stream logic
Update HTTP parser to set the min inspect depth per transaction. This
allows for signatures to have their fast_pattern in the HTTP body,
while still being able to inspect the raw stream reliably with it.

The inspect depth is set per transaction as it:
- depends on the per personality config for min inspect size
- is set to the size of the actual body if it is smaller

After the initial inspection is done, it is set to 0 which disables
the feature for the rest of the transaction.

This removes the rescanning flush logic in commit
7e004f52c6 and provides an alternative
fix for bug #2522. The old approach caused too much rescanning of
HTTP body data leading to a performance degradation.

Bug #2522
7 years ago
Victor Julien 7186ce7b99 stream: introduce min inspect depth logic
Some rules need to inspect both raw stream data and higher level
buffers together. When this higher level buffer is a streaming
buffer itself, the risk of mismatch exists.

This patch allows an app-layer parser to set a 'min inspect depth'.
The value is used by the stream engine to keep at least this
depth worth of data, so that the detection engine can request
all of it for inspection.

For rules that have the SIG_FLAG_FLUSH flag set, data is inspected
not from offset raw_progress, but from raw_progress minus
min_inspect_depth.

At this time this is only used for sigs that have their fast_pattern
in a HTTP body and have raw stream match as well.
7 years ago
Jason Ish 9b86c7c5c0 defrag: break out of loop in linux profile when able to 7 years ago
Jason Ish aa98678662 defrag: remove fragments that have complete overlap
Instead of just marking fragments that have been completely
overlapped and won't be part of the assembled packet, remove
them from the fragment tree when detected.
7 years ago
Jason Ish fe6e96a8c1 defrag: use rb tree to store fragments 7 years ago
Victor Julien 023a2fe9ab unittests: fix format-truncation warning 7 years ago
Victor Julien 269313a53e stream/segments: change packing to reduce size
Change the way fields are ordered to reduce TcpSegment structure
with 8 bytes.
7 years ago
Victor Julien b6b9b56e45 stream/segments: keep track of tree right edge
Use this in places where we need to use the outer right
edge of our sequence space.

This way we can avoid walking the tree to find this, which
is a potentially expensive operation.
7 years ago
Victor Julien ea771c69af streaming/sbb: convert RB_MIN to 'head' 7 years ago
Victor Julien bbf1f78ffe streaming: keep track of tree 'head' 7 years ago
Victor Julien 450500e667 streaming: use rbtree for stream blocks
Switch StreamBufferBlocks implementation to use RBTREE instead of
a list. This makes inserts/removals and lookups a lot cheaper if
the number of data gaps is large.

Use separate compare functions for inserts and regular lookups.
Inserts care about the offset, while lookups care about the blocks
right edge as well.
7 years ago
Victor Julien 9bda558c59 stream/sack: optimize SACK size handling
Optimize by keeping count during insert/remove instead of
walking the tree per check.
7 years ago
Victor Julien 7ec7d234cc stream/sack: turn SACK record list into rbtree
Convert to rbtree from linked list. These ranges, of which there can
be multiple per packet, are fully controlled by an attacked. The
attacker could craft a stream of packet in such a way that the list
would grow very large. This would make inserts/removals very expensive,
as well as the list walk that is done and size calculation and pruning
operations.

The RBTREE makes inserts/removals much cheaper, at a slight overhead
for 'normal' operations and slightly higher per record memory use.
7 years ago
Victor Julien 51ce03e76a stream/segments: speed up inserts
Don't try to do a 'fast path' by checking RB_MAX. RB_MAX walks the
tree which means it can be quite expensive. This cost would be paid
for virtually every data segment. The actual insert that follows would
walk the tree again.

Instead, simply insert it. There is a slight cost of the unnecessary
overlap check, but this is much less than the tree walk in a full
tree.
7 years ago
Victor Julien f4ff33969e stream/segments: remove RB_MIN/RB_MAX 7 years ago
Victor Julien 00e65e3cfa stream/segments: optimize overlap tree operations
Now that with the RBTREE we have a properly sorted Segment tree,
where with exact SEQ matches the tree is sorted by payload_len
smallest to largest, we can avoid walking backwards when checking
for overlaps. Our direct RB_PREV either overlaps or not and that
is a reliable verdict for the rest of the tree.
7 years ago
Victor Julien 26b5e1ed13 stream/segments: turn linked list into rbtree
To improve worst case performance turn the segments list into a rbtree.
This greatly improves inserts, lookups and removals if the number of
segments gets very large.

The tree is sorted by the segment sequence number as its primary key.
If 2 segments have the same seq, the payload_len (segment length) is
used. Then the larger segment will be places after the smaller segment.
Exact matches are not added to the tree.
7 years ago
Victor Julien 6a0cf0dd74 tree: add scan-build assertions to suppress FPs 7 years ago
Victor Julien 555fb15ab4 tree: add 2-clause BSD licensed tree.h 7 years ago
Victor Julien 9266334430 stream: expand GAP detection 7 years ago
Victor Julien 3a0eca9fde detect/mpm: minor code cleanup 7 years ago
Victor Julien 164252e381 detect/file: fix minor scan-build warnings 7 years ago
Victor Julien 11f213fb80 stream/tcp: be more liberal in last_ack
Don't set even if seq is before next_seq, as this could still be
a valid packet that was sent before the state was reached.
7 years ago
Victor Julien d7dae87e8b stream/tcp: add debug statements to state dispatcher 7 years ago
Mats Klepsland eba0d04171 app-layer-ssl: don't decode empty extensions 7 years ago
Mats Klepsland 4d38d0844b doc: add documentation for Lua function 'TlsGetVersion' 7 years ago
Mats Klepsland 04e78ace0a lua: add function 'TlsGetVersion'
Add another function to get TLS version, since 'TlsGetCertInfo' only
works when a TLS session contains a clear text certificate, which is
not the case in TLSv1.3 or when a session is resumed.
7 years ago
Mats Klepsland 10fcc8d2ca doc: update tls.version documentation 7 years ago
Mats Klepsland df9853b75c detect-tls-version: add support for 'raw' matching
Add support for matching a 'raw' TLS version using a hex string, e.g:

  tls.version:0x7f12;

The above example matches TLSv1.3 draft 16.
7 years ago
Mats Klepsland 4323e7840f detect-tls-version: add support for TLSv1.3 7 years ago
Mats Klepsland e813842731 detect-ssl-version: add support for TLSv1.3 7 years ago
Mats Klepsland db2fc9208a lua: use 'SSLVersionToString' in TlsGetCertInfo() 7 years ago
Mats Klepsland 13918be589 tlslog: add support for TLSv1.3 7 years ago
Mats Klepsland 25fb02da9a output-json-tls: add support for TLSv1.3 7 years ago
Mats Klepsland 97cc3475bf app-layer-ssl: add function to get string from version
Add 'SSLVersionToString' to get string from version.
7 years ago
Mats Klepsland 91acd3831f app-layer-ssl: add support for earlier TLSv1.3 drafts
Add support for TLSv1.3 draft 1 to draft 21.
7 years ago
Mats Klepsland 831ddb62d2 app-layer-ssl: add support for TLSv1.3 from draft 22
Add support for draft 22 to draft 28 and for the final
version (RFC8446) of TLSv1.3.
7 years ago
Mats Klepsland e0ef578c46 app-layer-ssl: add support for session tickets
Add support for logging a session as 'resumed' when using a non-empty
session ticket extension in the client hello record.
7 years ago
Mats Klepsland 21897a4d7a app-layer-ssl: add better session id support
Verify that the session id from both the client hello record and the
server hello record matches before marking the session as 'resumed'.
7 years ago
Mats Klepsland f22bd5a75b app-layer-ssl: decode server hello record
Decoding server hello is needed to do a better implementation of
session resumption.
7 years ago
jason taylor 0f1c8711ce doc: README.md minor fixes
- Capitalization update
- Typo fix
- Spacing update

Signed-off-by: jason taylor <jtfas90@gmail.com>
7 years ago
Victor Julien 93364b9175 flow/timeout: code simplification and cleanup 7 years ago
Victor Julien c8ecca59f8 stream: minor code cleanup 7 years ago
Victor Julien af6f52cc09 rules: hide 'template' from --list-keywords 7 years ago
Victor Julien b0577402b6 rules: hide internal keywords from --list-keywords 7 years ago
Victor Julien 8c7aee92eb flow-manager: fix unittest initialization 7 years ago