aiden
/
suricata
mirror of https://github.com/OISF/suricata
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,292 Commits
8 Branches
165 Tags
229 MiB
main
main-7.0.x
main-8.0.x
master-6.0.x
master-5.0.x
master-4.1.x
master-4.0.x
master-3.2.x
suricata-8.0.2
suricata-7.0.13
suricata-8.0.1
suricata-7.0.12
suricata-8.0.0
suricata-7.0.11
suricata-8.0.0-rc1
suricata-8.0.0-beta1
suricata-7.0.10
suricata-7.0.9
suricata-7.0.8
suricata-7.0.7
suricata-7.0.6
suricata-6.0.20
suricata-7.0.5
suricata-6.0.19
suricata-6.0.18
suricata-7.0.4
suricata-6.0.17
suricata-7.0.3
suricata-6.0.16
suricata-7.0.2
suricata-6.0.15
suricata-7.0.1
suricata-6.0.14
suricata-7.0.0
suricata-7.0.0-rc2
suricata-6.0.13
suricata-6.0.12
suricata-6.0.11
suricata-7.0.0-rc1
suricata-6.0.10
suricata-6.0.9
suricata-7.0.0-beta1
suricata-6.0.8
suricata-6.0.7
suricata-6.0.6
suricata-5.0.10
suricata-6.0.5
suricata-5.0.9
suricata-6.0.4
suricata-5.0.8
suricata-6.0.3
suricata-5.0.7
suricata-6.0.2
suricata-5.0.6
suricata-6.0.1
suricata-5.0.5
suricata-4.1.10
suricata-4.1.9
suricata-5.0.4
suricata-6.0.0
suricata-6.0.0-rc1
suricata-6.0.0-beta1
suricata-5.0.3
suricata-4.1.8
suricata-4.1.7
suricata-5.0.2
suricata-4.1.6
suricata-5.0.1
suricata-5.0.0
suricata-5.0.0-rc1
suricata-4.1.5
suricata-5.0.0-beta1
suricata-4.1.4
suricata-4.1.3
suricata-4.0.7
suricata-4.1.2
suricata-4.1.1
suricata-4.1.0
suricata-4.0.6
suricata-4.1.0-rc2
suricata-4.1.0-rc1
suricata-4.0.5
suricata-4.1.0-beta1
suricata-4.0.4
suricata-4.0.3
suricata-4.0.2
suricata-3.2.5
suricata-4.0.1
suricata-3.2.4
suricata-4.0.0
suricata-4.0.0-rc2
suricata-3.2.3
suricata-4.0.0-rc1
suricata-4.0.0-beta1
suricata-3.2.2
suricata-3.1.4
suricata-3.2.1
suricata-3.2
suricata-3.2RC1
suricata-3.1.3
suricata-3.2beta1
suricata-3.1.2
suricata-3.1.1
suricata-3.1
suricata-3.0.2
suricata-3.1RC1
suricata-3.0.1
suricata-3.0.1RC1
suricata-3.0
suricata-2.0.11
suricata-3.0RC3
suricata-3.0RC2
suricata-3.0RC1
suricata-2.0.10
suricata-2.0.9
suricata-2.1beta4
suricata-2.0.8
suricata-2.0.7
suricata-2.1beta3
suricata-2.0.6
suricata-2.0.5
suricata-2.1beta2
suricata-2.0.4
suricata-2.1beta1
suricata-2.0.3
suricata-2.0.2
suricata-2.0.1
suricata-2.0.1rc1
suricata-2.0
suricata-2.0rc3
suricata-2.0rc2
suricata-2.0rc1
suricata-2.0beta2
suricata-1.4.7
suricata-1.4.6
suricata-1.4.5
suricata-2.0beta1
suricata-1.4.4
suricata-1.4.3
suricata-1.4.2
suricata-1.4.1
suricata-1.3.6
suricata-1.4
suricata-1.3.5
suricata-1.4rc1
suricata-1.3.4
suricata-1.4beta3
suricata-1.3.3
suricata-1.4beta2
suricata-1.3.2
suricata-1.4beta1
suricata-1.3.1
suricata-1.3
suricata-1.3rc1
suricata-1.3beta2
suricata-1.3beta1
suricata-1.2.1
suricata-1.2
suricata-1.2rc1
suricata-1.2beta1
suricata-1.1.1
suricata-0.8.2
suricata-1.0.0
suricata-1.0.1
suricata-1.0.2
suricata-1.0.3
suricata-1.0.4
suricata-1.0.5
suricata-1.1
suricata-1.1beta1
suricata-1.1beta2
suricata-1.1beta3
suricata-1.1rc1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'c446abeb47'
${ noResults }
Commit Graph

5783 Commits (c446abeb471d8be516bf11f0106797526b7122b3)

Author SHA1 Message Date
Victor Julien 4dd53c8f03 lua: dns support 11 years ago
Victor Julien 433e511b63 dns: generic inspect engines for DNS 11 years ago
Victor Julien 110e23964e detect: add AppLayerTxMatch call 11 years ago
Victor Julien e6129f7b47 dns: generic request/response detect lists 11 years ago
Victor Julien 2c8e8c2516 dns: rename type so it's purpose is more clear 11 years ago
Victor Julien a1e50b3138 lua: dns extensions 
Add DNS lua calls for getting queries, answers, authorities. Also
rcode and recusion desired flags.
 11 years ago
Victor Julien c46d472921 lua: initial DNS logging support 11 years ago
Eric Leblond 159a6d1cb4 tls-store: avoid log flooding 
In case we can't write in the certs directory, this is possible
we flood the log for each TLS session or even worse each TLS
packet.  So this patch puts a limit in the number of logged
messages related to file creation.
 11 years ago
Eric Leblond cbf5d88447 filestore: use SCFree instead of free 11 years ago
Eric Leblond b77cd22b98 tls-store: backward compatibility 
This patch implements backward compatibility in suricata.yaml
file. In case the new 'tls-store' output is not present in the
YAML we have to use the value defined in 'tls-log'.
 11 years ago
Eric Leblond 4db0a35f25 tls-store: now a separate module 
An design error was made when doing the TLS storage module which
has been made dependant of the TLS logging. At the time there was
only one TLS logging module but there is now two different ones.

By putting the TLS store module in a separate module, we can now
use EVE output and TLS store at the same time.
 11 years ago
Jeff Barber 893fc9660d Support for reconnecting unix domain socket log files 
Issue #1423
 11 years ago
Jason Ish b512580bbe logging: integrate rotation into SCConfLogOpenGeneric. 
Addresses issue 1492, and will make it harder to omit
rotation on new outputs.
 11 years ago
Jason Ish 14981cb2a8 rule vars: strip leading white space before looking up var. 11 years ago
sfd e58cfb6a05 Fix compile bad dereferences 
The src/source-erf-dag.c code was not compiling. It looks like some stats counters were added but not tested as the dereferences are incorrect.
 11 years ago
Zopieux cd038419fd stream_size operator comparison (fix issue #1488) 
`DetectStreamSizeParse` was first checking if mode[0] is '<', which is true for both '<' and '<=', thus '<=' (and resp. '>=') is never matched. This patch does the `strcmp` to '<=' (resp. '>=') within the if block of '<' (resp. '>') to fix #1488.
 11 years ago
Victor Julien 45fc619f79 logging: json output 
Make JSON output optional.

Cleanups.
 11 years ago
Victor Julien 126ecb3ebf logging: fix per output log formats 11 years ago
Victor Julien b51c4e608f logging: optional colors output 
Construct message per output method.
 11 years ago
Victor Julien b13de5bf08 logging: change newline handling 11 years ago
Victor Julien 1927b3000c output: cleanup 11 years ago
Victor Julien b30bdc21b5 logging: cleanup output API 
Make SCLogMessage master of the logging. Reduces complexity
of the SCLog macro's.
 11 years ago
Victor Julien d6fc6e874f log: reorganize SCLogOPIfaceCtx to make it more efficient 11 years ago
Victor Julien c2f4031a8c detect: fix settings override for reloads 11 years ago
Alexander Gozman cd9cc2559e Issue 1491: fix capabilities for pf_ring mode when running under non-root account 11 years ago
Alexander Gozman d36eba4e5e Fix issues #1493 and #1494 11 years ago
Giuseppe Longo d592d57039 file_data: check for signature alproto and flow 
Currently the following rule can't be loaded:
alert tcp any any -> any 25 (msg:"SMTP file_data test"; flow:to_server,established; file_data; content:"abc";sid:1;)
and produces the error output:
"Can't use file_data with flow:to_server or from_client with http or smtp."

This checks if the alproto is not http in a signature,
so permits to use flow keyword also.

Issue reported by rmkml.
 11 years ago
Victor Julien e583de0582 Minor unittest cleanups 11 years ago
Victor Julien f4f53924bb app-layer: fix coverity warnings 11 years ago
Victor Julien 6c792cb4cc erspan: respect vlan.use-for-tracking setting 11 years ago
Victor Julien b8211e8c04 htp: hide BUG_ON's behind DEBUG_VALIDATION 11 years ago
Victor Julien bd73553027 smtp json: fix potential crash on malloc failure 
** CID 1298888:    (FORWARD_NULL)
/src/output-json-email-common.c: 117 in JsonEmailLogJson()
/src/output-json-email-common.c: 140 in JsonEmailLogJson()
 11 years ago
Victor Julien 1ed8d7b538 detect analizer: fix minor coverity warning 
** CID 1298889:  Integer handling issues  (NEGATIVE_RETURNS)
/src/detect-engine-analyzer.c: 102 in EngineAnalysisFP()
 11 years ago
Victor Julien dd2afd51f7 file_data smtp: fix minor coverity warning 
CID 1298891:  Null pointer dereferences  (REVERSE_INULL)
Null-checking "curr_file" suggests that it may be null, but it has already been dereferenced on all paths leading to the check.
 11 years ago
Victor Julien bcff35fd9f smb: fix coverity warning 
** CID 1298892:  Incorrect expression  (UNUSED_VALUE)
/src/app-layer-smb.c: 1486 in SMBProbingParser()
 11 years ago
Victor Julien aa095864d3 pfring runmode: remove set that is never read 11 years ago
Victor Julien 21db5ee691 counters: reduce global usage 11 years ago
Victor Julien a5168d5977 dce_opnum: improve memory handling on parsing error 11 years ago
Victor Julien b3b7625be5 htp: fix test 11 years ago
Victor Julien c0807c3df5 fast log: clean up tests 11 years ago
Victor Julien 29fbcce50d detect hsbd: simplify resize logic 11 years ago
Victor Julien 4e7cb7b863 app-layer: update all protocols to accept NULL+EOF 
Update all non-HTTP protocol parsers to accept a NULL+EOF input.
 11 years ago
Victor Julien cf9ff6adbd app-layer: improve EOF handling 
On receiving TCP end of stream packets (e.g. RST, but also sometimes FIN
packets), in some cases the AppLayer parser would not be notified. This
could happen in IDS mode, but would especially be an issue in IPS mode.

This patch changes the logic of the AppLayer API to handle this. When no
new data is available, and the stream ends, the AppLayer API now gets
called with a NULL/0 input, but with the EOF flag set.

This allows the AppLayer parser to call it's final routines still in the
context of a real packet.
 11 years ago
Victor Julien df79c1019f print: make PrintRawDataFp take a const arg 11 years ago
Victor Julien 7451d33396 stream: update StreamMsg to don't have fixed size 
StreamMsg would have a fixed size buffer. This patch replaces the buffer
by a dynamically allocated buffer.

Preparation of allowing bigger and customizable buffer sizes.
 11 years ago
Victor Julien 3ffa01d150 stream: remove STREAMTCP_STREAM_FLAG_CLOSE_INITIATED logic 11 years ago
Giuseppe Longo 32563d51d1 http: rework UT 
This reworks UT based on
commit 3203555708
 11 years ago
Victor Julien 573d082219 http: memcap HTTP server inspect body code 11 years ago
Victor Julien 3163a57577 smtp file_data: fix wrong free 11 years ago
Victor Julien 9f52bdd1e5 flow timeout: prevent dead locks 
The flow timeout mechanism called both from the flow manager at run time
and at shutdown creates pseudo packets. For this it has it's own packet
pool, which can be depleted if the timeout logic is faster than the packet
processing threads. In this case the flow timeout would enter a wait loop.
The problem however, is that this wait loop would happen while keeping a
flow locked. This could lead to a race condition when the packet thread(s)
are waiting for the lock that the flow manager has.

This patch introduces a new packet pool call 'PacketPoolWaitForN', meant
to make sure that the thread's packet pool has at least N available
packets. The flow timeout paths use this to make sure enough packets are
available *before* grabbing the flow lock. If there aren't enough packets
available yet, the wait happens before the lock as well.

This still means the wait can happen while the flow hash row is locked, so
we do make sure some more packets are available when entering that. But
perhaps in the future we need a more precise logic there as well.
 11 years ago
Victor Julien 423d5fd0de http: improve inline body tracking 
Rewrite the sliding window handling for IPS mode for the server body.

The buffer used will have the following properties:
    left edge: inspected data - window_size
    right edge: the most recent data

Due to handling of compressed bodies, the data can be much bigger than
the configured window size.
 11 years ago
Victor Julien 79d34f4575 http: body pruning update 
Take inspect limits into account. Consider per direction inspect settings.
 11 years ago
Giuseppe Longo 8f1998e35f hsbd: inspect buffer depending on the engine mode 
Currently, data is buffered up to response-body-minimal size
and response-body-inspect-window before being inspected.
With this, in IPS mode, inspect data as it comes in up.

The sliding window concept is used here,
some data chunks are copied into the window (buffer)
then it's inspected.
 11 years ago
Giuseppe Longo a459376d2e app-layer-htp: add http_body_inline setting 11 years ago
Victor Julien 80615f7d43 ips: move counters in common struct and funcs 11 years ago
Victor Julien 813930f027 nfq: add ips stats 11 years ago
Victor Julien 14466a803d decode: create util function for basic counter updates 11 years ago
Jason Ish e3ce29f694 json-stats: log deltas 
If "deltas" is yes, log delta values as the name of the value
suffixed with _delta.
 11 years ago
Victor Julien 3e14aa729a detect-state: remove/hide BUG_ON statements 11 years ago
Victor Julien b7f1e9e370 file extract: add app_proto to logging 11 years ago
Victor Julien 925aac854e alproto: improve AppProtoToString 
Make AppProtoToString compilation fail on missing 'case's.
 11 years ago
Eric Leblond 0bafc71689 app-layer: add modbus to AppProtoToString 
It was missing causing protocol identified as modbus not to be
displayed in netflow events.
 11 years ago
Victor Julien ebb42f831c suppress: add track by_either mode 
So far suppress rules would apply to src or dst addresses of a packet.
This meant that if a ip would need to suppressed both as src and as dst,
2 suppress rules would be needed.

This patch introduces track by_either, which means that the ip(s) in the
suppress rule are tested against both the packets source and dest ip's.
If either of them is on the suppress list, the alert is suppressed.
 11 years ago
Victor Julien e85a44c383 suppress: support ip-lists 
Ticket: 1137

Support supplying a list of IP's to the suppress keyword. Variables from
the address-groups and negation is supported. The same logic (and code) is
used that is also used in parting the IP portions of regular detection
rules.
 11 years ago
Jason Ish 26fc5682ad hostbits: ignore leading and trailing white space 
Ignore leading and trailing space around the name and
direction tokens.
 11 years ago
Jason Ish 7c40c73482 json-stats: reorg threads and totals 
Totals are now placed at the top level instead of under a "Total"
object.

Threads are placed under a "threads" object.
 11 years ago
Jason Ish 1f2caf78c3 json-stats: log uptime in seconds, instead of a string 11 years ago
Jason Ish 0f1dd0d7ea flowbits: strip leading and trailing spaces in name 
Redmine bug 1481. Strip leading and trailing white space. Factor
out parsing from setup while in here.
 11 years ago
Zachary Rasmor 0edf28a4f8 Add Feature #1454. Generic eve-log prefix support. 11 years ago
Victor Julien a083513c49 decode: optimize DecodeThreadVars layout 
Put common counters on the first cache line. Please the flow output
pointer last as it's use depends on the flow logging being enabled
and even then it's only called very rarely.
 11 years ago
Victor Julien fe5a85aea0 decode: add erspan counter 11 years ago
Victor Julien 928957f0a3 decode: add ERSPANv1 decoder 
Only allow v1 to be parsed as thats what is tested.

Take vlan_id from the ERSPAN layer.
 11 years ago
Victor Julien aa6b24f814 decode: clean up tunnel decode logic 
Don't use mix of existing and custom types to indicate the next
layer.
 11 years ago
Victor Julien ef7cd043cc detect: various header cleanups 11 years ago
Victor Julien 5483b800c5 detect: remove struct/union tricks from Signature 11 years ago
Victor Julien 8949054212 detect: remove unused match_flags from inspect engines 11 years ago
Victor Julien 9fa2f85cc7 http: improve body pruning 
Take inspect window into account.
 11 years ago
Victor Julien 0bbc818b2d http: fix body tracking 
In HTTP body tracking for response bodies, pruning body chunks was broken
as the body parsing code wouldn't update HtpBody::body_parsed.
 11 years ago
Victor Julien 3203555708 http-client-body: create unittest util func 11 years ago
Eric Leblond d837562441 logging: fix modules ordering during logging 
With the previous code the order of the logging modules in the
YAML were determining which module was run first. This was not
wished and a consequences was that the EVE fileinfo module was
not correctly displaying the key 'stored' because it was
depending on a flag set alter by the filestore module.

This patch adds a priority file to the TmModule structure. The
higher the priority is set, the sooner the module is run in the
logging process. The RunModeOutput structure has also been
updated to contain the name of the original TmModule. Thus allowing
to define a priority for a RunModeOutput.

Currently only the filestore has a priority set. The rest of them is
set to the default value of zero.
 11 years ago
Eric Leblond be07620a60 output-lua: sync variable name with yaml 
'script-dir' was used in the code but we had 'scripts-dir' in the
configuration file. This patch fixes it to 'scripts-dir'.
 11 years ago
Jason Ish ae23144b67 --set - handle spaces on either side of '=' 
Discard spaces when provided as part of --set around the '='. For
example, "val=key", "val = key", "val= key" and "val =key" are
all equivalent now.
 11 years ago
Jason Ish d9fe95bc8a conf - function declaration style 
Use consistent style - function return type and declaration on
same line.
 11 years ago
DIALLO David 0a4fd39f9c modbus: fix heap-buffer-overflow in Modbus parser 
Modbus parser does not check length to extract/read data (read or write address,
quantity of data, etc.) that should be present.

In case of malformated data (invalid length in header), Modbus parser reads data
over the input data length.

Add check before extracting/reading data from input buffer to avoid head buffer
overflow.
 11 years ago
Victor Julien 07efec550d counters: use ptr to name instead of copy 
All counters have hardcoded names, so copies are not needed.
 11 years ago
Victor Julien 7e66c70507 counters: don't run if no counters have been registered 11 years ago
Victor Julien cb5aa8f8d5 counters: work around unix-socket init issues 11 years ago
Victor Julien e48153c6b0 counters: make threads cleanup all memory 11 years ago
Victor Julien 81548ae3e8 counters: clean up global context 11 years ago
Victor Julien 84b8829cb4 counters: turn flow.memuse into a global counter 11 years ago
Victor Julien 0a262acdfb counters: make DNS counters globals 11 years ago
Victor Julien ac069c579a counters: make tcp.memuse a global counter 11 years ago
Victor Julien cddbb0f606 http: make http.memuse a global counter 
http.memcap as well.
 11 years ago
Victor Julien f05d0692ef counters: remove references to 'perf' counters 11 years ago
Victor Julien faef92f8da counters: remove last and now unused tm_name reference 11 years ago
Victor Julien 83f27ae2a5 counters: remove old unix socket json logic 11 years ago
Victor Julien 41ead6611a counters: minor internal API cleanups 11 years ago
Victor Julien d2a9ef2680 counters: rename unparsable SCPCAElem to StatsLocalCounter 11 years ago
Victor Julien 4c3ccda72e counters: minor header cleanup 11 years ago
Victor Julien 752f03e7a4 counters: remaining s/SCPerf/Stats/g 11 years ago
Victor Julien 4362d0a6e9 counters: s/SCPerfPrivateContext/StatsPrivateThreadContext/g 11 years ago
Victor Julien 628c3b1bc7 counters: s/SCPerfPublicContext/StatsPublicThreadContext/g 11 years ago
Victor Julien 7e70f136ec counters: various renames and cleanups 11 years ago
Victor Julien 30cce2bd29 counters: s/SCPerfCounterSetUI64/StatsSetUI64/g 11 years ago
Victor Julien 1c0b4ee0ae counters: s/SCPerfCounterIncr/StatsIncr/g 11 years ago
Victor Julien 8992275b0c counters: s/SCPerfCounterAddUI64/StatsAddUI64/g 11 years ago
Victor Julien 60d9eb6790 counters: clean up defines 11 years ago
Victor Julien 1ef786e7cb counters: rename register API calls 
Also remove 'type' parameter which was always the same.
 11 years ago
Victor Julien 3fab736539 log-stats: make global/threads logging configurable 11 years ago
Victor Julien 2c9a2c8327 stats: support per thread stats in json output 
Default is only to output totals. Optionally per thread can be added.

Both can be enabled together.
 11 years ago
Victor Julien 175831331c stats json: replace strndup 
strndup is a banned function.
 11 years ago
Victor Julien 6565c86f96 stats-json: fixes and improvements 
Use proper LogFileCtx and MemBuffer handling so we can have multiple
loggers active at the same time.

Change 'date' field to timestamp, and use ISO notation to make it
the same as the other JSON outputs.
 11 years ago
Tom DeCanio e4e07d0c3b eve-log: stats logging code cleanup. 11 years ago
Tom DeCanio 117eed0385 eve-log: add JSON stats logging 
Support for counters in stats.log in eve output JSON stream.
 11 years ago
Victor Julien 23f17950bc counters: pass per thread stats to output api 
As well as the global (merged) stats.
 11 years ago
Victor Julien de82b6d31e counters: rename widely used pctmi var to sts (stats thread store) 11 years ago
Victor Julien 60c5ad4649 counters: call global counters funcs 11 years ago
Victor Julien 33756abd87 counters: split API init 
Split into early ctx initialization and post-config setup.
 11 years ago
Victor Julien d05eed3735 counters: start using Stats prefix 11 years ago
Victor Julien f300ad253e counters: simplify and speedup counters sync 11 years ago
Victor Julien 0478407833 counters: remove threadvars arg from SCPerfAddToClubbedTMTable 11 years ago
Victor Julien 2346a88db7 counters: remove thread module name from counters API 11 years ago
Victor Julien b5bd3dee13 stream: make tcp.reassembly_memuse counter global 
Fixes bugs #632 and #1178
 11 years ago
Victor Julien 06461e37da counters: global counters registration 11 years ago
Victor Julien 9bbef55c4d Fix harmless typo in IPOnlyCIDRItemNew's SCReturnPtr use 11 years ago
Victor Julien b293a4b7d0 counters: remove unused description 11 years ago
Victor Julien 711cd7b59b counters: merge counters from threads for output 
Merge counters so the table contains combined values from counters
from each thread.

Use global counter id's, track them in a hash.

Rename SCPCAElem members

Fix and improve average counters
 11 years ago
Victor Julien 7da657dc3d counters: remove unused public API calls and make them private 11 years ago
Victor Julien ac6e24c06a counters: make SCPerfSetupPrivate a function 11 years ago
Victor Julien 66635f0741 counters: minor cleanups 11 years ago
Victor Julien 74ab84c194 counters: introduce SCPerfSetupPrivate for thread setup 11 years ago
Victor Julien 799640f906 counters: make threadvars::perf_private_ctx static 
Update SCPerfGetAllCountersArray and add a UT workaround.
 11 years ago
Victor Julien 55cfab89e4 counters: SCPerfGetLocalCounterValue cleanup 
Return u64, update arguments.
 11 years ago
Victor Julien b34c6dc93a counters: remove references to SCPerfCounterAddDouble 
They were all in comments anyway.
 11 years ago
Victor Julien e9b067c1eb counters: make increment call take threadvars 
This hides the implementation from the caller.
 11 years ago
Victor Julien 9a8bff7d96 counters: threadvars s/sc_perf_pca/perf_private_ctx/g 11 years ago
Victor Julien 50bb995458 counters: rename threadvars public counters 11 years ago
Victor Julien 6ffbc3a362 counters: s/SCPerfContext/SCPerfPublicContext/g 11 years ago
Victor Julien 0a5ae1b403 counters: s/SCPerfCounterArray/SCPerfPrivateContext/g 
Goal is to make it's purpose clear.
 11 years ago
Victor Julien 9f584483be counters: minor cleanups 11 years ago
Victor Julien 1e8142c699 logfile: rename ALERT_ types to LOGFILE_TYPE_ 11 years ago
Eric Leblond 4c6a7bea30 output-json: suppress global variable 
It uses the new type field in the LogFileCtx instead.

This fixes the problem of not being able to use two eve-json
instance with different logging methods.
 11 years ago
Eric Leblond 636e3d93c0 log file: add type flag 
It will be used to store if the file is syslog or a real file.
 11 years ago
Eric Leblond 39d667ff56 output-json: fix type of data parameter 
The cast of data to AlertJsonThread was not correct as the real
type of the void pointer is a OutputJsonCtx. This was working by
luck because they both have a file_ctx as first element.
 11 years ago
Alexander Gozman f11e237d77 Feature #1440: support wildcards in rule filenames 11 years ago
David Cannings 4f8f53d080 Fix rcode parsing, as noticed by Coverity. 
Without support for OPT RR from RFC6891 (Extension mechanisms for DNS)
values of RCODE above 15 are not possible.  Remove dead code which will
never match.
 11 years ago
Jason Ish 9fdae82815 conf - process includes even if not at root node. 11 years ago
Jason Ish 56f6e37304 radix-tree - prevent out of bounds array access 
An IPv6 entry specified before an IPv4 entry on the host-os-policy
table can cause the stream byte array to be access one byte after
the end of the allocated memory at util-radix-tree.c:578.
 11 years ago
Jason Ish 3e5b8f48b1 Bug 1281 - Add tests for rule content of lengths > 255. 11 years ago
Jason Ish e2b04635a7 Bug 1281 - Accept rule content with lengths greater than 255. 11 years ago
Victor Julien 0e22e95e47 alert-json: fix stream logging for IPS mode 
Switch direction in IPS mode.
 11 years ago
Victor Julien 5037ea93f3 threads: add untimed control cond call 
The control conditions so far could only do timed waits, not normal
waits.
 11 years ago
Victor Julien c7bc9ae6a8 detect: minor cleanups 11 years ago
Victor Julien bc2b53f10b parsing: s/strtok/strtok_r/g 
Remove all strtok uses and replace them by strtok_r.

Do the same for Windows builds. Cygwin builds fine with strtok_r.

Add strtok to banned function list.
 11 years ago
Victor Julien fb479902e4 threading: explain purpose of threadvars mucond 11 years ago
Victor Julien 478719ee9d flow: don't hold tv_root_lock longer than needed 
Don't hold it longer than needed in shutting down.
 11 years ago
Victor Julien c96805e839 threading: remove unused cmd thread create func 11 years ago
Victor Julien df5e9d44ca unix-manager: convert to thread module 
Sync command thread for unix manager with other managers and make
it a full thread module.
 11 years ago
Victor Julien cc01b5f6b6 reference/classification: call global init for unittests 11 years ago
Victor Julien 34f2ff067b reference: update pcre globals use 
Don't update globals each time we parse, but instead do it once at
startup.
 11 years ago
Victor Julien 46d401e3bb classification: update pcre globals use 
Don't update globals each time we parse, but instead do it once at
startup.
 11 years ago
Victor Julien b2da57c827 reference: remove global 11 years ago
Victor Julien 393689ce44 classification: remove global from parsing 
Parsing code used a 'fd' global. Remove this.
 11 years ago
Victor Julien 9764a35604 stream: fix --disable-detection reassembly issue 
Due to an error at initialization, the stream engine would not disable
'raw' reassembly automatically when --disable-detection was used.

This lead to segments not getting cleared from the segment lists.
 11 years ago
Victor Julien c1558f5ac4 stream: remove FLOW_NO_APPLAYER_INSPECTION flag 
Instead, intruduce StreamTcpDisableAppLayer to disable app layer
tracking and reassembly. StreamTcpAppLayerIsDisabled can be used
to check it.

Replace all uses of FlowSetSessionNoApplayerInspectionFlag and
the FLOW_NO_APPLAYER_INSPECTION.
 11 years ago
Victor Julien b6798495c5 stream: remove FLOW_NO_APPLAYER_INSPECTION use from tests 11 years ago
Victor Julien b2e1854e2a stream: improve 'no app layer' handling 
When the session/flow was flagged as 'no applayer inspect', which
could happen as a result various reasons, packets would still be
considered by the app layer reassembly.

When ACK'd, they would be removed again. Depending also on the raw
reassembly.

In very long sessions however, this meganism could fail leading to
virtually endlessly growing segment lists.

This patch makes sure that segments that come in on a 'no app layer'
session are tagged properly or even not added at all.

Use a new ssn flag instead of flow flag for no app tracking.
 11 years ago
Victor Julien 22a810813c app-layer: add DisableAppLayer 
Move various app layer related flag setting calls into a utility
function "DisableAppLayer"
 11 years ago
Victor Julien f536099a67 app-layer: de_state optimization 
Add API to bypass expensive TX list walks. This API call is optional.

Implement it for HTTP and DNS.
 11 years ago
Victor Julien 5f0678120d detect-state: update test to check state storing 11 years ago
Victor Julien 37f0bd57b6 detect-state: handle duplicate inspect/match 
If for a packet we have a TX N that has detect state and a TX N+1 that
has no detect state, but does have 'progress', we have a corner case
in stateful detection.

ContinueDetection inspects TX N, but cannot flag the rule in the
de_state_sig_array as the next (TX N+1) has already started and needs
to be inspected. 'StartDetection' however, is then unaware of the fact
that ContinueDetection already inspected the rule. It uses the per
session 'inspect_id' that is only moved forward at the end of the
detection run.

This patch adds a workaround. It uses the DetectEngineThreadCtx::
de_state_sig_array to store an offset between the 'base' inspect_id
and the inspect_id that StartDetection should use. The data type is
limited, so if the offset would be too big, a search based fall back
is implemented as well.
 11 years ago
Victor Julien bc6e4140be detect: add de_state duplication check 
Add test to check if no duplicate destate is created.

Only enabled with DEBUG_VALIDATION.
 11 years ago
Victor Julien 9d198e6662 detect-state: fix state storing 
Fix storing state and bypassing detection. Previously we'd store
on a match only, meaning that StartDetection would rerun often.

Make sure StartDetection only stores if there is something to store.
 11 years ago
Victor Julien 359e2d68f5 detect-http-header: improve buffer handling 
Previously we could never be calling DetectEngineHHDGetBufferForTX
for TX N and then afterwards for TX N - 1. Due to changes in the
stateful detection code this is now possible.

This patch changes the buffer logic to take the 'inspect_id' as it's
base, instead of the first transaction that we are called with.
 11 years ago
Victor Julien 62e937672d detect-events: set SIG_MASK_REQUIRE_*_STATE for events 
Set SIG_MASK_REQUIRE_*_STATE for event rules to earlier discard
them.
 11 years ago
Victor Julien 8d0b090150 engine-analysis: print fast_pattern summary 
When using engine analysis for print fast_pattern stats, print a
short summary at the end containing per buffer:
- smallest fp
- biggest fp
- number of patterns
- avg fp len
 11 years ago
Victor Julien 21f9328989 lua: fix error handling 11 years ago
Victor Julien 52195a4192 http: add event for leading spaces on request line 
Libhtp will issue a warning in this case, so we can match on this.
This patch adds event, rule and unittest.
 11 years ago
Victor Julien 2f85308afe threads: fix missing unlock in error handling 
If TmThreadsUnregisterThread was called with out of range 'id', a lock
would not be cleared after returning from the function.

** CID 1264421:  Missing unlock  (LOCK)
/src/tm-threads.c: 2186 in TmThreadsUnregisterThread()
 11 years ago
Eric Leblond 0303245761 af-packet: use max packet size as snaplen 
If default_packet_size is set to 0, then we use the maximum packet
size as snaplen.
 11 years ago
Eric Leblond 43f691fef8 util-device: fix LiveBuildDeviceListCustom 
The code was assuming that the dictionnary containing the parameter
of a interface was ordered. But for YAML, the order is not assumed
so in case the configuration is generated we may not be able to
parse correctly the configuration file.

By iterating on child on main node and then iterating on subchild
and doing a match on the name, we are able to find the interface
list. In term of code, this algorithm was obtained by simply
removing the test on the name of the first child.
 11 years ago
Eric Leblond 268285c49f output-json-http: output status as an integer 
HTTP status is an integer and it should be written as such in the
JSON events. This will allow to have improved matching in log
analysis tools.
 11 years ago
Eric Leblond 58582df1c6 decode-der: decode DC keyword 
'DC' is used by some certificates and it was not currently translated
to a string.
 11 years ago
Giuseppe Longo 26ba647d58 filedata: read inspected tracker settings from suricata.yaml 11 years ago
Giuseppe Longo 4b5848616f filedata: implement inspected tracker 11 years ago
Giuseppe Longo 1f52410d0f UT: implement tests for inspection code 11 years ago
Giuseppe Longo d2657becc9 app-layer-smtp: make functions as public 11 years ago
Giuseppe Longo 84dc73d9de mpm: implement prefiltering for smtp 11 years ago
Giuseppe Longo f0c54d4764 Detect engine for smtp file_data file_data: inspecting smtp attachments 
Create a buffer to store reassembled file chunks,
and inspect the content.
 11 years ago
Giuseppe Longo b9468aba7c FileData: add stream_offset field 
This is required to store the offset for reassembling chunks.
 11 years ago
Giuseppe Longo 68cf3dd621 file_data: register keyword for smtp and tcp protocol 
Permits to use file_data keyword with smtp or tcp proto.
Also adds some unit tests
 11 years ago
Giuseppe Longo 04561f13d3 signature: set flags and test the protocol 
This checks if the signature's protocol is http
when setup the content keyword.

Also sets the proper flags based by protocol
since the flag SIG_FLAG_TOSERVER has to be set
if the proto is smtp, otherwise SIG_FLAG_TOCLIENT
is it's http.
 11 years ago
Giuseppe Longo 41a1a9f4af find and replace HSBDMATCH by FILEDATA 
This commit do a find and replace of the following:

- DETECT_SM_LIST_HSBDMATCH by DETECT_SM_LIST_FILEDATA
  sed -i 's/DETECT_SM_LIST_HSBDMATCH/DETECT_SM_LIST_FILEDATA/g' src/*

- HSBD by FILEDATA:
  sed -i 's/HSBDMATCH/FILEDATA/g' src/*
 11 years ago
Ken Steele eac83be121 Formatting cleanup in detect-replace.c 
Wrap lines longer than 80 characters
Add "static" for unit tests.
Use (void) for () for function arguments.
Add space after "while(" -> "while ("
Remove space after function names.
Put open bracket of function on a new line.
 11 years ago
Ken Steele ddec92676d Add a comment for DetectReplaceList 
Reworded a quote in PR 742 by Regit from Inliniac to explain why adding
the head of the list (really a FIFO) is the correct behavior.
 11 years ago
Ken Steele cf9da2be15 Fix DetectReplaceAddToList 
I see two problems:
1) If allocating a newlist fails, the function returns NULL, which then
   leaks any existing list elements.
2) The code to add the new value to the list works for the first two, but
   for not the third. For example, replist=A, A->next=B, B->next=NULL, then
   adding C results in replist=A, A->next=C, C->next=NULL, B is lost.

The fix pushes new values onto the head of the list, which might not be
what is needed, but there are no comments on what the function does, so I
made an assumption.
 11 years ago
Alexander Gozman d44eab82c1 Fix bug #1435 (data loss when dumping payloads to JSON) 11 years ago
Zachary Rasmor f0c659f82f Fix Bug #1204 
Fix typo that causes eve syslog settings code to be unreachable.
 11 years ago
Victor Julien 208d27abc7 stream: next_seq handling improvements 
Allow next_seq updating to recover from cases where last_ack has been
moved beyond it. This can happen if ACK's have been accepted for missing
data that is later retransmitted.

This undoes some of the previous last_ack update changes
 11 years ago
Alexander Gozman b12c53cd51 Add timezone to timestamp in JSON logs 11 years ago
David Cannings 2918a75da1 Added support for full parsing of the rcode header in DNS answer 
packets. Where rcode isn't "no error" this is displayed in both DNS and
JSON logs.

Note that this changes the current "No such domain" to "NXDOMAIN" in DNS
logs. This could be fixed if desired to maintain compatibility with
anybody crazy enough to parse the DNS log.

When the rcode is not "no error" (for example NXDOMAIN or SERVFAIL) it
is unlikely that there will be answer RRs. Therefore the rname from the
query is used.

Because the rcode applies to a whole answer packet (not individual
queries) it is impossible to determine which query RR caused the error.
Because of this most DNS servers currently reject multiple queries per
packet. Therefore each query RR is output instead with the relevant
error code, likely to be FORMERR if queries > 1.
 11 years ago
Victor Julien cf839c931f tls: force 'raw' reassembly after each record 
Trigger raw reassembly after each record and after the handshake.
 11 years ago
DIALLO David 04f3f5066d app-layer-modbus: fix deadlock in parsers 11 years ago
Victor Julien 6d170cadd7 smtp: fix mime boundary parsing issue 
If a boundary was longer than 254 bytes a stack overflow would result
in mime decoding.

Ticket #1449

Reported-by: Kostya Kortchinsky of the Google Security Team
 11 years ago
Victor Julien a4a1c396e1 pcap-file: fix malformed timestamp crash 
A bad timestamp would lead to SCLocalTime returning NULL. This case
wasn't checked, leading to a NULL deref.

Reported-by: Kostya Kortchinsky of the Google Security Team
 11 years ago
Pierre Chifflier fa73a0bb8f Fix possible wrap in uint32_t addition in DER parser 
Signed-off-by: Pierre Chifflier <pierre.chifflier@ssi.gouv.fr>
 11 years ago
Victor Julien a2d0441380 Bug 1340: fix missing flow cleanup 
Fix missing flow output cleanup function leading to a crash in the
unix socket mode.
 11 years ago
Victor Julien 04ccfda639 pcap: implement LINKTYPE_NULL 
Implement LINKTYPE_NULL for pcap live and pcap file.

From: http://www.tcpdump.org/linktypes.html

"BSD loopback encapsulation; the link layer header is a 4-byte field,
 in host byte order, containing a PF_ value from socket.h for the
 network-layer protocol of the packet.

 Note that ``host byte order'' is the byte order of the machine on
 which the packets are captured, and the PF_ values are for the OS
 of the machine on which the packets are captured; if a live capture
 is being done, ``host byte order'' is the byte order of the machine
 capturing the packets, and the PF_ values are those of the OS of
 the machine capturing the packets, but if a ``savefile'' is being
 read, the byte order and PF_ values are not necessarily those of
 the machine reading the capture file."

Feature ticket #1445
 11 years ago
Tom DeCanio 97a2d1ac26 fix reputation parser so that it accepts ipv6 addresses in configuration file. 11 years ago
Jason Ish 11d6770ae4 afl - SCHINfoLoadFromConfig - check for NULL before parsing. 
Found by AFL on suricata.yaml.
 11 years ago
Jason Ish 75d21851cf afl - Don't fail if app-layer proto enabled value is NULL. 
Found by using AFL on suricata.yaml.
 11 years ago
Eric Leblond 0376b60da9 email-json: free temporary 'cc' string 11 years ago
Eric Leblond bd67000b69 email-json: free temporary 'to' string 11 years ago
Eric Leblond 7b8184947a app-layer-smtp: fix SMTPTransactionFree function 
A typo was causing some freeing tasks not to be executed.
 11 years ago
Victor Julien 91ddf85323 profiling: fix sorting on very long runs 
Fix poor int handling breaking sorts of profiling on long runs
where the numbers could get really big.
 11 years ago
Victor Julien 1a83fee5f5 xbits: fix coverity warnings 
Switch statement would contain NOALERT even though this was
unreachable.
 11 years ago
Victor Julien 54d5e2eed5 coverity fix: don't do pointer check on static array 11 years ago
Victor Julien b9aaf5a9ab Fix potential deadlock in output 
Coverity:
** CID 1296115:  Program hangs  (ORDER_REVERSAL)
/src/tm-threads.c: 1670 in TmThreadClearThreadsFamily()

The problem is with the by default unused '%m' output parameter.
To get the thread vars it takes the tv_root_lock. This may already
be locked by the calling thread. Also, it could lead to a case of
wrong lock order between the tv_root_lock and the thread_store_lock.

Very unlikely to happen though.

As the %m param isn't really used (by default) this patch just
disables it.
 11 years ago
Victor Julien 94321b8a2f packet pool: fix memleaks 
Don't kill flow manager and recyclers before the rest of the threads. The
packet threads may still have packets from their pools. As the flow threads
would destroy their pools the packets would be lost.

This patch doesn't kill the threads, it just pulls them out of their run
loop and into a wait loop. The packet pools won't be cleared until all
threads are killed.

Wait for flow management threads to close before moving on to the
next steps in the shutdown process.

Don't destroy flow force reassembly packet pool too early. Worker
threads may still want to return packets to it.
 11 years ago
Victor Julien 22142d9b8b threshold: cleanups 
Use pcre_copy_substring in the most common part of threshold rule
parsing. Minor other cleanups.
 11 years ago
Victor Julien e86e1e0282 reference: use pcre_copy_substring 11 years ago
Victor Julien cc38df8074 classification: cleanups 
Reduce hash table size as regular classification files are usually
below 100 in size. It's not performance critical anyway.

Convert pcre_get_substring calls to pcre_copy_substring.
 11 years ago
Victor Julien deb98fab0c packet pool: init pool for autofp workers as well 
Introduce a new 'PacketPoolInitEmpty' as these pools will not often
need packets.

Also, don't double 'destroy' in the main thread.
 11 years ago
Victor Julien 1884227019 autofp: reduce flow storage space requirement 
Use int16_t instead of a regular int to safe 2 bytes per flow.
 11 years ago
Victor Julien 643c30f67f threading: add missing locks to threading storage 
Lock threading root lock when walking or updating the threading
storage for 'ThreadVars' objects.
 11 years ago
maxtors e7658fd4f4 Added new WebDAV http methods. 
Added more WebDAV functions. A complete list of what http
methods libhtp can handle can be found at:
https://github.com/OISF/libhtp/blob/0.5.x/htp/htp_core.h#L260.
So now the methods array reflects these available functions.

The comments have also been changed to reflect the desired style.
 11 years ago
maxtors 3b26b07963 Reworked pattern registration structure and feedback issues. 
1) Reworked pattern registration for http methods and versions.

Instead of being a manual and verbose action of adding one
and one http method with N-amount if prefix spacings and
the same for HTTP versions (eg. HTTP/1.1) i moved it all
to be loop based actions reading values from char arrays.

In the future all that is needed is to add new methods
to the arrays and they will be added as a pattern.

2) Modified pattern registration after feedback.

Changed variable used in snprintf for http method registration
Should have been size of dest buffer at not another var (catsize)
that i had created. Also removed this variable.

Fixed a typo in the comment for registering http versions.
TO_CIENT -> TO_CLIENT.
 11 years ago
maxtors 7a9e963619 Added http methods for webdav (rfc4918). 
Included the following methods to app-layer-htp.c:
- PROPFIND
- PROPPATCH
- MKCOL
- COPY
- MOVE
- LOCK
- UNLOCK
 11 years ago
maxtors a7abaf538c Added DELETE and PATCH http methods to app-layer-htp.c 
Added two new http methods to the http decoding done in
app-layer-htp.c so that these messages will be decoded as well.
 11 years ago
Victor Julien 915eb992ca app-layer: update protocol detection tests 
AppLayerTest09, AppLayerTest10 and AppLayerTest11 depended on a max
protocol detection pattern size of < 17. Update the tests to pass one
extra byte to the app layer. This makes the protocol detection code
flag the session as 'proto detection completed' again.
 11 years ago
David Maciejak 48971afdfb Fix Unit ID check 11 years ago
Eric Leblond f35001a614 util-running-mode: setup config file 
Without that we get warning message.
 11 years ago
Eric Leblond 6f5c4c12e1 list keywords: fix regression on app layer name 
It was not anymore displayed.
 11 years ago
Eric Leblond d98060410c list keywords: restore 2.0 similar output 
Unset was previously displayed if the application layer was not
set. Code before this patch was displaying '(null)' which is not
user friendly.
 11 years ago
Alexander Gozman 568635c2e1 Reject ICMP traffic, not only TCP/UDP 11 years ago
Jason Ish 1f9d9256e7 spelling - dubbed -> duped (for duplicate) 11 years ago
Jason Ish 8911b04077 DetectGidSet - safer stripping of quotes. 
Discovered by AFL when using a rule like:
    alert tcp any any -> any any (content:"ABC"; gid:";)
resulting a negative array index.
 11 years ago
Jason Ish da88b3b787 DetectSidSetup - safer stripping of quotes. 
Discovered by AFL when using a rule like:
    alert tcp any any -> any any (content:"ABC"; sid:";)
would result in a negative array index.
 11 years ago
Jason Ish ca60d8c370 ParseSizeString - don't abort on unknown unit type. 11 years ago
Victor Julien 00d6298666 host: handle timeout 
Set expiration for regular hostbits keyword.
 11 years ago
Victor Julien ab7882fbf3 ippair: handle timeouts in the flow manager 
Like host and defrag, handle timeouts of ippair in the first flow
manager thread.
 11 years ago
Victor Julien e8a5925012 ippair: implement basic timeout check 
The only user is the xbits subsys, so it's timeout controls all.
 11 years ago
Victor Julien f2349e10ad hosts: consider hostbits/xbits status in timeout 
Consider the host's xbits expiry status when checking the host for
timeout. If a single active non-expired bit is found, the host won't
be timeout just yet.
 11 years ago
Victor Julien 67dd5c0430 host: implement hostbits/xbits expire 11 years ago
Victor Julien ca1f3e68d4 ippair: implement xbits expiration 11 years ago
Victor Julien fbdcffabc2 xbits: expire (first steps) 11 years ago
Victor Julien 7b79b9458d xbits: implement ip_src/ip_dst support 
Just a wrapper for regular 'hostbits'.
 11 years ago
Victor Julien 57d93cbcfe hostbits: prepare for xbits integration 11 years ago
Victor Julien 8e6453faac xbits: prepare for non-ippair support 11 years ago
Victor Julien dea8dda3c1 vars: redo var name <-> idx handling 
Can't use sm type anymore as multiple var carriers (host/flow/etc) will
all have xbits.
 11 years ago
Victor Julien f77c3d9a2c xbits: hostbits use xbits type 
Make hostbits use xbits type.
 11 years ago
Victor Julien a716dded04 xbit: move to util-var 
We can reuse the type for hostbits and flowbits as well.
 11 years ago
Victor Julien 61cb2abc8d ippair: xbit implementation 
detect xbits for ippair: initial implementation
 11 years ago
Victor Julien 4c6d564211 hostbits: unittest cleanups 11 years ago
Victor Julien 2cb0df7cf9 hostbits: allow for src/dst parameter 
By default, the hostbit checks the 'src' host. By adding the dst option
the dst host is used instead.
 11 years ago
Victor Julien be725a5ba2 hostbits: add more tests, include sigorder 11 years ago
Victor Julien d67fd306f8 hostbits: implement sigorder 
Like with flowbits, make sure that 'set's are evaluated before
'isset's, etc.
 11 years ago
Victor Julien 5c880377ae detect: hostbits keyword 
Per host bit similar to 'flowbits'.

Initial code that uses just the 'src' ip for the operations.
 11 years ago
Victor Julien 99ae643e4e ippair: track ippairs, enable tests 11 years ago
Alexander Gozman 45ba20bcb0 DetectAddressParseString(): fix IPv6 address handling 11 years ago
Eric Leblond 605ef33972 runmodes: improve listing output 
This patch removes some multiple line displayed before the
information about unix socket mode.
 11 years ago
Eric Leblond e5ae808b0f runmodes: fix typo in output 11 years ago
Eric Leblond ef3290bb2e flow-manager: don't set cpu affinity twice 
It is already set in the thread creation function.
 11 years ago
Victor Julien 8e83d0073e stream: fix bad last_ack update leading to gaps 
A bad last_ack update where it would be set beyond next_seq could
lead to rejection of valid segments and thus stream gaps.

Update tests to reflect new last_ack/next_seq behaviour.
 11 years ago
Victor Julien 4e177bc9d6 detect-state: cleanups and comments 11 years ago
Victor Julien 304c711a27 detect-state: use f->protomap instead of FlowGetProtoMapping(f->proto) 11 years ago
Victor Julien e1efa7a0b5 detect-flowbits: locking update 
Make matches that can be in the POSTMATCH list aware of the lock hint.
 11 years ago
Victor Julien 4609495c60 detect-flowint: conditional locking 11 years ago
Victor Julien 5111aa2ec0 detect-state: handle 'post match' locking 
The post match list was called with an unlocked flow until now.
However, recent de_state handling updates changed this. The stateful
detection code can now call the post match functions while keeping
the flow locked. The normal detection code still calls it with an
unlocked flow.

This patch adds a hint to the DetectEngineThreadCtx called
'flow_locked' that is set to true if the caller has already locked
the flow.
 11 years ago
Victor Julien 9bcb02119f detect-flowint: fix unlocked flow access 
Some of the access to the flow and to structures retrieved from the
flow was unlocked.

This patch changes the logic to be wrapped in lock calls.
 11 years ago
Victor Julien 18f0351d9b detect-state: fix profiling 11 years ago
Victor Julien 0b262cbfda dns: optimize tx list walks 
The detection engine and log engines can walk the tx list indirectly,
by looping AppLayerParserGetTx. This would lead to new list walks in
the DNS tx list though. Leading to bad performance.

This patch stores the last returned tx and uses that to determine if
the next tx is what we need next. If so, we can return that w/o list
walk.
 11 years ago
Victor Julien 0704ece4d7 detect-reload: enable unconditionally 
Reloading is available unconditionally now.
 11 years ago
Victor Julien 724c7044e1 detect-reload: 0 detect threads is no error 
The reload code would consider 0 detect threads to be an error,
but it's not in case of unix socket mode.
 11 years ago
Victor Julien 7c9e015748 unix-socket: implement reload-rules 
Implement the reload-rules unix socket command. The unix command
thread signals the main thread to do the reload and it waits for
it to complete.
 11 years ago
Victor Julien 71d01f06b9 detect reload: load config 
Load the YAML into a prefix "detect-engine-reloads.N" where N is the
reload counter. This way we can load the updated config w/o overwriting
the current one.
 11 years ago
Victor Julien b51075e804 detect: remove config at prefix 
Remove config at prefix when freeing a detect engine.
 11 years ago
Victor Julien 85e12f2bc6 rule vars: support prefix 
Support the detection engine's prefix when retrieving rule vars.
 11 years ago
Victor Julien 0bc27c7dc7 rule-vars: take detect engine as arg 11 years ago
Victor Julien 3083f51cc6 detect:pass DetectEngineCtx to port parsing 
Preparation for prefix handling in port parsing.
 11 years ago
Victor Julien 2be9ccfe9c detect: pass DetectEngineCtx to address parsing 
Preparation for prefix handling in address parsing.
 11 years ago
Victor Julien 7108085d33 detect: initialize detection engine by prefix 
Initalize detection engine by configuration prefix.

    DetectEngineCtxInitWithPrefix(const char *prefix)

Takes the detection engine configuration from:
<prefix>.<config>

If prefix is NULL the regular config will be used.

Update sure that DetectLoadCompleteSigPath considers the prefix when
retrieving the configuration.
 11 years ago
Victor Julien 97d77e3540 conf: add ConfYamlLoadFileWithPrefix 
Add function to load a yaml file and insert it into the conf tree at
a specific prefix.

Example YAML:
somefile: myfile.txt

If loaded using ConfYamlLoadFileWithPrefix with prefix "myprefix", it
can be retrieved by the name of "myprefix.somefile".
 11 years ago
Victor Julien a80cc696d7 detect: allow det_ctx->de_ctx to be NULL 
When freeing det_ctx, allow de_ctx to be NULL.
 11 years ago
Victor Julien c9a8262ccf detect: reload thread init cleanup 
Rename the thread init function DetectEngineThreadCtxInitForLiveRuleSwap
to DetectEngineThreadCtxInitForReload and change it's logic to take the
new detection engine as argument and let it return the
DetectEngineThreadCtx or NULL on error.

The old approach used the thread init API format, but it wasn't used in
that way.
 11 years ago
Victor Julien 55e7370fc5 detect reload: allow master update during reload 
Add DetectEngineReference, which takes a reference to a detect engine,
and make DetectEngineThreadCtxInitForLiveRuleSwap use it. This way
reload will not depend on master staying the same. This allows master
to be updated in between w/o affecting the reload that is in progress.
 11 years ago
Victor Julien b1c54a8673 detect: remove old live reload implementation 
Remove code that ran the reload in it's own thread. Simplify the
signal handling.
 11 years ago
Victor Julien 0c263105cd detect: move reload into main loop 
Use new DetectEngineReload() function. It's called from the main loop
instead of it being spawned into it's own temporary thread. This greatly
simplifies the signal handling.

An added advantage is that this seems to improve the memory usage.

Related to bug #1358
 11 years ago
Victor Julien e7882da178 detect: introduce 'minimal' detect engine 
The minimal detect engine has only the minimal memory use and setup
time. It's to be used for 'delayed' detect where the first detection
engine is essentially empty.

The threads setup are also minimal.
 11 years ago
Victor Julien f4617d5357 threading: remove 'dummy' slot logic 
Now that delayed detect is not using it anymore, the dummy logic
can be removed.
 11 years ago
Victor Julien b0cb4c17ec detect: unify delayed detect and reload 
Instead of threading logic with dummy slots and all, use the regular
reload logic for delayed detect.

This means we pass a empty detect engine to the threads and then
reload (live swap) it as soon as the engine is running.
 11 years ago
Victor Julien 38b349af1e runmodes: remove DetectEngineCtx passing from API 
No longer pass a pointer to the current detection engine to the
runmode API calls.

Note: breaks delayed detect. Will be fixed in a future commit.
 11 years ago
Victor Julien b038b6a2f8 unittests: add exception to detect engine setup 
Add code to allow for unittests not following the complete api.

Update replace tests as they don't use the unittests runmode that
powers the workaround based on RunmodeIsUnittests().
 11 years ago
Victor Julien d66fa1add1 detect: update detect engine management 
Update detect engine management to make it easier to reload the detect
engine.

Core of the new approach is a 'master' ctx, that keeps a list of one or
more detect engines. The detect engines will not be passed to any thread
directly, but instead will only be accessed through the detect engine
thread contexts. As we can replace those atomically, replacing a detect
engine becomes easier.

Each thread keeps a reference to its detect context. When a detect engine
is replaced or removed, it's added to a free list. Once its reference
count reaches 0, it is freed.
 11 years ago
Victor Julien 092ddc1853 detect: no exit on reference/classification errors 
Don't exit on errors during classification and reference parsing.

Add some suppression of error messages when in unittest mode.
 11 years ago
Victor Julien 49bad2cfba detect: consolidate more setup into DetectEngineCtxInit 
Loading of classifications, references and action order was done
unconditionally, so can be done in one place.
 11 years ago
Victor Julien 69f99245c5 unix-command: fix duplicate registration error msg 11 years ago
Victor Julien 60a49657df Bug 1401: on midstream pickup, fix packet direction 
On midstream SYN/ACK pickups, we would flip the direction of packets
after the first. This meant the first (pickup) packet's direction
was wrong.

This patch fixes that.
 11 years ago
Jason Ish ee7e813256 Bug 1417 - Record sequence nodes as sequences. 
Nodes that are sequences weren't being recorded as such, causing
rules to fail to load.

Change sequence test name to reflect better what it tests, and
test that the sequence node is detected as a sequence.
 11 years ago
Victor Julien e250040b72 detect-state: implement tx state reset for reload 
In case of Detect Reload, we need to reset active tx' state.
 11 years ago
Victor Julien da3e8ad8f6 detect-state: split flow and tx state 
Use separate data structures for storing TX and FLOW (AMATCH) detect
state.

- move state storing into util funcs
- remove de_state_m
- simplify reset state logic on reload
 11 years ago
Victor Julien 840efe17fe modbus: tx de_state 11 years ago
Victor Julien e984a57203 smtp: tx de_state 11 years ago
Victor Julien 2a23627a82 dns: implement tx de_state 11 years ago
Victor Julien 774bb90351 http: clean up tx destate at tx destroy 11 years ago
Victor Julien 6279da0fbd http: support per TX destate storage 11 years ago
Victor Julien 1cf02560c8 app-layer: per tx destate 
Add API calls for storing detection state in the TX.
 11 years ago
Victor Julien 866d9684ea detect-state: fix profiling 11 years ago
Victor Julien 7e75279977 detect-state: various cleanups 11 years ago
Victor Julien eec22ce19b detect-state: rip per sig detect out of ContinueDetect 11 years ago
Victor Julien bf818b8fb2 detect-state: remove DeStateResetFileInspection 
It was effectively unused.
 11 years ago
Victor Julien 206f9d4010 detect-state: remove redundant code 11 years ago
Victor Julien e390e24a7c detect-state: add helper to test state 
Add little helper function StateIsValid() to test if the state
can be inspected safely.

Cleans up stateful detection loops.
 11 years ago
Victor Julien 072ae12771 detect-state: add helper to indicate last tx 
Add little helper to indicate current tx is that last we have.
 11 years ago
Victor Julien b710f2dd59 detect-state: cleanup ContinueDetection 
Only lock f->de_state->m when we start to access it. So after
declaration and initialization of local vars.
 11 years ago
Victor Julien 54cb2b6877 detect-state: cleanup retvals 
Use DETECT_ENGINE_INSPECT_SIG_* instead of 0, 1, 2 and 3.
 11 years ago
Victor Julien 97cab030d7 modbus: shrink data structure 11 years ago
Victor Julien 9f1b417660 file: don't 'close' file if we need to track it 11 years ago
Victor Julien 6723d03c7e http: add inspection engine for http request line 
No MPM though.
 11 years ago
Victor Julien c0f265fd82 http: fix stat_msg and stat_code state tracking 11 years ago
Victor Julien 5b8c94db30 Remove spinning PacketPoolWait 
PacketPoolWait in autofp can wait for considerable time. Until now
it was essentially spinning, keeping the CPU 100% busy.

This patch introduces a condition to wait in such cases.

Atomically flag pool that consumer is waiting, so that we can sync
the pending pool right away instead of waiting for the
MAX_PENDING_RETURN_PACKETS limit.
 11 years ago
Victor Julien 2d7f79a62b drop json log: log out 'drop' signature 
If no normal sig was logged as the 'drop' reason, try the stored
drop signature instead, this will also log out 'noalert' sigs.
 11 years ago
Victor Julien 2e754ca6fa drop json: make alerts logging optional 
Make logging out alerts that caused the drop optional.
 11 years ago
Victor Julien 6b172bb010 drop json: add sids (if applicable) 
If a drop is caused by a SID match, add it to the drop record.
 11 years ago
Victor Julien 8a97bb0d04 alert json: move alert info into function 
Move adding the alert info (sid,rev,gid,etc) into it's own function,
so it can be called from other outputs as well.
 11 years ago
Victor Julien e9857200b3 detect: set action from utility function 
Set actions that are set directly from Signatures using the new
utility function DetectSignatureApplyActions. This will apply
the actions and also store info about the 'drop' that first made
the rule drop.
 11 years ago
Victor Julien c914f7bcdc detect: cleanup, remove unused order_id 
No longer used, so remove.
 11 years ago
Eric Leblond b8e7d3a259 flow-timeout: fix init of pseudo packet 
The code was not checking if we had enough room in the direct
data. In case default_packet_size was set really small, this was
resulting in data being written over the data and causing a crash.

The patch fixes the issue by forcing an allocation if the direct
data size in the Packet is to small.
 11 years ago
Eric Leblond e138a2ac1e decode: introduce PacketCallocExtPkt function 
In flow timeout handling we need a function that allocate and blank
a place that will be used to put constructed packet data. This new
function has no other goal.
 11 years ago
Jason Ish ab1d69fc4e When re-opening a log file on HUP, always append. 
This will prevent log files that have not been rotated by some
external tool from being deleted, but log files that were
rotated (moved out of the way) will be re-opened.

This is a better default behaviour, especially when not all
log files are rotated at the same time.

Thanks to iro on IRC.
 11 years ago
Jason Ish 6ed246c041 Don't attempt to load the rule files if the rule-files configuration 
node is not a sequence.  Instead log a warning as this is usually
a configuration error.
 11 years ago
Jason Ish a243a42bdf New function to test if a configuration node is a sequence or not. 11 years ago
Eric Leblond 290b01f95e af-packet: don't unlock twice the bpf mutex 11 years ago
Eric Leblond 47d9c7b211 json-alert: use getter for appstate 11 years ago
Eric Leblond 54b13851cc flow: constify getters param 
Some potential callers are already using constified values so it
is good to do it.
 11 years ago
Eric Leblond 881f32cc02 json-alert: add SSH fields in alert logging 11 years ago
Eric Leblond c1970a3655 json-ssh: export logging function 
It will be use in alert logging to display SSH information.
 11 years ago
Eric Leblond 180faece7c json-alert: log tls info in alert 
This patch adds the capabilities to log the TLS information the
same way it is currently possible to do with HTTP. As it is
quite hard to read ASN.1 directly in the stream, this will help
people to understand why suricata is firing on alert relative
to TLS.
 11 years ago
Eric Leblond 22182e7a84 json-tls: refactor to export logging function 
To be able to add TLS data in alert we need to do the same as what
is done with HTTP ie export the logging functions.
 11 years ago
Ken Steele fb0ecaba05 Inject pseudo packet periodically when there is not traffic in mPIPE. 
To prevent pseudo packets from not being processed when there is no traffic,
inject a pseudo packet if no traffic is seen by a thread for ~100ms.
 11 years ago
Victor Julien 20b9849999 util-magic: make unittests less specific 
So they pass on CentOS 5.11 as well.
 11 years ago
Victor Julien ec2be2643b app-layer: init flow in tests 11 years ago
Eric Leblond ee7422de0a pcap-file: add missing atomic init 
It is mandatory to init all atomic to avoid problem on system
without atomic support.
 11 years ago
Eric Leblond ed147d3b91 runmode-pcap-file: suppress useless include 11 years ago
Victor Julien ae6ef9acb4 flow-manager: init global atomics 11 years ago
Victor Julien ae8f2a4e62 stream: fix unittests wrt flow 11 years ago
Victor Julien 256388325d Fix compiler warning on CentOS 5.11 
cc1: warnings being treated as errors
app-layer-smtp.c: In function ‘SMTPParseCommandBDAT’:
app-layer-smtp.c:908: warning: dereferencing type-punned pointer will break strict-aliasing rules
 11 years ago
Eric Leblond bed5b28412 util-ioctl: don't build code RX ring on old system 
If ETHTOOL_GRXRINGS is undefined we will not be able to build the
RX rings code. So we can make the build conditional to the
definition of ETHTOOL_GRXRINGS.
 11 years ago
Victor Julien e78e33a428 http: add event for suspicious method delimeter 
Add event and rule for suspicious delim(s) between method and uri.

Add unittests as well.
 11 years ago
Victor Julien 5ad7198dc0 http: add libhtp uri warning event 
Add event for libhtp warning added 0.5.17 for URI's with suspicious
delimeters.
 11 years ago