aiden
/
suricata
mirror of https://github.com/OISF/suricata
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,292 Commits
8 Branches
165 Tags
229 MiB
main
main-7.0.x
main-8.0.x
master-6.0.x
master-5.0.x
master-4.1.x
master-4.0.x
master-3.2.x
suricata-8.0.2
suricata-7.0.13
suricata-8.0.1
suricata-7.0.12
suricata-8.0.0
suricata-7.0.11
suricata-8.0.0-rc1
suricata-8.0.0-beta1
suricata-7.0.10
suricata-7.0.9
suricata-7.0.8
suricata-7.0.7
suricata-7.0.6
suricata-6.0.20
suricata-7.0.5
suricata-6.0.19
suricata-6.0.18
suricata-7.0.4
suricata-6.0.17
suricata-7.0.3
suricata-6.0.16
suricata-7.0.2
suricata-6.0.15
suricata-7.0.1
suricata-6.0.14
suricata-7.0.0
suricata-7.0.0-rc2
suricata-6.0.13
suricata-6.0.12
suricata-6.0.11
suricata-7.0.0-rc1
suricata-6.0.10
suricata-6.0.9
suricata-7.0.0-beta1
suricata-6.0.8
suricata-6.0.7
suricata-6.0.6
suricata-5.0.10
suricata-6.0.5
suricata-5.0.9
suricata-6.0.4
suricata-5.0.8
suricata-6.0.3
suricata-5.0.7
suricata-6.0.2
suricata-5.0.6
suricata-6.0.1
suricata-5.0.5
suricata-4.1.10
suricata-4.1.9
suricata-5.0.4
suricata-6.0.0
suricata-6.0.0-rc1
suricata-6.0.0-beta1
suricata-5.0.3
suricata-4.1.8
suricata-4.1.7
suricata-5.0.2
suricata-4.1.6
suricata-5.0.1
suricata-5.0.0
suricata-5.0.0-rc1
suricata-4.1.5
suricata-5.0.0-beta1
suricata-4.1.4
suricata-4.1.3
suricata-4.0.7
suricata-4.1.2
suricata-4.1.1
suricata-4.1.0
suricata-4.0.6
suricata-4.1.0-rc2
suricata-4.1.0-rc1
suricata-4.0.5
suricata-4.1.0-beta1
suricata-4.0.4
suricata-4.0.3
suricata-4.0.2
suricata-3.2.5
suricata-4.0.1
suricata-3.2.4
suricata-4.0.0
suricata-4.0.0-rc2
suricata-3.2.3
suricata-4.0.0-rc1
suricata-4.0.0-beta1
suricata-3.2.2
suricata-3.1.4
suricata-3.2.1
suricata-3.2
suricata-3.2RC1
suricata-3.1.3
suricata-3.2beta1
suricata-3.1.2
suricata-3.1.1
suricata-3.1
suricata-3.0.2
suricata-3.1RC1
suricata-3.0.1
suricata-3.0.1RC1
suricata-3.0
suricata-2.0.11
suricata-3.0RC3
suricata-3.0RC2
suricata-3.0RC1
suricata-2.0.10
suricata-2.0.9
suricata-2.1beta4
suricata-2.0.8
suricata-2.0.7
suricata-2.1beta3
suricata-2.0.6
suricata-2.0.5
suricata-2.1beta2
suricata-2.0.4
suricata-2.1beta1
suricata-2.0.3
suricata-2.0.2
suricata-2.0.1
suricata-2.0.1rc1
suricata-2.0
suricata-2.0rc3
suricata-2.0rc2
suricata-2.0rc1
suricata-2.0beta2
suricata-1.4.7
suricata-1.4.6
suricata-1.4.5
suricata-2.0beta1
suricata-1.4.4
suricata-1.4.3
suricata-1.4.2
suricata-1.4.1
suricata-1.3.6
suricata-1.4
suricata-1.3.5
suricata-1.4rc1
suricata-1.3.4
suricata-1.4beta3
suricata-1.3.3
suricata-1.4beta2
suricata-1.3.2
suricata-1.4beta1
suricata-1.3.1
suricata-1.3
suricata-1.3rc1
suricata-1.3beta2
suricata-1.3beta1
suricata-1.2.1
suricata-1.2
suricata-1.2rc1
suricata-1.2beta1
suricata-1.1.1
suricata-0.8.2
suricata-1.0.0
suricata-1.0.1
suricata-1.0.2
suricata-1.0.3
suricata-1.0.4
suricata-1.0.5
suricata-1.1
suricata-1.1beta1
suricata-1.1beta2
suricata-1.1beta3
suricata-1.1rc1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'c446abeb47'
${ noResults }
Commit Graph

5783 Commits (c446abeb471d8be516bf11f0106797526b7122b3)

Author SHA1 Message Date
Victor Julien 8673801ea3 multi-detect: add tenant id to alert json output 
Add a integer field "tenant_id" to the JSON alert output.
 11 years ago
Victor Julien f4c9915066 multi-detect: store tenant id in packet 
Store tenant id in the packet so that the output API's can log it.
 11 years ago
Victor Julien 642c267dc4 multi-detect: refuse to add duplicate tenant 
Generate error if tentant to be added is already loaded.
 11 years ago
Victor Julien 4e0683c3f5 multi-detect: cleanup, reuse tenant loading code 
Reuse tenant loading from YAML code for Unix Socket.
 11 years ago
Victor Julien 646eb4c2a8 multi-detect: load tenants from yaml file 
Load tenants and mappings from the suricata.yaml when available.
 11 years ago
Victor Julien 216638c342 multi-detect: implement unregister-tenant-handler 
Remove a tenant handler from the list and apply it.
 11 years ago
Victor Julien 93f856a1b3 detect: don't error out on no de_ctx 
This can happen on a multi-detect setup with no registered
engines yet.
 11 years ago
Victor Julien b6f290fac7 multi-detect: set selector from yaml 
Yaml setting is: multi-detect.selector

Implement 'vlan' and 'direct'.
 11 years ago
Victor Julien c72b7f83b8 multi-detect: error on start if no selector registered 
Force user to select the method at startup.
 11 years ago
Victor Julien 1127ad66b4 multi-detect: register counters on 'master' det_ctx 
Otherwise counters are only registered after the stats api is
already fixed.
 11 years ago
Victor Julien 7c581c0ffc multi-detect: allow start up with 0 tenants 11 years ago
Victor Julien 6d92e8d220 unix-socket: implement register-tenant-handler 
Register tenant handlers/selectors based on what the unix command
"register-tenant-handler" tells.

Check traffic id before adding it. No duplicated registrations for
a traffic id are allowed.
 11 years ago
Victor Julien 1893c5edb1 multi-detect: initial selectors for tenants 
The Detection Thread has the TenantGetId pointer which allows it
to select a tenant id based on the packet.
 11 years ago
Victor Julien 0ff6d3dcfd detect: select detect engine at Detect entry 
Limited to Pcap only currently.
 11 years ago
Victor Julien bc7e9be5c6 unix-socket: allow tenant id with pcap-file 
Register the tenant id that the pcap-file optionally got.
 11 years ago
Victor Julien bea66156b9 pcap-file: set tenant-id if available 
Set the id to each packet's 'pcap_v' structure.
 11 years ago
Victor Julien 98d265f40b detect: use multi tenant thread init if MT enabled 11 years ago
Victor Julien b653479815 detect: make multi tenancy a global switch 
At start up we will set this flag based on "multi-detect.enabled".
 11 years ago
Victor Julien 3f12bdd4f8 tenants: apply added/removed tenant 
Apply to the engine.
 11 years ago
Victor Julien def2b58725 detect: initial MT lookup logic 
In the DetectEngineThreadCtx, store another DetectEngineThreadCtx per
tenant.

Currently it's just a simple array indexed by the tenant id.
 11 years ago
Victor Julien 147a6d2bfd multi-detect: (un)register-tenant unix socket commands 
Make available to live mode and unix socket mode.

register-tenant:
    Loads a new YAML, does basic validation.
    Loads a new detection engine
    Loads rules
    Add new de_ctx to master store and stores tenant id in the de_ctx so
        we can look it up by tenant id later.

unregister-tenant:
    Gets the de_ctx, moves it to the freelist
    Removes config

Introduce DetectEngineGetByTenantId, which gets a reference to the
detect engine by tenant id.
 11 years ago
Victor Julien 3aa58f25ad eve alert: fix stream payload printing 11 years ago
Victor Julien 2ef0ebb24b detect: fix pass transaction handling 
If a flow was 'pass'd, it means that no packet of it will flow be handled
by the detection engine. A side effect of this was that the per flow
inspect_id would never be moved forward. This in turn lead to a situation
where transactions wouldn't be freed.

This patch addresses this case by incrementing the inspect_id anyway for
the pass case.
 11 years ago
Victor Julien 06ee2bc87e detect: set flow noinspect on pass in applayer/stream 
If a pass rule matches in the reassembled stream and/or in the
app-layer state, it means the rest of the flow should not be
inspected.
 11 years ago
Victor Julien bbc9874b81 http: destroy htp_tx_t even if incomplete 11 years ago
Victor Julien 814f0b4094 detect: make http prefilter use disrupt flags 11 years ago
Victor Julien b592f98727 detect: optimize http prefilter handing 11 years ago
Victor Julien 6946e0be55 detect: pass flags to inspect_id update logic 11 years ago
Victor Julien c087708fa9 output-tx: use disrupt flags 11 years ago
Victor Julien 00ef789ffc app-layer: pass full flags around in tx handling 11 years ago
Victor Julien ea571add73 app-layer: disruption flags 
Stream GAPs and stream reassembly depth are tracked per direction. In
many cases they will happen in one direction, but not in the other.

Example:
HTTP requests a generally smaller than responses. So on the response
side we may hit the depth limit, but not on the request side.

The asynchronious 'disruption' has a side effect in the transaction
engine. The 'progress' tracking would never mark such transactions
as complete, and thus some inspection and logging wouldn't happen
until the very last moment: when EOF's are passed around.

Especially in proxy environments with _very_ many transactions in a
single TCP connection, this could lead to serious resource issues. The
EOF handling would suddenly have to handle thousands or more
transactions. These transactions would have been stored for a long time.

This patch introduces the concept of disruption flags. Flags passed to
the tx progress logic that are and indication of disruptions in the
traffic or the traffic handling. The idea is that the progress is
marked as complete on disruption, even if a tx is not complete. This
allows the detection and logging engines to process the tx after which
it can be cleaned up.
 11 years ago
Victor Julien 8125e04b39 detect: clean up flag usage 11 years ago
Victor Julien 6e2c90a83b detect: constify some DetectMpmPrefilter args 11 years ago
Victor Julien c419f33f44 dns: fix state progress handling 11 years ago
Victor Julien b0f5f7ee97 app-layer: fix args to state progress calls 11 years ago
Victor Julien 62fa9f09d4 Sync alversion/appversion types 
The app layer state 'version' field is incremented with each update
to the state. It is used by the detection engine to see if the current
version of the state has already been inspected. Since app layer and
detect always run closely together there is no need for a big number
here. The detect code really only checks for equal/not-equal, so wrap
arounds are not an issue.
 11 years ago
Victor Julien 11d3f5f67a http: harden tx inspection code 11 years ago
Victor Julien be2849044b flow/stream: xfer noinspect flags to pseudo pkts 
Set noinspection flags for payloads and packets on flow and stream
pseudo packets. Without these, the pseudo packets could trigger
inspection even though this was disabled for a flow.
 11 years ago
Victor Julien a0899cbe85 detect: optimize Signature layout 11 years ago
Victor Julien baadcab1b8 detect: default to u32 for SigIntId 11 years ago
Eric Leblond 1cd97713c2 file-json: add file_id to message 
This will allow to get the filename and by consequence the file
after a parsing of the EVEV log file.
 11 years ago
Eric Leblond 8fde842f97 af-packet: implement rollover option 
This patch implements the rollover option in af_packet capture.
This should heavily minimize the packet drops as well as the
maximum bandwidth treated for a single flow.

The option has been deactivated by default but it is activated in
the af_packet default section. This ensure there is no change for
old users using an existing YAML. And new users will benefit from
the change.

This option is available since Linux 3.10. An analysis of af_packet
kernel code shows that setting the flag in all cases should not
cause any trouble for older kernel.
 11 years ago
Eric Leblond dc306f3bad af-packet: implement new load balancing modes 
This patch implements the fanout load balancing modes available
in kernel 4.0. The more interesting is cluster_qm that does the
load balancing based on the RSS queues. So if the network card
is doing a flow based load balancing then a given socket will
receive all packets of a flow indepently of the CPU affinity.
 11 years ago
Eric Leblond 851fcef962 af-packet: sync header with latest features 
Sync the replacement define with the latest Linux code.
This patch also updates the detection part in configure.ac
to do a declaration of all fields if the newest features are
not present.
 11 years ago
Aleksey Katargin caa2438b98 netmap: support SW rings 
Netmap uses SW rings to send and receive packets from OS.
 11 years ago
Aleksey Katargin 823167bde2 netmap: strict check for zero copy mode 
Netmap does not guarantees that mmap'ed regions for different interfaces would be the same.
 11 years ago
Aleksey Katargin 5104b02f32 netmap: fixed autofp mode. 
Previous implementation does not work with this mode.
 11 years ago
Aleksey Katargin 491fca468d netmap: support non-equal count of Rx and Tx rings on interface. 
Netmap does not guarantees that NIC will have equal number of transmit and receive rings.
 11 years ago
Schnaffon 6484299701 Use unlikely for error treatment in alert-prelude.c 
When handling error cases on creation of a new idmef field, we are in an unlikely case. This patch adds the unlikely() expression to indicate this to gcc.
 11 years ago
Victor Julien 334e8656bf introduce fatal error macro's 
Add 'FatalError' and 'FatalErrorConditonal' that will take the same
args as SCLogError.

FatalError logs the error using SCLogError and then exits with return
code EXIT_FAILURE.

FatalErrorOnInit does the same only during init and with
--init-errors-fatal enabled, otherwise it just calls SCLogWarning. So
then the macro returns to the caller.

Implement this for output setup.
 11 years ago
Victor Julien 4dd53c8f03 lua: dns support 11 years ago
Victor Julien 433e511b63 dns: generic inspect engines for DNS 11 years ago
Victor Julien 110e23964e detect: add AppLayerTxMatch call 11 years ago
Victor Julien e6129f7b47 dns: generic request/response detect lists 11 years ago
Victor Julien 2c8e8c2516 dns: rename type so it's purpose is more clear 11 years ago
Victor Julien a1e50b3138 lua: dns extensions 
Add DNS lua calls for getting queries, answers, authorities. Also
rcode and recusion desired flags.
 11 years ago
Victor Julien c46d472921 lua: initial DNS logging support 11 years ago
Eric Leblond 159a6d1cb4 tls-store: avoid log flooding 
In case we can't write in the certs directory, this is possible
we flood the log for each TLS session or even worse each TLS
packet.  So this patch puts a limit in the number of logged
messages related to file creation.
 11 years ago
Eric Leblond cbf5d88447 filestore: use SCFree instead of free 11 years ago
Eric Leblond b77cd22b98 tls-store: backward compatibility 
This patch implements backward compatibility in suricata.yaml
file. In case the new 'tls-store' output is not present in the
YAML we have to use the value defined in 'tls-log'.
 11 years ago
Eric Leblond 4db0a35f25 tls-store: now a separate module 
An design error was made when doing the TLS storage module which
has been made dependant of the TLS logging. At the time there was
only one TLS logging module but there is now two different ones.

By putting the TLS store module in a separate module, we can now
use EVE output and TLS store at the same time.
 11 years ago
Jeff Barber 893fc9660d Support for reconnecting unix domain socket log files 
Issue #1423
 11 years ago
Jason Ish b512580bbe logging: integrate rotation into SCConfLogOpenGeneric. 
Addresses issue 1492, and will make it harder to omit
rotation on new outputs.
 11 years ago
Jason Ish 14981cb2a8 rule vars: strip leading white space before looking up var. 11 years ago
sfd e58cfb6a05 Fix compile bad dereferences 
The src/source-erf-dag.c code was not compiling. It looks like some stats counters were added but not tested as the dereferences are incorrect.
 11 years ago
Zopieux cd038419fd stream_size operator comparison (fix issue #1488) 
`DetectStreamSizeParse` was first checking if mode[0] is '<', which is true for both '<' and '<=', thus '<=' (and resp. '>=') is never matched. This patch does the `strcmp` to '<=' (resp. '>=') within the if block of '<' (resp. '>') to fix #1488.
 11 years ago
Victor Julien 45fc619f79 logging: json output 
Make JSON output optional.

Cleanups.
 11 years ago
Victor Julien 126ecb3ebf logging: fix per output log formats 11 years ago
Victor Julien b51c4e608f logging: optional colors output 
Construct message per output method.
 11 years ago
Victor Julien b13de5bf08 logging: change newline handling 11 years ago
Victor Julien 1927b3000c output: cleanup 11 years ago
Victor Julien b30bdc21b5 logging: cleanup output API 
Make SCLogMessage master of the logging. Reduces complexity
of the SCLog macro's.
 11 years ago
Victor Julien d6fc6e874f log: reorganize SCLogOPIfaceCtx to make it more efficient 11 years ago
Victor Julien c2f4031a8c detect: fix settings override for reloads 11 years ago
Alexander Gozman cd9cc2559e Issue 1491: fix capabilities for pf_ring mode when running under non-root account 11 years ago
Alexander Gozman d36eba4e5e Fix issues #1493 and #1494 11 years ago
Giuseppe Longo d592d57039 file_data: check for signature alproto and flow 
Currently the following rule can't be loaded:
alert tcp any any -> any 25 (msg:"SMTP file_data test"; flow:to_server,established; file_data; content:"abc";sid:1;)
and produces the error output:
"Can't use file_data with flow:to_server or from_client with http or smtp."

This checks if the alproto is not http in a signature,
so permits to use flow keyword also.

Issue reported by rmkml.
 11 years ago
Victor Julien e583de0582 Minor unittest cleanups 11 years ago
Victor Julien f4f53924bb app-layer: fix coverity warnings 11 years ago
Victor Julien 6c792cb4cc erspan: respect vlan.use-for-tracking setting 11 years ago
Victor Julien b8211e8c04 htp: hide BUG_ON's behind DEBUG_VALIDATION 11 years ago
Victor Julien bd73553027 smtp json: fix potential crash on malloc failure 
** CID 1298888:    (FORWARD_NULL)
/src/output-json-email-common.c: 117 in JsonEmailLogJson()
/src/output-json-email-common.c: 140 in JsonEmailLogJson()
 11 years ago
Victor Julien 1ed8d7b538 detect analizer: fix minor coverity warning 
** CID 1298889:  Integer handling issues  (NEGATIVE_RETURNS)
/src/detect-engine-analyzer.c: 102 in EngineAnalysisFP()
 11 years ago
Victor Julien dd2afd51f7 file_data smtp: fix minor coverity warning 
CID 1298891:  Null pointer dereferences  (REVERSE_INULL)
Null-checking "curr_file" suggests that it may be null, but it has already been dereferenced on all paths leading to the check.
 11 years ago
Victor Julien bcff35fd9f smb: fix coverity warning 
** CID 1298892:  Incorrect expression  (UNUSED_VALUE)
/src/app-layer-smb.c: 1486 in SMBProbingParser()
 11 years ago
Victor Julien aa095864d3 pfring runmode: remove set that is never read 11 years ago
Victor Julien 21db5ee691 counters: reduce global usage 11 years ago
Victor Julien a5168d5977 dce_opnum: improve memory handling on parsing error 11 years ago
Victor Julien b3b7625be5 htp: fix test 11 years ago
Victor Julien c0807c3df5 fast log: clean up tests 11 years ago
Victor Julien 29fbcce50d detect hsbd: simplify resize logic 11 years ago
Victor Julien 4e7cb7b863 app-layer: update all protocols to accept NULL+EOF 
Update all non-HTTP protocol parsers to accept a NULL+EOF input.
 11 years ago
Victor Julien cf9ff6adbd app-layer: improve EOF handling 
On receiving TCP end of stream packets (e.g. RST, but also sometimes FIN
packets), in some cases the AppLayer parser would not be notified. This
could happen in IDS mode, but would especially be an issue in IPS mode.

This patch changes the logic of the AppLayer API to handle this. When no
new data is available, and the stream ends, the AppLayer API now gets
called with a NULL/0 input, but with the EOF flag set.

This allows the AppLayer parser to call it's final routines still in the
context of a real packet.
 11 years ago
Victor Julien df79c1019f print: make PrintRawDataFp take a const arg 11 years ago
Victor Julien 7451d33396 stream: update StreamMsg to don't have fixed size 
StreamMsg would have a fixed size buffer. This patch replaces the buffer
by a dynamically allocated buffer.

Preparation of allowing bigger and customizable buffer sizes.
 11 years ago
Victor Julien 3ffa01d150 stream: remove STREAMTCP_STREAM_FLAG_CLOSE_INITIATED logic 11 years ago
Giuseppe Longo 32563d51d1 http: rework UT 
This reworks UT based on
commit 3203555708
 11 years ago
Victor Julien 573d082219 http: memcap HTTP server inspect body code 11 years ago
Victor Julien 3163a57577 smtp file_data: fix wrong free 11 years ago
Victor Julien 9f52bdd1e5 flow timeout: prevent dead locks 
The flow timeout mechanism called both from the flow manager at run time
and at shutdown creates pseudo packets. For this it has it's own packet
pool, which can be depleted if the timeout logic is faster than the packet
processing threads. In this case the flow timeout would enter a wait loop.
The problem however, is that this wait loop would happen while keeping a
flow locked. This could lead to a race condition when the packet thread(s)
are waiting for the lock that the flow manager has.

This patch introduces a new packet pool call 'PacketPoolWaitForN', meant
to make sure that the thread's packet pool has at least N available
packets. The flow timeout paths use this to make sure enough packets are
available *before* grabbing the flow lock. If there aren't enough packets
available yet, the wait happens before the lock as well.

This still means the wait can happen while the flow hash row is locked, so
we do make sure some more packets are available when entering that. But
perhaps in the future we need a more precise logic there as well.
 11 years ago
Victor Julien 423d5fd0de http: improve inline body tracking 
Rewrite the sliding window handling for IPS mode for the server body.

The buffer used will have the following properties:
    left edge: inspected data - window_size
    right edge: the most recent data

Due to handling of compressed bodies, the data can be much bigger than
the configured window size.
 11 years ago
Victor Julien 79d34f4575 http: body pruning update 
Take inspect limits into account. Consider per direction inspect settings.
 11 years ago
Giuseppe Longo 8f1998e35f hsbd: inspect buffer depending on the engine mode 
Currently, data is buffered up to response-body-minimal size
and response-body-inspect-window before being inspected.
With this, in IPS mode, inspect data as it comes in up.

The sliding window concept is used here,
some data chunks are copied into the window (buffer)
then it's inspected.
 11 years ago
Giuseppe Longo a459376d2e app-layer-htp: add http_body_inline setting 11 years ago
Victor Julien 80615f7d43 ips: move counters in common struct and funcs 11 years ago
Victor Julien 813930f027 nfq: add ips stats 11 years ago
Victor Julien 14466a803d decode: create util function for basic counter updates 11 years ago
Jason Ish e3ce29f694 json-stats: log deltas 
If "deltas" is yes, log delta values as the name of the value
suffixed with _delta.
 11 years ago
Victor Julien 3e14aa729a detect-state: remove/hide BUG_ON statements 11 years ago
Victor Julien b7f1e9e370 file extract: add app_proto to logging 11 years ago
Victor Julien 925aac854e alproto: improve AppProtoToString 
Make AppProtoToString compilation fail on missing 'case's.
 11 years ago
Eric Leblond 0bafc71689 app-layer: add modbus to AppProtoToString 
It was missing causing protocol identified as modbus not to be
displayed in netflow events.
 11 years ago
Victor Julien ebb42f831c suppress: add track by_either mode 
So far suppress rules would apply to src or dst addresses of a packet.
This meant that if a ip would need to suppressed both as src and as dst,
2 suppress rules would be needed.

This patch introduces track by_either, which means that the ip(s) in the
suppress rule are tested against both the packets source and dest ip's.
If either of them is on the suppress list, the alert is suppressed.
 11 years ago
Victor Julien e85a44c383 suppress: support ip-lists 
Ticket: 1137

Support supplying a list of IP's to the suppress keyword. Variables from
the address-groups and negation is supported. The same logic (and code) is
used that is also used in parting the IP portions of regular detection
rules.
 11 years ago
Jason Ish 26fc5682ad hostbits: ignore leading and trailing white space 
Ignore leading and trailing space around the name and
direction tokens.
 11 years ago
Jason Ish 7c40c73482 json-stats: reorg threads and totals 
Totals are now placed at the top level instead of under a "Total"
object.

Threads are placed under a "threads" object.
 11 years ago
Jason Ish 1f2caf78c3 json-stats: log uptime in seconds, instead of a string 11 years ago
Jason Ish 0f1dd0d7ea flowbits: strip leading and trailing spaces in name 
Redmine bug 1481. Strip leading and trailing white space. Factor
out parsing from setup while in here.
 11 years ago
Zachary Rasmor 0edf28a4f8 Add Feature #1454. Generic eve-log prefix support. 11 years ago
Victor Julien a083513c49 decode: optimize DecodeThreadVars layout 
Put common counters on the first cache line. Please the flow output
pointer last as it's use depends on the flow logging being enabled
and even then it's only called very rarely.
 11 years ago
Victor Julien fe5a85aea0 decode: add erspan counter 11 years ago
Victor Julien 928957f0a3 decode: add ERSPANv1 decoder 
Only allow v1 to be parsed as thats what is tested.

Take vlan_id from the ERSPAN layer.
 11 years ago
Victor Julien aa6b24f814 decode: clean up tunnel decode logic 
Don't use mix of existing and custom types to indicate the next
layer.
 11 years ago
Victor Julien ef7cd043cc detect: various header cleanups 11 years ago
Victor Julien 5483b800c5 detect: remove struct/union tricks from Signature 11 years ago
Victor Julien 8949054212 detect: remove unused match_flags from inspect engines 11 years ago
Victor Julien 9fa2f85cc7 http: improve body pruning 
Take inspect window into account.
 11 years ago
Victor Julien 0bbc818b2d http: fix body tracking 
In HTTP body tracking for response bodies, pruning body chunks was broken
as the body parsing code wouldn't update HtpBody::body_parsed.
 11 years ago
Victor Julien 3203555708 http-client-body: create unittest util func 11 years ago
Eric Leblond d837562441 logging: fix modules ordering during logging 
With the previous code the order of the logging modules in the
YAML were determining which module was run first. This was not
wished and a consequences was that the EVE fileinfo module was
not correctly displaying the key 'stored' because it was
depending on a flag set alter by the filestore module.

This patch adds a priority file to the TmModule structure. The
higher the priority is set, the sooner the module is run in the
logging process. The RunModeOutput structure has also been
updated to contain the name of the original TmModule. Thus allowing
to define a priority for a RunModeOutput.

Currently only the filestore has a priority set. The rest of them is
set to the default value of zero.
 11 years ago
Eric Leblond be07620a60 output-lua: sync variable name with yaml 
'script-dir' was used in the code but we had 'scripts-dir' in the
configuration file. This patch fixes it to 'scripts-dir'.
 11 years ago
Jason Ish ae23144b67 --set - handle spaces on either side of '=' 
Discard spaces when provided as part of --set around the '='. For
example, "val=key", "val = key", "val= key" and "val =key" are
all equivalent now.
 11 years ago
Jason Ish d9fe95bc8a conf - function declaration style 
Use consistent style - function return type and declaration on
same line.
 11 years ago
DIALLO David 0a4fd39f9c modbus: fix heap-buffer-overflow in Modbus parser 
Modbus parser does not check length to extract/read data (read or write address,
quantity of data, etc.) that should be present.

In case of malformated data (invalid length in header), Modbus parser reads data
over the input data length.

Add check before extracting/reading data from input buffer to avoid head buffer
overflow.
 11 years ago
Victor Julien 07efec550d counters: use ptr to name instead of copy 
All counters have hardcoded names, so copies are not needed.
 11 years ago
Victor Julien 7e66c70507 counters: don't run if no counters have been registered 11 years ago
Victor Julien cb5aa8f8d5 counters: work around unix-socket init issues 11 years ago
Victor Julien e48153c6b0 counters: make threads cleanup all memory 11 years ago
Victor Julien 81548ae3e8 counters: clean up global context 11 years ago
Victor Julien 84b8829cb4 counters: turn flow.memuse into a global counter 11 years ago
Victor Julien 0a262acdfb counters: make DNS counters globals 11 years ago
Victor Julien ac069c579a counters: make tcp.memuse a global counter 11 years ago
Victor Julien cddbb0f606 http: make http.memuse a global counter 
http.memcap as well.
 11 years ago
Victor Julien f05d0692ef counters: remove references to 'perf' counters 11 years ago
Victor Julien faef92f8da counters: remove last and now unused tm_name reference 11 years ago
Victor Julien 83f27ae2a5 counters: remove old unix socket json logic 11 years ago
Victor Julien 41ead6611a counters: minor internal API cleanups 11 years ago
Victor Julien d2a9ef2680 counters: rename unparsable SCPCAElem to StatsLocalCounter 11 years ago
Victor Julien 4c3ccda72e counters: minor header cleanup 11 years ago
Victor Julien 752f03e7a4 counters: remaining s/SCPerf/Stats/g 11 years ago
Victor Julien 4362d0a6e9 counters: s/SCPerfPrivateContext/StatsPrivateThreadContext/g 11 years ago
Victor Julien 628c3b1bc7 counters: s/SCPerfPublicContext/StatsPublicThreadContext/g 11 years ago
Victor Julien 7e70f136ec counters: various renames and cleanups 11 years ago
Victor Julien 30cce2bd29 counters: s/SCPerfCounterSetUI64/StatsSetUI64/g 11 years ago
Victor Julien 1c0b4ee0ae counters: s/SCPerfCounterIncr/StatsIncr/g 11 years ago
Victor Julien 8992275b0c counters: s/SCPerfCounterAddUI64/StatsAddUI64/g 11 years ago
Victor Julien 60d9eb6790 counters: clean up defines 11 years ago
Victor Julien 1ef786e7cb counters: rename register API calls 
Also remove 'type' parameter which was always the same.
 11 years ago
Victor Julien 3fab736539 log-stats: make global/threads logging configurable 11 years ago
Victor Julien 2c9a2c8327 stats: support per thread stats in json output 
Default is only to output totals. Optionally per thread can be added.

Both can be enabled together.
 11 years ago
Victor Julien 175831331c stats json: replace strndup 
strndup is a banned function.
 11 years ago
Victor Julien 6565c86f96 stats-json: fixes and improvements 
Use proper LogFileCtx and MemBuffer handling so we can have multiple
loggers active at the same time.

Change 'date' field to timestamp, and use ISO notation to make it
the same as the other JSON outputs.
 11 years ago
Tom DeCanio e4e07d0c3b eve-log: stats logging code cleanup. 11 years ago
Tom DeCanio 117eed0385 eve-log: add JSON stats logging 
Support for counters in stats.log in eve output JSON stream.
 11 years ago
Victor Julien 23f17950bc counters: pass per thread stats to output api 
As well as the global (merged) stats.
 11 years ago
Victor Julien de82b6d31e counters: rename widely used pctmi var to sts (stats thread store) 11 years ago
Victor Julien 60c5ad4649 counters: call global counters funcs 11 years ago
Victor Julien 33756abd87 counters: split API init 
Split into early ctx initialization and post-config setup.
 11 years ago
Victor Julien d05eed3735 counters: start using Stats prefix 11 years ago
Victor Julien f300ad253e counters: simplify and speedup counters sync 11 years ago
Victor Julien 0478407833 counters: remove threadvars arg from SCPerfAddToClubbedTMTable 11 years ago
Victor Julien 2346a88db7 counters: remove thread module name from counters API 11 years ago
Victor Julien b5bd3dee13 stream: make tcp.reassembly_memuse counter global 
Fixes bugs #632 and #1178
 11 years ago
Victor Julien 06461e37da counters: global counters registration 11 years ago
Victor Julien 9bbef55c4d Fix harmless typo in IPOnlyCIDRItemNew's SCReturnPtr use 11 years ago
Victor Julien b293a4b7d0 counters: remove unused description 11 years ago
Victor Julien 711cd7b59b counters: merge counters from threads for output 
Merge counters so the table contains combined values from counters
from each thread.

Use global counter id's, track them in a hash.

Rename SCPCAElem members

Fix and improve average counters
 11 years ago
Victor Julien 7da657dc3d counters: remove unused public API calls and make them private 11 years ago
Victor Julien ac6e24c06a counters: make SCPerfSetupPrivate a function 11 years ago
Victor Julien 66635f0741 counters: minor cleanups 11 years ago
Victor Julien 74ab84c194 counters: introduce SCPerfSetupPrivate for thread setup 11 years ago
Victor Julien 799640f906 counters: make threadvars::perf_private_ctx static 
Update SCPerfGetAllCountersArray and add a UT workaround.
 11 years ago
Victor Julien 55cfab89e4 counters: SCPerfGetLocalCounterValue cleanup 
Return u64, update arguments.
 11 years ago
Victor Julien b34c6dc93a counters: remove references to SCPerfCounterAddDouble 
They were all in comments anyway.
 11 years ago
Victor Julien e9b067c1eb counters: make increment call take threadvars 
This hides the implementation from the caller.
 11 years ago
Victor Julien 9a8bff7d96 counters: threadvars s/sc_perf_pca/perf_private_ctx/g 11 years ago
Victor Julien 50bb995458 counters: rename threadvars public counters 11 years ago
Victor Julien 6ffbc3a362 counters: s/SCPerfContext/SCPerfPublicContext/g 11 years ago
Victor Julien 0a5ae1b403 counters: s/SCPerfCounterArray/SCPerfPrivateContext/g 
Goal is to make it's purpose clear.
 11 years ago
Victor Julien 9f584483be counters: minor cleanups 11 years ago
Victor Julien 1e8142c699 logfile: rename ALERT_ types to LOGFILE_TYPE_ 11 years ago
Eric Leblond 4c6a7bea30 output-json: suppress global variable 
It uses the new type field in the LogFileCtx instead.

This fixes the problem of not being able to use two eve-json
instance with different logging methods.
 11 years ago
Eric Leblond 636e3d93c0 log file: add type flag 
It will be used to store if the file is syslog or a real file.
 11 years ago
Eric Leblond 39d667ff56 output-json: fix type of data parameter 
The cast of data to AlertJsonThread was not correct as the real
type of the void pointer is a OutputJsonCtx. This was working by
luck because they both have a file_ctx as first element.
 11 years ago
Alexander Gozman f11e237d77 Feature #1440: support wildcards in rule filenames 11 years ago
David Cannings 4f8f53d080 Fix rcode parsing, as noticed by Coverity. 
Without support for OPT RR from RFC6891 (Extension mechanisms for DNS)
values of RCODE above 15 are not possible.  Remove dead code which will
never match.
 11 years ago
Jason Ish 9fdae82815 conf - process includes even if not at root node. 11 years ago
Jason Ish 56f6e37304 radix-tree - prevent out of bounds array access 
An IPv6 entry specified before an IPv4 entry on the host-os-policy
table can cause the stream byte array to be access one byte after
the end of the allocated memory at util-radix-tree.c:578.
 11 years ago
Jason Ish 3e5b8f48b1 Bug 1281 - Add tests for rule content of lengths > 255. 11 years ago
Jason Ish e2b04635a7 Bug 1281 - Accept rule content with lengths greater than 255. 11 years ago
Victor Julien 0e22e95e47 alert-json: fix stream logging for IPS mode 
Switch direction in IPS mode.
 11 years ago
Victor Julien 5037ea93f3 threads: add untimed control cond call 
The control conditions so far could only do timed waits, not normal
waits.
 11 years ago
Victor Julien c7bc9ae6a8 detect: minor cleanups 11 years ago
Victor Julien bc2b53f10b parsing: s/strtok/strtok_r/g 
Remove all strtok uses and replace them by strtok_r.

Do the same for Windows builds. Cygwin builds fine with strtok_r.

Add strtok to banned function list.
 11 years ago
Victor Julien fb479902e4 threading: explain purpose of threadvars mucond 11 years ago
Victor Julien 478719ee9d flow: don't hold tv_root_lock longer than needed 
Don't hold it longer than needed in shutting down.
 11 years ago
Victor Julien c96805e839 threading: remove unused cmd thread create func 11 years ago
Victor Julien df5e9d44ca unix-manager: convert to thread module 
Sync command thread for unix manager with other managers and make
it a full thread module.
 11 years ago
Victor Julien cc01b5f6b6 reference/classification: call global init for unittests 11 years ago
Victor Julien 34f2ff067b reference: update pcre globals use 
Don't update globals each time we parse, but instead do it once at
startup.
 11 years ago
Victor Julien 46d401e3bb classification: update pcre globals use 
Don't update globals each time we parse, but instead do it once at
startup.
 11 years ago
Victor Julien b2da57c827 reference: remove global 11 years ago
Victor Julien 393689ce44 classification: remove global from parsing 
Parsing code used a 'fd' global. Remove this.
 11 years ago
Victor Julien 9764a35604 stream: fix --disable-detection reassembly issue 
Due to an error at initialization, the stream engine would not disable
'raw' reassembly automatically when --disable-detection was used.

This lead to segments not getting cleared from the segment lists.
 11 years ago
Victor Julien c1558f5ac4 stream: remove FLOW_NO_APPLAYER_INSPECTION flag 
Instead, intruduce StreamTcpDisableAppLayer to disable app layer
tracking and reassembly. StreamTcpAppLayerIsDisabled can be used
to check it.

Replace all uses of FlowSetSessionNoApplayerInspectionFlag and
the FLOW_NO_APPLAYER_INSPECTION.
 11 years ago
Victor Julien b6798495c5 stream: remove FLOW_NO_APPLAYER_INSPECTION use from tests 11 years ago
Victor Julien b2e1854e2a stream: improve 'no app layer' handling 
When the session/flow was flagged as 'no applayer inspect', which
could happen as a result various reasons, packets would still be
considered by the app layer reassembly.

When ACK'd, they would be removed again. Depending also on the raw
reassembly.

In very long sessions however, this meganism could fail leading to
virtually endlessly growing segment lists.

This patch makes sure that segments that come in on a 'no app layer'
session are tagged properly or even not added at all.

Use a new ssn flag instead of flow flag for no app tracking.
 11 years ago
Victor Julien 22a810813c app-layer: add DisableAppLayer 
Move various app layer related flag setting calls into a utility
function "DisableAppLayer"
 11 years ago
Victor Julien f536099a67 app-layer: de_state optimization 
Add API to bypass expensive TX list walks. This API call is optional.

Implement it for HTTP and DNS.
 11 years ago
Victor Julien 5f0678120d detect-state: update test to check state storing 11 years ago
Victor Julien 37f0bd57b6 detect-state: handle duplicate inspect/match 
If for a packet we have a TX N that has detect state and a TX N+1 that
has no detect state, but does have 'progress', we have a corner case
in stateful detection.

ContinueDetection inspects TX N, but cannot flag the rule in the
de_state_sig_array as the next (TX N+1) has already started and needs
to be inspected. 'StartDetection' however, is then unaware of the fact
that ContinueDetection already inspected the rule. It uses the per
session 'inspect_id' that is only moved forward at the end of the
detection run.

This patch adds a workaround. It uses the DetectEngineThreadCtx::
de_state_sig_array to store an offset between the 'base' inspect_id
and the inspect_id that StartDetection should use. The data type is
limited, so if the offset would be too big, a search based fall back
is implemented as well.
 11 years ago
Victor Julien bc6e4140be detect: add de_state duplication check 
Add test to check if no duplicate destate is created.

Only enabled with DEBUG_VALIDATION.
 11 years ago
Victor Julien 9d198e6662 detect-state: fix state storing 
Fix storing state and bypassing detection. Previously we'd store
on a match only, meaning that StartDetection would rerun often.

Make sure StartDetection only stores if there is something to store.
 11 years ago
Victor Julien 359e2d68f5 detect-http-header: improve buffer handling 
Previously we could never be calling DetectEngineHHDGetBufferForTX
for TX N and then afterwards for TX N - 1. Due to changes in the
stateful detection code this is now possible.

This patch changes the buffer logic to take the 'inspect_id' as it's
base, instead of the first transaction that we are called with.
 11 years ago
Victor Julien 62e937672d detect-events: set SIG_MASK_REQUIRE_*_STATE for events 
Set SIG_MASK_REQUIRE_*_STATE for event rules to earlier discard
them.
 11 years ago
Victor Julien 8d0b090150 engine-analysis: print fast_pattern summary 
When using engine analysis for print fast_pattern stats, print a
short summary at the end containing per buffer:
- smallest fp
- biggest fp
- number of patterns
- avg fp len
 11 years ago
Victor Julien 21f9328989 lua: fix error handling 11 years ago
Victor Julien 52195a4192 http: add event for leading spaces on request line 
Libhtp will issue a warning in this case, so we can match on this.
This patch adds event, rule and unittest.
 11 years ago
Victor Julien 2f85308afe threads: fix missing unlock in error handling 
If TmThreadsUnregisterThread was called with out of range 'id', a lock
would not be cleared after returning from the function.

** CID 1264421:  Missing unlock  (LOCK)
/src/tm-threads.c: 2186 in TmThreadsUnregisterThread()
 11 years ago
Eric Leblond 0303245761 af-packet: use max packet size as snaplen 
If default_packet_size is set to 0, then we use the maximum packet
size as snaplen.
 11 years ago
Eric Leblond 43f691fef8 util-device: fix LiveBuildDeviceListCustom 
The code was assuming that the dictionnary containing the parameter
of a interface was ordered. But for YAML, the order is not assumed
so in case the configuration is generated we may not be able to
parse correctly the configuration file.

By iterating on child on main node and then iterating on subchild
and doing a match on the name, we are able to find the interface
list. In term of code, this algorithm was obtained by simply
removing the test on the name of the first child.
 11 years ago
Eric Leblond 268285c49f output-json-http: output status as an integer 
HTTP status is an integer and it should be written as such in the
JSON events. This will allow to have improved matching in log
analysis tools.
 11 years ago
Eric Leblond 58582df1c6 decode-der: decode DC keyword 
'DC' is used by some certificates and it was not currently translated
to a string.
 11 years ago
Giuseppe Longo 26ba647d58 filedata: read inspected tracker settings from suricata.yaml 11 years ago
Giuseppe Longo 4b5848616f filedata: implement inspected tracker 11 years ago
Giuseppe Longo 1f52410d0f UT: implement tests for inspection code 11 years ago
Giuseppe Longo d2657becc9 app-layer-smtp: make functions as public 11 years ago
Giuseppe Longo 84dc73d9de mpm: implement prefiltering for smtp 11 years ago
Giuseppe Longo f0c54d4764 Detect engine for smtp file_data file_data: inspecting smtp attachments 
Create a buffer to store reassembled file chunks,
and inspect the content.
 11 years ago
Giuseppe Longo b9468aba7c FileData: add stream_offset field 
This is required to store the offset for reassembling chunks.
 11 years ago
Giuseppe Longo 68cf3dd621 file_data: register keyword for smtp and tcp protocol 
Permits to use file_data keyword with smtp or tcp proto.
Also adds some unit tests
 11 years ago
Giuseppe Longo 04561f13d3 signature: set flags and test the protocol 
This checks if the signature's protocol is http
when setup the content keyword.

Also sets the proper flags based by protocol
since the flag SIG_FLAG_TOSERVER has to be set
if the proto is smtp, otherwise SIG_FLAG_TOCLIENT
is it's http.
 11 years ago
Giuseppe Longo 41a1a9f4af find and replace HSBDMATCH by FILEDATA 
This commit do a find and replace of the following:

- DETECT_SM_LIST_HSBDMATCH by DETECT_SM_LIST_FILEDATA
  sed -i 's/DETECT_SM_LIST_HSBDMATCH/DETECT_SM_LIST_FILEDATA/g' src/*

- HSBD by FILEDATA:
  sed -i 's/HSBDMATCH/FILEDATA/g' src/*
 11 years ago
Ken Steele eac83be121 Formatting cleanup in detect-replace.c 
Wrap lines longer than 80 characters
Add "static" for unit tests.
Use (void) for () for function arguments.
Add space after "while(" -> "while ("
Remove space after function names.
Put open bracket of function on a new line.
 11 years ago
Ken Steele ddec92676d Add a comment for DetectReplaceList 
Reworded a quote in PR 742 by Regit from Inliniac to explain why adding
the head of the list (really a FIFO) is the correct behavior.
 11 years ago
Ken Steele cf9da2be15 Fix DetectReplaceAddToList 
I see two problems:
1) If allocating a newlist fails, the function returns NULL, which then
   leaks any existing list elements.
2) The code to add the new value to the list works for the first two, but
   for not the third. For example, replist=A, A->next=B, B->next=NULL, then
   adding C results in replist=A, A->next=C, C->next=NULL, B is lost.

The fix pushes new values onto the head of the list, which might not be
what is needed, but there are no comments on what the function does, so I
made an assumption.
 11 years ago
Alexander Gozman d44eab82c1 Fix bug #1435 (data loss when dumping payloads to JSON) 11 years ago
Zachary Rasmor f0c659f82f Fix Bug #1204 
Fix typo that causes eve syslog settings code to be unreachable.
 11 years ago
Victor Julien 208d27abc7 stream: next_seq handling improvements 
Allow next_seq updating to recover from cases where last_ack has been
moved beyond it. This can happen if ACK's have been accepted for missing
data that is later retransmitted.

This undoes some of the previous last_ack update changes
 11 years ago
Alexander Gozman b12c53cd51 Add timezone to timestamp in JSON logs 11 years ago
David Cannings 2918a75da1 Added support for full parsing of the rcode header in DNS answer 
packets. Where rcode isn't "no error" this is displayed in both DNS and
JSON logs.

Note that this changes the current "No such domain" to "NXDOMAIN" in DNS
logs. This could be fixed if desired to maintain compatibility with
anybody crazy enough to parse the DNS log.

When the rcode is not "no error" (for example NXDOMAIN or SERVFAIL) it
is unlikely that there will be answer RRs. Therefore the rname from the
query is used.

Because the rcode applies to a whole answer packet (not individual
queries) it is impossible to determine which query RR caused the error.
Because of this most DNS servers currently reject multiple queries per
packet. Therefore each query RR is output instead with the relevant
error code, likely to be FORMERR if queries > 1.
 11 years ago
Victor Julien cf839c931f tls: force 'raw' reassembly after each record 
Trigger raw reassembly after each record and after the handshake.
 11 years ago
DIALLO David 04f3f5066d app-layer-modbus: fix deadlock in parsers 11 years ago
Victor Julien 6d170cadd7 smtp: fix mime boundary parsing issue 
If a boundary was longer than 254 bytes a stack overflow would result
in mime decoding.

Ticket #1449

Reported-by: Kostya Kortchinsky of the Google Security Team
 11 years ago
Victor Julien a4a1c396e1 pcap-file: fix malformed timestamp crash 
A bad timestamp would lead to SCLocalTime returning NULL. This case
wasn't checked, leading to a NULL deref.

Reported-by: Kostya Kortchinsky of the Google Security Team
 11 years ago
Pierre Chifflier fa73a0bb8f Fix possible wrap in uint32_t addition in DER parser 
Signed-off-by: Pierre Chifflier <pierre.chifflier@ssi.gouv.fr>
 11 years ago
Victor Julien a2d0441380 Bug 1340: fix missing flow cleanup 
Fix missing flow output cleanup function leading to a crash in the
unix socket mode.
 11 years ago
Victor Julien 04ccfda639 pcap: implement LINKTYPE_NULL 
Implement LINKTYPE_NULL for pcap live and pcap file.

From: http://www.tcpdump.org/linktypes.html

"BSD loopback encapsulation; the link layer header is a 4-byte field,
 in host byte order, containing a PF_ value from socket.h for the
 network-layer protocol of the packet.

 Note that ``host byte order'' is the byte order of the machine on
 which the packets are captured, and the PF_ values are for the OS
 of the machine on which the packets are captured; if a live capture
 is being done, ``host byte order'' is the byte order of the machine
 capturing the packets, and the PF_ values are those of the OS of
 the machine capturing the packets, but if a ``savefile'' is being
 read, the byte order and PF_ values are not necessarily those of
 the machine reading the capture file."

Feature ticket #1445
 11 years ago
Tom DeCanio 97a2d1ac26 fix reputation parser so that it accepts ipv6 addresses in configuration file. 11 years ago
Jason Ish 11d6770ae4 afl - SCHINfoLoadFromConfig - check for NULL before parsing. 
Found by AFL on suricata.yaml.
 11 years ago
Jason Ish 75d21851cf afl - Don't fail if app-layer proto enabled value is NULL. 
Found by using AFL on suricata.yaml.
 11 years ago
Eric Leblond 0376b60da9 email-json: free temporary 'cc' string 11 years ago
Eric Leblond bd67000b69 email-json: free temporary 'to' string 11 years ago
Eric Leblond 7b8184947a app-layer-smtp: fix SMTPTransactionFree function 
A typo was causing some freeing tasks not to be executed.
 11 years ago
Victor Julien 91ddf85323 profiling: fix sorting on very long runs 
Fix poor int handling breaking sorts of profiling on long runs
where the numbers could get really big.
 11 years ago
Victor Julien 1a83fee5f5 xbits: fix coverity warnings 
Switch statement would contain NOALERT even though this was
unreachable.
 11 years ago
Victor Julien 54d5e2eed5 coverity fix: don't do pointer check on static array 11 years ago
Victor Julien b9aaf5a9ab Fix potential deadlock in output 
Coverity:
** CID 1296115:  Program hangs  (ORDER_REVERSAL)
/src/tm-threads.c: 1670 in TmThreadClearThreadsFamily()

The problem is with the by default unused '%m' output parameter.
To get the thread vars it takes the tv_root_lock. This may already
be locked by the calling thread. Also, it could lead to a case of
wrong lock order between the tv_root_lock and the thread_store_lock.

Very unlikely to happen though.

As the %m param isn't really used (by default) this patch just
disables it.
 11 years ago
Victor Julien 94321b8a2f packet pool: fix memleaks 
Don't kill flow manager and recyclers before the rest of the threads. The
packet threads may still have packets from their pools. As the flow threads
would destroy their pools the packets would be lost.

This patch doesn't kill the threads, it just pulls them out of their run
loop and into a wait loop. The packet pools won't be cleared until all
threads are killed.

Wait for flow management threads to close before moving on to the
next steps in the shutdown process.

Don't destroy flow force reassembly packet pool too early. Worker
threads may still want to return packets to it.
 11 years ago
Victor Julien 22142d9b8b threshold: cleanups 
Use pcre_copy_substring in the most common part of threshold rule
parsing. Minor other cleanups.
 11 years ago
Victor Julien e86e1e0282 reference: use pcre_copy_substring 11 years ago
Victor Julien cc38df8074 classification: cleanups 
Reduce hash table size as regular classification files are usually
below 100 in size. It's not performance critical anyway.

Convert pcre_get_substring calls to pcre_copy_substring.
 11 years ago
Victor Julien deb98fab0c packet pool: init pool for autofp workers as well 
Introduce a new 'PacketPoolInitEmpty' as these pools will not often
need packets.

Also, don't double 'destroy' in the main thread.
 11 years ago
Victor Julien 1884227019 autofp: reduce flow storage space requirement 
Use int16_t instead of a regular int to safe 2 bytes per flow.
 11 years ago
Victor Julien 643c30f67f threading: add missing locks to threading storage 
Lock threading root lock when walking or updating the threading
storage for 'ThreadVars' objects.
 11 years ago
maxtors e7658fd4f4 Added new WebDAV http methods. 
Added more WebDAV functions. A complete list of what http
methods libhtp can handle can be found at:
https://github.com/OISF/libhtp/blob/0.5.x/htp/htp_core.h#L260.
So now the methods array reflects these available functions.

The comments have also been changed to reflect the desired style.
 11 years ago
maxtors 3b26b07963 Reworked pattern registration structure and feedback issues. 
1) Reworked pattern registration for http methods and versions.

Instead of being a manual and verbose action of adding one
and one http method with N-amount if prefix spacings and
the same for HTTP versions (eg. HTTP/1.1) i moved it all
to be loop based actions reading values from char arrays.

In the future all that is needed is to add new methods
to the arrays and they will be added as a pattern.

2) Modified pattern registration after feedback.

Changed variable used in snprintf for http method registration
Should have been size of dest buffer at not another var (catsize)
that i had created. Also removed this variable.

Fixed a typo in the comment for registering http versions.
TO_CIENT -> TO_CLIENT.
 11 years ago
maxtors 7a9e963619 Added http methods for webdav (rfc4918). 
Included the following methods to app-layer-htp.c:
- PROPFIND
- PROPPATCH
- MKCOL
- COPY
- MOVE
- LOCK
- UNLOCK
 11 years ago
maxtors a7abaf538c Added DELETE and PATCH http methods to app-layer-htp.c 
Added two new http methods to the http decoding done in
app-layer-htp.c so that these messages will be decoded as well.
 11 years ago
Victor Julien 915eb992ca app-layer: update protocol detection tests 
AppLayerTest09, AppLayerTest10 and AppLayerTest11 depended on a max
protocol detection pattern size of < 17. Update the tests to pass one
extra byte to the app layer. This makes the protocol detection code
flag the session as 'proto detection completed' again.
 11 years ago
David Maciejak 48971afdfb Fix Unit ID check 11 years ago
Eric Leblond f35001a614 util-running-mode: setup config file 
Without that we get warning message.
 11 years ago
Eric Leblond 6f5c4c12e1 list keywords: fix regression on app layer name 
It was not anymore displayed.
 11 years ago
Eric Leblond d98060410c list keywords: restore 2.0 similar output 
Unset was previously displayed if the application layer was not
set. Code before this patch was displaying '(null)' which is not
user friendly.
 11 years ago
Alexander Gozman 568635c2e1 Reject ICMP traffic, not only TCP/UDP 11 years ago
Jason Ish 1f9d9256e7 spelling - dubbed -> duped (for duplicate) 11 years ago
Jason Ish 8911b04077 DetectGidSet - safer stripping of quotes. 
Discovered by AFL when using a rule like:
    alert tcp any any -> any any (content:"ABC"; gid:";)
resulting a negative array index.
 11 years ago
Jason Ish da88b3b787 DetectSidSetup - safer stripping of quotes. 
Discovered by AFL when using a rule like:
    alert tcp any any -> any any (content:"ABC"; sid:";)
would result in a negative array index.
 11 years ago
Jason Ish ca60d8c370 ParseSizeString - don't abort on unknown unit type. 11 years ago
Victor Julien 00d6298666 host: handle timeout 
Set expiration for regular hostbits keyword.
 11 years ago
Victor Julien ab7882fbf3 ippair: handle timeouts in the flow manager 
Like host and defrag, handle timeouts of ippair in the first flow
manager thread.
 11 years ago
Victor Julien e8a5925012 ippair: implement basic timeout check 
The only user is the xbits subsys, so it's timeout controls all.
 11 years ago
Victor Julien f2349e10ad hosts: consider hostbits/xbits status in timeout 
Consider the host's xbits expiry status when checking the host for
timeout. If a single active non-expired bit is found, the host won't
be timeout just yet.
 11 years ago
Victor Julien 67dd5c0430 host: implement hostbits/xbits expire 11 years ago
Victor Julien ca1f3e68d4 ippair: implement xbits expiration 11 years ago
Victor Julien fbdcffabc2 xbits: expire (first steps) 11 years ago
Victor Julien 7b79b9458d xbits: implement ip_src/ip_dst support 
Just a wrapper for regular 'hostbits'.
 11 years ago
Victor Julien 57d93cbcfe hostbits: prepare for xbits integration 11 years ago
Victor Julien 8e6453faac xbits: prepare for non-ippair support 11 years ago
Victor Julien dea8dda3c1 vars: redo var name <-> idx handling 
Can't use sm type anymore as multiple var carriers (host/flow/etc) will
all have xbits.
 11 years ago