aiden
/
suricata
mirror of https://github.com/OISF/suricata
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,292 Commits
8 Branches
165 Tags
229 MiB
main
main-7.0.x
main-8.0.x
master-6.0.x
master-5.0.x
master-4.1.x
master-4.0.x
master-3.2.x
suricata-8.0.2
suricata-7.0.13
suricata-8.0.1
suricata-7.0.12
suricata-8.0.0
suricata-7.0.11
suricata-8.0.0-rc1
suricata-8.0.0-beta1
suricata-7.0.10
suricata-7.0.9
suricata-7.0.8
suricata-7.0.7
suricata-7.0.6
suricata-6.0.20
suricata-7.0.5
suricata-6.0.19
suricata-6.0.18
suricata-7.0.4
suricata-6.0.17
suricata-7.0.3
suricata-6.0.16
suricata-7.0.2
suricata-6.0.15
suricata-7.0.1
suricata-6.0.14
suricata-7.0.0
suricata-7.0.0-rc2
suricata-6.0.13
suricata-6.0.12
suricata-6.0.11
suricata-7.0.0-rc1
suricata-6.0.10
suricata-6.0.9
suricata-7.0.0-beta1
suricata-6.0.8
suricata-6.0.7
suricata-6.0.6
suricata-5.0.10
suricata-6.0.5
suricata-5.0.9
suricata-6.0.4
suricata-5.0.8
suricata-6.0.3
suricata-5.0.7
suricata-6.0.2
suricata-5.0.6
suricata-6.0.1
suricata-5.0.5
suricata-4.1.10
suricata-4.1.9
suricata-5.0.4
suricata-6.0.0
suricata-6.0.0-rc1
suricata-6.0.0-beta1
suricata-5.0.3
suricata-4.1.8
suricata-4.1.7
suricata-5.0.2
suricata-4.1.6
suricata-5.0.1
suricata-5.0.0
suricata-5.0.0-rc1
suricata-4.1.5
suricata-5.0.0-beta1
suricata-4.1.4
suricata-4.1.3
suricata-4.0.7
suricata-4.1.2
suricata-4.1.1
suricata-4.1.0
suricata-4.0.6
suricata-4.1.0-rc2
suricata-4.1.0-rc1
suricata-4.0.5
suricata-4.1.0-beta1
suricata-4.0.4
suricata-4.0.3
suricata-4.0.2
suricata-3.2.5
suricata-4.0.1
suricata-3.2.4
suricata-4.0.0
suricata-4.0.0-rc2
suricata-3.2.3
suricata-4.0.0-rc1
suricata-4.0.0-beta1
suricata-3.2.2
suricata-3.1.4
suricata-3.2.1
suricata-3.2
suricata-3.2RC1
suricata-3.1.3
suricata-3.2beta1
suricata-3.1.2
suricata-3.1.1
suricata-3.1
suricata-3.0.2
suricata-3.1RC1
suricata-3.0.1
suricata-3.0.1RC1
suricata-3.0
suricata-2.0.11
suricata-3.0RC3
suricata-3.0RC2
suricata-3.0RC1
suricata-2.0.10
suricata-2.0.9
suricata-2.1beta4
suricata-2.0.8
suricata-2.0.7
suricata-2.1beta3
suricata-2.0.6
suricata-2.0.5
suricata-2.1beta2
suricata-2.0.4
suricata-2.1beta1
suricata-2.0.3
suricata-2.0.2
suricata-2.0.1
suricata-2.0.1rc1
suricata-2.0
suricata-2.0rc3
suricata-2.0rc2
suricata-2.0rc1
suricata-2.0beta2
suricata-1.4.7
suricata-1.4.6
suricata-1.4.5
suricata-2.0beta1
suricata-1.4.4
suricata-1.4.3
suricata-1.4.2
suricata-1.4.1
suricata-1.3.6
suricata-1.4
suricata-1.3.5
suricata-1.4rc1
suricata-1.3.4
suricata-1.4beta3
suricata-1.3.3
suricata-1.4beta2
suricata-1.3.2
suricata-1.4beta1
suricata-1.3.1
suricata-1.3
suricata-1.3rc1
suricata-1.3beta2
suricata-1.3beta1
suricata-1.2.1
suricata-1.2
suricata-1.2rc1
suricata-1.2beta1
suricata-1.1.1
suricata-0.8.2
suricata-1.0.0
suricata-1.0.1
suricata-1.0.2
suricata-1.0.3
suricata-1.0.4
suricata-1.0.5
suricata-1.1
suricata-1.1beta1
suricata-1.1beta2
suricata-1.1beta3
suricata-1.1rc1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'c446abeb47'
${ noResults }
Commit Graph

5783 Commits (c446abeb471d8be516bf11f0106797526b7122b3)

Author SHA1 Message Date
Eric Leblond d9b602fc0f json-smtp: add tx_id param to metadata generation 
In all metadata generation contexts we know the tx_id so we better
used it to log the correct transaction and not an other one.
 10 years ago
Eric Leblond 0ef0f1d526 json-smtp: change copyright date 10 years ago
Eric Leblond f03a7a032f json-alert: add smtp elements in alert 10 years ago
Eric Leblond 77a5273cde json-smtp: fix a debug message 10 years ago
Eric Leblond 946f2a6acc email-json: add bcc to extended fields 10 years ago
Eric Leblond fbd6428f1b smtp-layer: add MAIL FROM parsing test in unittest 10 years ago
Eric Leblond 534360fc02 smtp-layer: add HELO parsing test in unittest 10 years ago
Eric Leblond f81f353d1f email-json: add 'date' field extraction 10 years ago
Eric Leblond 6f033747ec util-decode-mime: add unittests for field fetching 10 years ago
Eric Leblond 17edff6c5e unittests: finally register MIME tests 10 years ago
Eric Leblond 6e0668125c email-json: improve log message 10 years ago
Eric Leblond dad1f85edb email-json: add author 
Add myself as author and change the copyright date.
 10 years ago
Eric Leblond caa8982b43 email-json: add some fields 
This patch adds some fields to the list of extracted fields.
 10 years ago
Eric Leblond ca52fa91dd email-json: body md5 logging is optional 10 years ago
Eric Leblond a719ea3c92 email-json: add capa to display subject md5 
To be able to identify mails with identical subjects without
using the subject itself as a key, it is possible to use the md5
hash of the subjet string. This allows to limit the privacy impact.
 10 years ago
Eric Leblond 881aa3efce email-json: don't log subject by default 
It seems to be a bit too intrusive for the privacy so this patch
adds this field to the extended logging only.
 10 years ago
Eric Leblond 7bb38f7c30 decode-mime: fix typo in comment 10 years ago
Eric Leblond 431dc155aa email-json: delete white space from "from" 
The From field is handled separatly and it could also starts by
white spaces.
 10 years ago
Eric Leblond abcaf46193 email-json: delete leading white spaces 
Some mail clients are using tabulation and/or space for comma
separated list. This patch removes them so the event will contain
only significative characters.
 10 years ago
Eric Leblond 4c0f8803e7 email-json: factorize the code 10 years ago
Eric Leblond a7ef0c05ba email-json: add LOG_EMAIL_COMMA type 
extract these data types by treating them as a comma separated list.
 10 years ago
Eric Leblond 3456ec467f email-json: add custom fields support 
This patch adds a way to specify which MIME fields to log via
the custom keyword in the EVE configuration. it also adds an
extended logging where some fields are added. The logging support
mono value fields as well as multivalue fields via the use of
JSON array.
 10 years ago
Eric Leblond 714c30a127 decode-mime: introduce MimeDecFindFieldsForEach 
This patch introduces a new function that can be used to handle
multivalued MIME fields. A callback function can be called for
each corresponding field value.
 10 years ago
Eric Leblond 54038f5691 file-json: log 'email' information 
Log information coming from email/MIME decoding in the message.
 10 years ago
Eric Leblond ab941305d5 email-json: add function to export data 10 years ago
Eric Leblond 77119a3186 file-json: output smtp proto info 10 years ago
Eric Leblond 47a199ee97 smtp-json: introduce function to output smtp data 10 years ago
Eric Leblond 94dbd303e4 file-json: log http data using common function 10 years ago
Eric Leblond 4ef12dcf5d alert-json: use new JsonHttpAddMetadata function 
This patch uses the newly introduced function to handle the logging
of HTTP data.
 10 years ago
Eric Leblond bccabe3813 http-json: introduce JsonHttpAddMetadata function 
This function will be usable in other logging components to add
the http data to their messages.
 10 years ago
Eric Leblond d7e13c2c03 email-json: output MIME parsing status 
If the status is not PARSE_DONE then in that case we may have
imcomplete information. Increasing the stream reassemly depth
in that case would be a good idea.
 10 years ago
Eric Leblond a233a982ea decode-mime: add function to get status 
This new function return the textual status of MIME parsing.
 10 years ago
Eric Leblond 9900558428 smtp: add 'body-md5' mime option 
This option will allow the user to select weither or not he wants
to journalize the md5 of the mail body.
 10 years ago
Eric Leblond ea311c1594 email-json: export md5sum of body 
The body_md5 has been added and contain the value of the md5sum
of the body.

This patch is using the state PARSE_DONE on the MIME state to
detect when a message has been completely parsed.
 10 years ago
Eric Leblond d39009ca58 decode-mime: compute body md5 
This patch is computing the md5 sum of the body of the MIME message.
This will allow to detect messages with same content and sent to
different people.
 10 years ago
Eric Leblond e43eb76abd app-layer-stmp: simplify code 
Delete a only used once goto to a point where we only do a return.
 10 years ago
Eric Leblond 0f3979cc81 output-json-smtp: output RCPT TO fields 
This patch uses an array to output the RCPT TO fields to the
JSON message.
 10 years ago
Eric Leblond 752fdba957 app-layer-smtp: parse and extract RCPT TO fields 
Add the RCPT TO fields to a linked list stored in the transaction.
 10 years ago
Eric Leblond 2abae3f0a1 smtp-json: update SMTP EVE messages 
This patch updates SMTP message to have them feature a 'smtp'
section which will contain all fields coming from the smtp
protocol.
 10 years ago
Eric Leblond 7bca8268bc app-layer-smtp: extract and store HELO and MAIL FROM 
This patch updates the SMTP transaction and SMTP state to be able
to contain the HELO and MAIL FROM fields.
 10 years ago
Eric Leblond 5c26a2f2c8 email-json: move email fields to email section 
This patch changes the way smtp message are written. It is using
the "email" key to store the email related fields. This will
allow to do the same search through SMTP and IMAP if we implement
this last one.
 10 years ago
Victor Julien 77302e5d51 threshold: remove debug message from info loglevel 10 years ago
Victor Julien fc7f090cd3 flow: add missing storage size to checks, output 10 years ago
Victor Julien 37fa4a4876 host: update host size logic 
Instead of using (sizeof(Host)+HostStorageSize()) in many places,
create a simple size variable that is set during setup.
 10 years ago
Victor Julien 480e91edac ippair: update ippair size logic 
Instead of using (sizeof(IPPair)+IPPairStorageSize()) in many places,
create a simple size variable that is set during setup.
 10 years ago
Victor Julien ff769b73a7 stream: improve retransmission detection 
Consider packets starting before last_ack and ending after it also
to be retransmissions. This way we can see if they are having
different data.
 10 years ago
Victor Julien 6b2f831a70 mpm: SGH maxlen was actually minlen, so rename 10 years ago
Victor Julien 2716c78628 mpm: improve SGH content len tracking 
SGH's track content length for rule grouping.

This patch changes the logic to only consider the pattern that is
used in the mpm for a sig.
 10 years ago
Victor Julien e529ebb50e mpm: redo uri maxlen logic 
The mpm_uricontent_maxlen logic was meant to track the shortest
possible pattern in the MPM of a SGH. So a minlen more than a maxlen.

This patch replaces the complicated tracking logic by a simpler
scheme. When the SGH's are finalize, the minlen is calculated.

It also fixes a small corner case where the calculated "maxlen" could
be wrong. This would require a smaller pattern in a rule to be forced
as fast pattern.
 10 years ago
Victor Julien df95d375bb detect: improve comments on mpm 10 years ago
Victor Julien 496f9800ac mpm: remove used counter 10 years ago
Victor Julien c53c9b4b20 mpm: remove bloated counting logic 
Counters were only used to print debug info.
 10 years ago
Victor Julien da7bad7c1b mpm: improve debug output 10 years ago
Victor Julien 977074930b mpm: use IPPROTO_TCP for readability 10 years ago
Victor Julien a559c41295 mpm: optimize & debug validate 
Wrappers are called only if a mpm_ctx is available. So remove the test
for a null ctx and replace it by a debug validation BUG_ON.
 10 years ago
Victor Julien 0dd3b73db2 mpm: assume we'll likely have a mpm_ctx 10 years ago
Victor Julien 7c336f4190 mpm: indent fix, no functional change 10 years ago
Victor Julien a00d83f1f5 mpm: change direction checking in mpm wrappers 
Instead of having reachable assertions, use DEBUG_VALIDATE_BUG_ON
 10 years ago
Victor Julien 804f861967 debug validation: introduce DEBUG_VALIDATE_BUG_ON 
DEBUG_VALIDATE_BUG_ON(exp) will call BUG_ON(exp) if debug validation
is compiled in. Otherwise it's a no-op.
 10 years ago
Victor Julien e755913b4b mpm: minor fixes and cleanups 10 years ago
Victor Julien cacf425bd3 stream: improve handling of GAPs at stream start 
Detect and handle gaps at the start of the stream, when there may
be no segments in the list (yet).
 10 years ago
Victor Julien 574ef0ad2a stream: RST last_ack update fix 
Only use ACK if ACK flag was set and ACK value is valid.
 10 years ago
Victor Julien 3ca44219dc proto detect: more bypass conditions 
More exceptional cases for protocol detection. In very unbalanced flows,
where just a few bytes are sent toserver and many toclient, proto detect
might not complete in time on the toserver direction. This can lead to
queuing up many segments in the toclient direction.

Another case is that in come cases the stream is flagged as proto detect
done, but the flows proto detect flags are not set. This is now handled
by the ProtoDetectDone() check.
 10 years ago
Victor Julien fa8dc77dcc debug validation: add segment list sanity check 10 years ago
Victor Julien e67188e437 detect: fix issue with smsg and seq wraps 
Due to a broken sequence number check, detect could fail to process
smsgs in case of a sequence wrap. This could lead to excessive use
of smsg's but also of segments, since these aren't cleared until the
smsg containing them is.
 10 years ago
Victor Julien 8ac49d9129 stream: allow next_seq catch up after pkt loss 
If next_seq falls behind last_ack, force update it.
 10 years ago
Victor Julien 596465b76d stream: use reassembly fast path after proto detect 
Use the reassembly fast paths only after protocol detection has completed.
In some corner cases the sending of smaller segments lead to protocol
detection failing.
 10 years ago
Victor Julien 34ed15e182 stream: fix protocol detection issue for GAPs 
If the protocol required TOSERVER data first, but the SSN started with
a GAP, then the TOCLIENT side would get stuck in an expensive path:

1. it would run detection on TOCLIENT
2. it would try to force reassembly for TOSERVER
3. it would reset the detected protocol as TOSERVER failed
4. it would not evict any segment

This had 2 consequences:
1. on long running sessions this could lead to using lots of memory
   on segments, denying other sessions resources
2. wasted cycles on protocol detection and segment list management

This patch introduces a fix. It checks in the (2) stage above, whether
the opposing stream (that we depend on) it is a NOREASSEMBLY state. If
so, it gives up on this side of the session as well.
 10 years ago
Victor Julien 708e80c900 stream: optimize proto detect segment handling 
In case of protocol detection not yet being complete, the segment
list was walked unconditionally to unset the app layer processed
flag. Optimize this to bail on the first segment that doesn't have
the flag set.
 10 years ago
Eric Leblond a286715367 host-storage: document host storage API 10 years ago
Jason Ish 06beca62f5 app-layer: template for application layer content inspection 10 years ago
Jason Ish bcda92134d app-layer: template for application layer tx logger 10 years ago
Jason Ish c1b921265f app-layer: template for application layer parser 10 years ago
Victor Julien cb060f16fd detect plugin: add template 10 years ago
Victor Julien 830dd32417 decoder: add template/example 
Add fictional example / template for a packet decoder. It's not invoked
anywhere.
 10 years ago
Victor Julien 371648a8c6 lua: TLS support 
Support TLS in Lua detection scripts.

function init (args)
    local needs = {}
    needs["tls"] = tostring(true)
    return needs
end

function match(args)
    version, subject, issuer, fingerprint = TlsGetCertInfo();
    if version == nil then
        return 0
    end
    str = string.format("Version %s\nIssuer %s\nSubject %s\nFingerprint %s",
                        version, issuer, subject, fingerprint)
    SCLogInfo(str);
    return 1
end
 10 years ago
Victor Julien 1e84950017 detect-lua: set direction 10 years ago
Victor Julien 8b3afb6986 lua: add direction support 10 years ago
Victor Julien a798cf9518 DNS: refactor tx completion logic 
Use simple bool values to track the transaction state in both directions.

A tx is only created in two cases:
1. full request parsed
2. response parsed (request missing)

This is true even for multi-packet TCP requests.

This leads to the following tx completion logic for the request side:
the presence of a tx implies the request is complete

On the response side, we consider the tx complete when we have seen
the response. If the DNS parser thinks the response was lost, we also
flag the response side as complete.
 10 years ago
Victor Julien 6b2fb3fd5d Fix minor format string issues 10 years ago
Victor Julien ba81c4d290 autotools: cleanup 
Remove most of the CFLAGS updates from configure. Flags are now (mostly)
set in AM_CLFLAGS.

Update all -DBLAH additions to CFLAGS to use AC_DEFINE([BLAH], ...)

Improve Lua vs LuaJIT checking.

Improve the configure output a bit.

Lots of smaller cleanups.
 10 years ago
Antti Tönkyrä 63a47967d7 source-pfring: don't set cluster mode when using ZC and VLAN tracking is disabled 10 years ago
gureedo c31660e5fd netmap: enable zero-copy mode only when copy-mode is specified. 
Perform zero-copy checks only when copy-mode is specified.
 10 years ago
Victor Julien cbf46c44ec transaction inspection: fix limit enforcement 
Make sure we're never wrapping around the size value.
 10 years ago
Antti Tönkyrä 834c36659e Allow colon in SSH version, at least some trojaned PuTTY clients have version like Putty-Local: Timestamp HH:MM:SS 11 years ago
Victor Julien 1235c578b2 xff: support ports and more ipv6 notations 
It's not uncommon to see an header like:
X-Forwarded-For: 1.2.3.4:56789

This patch recognizes this case and ignores the port. It also supports
this for IPv6 if the address has the following notation:
X-Forwarded-For: [12::34]:1234

This patch also adds unittests.
 11 years ago
Victor Julien 979bd35277 detect loader: move to own file 11 years ago
Victor Julien cfeaf42cab detect-loaders: configurable amount of loaders 11 years ago
Victor Julien f871c0e1b8 debug: packet pool init/destroy validation 
Validate packet pool handling:
- pools are initialized before use
- pools are not used after destroy
- pools are not double initialized/destroyed
 11 years ago
Victor Julien e44fd97d72 unittests: use a global packetpool 11 years ago
Victor Julien 99c0a7ad72 multi-detect: improve memory handling of setup code 11 years ago
Victor Julien b7b27684c2 multi-detect: detect loader for unix socket 
Move the tenant load and reload commands to be executed by the detect
loader thread(s).

Limitation: no yaml parsing in parallel. The Conf API is currently not
thread safe, so don't load the tenant config (yaml) in parallel.
 11 years ago
Victor Julien eb09118d64 detect: create loader threads 
To speed up startup with many tenants, tenant loading will be parallelized.
As no tempary threads should be used for these memory allocation heavy
tasks, this patch adds new type of 'command' thread that can be used to
load and reload tenants.

This patch hardcodes the number of loaders to 4. Future work will make it
dynamic.

The loader thread essentially sleeps constantly. When a tasks is sent to
it, it will wake up and execute it.
 11 years ago
Victor Julien 82aa419431 multi-detect: set tenant id on pseudo packets 
Store the tenant id in the flow and use the stored id when setting
up pesudo packets.

For tunnel and defrag packets, get tenant from parent. This will only
pass tenant_id's set at capture time.

For defrag packets, the tenant selector based on vlan id will still
work as the vlan id(s) are stored in the defrag tracker before being
passed on.
 11 years ago
Victor Julien e19c41a807 multi-detect: hash lookup for tenants 
Use hash for storing and looking up det_ctxs.
 11 years ago
Victor Julien 722c56dbf3 detect: clean up thread free code 
Introduce DetectEngineThreadCtxFree that doesn't need a 'ThreadVars'
pointer.
 11 years ago
Victor Julien 4f77e8967b multi-detect: make threshold prefix aware 
Make threshold loading prefix aware, so it can be part of tenant
configuration.

If the setting is missing from the tenant, the global setting is tried
and if that too is missing, the global default is used.

Note: currently per host thresholds are tracked globally and NOT per
tenant.
 11 years ago
Victor Julien a6e3cec9e5 multi-detect: make reference prefix aware 
Make reference loading prefix aware, so it can be part of tenant
configuration.

If the setting is missing from the tenant, the global setting is tried
and if that too is missing, the global default is used.
 11 years ago
Victor Julien 5fff250749 multi-detect: make classification prefix aware 
Make classification loading prefix aware, so it can be part of tenant
configuration.

If the setting is missing from the tenant, the global setting is tried
and if that too is missing, the global default is used.
 11 years ago
Victor Julien d8181802d3 multi-detect: add reload-tenant command 
Allow for a tenant to be reloaded. The command is the same as the
register-tenant command, so with a yaml and tenant-id as argument.
However this replaces an existing tenant.
 11 years ago
Victor Julien 8673801ea3 multi-detect: add tenant id to alert json output 
Add a integer field "tenant_id" to the JSON alert output.
 11 years ago
Victor Julien f4c9915066 multi-detect: store tenant id in packet 
Store tenant id in the packet so that the output API's can log it.
 11 years ago
Victor Julien 642c267dc4 multi-detect: refuse to add duplicate tenant 
Generate error if tentant to be added is already loaded.
 11 years ago
Victor Julien 4e0683c3f5 multi-detect: cleanup, reuse tenant loading code 
Reuse tenant loading from YAML code for Unix Socket.
 11 years ago
Victor Julien 646eb4c2a8 multi-detect: load tenants from yaml file 
Load tenants and mappings from the suricata.yaml when available.
 11 years ago
Victor Julien 216638c342 multi-detect: implement unregister-tenant-handler 
Remove a tenant handler from the list and apply it.
 11 years ago
Victor Julien 93f856a1b3 detect: don't error out on no de_ctx 
This can happen on a multi-detect setup with no registered
engines yet.
 11 years ago
Victor Julien b6f290fac7 multi-detect: set selector from yaml 
Yaml setting is: multi-detect.selector

Implement 'vlan' and 'direct'.
 11 years ago
Victor Julien c72b7f83b8 multi-detect: error on start if no selector registered 
Force user to select the method at startup.
 11 years ago
Victor Julien 1127ad66b4 multi-detect: register counters on 'master' det_ctx 
Otherwise counters are only registered after the stats api is
already fixed.
 11 years ago
Victor Julien 7c581c0ffc multi-detect: allow start up with 0 tenants 11 years ago
Victor Julien 6d92e8d220 unix-socket: implement register-tenant-handler 
Register tenant handlers/selectors based on what the unix command
"register-tenant-handler" tells.

Check traffic id before adding it. No duplicated registrations for
a traffic id are allowed.
 11 years ago
Victor Julien 1893c5edb1 multi-detect: initial selectors for tenants 
The Detection Thread has the TenantGetId pointer which allows it
to select a tenant id based on the packet.
 11 years ago
Victor Julien 0ff6d3dcfd detect: select detect engine at Detect entry 
Limited to Pcap only currently.
 11 years ago
Victor Julien bc7e9be5c6 unix-socket: allow tenant id with pcap-file 
Register the tenant id that the pcap-file optionally got.
 11 years ago
Victor Julien bea66156b9 pcap-file: set tenant-id if available 
Set the id to each packet's 'pcap_v' structure.
 11 years ago
Victor Julien 98d265f40b detect: use multi tenant thread init if MT enabled 11 years ago
Victor Julien b653479815 detect: make multi tenancy a global switch 
At start up we will set this flag based on "multi-detect.enabled".
 11 years ago
Victor Julien 3f12bdd4f8 tenants: apply added/removed tenant 
Apply to the engine.
 11 years ago
Victor Julien def2b58725 detect: initial MT lookup logic 
In the DetectEngineThreadCtx, store another DetectEngineThreadCtx per
tenant.

Currently it's just a simple array indexed by the tenant id.
 11 years ago
Victor Julien 147a6d2bfd multi-detect: (un)register-tenant unix socket commands 
Make available to live mode and unix socket mode.

register-tenant:
    Loads a new YAML, does basic validation.
    Loads a new detection engine
    Loads rules
    Add new de_ctx to master store and stores tenant id in the de_ctx so
        we can look it up by tenant id later.

unregister-tenant:
    Gets the de_ctx, moves it to the freelist
    Removes config

Introduce DetectEngineGetByTenantId, which gets a reference to the
detect engine by tenant id.
 11 years ago
Victor Julien 3aa58f25ad eve alert: fix stream payload printing 11 years ago
Victor Julien 2ef0ebb24b detect: fix pass transaction handling 
If a flow was 'pass'd, it means that no packet of it will flow be handled
by the detection engine. A side effect of this was that the per flow
inspect_id would never be moved forward. This in turn lead to a situation
where transactions wouldn't be freed.

This patch addresses this case by incrementing the inspect_id anyway for
the pass case.
 11 years ago
Victor Julien 06ee2bc87e detect: set flow noinspect on pass in applayer/stream 
If a pass rule matches in the reassembled stream and/or in the
app-layer state, it means the rest of the flow should not be
inspected.
 11 years ago
Victor Julien bbc9874b81 http: destroy htp_tx_t even if incomplete 11 years ago
Victor Julien 814f0b4094 detect: make http prefilter use disrupt flags 11 years ago
Victor Julien b592f98727 detect: optimize http prefilter handing 11 years ago
Victor Julien 6946e0be55 detect: pass flags to inspect_id update logic 11 years ago
Victor Julien c087708fa9 output-tx: use disrupt flags 11 years ago
Victor Julien 00ef789ffc app-layer: pass full flags around in tx handling 11 years ago
Victor Julien ea571add73 app-layer: disruption flags 
Stream GAPs and stream reassembly depth are tracked per direction. In
many cases they will happen in one direction, but not in the other.

Example:
HTTP requests a generally smaller than responses. So on the response
side we may hit the depth limit, but not on the request side.

The asynchronious 'disruption' has a side effect in the transaction
engine. The 'progress' tracking would never mark such transactions
as complete, and thus some inspection and logging wouldn't happen
until the very last moment: when EOF's are passed around.

Especially in proxy environments with _very_ many transactions in a
single TCP connection, this could lead to serious resource issues. The
EOF handling would suddenly have to handle thousands or more
transactions. These transactions would have been stored for a long time.

This patch introduces the concept of disruption flags. Flags passed to
the tx progress logic that are and indication of disruptions in the
traffic or the traffic handling. The idea is that the progress is
marked as complete on disruption, even if a tx is not complete. This
allows the detection and logging engines to process the tx after which
it can be cleaned up.
 11 years ago
Victor Julien 8125e04b39 detect: clean up flag usage 11 years ago
Victor Julien 6e2c90a83b detect: constify some DetectMpmPrefilter args 11 years ago
Victor Julien c419f33f44 dns: fix state progress handling 11 years ago
Victor Julien b0f5f7ee97 app-layer: fix args to state progress calls 11 years ago
Victor Julien 62fa9f09d4 Sync alversion/appversion types 
The app layer state 'version' field is incremented with each update
to the state. It is used by the detection engine to see if the current
version of the state has already been inspected. Since app layer and
detect always run closely together there is no need for a big number
here. The detect code really only checks for equal/not-equal, so wrap
arounds are not an issue.
 11 years ago
Victor Julien 11d3f5f67a http: harden tx inspection code 11 years ago
Victor Julien be2849044b flow/stream: xfer noinspect flags to pseudo pkts 
Set noinspection flags for payloads and packets on flow and stream
pseudo packets. Without these, the pseudo packets could trigger
inspection even though this was disabled for a flow.
 11 years ago
Victor Julien a0899cbe85 detect: optimize Signature layout 11 years ago
Victor Julien baadcab1b8 detect: default to u32 for SigIntId 11 years ago
Eric Leblond 1cd97713c2 file-json: add file_id to message 
This will allow to get the filename and by consequence the file
after a parsing of the EVEV log file.
 11 years ago
Eric Leblond 8fde842f97 af-packet: implement rollover option 
This patch implements the rollover option in af_packet capture.
This should heavily minimize the packet drops as well as the
maximum bandwidth treated for a single flow.

The option has been deactivated by default but it is activated in
the af_packet default section. This ensure there is no change for
old users using an existing YAML. And new users will benefit from
the change.

This option is available since Linux 3.10. An analysis of af_packet
kernel code shows that setting the flag in all cases should not
cause any trouble for older kernel.
 11 years ago
Eric Leblond dc306f3bad af-packet: implement new load balancing modes 
This patch implements the fanout load balancing modes available
in kernel 4.0. The more interesting is cluster_qm that does the
load balancing based on the RSS queues. So if the network card
is doing a flow based load balancing then a given socket will
receive all packets of a flow indepently of the CPU affinity.
 11 years ago
Eric Leblond 851fcef962 af-packet: sync header with latest features 
Sync the replacement define with the latest Linux code.
This patch also updates the detection part in configure.ac
to do a declaration of all fields if the newest features are
not present.
 11 years ago
Aleksey Katargin caa2438b98 netmap: support SW rings 
Netmap uses SW rings to send and receive packets from OS.
 11 years ago
Aleksey Katargin 823167bde2 netmap: strict check for zero copy mode 
Netmap does not guarantees that mmap'ed regions for different interfaces would be the same.
 11 years ago
Aleksey Katargin 5104b02f32 netmap: fixed autofp mode. 
Previous implementation does not work with this mode.
 11 years ago
Aleksey Katargin 491fca468d netmap: support non-equal count of Rx and Tx rings on interface. 
Netmap does not guarantees that NIC will have equal number of transmit and receive rings.
 11 years ago
Schnaffon 6484299701 Use unlikely for error treatment in alert-prelude.c 
When handling error cases on creation of a new idmef field, we are in an unlikely case. This patch adds the unlikely() expression to indicate this to gcc.
 11 years ago
Victor Julien 334e8656bf introduce fatal error macro's 
Add 'FatalError' and 'FatalErrorConditonal' that will take the same
args as SCLogError.

FatalError logs the error using SCLogError and then exits with return
code EXIT_FAILURE.

FatalErrorOnInit does the same only during init and with
--init-errors-fatal enabled, otherwise it just calls SCLogWarning. So
then the macro returns to the caller.

Implement this for output setup.
 11 years ago
Victor Julien 4dd53c8f03 lua: dns support 11 years ago
Victor Julien 433e511b63 dns: generic inspect engines for DNS 11 years ago
Victor Julien 110e23964e detect: add AppLayerTxMatch call 11 years ago
Victor Julien e6129f7b47 dns: generic request/response detect lists 11 years ago
Victor Julien 2c8e8c2516 dns: rename type so it's purpose is more clear 11 years ago
Victor Julien a1e50b3138 lua: dns extensions 
Add DNS lua calls for getting queries, answers, authorities. Also
rcode and recusion desired flags.
 11 years ago
Victor Julien c46d472921 lua: initial DNS logging support 11 years ago
Eric Leblond 159a6d1cb4 tls-store: avoid log flooding 
In case we can't write in the certs directory, this is possible
we flood the log for each TLS session or even worse each TLS
packet.  So this patch puts a limit in the number of logged
messages related to file creation.
 11 years ago
Eric Leblond cbf5d88447 filestore: use SCFree instead of free 11 years ago
Eric Leblond b77cd22b98 tls-store: backward compatibility 
This patch implements backward compatibility in suricata.yaml
file. In case the new 'tls-store' output is not present in the
YAML we have to use the value defined in 'tls-log'.
 11 years ago
Eric Leblond 4db0a35f25 tls-store: now a separate module 
An design error was made when doing the TLS storage module which
has been made dependant of the TLS logging. At the time there was
only one TLS logging module but there is now two different ones.

By putting the TLS store module in a separate module, we can now
use EVE output and TLS store at the same time.
 11 years ago
Jeff Barber 893fc9660d Support for reconnecting unix domain socket log files 
Issue #1423
 11 years ago
Jason Ish b512580bbe logging: integrate rotation into SCConfLogOpenGeneric. 
Addresses issue 1492, and will make it harder to omit
rotation on new outputs.
 11 years ago
Jason Ish 14981cb2a8 rule vars: strip leading white space before looking up var. 11 years ago
sfd e58cfb6a05 Fix compile bad dereferences 
The src/source-erf-dag.c code was not compiling. It looks like some stats counters were added but not tested as the dereferences are incorrect.
 11 years ago
Zopieux cd038419fd stream_size operator comparison (fix issue #1488) 
`DetectStreamSizeParse` was first checking if mode[0] is '<', which is true for both '<' and '<=', thus '<=' (and resp. '>=') is never matched. This patch does the `strcmp` to '<=' (resp. '>=') within the if block of '<' (resp. '>') to fix #1488.
 11 years ago
Victor Julien 45fc619f79 logging: json output 
Make JSON output optional.

Cleanups.
 11 years ago
Victor Julien 126ecb3ebf logging: fix per output log formats 11 years ago
Victor Julien b51c4e608f logging: optional colors output 
Construct message per output method.
 11 years ago
Victor Julien b13de5bf08 logging: change newline handling 11 years ago
Victor Julien 1927b3000c output: cleanup 11 years ago
Victor Julien b30bdc21b5 logging: cleanup output API 
Make SCLogMessage master of the logging. Reduces complexity
of the SCLog macro's.
 11 years ago
Victor Julien d6fc6e874f log: reorganize SCLogOPIfaceCtx to make it more efficient 11 years ago
Victor Julien c2f4031a8c detect: fix settings override for reloads 11 years ago
Alexander Gozman cd9cc2559e Issue 1491: fix capabilities for pf_ring mode when running under non-root account 11 years ago
Alexander Gozman d36eba4e5e Fix issues #1493 and #1494 11 years ago
Giuseppe Longo d592d57039 file_data: check for signature alproto and flow 
Currently the following rule can't be loaded:
alert tcp any any -> any 25 (msg:"SMTP file_data test"; flow:to_server,established; file_data; content:"abc";sid:1;)
and produces the error output:
"Can't use file_data with flow:to_server or from_client with http or smtp."

This checks if the alproto is not http in a signature,
so permits to use flow keyword also.

Issue reported by rmkml.
 11 years ago
Victor Julien e583de0582 Minor unittest cleanups 11 years ago
Victor Julien f4f53924bb app-layer: fix coverity warnings 11 years ago
Victor Julien 6c792cb4cc erspan: respect vlan.use-for-tracking setting 11 years ago
Victor Julien b8211e8c04 htp: hide BUG_ON's behind DEBUG_VALIDATION 11 years ago
Victor Julien bd73553027 smtp json: fix potential crash on malloc failure 
** CID 1298888:    (FORWARD_NULL)
/src/output-json-email-common.c: 117 in JsonEmailLogJson()
/src/output-json-email-common.c: 140 in JsonEmailLogJson()
 11 years ago
Victor Julien 1ed8d7b538 detect analizer: fix minor coverity warning 
** CID 1298889:  Integer handling issues  (NEGATIVE_RETURNS)
/src/detect-engine-analyzer.c: 102 in EngineAnalysisFP()
 11 years ago
Victor Julien dd2afd51f7 file_data smtp: fix minor coverity warning 
CID 1298891:  Null pointer dereferences  (REVERSE_INULL)
Null-checking "curr_file" suggests that it may be null, but it has already been dereferenced on all paths leading to the check.
 11 years ago
Victor Julien bcff35fd9f smb: fix coverity warning 
** CID 1298892:  Incorrect expression  (UNUSED_VALUE)
/src/app-layer-smb.c: 1486 in SMBProbingParser()
 11 years ago
Victor Julien aa095864d3 pfring runmode: remove set that is never read 11 years ago
Victor Julien 21db5ee691 counters: reduce global usage 11 years ago
Victor Julien a5168d5977 dce_opnum: improve memory handling on parsing error 11 years ago
Victor Julien b3b7625be5 htp: fix test 11 years ago
Victor Julien c0807c3df5 fast log: clean up tests 11 years ago
Victor Julien 29fbcce50d detect hsbd: simplify resize logic 11 years ago
Victor Julien 4e7cb7b863 app-layer: update all protocols to accept NULL+EOF 
Update all non-HTTP protocol parsers to accept a NULL+EOF input.
 11 years ago
Victor Julien cf9ff6adbd app-layer: improve EOF handling 
On receiving TCP end of stream packets (e.g. RST, but also sometimes FIN
packets), in some cases the AppLayer parser would not be notified. This
could happen in IDS mode, but would especially be an issue in IPS mode.

This patch changes the logic of the AppLayer API to handle this. When no
new data is available, and the stream ends, the AppLayer API now gets
called with a NULL/0 input, but with the EOF flag set.

This allows the AppLayer parser to call it's final routines still in the
context of a real packet.
 11 years ago
Victor Julien df79c1019f print: make PrintRawDataFp take a const arg 11 years ago
Victor Julien 7451d33396 stream: update StreamMsg to don't have fixed size 
StreamMsg would have a fixed size buffer. This patch replaces the buffer
by a dynamically allocated buffer.

Preparation of allowing bigger and customizable buffer sizes.
 11 years ago
Victor Julien 3ffa01d150 stream: remove STREAMTCP_STREAM_FLAG_CLOSE_INITIATED logic 11 years ago
Giuseppe Longo 32563d51d1 http: rework UT 
This reworks UT based on
commit 3203555708
 11 years ago
Victor Julien 573d082219 http: memcap HTTP server inspect body code 11 years ago
Victor Julien 3163a57577 smtp file_data: fix wrong free 11 years ago
Victor Julien 9f52bdd1e5 flow timeout: prevent dead locks 
The flow timeout mechanism called both from the flow manager at run time
and at shutdown creates pseudo packets. For this it has it's own packet
pool, which can be depleted if the timeout logic is faster than the packet
processing threads. In this case the flow timeout would enter a wait loop.
The problem however, is that this wait loop would happen while keeping a
flow locked. This could lead to a race condition when the packet thread(s)
are waiting for the lock that the flow manager has.

This patch introduces a new packet pool call 'PacketPoolWaitForN', meant
to make sure that the thread's packet pool has at least N available
packets. The flow timeout paths use this to make sure enough packets are
available *before* grabbing the flow lock. If there aren't enough packets
available yet, the wait happens before the lock as well.

This still means the wait can happen while the flow hash row is locked, so
we do make sure some more packets are available when entering that. But
perhaps in the future we need a more precise logic there as well.
 11 years ago