Commit Graph

4783 Commits (suricata-2.0.8)

Author SHA1 Message Date
Eric Leblond eea760de3c decode: fix typo in comment 12 years ago
Anoop Saldanha 167597cfb0 Update ftp parser protocol detection to use lowercase patterns. 12 years ago
Anoop Saldanha 36bd444406 Introduce new API to allow case insensitive protocol detection patterns. 12 years ago
Anoop Saldanha 90827ea990 Remove the obsolete DetectFtpBounceMatch() function. 12 years ago
Ken Steele 9fce4da84d Spell fixes in threads-arch-tile.h 12 years ago
Anoop Saldanha e8cd15c823 Support for feature #983.
Provide support for icmvp4 and icmpv6 as well.  You can now use

alert icmpv4 and
alert icmpv6 as well, apart from the existing

alert icmp, which created a rule that applied to both icmpv4 and icmpv6.
12 years ago
Nelson Escobar cf9f1e3191 Build cuda kernel for capability 3.5 devices. 12 years ago
Victor Julien 3f8b9dde04 Dead code removal 12 years ago
Victor Julien 84af1ee277 storage: fix and small optimization 12 years ago
Victor Julien 77ae8b8878 flow: set correct family in FLOW_COPY_IPV6_ADDR_TO_PACKET 12 years ago
Victor Julien 2a4f821284 Fix 2 unittests 12 years ago
Victor Julien 8516000208 Minor code cleanup/fixes to fast pattern unittests
cppcheck:
[detect-fast-pattern.c:1183] -> [detect-fast-pattern.c:1183]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1217] -> [detect-fast-pattern.c:1217]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1449] -> [detect-fast-pattern.c:1449]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1479] -> [detect-fast-pattern.c:1479]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1509] -> [detect-fast-pattern.c:1509]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1539] -> [detect-fast-pattern.c:1539]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1570] -> [detect-fast-pattern.c:1570]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1686] -> [detect-fast-pattern.c:1686]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1716] -> [detect-fast-pattern.c:1716]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1746] -> [detect-fast-pattern.c:1746]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1776] -> [detect-fast-pattern.c:1776]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1806] -> [detect-fast-pattern.c:1806]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1836] -> [detect-fast-pattern.c:1836]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1866] -> [detect-fast-pattern.c:1866]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1896] -> [detect-fast-pattern.c:1896]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1926] -> [detect-fast-pattern.c:1926]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:2022] -> [detect-fast-pattern.c:2022]: (style) Same expression on both sides of '&'.
12 years ago
Victor Julien af311aee4e Minor fix for detection engine setup error check
cppcheck said:
[detect-engine-mpm.c:2075] -> [detect-engine-mpm.c:2075]: (style) Same expression on both sides of '||'.
12 years ago
Victor Julien 974e86e450 Minor pppoe cleanup
cppcheck said:
[decode-pppoe.c:58] -> [decode-pppoe.c:60]: (performance, inconclusive) Variable 'pppoedh' is reassigned a value before the old one has been used if variable is no semaphore variable.
12 years ago
Victor Julien 209946b07c Fix broken check in stream.max-synack-queued parsing (coverity 1038103) 12 years ago
Victor Julien bec59f426e Fix sanity check in AppInspectionEngine registration code 12 years ago
Jason Ish 2953b3f640 Feature #901 - VLAN defrag support.
Take VLAN IDs into account when re-assembling fragments.

Prevents fragments that would otherwise match, but on different
VLANs from being reassembled with each other.
12 years ago
Ken Steele 2d3dc23026 Correct indentation and wording of comments. 12 years ago
Ken Steele a63b87df9e Clean up function syntax
Remove space before ( in function names. Put { on new line.
Make tests static.
12 years ago
Anoop Saldanha 619414c59e Add a /* fall through */ comment for all switch case fall throughs.
This should server as a message to coverity that the fall through is
intentional.
12 years ago
Victor Julien b6efaeb0c0 storage: fix freeing storage
Fix freeing storage. Also add workaround for unittests that don't
(fully) setup storage.

Bug #991.
12 years ago
Victor Julien 37669bfdd2 threshold: register threshold host storage. Related to bug #991 12 years ago
Victor Julien 1b11165864 Reset both sides of the de_state on rule reload. Bug #998. 12 years ago
Victor Julien 74d8d95f83 Don't initialize threshold before rules on delayed detect. Bug #999. 12 years ago
Victor Julien 64203be3ba iprep: fix reputation loading and reloading
When an IP is listed in multiple categories, each new "load" would clear the
previous loads for that IP.

Bug #976
12 years ago
Victor Julien c583c9e205 tag: fix session seconds tracking
Fix bug #995. Tag time setting was initialized using "usec" field
instead of "sec" field. This led to immediate timing out of tag.

Added proper matching unittests for all tagging types.

Bug #995.
12 years ago
Victor Julien 1822a897ff tag: add some debug statements 12 years ago
Victor Julien a26243a23c Clean up rule reload logging 12 years ago
Anoop Saldanha b24fb72247 fix for bug #987.
We don't support jabber protocol detection atm.  Disable the code check
inside suricata to check if jabber protocol detection is enabled in the
yaml file.

Also updated an error log message for app layer.
12 years ago
Anoop Saldanha 83a72d50dd API renaming/beautification. 12 years ago
Anoop Saldanha 1ea5d27508 Fix for bug #989.
In case of recursive call to protocol detection from within protocol
detection, and the recursively invoked stream still hasn't been ack'ed
yet, protocol detection doesn't take place.  In such cases we will end up
still calling the app layer with the wrong direction data.  Introduce a
check to not call app layer with wrong direction data.

When sockets are re-used reset all relevant vars correctly.

This commit fixes a bug where we were not reseting app proto detection
vars.

While fixing #989, we discovered some other bugs which have also been
fixed, or rather some features which are now updated.  One of the feature
update being if we recieve wrong direction data first, we don't reset the
protocol values for the flow.  We let the flow retain the detected
values.

Unittests have been modified to accomodate the above change.
12 years ago
Anoop Saldanha 836bad85a4 Reset app layer processed flag for segments that have been sent for proto
detection, but we failed to figure out the proto.

Updated a unittest to reflect the above change.
12 years ago
Anoop Saldanha 87edd2ade9 Inside PP parser, we were using the return value from DetectPortParse as
the ip_proto value,  which is wrong.  We have fixed this now.
12 years ago
Anoop Saldanha 73be9d3ef7 Update ssl parser protocol detection pattern strings. 12 years ago
Victor Julien 1d18155a16 XFF: use per alert tx id
Use the tx id stored for each alert to find the correct XFF address
to add to the extra-data field.

In overwrite mode we still only grab the first available XFF addr,
as this address is set in the header preceeding the individual alerts.

Issue #904.
12 years ago
Victor Julien e7df53b136 Display TX id in alert debuglog. 12 years ago
Victor Julien edeeb7ed44 Store TX id with alerts
When generating an alert and storing it in the packet, store the tx_id
as well. This way the output modules can log the tx_id and access the
proper tx for logging.

Issue #904.
12 years ago
Victor Julien 51c2e1eaf6 htp: for apache and apache_2_2 personalities, that are no longer supported by libhtp, fall back to apache_2 with a warning. 12 years ago
Victor Julien 958938bf01 Bug 640: add more tests to validate that issue is fixed 12 years ago
Eric Leblond 2be194d03f suricata: add -v[v] option to increase verbosity
This patch adds a -v option to suricata. It increases the log level
defined in the YAML.
12 years ago
Eric Leblond 4a4600539d suricata: info message after log init
This patch moves version display after log init so we can have an
homogeneous display.
12 years ago
Eric Leblond fdc1757e34 suricata: reorder start
Initalizing output just after configuration file parsing allow to
log almost all messages accordingly to configuration.
12 years ago
Eric Leblond 7bcacc712a log: change default log level to notice
This patch updates the log level of meaningful start messages to
notice. It also sets the default log level to notice.
12 years ago
Victor Julien c1190545cf Revert change in queue handler wait logic. Bug #988. 12 years ago
Victor Julien 8d6bca72f7 Improve 'host-mode' info message 12 years ago
Victor Julien 57abba2e64 Coverity 1100842: add missing return statement 12 years ago
Victor Julien afaa10b37d Coverity 1100843: remove unnecessary check 12 years ago
Victor Julien cb15000387 http: add new events for invalid host header and host part of uri 12 years ago
Victor Julien 43b39d333f http: fix some decoder events
Some events we retrieved from error messages are flag now, so check
those. Not all can be converted though. These are no longer set:

HTTP_DECODER_EVENT_INVALID_TRANSFER_ENCODING_VALUE_IN_RESPONSE
HTTP_DECODER_EVENT_INVALID_AUTHORITY_PORT

Part of Bug #982.
12 years ago
Victor Julien 636791751e http: fix field too long events 12 years ago
Victor Julien 5d10bafdba http: don't call HTPHandleWarning before HTPHandleError as the latter handles warnings and errors. 12 years ago
Victor Julien 129b6a65ca http: add test for HTTP_DECODER_EVENT_UNKNOWN_ERROR event as a result of a too long request 12 years ago
Eric Leblond 2c50e41153 reject: try to fail more gracefully
In the case of reject both, a failure in sending one way do not lead to
abort the reset procedure.
12 years ago
Eric Leblond 10b05a6361 reject: clean respond-reject code. 12 years ago
Eric Leblond 6f1cf9728e reject: delete debug line 12 years ago
Eric Leblond f05efeb46f Add reject for IPv6
With this patch reject is now available in IPv6.
12 years ago
Eric Leblond 5f224f87d1 reject: update computation of seq and ack
We have follow TCP RFC (http://tools.ietf.org/html/rfc793#section-3.4).
There is two cases depending on wether the original packet contains a
ACK.
If packet has no ACK, the RST seq number is 0 and the ACK is built the
standard way.
If packet has a ACK, the seq of the RST packet is equal to the ACK of
incoming packet and the ACK is build using packet sequence number and
size of the data.

Regarding standard Ack number, it is computed using seq number of captured
packet added to packet length. Finally 1 is added so we respect the
RFC:
    If the ACK control bit is set this field contains the value of the
    next sequence number the sender of the segment is expecting to
    receive.  Once a connection is established this is always sent.

With this patch we have some correct results. With the following rule:
    reject ssh any any -> 192.168.56.3 any (msg:"no SSH  way"; sid:3; rev:1;)
ssh connection to 192.168.56.3 is correctly resetted on client side.

But this is not perfect. If we have the following rule:
    reject tcp any any -> 192.168.56.3 22 (msg:"no way"; sid:2; rev:1;)
then the connection is not resetted on a standard ethernet network. But
if we introduce 20ms delay on packets, then it is correctly resetted.
This is explained when looking at the network trace. The reset is sent
as answer to the SYN packet and it is emitted after the SYN ACK from
server because the exchange is really fast. So this is discarded by the
client OS which has already seen a ACK for the same sequence number.

This should fix #895.
12 years ago
Eric Leblond 4e15cf2245 reject: fix typo 12 years ago
Eric Leblond efc12b24ae reject: use host-mode to set interface
This patch update reject code to send the packet on the interface
it comes from when 'host-mode' is set to 'sniffer-only'. When
'host-mode' is set to 'router', the reject packet is sent via
the routing interface.

This should fix #957.
12 years ago
Eric Leblond 9bbd2a103d reject: reindent and code cleaning
Reindent file and use some switch instead of if else if.
12 years ago
Eric Leblond 6cf7da30e2 Introduce host-mode.
This variable can be used to indicate to suricata that the host
running is running as a router or is in sniffing only mode.
This will used at least to determine which interfaces are used to
send reject message.
12 years ago
Victor Julien d8cb821875 locks: clean up locks declarations
Split threads.h into several files, where each of these files defines
all lock types and macro's.

threads.h defines the normal case
threads-debug.h defines the debug variants
threads-profile.h defines the lock profiling variants

Finally, threads-arch-tile.h moves the Tilera specifics out
12 years ago
Anoop Saldanha c5cd3562d0 Stateful detection inspection continuation API call should update per
signature's Sigmatch entry as well.
12 years ago
Victor Julien 7f0cc97f5b Thresholding: move parsing code into separate func 12 years ago
Victor Julien 8ce38ac8fe Split Thresholds and Suppression
Thresholds and suppression can be handled independently. Suppression
only suppresses output, and is not related to Threshold state tracking.

This simplifies mixing suppression and thresholding rules.

Part of the Bug #425 effort.
12 years ago
Ken Steele 592d48aab7 Use Spin locks on Tile
On Tile, replace pthread_mutex_locks with queued spin locks (ticket
locks) for dataplane processing code. This is safe when running on
dataplane cores with one thread per core. The condition variables are
no-ops when the thread is spinning anyway.

For control plane threads, unix-manager, stats-logs, thread startup,
use pthread_mutex_locks. For these locks replaced SCMutex with SCCtrlMutex
and SCCond with SCCtrlCond.
12 years ago
Victor Julien 2f4e11b1ca Fix compiler warning
app-layer-parser.c: In function ‘AppLayerPPTestData’:
app-layer-parser.c:2525:9: error: variable ‘dir’ set but not used [-Werror=unused-but-set-variable]
     int dir = 0;
         ^
12 years ago
Ken Steele 85a51638c9 Improve Signature sorting speed
Changed the signature sorting code to use a a single merge sort instead
of the multiple pass sorting that was being used. This reduces startup
time on Tile by a factor of 3.

Also replace the user array of pointers to ints with a simpler array of
ints.
12 years ago
Victor Julien 5c08b2296f DNS: copy only the length of the hardcoded string, not the length of the destination buffer. 12 years ago
Anoop Saldanha 57ed5dfd32 Fix return value from DetectProtoParse() which is used by probing
parser.
12 years ago
Anoop Saldanha ac65784cbc Fix coverity scan defect #1099714.
Sending back uninitialized variable in DetectParseProto().
12 years ago
Anoop Saldanha e383cc27cd Fix a leak in probing parsers. We were freeing just the head of the list,
instead of all the members.
12 years ago
Anoop Saldanha 980934d670 Fix a leak in app layer parser proto code. Free the proto signatures
allocated internally for PM parser.
12 years ago
Anoop Saldanha fc82614025 Fix mem leak in b2g. 12 years ago
Anoop Saldanha 06db1e4cb8 Remove unused vars alp_content_module_handle and proto_map from
struct AlpProtoDetectCtx.
12 years ago
Anoop Saldanha 558f5705eb Remove the unused flow flags - FLOW_TS_PM_PP_ALPROTO_DETECT_DONE and
FLOW_TC_PM_PP_ALPROTO_DETECT_DONE.
12 years ago
Anoop Saldanha 36220b689b Reset some flow flags when port numbers are re-used and we re-use the
flow as a part of a new session.
12 years ago
Anoop Saldanha af1df7a89d Remove the smtp parser restriction that it accepts data only in to client
direction first.
12 years ago
Anoop Saldanha 3ec411486e Fix compilation failure when we don't enable unittests. Got to #ifdef
ALPROTO_TEST.
12 years ago
Anoop Saldanha d76a5bedbc Update stream inline to use the improved app proto detection. 12 years ago
Anoop Saldanha 96d1ba9106 Cosmetic changes to app parser struct.
Removed a flag parameter introuced earlier to indicate the data
that is first acceptable by the parser.  We now use a differently
named parameter to carry out the same activity.
12 years ago
Anoop Saldanha 2cb5bdd3fa Cosmetic changes to code. Introduce human readabel flag values for some constants. Here the parameter in question is "data_first_seen_dir" for session context. 12 years ago
Anoop Saldanha e42905f3b9 indentation fix. 12 years ago
Anoop Saldanha 6bef5fda06 If we have proto mismatch from 2 directions, use one of the protos, instead of erroring out and not sending the data further to the parser.
The logic we use currently is if we have already sent some data to
a parser before we figure out we have a proto mismatch, we use the
proto from the first direction from which we have already sent the
data to the parser, else we stick to the the to client direction.
12 years ago
Anoop Saldanha 976a86def4 Introduce convenience macro to set Stream app proto completion flag. 12 years ago
Anoop Saldanha 16144fe38a Rename function pointer var to use the FuncPtr typing convention. Resupply "dns" as the alproto name for ALPROTO_DNS. 12 years ago
Anoop Saldanha 8ae92c7a5e Add unittest to test for http ambiguous host header.
Previously we would not check the port part of the host from the uri
hostname, while we did use the port part from the host header, leading
to FPs.
12 years ago
Anoop Saldanha d0c5f51293 Update rule engine relationship with regard to setting ip protocol between specifying protocol after action, ip_proto and app-layer-protocol.
Now we can specify alproto, ip_proto combinations this way

alert dns (ip_proto:[tcp/udp];)
alert ip (app-layer-protocol:dns;)
alert ip (app-layer-protocol:dns; ip_proto:tcp;)
alert tcp (app-layer-protocol:dns:)

so on.  Neater than using dnstcp/dnsudp.

This is related to feature #424.
12 years ago
Anoop Saldanha 6eb8f66f0a alert ipv4 and alert ipv6 specified proto rules should be treated and PROTO_ANY just like how we treat alert ip rules. 12 years ago
Anoop Saldanha f592c481dc Introduce a separate inspection engine for app events. 12 years ago
Anoop Saldanha 9e4eec200f Update htp event handler to both warning and error events regardless of any conditions. 12 years ago
Anoop Saldanha b1dffdfbe0 Add app layer protocol packet event detection support. 12 years ago
Anoop Saldanha 5e2d9dbdc3 Add and use EventGetInfo for getting info on an event.
Also update existing parsers and app-layer-event Setup to use this.
12 years ago
Anoop Saldanha 60a2b157b2 Fix duplicate packet decoder events. Add event entries that were missing as well. 12 years ago
Anoop Saldanha 1077acecd7 validate dns sigs that are reported as plain dns and not dnsudp or dnstcp. 12 years ago
Anoop Saldanha 6cb0014287 Move app event module registration as a part of app layer proto table. 12 years ago
Anoop Saldanha 64b0939b4a code cleanup. 12 years ago
Anoop Saldanha 0d7159b525 App layer protocol detection updated and improved. We now use
confirmation from both directions and set events if there's a mismatch
between the 2 directions.

FPs from corrupt flows have disappeared with this.
12 years ago
Anoop Saldanha 22c05da3cd Replace ssn appproto_detection_completed flag with individual stream ones. 12 years ago
Anoop Saldanha c044541b1c Provide convenience macros for setting flow flags on protocol matching by
PM and PP phase.

Replace the areas of the code that would otherwise rely on setting/reading
these flags with these macros.

Other minor tweaks to some api calls.
12 years ago
Anoop Saldanha 00f546e739 update pmp to return whole set of matches, rather than a single match. 12 years ago
Anoop Saldanha 4f7339c423 code cleanup. 12 years ago
Anoop Saldanha 8e8bc49063 Introduce detection parser function pointer. 12 years ago
Anoop Saldanha 94e40907e2 feature #727 - Add support for app-layer-protocol:<protocol> keyword 12 years ago
Anoop Saldanha 6f8cfd999f Allow detection ports for alproto to be specified via the conf file.
To understand the option have a look at the option

app-layer.protocols.tls.detection-ports
12 years ago
Anoop Saldanha ddde572fba Introduce new options into the conf file to enable/disable -
1. Proto detection
2. Parsers

For app layer protocols.

libhtp has now been moved to the section under app-layer.protocols.http,
but we still provide backward compatibility with older conf files.
12 years ago
Anoop Saldanha d9686fae57 Now supports accepting port addresses as strings, like the ones accepted in our rules. As a consequence we now accept port range, and other such combination. Support PP for ports based on ipproto as well. 12 years ago
Victor Julien 48b5513ed9 Properly clean up decoder event rules
Addresses:
~~Dr.M~~ Error #3: LEAK 120 direct bytes 0x08a26ac8-0x08a26b40 + 1871 indirect bytes
~~Dr.M~~ # 0 replace_malloc                               [/work/drmemory_package/common/alloc_replace.c:2292]
~~Dr.M~~ # 1 SigGroupHeadAlloc                            [/home/victor/dev/oisf/src/detect-engine-siggroup.c:144]
~~Dr.M~~ # 2 SigGroupHeadAppendSig                        [/home/victor/dev/oisf/src/detect-engine-siggroup.c:1014]
~~Dr.M~~ # 3 DetectEngineAddDecoderEventSig               [/home/victor/dev/oisf/src/detect.c:3026]
~~Dr.M~~ # 4 SigAddressPrepareStage2                      [/home/victor/dev/oisf/src/detect.c:3075]
~~Dr.M~~ # 5 SigGroupBuild                                [/home/victor/dev/oisf/src/detect.c:4311]
~~Dr.M~~ # 6 SigLoadSignatures                            [/home/victor/dev/oisf/src/detect.c:464]
~~Dr.M~~ # 7 LoadSignatures                               [/home/victor/dev/oisf/src/suricata.c:1706]
~~Dr.M~~ # 8 main                                         [/home/victor/dev/oisf/src/suricata.c:1994]
12 years ago
Victor Julien c43e078db8 ipproto: improve cleanup
To address:
~~Dr.M~~ Error #2: LEAK 16 direct bytes 0x08399688-0x08399698 + 2 indirect bytes
~~Dr.M~~ # 0 replace_malloc                      [/work/drmemory_package/common/alloc_replace.c:2292]
~~Dr.M~~ # 1 SigMatchAlloc                       [/home/victor/dev/oisf/src/detect-parse.c:201]
~~Dr.M~~ # 2 DetectIPProtoSetup                  [/home/victor/dev/oisf/src/detect-ipproto.c:523]
~~Dr.M~~ # 3 SigParseOptions                     [/home/victor/dev/oisf/src/detect-parse.c:510]
~~Dr.M~~ # 4 SigParseOptions                     [/home/victor/dev/oisf/src/detect-parse.c:523]
~~Dr.M~~ # 5 SigParse                            [/home/victor/dev/oisf/src/detect-parse.c:881]
~~Dr.M~~ # 6 SigInitHelper                       [/home/victor/dev/oisf/src/detect-parse.c:1309]
~~Dr.M~~ # 7 SigInit                             [/home/victor/dev/oisf/src/detect-parse.c:1456]
~~Dr.M~~ # 8 DetectEngineAppendSig               [/home/victor/dev/oisf/src/detect-parse.c:1728]
~~Dr.M~~ # 9 DetectLoadSigFile                   [/home/victor/dev/oisf/src/detect.c:334]
~~Dr.M~~ #10 SigLoadSignatures                   [/home/victor/dev/oisf/src/detect.c:422]
~~Dr.M~~ #11 LoadSignatures                      [/home/victor/dev/oisf/src/suricata.c:1706]
12 years ago
Victor Julien 1006d905d0 Improve memory cleanup for decoder-events
To address:

~~Dr.M~~ Error #1: LEAK 1 direct bytes 0x0892c108-0x0892c109 + 0 indirect bytes
~~Dr.M~~ # 0 replace_malloc                        [/work/drmemory_package/common/alloc_replace.c:2292]
~~Dr.M~~ # 1 DetectEngineEventParse                [/home/victor/dev/oisf/src/detect-engine-event.c:173]
~~Dr.M~~ # 2 _DetectEngineEventSetup               [/home/victor/dev/oisf/src/detect-engine-event.c:204]
~~Dr.M~~ # 3 DetectDecodeEventSetup                [/home/victor/dev/oisf/src/detect-engine-event.c:248]
~~Dr.M~~ # 4 SigParseOptions                       [/home/victor/dev/oisf/src/detect-parse.c:510]
~~Dr.M~~ # 5 SigParseOptions                       [/home/victor/dev/oisf/src/detect-parse.c:523]
~~Dr.M~~ # 6 SigParse                              [/home/victor/dev/oisf/src/detect-parse.c:881]
~~Dr.M~~ # 7 SigInitHelper                         [/home/victor/dev/oisf/src/detect-parse.c:1309]
~~Dr.M~~ # 8 SigInit                               [/home/victor/dev/oisf/src/detect-parse.c:1456]
~~Dr.M~~ # 9 DetectEngineAppendSig                 [/home/victor/dev/oisf/src/detect-parse.c:1728]
~~Dr.M~~ #10 DetectLoadSigFile                     [/home/victor/dev/oisf/src/detect.c:334]
~~Dr.M~~ #11 SigLoadSignatures                     [/home/victor/dev/oisf/src/detect.c:422]
12 years ago
Victor Julien 1be6a8a48b Fix small leak in ports validation at startup 12 years ago
Victor Julien 3601091952 flowint: further setup fixes and cleanups 12 years ago
Victor Julien 8080494e9a counters: consolidate counters after all ThreadInit functions of a thread have run. This prevents duplicate and overwriting memory allocations. 12 years ago
Victor Julien 7f8d256e7c Fix tests that didn't expect radix to be freed 12 years ago
Victor Julien d2d784e31a radix: actually free a tree in SCRadixReleaseRadixTree 12 years ago
Victor Julien c94b920874 flowint: fix compile warning 12 years ago
Victor Julien a8c416fc8b flowint: fix setup memory leaks 12 years ago
Victor Julien 16130cc974 ssh: fix memleaks during ssh.softwareversion init and cleanup 12 years ago
Victor Julien ec724a1e56 urilen: fix memory leak when freeing the rule 12 years ago
Anoop Saldanha cfa2cda42b fix for bug #973.
An alternative solution for bug #970.

For chopped patterns, which in it's whole is a duplicate of another
pattern we assign an unique content id.
12 years ago
Anoop Saldanha 4da2f29054 Unittest for bug #973. 12 years ago
Victor Julien 0bfba8352d pcre: check for pcre_free_study, fall back to pcre_free if it unavailable 12 years ago
Victor Julien dd76e679fe mpm: clean up stream thread ctx 12 years ago
Victor Julien 6f450785fc profiling: properly clean up thread local memory. 12 years ago
Victor Julien eca1a8d73a profiling: don't alloc 0 bytes block if no rules are used 12 years ago
Victor Julien 468a8e1ca3 Properly cleanup NSS ctx 12 years ago
Victor Julien eedd4329da Change ParseSize api to not leak memory and only setup pcre once. 12 years ago
Victor Julien 3d78cc8ca6 DNS: free TX events using proper function 12 years ago
Victor Julien 6f2cb141cf Http: improve tx data cleanup 12 years ago
Victor Julien 239ab202c9 stream: clean up queue list in all cases 12 years ago
Victor Julien 67c12c61d3 Http: fix memory leaks when cleaning up our per-tx storage 12 years ago
Victor Julien 6aed56d093 Dns: fix memory leak when events are set 12 years ago
Anoop Saldanha cd80dcbfd4 bug #955 - Fix SSL parsing issue.
The parser wasn't carrying out a bounds check on record length while
in the middle of parsing a handshake.  As a result we would step onto the
next record header and consider it a part of the current handshake.

- Contains an unittest to test the issue.
- Disable the duplicate parser unittest registration.

The issue came to light through an irregular ssl record, which was
reported by Sebastian Roschke, via CVE-2013-5919.

Thanks to Sebastian Roschke for reporting this issue.
12 years ago
Anoop Saldanha 8c1e855632 fix for bug #970(ac-gfbs).
Content strings that are a duplicate of a pattern from another sig, but
have a fast_pattern chop being applied, would end up being assigned the
same pattern id as the duplicate string.  But the string supplied to the
mpm would be the chopped string, which might result in the state_table
output_state content entry being over-riden by the the fuller string at
the final state of the smaller content length, because of which during a
match we might end up inspecting the search buffer against the fuller
content pattern, instead of the chopped pattern, which would end up being
an inspection beyond the buffer bounds.
12 years ago
Anoop Saldanha 92a8b2b738 Unittest to display bug #970(ac-gfbs). 12 years ago
Anoop Saldanha 496f30a5e4 fix for bug #970(ac-bs).
Content strings that are a duplicate of a pattern from another sig, but
have a fast_pattern chop being applied, would end up being assigned the
same pattern id as the duplicate string.  But the string supplied to the
mpm would be the chopped string, which might result in the state_table
output_state content entry being over-riden by the the fuller string at
the final state of the smaller content length, because of which during a
match we might end up inspecting the search buffer against the fuller
content pattern, instead of the chopped pattern, which would end up being
an inspection beyond the buffer bounds.
12 years ago
Anoop Saldanha af95df67a5 Unittest to display bug #970(ac-bs). 12 years ago
Victor Julien 68ba9df8a0 Fix valgrind warning on memrchr unittest. 12 years ago
Anoop Saldanha d2ea799d38 fix for bug #970.
Content strings that are a duplicate of a pattern from another sig, but
have a fast_pattern chop being applied, would end up being assigned the
same pattern id as the duplicate string.  But the string supplied to the
mpm would be the chopped string, which might result in the state_table
output_state content entry being over-riden by the the fuller string at
the final state of the smaller content length, because of which during a
match we might end up inspecting the search buffer against the fuller
content pattern, instead of the chopped pattern, which would end up being
an inspection beyond the buffer bounds.
12 years ago
Anoop Saldanha da75db9330 Unittest to display bug #970. 12 years ago
Victor Julien 397a55457d Add sanity checks for command line argument handling
Coverity 1075221.

Normally getopt_long should cover this case, but can't hurt to
add in some extra checks.
12 years ago
Victor Julien c8b71938ff Add a fallback memrchr implementation for those platforms that dont support it. Bug #963. 12 years ago
Victor Julien e77b21a7f7 Suppress compiler warning about comparing signed and unsigned vars 12 years ago
Victor Julien bb8298ffa2 Move header thread_affinity declaration to extern to avoid duplicate declarations. 12 years ago
Victor Julien 3470b07ea5 Fix several compile and runtime warnings found by clang 3.2 with the -fsanitize=address option. 12 years ago
Victor Julien c82ecf553a Tag: document in the code that 'tag' is compatible with ip only 12 years ago
Victor Julien d12761233c Don't set tag on pseudo packets 12 years ago
Victor Julien 02cbbd0b89 unified2: fix tags not being logged. Bug #968 12 years ago
Anoop Saldanha 3749fc98fd Modify handling of negated content.
The old behaviour of returning a failure if we found a pattern while
matching on negated content is now changed to continuing searching
for other combinations where we don't find the pattern for the
negated content.

Thanks to Will Metcalf for reporting this.
12 years ago
Victor Julien 8539791c7e Coverity 1038102: remove dead code from host hash 12 years ago
Victor Julien 8237bbf18a Coverity 1038101: remove dead code from host hash timeout code 12 years ago
Victor Julien 440124a4b9 Coverity 1038100: remove dead code from flow hash timeout code(2) 12 years ago
Victor Julien 243060a6b7 Coverity 1038099: remove dead code from flow hash timeout code 12 years ago
Victor Julien 2e82772a0a Coverity 1038098: remove dead code from flow hash 12 years ago
Victor Julien aecefd00bd Coverity 1038095: remove dead code from defrag hash timeout code 12 years ago
Victor Julien 16056d51f2 Coverity 1038094: remove dead code from defrag hash 12 years ago
Victor Julien 32503bafaa Coverity 1038089: error check fseek call 12 years ago
Victor Julien 4827a4dcef Coverity 400477: pcre_get_substring retval
Add missing return code check to pcre_get_substring call.
12 years ago
Victor Julien 790866656b Coverity 1038129 fix
Don't leak memory on malloc error in b2gm mpm implementation.
12 years ago
Victor Julien 33919559d0 Fix memory leak on invalid luajit signature. Coverity 1038520. 12 years ago
Victor Julien 51c6a333d9 geoip: never try to store more locations than possible (Coverity 1038517) 12 years ago
Victor Julien 3cf3b485f2 Coverity 1038138 fix
Clean up parsing code to suppress Coverity:
Dereference before null check (REVERSE_INULL)

Proper checking was already done.
12 years ago
Victor Julien 27ea4232fe Coverity 1038134 fix
Cleaned up error check. "ipdup" can only be non-NULL there, so remove check
that confused coverity.
12 years ago
Victor Julien ecd5c7573b Coverity 1038135 fix
Small cleanup in the error handling. The extra null check confused
Coverity.
12 years ago
Victor Julien 38b6103ff5 Coverity 1038133 fix
Clean up parsing code to suppress Coverity:
Dereference before null check (REVERSE_INULL)

Proper checking was already done.
12 years ago
Ken Steele 50f859e9f2 Move SIMD implementations out of detect.c
Move SIMD the implementations of SigMatchSignaturesBuildMatchArray()
for SSE3 and Tile out of detect.c to reduce the size of the file.

Also moved SIMD unit tests to detect-simd.c
12 years ago
Victor Julien 7f140f6726 Coverity 1038111: fix local overrun of a string in app layer proto detect setup code. 12 years ago
Ken Steele eb4f0da97f Change one more atomic size in detect.h
Change uint16_t to int for better tile atomic performance. Checked with
pahole that it doesn't increase the size of the structure.
12 years ago
Ken Steele b08ddfa7f1 Support for Tile Gx atomic instructions
Tilera's GCC supports the GCC __sync_ intrinsics.

Increase the size of some atomic variables for better performance on
Tile.  The Tile-Gx architecture has native support for 32-bit and
64-bit atomic operations, but not 8-bit and 16-bit, which are emulated
using 32-bit atomics, so changing some 16-bit and 8-bit atomic into
ints improves performance.

Increasing the size of the atomic variables modified in this change
does not increase the total size of the structures in which they
reside because of existing padding requirements. The one case that
would increase the size of the structure (Flow_) was confitionalized
to only change the size on Tile.
12 years ago
Anoop Saldanha 54847e396f unittests for gzip, deflate http compression, multiple stacked
compressions, cunning compression that's not what it says it is, etc.

These unittests are tweaked to pass.  When libhtp fixes these issues
we will have to reenable them.
12 years ago
Anoop Saldanha 94e2527606 Introduce a saner way to validate the completion of request and
response bodies.

Also don't change app state for http from inside inspection.
12 years ago
Anoop Saldanha dcdcbd9721 Fix creating a backup of htp config. This is used by unittests that
changed htp config.
12 years ago
Ken Steele 62540eff3e Align some structures to cacheline
Align strucutres with pthread mutex locks to start on cachelines to keep
the lock within one cacheline.
12 years ago
Ken Steele d84079ba7d Move FlowIncrUsecnt to header file to allow for inlining.
Move FlowIncrUsecnt() and FlowDecrUsecnt() from flow.c to flow.h to
allow for inlining.
12 years ago
Ken Steele e05034f5dd New Multi-pattern matcher, ac-tile, optimized for Tile architecture.
Aho-Corasick mpm optimized for Tilera Tile-Gx architecture. Based on the
util-mpm-ac.c code base. The primary optimizations are:
1) Matching function used Tilera specific instructions.
2) Alphabet compression to reduce delta table size to increase cache
   utilization  and performance.

The basic observation is that not all 256 ASCII characters are used by
the set of multiple patterns in a group for which a DFA is
created. The first reason is that Suricata's pattern matching is
case-insensitive, so all uppercase characters are converted to
lowercase, leaving a hole of 26 characters in the
alphabet. Previously, this hole was simply left in the middle of the
alphabet and thus in the generated Next State (delta) tables.

A new, smaller, alphabet is created using a translation table of 256
bytes per mpm group. Previously, there was one global translation
table for converting upper case to lowercase.

Additional, unused characters are found by creating a histogram of all
the characters in all the patterns. Then all the characters with zero
counts are mapped to one character (0) in the new alphabet. Since
These characters appear in no pattern, they can all be mapped to a
single character and still result in the same matches being
found. Zero was chosen for the value in the new alphabet since this
"character" is more likely to appear in the input. The unused
character always results in the next state being state zero, but that
fact is not currently used by the code, since special casing takes
additional instructions.

The characters that do appear in some pattern are mapped to
consecutive characters in the new alphabet, starting at 1. This
results in a dense packing of next state values in the delta tables
and additionally can allow for a smaller number of columns in that
table, thus using less memory and better packing into the cache. The
size of the new alphabet is the number of used characters plus 1 for
the unused catch-all character.

The alphabet size is rounded up to the next larger power-of-2 so that
multiplication by the alphabet size can be done with a shift.  It
might be possible to use a multiply instruction, so that the exact
alphabet size could be used, which would further reduce the size of
the delta tables, increase cache density and not require the
specialized search functions. The multiply would likely add 1 cycle to
the inner search loop.

Since the multiply by alphabet-size is cleverly merged with a mask
instruction (in the SINDEX macro), specialized versions of the
SCACSearch function are generated for alphabet sizes 256, 128, 64, 32
and 16.  This is done by including the file util-mpm-ac-small.c
multiple times with a redefined SINDEX macro. A function pointer is
then stored in the mpm context for the search function. For alpha bit
sizes of 8 or smaller, the number of states usually small, so the DFA
is already very small, so there is little difference using the 16
state search function.

The SCACSearch function is also specialized by the size of the value
stored in the next state (delta) tables, either 16-bits or 32-bits.
This removes a conditional inside the Search function. That
conditional is only called once, but doesn't hurt to remove
it. 16-bits are used for up to 32K states, with the sign bit set for
states with matches.

Future optimization:

The state-has-match values is only needed per state, not per next
state, so checking the next-state sign bit could be replaced with
reading a different value, at the cost of an additional load, but
increasing the 16-bit next state span to 64K.

Since the order of the characters in the new alphabet doesn't matter,
the new alphabet could be sorted by the frequency of the characters in
the expected input stream for that multi-pattern matcher. This would
group more frequent characters into the same cache lines, thus
increasing the probability of reusing a cache-line.

All the next state values for each state live in their own set of
cache-lines. With power-of-two sizes alphabets, these don't overlap.
So either 32 or 16 character's next states are loaded in each cache
line load. If the alphabet size is not an exact power-of-2, then the
last cache-line is not completely full and up to 31*2 bytes of that
line could be wasted per state.

The next state table could be transposed, so that all the next states
for a specific character are stored sequentially, this could be better
if some characters, for example the unused character, are much more
frequent.
12 years ago
Victor Julien 77b429c402 xff: fix unittest crashes 12 years ago
Victor Julien 05d68ce394 xff: don't do xff check if there are no alerts anyway. 12 years ago
Duarte Silva 7dbb305255 Adds X-Forwarded-For support to the Unified2 output format
- Added the Unified2 file format related constants
 - Added IPv6 support
 - Two modes of operation with a fall-back to "extra-data" mode if
   "overwrite" mode is not applicable
 - Changed the configuration loading code to handle the new
   configuration structure
 - When creating the packet that fakes the one that generated the alert
   the flow direction wasn't taken into account in overwrite mode
 - Fixed BUG_ON condition
12 years ago
Victor Julien 900918a5d1 Bug #948: detect thread local storage support 12 years ago
Ken Steele 0861d3a2a3 Minor optimization in time caching code.
Reduced the size of the cached string buffer from 128 to 32, which is
still larger than the largest possible time string, which is 26
characters.

Added a check for the user passing in an output buffer that is smaller
than the cached string. Previously, the code would have copied past
the end of the users buffer.
12 years ago
Anoop Saldanha 49dcb0ca84 fix for #925.
Log sensible error message when the user doesn't supply a value for
stream.prealloc-sessions or when the values supplied in invalid and
the engine resorts to using a default.
12 years ago
Anoop Saldanha db6ef81fb0 fix for #926.
Supply meaningful error message when user supplies invalid value for
host.prealloc.
12 years ago
Anoop Saldanha b90a56b626 fix for #927.
Print an error message when the user supplies an invalid value for
detect-thread-ratio in the conf file.
12 years ago
Anoop Saldanha bed3f605fa Fix for #922.
Add more relevant error message when we supply invalid value for
defrag.trackers and defrag.hash-size
12 years ago
Anoop Saldanha 6608e7f523 Introduce generic utility API to log message on invalid config entry. 12 years ago
Victor Julien 6d34834623 Runmode fixes and cleanups
Bug #939: thread name buffers are sized inconsistently
These buffers are now all fixed at 16 bytes.

Bug #914: Having a high number of pickup queues (216+) makes suricata crash
Fixed so that we can now have 256 pickup queues, which is the current built-in
maximum. Improved the error reporting.

Bug #928: Max number of threads
Error reporting improved. Issue was the same as #914.
12 years ago
Anoop Saldanha 56143131da Fix unittests that use chunked encoding. 12 years ago
Nelson Escobar ef4d11aeb5 Use the Async versions of SCCudaMemcpy* to improve gpu performance. 12 years ago
Eric Leblond 77f2b9968e autotools: use builddir instead of srcdir
srcdir is supposed to be read-only when running distcheck so it is better to
create the log directory in builddir.
12 years ago
Ignacio Sanchez 1b2f251866 Various custom http logging improvements
Cookie is parsed now using uint8_t pointers (inliniac PR comments)
Changed buffer size to a power of 2 (8192) and cookie value extraction function to static (inliniac PR comments)
Added %b for request size (vinfang patch)
Writing "-" if an unknown % directive is used (vinfang patch)
Fixed bug in cookie parser
Fixed format string issue logging literal values
Improve error handling (Victor Julien comments)

(patchset rebased and reworded by Victor Julien)
12 years ago
Ignacio Sanchez 8051dc8a6a Added modifications suggested by Charles Smutz (https://redmine.openinfosecfoundation.org/issues/602) 12 years ago
Ignacio Sanchez 796bfab231 Added support for %{cookiename}C
Added support for the definition of maximun length. ie: %[50]{user-agent}i
Some small bugfixes
12 years ago
Eric Leblond 3dbf6c6fee solaris: fix compilation failure
This patch fixes a compilation failure on Solaris. Compiler does
not support when a function returning void is used in return of
an other function returning void.
12 years ago
Ken Steele a2b502a30c Formatting change for function call.
Put open brace { for function on a new line to match coding standard.

Changed:

int foo(int x) {
}

to:

int foo(int x)
{
}
12 years ago
Ken Steele d4dd18eb85 Clean up SCLocalTime() usage
Remove cast of return type from SCLocalTime() as it is not needed.
Replace last use of localtime_r() with SCLocalTime().
12 years ago
Ken Steele 77fae5313d On Open BSD systems don't cache time.
Open BSD doesn't support __thread, which is used for time caching, so
don't do time chaching for BSD systems.
12 years ago
Ken Steele 2feb37c155 Cache time conversions for localtime() and CreateTimeString()
When converting a time in seconds (64-bit seconds since 1970) to
Month/Day/Year hours minutes, Suricata calls localtime_r(), which
always aquires a lock and then does complex comutation based on the
current time zone. The time zone can be specified in the TZ
environment variable, which is only parsed the first time it is used,
or from a file. The default file is /etc/localtime. The file is
checked each time to see if it might have changed and is reparsed if
it has changed.

The GLIBC library has a lock inside localtime_r(), which limits
parallelism, which is a problem when the rate of generating alerts is
high, since Suricata generates a new ascii time string for each alert
into fast.log.

This change caches the value returned by localtime_t() and then sets
the seconds within the minute based on the cached start-of-minute
time. All of the values return, expect for the seconds, is constant
within the same minute. Switching to a new seconds could change all
the other values, year, month, day, hour. The cache stores the current
and previous minute values.

The same trick is used in CreateTimeString() for generated time
string. The string, up to the minutes, is cached and then copied into
the result string, followed by printing the new seconds into the
result string.

The seconds within a minute are calculated as the difference in
seconds from the start of the current minute.
12 years ago
Ken Steele 68d26dcec7 Merge multiple copies of CreateTimeString() to one copy.
There were 8 identical copies of CreateTimeString() in 8 files.
Most used SCLocalTime, to replace localtime_r(), but some did not.
Created one copy in util-time.c.
12 years ago
Ken Steele 5532af4621 Create SCMUTEX_INITIALIZER to abstract out PTHREAD_MUTEX_INITIALIZER
This allows replacing pthread mutexes with other types of mutex.
12 years ago
Ken Steele 784843b146 Use Tilera SIMD for Signature matching ala SSE3
Makes use of 8-wide byte compare instructions in signature matching.

For allocating aligned memory, _mm_malloc() is SSE only, so added
check for __tile__ to use memalign() instead.

Shows a 13% speed up.
12 years ago
Ken Steele 22225a7e99 Tile SIMD implementation of SCMemcmp and SCMemcmpLowercase
Based on the SSE3 implementation, it checks 8 bytes at a time.
12 years ago
Anoop Saldanha e68d44b051 fix for #932.
ipv6 tunnel decoder wrongly treats the tunneled ipv6 packets as an ipv4
packet.
12 years ago
Anoop Saldanha e2f4144d99 fix for #920.
Cull the space before the address specified in address var variables.
12 years ago
Duarte Silva ab215c72f6 Now using the common functions 12 years ago
Duarte Silva 0a5c798729 Now using the common functions
- Removed some non printable ANSI characters
- Removed unecessary include
12 years ago
Duarte Silva 8ce95af09c Added the new files containing the repeated functions
- Renamed the functions to something more generic
- Added the source and include files to the Makefile
12 years ago
Anoop Saldanha a44d42b124 Fixes segv inside rule swap under low mem conditions.
We now gracefully exit rule swap on any allocation or other failures.
12 years ago
Anoop Saldanha 8516ba24c9 Rearrange ac state.
Notice a minor speed bump of around 2% on runs.  More updates to follow.
12 years ago
Ken Steele 4b8bb11454 Enable using Tile cycle counter.
The Tile processors all have a cycle counter with a simple interface. Use
that for UtilCpuGetTicks.
12 years ago
Victor Julien 38aaae1fd7 IsRuleReloadSet() shouldn't return an uninitialized value 12 years ago
Eric Leblond 189327981a unittests: fix stream-tcp.c
Lock and recycle fixes for stream-tcp.c
12 years ago
Eric Leblond cd3e32ce19 unittests: some functions needs a flow lock.
In debug validation mode, it is required to call application layer
parsing and other functions with a lock on flow. This patch updates
the code to do so.
12 years ago
Eric Leblond c5bd04f102 unittest: recycle packet before exit
To avoid an issue with flow validation, we need to recycle the packet
before cleaning the flow.
12 years ago
Anoop Saldanha d292f1a529 fix for #915. Fix segv when we send NULL to snprintf. 12 years ago
Eric Leblond c6e8c5bf1f pf_ring: avoid to ask for extended header.
This patch update pf_ring capture to avoid to ask for extended
header. They are only needed when rxonly checksum checks is used
and this is only possible when interface is not a DNA interface.
12 years ago
Victor Julien ff668c2030 Fix Tile compile 12 years ago
Eric Leblond 20ca270dc3 fix pf_ring build 12 years ago
Eric Leblond 2a46f0dae4 suricata: rename SuriInstance to SCInstance. 12 years ago
Eric Leblond 9b422f3a8c suricata: suppress Suri prefix
Suppress Suri prefix in internal function name.
12 years ago
Eric Leblond 18ced653c3 Use a typedef for SuriInstance. 12 years ago
Eric Leblond 2d77e53f2c Add offline flag to SuriInstance and some refactoring 12 years ago
Eric Leblond 34abd818dd Prefix util-conf function with Config 12 years ago
Eric Leblond 7242cb30e7 Move CreateLowercaseTable to GLobalInits 12 years ago
Eric Leblond 02e9851315 Generic code don't need ifdef 12 years ago
Eric Leblond 8c00a963aa Use function for delayed detect setup. 12 years ago
Eric Leblond 4296e5f29e Add functions for elapsed time computation. 12 years ago
Eric Leblond 9d1d08c7a4 Factorize Signature loading 12 years ago
Eric Leblond 20c5683b60 Use function for daemonification and signal handler 12 years ago
Eric Leblond 90aaf55201 set rule_reload as part of SuriInstance 12 years ago
Eric Leblond bb19ce1847 SetBPfString is part of command line parsing 12 years ago
Eric Leblond 1a6983ee19 suricata: use function to print version 12 years ago
Eric Leblond 4f789dbe84 Add function for internal running mode 12 years ago
Eric Leblond d3cb043001 suricata: windows specific in one function 12 years ago
Eric Leblond 4401c048ba Running mode is set earlier so out earlier 12 years ago
Eric Leblond 40a25112a0 kill remaining run_mode usage 12 years ago
Eric Leblond 75fa1e20d7 engine analysis is a running mode 12 years ago
Eric Leblond c0d5ee77f9 get (almost) rid of run_mode variable. 12 years ago
Eric Leblond 80542816cd add internal running mode 12 years ago
Eric Leblond e07fdb20a8 Add SuriInstance structure
To be able to split code in functions in main, we need to pass
information about the current running Suricata to functions.
For that we create a structure to store suricata run parameters.

In this patch it allows to separate command line parsing and to
treat internal running mode in a switch just after command line
parsing.
12 years ago
Eric Leblond 325462d396 Export IsRuleReloadSet and use it. 12 years ago
Eric Leblond 6d9a66d522 unittest: make check use a qa/log dir for logging
This patch is using the qa/log directory to store the output
of the check. In case of success, the directory is deleted.
In case of failure, the directory remains in place.

This should fixes #910.
12 years ago
Eric Leblond 4424f5a231 af-packet: add sanity check in free function 12 years ago
Eric Leblond 8e68b357c7 Suppress Suri prefix. 12 years ago
Eric Leblond 42011e2d32 suricata: function for lowercase table creation 12 years ago
Eric Leblond 132bebb2b2 Simplify code by removing comment 12 years ago
Eric Leblond 07ef1f9837 suricata: add wrapper for interface listing 12 years ago
Eric Leblond 54006de40c Use new function GetLogDirectory() 12 years ago
Eric Leblond 2be7c8aea8 Add util-conf for config util 12 years ago
Eric Leblond 27752818c2 suricata: add some wrapper for config file handling 12 years ago
Eric Leblond b2fa4edd36 move unittest out of suricata.c 12 years ago
Eric Leblond 9a0bf0956b suricata: list cuda cards in separate function 12 years ago
Eric Leblond bed48e3a54 suricata: separate keyword and app layer listing code
The list-keyword and app-layer listing code was spread over all the
init code. This patch introduces a separate file to store non standard
running mode like these ones.
12 years ago
Eric Leblond 135ef0186b runmodes: fix comment 12 years ago
Victor Julien 5a7bf53a6b Storage: rename Init to Alloc to reflect actual functioning. Comment updates. 12 years ago
Victor Julien f06694d0c1 Storage API: add safety check for cases when there is no storage used. 12 years ago
Eric Leblond caf730d988 engine-tag: rename var and add sanity check 12 years ago
Eric Leblond fb55931c30 flow tag: conversion to flow storage API
This patch is updating the flow tag system to use the flow
storage API. The tag_list member of Flow structure is suppressed
and its cleaning operation are suppressed too as this is handled
transparently by the flow storage API.
12 years ago
Eric Leblond 4db2fc2cbb Add per-flow generic storage
This patch adds a per-flow storage that can be created via the functions
available in flow-storage.c.
12 years ago
Eric Leblond 6d08807b2d Host: use global free storage function
This patch is here to avoid that all modules using a local storage
have to update host code to add their free function. It modifies
previous behavior by calling HostFreeStorage in any case.
12 years ago
Victor Julien 27023872de Use Host Storage API for per host thresholding 12 years ago
Victor Julien c08b395c2c Init storage api at start up 12 years ago
Victor Julien 5919901675 Storage API: add registration check closed test in debug mode. 12 years ago
Victor Julien 3447324c36 Move Host Tag storage to Host Storage API. 12 years ago
Victor Julien 0d2a6e515e Add Host specific wrapper to StorageRegister() 12 years ago
Victor Julien b5ccf0b9c7 storage: allow preallocated storage 12 years ago
Victor Julien e2b006f523 host: use storage api 12 years ago
Victor Julien 022c0e466e Initial storage api work 12 years ago
Victor Julien 1c06d52208 Misc fixes after make check feedback 12 years ago
Ken Steele 316190c6b9 Add TILE-Gx mPIPE packet processing support.
The TILE-Gx processor includes a packet processing engine, called
mPIPE, that can deliver packets directly into user space memory. It
handles buffer allocation and load balancing (either static 5-tuple
hashing, or dynamic flow affinity hashing are used here). The new
packet source code is in source-mpipe.c and source-mpipe.h

A new Tile runmode is added that configures the Suricata pipelines in
worker mode, where each thread does the entire packet processing
pipeline.  It scales across all the Gx chips sizes of 9, 16, 36 or 72
cores. The new runmode is in runmode-tile.c and runmode-tile.h

The configure script detects the TILE-Gx architecture and defines
HAVE_MPIPE, which is then used to conditionally enable the code to
support mPIPE packet processing. Suricata runs on TILE-Gx even without
mPIPE support enabled.

The Suricata Packet structures are allocated by the mPIPE hardware by
allocating the Suricata Packet structure immediatley before the mPIPE
packet buffer and then pushing the mPIPE packet buffer pointer onto
the mPIPE buffer stack.  This way, mPIPE writes the packet data into
the buffer, returns the mPIPE packet buffer pointer, which is then
converted into a Suricata Packet pointer for processing inside
Suricata. When the Packet is freed, the buffer is returned to mPIPE's
buffer stack, by setting ReleasePacket to an mPIPE release specific
function.

The code checks for the largest Huge page available in Linux when
Suricata is started. TILE-Gx supports Huge pages sizes of 16MB, 64MB,
256MB, 1GB and 4GB. Suricata then divides one of those page into
packet buffers for mPIPE.

The code is not yet optimized for high performance. Performance
improvements will follow shortly.

The code was originally written by Tom Decanio and then further
modified by Tilera.

This code has been tested with Tilera's Multicore Developement
Environment (MDE) version 4.1.5. The TILEncore-Gx36 (PCIe card) and
TILEmpower-Gx (1U Rack mount).
12 years ago
Victor Julien 04f3f14541 ipv6: fix parsing of malformed ext hdr. Bug #908. 12 years ago
Victor Julien 4b4111e9e2 icmpv6: fix icmp_id and icmp_seq keywords
Bug #907
12 years ago
Victor Julien d82ce3f50c Fix compiler warning due to missing include
decode.c: In function 'DecodeThreadVarsAlloc':
decode.c:437: error: implicit declaration of function 'ConfGetBool'
12 years ago
Victor Julien 16c3487444 Add yaml option to disable vlan ids hashing
In some cases using the vlan id(s) in flow hashing is problematic. Cases
of broken routers have been reported. So this option allows for disabling
the use of vlan id(s) while calculating the flow hash, and in the future
other hashes.

Vlan tracking for flow is enabled by default.
12 years ago
Victor Julien 58ed1f2411 flow: take vlan_id's into account in the flow hash
In VLAN we can have 2 layers of encapsulation. In this patch both
layers are used in the flow hash to distinguish between encapsulated
traffic.
12 years ago
Victor Julien 055b422c28 Remove obsolete code: flow alert sid storage 12 years ago
Victor Julien 9faa4b740d Add --unittests-coverage option to list how many code modules have tests 12 years ago
Victor Julien fc7879322e Rename GetIfaceMaxPayloadSize to GetIfaceMaxPacketSize to reflect the actual function. 12 years ago
Victor Julien bd21b5ed9c Pcap: fix snaplen autodetection, GetIfaceMTU doesn't include link layer length 12 years ago
Anoop Saldanha ee0b21652b fix bug where we were not printing http hostname(printing <unknown>
previously) in httplog, filestore meta and file log.
12 years ago
Victor Julien 7edcc13514 NFQ: fix packets not getting freed 12 years ago
Anoop Saldanha cdaa13012a fix for #882.
Refactor the code that initializes the cuda mpm environment.
12 years ago
Victor Julien 9f3e2f7a92 NFQ: adapt to ReleasePacket API 12 years ago
Ken Steele b076a26cdc Replace ReleaseData function on Packet Structure with ReleasePacket.
This commit allows handling Packets allocated by different methods.
The ReleaseData function pointer in the Packet structure is replaced
with ReleasePacket function pointer, which is then always called to
release the memory associated with a Packet.

Currently, the only usage of ReleaseData is in AF Packet. Previously
ReleaseData was only called when it was not NULL. To implement the
same functionality as before in AF Packet, a new function is defined
in AF Packet to first call the AFP specific ReleaseData function and
then releases the Packet structure.

Three new general functions are defined for releasing packets in the
default case:
    1) PacketFree() - To release a packet alloced with SCMalloc()
    2) PacketPoolReturnPacket() - For packets allocated from the Packet Pool.
                                  Calls RECYCLE_PACKET(p)
    3) PacketFreeOrRelease() - Calls PacketFree() or PacketPoolReturnPacket()
                                 based on the PKT_ALLOC flag.

Having these functions removes the need to check the PKT_ALLOC flag
when releasing a packet in most cases, since the ReleasePacket
function encodes how the Packet was allocated. The PKT_ALLOC flag is
still set and is needed when AF Packet releases a packet, since it
replaces the ReleasePacket function pointer with its own function and
then calls PacketFreeOfRelease(), which uses the PKT_ALLOC flag.
12 years ago
Anoop Saldanha f85a2dc84b fix for #875.
Update configure.ac to check for either 0.5.5 and 0.5.x version of libhtp.
12 years ago
Anoop Saldanha 9698a5d78c Code to enable cuda support for pfring live mode. 12 years ago
Victor Julien 91fb47475b DNS: break out of DNSResponseGetNameByOffset if we're in there too long. Can happen on bad data. 12 years ago
Victor Julien aa26dae5a1 Stream: don't inject stream end pseudo pkt on FinWait2 state. Bug #883. 12 years ago
Victor Julien 2f3f577fb6 DNS: convert info logs to debugs 12 years ago
Victor Julien 97f51c1011 Fix ac-bs and ac-gfbs mpm-algo settings leading to fatal error if CUDA is enabled. Workaround for #882. 12 years ago
Eric Leblond e2334fbfe8 unix socket: fix typo in error message 12 years ago
Eric Leblond c2cbb43776 autotool: INCLUDES usage is deprecated 12 years ago
Eric Leblond 281d2f27f8 Fix compilation warning
A goto could lead to the use de_ctx without declaring it.
12 years ago
Victor Julien f4dcba6de3 In case of fragments, don't consider ports. Bug #847. 12 years ago
Anoop Saldanha e7f09f24c8 Code to enable cuda support for live mode pcap and af-packet. Keep an eye
out on the mailing list and http://planet.suricata-ids.org for performance
and other profiling data.
12 years ago
Victor Julien 51d6c63860 Luajit: fix compilation and tests after libhtp upgrade 12 years ago
Anoop Saldanha 48cf0585fb Suricata upgrade to libhtp 0.5.x.
Remove the support for now unsupported personalities from libhtp -
TOMCAT_6_0, APACHE and APACHE_2_2.  We instead use the APACHE_2
personality.
12 years ago
Victor Julien 080c15b3fc Enable libhtp 0.3.0 compilation and crash free UT run. Still see 5 failed tests. 12 years ago
Victor Julien 538da26812 Fix sgh mpm flags assignment 12 years ago
Eric Leblond 150cd39c6e detect-engine: do a direct update of flag
There is no reason not to update the flag directly. So do it
to avoid to crash the test.
12 years ago
Eric Leblond 2f2916d9ec ccccinelle: add formatted comment for flag test 12 years ago
Victor Julien aafc65c757 Autotools: move libhtp conditionals to configure
In preparation of the libhtp upgrade, move all libhtp related conditionals
to configure. This allows for one set of build scripts that works regardless
of the presence of a local libhtp dir.
12 years ago
Victor Julien 73e27c1fb7 Generate proper errors if sid,gid,rev values are out of range. Bug #779. 12 years ago
Victor Julien 164d60e8cd Yaml: give a more detailed error if the user supplies a directory instead of a yaml file. Bug #803. 12 years ago
Victor Julien a4e838c1d3 TLS: create certs directory during startup if it doesn't exist yet. Bug #710. 12 years ago
Eric Leblond 0e92469222 nfq: be sure to always verdict packets
To be sure to always verdict packets (bug #769), this patch adds
a ReleaseData function to NFQ packets. The release function simply
drop the packet if it has not been verdicted before.
12 years ago
Victor Julien 4a0050b9ea Print pkt src to alert-debug log 12 years ago
Victor Julien 1c371da46d DNS: better handle TX' with lost replies 12 years ago
Victor Julien 0fd9b0c4fa HTP: free TX from transaction free API call 12 years ago
Victor Julien 1367074c75 App layer: clean up TX before lowest active one
Update DNS to handle cleaning up this way.
12 years ago
Victor Julien 0b229ec8b9 DNS: suppress log-dns registration message 12 years ago
Victor Julien f59f90331d Applayer: remove obsolete StateUpdateTransactionId
Also, update StateTransactionFree to take an u64 tx id, so it's
consistant with the rest of the engine.

To reflect these changes, AppLayerRegisterTransactionIdFuncs has
been renamed to AppLayerRegisterTxFreeFunc.

HTP, DNS, SMB, DCERPC parsers updated.
12 years ago
Victor Julien ebab9aee83 DNS: move internal tx id tracking to u64 12 years ago
Victor Julien e8ad876b48 App layer: add 'StateHasEvents' API call
Per TX decoder events resulted in significant overhead to the
detection engine, as it walked all TX' all the time to check
if decoder events were available.

This commit introduces a new API call StateHasEvents, which speeds
up this process, at the expense of keeping a counter in the state.

Implement this for DNS as well.
12 years ago
Anoop Saldanha cd7b4fac40 remove unused pattern id assignment functions. Goodbye 12 years ago
Victor Julien f353fb630c DNS: convert dns_query to sticky buffer 12 years ago
Victor Julien 7292998a58 Content: set up sticky buffers like file_data and dce_stub_data w/o flags, but with a list variable 12 years ago
Victor Julien d476e4e50d Coverity 1040312, 1040313, 1040314 1040315: improve pool thread error handling. 12 years ago
Victor Julien 1373a20e8a Thread: remove thread id 12 years ago
Victor Julien d7aaa9464c Stream: use per thread ssn_pool_id instead of thread id. 12 years ago
Victor Julien 92b7ffad69 Improve memory cleanup in some unittests 12 years ago
Victor Julien fd7899cc8b Stream: fix unittests after ssn pool changes. 12 years ago
Victor Julien aa449d51ca Stream: use per thread ssn pool
Use per thread pools to store and retrieve SSN's from. Uses PoolThread
API.

Remove max-sessions setting. Pools are set to unlimited, but TCP memcap
limits the amount of sessions.

The prealloc_session settings now applies to each thread, so lowered the
default from 32k to 2k.
12 years ago
Victor Julien b6af6cb241 pool: add error msgs and improve memory layout 12 years ago
Victor Julien 5b9ef94f34 pool: add api for per thread pools
This API is a wrapper around the regular pools where the thread pools
are arrays of locks+pools.
12 years ago
Victor Julien 016d03bdaf pool: add error msgs and improve memory layout 12 years ago
Victor Julien 46af6b7e0f Add a per threadvars thread local thread id, that starts at 0 and increments for each thread. 12 years ago
Victor Julien b3b554c269 Coverity 1038959: DNS mpm might use initialized variable 12 years ago
Anoop Saldanha fba95e9125 Remove mpm ctxs in the wrong direction.
A lot of http mpm ctxs have now been removed as a result of this.
12 years ago
Anoop Saldanha 3c2ddf04c1 Update mpm init ctx to not accept the final cuda_rc_module argument.
It was a part of our older architecture and is no longer used.
12 years ago
Victor Julien 33818c0272 DNS: fix CUDA build 12 years ago
Victor Julien be7e6cdd7a DNS: fix warning when debug is not enabled 12 years ago
Victor Julien 571b8ac186 DNS: add support for per TX decoder events. 12 years ago
Victor Julien 9dc04d9fab app layer: add support for per TX decoder events 12 years ago
Victor Julien 72e35efbc6 Reset app layer events when we start inspecting a new TX 12 years ago
Victor Julien 28a6c1d9f8 DNS: add test for app layer event match 12 years ago
Victor Julien 6645620c03 Merge SIG_FLAG_MPM_HTTP and SIG_FLAG_MPM_DNS into SIG_FLAG_MPM_APPLAYER, do the same for the _NEG variant. 12 years ago
Victor Julien 43ba5a677e DNS: enable mpm/fast_pattern support for dns_query 12 years ago
Victor Julien 4817e1305f DNS: add /F modifier to pcre to inspect DNS query name 12 years ago
Victor Julien e567e12230 DNS: add unittests for UDP and TCP for dns_query keyword 12 years ago
Victor Julien f10dd603ff DNS: adding dns_request content modifier 12 years ago
Victor Julien 6674f4892c DNS: add per tx internal id
Add per TX id. Rename transaction_cnt to transaction_max (id) and increment it on tx creation.
12 years ago
Victor Julien 59780ca770 Hacks to enable alert dns even though we have dnstcp and dnsudp parsers. Needs proper solution later. 12 years ago
Victor Julien 8e01cba85d DNS TCP and UDP parser and DNS response logger 12 years ago
Eric Leblond 4521de2dfd Use PACKET_* macro instead of UPDATE
Setting the ACTION_DROP flag can be done via PACKET_DROP instead
of using PACKET_UPDATE_ACTION.
12 years ago
Eric Leblond c0c59fbd17 decode: factorize macro code
PACKET_* are now wrapper to the newly introduced PACKET_SET_ACTION
macro.
12 years ago
Eric Leblond 3f107fa130 decode: Packet action start with PACKET
Rename all Packet action macro to have them prefixed by PACKET.
12 years ago
Anoop Saldanha 3304c91c91 Don't let geoip match on pseudo packets. 12 years ago
Anoop Saldanha 1fb4aae993 Coverity 1038523: Fix using cuda buffer slice that has been returned to the pool. 12 years ago
Victor Julien 51cdd464a6 stream: detect keep-alive and keep-alive ACK 12 years ago
Victor Julien 03c3ff5632 stream: fix typo in function name 12 years ago
Eric Leblond 4c6595f437 Coverity 1038106: fix FP out-of-bond access
A cast during the reading of a configuration variable was invalid
because a 16 bit integer was cast to a 32 bit integer. The called
function is only setting the pointer value to 1 or 0 so there is
no real issue there.
12 years ago
Eric Leblond 7df156ef50 Coverity: 1038139 suppress sanity check
The sanity check was really useless as the NULL value is checked in
the code flow.
12 years ago
Eric Leblond 75cd1f6096 Coverity 1038515: check function return
This is more cosmetic than useful but it is cleaner anyway.
12 years ago
Anoop Saldanha b4e750068f Cuda make distcheck fix for cuda-ptxdump.h 12 years ago
Anoop Saldanha 7d46d59cdd Coverity 1038522: fix memset inside cuda code. Wrong size specified to memset. 12 years ago
Victor Julien d6fcd07a31 Coverity 1038085: remove 'default' statement in SCErrorToString. This way a warning will be given if an error is defined w/o updating this function. 12 years ago
Victor Julien e2444f0ed5 Coverity 1038092 & 1038093: remove dead code 12 years ago
Victor Julien 0902c7f3aa Coverity 1038518: fix wrong error check 12 years ago
Victor Julien db1dad8cc6 Coverity 1038124: memory leak on 'seq' keyword parsing failure 12 years ago
Victor Julien b2e962da03 Coverity 1038123: memory leak on 'flowint' keyword parsing failure 12 years ago
Victor Julien 5c5b2f98dd Coverity 1038116 & 1038117: memory leaks on 'app-layer-event' keyword parsing failure 12 years ago
Victor Julien 778851626c Coverity 1038115: memory leak on 'ack' keyword parsing failure 12 years ago
Victor Julien 98dbf3e62c Coverity 1038113: possibly out of bounds read 12 years ago
Anoop Saldanha 602c91ed41 Minor cosmetic changes to the cuda code.
Moved a couple of functions to more cuda relevant files;
Re-structured some data types.
12 years ago
Anoop Saldanha c9f076def3 Modified CudaBufferCullCompletedSlices.
Allow readers specify max size of data they want to read.
12 years ago
Anoop Saldanha 70cb4d30eb Add a usleep to CudaBuffer culling process. Would lead to a situation where the thread wouldn't care to yield to others." 12 years ago
Anoop Saldanha 17c763f855 Version 1 of AC Cuda. 12 years ago
Anoop Saldanha 2de59fc235 Version 1 of CudaBuffer API. Introduced to buffer data to the gpu.
This version allows async writes to a buffer by threads.  Allows only
sequential reads though.
12 years ago
Anoop Saldanha 557cab3dc9 We call packet and stream mpm as late as possible now. Won't affect the working of the engine.
The rationale behind this is, if we have pkt buffered to the gpu, we'd want
to delay processing the pkt as much as possible.
12 years ago
Anoop Saldanha d2063d98ad pool now uses a queue kinda behaviour when getting/inserting data through poolbuckets. 12 years ago
Anoop Saldanha f4c719b83a code refactoring. Call mpmprefilter slightly later than where it's called atm 12 years ago
Anoop Saldanha b787da5643 Remove all cuda related code in the engine except for the cuda api wrappers 12 years ago
Anoop Saldanha e2a6cfb6a6 update cuda API wrappers 12 years ago
Eric Leblond d8ce2b1ca4 unix-socket: fix OSX build
MSG_NOSIGNAL is not defined on macOSX and SO_NOSIGPIPE is used
instead.
12 years ago
Eric Leblond a35c367942 action handling: use macro for test.
Use test macro instead of direct access to action field.

This patch has been obtained by using the following
spatch file:

  @@
  Packet *p;
  expression E;
  @@

  - p->action & E
  + TEST_PACKET_ACTION(p, E)
12 years ago
Eric Leblond efaa9a7302 action handling: define and use macros
The action field in Packet structure should not be accessed
directly as the tunneled packet needs to update the root packet
and not the initial packet.

This patch is fixing issue #819 where suricata was not able to
drop fragmented packets in AF_PACKET IPS mode. It also fixes
drop capability for tunneled packets.
12 years ago
Anoop Saldanha 429b5cec10 Fix magic unittets.
Fix segv, when magic_load() fails due to the non-availability of default
magic files.
12 years ago
Anoop Saldanha 058e9278c5 Fix wrong casting of htp pointer. Fixed it back to (HTPState *) inside
htp utility functions.
12 years ago
Anoop Saldanha 21f9cc3a39 discontinue matching on buffer if urilen returns a match failure. 12 years ago
Victor Julien 56c6dd9bb2 bytetest: add unittest showing missed detection
Tests recursive and relative negative byte_test matching.
12 years ago
Anoop Saldanha c3d98f9640 Fix the bug specified in the previous commit.
Bug emanates from byte_test, byte_jump and byte_extract keyword being
unable to handle negative offsets when the inspection pointer is at the
end of the buffer.
12 years ago
Anoop Saldanha bd6896bee1 Unit-tests exposing a bug in byte_test, byte_jump and byte_extract.
Bug emanates from all the keywords being unable to handle negative offsets
when the inspection pointer is at the end of the buffer.
12 years ago
Victor Julien 0fbfaadd53 bytetest: fix debug messages not printing negative offset correctly 12 years ago
Anoop Saldanha ff222b51e7 Http trailer headers unittests added. 12 years ago
Anoop Saldanha ab4b15c2e7 fix for #788.
Now depth is kept in mind when we inspect chunks in client/server body.
This takes care of FPs originating from inspecting subsequent chunks that
match with depth, but shouldn't.
12 years ago
Victor Julien f29e5459e6 luajit/flowint: add ScFlowintIncr & ScFlowintDecr
Add flowint lua functions for incrementing and decrementing flowints.

First use creates the var and inits to 0. So a call:

    a = ScFlowintIncr(0)

Results in a == 1.

If the var reached UINT_MAX (2^32), it's not further incremented. If the
var reaches 0 it's not decremented further.

Calling ScFlowintDecr on a uninitialized var will init it to 0.

Example script:

    function init (args)
        local needs = {}
        needs["http.request_headers"] = tostring(true)
        needs["flowint"] = {"cnt_incr"}
        return needs
    end

    function match(args)
        a = ScFlowintIncr(0);
        if a == 23 then
            return 1
        end

        return 0
    end
    return 0

This script matches the 23rd time it's invoked on a flow.
12 years ago
Victor Julien f312486c6e flowvar/luajit: make 'sets' real time. Needed for cross HTTP-header matching. 12 years ago
Victor Julien 72f6bc2aed luajit: add flowint support
Expose ScFlowintGet and ScFlowintSet functions to luajit. These set
flowints in real time, regardless of rule and/or script match.

Example:

function init (args)
    local needs = {}
    needs["http.request_headers"] = tostring(true)
    needs["flowint"] = {"cnt"}
    return needs
end

function match(args)
    a = ScFlowintGet(0);
    if a then
        ScFlowintSet(0, a + 1)
    else
        ScFlowintSet(0, 1)
    end

    a = ScFlowintGet(0);
    if a == 23 then
        return 1
    end

    return 0
end

return 0

Script's init call first registers "cnt" at id 0, then 0 is used to use
this var.
12 years ago
Victor Julien c3c3cd76e5 flowvar/flowint: split set functions into normal and NoLock version, where the latter won't lock the flow. 12 years ago
Victor Julien 57d3cd97f3 flowvar/flowint: make local function static 12 years ago
Victor Julien 6e18ed0489 luajit flowvar support
This patch adds flowvar support to luajit. It does so by exposing two special
C functions to the luajit scripts: ScFlowvarGet and ScFlowvarSet.
12 years ago
Ken Steele 9d677ea006 Clear the PKT_ALLOC flag when storing Packets into the Packet pool.
The PKT_ALLOC flag is set by PacketGetFromAlloc(), which needs to be
cleared for Packets in the Packet Pool, so clear the flag here.
12 years ago
Ken Steele 9c7b411a5d More PacketGetFromMalloc() to allocate packets. 12 years ago
Ken Steele 394f99e32c Use PacketGetfromAlloc() for packet allocation instead of SCMalloc.
Only changed in one file for testing.
12 years ago
Anoop Saldanha ab1f8afbc3 Removed Signature->order_id and replaced it with Signature->num. 12 years ago
Anoop Saldanha 43d1229dfa 1. Fix assignment of signums, which affected how we used read
sigs(priority wise) inside staging.

   Previously we would assign signums before sig ordering, and hence the
   order didn't actually reflect the order of the sig in the
   sig_list(assuming sig reordering changed the sig_list).  Staging would
   use the old sig_nums to decide the priority of sigs.
2. Fix sig ordering for flowvar, flowbits, flowint, pktvar sigs.   We have
   introduced a new priority to treat sigs with set + read as lower
   priority compared to set only sigs.
3. Previously we treated sigs with a "priority(keyword)" > another sig's
   priority, as a sig with greater priority than the later.  We have
   reversed it.  Now the sig priority ordering is 1,2,.etc.  Updated
   sigordering unittests to reflect the same.
12 years ago
Anoop Saldanha 9219079e1a Allow protocols to have both app layer keywords, as well as transaction
based ones.

Our general logic and assumption is protocols either support one of the
above and not have both.
12 years ago
Anoop Saldanha a490176c8a More lock fixes for the transaction update. Issues reported by Coverity. 12 years ago
Anoop Saldanha 7cf4042337 Fix luajit compilation failure introduced by the transaction update.
Fix coverity lock issues reported by transaction update as well.
12 years ago
Anoop Saldanha d4d18e3136 Transaction engine redesigned.
Improved accuracy, improved performance.  Performance improvement
noticeable with http heavy traffic and ruleset.

A lot of other cosmetic changes carried out as well.  Wrappers introduced
for a lot of app layer functions.

Failing dce unittests disabled.  Will be reintroduced in the updated dce
engine.

Cross transaction matching taken care of.  FPs emanating from these
matches have now disappeared.  Double inspection of transactions taken
care of as well.
12 years ago
Anoop Saldanha 6dcde9d7e9 hsbd mpm and packet mpm share same mpm ctx id.
This is a bug emanating from we having a var reference for hsbd mpm,
but failing to initialize it, and we default to using the packet mpm.
12 years ago
Ken Steele 93e7304117 Preserve PKT_ALLOC flag inside PACKET_RECYCLE().
The PKT_ALLOC flag was being cleared by PACKET_RECYCLE(), which could
then result in a packet being pushed back to the Packet ring buffer
incorrectly.
12 years ago
Ken Steele 699d9e01f1 Move memset() out of PACKET_INITIALIZE()
The memset() inside PACKET_INITIALIZE() is redundant in some cases and
it is cleaner to do as part of the memory allocation. This simplifies
changes for integrating Tilera mPIPE support because the size of memory
cleared in that case is different from SIZE_OF_PACKET.

For the cases where Packets are directly allocated and then call
PACKET_INITIALIZE() without memset() first, this patch adds memset() calls.

A further change would use GetPacketFromAlloc() directly.
12 years ago
Victor Julien 724ad9e8e7 Detect L1 cache line size at build time. Fall back to 64 bytes if detection failed. 12 years ago
Victor Julien 53fe756798 NFQ: convert batchcount related yaml errors to warnings. 12 years ago
Eric Leblond 703e5848e4 nfq: add errno display when verdict fail
In case of error, errno is set by sendmsg which is called by
nfnetlink and which is called by libnetfilter_queue. This patch
displays the string expression of errno if verdict has failed.
12 years ago
Florian Westphal 8da02115c9 nfq: add support for batch verdicts
Normally, there is one verdict per packet, i.e., we receive a packet,
process it, and then tell the kernel what to do with that packet (eg.
DROP or ACCEPT).

recv(), packet id x
send verdict v, packet id x
recv(), packet id x+1
send verdict v, packet id x+1
[..]
recv(), packet id x+n
send verdict v, packet id x+n

An alternative is to process several packets from the queue, and then send
a batch-verdict.

recv(), packet id x
recv(), packet id x+1
[..]
recv(), packet id x+n
send batch verdict v, packet id x+n

A batch verdict affects all previous packets (packet_id <= x+n),
we thus only need to remember the last packet_id seen.

Caveats:
- can't modify payload
- verdict is applied to all packets
- nfmark (if set) will be set for all packets
- increases latency (packets remain queued by the kernel
  until batch verdict is sent).

To solve this, we only defer verdict for up to 20 packets and
send pending batch-verdict immediately if:
- no packets are currently queue
- current packet should be dropped
- current packet has different nfmark
- payload of packet was modified

This patch adds a configurable batch verdict support for workers runmode.
The batch verdicts are turned off by default.

Problem is that batch verdicts only work with kernels >= 3.1, i.e.
using newer libnetfilter_queue with an old kernel means non-working
suricata. So the functionnality has to be disabled by default.
12 years ago
Florian Westphal 6678c9feb9 nfq: avoid extra copy when running in workers mode
currently, the packet payload recv()d from the nfqueue netlink
socket is copied into a new packet buffer.

This is required because the recv-buffer space used is tied
to the current thread, but a packet may be handed off to other
threads, and the recv-buffer can be re-used while the packet
is handled by another thread.

However, in worker runmode, the packet will always be handled
by the current thread, and the recv-buffer will only be reused
after the entire packet processing stack is done with the packet.

Thus, in worker runmode, we can avoid the copy and assign
the packet data area directly.
12 years ago
Victor Julien b68d566c44 alert-debuglog: cleanup TCP check 13 years ago
Victor Julien 4b3166b193 unified2: more udp fixes 13 years ago
Victor Julien bc3f941acb profiling: enabled app layer profiling for UDP app layer modules 13 years ago
Victor Julien 782aa5adae prelude: only call stream callback for TCP 13 years ago
Victor Julien b54a19937f unified2: only call stream callback for TCP 13 years ago
Victor Julien 00948882e7 Suppress warnings when StreamSegmentForEach is called for UDP or SCTP, unless debug is compiled in. 13 years ago
Victor Julien 3b68a9d1c6 UDP: inspection app layer state as soon as we have it. 13 years ago
Victor Julien f15d97b916 Bug 780 unittests, showing no problem. 13 years ago
Victor Julien b6995f7664 Bug 794: stream SACK list needs to respect memcap 13 years ago
Victor Julien a4fca88ba7 stream: default 'random' setting when running unittests is disabled, so that test results are predictable. 13 years ago
Eric Leblond 9b235b3d9e streaming: randomize chunk size
By randomizing chunk size around the choosen value, it is possible
to escape some evasion technics that are using the fact they know
chunk size to split the attack at the correct place.
This patch activates randomization by default and set the random
interval to chunk size value +- 10%.
13 years ago
Victor Julien 6ba52230ed Update DetectContentDataParse to reflect the actual data types content uses. 13 years ago
Victor Julien 3ad497e74f Remove filemagic debug statement 13 years ago
Victor Julien 19511cda97 Remove obsolete DetectParseContentString function, it has been replaced by DetectContentDataParse 13 years ago
Victor Julien 4d4f8fd358 file: make fileext, filename and filemagic use the same rule parsing function as others. This has as a side effect that we enforce doubly qouted values now. 13 years ago
Victor Julien 8023007fbd flowvar: cleanup keyword argument parsing. Should also address Coverity 400655. 13 years ago
Victor Julien 07b751b0df Coverity 1005134: fix minor memory leak on flowvar rule setup errors. 13 years ago
Victor Julien e45f683c19 Coverity 1005133: fix unlikely case where malformed pcre statement in rule would lead to null-deref. 13 years ago
Victor Julien 4c6463f378 stream: handle extra different SYN/ACK
Until now, when processing the TCP 3 way handshake (3whs), retransmissions
of SYN/ACKs are silently accepted, unless they are different somehow. If
the SEQ or ACK values are different they are considered wrong and events
are set. The stream events rules will match on this.

In some cases, this is wrong. If the client missed the SYN/ACK, the server
may send a different one with a different SEQ. This commit deals with this.

As it is impossible to predict which one the client will accept, each is
added to a list. Then on receiving the final ACK from the 3whs, the list
is checked and the state is updated according to the queued SYN/ACK.
13 years ago
Victor Julien 00a691fc1b flowvar: clean up properly on signature clean up. 13 years ago
Victor Julien 70e2adeb01 flowvar: add unittests for #802. 13 years ago
Victor Julien 4cd736fcc9 flowvar: fix deadlock with http buffers
Bug #802

Flowvars are set from pcre, and lock the flow when being set. However
when HTTP buffers were inspected, flow was already locked: deadlock.

This patch introduces a post-match list in the detection engine thread
ctx, where store candidates are kept. Then a post-match function is used
to finalize the storing if the rule matches.

Solves the deadlock and brings the handling of flowvars more in line
with flowbits and flowints.
13 years ago
Victor Julien 4c2e6a8402 flowvars: update funcs to accept u16 id
All id's are u16, but flowvar functions would only accept u8.

Minor cleanups.
13 years ago
Victor Julien ffffe6c10e profiling: add formatted totals, percents to packet stats 13 years ago
Victor Julien 4165de4771 Minor SigValidate cleanup 13 years ago
Anoop Saldanha 0d7305dfc7 Update the way we handle http_host keywords.
Previously we would have forced all users to use nocase with http_host
keywords(since the hostname buffer is lowercase).

We now error out on sigs that has nocase set with http_host set.  Also if
the http_host pattern or http_host pcre has an uppercase character set, we
invalidate such sigs.  Unittests also updated to reflect the above change.
13 years ago
Victor Julien 9ea4d36f7a Minor reshuffling of Signature struct. 13 years ago
Victor Julien eb11280888 Use define instead of magic number for pmq's per detect thread 13 years ago
Victor Julien 0fa38c13d1 detection engine: consolidate thread setup
DetectEngineThreadCtxInit and DetectEngineThreadCtxInitForLiveRuleSwap did
pretty much the same thing, except for a counters registration. As can be
predicted with code duplication like this, things got out of sync. To make
sure this doesn't happen again, I created a helper function that does the
heavy lifting in this function.
13 years ago
Victor Julien 73158fea33 Fix PmqSetup calls in Liveswap thread init. Func was out of sync with normal thread init. 13 years ago
Victor Julien b8078742c3 stream: intro function for SYN/ACK state update
As the TCP SSN state can be updated from several points in the state
machine on accepting a SYN/ACK, move the update logic into a separate
function.
13 years ago
Victor Julien 28ea129d9b stream: remove unused 'pause' feature 13 years ago
Victor Julien ea8b6078d8 stream: zero ts is a per stream flag
Ssn flag STREAMTCP_FLAG_ZERO_TIMESTAMP was used in stream only. Due to
it's value it did not conflict with a real stream flag. Renamed it to
STREAMTCP_STREAM_FLAG_ZERO_TIMESTAMP.
13 years ago
Victor Julien 374187bf65 stream: don't use ssn timestamp flag in stream
The STREAMTCP_FLAG_TIMESTAMP flag is a ssn flag, however it was used in
the stream flag field. As it has the same value as
STREAMTCP_STREAM_FLAG_DEPTH_REACHED it's possible that stream reassembly
got confused by the timestamp.
13 years ago
Victor Julien 40a5ce8f5f Change logic of SCErrorToString causing any missing entries to result in a compiler warning. 13 years ago
Anoop Saldanha 71ffed5128 Handle the case of pcre combined with a relative content, where pcre has the
set to match from start of line and we discontinue matching on not finding
match.
13 years ago
Anoop Saldanha aa363a8144 unittest to display #784. 13 years ago
Eric Leblond 26b7af1483 Don't try to sniff 'default' interface
Whan running suricata via 'suricata --af-packet', the list of interfaces
was containing the 'default' interface and sniffing it was attempted.
This was not wanted.
13 years ago
Eric Leblond 539de3f5ea bpf filter: use SCLogError instead of fprintf 13 years ago
Eric Leblond b7e78d33b1 af-packet: warn about BPF filter consequence in IPS mode
This patch add a message to warn user about the impact of using a
BPF filter in IPS mode.
13 years ago
Eric Leblond dfbb31df8a Exit if bpf is used in IPS mode 13 years ago
Victor Julien ce99a07582 After some discussion we decided that var declarations inside a for statement are not in line with our coding style. So removing a bunch. Decision was not unanimous ^^. 13 years ago
Anoop Saldanha 8bf034e8c4 Live rule swap logs added to report SigLoadSignatures() failure. Also set
thread_closed flag on exit for live swap thread.
13 years ago
Anoop Saldanha a3212f6a0f Minor fixes against the last set of patches for #564, 565, 581 + fp automation.
Rename struct DetectFigureFPAndId_t_ to DetectFPAndItsId_ and move it's
definition from inside the function where it's used to the global namespace,
as requested on #suricata.

Rename DetectEngineContentModifiedBufferSetup to DetectEngineContentModifierBufferSetup.

Also rename DetectFigureFPAndId() to DetectSetFastPatternAndItsId().

Updated DetectSetFastPatternAndItsId() to not exit on failure and return error.
13 years ago
Anoop Saldanha 6de8b1ed53 fix for #564.
Get rid of the hash table, and use a single-one_time_alloc'ed array for
pattern id assignment.
13 years ago
Anoop Saldanha f58c6589b4 We now print content flags in engine fp analyzer. 13 years ago
Anoop Saldanha e77fd1c883 We now assign ids to fp patterns only. Rest of them don't need one. 13 years ago
Anoop Saldanha 4c6efa2d40 Update content id assignment.
All fp id assignment now happens in one go.
Also noticing a slight perf increase, probably emanating from improved cache
perf.
Removed irrelevant unittests as well.
13 years ago
Anoop Saldanha 60be1751d5 Figure out sig fp during validation stage, instead of staging stage. 13 years ago
Anoop Saldanha 45ff67a2e0 Enable a conf option to enable/disable legacy keywords.
Currently, uricontent is declared a legacy keyword, and is enabled by default.
13 years ago
Anoop Saldanha 601836d831 Fast pattern setup now configurable in our code.
You can either enable/disable fp for a particular type + set priority.
13 years ago
Anoop Saldanha c63317d02e Detect sm_list rearranged for performance reasons. 13 years ago
Anoop Saldanha f8ae53ac02 Further customize content modifier buffer registration.
Allow modifier setups functions to have CustomCallbacks to enable their
internal conditions.
13 years ago
Anoop Saldanha a304a98d1d http_* setup unified. 13 years ago
Anoop Saldanha 434bdca9e2 uricontent simplified to use the existing content + http_uri infrastructure. 13 years ago
Anoop Saldanha 0b5d277254 code cleanup for all content based keywords. 13 years ago
Anoop Saldanha 51dcf19817 turn dce_stub_data into a sticky buffer. 13 years ago
Anoop Saldanha a308d718ae Allow the use of relative without the presence of a related previous keyword. 13 years ago
Victor Julien 4845631335 tcp stream: don't move to LAST_ACK on toserver resent of FIN 13 years ago
Victor Julien 3163243a55 Coverity 989710 and 989711: small recourse leaks in filemd5 parsing code. 13 years ago
Anoop Saldanha 12e4105dc3 fix for #770.
Invalidate sigs with negative depth.
13 years ago
Anoop Saldanha d041b98d95 fix for #771.
Fix /etc/protocols parsing.  Remove trailing newspace stored under some cases.
13 years ago
Victor Julien 37c80ea508 If an IP-only pass rule matches, set the no inspect flag for that flow. Bug #718. 13 years ago
Anoop Saldanha 75130f9702 fix for #769.
Packet inserted by live swap flagged as pseudo packet.
13 years ago
Victor Julien 274641abc2 Fix valgrind error/warning in ip reputation parsing code 13 years ago
Anoop Saldanha c6ec23ca87 fix for #758. Add redmine wiki link and desc for icmp-id keyword. 13 years ago
Victor Julien eeb439c1a3 Open 2.0 dev branch 13 years ago
Victor Julien b66af2c2ed nfq: add missing error string 13 years ago
Eric Leblond 7ec820d3ab Fix potential Null deref. 13 years ago
Victor Julien 8924d7598d Fix potential iprep file parsing issue (2). 13 years ago
Victor Julien 754ae8a1be Fix potential iprep file parsing issue. 13 years ago
Victor Julien 1b363ecb1d Fix test AddressTestParse36 on Big Endian systems 13 years ago
Anoop Saldanha 0febe5a410 fix for #760.
If udpv4 csum isn't calculated, udpv4-csum detection shouldn't run on the
csum.
13 years ago
Anoop Saldanha ce7d78dd69 fix for #725.
Update trec_len, trec_pos to 32 bits from 16 bits.
Handle handshakes that are fragmented across records.
13 years ago
Anoop Saldanha c6d50764e5 temporarily patched smb + dcerpc parsers for direction demaraction. 13 years ago
Eric Leblond 5b067e1abb pcap-file: treat the case of unsupported pcap link
In unix socket mode, Suricata was stopping processing pcap files
when a pcap file with an unsupported datalink was treated. This
patch updates error handling to allow Suricata to treat other
pcap files.
13 years ago
Eric Leblond 350d761961 af-packet: leave reading loop at each turn
The idea of this patch is to be sure to leave the ring reading loop
enough to be able to sync counters. This should fix #706.
13 years ago
Eric Leblond df0e7af8f2 unix-manager: fix thread killing function
The name of the thread was not searched in the correct family.

Reported-by: iswalker <mail2cissp@gmail.com>
13 years ago
Eric Leblond 31c03d38b9 unix socket: add 'dump-counters' command
This patch adds a 'dump-counters' command which answer an output of
all performance counter.
13 years ago
Eric Leblond 5722d8846a unix socket: add 'help' as alias to 'command-list' 13 years ago
Eric Leblond 84322fa556 unix socket: add 'conf-get' command
This patch adds a 'conf-get' command which get the configuration
value from suricata. Argument of the command is the name of the
variable to fetch.
The command syntax is the following:
{
 "command": "conf-get",
 "arguments": { "variable":value}
}
13 years ago
Eric Leblond c961056ed8 unix socket: add 'capture-mode' command
This patch displays what capture mode is used.
13 years ago
Eric Leblond 74a9fc4b66 Add function to display current capture mode
This patch adds a function to display the capture mode.
13 years ago
Eric Leblond 2f30485f7b unix socket: add 'runnning-mode' command
This command displays the active running mode ('autofp' for
example).
13 years ago
Eric Leblond f4faff6ff9 unix socket: add 'uptime' command
This command displays the nuber of second since the start of
Suricata.
13 years ago
Eric Leblond c6b38ebf67 unix socket: add 'version' command 13 years ago
Eric Leblond 78b5812ae6 unix runmode: add 'pcap-current' command
This command outputs the currently processed file name or 'None'
if no file is currently processed.
13 years ago
Eric Leblond fc7e6c4a3d unix socket: implement command-list command 13 years ago
Eric Leblond 346d5662b5 cuda: fix invalid use of sizeof 13 years ago
Anoop Saldanha 71609229cc sigorder cleaned up. 13 years ago
Eric Leblond 21dda8674d Fix build with old pcap library.
Pcap snaplen related modification broke compilation of Suricata for
system having old pcap library. This patch fixes the issue and allow
old pcap library to honour the snaplen value.
13 years ago
Eric Leblond 6d225378e4 Workaround function missing in libhtp include
As reported in bug #688, htp_config_set_path_decode_u_encoding
function is not included in libhtp header before 0.3.0. Result
is that suricata compilation fail with an external htp library.
The following patch detect the issue and adds the missing
declaration.
13 years ago
Anoop Saldanha 66f3c37016 code cleanup + unittests added against http_host and http_raw_host keywords,
against various combinations of hostname in uri and host header.
13 years ago
Anoop Saldanha 3511f91bba Add support for the new keyword - http_raw_host header.
The corresponding pcre modifier would be 'Z'.
13 years ago
Anoop Saldanha c4ce19a1be Add support for a new keyword to inspect http_host header.
The corresponding content keyword would now be - http_host.
The corresponding pcre modifier would be W.
13 years ago
Matt Keeler ebccb9ffcd Added host buffer allowance and stream configuration for Napatech 3GD
Added a napatech section in the yaml configuration.
	hba - host buffer allowance
	use-all-streams - whether all streams should be used
	streams - list of stream numbers to use when use-all-streams is no

The source-napatech.* files were modified to support the host buffer allowance configuration.
The runmode-napatech.c file was modified to support both the host buffer allowance configuration and stream configuration

Signed-off-by: Matt Keeler <mk@npulsetech.com>
13 years ago
Anoop Saldanha 0c24a8a92f fix(more like a feature update) for bug #708.
Add support for flowint based sig ordering.
13 years ago
Eric Leblond 2f0927fe9b pcap: add snaplen YAML variable
This patch introduces 'snaplen' a new YAML variable in the pcap section.
It can be set per-interface to force pcap capture snaplen. If not set
it defaults to interface MTU if MTU can be known via a ioctl call and to
full capture if not.
13 years ago
Eric Leblond e14a817fbd pfring: delete unused define. 13 years ago
Eric Leblond 786cbb1244 log-pcap: don't limit snaplen. 13 years ago
Eric Leblond e8aa66a44c pcap: add 'promisc' YAML configuration variable
This patch adds a promisc variable to pcap configuration. It is
used to decided if interface is switched to promiscuous mode.
13 years ago
Eric Leblond 1aaa828b63 pcap: set snaplen to MTU if available.
Main objective of this patch is to use a dynamic snaplen to avoid
to truncate packet at the currently fixed snaplen.

It set snaplen to MTU length if the MTU can be retrieved. If not, it
does not set the snaplen which results in using a 65535 snaplen.

libpcap is trying to use mmaped capture and setup the ring by using buffer_size
as the total memory. It also use "rounded" snaplen as frame size. So if we set
snaplen to MTU when available we are optimal regarding the building of the ring.
13 years ago
Victor Julien cc51eec59d Use new libhtp query string normalization. Bug #739. 13 years ago
Eric Leblond 2732faf05c teredo: update protocol decoding.
This patch fixes an error in pointer arythmetic and add some
comments to increase maintanability of the code. It also
simplify the decoding code as a careful RFC reading indicate
that if we discard packet containing an authentication field,
it is only possible to have a single origin indication field.
13 years ago
Eric Leblond 8d7b9703af Fix latest build-info modification
The creation of build-info.h should have been made in build
directory and not in source directory. This should fix changes
introduced in #738.
13 years ago
Eric Leblond 84f50ba49f build-info: use printf instead of SCLogInfo
This change results in a more readable and reusable output.
13 years ago
Eric Leblond 668113af77 add configure summary to build-info output 13 years ago
Eric Leblond f5ba8eb6db suricata: add information to build-info
This patch adds information about luajit and jansson to the
output of --build-info command. This should fix #696.
13 years ago
Anoop Saldanha 5fe9394d07 bug #737. Display a more apt error message when wrong argument's supplied to
reference keyword.
13 years ago
Jake Gionet 1ac8938787 Adding support for Feature #667 13 years ago
Victor Julien d0c1410cf5 Fix sig grouping bug when certain sigs are mixed. Add tests. 13 years ago
Victor Julien afb2d4eddf Fix stateful inspection not always inspecting at stream end. 13 years ago
Anoop Saldanha f59ce70c17 fix for #694.
Invalidate any address/port vars in the conf that uses a sequence
without quotes.
13 years ago
Anoop Saldanha 51868f17ae unittest to show the seg fault from bug_694 13 years ago
Anoop Saldanha 34a9c047fc updated to fix unix shutdown sequence
Should fix crashes occuring from unix mode shutdown/cleanup phase.
13 years ago
Ignacio Sanchez d771e08156 Adds support for the geoip keyword
Adds support for match-on conditions (src, dst, any, both)
Uses GEOIP_MEMORY_CACHE for performance reasons
Adds support for negation and multiple countries in the same rule

Bug fixes

Changed to take flow direction from rule, if present

Comments addressed. Unit tests added.
13 years ago
Eric Leblond 6dfd106139 conf: add unittest for WithDefault functions. 13 years ago
Eric Leblond f59c63c457 pcap: add support for 'default' interface 13 years ago
Eric Leblond feabe6e9a2 pfring: add support for 'default' interface 13 years ago
Eric Leblond 4ae27756b0 af-packet: add support for 'default' interface
This patch adds support for 'default' interface which is used to get
parameter values when per-interface is not defined.
13 years ago
Eric Leblond 0bddf4f02f conf: introduce WithDefault function
This patch introduces a new set of functions to the ConfGetChildValue
family. They permit to look under a default node if looking under
base node as failed. This will be used to access to default parameters
for a data type (for instance, first usage will be interface).
13 years ago
Eric Leblond 6b81430bcb pcap-file: don't kill engine in unix socket mode
This patch updates the cleaning code to avoid to exit from suricata
in unix socket mode when a invalid pcap is given.
13 years ago
Jamie Strandboge bc04090bc9 suppress: DETECT_SUPPRESS_REGEX should support IPv6 addresses too. Bug #697. 13 years ago
Victor Julien 80ed1ba008 file md5: print filename and line number on md5 parse errors. Bug #693. 13 years ago
Nikolay Denev 9480559c65 preserve the existing error code order
restore SC_WARN_IPFW_SETSOCKOPT
move SC_ERR_IPFW_SETSOCKOPT at the end of the enum
13 years ago
Nikolay Denev 894ad21be5 setsockopt() failures are already fatal,
so treat them as such and print error instead of warning.
13 years ago
Nikolay Denev 29b69fb026 set SO_BROADCAST on the divert socket so that broadcast
packets can be reinjected.
13 years ago
Victor Julien 6783463eee Fix ftpbounce address calc failing on PPC64 13 years ago
Victor Julien 0c84a7a2a9 Use _mm_free for memory allocated by _mm_alloc. Bug 703. Minor compiler warning fixes. 13 years ago
Victor Julien 34d063adea Fix double definition of CPU_* macro's for Darwin/OSX. Bug 701. 13 years ago
Victor Julien f0578c474e Fix byte order detection on Mac OS X/Darwin. Bug 700. 13 years ago
Victor Julien 5f4c52801e Fix protocol check for IP-only (#689). 13 years ago
Victor Julien 1eed3f2233 ipv6: add event for ipv6 packet with icmpv4 header 13 years ago
Anoop Saldanha 53c023342c fix for 653.
break out of afp readring loop if shutdown is initiated.
13 years ago
Victor Julien a55ff64a1b Use GET_PKT_LEN and GET_PKT_DATA macro's 13 years ago
Eric Leblond e690b3bbc9 magic: freebsd magic return differently
FreeBSD don't return "Microsoft Office Document" but
"OLE 2 Compound Document". This patch takes this into account.
13 years ago
Anoop Saldanha a30a1e5950 fix for bug 675.
Fix icmpv6-csum to send the right length to calculate the csum.
13 years ago
Anoop Saldanha af92c2fa4b Unittest to show the issue we have with 674 - csum-icmpv6 sends
wrong length for csum calculation)
13 years ago
Victor Julien 150b0c5ae0 ipv6: add option to detect HOP/DST headers with only padding. Detect unknown DST/HOP opts. 13 years ago
Victor Julien ba367dad3c icmpv6: fix payload handling 13 years ago
Victor Julien 538a941486 decoder events: fix bug causing some rules not to be inspected if the decoder completed with warnings 13 years ago
Victor Julien f5cd7c6a92 decode events: add debug statement 13 years ago
Victor Julien 82769a1b37 profiling: fix missing profile names 13 years ago
Victor Julien 72443a0d62 unified2: append open instead of trucate open so that in case we rotate within a second we don't overwrite files. Instead we violate the limit. 13 years ago
Victor Julien 298d21372b flow: only BUG_ON use_cnt in flows when compiled with debug-validation 13 years ago
Anoop Saldanha b22a0cffbb cleanup flowtimeout threadvars retrieval +
throw back pseudo pkt back to packetpool inside flow timeout.
13 years ago
Victor Julien abecef5d82 stream: send eof to app layer from stream end pkt if necessary 13 years ago
Ludovico Cavedon ac8b087717 Wait until both sides close the TCP connection before initiating cleanup 13 years ago
Eric Leblond 2accda78a1 unix runmode: fix error handling.
If 'output-dir' argument was not given it was possible to reach a
possibly problematic condition.
13 years ago
Eric Leblond 1fd47cfb96 Remove useless code. 13 years ago
Eric Leblond b3d4285982 fix logic error in sanity check 13 years ago
Eric Leblond 9c47ada771 Add removal safe TAILQ iterator.
TAILQ_FOREACH macro was not safe for element removal as it was
accessing the next element in case of a free. This patch is inspired
by Linux list handling and provide a new macro TAILQ_FOREACH_SAFE.
This macro is removal safe and only differs by a last argument being
a temporaty pointer to an element.
13 years ago
Eric Leblond 06751ecd75 prelude: don't build string objet for NULL string
prelude_string_set_ref don't like when it is called with a NULL
parameter. This patch adds check for NULL value. This is formally
good as there is no use of a NULL description.
13 years ago
Jason Ish 005f7a2399 Feature 638: Display DAG drop counts on exit; add DAG packet and drop stats to live stats. 13 years ago
Ludovico Cavedon b617c9c3f2 Fix length check on user-agent header 13 years ago
Ludovico Cavedon 5dd0a1d917 Add User-Agent header content to file metadata 13 years ago
Anoop Saldanha 34d5aadcb8 warn users that we don't support content strings whose length's > 255. 13 years ago
Ludovico Cavedon 2f4c9198a6 Initialize flow_manager_mutex 13 years ago
Anoop Saldanha 464ed95f71 fix for bug #526.
Insert pseudo packet under low load conditions to complete rule swap.
This is necessary when we use autofp active packets where most packets
would be sent to the first queue under low load conditions.
13 years ago
Victor Julien 389c48f222 Fix detection of spin locks supported. Clean up how we handle falling back to mutex if spinlocks aren't supported. 13 years ago
Eric Leblond df3d10865a host: suppress double memory clear
HostFree() is calling HostClearMemory() so calling HostClearMemory()
before HostFree() is useless.
13 years ago
Eric Leblond 12fd60b545 unix-socket: cleanup host table instead of destroying it
This patch should fix the bug #637. Between pcap files, it uses a
new function HostCleanup() to clear tag and threshold on host with
an IP regputation. An other consequence of this modification is
that Host init and shutdown are now init and shutdown unconditionaly.
13 years ago
Eric Leblond d9eaa0d340 host: don't destroy reference counter
The reference counter should not be destroyed in HostClearMemory()
as the host can be reused directly (without going through Init
function).
13 years ago
Eric Leblond ca1a70a04b pfring: fix build failure 13 years ago
Anoop Saldanha b1ce94babe Temporary fix for bug #599.
Treat sigs with negated addresses as non ip-only.

This fix exposes bug #608, which results in 2 failed unittest which
have now been disabled by this commit.  Would be reenabled when we
have #608 fix in.
13 years ago
Anoop Saldanha fdc666f732 unittest to show failure for bug #599. 13 years ago
Victor Julien 9f519e95a2 http: add event for libhtp detection of request port not matching tcp port. 13 years ago
Victor Julien 3ab1458abf pcap: fix windows commandline mangling win device string 13 years ago
Victor Julien a698a7600d clang: fix warnings when debug is enabled 13 years ago
Victor Julien 40bbf96f22 reputation: don't give error if config is missing/commented out 13 years ago
Victor Julien 0f42f0e890 Minor fixes 13 years ago
Eric Leblond 6b3ebc810d unix runmode: improve JSON handling
The jansson function with new in their name take care of ref
counting. The this patch fixes a memory leak.
13 years ago
Eric Leblond 195b144daa unix-manager: fix error and JSON handling 13 years ago
Eric Leblond a05113a2b1 unix-manager: memory handling fixes.
This patch adds unlikey() for memory error handling and fixes a few
error cases.
13 years ago
Eric Leblond 028a37f6e7 unix runmode: use unlikely for memory error 13 years ago
Eric Leblond 547c55114e unix runmode: fix FIXME 13 years ago
Eric Leblond f38b8fe4eb unix runmode: fix JSON mem handling
json_decref was not correctly used through the code. This patch
fixes it.
13 years ago
Eric Leblond 13237b8af2 unix manager: add static 13 years ago
Eric Leblond 936c36d5f1 Disable 'reload-rules' command. 13 years ago
Eric Leblond d5457ad70e unix-manager: doc and whitespace fixes 13 years ago
Eric Leblond af16c418b7 unix-socket: fix build when jansson not present 13 years ago
Eric Leblond ef64648cf8 unix-command: add drop counter to iface-stat message 13 years ago
Eric Leblond 8d0260b27e Add atomic counter for iface drop. 13 years ago
Eric Leblond cc71c993f4 unix-command: add iface information command.
This patch adds two commands to unix-command. 'iface-list' displays
the list of interface which are sniffed by Suricata and 'iface-stat'
display the available statistics for a single interface. For now,
this is the number of packets and the number of invalid checksums.
13 years ago
Eric Leblond c78e112e3e af-packet: update runmode copyright date. 13 years ago
Eric Leblond 6f0a851087 unix-manager: fix error treatment in accept phase 13 years ago
Eric Leblond f2a17f47d3 unix-manager: implement multi client support
This patch implements the support of multiple clients connected
at once to the unix socket.
13 years ago
Eric Leblond a9cb8ce89f affinity: avoid to init structure twice
In unix socket mode, suricata was doing multiple init of the
structure. This was not needed and caused a memory leak in
mutex creation.
13 years ago
Eric Leblond 93f801b3a9 pcap-file: update affinity setting code
The affinity setting code was using the old API. This patch updates
to the new API and also adds a call to RunModeInitiaze() which was
missing in Single running mode.
13 years ago
Eric Leblond cfd80e7063 unix-mode: fix return of pcap-file command 13 years ago
Eric Leblond f8921d8a28 unix-socket: introduce API to add commands and tasks
This patch transforms the unix socket into a flexible system to
add commands (triggered by user) and taks (run periodically).
It introduces two functions UnixManagerRegisterCommand and
UnixManagerRegisterBackroundTask to registed commands and tasks.

Other part of Suricata can then declare a new command via a simple
call of the function. In the case of a command the caller is
responsible of building the answer message using Jansson API. The
sending of the message is made by unix manager code.
13 years ago
Eric Leblond 20a8b9dbe5 unix-manager: add unix command socket and associated script
This patch introduces a unix command socket. JSON formatted messages
can be exchanged between suricata and a program connecting to a
dedicated socket.
The protocol is the following:
 * Client connects to the socket
 * It sends a version message: { "version": "$VERSION_ID" }
 * Server answers with { "return": "OK|NOK" }
If server returns OK, the client is now allowed to send command.

The format of command is the following:
 {
   "command": "pcap-file",
   "arguments": { "filename": "smtp-clean.pcap", "output-dir": "/tmp/out" }
 }
The server will try to execute the "command" specified with the
(optional) provided "arguments".
The answer by server is the following:
 {
   "return": "OK|NOK",
   "message": JSON_OBJECT or information string
 }

A simple script is provided and is available under scripts/suricatasc. It
is not intended to be enterprise-grade tool but it is more a proof of
concept/example code.  The first command line argument of suricatasc is
used to specify the socket to connect to.

Configuration of the feature is made in the YAML under the 'unix-command'
section:
  unix-command:
    enabled: yes
    filename: custom.socket
The path specified in 'filename' is not absolute and is relative to the
state directory.

A new running mode called 'unix-socket' is also added.
When starting in this mode, only a unix socket manager
is started. When it receives a 'pcap-file' command, the manager
start a 'pcap-file' running mode which does not really leave at
the end of file but simply exit. The manager is then able to start
a new running mode with a new file.

To start this mode, Suricata must be started with the --unix-socket
 option which has an optional argument which fix the file name of the
socket. The path is not absolute and is relative to the state directory.

THe 'pcap-file' command adds a file to the list of files to treat.
For each pcap file, a pcap file running mode is started and the output
directory is changed to what specified in the command. The running
mode specified in the 'runmode' YAML setting is used to select which
running mode must be use for the pcap file treatment.

This requires modification in suricata.c file where initialisation code
is now conditional to the fact 'unix-socket' mode is not used.

Two other commands exists to get info on the remaining tasks:
 * pcap-file-number: return the number of files in the waiting queue
 * pcap-file-list: return the list of waiting files
'pcap-file-list' returns a structured object as message. The
structure is the following:
 {
  'count': 2,
  'files': ['file1.pcap', 'file2.pcap']
 }
13 years ago
Eric Leblond 6be63bdc4f tm-threads: add TM_ECODE_DONE state
This patch adds a nex return state which can be used by threads
to warn that a task has been done. In this case, suricata does not
leave.
13 years ago
Eric Leblond 412482f6b1 filestore: create file store directory if needed
This patch modifies the file store system to have it create the
file store directory if needed. It dos not create the full
directory tree as the parent directory must have already been
created.
13 years ago
Eric Leblond 7b1d346c22 counters: management cpu set was set twice
Setting the management CPU set on perf threads is already done in
the TmThreadCreateMgmtThread() function used to create the threads.
13 years ago
Eric Leblond 84f2645e3e pcap-file: free thread var at deinit. 13 years ago
Eric Leblond 28b4bed141 tm-threads: fix potential access to NULL pointer. 13 years ago
Eric Leblond 1b26660ac4 counter: defensive set to NULL in free. 13 years ago
Eric Leblond 09b79cb5bf stream-tcp: fix double call to debug print function 13 years ago
Last G 8ae11f73b2 Added parentheses to fix Eclipse static code analysis
Fixed bug in action priority (REJECT_DST had lowest prio)
13 years ago
Last G e236351c52 Fixed missing "|" in "||" operation 13 years ago
Last G edcb8fdb87 Added parenthesis for right operation order 13 years ago
Last G 8bb9c3af35 Added return value to non-void function with "forever"-loop to fit
Eclipse static code analysis
13 years ago
Eric Leblond 40891223e9 list-keyword: detect non built keyword
This patch update the glafs list to be able to indicate that a
flag is not supported. This information is used by list-keyword to
display information to the user.
13 years ago
Eric Leblond 8f13694988 luajit: no link with HTTP when not build.
Even when not built-in, luajit is not linked with HTTP.
13 years ago
Eric Leblond 6842545331 Add documentation url in list-keyword output.
The output of the list-keyword is modified to include the url to
the keyword documentation when this is available. All documented
keywords should have their link set.

list-keyword can be used with an optional value:
 no option or short: display list of keywords
 csv: display a csv output on info an all keywords
 all: display a human readable output of keywords info
 $KWD: display the info about one keyword.
13 years ago
Eric Leblond fa900a9f6b suricata: add information about BPF filter usage 13 years ago
Eric Leblond 7e14fe62f5 suricata: add '-V' info to usage message. 13 years ago
Eric Leblond fd3a1346e4 suricata: add build-info command to usage message. 13 years ago
Eric Leblond 4e0f5b7f02 suricata: don't display msg in list-keyword mode.
In list-keywords and list-app-layer mode, suricata now only
displays the messages linked with the feature. This allow users
to redirect the output and easily work on it. For exemple, the
csv output will be easily imported into a spreadsheet.
13 years ago
Eric Leblond 5e4552fdcd suricata: update list-keyword command
This patch update the list-keyword command. Without any option,
the previous behavior is conserved. If 'all' is used as option,
suricata print a csv formatted output of keyword information:
	name;features;description
If a keyword name is used as argument, suricata print a readable
message:
tls.subject
Features: state inspecting
Description: Match TLS/SSL certificate Subject field
13 years ago
Eric Leblond 86709f5e9d rule analyser: display message for invalid signatures 13 years ago
Eric Leblond c7cfbb71c9 engine-analyzer: fix typo in message 13 years ago
Eric Leblond cd42e6a3ef Listing of app layers does not depend on unittests 13 years ago
Eric Leblond 42ace54137 list-keywords: fix when not using default install
As we don't parse the YAML file when listing of keywords is asked,
suricata make a test on existence of the build-default directory.
So with a non standard (working) install (even a single configure
without option lead to a failure), the keyword listing fails
because the default logging directory does not exist.
13 years ago
Eric Leblond b0471fb8e4 rule analyser: add msg if rule is ipv4 or ipv6 only 13 years ago
Victor Julien 83bfe3810b reputation: report error if host table memcap reached. Work around compilation failure with atomic fallback code. 13 years ago
Victor Julien 18535e6ef9 Host: ignore usecnt add/sub result. Expose HostPrintStats. 13 years ago
Victor Julien e30b1bfe64 Simple IP reputation implementation 13 years ago
Victor Julien 9140aa6ac5 cygwin supports the thread cpu affinity code now 13 years ago
Victor Julien b20bfa04ef clang warning squashing 13 years ago
Victor Julien 84bad6db77 Silence compiler warnings found by clang 13 years ago
Victor Julien b63c2eda6a build: more cygwin cleanups 13 years ago
Victor Julien dc465b92e5 Fix use of byte swap function 13 years ago
Victor Julien 506c144c60 build: reshuffle including headers to fix build on cygwin 13 years ago
Anoop Saldanha e1cabae0f4 fix uninit var usage in hhd 13 years ago
Eric Leblond 4726e02afb logging: add warning if no output module is selected
If no daemon compatible logging module is selected, a message is
displayed to avoid the user to look like mad for messages.
13 years ago
Eric Leblond 9f4da93a4b suricata: don't exit if pidfile can't be created 13 years ago
Eric Leblond e148b2b82a suricata: display PID file name in case of error. 13 years ago
Victor Julien 93bdaa49d8 byte_jump: when from_beginning option is used, the number of bytes to convert should not be used in the jump. Bug 627. 13 years ago
Eric Leblond 7854c84972 pcap: add capture counters in stats.log.
This patch adds three counters to stats.log:
    capture.kernel_packets    | RxPcapwlan0               | 4218
    capture.kernel_drops      | RxPcapwlan0               | 0
    capture.kernel_ifdrops    | RxPcapwlan0               | 0
This patch meant to fix bug #625.
13 years ago
Victor Julien bcaec1e963 pkt-data: don't compile unittest unless unittests are enabled 13 years ago
Victor Julien 472e061c6d build: more checking for includes 13 years ago
Victor Julien 2a42f554b1 build cleanup, build source files in alphabetical order 13 years ago
Victor Julien 042d0c6ee8 build cleanups 13 years ago
Victor Julien 5a6c8c0f01 minor misc changes: update htp ver, add htp ver to --build-info, clean up 13 years ago
Xavier Lange 234922f3c6 Keyword pkt_data 13 years ago
Eric Leblond b9a2f91a76 napatech: treat malloc error 13 years ago
Eric Leblond a1d1abfc05 suricata: add daemon-directory config variable
It is now possible to use the 'daemon-directory' configuration
variable to specify the working directory of suricata in daemon
mode. This will permit to specify the place for core and other
related files.
13 years ago
Eric Leblond 3061452c5e suricata: avoid concurrent run in daemon mode
This patch creates a pid file per default and use it to avoid to be
able to run two Suricata. Separate pid file have to be provided to
be able to do it.
13 years ago
Eric Leblond 24d10de8af suricata: change dir to / in daemon mode.
By changing directory to /, we will not block the directory where
suricata has been started.
13 years ago
Matt Keeler 37e3de8425 Refactor Napatech 3GD to just Napatech as Suricata is only going to support 3GD.
Signed-off-by: Matt Keeler <mk@npulsetech.com>
13 years ago
Matt Keeler 5786a32d0f Remove Napatech 2GD support
Removed the Napatech 2GD support

runmode-napatech-3gd.c had an include from runmode-napatech.h which was erroneous and has been removed as well.

Signed-off-by: Matt Keeler <mk@npulsetech.com>
13 years ago
Victor Julien 57d7783402 Remove unnecessary debug message 13 years ago
Victor Julien 829238e49c OpenBSD 5.2 build fixes, Unit test fix. 13 years ago
Eric Leblond fc9e0df33b suricata: add run-as.user and run-as.group yaml var
This patch update the YAML to be able to specify the user or the
group to run Suricata as:
 run-as:
   user: suri
   group: suri
13 years ago
Eric Leblond 961eda2108 pcap: ref config according to threads count 13 years ago
Victor Julien b645425331 Silence compiler warning if napatech3 support is disabled 13 years ago
Matt Keeler 844e4dba11 Napatech 3GD Support
For use with Network Cards from Napatech utilizing the 3GD driver/api.

    - Implemented new run modes in runmode-napatech-3gd.*
    - Implemented capture/decode threads in source-napatech-3gd.*
    - Integrated the new run modes and source into the build infrastructure.

    New configure switches
    --enabled-napatech-3gd : Turns on the NT 3GD support
    --with-napatech-3gd-includes : The directory containing the NT 3GD header files
    --with-napatech-3gd-libraries : The directory containing the NT 3GD libraries to link against.

    New CLI switch
    --napatech-3gd : Uses the Napatech 3GD run mode

    Runmodes Supported:
    - auto
    - autofp
    - workers

    Notes:
    - tested with 1 Gbps sustained traffic (no drops)

Signed-off-by: Matt Keeler <mk@npulsetech.com>
13 years ago
Anoop Saldanha b8164b8797 fix wrong record hdr len check in ssl parser 13 years ago
Victor Julien d1573a366d Fix GetUsed functions for Host, Flow and Defrag. 13 years ago
Eric Leblond 4542cd0eec ipfw: suppress non loop receive function 13 years ago
Eric Leblond e3a38810b6 nfq: suppress non loop receive function 13 years ago
Victor Julien 966c731e73 flow: fix crash when flow engine under extreme stress, and unable to force free any existing flow 13 years ago
Victor Julien da7f1d22cc http: don't assume http tx to have header alloc'd. Can happen in OOM conditions. Bug #587. 13 years ago
Victor Julien 18ecd4b287 Don't use SCStrdup in SCLogMessage as we call it on OOM condition, leading to endless recursion. SCStrdup failure calling SCLogMessage... 13 years ago
Victor Julien 70bc9e2494 filestore: fix logic flag in continued stateful detection 13 years ago
Eric Leblond 8957113550 pf-ring: fix build 13 years ago
Victor Julien d386606b80 Remove pcre jit warning. Bug #579. 13 years ago
Eric Leblond d3195b0f70 pf_ring: don't set cluster for DNA interface. 13 years ago
Anoop Saldanha 7a7cd6999e feature #558.
Print FP info in rule analysis + other cleanup.
13 years ago
Victor Julien a3f963f630 filestore: fix a case where a matching non-filestore sig could trigger the store of a partially matching filestore sig. 13 years ago
Victor Julien 3156407746 http: fix client and server body sometimes being inspected in wrong order 13 years ago
Eric Leblond b12967534a stream.inline: add 'auto' mode
stream.inline YAML configuration variable now support the 'auto' value.
In this case, inline mode is activated for IPS running mode (NFQ and
IPFW) and is deactivated for IDS mode. This patch should fix bug #592.
13 years ago
Eric Leblond b26ec60398 af-packet: fix possible infinite loop.
If no packet arrives to a capture thread, it is possible that the
AFPReadLoop() function goes into an infinite loop. This could cause
suricata to hang at exit on non busy system.
This patch adds a counter to detect when Suricata start looping in
the ring to stop when it reaches this point.
13 years ago
Eric Leblond e8a4a4c47c af-packet: dump counter every seconds.
This patch updates to kernel counters handling to be almost sure to
update at least once per second.
13 years ago
Eric Leblond 3acdd4da1d pf-ring: add counter for kernel drop and packets
This patch adds a counter for kernel drop and packets by using the
same strategy as the one used in af-packet.
13 years ago
Victor Julien 80d62b59ec Fix drop (and other actions) not being applied to thresholded packets. Bug #613. 13 years ago
Anoop Saldanha bca1b7c52a change default mpm to ac. Also default sgh-mpm-context is full. 13 years ago
Victor Julien fd6df00684 Bug 585: use per detect thread libmagic ctx 13 years ago
Victor Julien ea6fcb355b magic: add test showing payload resulting in libmagic invalid read as reported by valgrind. 13 years ago
Anoop Saldanha fdab6f2ab1 fix flow deadlock issue in detection engine state introduced by tx api.
Issue discovered by coverity.
13 years ago
Eric Leblond 00b95c69c0 suricata: list-keywords does not depend on unittest 13 years ago
Victor Julien 83ffd1f743 luajit: suppress compiler warning 13 years ago
Anoop Saldanha 2ab62920aa fix segv in hcbd and hsbd buffering.
Increase bufffers_list_len, only we open up a space for a new tx.
13 years ago
Anoop Saldanha b359bc03a9 unittest to reveal a bug/segv in our hsbd buffering code. 13 years ago
Victor Julien 4fab8ea6d6 http: fix http header reassembly bug causing some headers to be left out of the inspected buffer 13 years ago
Victor Julien 5cd46433d3 http: now that htp_state has a cfg reference, use it for body limits 13 years ago
Victor Julien 2763a61213 http: allow configuration of request and response body inspection limits. Issue #560. 13 years ago
Anoop Saldanha b99f9fe890 New app inspection engine introduced. Moved existing inspecting engines to use it. 13 years ago
Anoop Saldanha 7b4eac3e8d Change all inspect callbacks to accept TV and a tx_id param. 13 years ago
Anoop Saldanha 10a6e6a3eb Engine cleanup. Remove all old engine inspection and mpm functions. 13 years ago
Anoop Saldanha b0e20a486c update client/server/http_header to use a different form of
buffering/buffer_retrieval.

Now it happens per tx, based on tx id.  Also notice a perf improvement with
this.
13 years ago
Victor Julien e1321f9ae6 stream: change how retransmissions are handled and detected. 13 years ago
Victor Julien b621ed8423 stream: fix retransmission on closewait being considered out of window 13 years ago
Victor Julien a25629b250 stream: detect retransmissions on timewait state 13 years ago
Victor Julien 6326390120 stream: accept ack with next_seq + 1 on last_ack state 13 years ago
Victor Julien bc37cb6b8e stream: detect retransmissions on closewait and finwait2 states 13 years ago
Victor Julien 305ed3f23b stream: don't flag zero window probe packets as out of window. Bug #604. 13 years ago
Victor Julien 13e60c0040 stream: detect keep-alive packets so we don't consider those invalid 13 years ago
Victor Julien 9094eb4783 stream: ignore ack value if ack flag is not set. Add stream.pkt_broken_ack event for when ack value is not 0 and ack flag not set. 13 years ago
Victor Julien a5d9442c2d stream: handle retransmission of lost data packet on TIME_WAIT state 13 years ago
Victor Julien 037d67cc66 stream: go from FIN_WAIT_1 to CLOSING on simultaneous close. 13 years ago
Victor Julien 6544475670 stream: don't reject RST as response to SYN because of ACK 13 years ago
Victor Julien 6f76ac176d stream: add option to match on overlapping data
Set event on overlapping data segments that have different data.

Add stream-events option stream-event:reassembly_overlap_different_data and
add an example rule.

Issue 603.
13 years ago
Victor Julien d68fd54a76 Fix/suppress a couple of harmless compiler warnings. 13 years ago
Anoop Saldanha 870a98b528 Remove dead comment about flow reference api duplicate 13 years ago
Anoop Saldanha f08497d1e4 Move Flow Reference/Dereferene api from flow-util.h to flow.h.
Remove duplicate FlowDeReference from decode.h
13 years ago
Anoop Saldanha 67981d1c5c Update suricata to use FlowReference/FlowDeReference for the ones left out
from last update.
13 years ago
Victor Julien 72782e5a6a profiling: fix rule profiling output sometimes missing sid,rev,gid. Bug #576. 13 years ago
Victor Julien 10a11b750d Add dsize check to prefilter stage
Many sigs with dsize have a weak fast_pattern. Those patterns
are likely to match. By filtering on dsize early, we safe a lot
of cycles later.
13 years ago
Victor Julien 45cbef0735 For signatures with the dsize option set depth on any content match in that sig. 13 years ago
Victor Julien 4464657ca2 remove reference to non-existing file from Makefile.am 13 years ago
Victor Julien a01130d2ed packet src: move pkt_src field up in the structure to fix in an existing hole (found with pahole -C Packet_ src/.libs/suricata). 13 years ago
Anoop Saldanha b33986c887 Add a packet src for every packet generated inside suricata. 13 years ago
Eric Leblond 19756488ab nfq: close the queue when leaving acquisition.
This patch adds a call to close the queue when the acquisition
loop is ending. This way the incoming packets will be accepted
during all the shutdown phase (if the queue-bypass option of
NFQUEUE is used). At the same time the currently processed packets
will be dropped but the time scale are different: suricata will
drop 20 ms of packets and the shutdown can take 0.5 seconds.

Patch based on an idea of Victor Julien.
13 years ago
Victor Julien 75cddabd8a fast_pattern: don't consider http_method, http_stat_code and http_stat_msg when automatically giving preference to a HTTP pattern over a stream pattern. 13 years ago
Eric Leblond 928ade1d04 pf-ring: suppress unused variable. 13 years ago
Eric Leblond c3b9a5e97f pf-ring: add missing header. 13 years ago
Eric Leblond 7731cef782 pf-ring: protect definition of (un)likely
This patch makes (un)likely declared if and only if they are not
declared before.
13 years ago
Anoop Saldanha fd977601b6 fix for bug #574.
More of a temporary solution to prevent any possible FPs.  Disable content
inspection bypass for mpm patterns.
13 years ago
Anoop Saldanha 51c9955c79 fix for bug #577.
If a pattern has matched on mpm, don't re-inspect it later, subject to certain
conditions met by the pattern - namely, not negated, right chop, no replacet
attached to it.
13 years ago
Victor Julien aa4ae98d37 http: fix multipart parsing leading to missing chunks of files in file extraction. 13 years ago
Anoop Saldanha 028c6c1782 Make available custom features of libhtp.
The power of libhtp customisation now available to users.

Options available -

path-backslash-separators: yes
path-compress-separators: yes
path-control-char-handling: none
path-convert-utf8: yes
path-decode-separators: yes
path-decode-u-encoding: yes
path-invalid-encoding-handling: preserve_percent
path-invalid-utf8-handling: none
path-nul-encoded-handling: none
path-nul-raw-handling: none
set-path-replacement-char: ?
set-path-unicode-mapping: bestfit

You can use this for your libhtp customisation.  Options explained in our
wiki.

https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Advanced_libhtp_customization
13 years ago
Anoop Saldanha 340542c44e refactor htpconfigure() 13 years ago
Victor Julien 33b0b07107 bug #572: make sure we use profiling fallback for all architectures except x86_64 and i386. 13 years ago
Victor Julien 50da0e80d5 Fix flow keyword compilation failure. 13 years ago
Anoop Saldanha 3d74fa964a Update all flow referencing to use the new FlowReference and FlowDeReference
macros.
13 years ago
Anoop Saldanha 6c68f86b8c fix for bug #557.
In FFRv2, dereference flow from a packet using the new reference/dereference
util macros.  This allows the decr use_cnt for flow and reseting the flow
pointer to NULL for the pseudo pkt to happen simultaneously, in case there we
fail to retrieve a pseudo_packet and have to return the already obtained
pseudo packets, back to the packetpool.
13 years ago
Anoop Saldanha 88e89d6302 Introduce utility flow macros to help referencing/dereferencing flows. 13 years ago
Anoop Saldanha 4d501778e9 fix for bug #557.
Reset hhd buffers list len if we exit before allocating the buffer.
13 years ago
Anoop Saldanha 855726f372 fix for bug #575.
If sig has no_stream set, don't mask it as requiring flow.  Should get rid of
FNs any.
13 years ago
Victor Julien 1598425a40 detect: properly store a stateful match if it happens at the start of inspection 13 years ago
Victor Julien c3f4f8d46a Dead code cleanup. Coverity 728047, 728048, 728049. 13 years ago
Victor Julien ee5d6fdb6f profiling: fix some profiling info missing from output 13 years ago
Eric Leblond ffbbff9d6c tm-thread: detect thread death
When a thread is dead at init the THV_INIT_DONE flag is not set
and the spawn function can freeze (see bug #553 for an example).
In this case THV_RUNNING_DONE is set and we can also check on this
state for leaving the function. This should fix #bug553
13 years ago
Anoop Saldanha 4e3b206f7b fix http server/client body handling. Update body status based on tx state. 13 years ago
Victor Julien 82fc61770b threshold: allow threshold.config to override rule
Allow threshold.conf to override rule thresholds in the following
cases:

- threshold.config rule uses threshold or event_filter AND
- threshold.config rule applies to a single signature (so no
  gid 0 or sid 0)

Confirmed to work with both threshold and detection_filter rule
keywords.

Part of bug #425.
13 years ago
Victor Julien a0c43a8a1c Minor parsing cleanups in detect-engine options. 13 years ago
Eric Leblond 9f13572843 Fix indentation of win32 files. 13 years ago
Eric Leblond 710d237724 Add missing sctrdup test 13 years ago
Eric Leblond e176be6fcc Use unlikely for error treatment.
When handling error case on SCMallog, SCCalloc or SCStrdup
we are in an unlikely case. This patch adds the unlikely()
expression to indicate this to gcc.

This patch has been obtained via coccinelle. The transformation
is the following:

@istested@
identifier x;
statement S1;
identifier func =~ "(SCMalloc|SCStrdup|SCCalloc)";
@@

x = func(...)
... when != x
- if (x == NULL) S1
+ if (unlikely(x == NULL)) S1
13 years ago
Eric Leblond d292004880 Add some missing checks of SCStrdup return. 13 years ago
Eric Leblond 655577cbbc Add some missing checks of SCMalloc return. 13 years ago
Victor Julien d8667448c1 threshold: allow suppression for sigs with threshold set. Part of #425. 13 years ago
Anoop Saldanha f9a6c890d4 fix for #529
Respect pcre's anchor during content inspection.
13 years ago
Anoop Saldanha 19e8f82f25 Unittest to display #bug 529. pcre anchor not respected 13 years ago
Anoop Saldanha b0b4052860 detect-pcre.c cleanup. Delete old pcre functions that we no longer use. 13 years ago
Eric Leblond 680e941a8f af-packet: clean APFPacketVar before release.
This patch resets the AFPPacketVar linked to a Packet in the release
function to avoid any side effect when the packet is reused. To do
so a new AFPV_CLEANUP macro has been introduced.
13 years ago
Eric Leblond 775f379e2b decode: clean release function 13 years ago
Anoop Saldanha 21f92c0a89 Give priority to non stream content over stream content when selecting fast
pattern.
13 years ago
Victor Julien a08a0e9161 Minor output cleanup 13 years ago
Victor Julien abc3f903f9 Fix defrag compilation warning. 13 years ago
Victor Julien 525367113a Fix compilation if luajit is disabled. 13 years ago
Victor Julien d1abd552e9 luajit: correct offset passed to script for lua's array idx starting at 1. Add http.response_headers and http.response_headers.raw buffers. 13 years ago
Victor Julien 20d2db085e reintroduce pool free func for cases where block alloc is not used. 13 years ago
Victor Julien 98484ffdcc luajit: prealloc lua states to increases chances of alloc success. Luajit requires them to be in memory <2GB. 13 years ago
Victor Julien f962e3de29 pool: only alloc one large block if it will actually be used. 13 years ago
Victor Julien 6f7e527e92 luajit: fix crash at shutdown / rule reload if lua script didn't properly init. 13 years ago
Eric Leblond 8192f6ce8c Add missing include in flow-manager
DefragTimeoutHash was not declared before being used.
13 years ago
Victor Julien 44b7d5551a luajit: fix crash if luaL_newstate fails 13 years ago
Victor Julien b29971bc92 luajit: buffer selection fixes 13 years ago
Victor Julien fcc21ae4cc http: fix multipart parsing bug 13 years ago
Victor Julien 8f337a3904 stream: never resend reassembled data to app layer. 13 years ago
Victor Julien 9a4b612126 app layer events: prefilter sigs that need an event 13 years ago
Victor Julien 575c87aeba engine events: prefilter sigs that need a event 13 years ago
Eric Leblond 5f12b23469 af-packet: little code cleaning
This patch cleans the code were two almost identical treatment on
the packet we're made. It may be linked by a merge error I've done
or to a simple mistake on my side.
13 years ago
Eric Leblond 0581a23f3c af-packet: fix IPS mode
There was an inversion in code resulting as all sockets being seen
as non IPS mode when doing the peering. This resulted in a crash at
first packet because it has no peer.
13 years ago
Eric Leblond 566674ae4a Fix logic operator.
Previous patches on the same subject did not fixed this error as it
was undetected because the code was not compiled on my setup.
13 years ago
Victor Julien 7a044a99ee Defrag engine
Big rewrite of defrag engine to make it more scalable and fix some
locking logic flaws.

Now uses a hash of trackers similar to Flow and Host hashes.
13 years ago
Victor Julien c91c359692 profiling: fix build on older systems 13 years ago
Victor Julien 7babb35aeb profiling: remove obsolete unit test 13 years ago
Eric Leblond cdbc9be1c3 pf_ring: set cluster_id even if only one thread is used. 13 years ago
Victor Julien 1f92307517 profiling: minor cleanup 13 years ago
Victor Julien 3da3e3264c profiling: make sure counters are reset after a reload. 13 years ago
Victor Julien 2343ff8950 profiling: fix memory error in case of rule reload. 13 years ago
Victor Julien ec7e79c748 Rule profiling update
- Remove usage of counters api.
- Store stats in detect engine thread ctx to remove locking
- Support rule reloads
13 years ago
Victor Julien ba6f564296 luajit: add http.uri.raw, cookie, ua, headers, headers.raw buffers. 13 years ago
Eric Leblond 9c28cd40fd Fix build if luajit is not available. 13 years ago
Eric Leblond 937ba71491 defrag: don't return after a cleaning.
This patch changes the policy of the timeout function by cleaning
every timeouted trackers.
Previous code was only freeing the first tracker and this was resulting
in calling the timeout function continuously. One of my previous patch
has modified the function to avoid to run it more than twice a second.
But as it was not taken into account the fact only the first tracker was
freed, the result was that a lot of tracker could not be allocated.
13 years ago
Victor Julien b6834cb6b2 luajit: support http.request_body (http_client_body) and http.response_body (file_data/http_server_body). 13 years ago
Victor Julien 42646579a8 luajit: clean up initialization 13 years ago
Eric Leblond 619014a280 pool: rename Free function to Cleanup
This patch renames Free functions to Cleanup as the free is made
by the pool system.
13 years ago
Eric Leblond f241312a36 defrag: don't use message for repetitive error
When nothing can be fetch from the pool, this can repeat frequently.
Thus displaying a message in the log will not help. This patch
uses a counter instead of a log message. As this is a sort of memcap
this is conformed to what is done for other issues of the same type.
13 years ago
Eric Leblond 6303b5d987 SC_LOG_ERROR is not an error. 13 years ago
Eric Leblond d51dd6a30e Fix warning about unused return of SC_ATOMIC func. 13 years ago
Eric Leblond c4f9d0e0e1 Fix invalid usage of operator. 13 years ago
Eric Leblond 7af9fd7735 freebsd: fix warning about redeclaration. 13 years ago
Eric Leblond 4d2305c0a8 freebsd: fix warning 13 years ago
Eric Leblond 6d55446655 ipfw: avoid critical error for broadcast
In some setup, suricata may receive broadcast packets and the call
to sendto may fail if the wrong interface is choosen by kernel.
This patch change the error treatment to avoid to leave when
this problem occurs.
13 years ago
Eric Leblond 41cb365a39 ipfw: add missing include 13 years ago
Eric Leblond e168824d80 freebsd: fix function usage.
The unlock function was not correctly used in error treatment.
13 years ago
Jason Ish ea020e2be6 Do not trim the FCS, pcaps converted to ERF will have have an FCS. 13 years ago
Eric Leblond 4a1a008009 af-packet: fix looping in ring buffer.
A crash can occurs in the following conditions:
 * Suricata running in other mode than "workers"
 * Kernel fill in the ring buffer
Under this conditions, it is possible that the capture thread reads
a packet that has not yet released by one of the treatment threads
because there is no modification done on the ring buffer entry when
a packet is read. Doing, this it access to memory which can be
released to the kernel and modified. This results in a kind of memory
corruption.

This bug has only been seen recently and this has to be linked with the
read speed improvement recently made in AF_PACKET support.

The patch fixes the issue by modifying the tp_status bitmask in the
ring buffer. It sets the TP_STATUS_USER_BUSY flag when it is confirmed
that the packet will be treated. And at the start of the read, it exits
from the reading loop (returning to poll) when it reaches a packet with
the flag set. As tp_status is set to 0 during packet release the flag
is destroyed when releasing the packet.

Regarding concurrency, we've got a sequence of modification. The
capture thread read the packet and set the flag, then it passes the
queue and the packet get processed by other threads. The change on
tp_status are thus made at different time.

Regarding the value of the flag, the patch uses the last bit of
tp_status to avoid be impacting by a change in kernel. I will
propose a patch to have TP_STATUS_USER_BUSY included in kernel
as this is a generic issue for multithreading application using
AF_PACKET mechanism.
13 years ago
Victor Julien 0d55950840 luajit: add http.uri and http.request_line buffers. 13 years ago
Victor Julien 597b6db8f2 luajit: fix filtering payload or pkt when not available yet 13 years ago
Victor Julien 69186cda12 luajit: force scripts to have 'init' function that returns a table of 'needs' such as packet or payload. 13 years ago
Eric Leblond cd8d215724 pool: improve error handling
Error handling during Pool creation was not perfect as a PoolBucket
could leak.
13 years ago
Victor Julien 829d975d63 Make sure defrag pool sizes are not initialized to 0, see #540. 13 years ago
Eric Leblond 01d3c14449 tls: fix error handling
Handling of error case was correct as pointed out by Coverity
717439.
13 years ago
Eric Leblond 41c72a537a tls: avoid double close.
This should fix issue 717441 reported by Coverity.
13 years ago
Eric Leblond 4e6a4c65f6 defrag: be sure to output NULL tracker
Coverity 720337 pointed out a use after free. We can't be dependent
to HashListTableAdd outputting a NULL tracker.
13 years ago