Commit Graph

4783 Commits (suricata-2.0.8)

Author SHA1 Message Date
Alexander Gozman 405baa3cb2 Fix compile-time error on old kernels and ethtool.h 11 years ago
sxhlinux 546ae9737b Update log-file.c
test whether tx_ud is NULL
11 years ago
Ken Steele f2e777e3a5 Fix lowercase table initialization (bug 1221)
The for loop needed to check for < 256, not < 255.
11 years ago
Victor Julien 27eb0f450a defrag: fix timeout setting when config is missing
When the config is missing, DefragPolicyGetHostTimeout will default
to returning -1. This will effectively set no timeout at all, leading
to defrag trackers being freed too early.
11 years ago
Eric Leblond 97ca02f0c5 defrag: fix reconstruction
This patch is fixing an issue in defragmentation code. The
insertion of a fragment in the list of fragments is done with
respect to the offset of the fragment. But the code was using
the original offset of the fragment and not the one of the
new reconstructed fragment (which can be different in the
case of overlapping segment where the left part is trimmed).

This case could lead to some evasion techniques by causing
Suricata to analyse a different payload.
11 years ago
Eric Leblond 09fd7060ec unix socket: fix valgrind issue
This patch fixes the following issue reported by valgrind:
 31 errors in context 1 of 1:
 Conditional jump or move depends on uninitialised value(s)
    at 0x8AB2F8: UnixSocketPcapFilesCheck (runmode-unix-socket.c:279)
    by 0x97725D: UnixCommandBackgroundTasks (unix-manager.c:368)
    by 0x97BC52: UnixManagerThread (unix-manager.c:884)
    by 0x6155F6D: start_thread (pthread_create.c:311)
    by 0x6E3A9CC: clone (clone.S:113)

The running field in PcapCommand was not initialized.
11 years ago
Eric Leblond a33d1e28e9 unix-manager: fix crash when client disconnect
This patch fixes an issue in unix socket handling. It is possible
that a socket did disconnect when analysing a command and because
the data treatment is done in a loop on clients this was leading
to a update of the list of clients during the loop. So we need
in fact to use TAILQ_FOREACH_SAFE instead of TAILQ_FOREACH.

Reported-by: Luigi Sandon <luigi.sandon@gmail.com>
Fix-suggested-by: Luigi Sandon <luigi.sandon@gmail.com>
11 years ago
Ken Steele 6ebc20f6d8 Rework Tile CPU affinity setting to handle non-contiguous sets of CPUs.
It is possible to have a non-contiguous CPU set, which was not being
handled correctly on the TILE architecture.

Added a "rank" field in the ThreadVar to store the worker's rank separately
from the cpu for this case.
11 years ago
Mats Klepsland 1f3fbbc992 Fix bug #1206
PF_RING ZC uses clusters in the same way as PF_RING DNA. Therefore,
this bug can be fixed as it was fixed for DNA (bug #598).
11 years ago
Victor Julien 896b61452c htp: make htp state handling function more robust
Also, fix wrong cast that worked only by luck.
11 years ago
Victor Julien eff85aba5e http: remove BUG_ON(1) statement
Remove BUG_ON(1) statement that was a leftover from debugging.

Bug #1189
Bug #1212
11 years ago
Alessandro Guido 13448aca1c Fix issue #1214
When applying wildcard thresholds (with sid = 0 and/or gid = 0) it's wrong
to exit on the first signature already having an event filter. Indeed,
doing so results in the theshold not being applied to all subsequent
signatures. Change the code in order to skip signatures with event
filters instead of breaking out of the loop.
11 years ago
Victor Julien 9de536efdb Bug 1098: improve invalid pcre/R handling
When not using a file_data or similar 'sticky buffer', a pcre/R option
needs a content in the same buffer.
11 years ago
Giuseppe Longo 8db3f214f0 nflog: fix memory leaks
This fixes the following memory leaks:

[src/source-nflog.c:222]: (error) Memory leak: ntv
[src/source-nflog.c:236]: (error) Memory leak: ntv
[src/source-nflog.c:253]: (error) Memory leak: ntv
[src/source-nflog.c:258]: (error) Memory leak: ntv
11 years ago
Victor Julien cc54250cf9 Fix live reload segv when startup isn't complete
If a live reload signal was given before the engine was fully started
up (e.g. pcap file thread waiting for a disk to spin up), a segv could
occur.

This patch only enables live reloads after the threads have been
started up completely.
11 years ago
Victor Julien 2c20c9d409 Fix Coverity 1220098 and 1220099
*** CID 1220098:  Missing unlock  (LOCK)
/src/log-droplog.c: 195 in LogDropLogNetFilter()
189         SCMutexLock(&dlt->file_ctx->fp_mutex);
190
191         if (dlt->file_ctx->rotation_flag) {
192             dlt->file_ctx->rotation_flag  = 0;
193             if (SCConfLogReopen(dlt->file_ctx) != 0) {
194                 /* Rotation failed, error already logged. */
>>>     CID 1220098:  Missing unlock  (LOCK)
>>>     Returning without unlocking "dlt->file_ctx->fp_mutex".
195                 return TM_ECODE_FAILED;
196             }
197         }
198
199         if (dlt->file_ctx == NULL) {
200             return TM_ECODE_FAILED;

*** CID 1220099:  Dereference before null check  (REVERSE_INULL)
/src/log-droplog.c: 199 in LogDropLogNetFilter()
193             if (SCConfLogReopen(dlt->file_ctx) != 0) {
194                 /* Rotation failed, error already logged. */
195                 return TM_ECODE_FAILED;
196             }
197         }
198
>>>     CID 1220099:  Dereference before null check  (REVERSE_INULL)
>>>     Null-checking "dlt->file_ctx" suggests that it may be null, but it has already been dereferenced on all paths leading to the check.
199         if (dlt->file_ctx == NULL) {
200             return TM_ECODE_FAILED;
201         }
202
203         char srcip[46] = "";
204         char dstip[46] = "";
11 years ago
Victor Julien 8a77e6bc8e Fix Coverity 1220097
*** CID 1220097:  Missing unlock  (LOCK)
/src/log-file.c: 160 in LogFileWriteJsonRecord()
154             }
155         }
156
157         /* Bail early if no file pointer to write to (in the unlikely
158          * event file rotation failed. */
159         if (aft->file_ctx->fp == NULL) {
>>>     CID 1220097:  Missing unlock  (LOCK)
>>>     Returning without unlocking "aft->file_ctx->fp_mutex".
160             return;
161         }
162
163         FILE *fp = aft->file_ctx->fp;
164         char timebuf[64];
165         AppProto alproto = FlowGetAppProtocol(p->flow);
11 years ago
Jason Ish fc2014ab40 Unregister for file rotation notification when a context is
de-initialized.  Required for unix-socket mode where
contexts come and go.
11 years ago
Jason Ish e1b97fed70 Add signal based file rotation for:
- alert debug log
- fast log
- stats log
- dns log
- drop log
- file log
- http log
- tls log
- eve/json log
11 years ago
Jason Ish 0a33e73417 Add macros for access to the underlying buffer and offset.
Useful for using passing the buffer through to another writer
such as LogFileCtx.
11 years ago
Jason Ish c1b6894ce3 Add a rotation flag to LogFileCtx which loggers can use to register
for log rotation.  Have the LogFileCtx handle the log rotation.
11 years ago
Jason Ish 698a0f7f48 Registration for SIGHUP notification - for loggers interested
in file rotation on SIGHUP.
11 years ago
Victor Julien 25cbf36d40 lua/luajit: use HAVE_LUA mostly
Only use HAVE_LUAJIT if things are done differently from HAVE_LUA,
like in the states pool.
11 years ago
Victor Julien a7118a4ff3 profiling: use wider columns in keyword output
Use wider columns in keyword output so that even on high end sensors
the stats tables remain readable.
11 years ago
jeka dc1599e0dc bugfix in debug mode:
removed function calls from SCReturnX macros
11 years ago
Victor Julien 0765bcc73e nflog: set socket timeout
Set socket timeout so that we can exit if there is no traffic.

It would hang after the SIGINT signal, until packets arived.
11 years ago
Victor Julien 26c0915375 nflog: warn if buffer-size is larger than max-size
If buffer-size is larger than max size, give a warning and adjust
buffer-size to max-size.
11 years ago
Victor Julien 462f9de134 dns: unify type to string logging utility
Both DNS loggers had their own CreateTypeString. This patch unifies
them.
11 years ago
Victor Julien 5e87257845 dns: add names for common types
Add names for SRV, NAPTR, DS, RRSIG, NSEC, NSEC3 types.
11 years ago
Victor Julien 0bbec75764 nflog: fix typo rising->raising 11 years ago
Victor Julien 0857a60fce nflog: improve error handling on NOBUFS
Don't fall through to handle_packet on any NOBUFS condition. Make
sure we catch all NOBUFS.
11 years ago
Giuseppe Longo 4d72911e17 This patch adds the fields into PacketVars struct to setup a packet from a nflog message 11 years ago
Giuseppe Longo 4dda018ede Adds nflog option 11 years ago
Giuseppe Longo 0368d5e4a4 Declare a wrapper to parse group option for nflog 11 years ago
Giuseppe Longo c35432b265 Implements NFLOG runmode 11 years ago
Giuseppe Longo 2ad8a8e111 Bootstraping NFLOG capture mode 11 years ago
Giuseppe Longo 0162e7e809 Adds nflog error code 11 years ago
Giuseppe Longo d213d89981 Updating the Tmm Id for declaration of nflog capture mode 11 years ago
Victor Julien db563ed4b0 tls: check SSL3/TLS version per record
Set event if SSL3/TLS record isn't within the acceptable range.
11 years ago
Victor Julien 8ddcf6a816 dns: add tests for TXT response parsing
Add valid and invalid examples.
11 years ago
Victor Julien bddb2c3bdc dns json: log TXT response data
Log TXT data in the rdata field.
11 years ago
Victor Julien 683d2d64e9 dns: parse and store TXT responses
This way the TXT data can be logged by the loggers.

Ticket #1158
11 years ago
Victor Julien 8ba8c0bf6f json output: don't set 'unknown' for missing data
Instead of setting 'unknown' or '<unknown>' just pass NULL to json_*
function, which results in omitting the data.
11 years ago
Tom DeCanio 11ca25ddca eve-log: swap ip/port pairs in dns answers 11 years ago
Victor Julien d4215fca84 http-json: fix coverity warning
*** CID 1211009:  Bad bit shift operation  (BAD_SHIFT)
/src/output-json-http.c: 265 in JsonHttpLogJSON()
259         /* log custom fields if configured */
260         if (http_ctx->fields != 0)
261         {
262             HttpField f;
263             for (f = HTTP_FIELD_ACCEPT; f < HTTP_FIELD_SIZE; f++)
264             {
>>>     CID 1211009:  Bad bit shift operation  (BAD_SHIFT)
>>>     In expression "1 << f", left shifting by more than 31 bits has undefined behavior.  The shift amount, "f", is as much as 46.
265                 if ((http_ctx->fields & (1<<f)) != 0)
266                 {
267                     /* prevent logging a field twice if extended logging is
268                        enabled */
269                     if (((http_ctx->flags & LOG_HTTP_EXTENDED) == 0) ||
270                         ((http_ctx->flags & LOG_HTTP_EXTENDED) !=

________________________________________________________________________________________________________
*** CID 1211010:  Bad bit shift operation  (BAD_SHIFT)
/src/output-json-http.c: 492 in OutputHttpLogInitSub()
486                         {
487                             if ((strcmp(http_fields[f].config_field,
488                                        field->val) == 0) ||
489                                 (strcasecmp(http_fields[f].htp_field,
490                                             field->val) == 0))
491                             {
>>>     CID 1211010:  Bad bit shift operation  (BAD_SHIFT)
>>>     In expression "1 << f", left shifting by more than 31 bits has undefined behavior.  The shift amount, "f", is as much as 46.
492                                 http_ctx->fields |= (1<<f);
493                                 break;
494                             }
495                         }
496                     }
497                 }
11 years ago
Victor Julien 5cdd9b460a unix-socket: reset logging api's properly
Lack of proper reset lead to logs not being written after the first
pcap had been processed.
11 years ago
Victor Julien fd56acd4b3 stream: cleanup
StreamTcpSetDisableRawReassemblyFlag() has the same effect as
AppLayerParserTriggerRawStreamReassembly in that it will force the
raw reassembly to flush out asap. So it is redundant to call both.
11 years ago
Victor Julien 3543150f42 stream: implement raw reassembly stop api
Implement StreamTcpSetDisableRawReassemblyFlag() which stops raw
reassembly for _NEW_ segments in a stream direction.

It is used only by TLS/SSL now, to flag the streams as encrypted.
Existing segments will still be reassembled and inspected, while
new segments won't be. This allows for pattern based inspection
of the TLS handshake.

Like is the case with completely disabled 'raw' reassembly, the
logic is that the segments are flagged as completed for 'raw' right
away. So they are not considered in raw reassembly anymore.

As no new segments will be considered, the chunk limit check will
return true on the next call.
11 years ago
Victor Julien b2184f936e stream: unify segment discard handling
Have a single function StreamTcpReturnSegmentCheck determine if a
segment is ready to be removed from the stream.

Handle FLOW_NOPAYLOAD_INSPECT in raw reassembly.
11 years ago
Victor Julien ad355c3c0a app-layer: improve no payload inspect flag
If setting APP_LAYER_PARSER_NO_INSPECTION_PAYLOAD, trigger raw
reassembly.
11 years ago
Victor Julien f0bdb009ed tls/heartbleed: fix test
Now that we continue to track ssl/tls after the handshake, we need
to fix tests that checked for the cutoff flags.
11 years ago
Victor Julien 31655aef7e tls/heartbleed: improve encrypted logic
Don't assume that if the type field isn't 01 or 02 it's an encrypted
heartbeat. Instead, use our knowledge of the SSL state.
11 years ago
Victor Julien fdbd9b3f25 tls/heartbleed: formatting fixes 11 years ago
Will Metcalf 26169ad8c5 Look for Mismatched Encrypted HB request and response sizes, along with multiple in-flight HB requests from the same direction 11 years ago
Victor Julien 0564a8da3c detect: add more defensive checks for flow handling
Don't unconditionally deref f->alparser in detection through
DeStateFlowHasInspectableState(). In very rare cases it can
be NULL.
11 years ago
Victor Julien 2002067fb1 http-json: init 'fields' to 0 before setting it
httplog_ctx->fields would not be initialized before setting flags in
it:

Scanbuild:
output-json-http.c:491:46: warning: The left expression of the compound assignment is an uninitialized value. The computed value will also be garbage
                            http_ctx->fields |= (1<<f);
                            ~~~~~~~~~~~~~~~~ ^
1 warning generated.

Drmemory:
~~27874~~ Error #1: UNINITIALIZED READ: reading register eax
~~27874~~ # 0 JsonHttpLogJSON                       [/home/buildbot/qa/buildbot/donkey/drmemory/Suricata/src/output-json-http.c:260]
~~27874~~ # 1 JsonHttpLogger                        [/home/buildbot/qa/buildbot/donkey/drmemory/Suricata/src/output-json-http.c:375]

Just memset the whole structure right after initialition.
11 years ago
Tom DeCanio 7df9b283f1 json: address custom output capability to http eve log review comments 11 years ago
Tom DeCanio 4838b9bf4f json: add custom output capability to http eve log 11 years ago
Victor Julien 9abf595122 rohash: fix potential bad shift
Fix issue detected byCoverity:

*** CID 1197756:  Bad bit shift operation  (BAD_SHIFT)
/src/util-rohash.c: 74 in ROHashInit()
68         }
69         if (hash_bits < 4 || hash_bits > 32) {
70             SCLogError(SC_ERR_HASH_TABLE_INIT, "invalid hash_bits setting, valid range is 4-32");
71             return NULL;
72         }
73
>>>     CID 1197756:  Bad bit shift operation  (BAD_SHIFT)
>>>     In expression "1U << hash_bits", left shifting by more than 31 bits has undefined behavior.  The shift amount, "hash_bits", is as much as 32.
74         uint32_t size = hashsize(hash_bits) * sizeof(ROHashTableOffsets);
75
76         ROHashTable *table = SCMalloc(sizeof(ROHashTable) + size);
77         if (unlikely(table == NULL)) {
78             SCLogError(SC_ERR_HASH_TABLE_INIT, "failed to alloc memory");
79             return NULL;

This was only a potential issue as ROHashInit was only called with
hash_bits 16 in the code.

Bug #1170.
11 years ago
Eric Leblond 1992a22769 af-packet: exit in case of a fatal error
During socket creation all error cases were leading to suricata to
retry the opening of capture. This patch updates this behavior to
have fatal and recoverable error case. In case of a fatal error,
suricata is leaving cleanly.
11 years ago
Jason Ish d28879f1a2 DAG: Pull some raw values out into defines. 11 years ago
Jason Ish 8ab962df7c DAG: Consistent code style. 11 years ago
Jason Ish c00ec5f4fc DAG: Sync dag packet and drop counts to live device on exit for better exit
logging.
11 years ago
Victor Julien 4d6cc1dbc6 json-file: improve error handling
If the functions getting uri, etc fail. Return "unknown" instead of
NULL pointer. This improves consistency.
11 years ago
Victor Julien 2d25f12cda json-file: check pointer before using
A check was missing to see if ht_ud was not null before using the
pointer. This should be rare, but it can happen.

Reported-by: Will Metcalf
11 years ago
Victor Julien d8481cb7cd stream: improve retransmission handling
Improve retransmission handling in the CLOSE_WAIT state.

Bug #1180.
11 years ago
Victor Julien 4929c840cd stream: update GAP detection
Change GAP detection logic. If we encounter missing data before
last_ack, we know we have missed data. The receiving host has ack'd
it already, so a retransmission of the missing data is highly
unlikely.
11 years ago
Victor Julien 5db228b9f6 stream: fix raw reassembly flag issue
AppLayer reassembly correctly only flags a segment as done when it's
completely used in reassembly. Raw reassembly could flag a partially
used segment as complete as well. In this case the segment could be
discarded early. Further reassembly would miss data, leading to a
gap. Due to this, up to 'window size' bytes worth of segments could
remain in the session for a long time, leading to memory resource
pressure.

This patch sets the flag correctly.
11 years ago
Victor Julien 539bf57a65 stream: improve StreamTcpPruneSession
Check if a segment is done for the app-layer using
StreamTcpAppLayerSegmentProcessed instead of the flag directly so the
gap case works better.
11 years ago
Victor Julien 3fa818d087 stream: flags cleanup
Stream flags are 16bit, but notation is still 8bit. Clean this up to
avoid confusion.
11 years ago
Victor Julien 1bd189a076 protocol detection: handle very unbalanced case
Some traffic is very unbalanced. For example a 4 bytes request
followed by 12mb of response data. If the 4 bytes don't lead to
the protocol being detected, then app layer processing won't
start, but it will not give up either. It will just wait for more
data. This leads to piling up data on the opposite side.

Observed:

TS: 4 bytes. PP failed (bit set), PM has not given up (bit not set).
    This makes sense as max_depth is not yet reached.

TC: 12mb. PP and PM failed (bits set).

As ts-PM never gives up, we never consider proto detect complete,
so all segments in either direction are still kept in the session.

This patch adds a cutoff for this case:
- IF for TS we have PP bit set, PM not set, AND
- we have TC both bits set, AND
- proto is unknown, AND
- TC data is 100k already, THEN
- give up on protocol detection.

The same for the opposite direction.
11 years ago
Victor Julien ed46fd715d stream: improve midstream reassembly gap detection
The reassembly gap detection makes use of the window. However, in
midstream mode the window size we use is unreliable, as we have to
assume window scaling is in place. This patch improves midstream
handling of those cases.
11 years ago
Victor Julien 165f129c61 stream: detect data gap at stream start
In midstream mode we may encounter a case where the data we is beyond
the isn, but below last_ack. This means we're missing some data, that
is already acked so it won't be retransmitted. Therefore, we can
conclude it's a data gap.
11 years ago
Victor Julien 32aafa6a48 proto detect: add cutoff for unbalanced traffic
If we're getting a lot of data in one direction and the proto for this
direction is unknown, proto detect will hold up segments in the segment
list in the stream. They are held so that if we detect the protocol on
the opposing stream, we can still parse this side of the stream as well.
However, some sessions are very unbalanced. FTP data channels, large
PUT/POST request and many others, can lead to cases where we would have
to store many megabytes worth of segments before we see the opposing
stream. This leads to risks of resource starvation.

In this patch, a cutoff point is enforced. If we've stored 100k in one
direction and we've seen no data in the other direction, we give up.

If we've given up, the applayer_proto_detection_skipped event is set.
    app-layer-event: applayer_proto_detection_skipped;
11 years ago
Victor Julien 44b8be3732 protocol detection: midstream handling update
If a TCP session is midstream, we may end up with a case where the
start of an HTTP request is missing. We won't detect HTTP based on
the request. However, the reply is fine, so we detect HTTP anyway.
This leads to passing the incomplete request to the htp parser.

This has been observed, where the http parser then saw many bogus
requests in the incomplete data. This is not limited to HTTP.

To counter this case, a midstream session MUST find it's protocol
in the toserver direction. If not, we assume the start of the
request/toserver is incomplete and no reliable detection and parsing
is possible. So we give up.
11 years ago
Victor Julien c1dcc7a268 stream: fix midstream syn/ack setup
If midstream is enabled and the first packet is the syn/ack packet from
the 3whs, initialized server.last_ack to the packets seq.

This fixes tracking the session.
11 years ago
Victor Julien 59ccd7b62d byte-test: switch to pcre_copy_substring
Fixes cppcheck:
[src/detect-bytejump.c:360]: (error) Uninitialized variable: str_ptr
[src/detect-bytejump.c:372]: (error) Uninitialized variable: str_ptr
[src/detect-bytejump.c:375]: (error) Uninitialized variable: str_ptr

Bug #1046.
11 years ago
Victor Julien 1de9256b18 byte-extract: switch to pcre_copy_substring
Fixes memory leaks (str_ptr was never freed).
Fixes cppcheck warning:
    [src/detect-byte-extract.c:268]: (error) Uninitialized variable: str_ptr

Bug #1046.
11 years ago
Victor Julien a25f532ce7 dns: improve response name parsing
Improve parsing of names with multiple pointers following each other.
12 years ago
Eric Leblond 18c7eeefca packet handling: fix release function
Extended data were freed before the release function was called.
The result was that, in AF_PACKET IPS mode, the release function
was only sending void data because it the content of the extended
data is the content of the packet.

This patch updates the code to have the freeing of extended data
done in the cleaning function for a packet which is called by the
release function. This improves consistency of the code and fixes
the bug.
12 years ago
Victor Julien 3df904475c proto-detect: masks cleanup
The direction specific masks were not used correctly. The toserver ones
were only used for 'dp' registrations, the toclient ones only for 'sp'.

The patch merges them.
12 years ago
Victor Julien 8252416c10 proto-detect: update port logic
If a flow matches both an 'sp' based PP registration and a 'dp' based,
until now we would only check the 'dp' one. This patch changes that. It
will inspect both.
12 years ago
Victor Julien eae5b1ba35 app-layer: proto detection update
Instead of the notion of toserver and toclient protocol detection, use
destination port and source port.

Independent of the data direction, the flow's port settings will be used
to find the correct probing parser, where we first try the dest port,
and if that fails the source port.

Update the configuration file format, where toserver is replaced by 'dp'
and toclient by 'sp'. Toserver is intrepreted as 'dp' and toclient as
'sp' for backwards compatibility.

Example for dns:

    dns:
      # memcaps. Globally and per flow/state.
      #global-memcap: 16mb
      #state-memcap: 512kb

      # How many unreplied DNS requests are considered a flood.
      # If the limit is reached, app-layer-event:dns.flooded; will match.
      #request-flood: 500

      tcp:
        enabled: yes
        detection-ports:
          dp: 53
      udp:
        enabled: yes
        detection-ports:
          dp: 53

Like before, progress of protocol detection is tracked per flow direction.

Bug #1142.
12 years ago
Victor Julien 7b0f1e9512 stream: improve retransmission handling
When connection are closing, don't reject retransmissions of data
packets.

Bug #1180.
12 years ago
Victor Julien 96adcf6829 refactor IDS/IPS engine mode logic
Instead of error phrone externs with macro's, use functions with a local
static enum var instead.

- EngineModeIsIPS(): in IPS mode
- EngineModeIsIDS(): in IDS mode

To set the modes:

- EngineModeSetIDS(): IDS mode (default)
- EngineModeSetIPS(): IPS mode

Bug #1177.
12 years ago
Ken Steele 354a24e2ef Fix unaligned load in AC-TILE MPM.
The SLOAD define using __insn_ld2s_L2 is used to provide a compiler
hint that the load will come from the L2 cache instead of the L1. It
also specifies that it is a 2 byte signed load. For the Tiny MPM, that
needs to be a 1-byte load, which is what is specified in util-ac-mpm-tile.c,
but the #undef was removing that definition.
12 years ago
Victor Julien fc559ce227 detect: fix alstate handling
Previously, the alstate use in the main detect loop was unsafe. The
alstate pointer would be set duing a lock, but it would again be used
after one or more lock/unlock cycles. If the data pointed to would
disappear, a dangling pointer would be the result.

Due to they way flows are cleaned up using reference counting and
such, changes of this happening were very small. However, at least
one path can lead to this situation. So it had to be fixed.
12 years ago
Victor Julien b6e2a6f525 detect: locking update continued
Make DeStateDetectContinueDetection get it's own alstate pointer instead
of using the one that was passed to it. We now get and use it only
inside a flow lock.
12 years ago
Victor Julien cf31e2cc74 detect: locking update
Make DeStateDetectStartDetection get it's own alstate pointer instead
of using the one that was passed to it. We now get and use it only
inside a flow lock.
12 years ago
Victor Julien 5e1bc99e5b detect: cleanup
Remove unused alstate and app layer flags arguments from
DetectEngineInspectPacketPayload()
12 years ago
Victor Julien 6e0112d737 detect: modify AMATCH locking
This is an intrusive change. This patch modifies the way AMATCH
inspection uses locking.

So far, each keyword did it's own locking. This lead to a situation
where a 'alstate' pointer was passed around that was not always
protected by a lock.

This patch moves the locking to the Stateful detection functions.
12 years ago
Eric Leblond 43b6cbd4bc af-packet: fix error handling
Only exit from synchronization loop on poll error and not in case
of a timeout.
12 years ago
Eric Leblond 6e77c4d1b4 util-ioctl: only get MTU when iface name is set
This patch fixes a warning message when suricata is started without
giving an interface name on the command line. The code was trying
to get the MTU even if pcap_dev was not set.
12 years ago
Victor Julien 79c924af8c Fix 2 compiler warnings
FreeBSD 10 32-bit with clang 3.3:

log-tlslog.c:172:14: error: format specifies type 'long' but the argument has type 'time_t' (aka 'int') [-Werror,-Wformat]
             p->ts.tv_sec,
             ^~~~~~~~~~~~
1 error generated.

detect-engine-payload.c:508:27: warning: format specifies type 'long' but the argument has type 'time_t' (aka 'int') [-Wformat]
    printf("%ld.%06ld\n", tv_diff.tv_sec, (long int)tv_diff.tv_usec);
            ~~~           ^~~~~~~~~~~~~~
            %d
1 warning generated.
12 years ago
Victor Julien 26778b8703 output-api: cleanup handling
Add output 'free list' that contains all the output ctx' that need
cleanup at shutdown. It differs from the runmode output list in that
it will also contain a 'parent' for the submodules that share the
context of it's parent.
12 years ago
Victor Julien c27304451e output api: complete shutdown functions
Add missing function for Filedata API. Clean up list in all functions.
12 years ago
Victor Julien b2d29a85e9 tls-json: add cleanup function
Properly clean up output context when shutting down.
12 years ago
Victor Julien 34069054ce drop-json: fix cleanup
Use proper function for sub-module cleanup. LogFileCtx is not managed
by the sub-module, so don't clean it.
12 years ago
Victor Julien bc1c06b9e4 eve-log: fix mem leak at shutdown
Make sure we free all memory in the shutdown function.
12 years ago
Victor Julien 90c4834709 ssh-json: add clean up functions
Add clean up functions for the SSH json logger.
12 years ago
Victor Julien 3a6be9772f http-json: add missing cleanup functions
Add cleanup functions.
12 years ago
Victor Julien 7ffd227133 file-json: cleanup at shutdown
Fix a memory leak at shutdown. Module didn't have a cleanup function.
12 years ago
Victor Julien 7ee3b456a3 dns-json: fix cleanup
Use specialized cleanup function for sub-module case. Freeing the
LogFileCtx is not the responsibility of a sub-module.
12 years ago
Victor Julien 1f2310bb34 alert-json: fix cleanup
Call specialized clean up function when running as a sub-module.
12 years ago
Victor Julien f62185c207 log-tls: run Disable at shutdown
Call OutputTlsLoggerDisable at cleanup.
12 years ago
Victor Julien f96a54535c drop loggers: call disable func
Call OutputDropLoggerDisable() on cleanup.
12 years ago
Victor Julien 9df045d086 output: add Disable funcs to mirror Enable
For the loggers that we allow only one instance for: tls, ssh, drop, we
track active loggers through Output*Enable functions. Add Disable
functions to mirror this. They are to be called from the shutdown funcs
those loggers use.
12 years ago
Victor Julien 4a104ae315 unified2: fix memory leak at shutdown
Module didn't properly free output context at shutdown. Led to a leak
in Unix Socket mode.
12 years ago
Eric Leblond 9961520316 output: clean file desc at exit.
This is a beginning of implementation for bug #1660:
 https://redmine.openinfosecfoundation.org/issues/1160

This patch adds a cleaning function for each logger of new type
(packet, tx and file). These functions are called in RunModeShutDown().

The state of this patch is that it is crashing suricata when sending
pcap to analyse:
 - At first pcap if tx and file cleaning function are called
 - At second pcap if only packet cleaning function is called

The cause in first case is unknown. In second case this is due to
the necessity of cleaning the list of logger registered to a logging
type.
12 years ago
Pierre Chifflier d476c654ee TLS: add detection for malicious heartbeats (AKA heartbleed)
The OpenSSL implementation of RFC 6520 (Heartbeat extension) does not
check the payload length correctly, resulting in a copy of at most 64k
of memory from the server (ref: CVE-2014-0160).
This patch adds support for decoding heartbeat messages (if not
encrypted), and checking several parts (type, length and padding).
When an anomaly is detected, a TLS event is raised.
12 years ago
Victor Julien ab503873ca erf-file: clean up decode thread local storage
Clean up the thread local data the decode portion of ERF users.

Bug #978
12 years ago
Victor Julien 6da8652a77 endace-dag: clean up decode thread local storage
Clean up the thread local data the decode portion of DAG uses.

Bug #978
12 years ago
Victor Julien 09ebbe08df mpipe: clean up decode thread local storage
Clean up the thread local data the decode portion of mpipe uses.

Bug #978
12 years ago
Victor Julien 8c16fede08 ipfw: clean up decode thread local storage
Clean up the thread local data the decode portion of ipfw uses.

Bug #978
12 years ago
Victor Julien 10c791c937 napatech: clean up decode thread local storage
Clean up the thread local data the decode portion of napatech uses.

Bug #978
12 years ago
Victor Julien c3e193e786 pcap: clean up decode thread local storage
Clean up the thread local data the decode portion of pcap uses.

Bug #978
12 years ago
Victor Julien 900fc6fdc7 pfring: clean up decode thread local storage
Clean up the thread local data the decode portion of pfring uses.

Bug #978
12 years ago
Victor Julien a8b1af3369 nfq: clean up decode thread local storage
Clean up the thread local data the decode portion of nfq uses.

Bug #978
12 years ago
Victor Julien 2864f9eef9 af-packet: clean up decode thread local storage
Clean up the thread local data of the decode part of afpacket.

Bug #978
12 years ago
Victor Julien bb2e9af40f pcap-file: clean up decode thread local storage
Clean up the thread local data the decode portion of pcap-file use.

Bug #978.
12 years ago
Victor Julien d26ceb2356 decode: introduce DecodeThreadVarsFree
As a mirror of DecodeThreadVarsAlloc, DecodeThreadVarsFree is used
to free the memory that DecodeThreadVarsAlloc alloc'd, including
AppLayer storage.
12 years ago
Victor Julien 552558894c app-layer: cleanups
Clean up AppLayerParserThreadCtxAlloc and AppLayerParserThreadCtxFree.
Both used confusing variables in loops, with the wrong types.
12 years ago
Jason Ish 7e268bd4d4 Force pidfile creation of --pidfile.
A pidfile can be useful when not in daemon mode, for example
when running under a process supervisor.
12 years ago
Tom DeCanio 4085f08602 json: somewhere along the way IP/port pairs had gotten swapped in
http and ssh eve logs
12 years ago
Victor Julien b9227ad20c tls: no event on 'new session ticket' in handshake
Don't set an event on encountering a 'new session ticket' (4) record
in the TLS handshake.
12 years ago
Victor Julien 1195f882b9 ipv6: add support for PAD1
Support PAD1 in IPv6 HOP options header and DST options header.
12 years ago
Victor Julien 7539372db7 icmpv6: add multicast types
Only add them to check if the code is 0 and to make sure the default
case doesn't set an 'unknown type' event.
12 years ago
Victor Julien eb3a9d3076 TLS: register patterns for tls-alerts
Register patterns for when server has an alert as the first message.
12 years ago
Victor Julien a96446d39e detect state: fix indent
AMATCH block was indented too far.
12 years ago
Victor Julien 6c8ca76942 detect state: remove alproto check for AMATCH
Not all AMATCHes set a alproto.
12 years ago
Victor Julien c9436a6aef Fix app-layer-protocol FP on multi TX flow
In case of multiple transactions, the stored AMATCH list would not have
been reset, but it would still be reconsidered. Even though none would
match, the engine would still conclude that the rule matched.
12 years ago
Eric Leblond 9e03550230 tls: fix problem with tls.store keyword
Pierre Chifflier pointed out that a rule like:
 alert tls any any -> any any (msg:"TLS store"; tls.issuerdn:!"C=FR"; tls.store;)
was alerting but not storing the certificate. If the filter was
removed:
 alert tls any any -> any any (msg:"TLS store"; tls.store;)
then tls.store is working as expected.

This was linked with fact that logging is only done once for a SSL
state. So without filter, once we have the info we can log and we
run the storage. But when there is a filter, we log and then there
is a filter analysis and alerting. And as logging as already be done
we don't enter in the logging function and there is no storage.

This patch forces the entrance in the log function when there is a
request for TLS storage. And it adds an exit in the logging function
to only do the storage part if the TLS state has already being logged.
12 years ago
Victor Julien 806844d852 af-packet: fix init sync with no traffic
Previously the sync code would depend on traffic to complete. This
patch adds poll support and can complete the setup if the poll timeout
is reached as well.

Part of bug #1130.
12 years ago
Eric Leblond 238ff23111 af-packet: move packet fanout code
The sooner is the better for that caode as these means we will get
all sockets binded to fanout group as fast as possible.
12 years ago
Eric Leblond 919377d4a5 af-packet: synchronize reading start
This patch is updating af-packet to discard packets that have been
sent to a socket before all socket in a fanout group have been setup.
Without this, there is no way to assure that all packets for a single
flow will be treated by the same thread.

Tests have been done on a system with an ixgbe network card. When using
'cluster_flow' load balancing and disactivating receive hash on the iface:
 ethtool -K IFACE rxhash off
then suricata is behaving as expected and all packets for a single flow
are treated by the same thread.

For some unknown reason, this is not the case when using cluster_cpu. It
seems that in that case the load balancing is not perfect on the card side.

The rxhash offloading has a direct impact on the cluster_flow load balancing
because load balancing is done by using a generic hash key attached to
each skb. This hash can be computed by the network card or can be
computed by the kernel. In the xase of a ixgbe network card, it seems there
is some issue with the hash key for TCP. This explains why it is necessary to
remove the rxhash offloading to have a correct behavior. This could also
explain why cluster_cpu is currently failing because the card is using the
same hash key computation to do the RSS queues load balancing.
12 years ago
Victor Julien 70efc66e33 fast-log: restore logging of Drop/wDrop
Restore drop print logic. Probably got lost in large refactoring round
introducing log APIs.

Bug #1138.
12 years ago
Victor Julien 1af39d55cf detect: allow alias registration for rule keywords
This allows for registering a keyword under another name while keeping
the old name active and supported.

Do this for 'luajit', which can now also be used as just 'lua'.
12 years ago
Victor Julien 010a334e1a json alert: fix action
It would give 'Pass' as an action when the rule is set to 'alert'.
12 years ago
Victor Julien e04b5f0dca eve-http: register with app-layer api
The HTTP module of Eve didn't register itself with the app-layer
for HTTP. This meant that if no other HTTP logger was active, the
HTTP logging in Eve wouldn't work.

This patch makes the HTTP Eve module register itself correctly.

Bug #1133.
12 years ago
Victor Julien 5d96ea570f eve-file: set event_type to fileinfo
To remain constistent with the other logs, set the event type to
the same name as the structure containing the defails. In this
case fileinfo.

Part of bug #1127.
12 years ago
Victor Julien 305da0248d eve-files: file -> fileinfo
Due to what appears to be an issue in logstash, the 'file' part of
the file event types was masked by a field that logstash-forwarder
added itself.

Since logstash-forwarder is an important part of the logstash stack,
this patch works around the issue by renaming our 'file' structure
to 'fileinfo', thus resolving the naming conflict.

Bug #1127
12 years ago
Victor Julien 3e7714aca2 eve-http: print <unknown> like in eve-files
When UA or Host are unknown, print <unknown> instead of <useragent
unknown> or <hostname unknown>.

Bug #1131.
12 years ago
Victor Julien c5041d35d5 Fix live reload
Fix memsets clearing out of bounds memory on live reload, causing
crashes and corrupted backtraces.

Bug #1128.
12 years ago
Victor Julien fdb1bd9668 pfring: call enable_ring after set_cluster
Move pfring_enable_ring to the start of ReceivePfringLoop() so that
it's guaranteed to be called after all threads have called
pfring_set_cluster first.

This is necessary because pfring will already make packets available
to thread N, while thread N+1 is still registering itself. This leads
to cases where the first packet(s) of a flow are processed by a
different thread in Suricata than the later ones.

This is a race condition only at start up. New flows after the pfring
initialization is complete will not be influenced by this.

Bug #1129.
12 years ago
Victor Julien a3020b5306 eve-log: output cleanup
Suppress debug messages and print in a nicer way which modules are
being enabled.
12 years ago
Victor Julien ed877c64d1 Bug 611: fix for iponly
Fix Bug 611 for ip-only rules as well. If 'alert ip' rule has ports,
don't match on protocols that don't have ports. Like ICMP.

Bug #611.
12 years ago
Eric Leblond 6c3c234ca5 output-json: update timestamp format
This patch updates the timestamp format used in eve loggin.
It uses a ISO 8601 comptatible string. This allow tools parsing
the output to easily detect adn/or use the timestamp.

In the EVE JSON output, the value of the timestamp key has been
changed to 'timestamp' (instead of 'time'). This allows tools
like Splunk to detect the timestamp and use it without configuration.

Logstash configuration is simple:

input {
   file {
      path => [ "/usr/local/var/log/suricata/eve.json" ]
      codec =>   json
      type => "suricata-log"
   }
}

filter {
   if [type] == "suricata-log" {
      date {
        match => [ "timestamp", "ISO8601" ]
      }
   }
}

In splunk, auto detection of the fle format is failling and it seems
you need to define a type to parse JSON in
$SPLUNK_DIR/etc/system/local/props.conf:

[suricata]
KV_MODE = json
NO_BINARY_CHECK = 1
TRUNCATE = 0

Then you can simply declare the log file in
$SPLUNK_DIR/etc/system/local/inputs.conf:

[monitor:///usr/local/var/log/suricata/eve.json]
sourcetype = suricata

In both cases the timestamp are correctly imported by
the tools.
12 years ago
Eric Leblond 1fa4233d67 pfring: get vlan id from header
PF_RING is delivering the packet with VLAN header stripped. This
patch updates the code to get the information from PF_RING extended
header information.

This patch uses the new function SCKernelVersionIsAtLeast to know
that we've got a old kernel that do not strip the VLAN header from
the message before sending it to userspace.
12 years ago
Eric Leblond cf30adcedc ipfw: fix build
Buildbot reported:
 runmode-ipfw.c: In function 'RunModeIpsIPFWAuto':
 runmode-ipfw.c:85: error: implicit declaration of function 'LiveDeviceHasNoStats'
12 years ago
Eric Leblond 606e19124b http: add info message about memcap
Display a message about http memcap when it is set in config file.
12 years ago