Commit Graph

5093 Commits (suricata-2.1beta3)

Author SHA1 Message Date
Ken Steele 5008d0a58b Remove the b2gm and b2gc MPMs
These MPMs have code that looks like it won't work and updating them to
for the new MPM optimization wasn't working.
11 years ago
Victor Julien 227a7de351 Global define of MIN
Some OS' provide it automatically, so make sure we define it
conditionally in one place.
11 years ago
Victor Julien bcfd61416f Fix a fix: defrag OOM condition
** CID 1257764:  Dereference after null check  (FORWARD_NULL)
/src/defrag.c: 291 in Defrag4Reassemble()

** CID 1257763:  Dereference after null check  (FORWARD_NULL)
/src/defrag.c: 409 in Defrag6Reassemble()

In the error case 'rp' can be both NULL or non-NULL.
11 years ago
Victor Julien 43a1007788 detect: add test for memcmp issue 11 years ago
Victor Julien 0d910bed1d Add test for memcmp issue. 11 years ago
Victor Julien 17dfd59bc3 memcmp: compare the first byte as well
MemcmpLowercase would not compare the first byte of both input buffers
leading to two non-identical buffers to be considered the same.

Affects SSE_4_1 and SSE_4_2 implementations of SCMemcmpLowercase, as well
as the non-SIMD implementation. SSE_3 and Tile version are not affected.
11 years ago
Victor Julien c51ce4d2c0 Fix OS X 10.10 unittest failure
Work around OS X 10.10 Yosemite returning EDEADLK on a rwlock wrlocked
then tested by wrtrylock. All other OS' (and versions of OS X that I
tested) seem to return EBUSY instead.
11 years ago
Victor Julien baa55ba239 Fix Tilera compilation
Use proper initializer for a static mutex declaration.

Credits: Ken Steele
11 years ago
Victor Julien 8e946b92b7 Fix compilation on OS X Yosemite
Due to our unconditional declaration of the strlcat and strlcpy
functions, compilation failed on OS X Yosemite.

Bug #1192
11 years ago
Victor Julien 485f34134e unix socket: support profiling 11 years ago
Victor Julien f32d79dfe0 smtp: fix tx handling
Fix issue where SMTPStateGetTxCnt would return the actual active tx'.

The 'GetCnt' API call is not named correctly. It should be 'GetMaxId',
as this is actually the expected behavior.
11 years ago
Victor Julien 105b4340c2 thread local storage: add to build-info 11 years ago
Victor Julien 623c2e78fd packet pool: make pending pool use more robust
Don't leave pointers dangling.
11 years ago
Victor Julien 6e174128c8 packet pool: memory fixes for non-TLS
If the posix TLS implementation is used, the packet pool is memset to
0 before use.

Also use proper 'free' function.
11 years ago
Victor Julien 2745cd2ce9 packet pool: fix wrong free call 11 years ago
Eric Leblond ff8dae3b75 app-layer: fix 'detection-only' keyword
If we follow the description in the yaml file, we should disable
parsing if 'detection-only' keyword is used.
11 years ago
Eric Leblond 969abc2ccd output-json: fix duplicate logging
This patches is fixing a issue in the OutputJSONBuffer function. It
was writing to file the content of the buffer starting from the start
to the final offset. But as the writing is done for each JSON string
we are duplicating the previous events if we are reusing the same
buffer.

Duplication was for example triggered when we have multiple alerts
attached to a packet. In the case of two alerts, the first one was
logged twice more as the second one.
11 years ago
Victor Julien dc5e2a515c stream: improve inline mode GAP handling
Don't conclude a GAP is 'final' until the missing data is ack'd.

Further, cleanup and unify more with the non-inline code.
11 years ago
Victor Julien b69ca16553 stream: move utility functions
This way they can be used by the *Inline* functions as well.
11 years ago
Victor Julien a095694945 host: register unittests
Host unittests were not registered so they wouldn't run.
11 years ago
Victor Julien 60b50e1ca5 packet-pool: free pending packets 11 years ago
Victor Julien 8b2dd81628 stats: stats threads don't need packet pools 11 years ago
Victor Julien ffd2248459 flow manager: destroy packet pool on close 11 years ago
Victor Julien c4e1324690 flow-timeout: use packet pool
Use packet pool for pseudo packets on flow timeout. Wait for a packet
if necessary.

For shutdown, alloc a new pool as the 'main()' thread calls this.
11 years ago
Victor Julien cef609bb73 threading: lock TmThreadKillThreadsFamily 11 years ago
Victor Julien 3499d682c4 flow timeout: cleanups
Rename FlowForceReassemblyForFlowV2 to just FlowForceReassemblyForFlow
as there is no V1.
11 years ago
Victor Julien 6e69b51123 flow timeout: cleanup
Remove now unused old flow timeout code.
11 years ago
Victor Julien de4bda14e6 stream: handle flow timeout stream end packets
Handle flow timeout packets in the stream engine. Previously the flow
timeout code would call reassembly code directly.
11 years ago
Victor Julien 0ffaad66eb flow-time: disable remainder of the old timeout code
Disable registration code that was looking for threadvars
and slots as timeout handling is now done in a live engine.
11 years ago
Victor Julien 8e86f387a6 flow-time: use live threads at shutdown
Update pktacq loop to process flow timeouts in a running engine.

Add a new step to the shutdown phase of packet acquisition loop
threads (pktacqloop).

The shutdown code lets the pktacqloop break out of it's packet
acquisition loop. The thread then enters a flow timeout loop, where
it processes packets from it's tv->stream_pq queue until it's
empty _and_ the KILL flag is set.

Make sure receive threads are done before moving on to flow hash
cleanup (recycle all). Without this the flow recycler could start
it's unconditional hash clean up while detect threads are still
running on the flows.

Update unix socket to match live modes.
11 years ago
Victor Julien c6ec92d9b1 flow-timeout: use live threads
Use live threads. Disable old timeout code.
11 years ago
Victor Julien 48eccf7d91 Assign thread_id to flow on first packet stream engine 11 years ago
Victor Julien 8c51b23e94 Thread registration: id's start at 1
Start thread id's at 1, so that in flow's we can use 0 to indicate
a thread id hasn't been set in it yet.
11 years ago
Victor Julien 7f80516563 Introduce Flow timeout injection api
Add function TmThreadsInjectPacketById that is to be used to inject flow
timeout packets into the threads stream_pq queue.

TmThreadsInjectPacketById will also wake up listening threads if
applicable.

Packets are passed all packets together in an NULL terminated array
to reduce locking overhead.
11 years ago
Victor Julien 51a782fd8c Define FlowThreadId and add it to the flow
16 bits id should be enough for threads for a while.
11 years ago
Victor Julien a260cba32b Give easy access for thread stream packet queue
Access it from ThreadVars. This allows for easy injection of packets
into the stream engine.
11 years ago
Victor Julien 489ee20560 Thread Registration API for ID's
Create thread registration and unregistration API for assigning unique
thread id's.

Threadvars is static even if a thread restarts, so we can do the
registration before the threads start.

A thread is unregistered when the ThreadVars are freed.
11 years ago
Duarte Silva e586644c25 Fix and improvements
- Added/removed missing/superfluous util-memrchr.h include
- Improved the extraction of a IP from the XFF chain of IPs
11 years ago
Duarte Silva 68f43ffffb Implemented the diferent behaviour depending on the proxy deployment
- In forward deployment mode the first IP will be returned
- In reverse deployment mode the last IP will be retuned
11 years ago
Duarte Silva 496200dd08 Prepared everything for the proxy deployment configuration
- Added the suricata.yaml configurations and updated the comments
- Renamed the field in the configuration structure to something generic
- Added two new constants and the warning codes
11 years ago
Duarte Silva 4e04cd2d1b Adding XFF support to EVE alert output
- Created app-layer-htp-xff.c and app-layer-htp-xff.h
- Added entries in the Makefile.am
- Added the necessary configuration options to EVE alert section
- Updated Unified2 XFF configuration comments and removed unnecessary whitespace
- Created a generic function to parse the configuration
- Release the flow locks sooner and remove debug logging
- Added XFF support to EVE alert output
11 years ago
DIALLO David 0bdf494b54 fix Cygwin build fails: array subscript has type char 11 years ago
DIALLO David bfc871ce85 Update AppLayerProtoDetectPrintProbingParsers with Modbus protocol 11 years ago
DIALLO David 83d9834e77 fix CID 1257762: Logically dead code(DEADCODE) 11 years ago
Victor Julien 84e8217fd8 unix-socket: allow socked in custom locations
Allow the socket to be set in any location. This allows for easy
setting up of a socket as a non-root user.
11 years ago
Victor Julien b978730486 unix-socket: fix restart/shutdown cycle
When cleaning up after a pcap was processed, the stats api was cleaned
up before the stats threads were killed, leading to a BUG_ON triggering.
11 years ago
Ken Steele 3f3481e4d2 Fix indentation 11 years ago
Ken Steele 3f86c5a83f Fix memory leak in ac-tile
Incorrectly reallocing the goto table after it was freed by calling
SCACTileReallocState() when really only want to realloc the output table.
This was causing a large goto table to be allocated and never used or
freed.
11 years ago
Ken Steele b9e20ab4b8 Clean up memory leaks in ac-tile code
Free some memory at exit that was not getting freed.

Change pid_pat_list to store copy of case-strings in the same block
of memory as the array of pointers.
11 years ago
Ken Steele 1faa94c314 Make bad copy-mode be an error in runmode-tile. 11 years ago
Victor Julien c779065d35 Bug 1329: error out on invalid rule protocol
Due to a logic error in AppLayerProtoDetectGetProtoByName invalid
protocols would not be detected as such. Instead of ALPROTO_UNKNOWN
ALPROTO_MAX was returned.

Bug #1329
11 years ago
Eric Leblond 9f22c878e8 unix-manager: fix cppcheck errors
This patch fixes the following errors:
 [src/unix-manager.c:306]: (error) Memory pointed to by 'client' is freed twice.
 [src/unix-manager.c:313]: (error) Memory pointed to by 'client' is freed twice.
 [src/unix-manager.c:323]: (error) Memory pointed to by 'client' is freed twice.
 [src/unix-manager.c:334]: (error) Memory pointed to by 'client' is freed twice.

Unix manager was treating the packet after closing the socket if message was
too long.
11 years ago
Victor Julien 096b85ab68 stream: don't send EOF to AppLayer too soon
Sending EOF too soon results in the AppLayer cleaning up prematurely.
11 years ago
Victor Julien 0bb2b15491 ipv6: check for MLD messages with HL not 1
MLD messages should have a hop limit of 1 only. All others are invalid.

Written at MLD talk of Enno Rey, Antonios Atlasis & Jayson Salazar during
Deepsec 2014.
11 years ago
Ken Steele 68e6c4e94b Correct flow memory usage bookkeeping error
Fix bug 1321 where flow_memuse was incremented more on allocation than
free.
11 years ago
Victor Julien d951de2f19 Bug 977: -T / --init-errors-fatal to process all rules
Have -T / --init-errors-fatal process all rules so that it's easier
to debug problems in ruleset. Otherwise it can be a lengthy fix, test
error cycle if multiple rules have issues.

Convert empty rulefile error into a warning.

Bug #977
11 years ago
Victor Julien e951afb911 afpacket: only check offloading once per iface
Instead of once per thread per iface.
11 years ago
Victor Julien fa10811585 ioctl: make all string args const pointers 11 years ago
Victor Julien c3c144d504 http: don't crash when normalizing uri on low memory 11 years ago
Victor Julien f8f2ff49de defrag: don't crash when out of memory
Handle memory allocation errors in defrag better. Could lead to
crashes if malloc errors happened.
11 years ago
Eric Leblond 56373e5b34 af-packet: no more threads than RSS queues
If we manage to read the number of RSS queues from an interface,
this means that the optimal number of capture threads is equal
to the minimum of this number and of the number of cores on the
system.

This patch implements this logic thanks to the newly introduced
function GetIfaceRSSQueuesNum.
11 years ago
Eric Leblond 123c58af4b util-ioctl: add message in case of failure 11 years ago
Eric Leblond b4bb6e67ba util-ioctl: Add function to get number of RSS queues on iface
The number of RSS queues can be fetched via a standard ioctl which
is independant of hardware.
11 years ago
Victor Julien c174c9d779 af-packet: threads: auto, default to workers
Add a new default value for the 'threads:' setting in af-packet: "auto".
This will create as many capture threads as there are cores.

Default runmode of af-packet to workers.
11 years ago
Victor Julien 7b4987abc3 Runmode: handle value 'auto'
Auto now selects the default runmode for the capture method.
11 years ago
Victor Julien 234d18ab68 threading: remove '1slot' functions
No longer in use after the 'auto' runmode removal.

All runmodes now use either varslot or pktacqloop support.
11 years ago
Victor Julien 7025aabe75 Runmodes: remove 'auto' runmodes
Remove 'auto' runmodes from all capture methods. It wasn't reliable
enough, as it didn't enforce inspection order of packets.
11 years ago
Victor Julien 81c42f4916 log-stats: expand membuffer if necessary
Many threads could lead to a membuffer size requirement bigger than
64k. So use the expansion call to grow the buffer as needed.
11 years ago
Victor Julien 6277d2e0e4 MemBuffer: add expansion call
For some of the buffer users it's hard to predict how big the data
will be. In the stats.log case this depends on chosen runmode and
number of threads.

To deal with this case a 'MemBufferExpand' call is added. This realloc's
the buffer.
11 years ago
Victor Julien 75397ed750 stats: expose stats to Lua output
Register with type 'stats':

    function init (args)
        local needs = {}
        needs["type"] = "stats"
        return needs
    end

The stats are passed as an array of tables:

    { 1, { name=<name>, tmname=<tm_name>, value=<value>, pvalue=<pvalue>}}
    { 2, { name=<name>, tmname=<tm_name>, value=<value>, pvalue=<pvalue>}}
    etc

Name is the counter name (e.g. decoder.invalid), tm_name is the thread name
(e.g. AFPacketeth05), value is current value, and pvalue is the value of the
last time the script was invoked.
11 years ago
Victor Julien 5d95b08172 output streaming: cleanup at runmode destruction 11 years ago
Victor Julien 51a540c27e stats: disable stats if no loggers are enabled 11 years ago
Victor Julien 6252d24e0b stats: initialize after outputs
Initialize stats after outputs so that we can check if we need to
initialize the stats api at all.
11 years ago
Victor Julien a95c95f74c stats: introduce global config
As the stats api calls the loggers at a global interval, the global
interval should be configured globally.

 # global stats configuration
 stats:
   enabled: yes
   # The interval field (in seconds) controls at what interval
   # the loggers are invoked.
   interval: 8

If this config isn't found, the old config will be supported.
11 years ago
Victor Julien e98346b555 Introduce stats log API, convert existing output
Convert regular 'stats.log' output to this new API.

In addition to the current stats value, also give the last value. This
makes it easy to display the difference.
11 years ago
Victor Julien ee8da21e36 pcre: fix var capture for non relative matches
Var capture setup depended on the match being relative due to a logic
error.
11 years ago
Giuseppe Longo 1ad2a231fe pfring: fixes memleaks
This fixes some memory leaks
Bug #1184
11 years ago
Victor Julien 16941468ce lua: in streaming api, indicate open/close
The SCStreamingBuffer call now also returns two booleans:
    data, data_open, data_close = SCStreamingBuffer()

The first indicates this is the first data of this type for this
TCP session or HTTP transaction.

The second indicates this is the last data.

Ticket #1317.
11 years ago
Ken Steele 8dcd99209e Update copyright year in detect-flowbits files. 11 years ago
Ken Steele e6f83a586c DetectFlowintData - remove unused idx in TargetVar.
The idx inside TargetVar inside DetectFlowintData is never used, so remove
it.
11 years ago
Ken Steele c547d39152 Fix bug in DetectFlowintParse() - Assigning to both parts of a Union
sfd->target.value was always being set, even if the targettype was
not FLOWINT_TARGET_VAL. This would cause the tvar to be overwritten
with garbage data.
11 years ago
Ken Steele fa72082491 Don't write target.tvar.idx in DetectFlowintParse
Match functions should not be writing to the SigMatch context. So just use
a local variable instead.
11 years ago
Ken Steele 5cdb21ec34 Remove an unused define COUNTER_DETECT_ALERTS
The only place this exists in the code is when it is defined.
11 years ago
Ken Steele 18e2de320b Coding style cleanup in detect-modbus files. 11 years ago
Ken Steele ff41c1c452 Correct size increase in SigGroupHeadStore()
The code was increasing the size of the allocated memory by 16, but
only increasing the stored size by 10. Now uses one variable for both
places.
11 years ago
DIALLO David 55c5081240 Detect-engine: Add Modbus detection engine
Management of Modbus Tx

Based on DNS source code.

Signed-off-by: David DIALLO <diallo@et.esia.fr>
11 years ago
DIALLO David b3bf2f9939 Detect: Add Modbus keyword management
Add the modbus.function and subfunction) keywords for public function match in rules (Modbus layer).
Matching based on code function, and if necessary, sub-function code
or based on category (assigned, unassigned, public, user or reserved)
and negation is permitted.

Add the modbus.access keyword for read/write Modbus function match in rules (Modbus layer).
Matching based on access type (read or write),
and/or function type (discretes, coils, input or holding)
and, if necessary, read or write address access,
and, if necessary, value to write.
For address and value matching, "<", ">" and "<>" is permitted.

Based on TLS source code and file size source code (address and value matching).

Signed-off-by: David DIALLO <diallo@et.esia.fr>
11 years ago
DIALLO David 5a0409959f App-layer: Add Modbus protocol parser
Decode Modbus request and response messages, and extracts
MODBUS Application Protocol header and the code function.

In case of read/write function, extracts message contents
(read/write address, quantity, count, data to write).

Links request and response messages in a transaction according to
Transaction Identifier (transaction management based on DNS source code).

MODBUS Messaging on TCP/IP Implementation Guide V1.0b
(http://www.modbus.org/docs/Modbus_Messaging_Implementation_Guide_V1_0b.pdf)
MODBUS Application Protocol Specification V1.1b3
(http://www.modbus.org/docs/Modbus_Application_Protocol_V1_1b3.pdf)

Based on DNS source code.

Signed-off-by: David DIALLO <diallo@et.esia.fr>
11 years ago
Christophe M 6c2ae469be Fix to output a JSON buffer to an Unix domain socket.
Create the JSON buffer and write to it like regular file.

Upper function SCConfLogOpenGeneric already handle it properly.

Closes issue #1246.
11 years ago
Victor Julien 27007cc7d5 Fix Coverity issue in SMTP output
** CID 1250327:  Uninitialized pointer read  (UNINIT)
/src/output-json-email-common.c: 117 in JsonEmailLogJson()
/src/output-json-email-common.c: 139 in JsonEmailLogJson()
11 years ago
Victor Julien 7c3b22da22 smtp: don't create a new tx for rset/quit
A tx is considered complete after the data command completed. However,
this would lead to RSET and QUIT commands setting up a new tx.

This patch simply adds a check that refuses to setup a new tx when these
commands are encountered after the data portion is complete.
11 years ago
Victor Julien f7c2c219cd filestore: fix crash if keyword setup fails
SigMatch would be added to list, then the alproto check failed, leading
to freeing of sm. But as it was still in the list, the list now contained
a dangling pointer.
11 years ago
Victor Julien 9d2a0c39e5 mime: fix output issues
When multiple email addresses were in the 'to' field, sometimes
they would be logged as "\r\n \"Name\" <email>".

The \r\n was added by GetFullValue in the mime decoder, for unknown
reasons. Disabling this seems to have no drawbacks.
11 years ago
Victor Julien ebd6737b65 mime: fix compiler warning 11 years ago
Victor Julien 20a175f315 mime: improve error checking 11 years ago
Victor Julien 5461294a52 smtp: fix SMTPParserTest14 on 32bit 11 years ago
Victor Julien 9d33131d37 smtp: improve ProcessDataChunk error checking 11 years ago
Victor Julien d209699a41 smtp: expand tx use
Instead of just using TX for mime decoding, it is now also used for
tracking decoder events.
11 years ago
Victor Julien d67289b60e output-filedata: close files even w/o data
If there is no data chunk but the file is closed/truncated anyway,
logging is still required.
11 years ago
Victor Julien 08b06bac3f smtp: register file truncate callback
Tag files as truncated from this callback so storing/logging displays
the correct info.
11 years ago