Commit Graph

5380 Commits (0b28943487424f4831072a7161b33ebb5fc22211)
 

Author SHA1 Message Date
Victor Julien 31eea0f143 output-lua: TxLogger use proper stack function
Use proper wrapper to setup the stack.
10 years ago
Victor Julien 329f55598f output-lua: improve error handling and documentation
Better document the various functions and improve error handling.
10 years ago
Victor Julien c5ff94a319 output-lua: register common callbacks
Clean up callback registration in the setup-stage and register
common callbacks.
10 years ago
Victor Julien 0070aef3d1 output-lua: support File logging
Add file logger support. The script uses:

function init (args)
    local needs = {}
    needs['type'] = 'file'
    return needs
end

The type is set to file to make it a file logger.
10 years ago
Victor Julien 1517a2ca0e output-lua: rename LuaPacketLogger to ..Alerts
As the script is called for each alert, not for each packet, name
the script LuaPacketLoggerAlerts.
10 years ago
Victor Julien fe6cf00a8a output-lua: add stack utility functions
Add utility functions for placing things on the stack for use
by the scripts. Functions for numbers, strings and byte arrays.

Add callback for returing IP header info: ip version, src ip,
dst ip, proto, sp, dp (or type and code for icmp and icmpv6):
SCPacketTuple
10 years ago
Victor Julien 53d7f800bf output-lua: initial packet support
Add key for storing packet pointer in the lua stack and a utility
function to retrieve it from lua callbacks.
10 years ago
Victor Julien 15052e58a2 output-lua: move LuaPrintStack to common
It's a utility function that will be used in several other places
as well.
10 years ago
Victor Julien 684afc7f4e output-lua: add example packet log script
Example packet log script that outputs to stdout in the alert-
fast log format.
10 years ago
Victor Julien b60e28e1a4 output-lua: packet logger support
Through 'needs' the script init function can indicate it wants to
see packets and select a condition function. Currently only alerts
is an option:

    function init (args)
        local needs = {}
        needs["type"] = "packet"
        needs["filter"] = "alerts"
        return needs
    end
10 years ago
Victor Julien 0bd4b9beca output-lua: new file for common functions
Add output-lua-common.[ch] to store functions common to various parts
of the lua output framework.
10 years ago
Victor Julien db30ed8c3e output: Lua HTTP log initial implementation
Initial version of a HTTP LUA logger. Execute lua scripts from the
Tx-log API.
10 years ago
Victor Julien 95e0eae69a output-lua: support submodules
Use the OutputCtx::submodules list to register additional log modules.
Currently this is hardcoded to the 'lua' module.
10 years ago
Victor Julien 1fd0f96b49 output-lua: display warning if no lua support
Display a warning that the lua module is not available if we're
not compiled against lua(jit).
10 years ago
Victor Julien eb5a70fe09 output: add submodules list to OutputCtx
Add a list to the OutputCtx that can contain OutputModule structures.
This will be used by a 'parent' module to register submodules directly.
10 years ago
Victor Julien 6493554663 streaming: pass tx_id to logger
This way we can distinguish between various tx' in the logger.
10 years ago
Victor Julien cd78705e3a streaming-loggers: add configuration
Add a (disabled by default) config to the yaml
10 years ago
Victor Julien ac2ef45a3d tcp-data-log: file and dir logging modes
Add a file logging mode, which logs all the data into a single log file.

Also, make the directory logging more configurable.
10 years ago
Victor Julien 3dec0e96f8 tcp-data: new streaming logger
tcp-data logs out reassembled stream data in a streaming fashion.

Records type to log into different directories.
10 years ago
Victor Julien bac6c3ab02 streaming logger: support Http Body logging
Add an argument to the registration to indicate which iterator
needs to be used: Stream or HttpBody

Add HttpBody Iterator, calling the logger(s) for each Http body chunk.
10 years ago
Victor Julien ab6fac884d output-streaming: StreamIterator
StreamIterator implementation for iterating over ACKed segments.

Flag each segment as logged when the log function has been called for it.

Set a 'OPEN' flag for the first segment in both directions.

Set a 'CLOSE' flag when the stream ends. If the last segment was already
logged, a empty CLOSE call is performed with NULL data.
10 years ago
Victor Julien 9d9ef983dd output-streaming: a Log API for streaming data
This patch adds a new Log API for streaming data such as TCP reassembled
data and HTTP body data. It could also replace Filedata API.

Each time a new chunk of data is available, the callback will be called.
10 years ago
Ken Steele fdcc7d18e7 Fix compiler warning about uninitialized variable in mpipe. 10 years ago
Duarte Silva 3a18db13dc Simple code fixes
- Removed unnecessary assignment of the data field
- Removed else condition (same function called for IPv4 and IPV6)
- Fixed constants to be a power of two (used in bitwise operations)
10 years ago
Victor Julien c20bd3bcb2 Optimize Packet Ext data freeing
Move the logic of PacketFreeExtData into a macro 'PACKET_FREE_EXTDATA'.
It was called for each packet.
10 years ago
Eric Leblond 80adc40f68 packet pool: fix ext_pkt cleaning
The field ext_pkt was cleaned before calling the release function.
The result was that IPS mode such as the one of AF_PACKET were not
working anymore because they were not able to send the data which
were initially pointed by ext_pkt.

This patch moves the ext_pkt cleaning to the cleaning macro. This
ensures that the cleaning is done for allocated and pool packets.
10 years ago
Victor Julien 3ee504a3dc packet recycle: remove mutex destroy/init
This was necessary earlier when there was a memset involved.
10 years ago
Victor Julien ed0b75e1e9 packet recycle: do most clean up on packet reuse
Call PACKET_RELEASE_REFS from PacketPoolGetPacket() so that
we only access the large packet structure just before actually
using it. Should give better cache behaviour.
10 years ago
Victor Julien 231b993f1f packet recycle: split macro
Split PACKET_RECYCLE into 2 parts. One part for cleanup to do before a
packet is returned to the pool, the other after it's retrieved from
the pool.
10 years ago
Victor Julien 04a0672f7a Fix decode tests calling PACKET_DO_RECYCLE instead of PACKET_RECYCLE 10 years ago
Victor Julien 7fa2b8769b Update Changelog for 2.1beta1 10 years ago
Victor Julien 2bcff80d32 Update Changelog for 2.0.3 10 years ago
Ken Steele 44aeb9c637 Fix GRE Source Routing Header definition
The Source Routing Header had routing defined as a char* for a field
of variable size. Since that field was not being used in the code, I
removed the pointer and added a comment.
10 years ago
Ken Steele c91b7fd3bc More structures that need to be marked Packed.
These structures are cast from raw packet data, so they should be packed.
The case is:

typedef struct Foo_ {
} Foo;

Foo *f = (Foo *)pkt;
10 years ago
Ken Steele 045966676d Add Packed attribute on Header structures
Structures that are used to cast packet data into fields need to be packed
so that the compiler doesn't add any padding to these fields. This also helps
Tile-Gx to avoid unaligned loads because the compiler will insert code to
handle the possible unaligned load.
10 years ago
Victor Julien da1fe75975 lua: improve configure checks
The base 'lua' library has different names on different OS' and even
Linux distro's. Instead of selecting the proper one, we now just try
all. This way no OS/distro specific knowledge about the name is needed.
10 years ago
Victor Julien f453fb810f alert-json: clean up flags
Make payload/packet logging code also use the flags field in
AlertJsonOutputCtx, instead of in the LogFileCtx.
10 years ago
Giuseppe Longo b188d93630 json-alert: include HTTP info on the alerts
Extends the JSON alert output to include the HTTP data
at the time of the alert.
10 years ago
Giuseppe Longo 288f0b1fb7 json-http: refactoring output code
Splits the output code in two public functions and permits
to call them from the alert function
10 years ago
Duarte Silva db9588a2ce Use extra data structure in json alert output
to store local configuration
10 years ago
Victor Julien 6b0ff0193d stream: detect and filter out bad window updates
Reported in bug 1238 is an issue where stream reassembly can be
disrupted.

A packet that was in-window, but otherwise unexpected set the
window to a really low value, causing the next *expected* packet
to be considered out of window. This lead to missing data in the
stream reassembly.

The packet was unexpected in various ways:
- it would ack unseen traffic
- it's sequence number would not match the expected next_seq
- set a really low window, while not being a proper window update

Detection however, it greatly hampered by the fact that in case of
packet loss, quite similar packets come in. Alerting in this case
is unwanted. Ignoring/skipping packets in this case as well.

The logic used in this patch is as follows. If:

- the packet is not a window update AND
- packet seq > next_seq AND
- packet acq > next_seq (packet acks unseen data) AND
- packet shrinks window more than it's own data size
THEN set event and skip the packet in the stream engine.

So in case of a segment with no data, any window shrinking is rejected.

Bug #1238.
10 years ago
Victor Julien 7cc63918c3 ipv6: fix dst/hop header option parsing
The extension header option parsing used a uint8_t internally. However
much bigger option sizes are valid.
10 years ago
Victor Julien 2b84cd9483 defrag: use 'struct timeval' for timeout tracking
Until now the time out handling in defrag was done using a single
uint32_t that tracked seconds. This lead to corner cases, where
defrag trackers could be timed out a little too early.
10 years ago
Victor Julien 7c05685421 ipv6: set event on unsupported nh
If a next header / protocol is encountered that we can't handle (yet)
set an event. Disabled the rule by default.

    decode-event:ipv6.unknown_next_header;
10 years ago
Victor Julien bbcdb657da ipv6: more robust ipv6 exthdr handling
Skip past Shim6, HIP and Mobility header.

Detect data after 'none' header.
    decode-event:ipv6.data_after_none_header;
10 years ago
Victor Julien 938602c55e ipv6: detect frag header reserved field non-zero
Frag Header length field is reserved, and should be set to 0.

    decode-event:ipv6.fh_non_zero_reserved_field;
10 years ago
Victor Julien 8c19e5ff63 ipv6: make exthdr parsing more robust
Improve data length checks. Detect PadN option with 0 length.
10 years ago
Victor Julien abee95ca4f ipv6: set flag on type 0 routing header
Type 0 Routing headers are deprecated per RFC 5095.

This patch sets an decode event flag that can be matched on through:
    decode-event:ipv6.rh_type_0;
10 years ago
Victor Julien 83b031b4e0 ipv6 defrag: fix unfragmentable exthdr handling
Fix or rather implement handling of unfragmentable exthdrs in ipv6.
The exthdr(s) appearing before the frag header were copied into the
reassembled packet correctly, however the stripping of the frag header
did not work correctly.

Example:
The common case is a frag header directly after the ipv6 header:

[ipv6 header]->[frag header]->[icmpv6 (part1)]
[ipv6 header]->[frag header]->[icmpv6 (part2)]

This would result in:
[ipv6 header]->[icmpv6]

The ipv6 headers 'next header' setting would be updated to point to
whatever the frag header was pointing to.

This would also happen when is this case:

[ipv6 header]->[hop header]->[frag header]->[icmpv6 (part1)]
[ipv6 header]->[hop header]->[frag header]->[icmpv6 (part2)]

The result would be:
[ipv6 header]->[hop header]->[icmpv6]

However, here too the ipv6 header would have been updated to point
to what the frag header pointed at. So it would consider the hop header
as if it was an ICMPv6 header, or whatever the frag header pointed at.

The result is that packets would not be correctly parsed, and thus this
issue can lead to evasion.

This patch implements handling of the unfragmentable part. In the first
segment that is stored in the list for reassembly, this patch detects
unfragmentable headers and updates it to have the last unfragmentable
header point to the layer after the frag header.

Also, the ipv6 headers 'next hdr' is only updated if no unfragmentable
headers are used. If they are used, the original value is correct.

Reported-By: Rafael Schaefer <rschaefer@ernw.de>

Bug #1244.
10 years ago
Victor Julien e66c73abcd packet pool: init pthread key before using it
In the packet pool code, it's critical to initialize the pthread key
before using it. Applies only to the code used if __thread isn't
supported.
10 years ago