Commit Graph

6452 Commits (suricata-3.0.1)
 

Author SHA1 Message Date
Victor Julien f7d890fe00 lua-output: add SCStreamingBuffer
Add SCStreamingBuffer lua function to retrieve the data passed
to the script per streaming API invocation.

Example:

    function log(args)
        data = SCStreamingBuffer()
        hex_dump(data)
    end
11 years ago
Victor Julien ca3be77008 output-lua: add support for streaming api
Add support to lua output for the streaming api. This allows for a
script to subscribe itself to streaming tcp data and http body data.
11 years ago
Victor Julien efb5c29698 output-lua: give access to packet payload
Add SCPacketPayload()

Example:
    function log(args)
        p = SCPacketPayload()
        print(p)
    end
11 years ago
Victor Julien 08b0d9a5b4 output-lua: expose http body data
Make normalized body data available to the script through
HttpGetRequestBody and HttpGetResponseBody.

There no guarantees that all of the body will be availble.

Example:
    function log(args)
        a, o, e = HttpGetResponseBody();
        --print("offset " .. o .. " end " .. e)

        for n, v in ipairs(a) do
            print(v)
        end
    end
11 years ago
Victor Julien 8360b707e8 output-lua: add HttpGetRequestHost callback
Get the host from libhtp's tx->request_hostname, which can either be
the host portion of the url or the host portion of the Host header.

Example:

    http_host = HttpGetRequestHost()
    if http_host == nil then
        http_host = "<hostname unknown>"
    end
11 years ago
Victor Julien a234a335ac output-lua: http alproto check 11 years ago
Victor Julien cb69cee4d8 output-lua: clean up flow lock handling 11 years ago
Victor Julien 19383fd428 output-lua: alproto string callback
SCFlowAppLayerProto: get alproto as string from the flow. If alproto
is not (yet) known, it returns "unknown".

    function log(args)
        alproto = SCFlowAppLayerProto()
        if alproto ~= nil then
            print (alproto)
        end
    end
11 years ago
Victor Julien 22dd14d560 output-lua: expose thread info
A new callback to give access to thread id, name and group name:
SCThreadInfo. It gives: tid (integer), tname (string), tgroup (string)

    function log(args)
        tid, tname, tgroup = SCThreadInfo()
11 years ago
Victor Julien 8802ba3f67 output-lua: expose flow start time string
SCFlowTimeString: returns string form of start time of a flow

Example:

    function log(args)
        startts = SCFlowTimeString()
        ts = SCPacketTimeString()
        if ts == startts then
            print("new flow")
        end
11 years ago
Victor Julien 07ff85a44e output-lua: add file callbacks
SCFileInfo: returns fileid (number), txid (number), name (string),
            size (number), magic (string), md5 in hex (string)

Example:

    function log(args)
        fileid, txid, name, size, magic, md5 = SCFileInfo()

SCFileState: returns state (string), stored (bool)

Example:
    function log(args)
        state, stored = SCFileState()
11 years ago
Victor Julien 3343060d85 output-lua: add SCPacketTimeString
Add SCPacketTimeString to get the packets time string in the format:
    11/24/2009-18:57:25.179869

Example use:

    function log(args)
        ts = SCPacketTimeString()
11 years ago
Victor Julien b3dfd3cd8e output-lua: rule info callback
SCRuleIds(): returns sid, rev, gid:

    function log(args)
        sid, rev, gid = SCRuleIds()

SCRuleMsg(): returns msg

    function log(args)
        msg = SCRuleMsg()

SCRuleClass(): returns class msg and prio:

    function log(args)
        class, prio = SCRuleClass()
        if class == nil then
            class = "unknown"
        end
11 years ago
Victor Julien d9efa7048a lua: add SCFlowTuple lua function
Like SCPacketTuple, only retrieves Tuple from the flow.

Minimal log function:

    function log(args)
        ipver, srcip, dstip, proto, sp, dp = SCFlowTuple()
        print ("Flow IPv" .. ipver .. " src " .. srcip .. " dst " .. dstip ..
               " proto " .. proto .. " sp " .. sp .. " dp " .. dp)
    end
11 years ago
Victor Julien f2da5dbbad detect-lua: convert extensions to use flow wrappers
Use the new flow wrapper functions in the lua flowvar and flowint
extensions.
11 years ago
Victor Julien affbd697ed lua: add flow store and retrieval wrappers
Add flow store and retrieval wrappers for accessing the flow through
Lua's lightuserdata method.

The flow functions store/retrieve a lock hint as well.
11 years ago
Victor Julien 599ec36b2c lua: introduce util-lua.[ch]
Shared functions for all lua parts of the engine.
11 years ago
Victor Julien 8bc01af581 output-lua: add all packets logging support
If the script needing a packet doesn't specify a filter, it will
be run against all packets. This patch adds the support for this
mode. It is a packet logger with a condition function that always
returns true.
11 years ago
Victor Julien fe3484fbc0 output-lua: improve error checking for init()
If init doesn't properly init the script, skip the script and error
out.
11 years ago
Victor Julien 0055a10b3a output-log: expose SCLog functions to lua scripts
The lua scripts can use SCLogDebug, SCLogInfo, SCLogNotice, SCLogWarning,
SCLogError. The latter 2 won't be able to add an error code though.
11 years ago
Victor Julien 51ab5e55c1 output-lua: make packet ptr available to all scripts
TxLogger and Packet logger need it to be able to use the Tuple
callback.
11 years ago
Victor Julien 1e836be3d8 output-lua: add SCLogPath callback
Add a lua callback for getting Suricata's log path, so that lua scripts
can easily get the logging directory Suricata uses.

Update the Setup logic to register callbacks before the scripts 'setup'
is called.

Example:

    name = "fast_lua.log"
    function setup (args)
        filename = SCLogPath() .. "/" .. name
        file = assert(io.open(filename, "a"))
    end
11 years ago
Victor Julien 31eea0f143 output-lua: TxLogger use proper stack function
Use proper wrapper to setup the stack.
11 years ago
Victor Julien 329f55598f output-lua: improve error handling and documentation
Better document the various functions and improve error handling.
11 years ago
Victor Julien c5ff94a319 output-lua: register common callbacks
Clean up callback registration in the setup-stage and register
common callbacks.
11 years ago
Victor Julien 0070aef3d1 output-lua: support File logging
Add file logger support. The script uses:

function init (args)
    local needs = {}
    needs['type'] = 'file'
    return needs
end

The type is set to file to make it a file logger.
11 years ago
Victor Julien 1517a2ca0e output-lua: rename LuaPacketLogger to ..Alerts
As the script is called for each alert, not for each packet, name
the script LuaPacketLoggerAlerts.
11 years ago
Victor Julien fe6cf00a8a output-lua: add stack utility functions
Add utility functions for placing things on the stack for use
by the scripts. Functions for numbers, strings and byte arrays.

Add callback for returing IP header info: ip version, src ip,
dst ip, proto, sp, dp (or type and code for icmp and icmpv6):
SCPacketTuple
11 years ago
Victor Julien 53d7f800bf output-lua: initial packet support
Add key for storing packet pointer in the lua stack and a utility
function to retrieve it from lua callbacks.
11 years ago
Victor Julien 15052e58a2 output-lua: move LuaPrintStack to common
It's a utility function that will be used in several other places
as well.
11 years ago
Victor Julien 684afc7f4e output-lua: add example packet log script
Example packet log script that outputs to stdout in the alert-
fast log format.
11 years ago
Victor Julien b60e28e1a4 output-lua: packet logger support
Through 'needs' the script init function can indicate it wants to
see packets and select a condition function. Currently only alerts
is an option:

    function init (args)
        local needs = {}
        needs["type"] = "packet"
        needs["filter"] = "alerts"
        return needs
    end
11 years ago
Victor Julien 0bd4b9beca output-lua: new file for common functions
Add output-lua-common.[ch] to store functions common to various parts
of the lua output framework.
11 years ago
Victor Julien db30ed8c3e output: Lua HTTP log initial implementation
Initial version of a HTTP LUA logger. Execute lua scripts from the
Tx-log API.
11 years ago
Victor Julien 95e0eae69a output-lua: support submodules
Use the OutputCtx::submodules list to register additional log modules.
Currently this is hardcoded to the 'lua' module.
11 years ago
Victor Julien 1fd0f96b49 output-lua: display warning if no lua support
Display a warning that the lua module is not available if we're
not compiled against lua(jit).
11 years ago
Victor Julien eb5a70fe09 output: add submodules list to OutputCtx
Add a list to the OutputCtx that can contain OutputModule structures.
This will be used by a 'parent' module to register submodules directly.
11 years ago
Victor Julien 6493554663 streaming: pass tx_id to logger
This way we can distinguish between various tx' in the logger.
11 years ago
Victor Julien cd78705e3a streaming-loggers: add configuration
Add a (disabled by default) config to the yaml
11 years ago
Victor Julien ac2ef45a3d tcp-data-log: file and dir logging modes
Add a file logging mode, which logs all the data into a single log file.

Also, make the directory logging more configurable.
11 years ago
Victor Julien 3dec0e96f8 tcp-data: new streaming logger
tcp-data logs out reassembled stream data in a streaming fashion.

Records type to log into different directories.
11 years ago
Victor Julien bac6c3ab02 streaming logger: support Http Body logging
Add an argument to the registration to indicate which iterator
needs to be used: Stream or HttpBody

Add HttpBody Iterator, calling the logger(s) for each Http body chunk.
11 years ago
Victor Julien ab6fac884d output-streaming: StreamIterator
StreamIterator implementation for iterating over ACKed segments.

Flag each segment as logged when the log function has been called for it.

Set a 'OPEN' flag for the first segment in both directions.

Set a 'CLOSE' flag when the stream ends. If the last segment was already
logged, a empty CLOSE call is performed with NULL data.
11 years ago
Victor Julien 9d9ef983dd output-streaming: a Log API for streaming data
This patch adds a new Log API for streaming data such as TCP reassembled
data and HTTP body data. It could also replace Filedata API.

Each time a new chunk of data is available, the callback will be called.
11 years ago
Ken Steele fdcc7d18e7 Fix compiler warning about uninitialized variable in mpipe. 11 years ago
Duarte Silva 3a18db13dc Simple code fixes
- Removed unnecessary assignment of the data field
- Removed else condition (same function called for IPv4 and IPV6)
- Fixed constants to be a power of two (used in bitwise operations)
11 years ago
Victor Julien c20bd3bcb2 Optimize Packet Ext data freeing
Move the logic of PacketFreeExtData into a macro 'PACKET_FREE_EXTDATA'.
It was called for each packet.
11 years ago
Eric Leblond 80adc40f68 packet pool: fix ext_pkt cleaning
The field ext_pkt was cleaned before calling the release function.
The result was that IPS mode such as the one of AF_PACKET were not
working anymore because they were not able to send the data which
were initially pointed by ext_pkt.

This patch moves the ext_pkt cleaning to the cleaning macro. This
ensures that the cleaning is done for allocated and pool packets.
11 years ago
Victor Julien 3ee504a3dc packet recycle: remove mutex destroy/init
This was necessary earlier when there was a memset involved.
11 years ago
Victor Julien ed0b75e1e9 packet recycle: do most clean up on packet reuse
Call PACKET_RELEASE_REFS from PacketPoolGetPacket() so that
we only access the large packet structure just before actually
using it. Should give better cache behaviour.
11 years ago