Commit Graph

62 Commits (08fb4d790e469a71e2c4e154443ad387194973d7)

Author SHA1 Message Date
Victor Julien d6903e70c1 file-log: remove and add warning
Feature was deprecated and scheduled for removal.

Ticket #2376
6 years ago
magenbluten 1378f376a1 filestore v2: print sid in json output 6 years ago
Victor Julien dadac4ae4f eve/fileinfo: don't alloc filename during logging 6 years ago
Maurizio Abba bf4398b15d output-json: ensure string is json-encodable
Substitute json_string with SCJsonString custom function.
SCJsonString will ensure string passed is json-encodable (utf-8).
If it's not, the string will be converted in such a way that any
non-printable character will be encoded in its hex form.
The resulting json object will be returned.

rust modification will encode any non-printable character during its
conversion in to_cstring.
6 years ago
Jason Ish 95481a9176 eve/json/xff - remove check for flow being NULL.
Fix Coverity issue:
** CID 1435535:  Null pointer dereferences  (REVERSE_INULL)
/src/output-json-file.c: 212 in JsonBuildFileInfoRecord()

Where we check a variable for being NULL, when all paths to the
code show that it can't be NULL.
7 years ago
Jason Ish 36ec1281b2 eve/files: use eve-level xff config by default
The files section can still have an xff configuration which
will take priority over the eve level xff config.
7 years ago
Maurizio Abba 2543930d74 xff: Use XFF configuration in eve and filestore
XFF configuration is already set in app-layer-htp-xff, and in
output-json-alert. Extending XFF configuration to files and HTTP allow
to get the same behavior as for alerts.

Extend the configuration of filestore json to let filestore metafile
dump be aware of xff. This is available only if write-fileinfo is set
to yes and file-store version is 2.
7 years ago
Eric Leblond 1012fc4466 file: update logger API to log direction
By adding the flow direction to the logger we can have an accurate
logging of fileinfo events that has source and destination IP
correctly set.
7 years ago
Victor Julien 67f0e27ca4 smb: add smb records to fileinfo 7 years ago
Victor Julien 710c7b821f output/json: update callers to use explicit directions 7 years ago
Victor Julien 44c4008f77 output/json: clean up CreateJSONHeader calls 7 years ago
Jason Ish ccbe7401b6 output-json-file: let caller decide if file is stored
Mainly for the filestore module, which may have its own
knowledge of the file being stored before others.
7 years ago
Jason Ish 59bb98afcc eve/fileinfo: split record creation from writing
Split the building of the fileinfo record from the writing
of the record so the building can be called from other code.
Specifically the new filestore output which uses fileinfo
records as the metadata.
7 years ago
Jason Ish 00e6cd4ced output: introduce init return type
The new OutputInitResult is a struct return type that allows
logger init functions to return a NULL context without
raising error.

Instead of returning NULL to signal error, the "ok" field will
be set to false. If ok, but the ctx is NULL, then silently
move on to the next logger.

Use case: multiple versions of a specific logger, and one
implementation decides the configuration is not for that
implemenation. It can return NULL, ok.
7 years ago
Victor Julien e8dae2e093 nfs: add to fileinfo events 7 years ago
Victor Julien 58af39131f rust/nfs: handle GAPs
In normal records it will try to continue parsing.

GAP 'data' will be passed to file api as '0's. New call is used
so that the file API does know it is dealing with a GAP. Such
files are flagged as truncated at the end of the file and no
checksums are calculated.
7 years ago
Victor Julien 944ab48b20 file: clarify file store id name 8 years ago
Victor Julien ab1200fbd7 compiler: more strict compiler warnings
Set flags by default:

    -Wmissing-prototypes
    -Wmissing-declarations
    -Wstrict-prototypes
    -Wwrite-strings
    -Wcast-align
    -Wbad-function-cast
    -Wformat-security
    -Wno-format-nonliteral
    -Wmissing-format-attribute
    -funsigned-char

Fix minor compiler warnings for these new flags on gcc and clang.
8 years ago
Eric Leblond 569cc5d238 util-file: introduce new functions for file size
This patch introduces the FileDataSize and FileTrackedSize functions.
The first one is just a renaming of the initial FilSize function
whereas the other one is using the newly introduced size field as
value.
8 years ago
Eric Leblond a098896b28 output-json-file: use size instead of FileSize
FileSize is not returning the actual value when file store is not
used.
8 years ago
Victor Julien 810e43f373 magic: make optional
Make libmagic optional. If installed it will be enabled by default in
configure. Use --disable-libmagic to disable.
8 years ago
Victor Julien 9477fd4628 eve-file: fix -Wshadow warnings 8 years ago
Jason Ish 3fab684f97 logging: don't log that json is disabled in each logger
A warning log is already emitted if eve-log is enabled in the
configuration but json support is not built so the logger
registration functions can be silent.
8 years ago
Duarte Silva 53ebe4c538 file-hashing: added configuration options and common parsing code 8 years ago
Duarte Silva 89eb935f73 file-hashing: added support for SHA-256 file hashing 8 years ago
Duarte Silva a6d928e269 file-hashing: added support for SHA-1 file hashing 8 years ago
Jason Ish 1b4ba4496c logging: rename registration functions to not have tmm
As the logging modules are no longer threading modules, rename
them so they don't look like they are being registered as
threading modules.

Also, move the registration to the output.c which will handle
registration of the loggers.
8 years ago
Jason Ish fa27a76462 logging: add profiling back for non-tmm loggers
The loggers moved away from a TMM required a new
profiling support.
8 years ago
Jason Ish f9bb9029c5 logging: convert file logging to non-thread module 8 years ago
Victor Julien 2aac437927 output: reduce verbosity on info level 8 years ago
Victor Julien e43ce0a9ec file: switch to streaming buffer API
Make the file storage use the streaming buffer API.

As the individual file chunks were not needed by themselves, this
approach uses a chunkless implementation.
9 years ago
maxtors 69863f7b1c Corrected and unified debugmessages for init data errors in *ThreadInit. 9 years ago
Victor Julien c446abeb47 jansson: include in suricata-common.h 9 years ago
Victor Julien 8bb1cf08ef eve: fix mishandling of big messages
When the string representation of a JSON message grew bigger than
64k, the JSON record would just be truncated. This lead to errors
in the parser(s) of the JSON stream.

This patch changes the buffer logic to grow the buffer on demand.
9 years ago
Tom DeCanio 559747e325 file-store: add force-filestore configuration option to enable writing all
extracted files to filesystem.
9 years ago
Eric Leblond 538f37bd38 output-json: add app_proto key in root
By adding the key in the root of *flow and fileinfo  events it
will be possible to get all events for one application layer by
using a 'event_type:proto OR app_proto:proto' filter. This will
permit to the analyst to get a good view of events related to
one protocol.

This patch also fixes a regression in file logging where app_proto
was available before 94dbd303e4 create
the regression.
9 years ago
Eric Leblond 16b210f1ea json-file: avoid allocation 9 years ago
Eric Leblond 96412e8921 json-email: JsonEmailAddMetadata update
Add tx_id to the list of params to be in sync with recent changes.
9 years ago
Eric Leblond bf6b0145e2 json-http: gen metadata function with tx_id param 9 years ago
Eric Leblond d9b602fc0f json-smtp: add tx_id param to metadata generation
In all metadata generation contexts we know the tx_id so we better
used it to log the correct transaction and not an other one.
9 years ago
Eric Leblond 54038f5691 file-json: log 'email' information
Log information coming from email/MIME decoding in the message.
9 years ago
Eric Leblond 77119a3186 file-json: output smtp proto info 9 years ago
Eric Leblond 94dbd303e4 file-json: log http data using common function 9 years ago
Eric Leblond 1cd97713c2 file-json: add file_id to message
This will allow to get the filename and by consequence the file
after a parsing of the EVEV log file.
9 years ago
Victor Julien b7f1e9e370 file extract: add app_proto to logging 9 years ago
Eric Leblond 39d667ff56 output-json: fix type of data parameter
The cast of data to AlertJsonThread was not correct as the real
type of the void pointer is a OutputJsonCtx. This was working by
luck because they both have a file_ctx as first element.
10 years ago
Tom DeCanio 31f8f5cf20 eve-log: SMTP JSON logger 10 years ago
Ken Steele 8f1d75039a Enforce function coding standard
Functions should be defined as:

int foo(void)
{
}

Rather than:
int food(void) {
}

All functions where changed by a script to match this standard.
10 years ago
Victor Julien de4e2221d8 eve: add tx_id to output for alerts and events
Add tx_id field for correlating alerts and events per tx.
10 years ago
Victor Julien 8ba8c0bf6f json output: don't set 'unknown' for missing data
Instead of setting 'unknown' or '<unknown>' just pass NULL to json_*
function, which results in omitting the data.
11 years ago