Restructure code to make it clearer that either 'basic', 'extended'
or 'custom' is being printed, by creating one function for each of
the possibilities.
Don't log a session as "resumed" if a ServerHello record has not been
seen. This makes sure that incomplete TLS sessions where the ClientHello
contains a session ticket, is not logged as a session resumption.
The new OutputInitResult is a struct return type that allows
logger init functions to return a NULL context without
raising error.
Instead of returning NULL to signal error, the "ok" field will
be set to false. If ok, but the ctx is NULL, then silently
move on to the next logger.
Use case: multiple versions of a specific logger, and one
implementation decides the configuration is not for that
implemenation. It can return NULL, ok.
Set flags by default:
-Wmissing-prototypes
-Wmissing-declarations
-Wstrict-prototypes
-Wwrite-strings
-Wcast-align
-Wbad-function-cast
-Wformat-security
-Wno-format-nonliteral
-Wmissing-format-attribute
-funsigned-char
Fix minor compiler warnings for these new flags on gcc and clang.
We assume session resumption has occurred if the Client Hello message
included a session id, we have not seen the server certificate, but
we have seen a Change Cipher Spec message from the server.
Previously, these transactions were not logged at all because the
server cert was never seen.
Ticket: https://redmine.openinfosecfoundation.org/issues/1969
All loggers were wrapping just the write in a lock with some
updating a counter. This moves the lock into the write function.
The log_ctx alerts counter was also removed as many modules have
stopped using this and the alert count is available elsewhere.
Should satisfy Coverity CID 1400798:
CID 1400798 (#1 of 1): Data race condition (MISSING_LOCK) 2.
missing_lock: Accessing log_ctx->rotation_flag without holding lock
LogFileCtx_.fp_mutex. Elsewhere, "LogFileCtx_.rotation_flag" is accessed
with LogFileCtx_.fp_mutex held 4 out of 5 times.
Which appears to be a false positive as all calls to SCLogFileWrite
were done under lock, but this will make it more explicit.
As the logging modules are no longer threading modules, rename
them so they don't look like they are being registered as
threading modules.
Also, move the registration to the output.c which will handle
registration of the loggers.
An design error was made when doing the TLS storage module which
has been made dependant of the TLS logging. At the time there was
only one TLS logging module but there is now two different ones.
By putting the TLS store module in a separate module, we can now
use EVE output and TLS store at the same time.
FreeBSD 10 32-bit with clang 3.3:
log-tlslog.c:172:14: error: format specifies type 'long' but the argument has type 'time_t' (aka 'int') [-Werror,-Wformat]
p->ts.tv_sec,
^~~~~~~~~~~~
1 error generated.
detect-engine-payload.c:508:27: warning: format specifies type 'long' but the argument has type 'time_t' (aka 'int') [-Wformat]
printf("%ld.%06ld\n", tv_diff.tv_sec, (long int)tv_diff.tv_usec);
~~~ ^~~~~~~~~~~~~~
%d
1 warning generated.
Pierre Chifflier pointed out that a rule like:
alert tls any any -> any any (msg:"TLS store"; tls.issuerdn:!"C=FR"; tls.store;)
was alerting but not storing the certificate. If the filter was
removed:
alert tls any any -> any any (msg:"TLS store"; tls.store;)
then tls.store is working as expected.
This was linked with fact that logging is only done once for a SSL
state. So without filter, once we have the info we can log and we
run the storage. But when there is a filter, we log and then there
is a filter analysis and alerting. And as logging as already be done
we don't enter in the logging function and there is no storage.
This patch forces the entrance in the log function when there is a
request for TLS storage. And it adds an exit in the logging function
to only do the storage part if the TLS state has already being logged.
The TM_FLAG_LOGAPI_TM flag indicates that a module is run by the log
api, not by the 'regular' thread module call functions.
Set flag in all all Log API users' registration code.
Purpose of this flag is in profiling. In profiling output it will be
used to list log api thread modules separately.
Both the drop and tls logs are currently not designed to have multiple
instances running. So until that is changed, error out if more than one
instance is started.
This patch converts log-tls to use the packet logger API. The packet
logger API was choosen as the TLS parser is not transaction aware.
To make sure the state is only logged once, the flag
SSL_AL_FLAG_STATE_LOGGED was added to the parser. This flag is checked
by the condition function, and set at the end of the Logger function.
Make all functions static. Remove separate ipv4 and ipv6 registration
functions. Move register function to the bottom so that we no longer
need function prototype declarations.
app-layer.[ch], app-layer-detect-proto.[ch] and app-layer-parser.[ch].
Things addressed in this commit:
- Brings out a proper separation between protocol detection phase and the
parser phase.
- The dns app layer now is registered such that we don't use "dnstcp" and
"dnsudp" in the rules. A user who previously wrote a rule like this -
"alert dnstcp....." or
"alert dnsudp....."
would now have to use,
alert dns (ipproto:tcp;) or
alert udp (app-layer-protocol:dns;) or
alert ip (ipproto:udp; app-layer-protocol:dns;)
The same rules extend to other another such protocol, dcerpc.
- The app layer parser api now takes in the ipproto while registering
callbacks.
- The app inspection/detection engine also takes an ipproto.
- All app layer parser functions now take direction as STREAM_TOSERVER or
STREAM_TOCLIENT, as opposed to 0 or 1, which was taken by some of the
functions.
- FlowInitialize() and FlowRecycle() now resets proto to 0. This is
needed by unittests, which would try to clean the flow, and that would
call the api, AppLayerParserCleanupParserState(), which would try to
clean the app state, but the app layer now needs an ipproto to figure
out which api to internally call to clean the state, and if the ipproto
is 0, it would return without trying to clean the state.
- A lot of unittests are now updated where if they are using a flow and
they need to use the app layer, we would set a flow ipproto.
- The "app-layer" section in the yaml conf has also been updated as well.
When http and/or tls logging is disabled, the app layer would still
be flagged as logging. This caused transactions not to be freed until
the end of the flow as the logged tx id would never increment.
This fix postpones the setting of the app layer parser "logger"
flag to the point where we know the logger is enabled.
There were 8 identical copies of CreateTimeString() in 8 files.
Most used SCLocalTime, to replace localtime_r(), but some did not.
Created one copy in util-time.c.
Improved accuracy, improved performance. Performance improvement
noticeable with http heavy traffic and ruleset.
A lot of other cosmetic changes carried out as well. Wrappers introduced
for a lot of app layer functions.
Failing dce unittests disabled. Will be reintroduced in the updated dce
engine.
Cross transaction matching taken care of. FPs emanating from these
matches have now disappeared. Double inspection of transactions taken
care of as well.
When handling error case on SCMallog, SCCalloc or SCStrdup
we are in an unlikely case. This patch adds the unlikely()
expression to indicate this to gcc.
This patch has been obtained via coccinelle. The transformation
is the following:
@istested@
identifier x;
statement S1;
identifier func =~ "(SCMalloc|SCStrdup|SCCalloc)";
@@
x = func(...)
... when != x
- if (x == NULL) S1
+ if (unlikely(x == NULL)) S1
Victor Julien
Jason Ish
Mats Klepsland
Jason Ish
Victor Julien
Ray Ruvinskiy
fooinha
Eric Leblond
Ken Steele
Jason Ish
Anoop Saldanha