This patch converts log-tls to use the packet logger API. The packet
logger API was choosen as the TLS parser is not transaction aware.
To make sure the state is only logged once, the flag
SSL_AL_FLAG_STATE_LOGGED was added to the parser. This flag is checked
by the condition function, and set at the end of the Logger function.
Make all functions static. Remove separate ipv4 and ipv6 registration
functions. Move register function to the bottom so that we no longer
need function prototype declarations.
This patch introduces a new API for logging transactions from
tx-aware app layer protocols. It runs all the registered loggers
from a single thread module. This thread module takes care of the
transaction handling and flow locking. The logger just gets a
transaction to log out.
All loggers for a protocol will be run at the same time, so there
will not be any timing differences.
Loggers will no longer act as Thread Modules in the strictest sense.
The Func is NULL, and SetupOuputs no longer attaches them to the
thread module chain individually. Instead, after registering through
OutputRegisterTxModule, the setup data is used in the single logging
module.
The logger (LogFunc) is called for each transaction once, at the end
of the transaction.
This patch introduces a new API for outputs that log based on the
packet, such as alert outputs. In converts fast-log to the new API.
The API gets rid of the concept of each logger being a thread module,
but instead there is one thread module that runs all packet loggers.
Through the registration function OutputRegisterPacketModule a log
module can register itself to be considered for each packet.
Each logger registers itself to this new API with 2 functions and the
OutputCtx object that was already used in the old implementation.
The function pointers are:
LogFunc: the log function
ConditionFunc: this function is called before the LogFunc and only
if this returns TRUE the LogFunc is called.
For a simple alert logger like fast-log, the condition function will
simply return TRUE if p->alerts.cnt > 0.
If a protocol parser is active without a logger when detection is
disabled, the transaction handling logic would fail. Now it will
return the proper tx id so we can clean up the complete transactions.
When running w/o detect, TX cleanup handling needs to ignore the
inspect_id as it's only updated by detect.
This patch introduces a new ActiveTx handler for logging only:
AppLayerTransactionGetActiveLogOnly
If --disable-detection is passed on the commandline, this function
is registered.
Flow timeout code keeps track of thread module running detect, and
fails (hard) if it doesn't find it.
This changeset retrieves the global g_detect_disabled and passes
it to the timeout handling code during setup.
This global will be set to TRUE if detect is disabled. The reason for
adding a global is that there currently is no clean way to pass
configuration options to management threads.
If FlowForceReassemblyForFlowV2 can't get packets to inject into the
engine, until now it would bail and retry later. In case of resource
starvation issues, this would cause a lot of lock contention, as the
flow manager would try over and over again.
This patch limits FlowForceReassemblyForFlowV2 to one try per flow,
if it fails... bad luck. It will only fail in serious conditions,
which means we must prefer the health of the engine over the proper
inspection of the flow in question.
AppLayer Proto detection code uses a mix of pattern search and
"probing parsers". The pattern search validates potential matches
using a single pattern search algo. The code was using SpmSearch
for this, but this made it inefficient as it builds a BoyerMoore
context for each search. This lead to significant memory pressure,
especially on high speed/bandwidth boxes.
This patch switches the search calls to BoyerMoore and BoyerMoore-
Nocase directly. This can be done as the ctx' were available already.
Move app layer event handling into app-layer-event.[ch].
Convert 'Set' macro's to functions.
Get rid of duplication in Set and SetRaw. Set now calls SetRaw.
Fix potentential int overflow condition in the event storage.
Update callers.
Don't use FlowGetProtoMapping at runtime, use f->protomap instead.
Add safety check to make sure its value is within range, as it's
used to index an array.
Update unittests to initialized flows (somewhat).
Remove unnecessay inlining.
Rename functions with wrong naming scheme. E.g. AllocAppLayer.. instead
of AppLayer..Alloc.
Use AppProto instead of uint16_t.
Convert u16 ipproto cases to u8.
This patch updates the DNS counters from the main AppLayer entry
functions. Due to the limited scope of AppLayerThreadCtx some of
the logic had to be implemented in app-layer.c, where it doesn't
belong.
To be able to register counters from AppLayerGetCtxThread, the
ThreadVars pointer needs to be available in it and thus in it's
callers:
- AppLayerGetCtxThread
- DecodeThreadVarsAlloc
- StreamTcpReassembleInitThreadCtx
Add memuse tracking and memcap checking to the DNS parsers. Memuse
is tracked globally and per flow (state).
Memcaps are also checked per flow and globally before memory allocs
are done.
In some cases the TLS state pointers to subject and issuerdn could
be overwritten by a new memory allocation, causing us to loose
track of the old.
This has been observed in the case of improper VLAN handling, where
it was suspected that multiple unrelated TLS streams were mangled
together.
The loop is freeing elements so we need to use the safe version
of TIALQ_FOREACH.
This fixes a valgrind error:
Thread 1 Suricata-Main:
Invalid read of size 8
at 0x8E129C: LiveDeviceListClean (util-device.c:167)
by 0x89B742: main (suricata.c:2284)
Address 0x8382988 is 24 bytes inside a block of size 40 free'd
at 0x4C2A70C: free (vg_replace_malloc.c:468)
by 0x8E1297: LiveDeviceListClean (util-device.c:179)
by 0x89B742: main (suricata.c:2284)
Victor Julien
Eric Leblond