The http_server_body content modifier modifies the previous content to inspect
the normalized (dechunked, unzipped) http_server_body. The workings are similar
to http_client_body. Additionally, a new pcre flag was introduced "/S".
To facilitate this change the signature flags field was changed to be 64 bit.
This patch adds a new alert format called pcap-info. It aims at
providing an easy to parse one-line per-alert format containing
the packet id in the parsed pcap for each alert. This permit to
add information inside the pcap parser.
This format is made to be used with suriwire which is a plugin for
wireshark. Its target is to enable the display of suricata results
inside wireshark.
This format doesn't use append mode per default because a clean file
is needed to operate with wireshark.
The format is a list of values separated by ':':
Packet number:GID of matching signature:SID of signature:REV of signature:Flow:To Server:To Client:0:0:Message of signature
The two zero are not yet used values. Candidate for usage is the
part of the packet that matched the signature.
This patch adds support for the replace keyword. It is used with
content to change selected part of the payload. The major point
with this patch is that having a replace keyword made necessary
to avoid all stream level check because we need to access to the
could-be-modified packet payload.
One of the main difficulty is to handle complex signature. If there is
other content check, we must do the substitution when we're sure all
match are valid. The patch adds an attribute to the thread context
variable to be able to deal with recursivity of the match function.
Replace is only activated in IPS mode and apply only to raw match.
Per packet per app layer parser profiling. Example summary output:
Per App layer parser stats:
App Layer IP ver Proto cnt min max avg
-------------------- ------ ----- ------ ------ ---------- -------
ALPROTO_HTTP IPv4 6 163394 126 38560320 42814
ALPROTO_FTP IPv4 6 644 117 26100 2566
ALPROTO_TLS IPv4 6 670 117 7137 799
ALPROTO_SMB IPv4 6 114794 126 225270 957
ALPROTO_DCERPC IPv4 6 5207 126 25596 1266
Also added to the csv out.
In the csv out there is a new column "stream (no app)" that removes the
app layer parsers from the stream tracking. So raw stream engine performance
becomes visible.
Per packet profiling uses tick based accounting. It has 2 outputs, a summary
and a csv file that contains per packet stats.
Stats per packet include:
1) total ticks spent
2) ticks spent per individual thread module
3) "threading overhead" which is simply calculated by subtracting (2) of (1).
A number of changes were made to integrate the new code in a clean way:
a number of generic enums are now placed in tm-threads-common.h so we can
include them from any part of the engine.
Code depends on --enable-profiling just like the rule profiling code.
New yaml parameters:
profiling:
# packet profiling
packets:
# Profiling can be disabled here, but it will still have a
# performance impact if compiled in.
enabled: yes
filename: packet_stats.log
append: yes
# per packet csv output
csv:
# Output can be disabled here, but it will still have a
# performance impact if compiled in.
enabled: no
filename: packet_stats.csv
Example output of summary stats:
IP ver Proto cnt min max avg
------ ----- ------ ------ ---------- -------
IPv4 6 19436 11448 5404365 32993
IPv4 256 4 11511 49968 30575
Per Thread module stats:
Thread Module IP ver Proto cnt min max avg
------------------------ ------ ----- ------ ------ ---------- -------
TMM_DECODEPCAPFILE IPv4 6 19434 1242 47889 1770
TMM_DETECT IPv4 6 19436 1107 137241 1504
TMM_ALERTFASTLOG IPv4 6 19436 90 1323 155
TMM_ALERTUNIFIED2ALERT IPv4 6 19436 108 1359 138
TMM_ALERTDEBUGLOG IPv4 6 19436 90 1134 154
TMM_LOGHTTPLOG IPv4 6 19436 414 5392089 7944
TMM_STREAMTCP IPv4 6 19434 828 1299159 19438
The proto 256 is a counter for handling of pseudo/tunnel packets.
Example output of csv:
pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading
1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337
First line of the file contains labels.
2 example gnuplot scripts added to plot the data.
This patch introduces 'nfq_set_mark' which is new rules option. If a packet
matches a rule using nfq_set_mark in NFQ mode, it is marked with the mark/mask
specified in the option during the verdict.
It is thus possible to trigger different behaviour on the packet inside
Linux/Netfilter.
Use the --dag <dagname> cmd line option to specify from which DAG card to read pkts
from.
Issue at the moment with pkts being ejected during shutdown -- at the moment we
ignore any packets that are not of link type Ethernet.
Stateful detection for app layer detection keywords, except uricontent. Stores it's partial results in the flow structure. Other modifications:
- Generalize transaction tracking, logging and inspection.
- Adapt http and dcerpc to use the new transaction handling.
- Stream engine now always notifies app layer of a stream eof.
This commit fixes bug #124.
Add support for reporting alerts to the Prelude SIEM system, using
libprelude to send IDMEF (RFC4765) messages.
Each message contains the alert description and reference (using
the SID/GID), and a normalized description (assessment, impact,
sources etc.)
libprelude handles the connection with the manager (collecting component),
spooling and sending the event asynchronously. It also offers transport
security (using TLS and trusted certificates) and reliability (events
are retransmitted if not sent successfully).
This modules requires a Prelude profile to work (see man prelude-admin
and the Prelude Handbook for help).
Signed-off-by: Pierre Chifflier <chifflier@edenwall.com>
Hello,
Below is a patch that gets "make distcheck" working. Its against the
current code in git. The project version was set to 0.1 in configure,
I changed that to 0.8.1 just so its actually relevant. You might want
to set that to something else.
After checking this patch, I find that there are several source code
files in src/ that are not getting compiled:
-app-layer-detect.c
-app-layer-detect.h
-app-layer-http.c
-reputation.h
Are these new or abandoned? Anyways...here's the patch.
-Steve