28 Commits (d908e707d72923cee0712644d223bd8bf6223d4a)

Author	SHA1	Message	Date
Victor Julien	d908e707d7	profiling: add per lock location profiling ... Add profiling per lock location in the code. Accounts how often a lock is requested, how often it was contended, the max number of ticks spent waiting for it, avg number of ticks waiting for it and the total ticks for that location. Added a new configure flag --enable-profiling-locks to enable this feature.	14 years ago
Victor Julien	979edf0b97	Add way to profile mutex/spin locks per thread module.	14 years ago
Victor Julien	0b9038b971	Add atomics to ticks unittests.	14 years ago
Victor Julien	89f83e714c	Introduce http_server_body keyword. ... The http_server_body content modifier modifies the previous content to inspect the normalized (dechunked, unzipped) http_server_body. The workings are similar to http_client_body. Additionally, a new pcre flag was introduced "/S". To facilitate this change the signature flags field was changed to be 64 bit.	14 years ago
Victor Julien	af51493da2	Mpm update: Toss out signatures that mix pkt and stream/state. Update profiling code to track new mpm.	14 years ago
Anoop Saldanha	4307ea2348	Replace all frees with SCFrees	14 years ago
Anoop Saldanha	13ea299ee0	Replace all mallocs with SCMallocs	14 years ago
Victor Julien	cc4e89fbe1	Profiling: convert all packet profile counters/variables to u64. Improve output for larger numbers.	14 years ago
Eileen Donlon	89599d3b9b	fixed bug 288; corrected config boolean parsing problems	14 years ago
Victor Julien	63f834d9a7	Add profiling to various HTTP buffer MPM calls.	14 years ago
Victor Julien	3693a7a9ee	Profiling: add accounting for several detection phases.	14 years ago
Victor Julien	e8e392fb1f	Profiling: add per packet accounting of how much ticks are spend in protocol detection.	14 years ago
Victor Julien	21f387d2c7	profiling: fix stream ticks miscalculation on stream end pseudo packets.	14 years ago
Victor Julien	fca541f40e	Add per app layer parser profiling ... Per packet per app layer parser profiling. Example summary output: Per App layer parser stats: App Layer IP ver Proto cnt min max avg -------------------- ------ ----- ------ ------ ---------- ------- ALPROTO_HTTP IPv4 6 163394 126 38560320 42814 ALPROTO_FTP IPv4 6 644 117 26100 2566 ALPROTO_TLS IPv4 6 670 117 7137 799 ALPROTO_SMB IPv4 6 114794 126 225270 957 ALPROTO_DCERPC IPv4 6 5207 126 25596 1266 Also added to the csv out. In the csv out there is a new column "stream (no app)" that removes the app layer parsers from the stream tracking. So raw stream engine performance becomes visible.	14 years ago
Victor Julien	820b0ded82	Add per packet profiling. ... Per packet profiling uses tick based accounting. It has 2 outputs, a summary and a csv file that contains per packet stats. Stats per packet include: 1) total ticks spent 2) ticks spent per individual thread module 3) "threading overhead" which is simply calculated by subtracting (2) of (1). A number of changes were made to integrate the new code in a clean way: a number of generic enums are now placed in tm-threads-common.h so we can include them from any part of the engine. Code depends on --enable-profiling just like the rule profiling code. New yaml parameters: profiling: # packet profiling packets: # Profiling can be disabled here, but it will still have a # performance impact if compiled in. enabled: yes filename: packet_stats.log append: yes # per packet csv output csv: # Output can be disabled here, but it will still have a # performance impact if compiled in. enabled: no filename: packet_stats.csv Example output of summary stats: IP ver Proto cnt min max avg ------ ----- ------ ------ ---------- ------- IPv4 6 19436 11448 5404365 32993 IPv4 256 4 11511 49968 30575 Per Thread module stats: Thread Module IP ver Proto cnt min max avg ------------------------ ------ ----- ------ ------ ---------- ------- TMM_DECODEPCAPFILE IPv4 6 19434 1242 47889 1770 TMM_DETECT IPv4 6 19436 1107 137241 1504 TMM_ALERTFASTLOG IPv4 6 19436 90 1323 155 TMM_ALERTUNIFIED2ALERT IPv4 6 19436 108 1359 138 TMM_ALERTDEBUGLOG IPv4 6 19436 90 1134 154 TMM_LOGHTTPLOG IPv4 6 19436 414 5392089 7944 TMM_STREAMTCP IPv4 6 19434 828 1299159 19438 The proto 256 is a counter for handling of pseudo/tunnel packets. Example output of csv: pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading 1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337 First line of the file contains labels. 2 example gnuplot scripts added to plot the data.	14 years ago
Victor Julien	1bd1a62526	Rename profile macro's and variables to reflect that they are for rule profiling.	14 years ago
William	bca8fbc79e	Add Num, Rev, and Gid columns to rule perf output	14 years ago
Victor Julien	ece8e5444b	Minor profiling fix: don't close stdout.	15 years ago
William	85643fe780	Convert to logging perf stats to file by default. Add a few columns to output avg ticks per match, avg ticks non match, allow sorting on based on them.	15 years ago
Jason Ish	a4d19e4130	Add new profiling sort option, maxticks.	15 years ago
Victor Julien	746ef0d2f8	Remove stray newline char from profiling output.	15 years ago
Jason Ish	9ad1924dba	Track the max ticks for each rule. ... This is the highest tick count this rule used to evaluate in a single sample.	15 years ago
Pablo Rincon	c1486d7f2e	Fix bug 217 (segv on profiling summary if no rule was specified	15 years ago
Victor Julien	8514132851	Cleanup thresholding code.	16 years ago
Jason Ish	ea4b7cc33b	add profiling to stateful detection engine + other fixups.	16 years ago
Jason Ish	a9ed9a8196	In profiling output, include a % for each rule.	16 years ago
William Metcalf	2eef905c07	GPL and Copyright header updates.	16 years ago
Jason Ish	18e5ac8cde	Basic rule profiling even though the results may be skewed by a bad rule in a grouping of rules.	16 years ago