199 Commits (b591813b8690753ee4a678f8b10ff1928de1fe15)

Author	SHA1	Message	Date
Victor Julien	e48153c6b0	counters: make threads cleanup all memory	11 years ago
Victor Julien	81548ae3e8	counters: clean up global context	11 years ago
Victor Julien	f05d0692ef	counters: remove references to 'perf' counters	11 years ago
Victor Julien	faef92f8da	counters: remove last and now unused tm_name reference	11 years ago
Victor Julien	83f27ae2a5	counters: remove old unix socket json logic	11 years ago
Victor Julien	41ead6611a	counters: minor internal API cleanups	11 years ago
Victor Julien	d2a9ef2680	counters: rename unparsable SCPCAElem to StatsLocalCounter	11 years ago
Victor Julien	752f03e7a4	counters: remaining s/SCPerf/Stats/g	11 years ago
Victor Julien	4362d0a6e9	counters: s/SCPerfPrivateContext/StatsPrivateThreadContext/g	11 years ago
Victor Julien	628c3b1bc7	counters: s/SCPerfPublicContext/StatsPublicThreadContext/g	11 years ago
Victor Julien	7e70f136ec	counters: various renames and cleanups	11 years ago
Victor Julien	30cce2bd29	counters: s/SCPerfCounterSetUI64/StatsSetUI64/g	11 years ago
Victor Julien	1c0b4ee0ae	counters: s/SCPerfCounterIncr/StatsIncr/g	11 years ago
Victor Julien	8992275b0c	counters: s/SCPerfCounterAddUI64/StatsAddUI64/g	11 years ago
Victor Julien	60d9eb6790	counters: clean up defines	11 years ago
Victor Julien	1ef786e7cb	counters: rename register API calls ... Also remove 'type' parameter which was always the same.	11 years ago
Victor Julien	23f17950bc	counters: pass per thread stats to output api ... As well as the global (merged) stats.	11 years ago
Victor Julien	de82b6d31e	counters: rename widely used pctmi var to sts (stats thread store)	11 years ago
Victor Julien	60c5ad4649	counters: call global counters funcs	11 years ago
Victor Julien	33756abd87	counters: split API init ... Split into early ctx initialization and post-config setup.	11 years ago
Victor Julien	d05eed3735	counters: start using Stats prefix	11 years ago
Victor Julien	f300ad253e	counters: simplify and speedup counters sync	11 years ago
Victor Julien	0478407833	counters: remove threadvars arg from SCPerfAddToClubbedTMTable	11 years ago
Victor Julien	2346a88db7	counters: remove thread module name from counters API	11 years ago
Victor Julien	06461e37da	counters: global counters registration	11 years ago
Victor Julien	b293a4b7d0	counters: remove unused description	11 years ago
Victor Julien	711cd7b59b	counters: merge counters from threads for output ... Merge counters so the table contains combined values from counters from each thread. Use global counter id's, track them in a hash. Rename SCPCAElem members Fix and improve average counters	11 years ago
Victor Julien	7da657dc3d	counters: remove unused public API calls and make them private	11 years ago
Victor Julien	ac6e24c06a	counters: make SCPerfSetupPrivate a function	11 years ago
Victor Julien	799640f906	counters: make threadvars::perf_private_ctx static ... Update SCPerfGetAllCountersArray and add a UT workaround.	11 years ago
Victor Julien	55cfab89e4	counters: SCPerfGetLocalCounterValue cleanup ... Return u64, update arguments.	11 years ago
Victor Julien	e9b067c1eb	counters: make increment call take threadvars ... This hides the implementation from the caller.	11 years ago
Victor Julien	9a8bff7d96	counters: threadvars s/sc_perf_pca/perf_private_ctx/g	11 years ago
Victor Julien	50bb995458	counters: rename threadvars public counters	11 years ago
Victor Julien	6ffbc3a362	counters: s/SCPerfContext/SCPerfPublicContext/g	11 years ago
Victor Julien	0a5ae1b403	counters: s/SCPerfCounterArray/SCPerfPrivateContext/g ... Goal is to make it's purpose clear.	11 years ago
Victor Julien	9f584483be	counters: minor cleanups	11 years ago
Victor Julien	fb479902e4	threading: explain purpose of threadvars mucond	11 years ago
Victor Julien	bc2c7f462e	stats api: call thread deinit API functions ... Thread deinit funcs weren't called. This meant the lua scripts 'deinit' functions weren't called either.	11 years ago
Victor Julien	8b2dd81628	stats: stats threads don't need packet pools	11 years ago
Victor Julien	51a540c27e	stats: disable stats if no loggers are enabled	11 years ago
Victor Julien	a95c95f74c	stats: introduce global config ... As the stats api calls the loggers at a global interval, the global interval should be configured globally. # global stats configuration stats: enabled: yes # The interval field (in seconds) controls at what interval # the loggers are invoked. interval: 8 If this config isn't found, the old config will be supported.	11 years ago
Victor Julien	e98346b555	Introduce stats log API, convert existing output ... Convert regular 'stats.log' output to this new API. In addition to the current stats value, also give the last value. This makes it easy to display the difference.	11 years ago
Ken Steele	28ccea51d3	Add error checking for pthread_setspecific() and pthread_key_create().	11 years ago
Jason Ish	fc2014ab40	Unregister for file rotation notification when a context is ... de-initialized. Required for unix-socket mode where contexts come and go.	12 years ago
Jason Ish	e1b97fed70	Add signal based file rotation for: ... - alert debug log - fast log - stats log - dns log - drop log - file log - http log - tls log - eve/json log	12 years ago
Victor Julien	399246881d	counters: fix 2 scan-build warnings ... counters.c:1069:13: warning: Potential leak of memory pointed to by 'temp' SCMutexUnlock(&sc_perf_op_ctx->pctmi_lock); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ./threads.h:121:28: note: expanded from macro 'SCMutexUnlock' ^~~~~~~~~~~~~~~~~~~~ counters.c:1156:16: warning: Potential leak of memory pointed to by 'pca' return NULL; ^~~~ /usr/include/clang/3.3/include/stddef.h:77:24: note: expanded from macro 'NULL' ^ 2 warnings generated.	12 years ago
Eric Leblond	1f07d1521e	Fix realloc error handling ... This patch is fixing realloc error handling. In case of a realloc failure, it free the initial memory and continue existing error handling. The patch has been obtained via the following semantic patch and a bit oh hand editing: @@ expression x, E; identifier f; @@ f(...) { + void *ptmp; <+... - x = SCRealloc(x, E); + ptmp = SCRealloc(x, E); ... when != x - if (x == NULL) + if (ptmp == NULL) { + SCFree(x); + x = NULL; ... - } + } else { + x = ptmp; + } ...+> } @@ expression x, E; identifier f; statement ES; @@ f(...) { + void *ptmp; <+... - x = SCRealloc(x, E); + ptmp = SCRealloc(x, E); ... when != x - if (x == NULL) ES + if (ptmp == NULL) { + SCFree(x); + x = NULL; + ES + } else { + x = ptmp; + } ...+> } @@ expression x, E; identifier f; @@ f(...) { + void *ptmp; <+... - x = SCRealloc(x, E); + ptmp = SCRealloc(x, E); ... when != x - if (unlikely(x == NULL)) + if (unlikely(ptmp == NULL)) { + SCFree(x); + x = NULL; ... - } + } else { + x = ptmp; + } ...+> } @@ expression x, E; identifier f; statement ES; @@ f(...) { + void *ptmp; <+... - x = SCRealloc(x, E); + ptmp = SCRealloc(x, E); ... when != x - if (unlikely(x == NULL)) ES + if (unlikely(ptmp == NULL)) { + SCFree(x); + x = NULL; + ES + } else { + x = ptmp; + } ...+> }	12 years ago
Eric Leblond	28c5c68192	error checking: add missing alloc error treatment ... The return of some malloc like functions was not treated in some places of the code.	12 years ago
Victor Julien	ed03196a20	Counter: fix accidental logic change	12 years ago
Victor Julien	45dfecafd4	Counters: remove unused updated field	12 years ago
Victor Julien	64f5129f12	Counters: remove unused tm_name comparison loops	12 years ago
Victor Julien	76c305c128	Counters: fix unix socket	12 years ago
Victor Julien	25aeeebdf7	Counters: merge SCPerfCounterName into SCPerfCounter as there was a 1 on 1 mapping	12 years ago
Victor Julien	3445d17ae5	Counters: remove SCPerfCounterValue struct as we no longer support multiple data types	12 years ago
Victor Julien	677cd03e52	Counters: more unused code removal	12 years ago
Victor Julien	8d4a61a789	Counters: remove unused code	12 years ago
Victor Julien	698ff4e4aa	Counters: remove all unused parts of the API	12 years ago
Ken Steele	592d48aab7	Use Spin locks on Tile ... On Tile, replace pthread_mutex_locks with queued spin locks (ticket locks) for dataplane processing code. This is safe when running on dataplane cores with one thread per core. The condition variables are no-ops when the thread is spinning anyway. For control plane threads, unix-manager, stats-logs, thread startup, use pthread_mutex_locks. For these locks replaced SCMutex with SCCtrlMutex and SCCond with SCCtrlCond.	12 years ago
Victor Julien	3470b07ea5	Fix several compile and runtime warnings found by clang 3.2 with the -fsanitize=address option.	12 years ago
Ken Steele	d4dd18eb85	Clean up SCLocalTime() usage ... Remove cast of return type from SCLocalTime() as it is not needed. Replace last use of localtime_r() with SCLocalTime().	12 years ago
Eric Leblond	34abd818dd	Prefix util-conf function with Config	12 years ago
Eric Leblond	54006de40c	Use new function GetLogDirectory()	12 years ago
Eric Leblond	7ec820d3ab	Fix potential Null deref.	13 years ago
Eric Leblond	31c03d38b9	unix socket: add 'dump-counters' command ... This patch adds a 'dump-counters' command which answer an output of all performance counter.	13 years ago
Anoop Saldanha	34a9c047fc	updated to fix unix shutdown sequence ... Should fix crashes occuring from unix mode shutdown/cleanup phase.	13 years ago
Eric Leblond	7b1d346c22	counters: management cpu set was set twice ... Setting the management CPU set on perf threads is already done in the TmThreadCreateMgmtThread() function used to create the threads.	13 years ago
Eric Leblond	1b26660ac4	counter: defensive set to NULL in free.	13 years ago
Eric Leblond	d1569337a7	affinity: add call to setup function in threads ... Threads created through TMThreadSpawn need to call the affinity function by themselves.	13 years ago
Eric Leblond	0eeccb4b17	affinity: tag management threads as such ... The management threads were not tagged for CPU affinity and thus the setting was not applied.	13 years ago
Eric Leblond	0227a87fcb	cleaning: fix warning when building with clang. ... clang was issuing some warnings related to unused return in function. This patch adds some needed error treatment and ignore the rest of the warnings by adding a cast to void.	14 years ago
Anoop Saldanha	31eb5fa2f6	Introduce util-signal.[ch]. Move our signal setup functions here	14 years ago
Anoop Saldanha	ecad4a24fa	live rule support added ... To reload ruleset during engine runtime, send the USR2 signal to the engine, and the ruleset would be reloaded from the same yaml file supplied at engine startup	14 years ago
Eric Leblond	a0e57f58e5	OpenBSD: introduce SCLocalTime function. ... This function is a wrapper to localtime_r. It is needed to avoid a compilation warning on OpenBSD. I'm forced to type the function to a non pointer first parameter. If not we will have to use two differents functions in OpenBSD where tv->tv_sec is a long (different from time_t).	14 years ago
Anoop Saldanha	bff2866aed	more coverity fixes	14 years ago
Anoop Saldanha	081b0e05a2	restructure disabling receive threads. Introduce new flag to indicate that threads have finised running	14 years ago
Victor Julien	8448333bdd	Remove trailing zero's from some counters output.	14 years ago
Anoop Saldanha	420befb180	Changed my email address to anoopsaldanha at gmail dot com from my current one	14 years ago
Victor Julien	1df3304655	Clean up for unittests code: only compile unittest api code when unittests are enabled. Fix unittest code that wasn't wrapped in the proper UNITTESTS ifdefs.	14 years ago
Victor Julien	8186565240	Fix a number of potential issues found by CLANG and cppcheck.	14 years ago
Victor Julien	b39acddf28	Add flow counters: memuse, pruning stats, emergency mode. Bug #348.	14 years ago
Anoop Saldanha	f7b1972263	update broken stats.log. Use pktacqloop funcs in pcap-file, pfring, pcap-live, af-pkt to sync counters - bug #343	14 years ago
Anoop Saldanha	6c95526423	Introduce a new wrapper macro that wait loops till the flag(s) in question have been set	14 years ago
Anoop Saldanha	b0a588beeb	Introduce another solution to solve stream timeout shutdown issue using thread flags. No more MSSyncPts	14 years ago
Eileen Donlon	89599d3b9b	fixed bug 288; corrected config boolean parsing problems	14 years ago
Victor Julien	820b0ded82	Add per packet profiling. ... Per packet profiling uses tick based accounting. It has 2 outputs, a summary and a csv file that contains per packet stats. Stats per packet include: 1) total ticks spent 2) ticks spent per individual thread module 3) "threading overhead" which is simply calculated by subtracting (2) of (1). A number of changes were made to integrate the new code in a clean way: a number of generic enums are now placed in tm-threads-common.h so we can include them from any part of the engine. Code depends on --enable-profiling just like the rule profiling code. New yaml parameters: profiling: # packet profiling packets: # Profiling can be disabled here, but it will still have a # performance impact if compiled in. enabled: yes filename: packet_stats.log append: yes # per packet csv output csv: # Output can be disabled here, but it will still have a # performance impact if compiled in. enabled: no filename: packet_stats.csv Example output of summary stats: IP ver Proto cnt min max avg ------ ----- ------ ------ ---------- ------- IPv4 6 19436 11448 5404365 32993 IPv4 256 4 11511 49968 30575 Per Thread module stats: Thread Module IP ver Proto cnt min max avg ------------------------ ------ ----- ------ ------ ---------- ------- TMM_DECODEPCAPFILE IPv4 6 19434 1242 47889 1770 TMM_DETECT IPv4 6 19436 1107 137241 1504 TMM_ALERTFASTLOG IPv4 6 19436 90 1323 155 TMM_ALERTUNIFIED2ALERT IPv4 6 19436 108 1359 138 TMM_ALERTDEBUGLOG IPv4 6 19436 90 1134 154 TMM_LOGHTTPLOG IPv4 6 19436 414 5392089 7944 TMM_STREAMTCP IPv4 6 19434 828 1299159 19438 The proto 256 is a counter for handling of pseudo/tunnel packets. Example output of csv: pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading 1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337 First line of the file contains labels. 2 example gnuplot scripts added to plot the data.	14 years ago
Anoop Saldanha	ff7284e7b7	Fix code that allows the engine to restart threads that have exited on failure	14 years ago
Anoop Saldanha	d3bc3f0fe5	coverity fix for counters api	15 years ago
Victor Julien	e19f6ebaf4	Various fixes for issues reported by clang.	15 years ago
Victor Julien	f5a2017f3c	Fix counter unittest on big endian.	15 years ago
Victor Julien	d48ff8f6aa	Extend 'append' option to stats.log as well. Small cleanups.	15 years ago
Victor Julien	3fcfaef9f7	Fix compiler warning in log-httplog.c & change stats.log to log as mm/dd/yyyy as well.	15 years ago
Victor Julien	355f237bfd	Fix compiler warnings, cleanup counters config code.	15 years ago
Gurvinder Singh	ba18110abd	support for stats.log configurable and fixed timezone issue in faslog and debuglog	15 years ago
Victor Julien	1c1c8cef89	Print engine uptime on the same line as date and time.	15 years ago
Gurvinder Singh	1d0492e2e7	added support to print the engine uptime in stats.log	15 years ago
Victor Julien	012e602c3f	Add a (disabled by default) flow pinned runmode for file pcap.	16 years ago
Anoop Saldanha	bbb5bf5c51	allow counters clubbing for detect TM	16 years ago
Gurvinder Singh	cda664a8c4	memroy leaks fixes in detection module, app layer and counters	16 years ago
Gerardo Iglesias Galvan	9f4fae5b1a	Fix inconsistent use of dynamic memory allocation	16 years ago