suricata

Commit Graph

Author	SHA1	Message	Date
Ken Steele	497575d38e	Add option on Tile-Gx for logging for fast.log alerts over PCIe When running on a TILEncore-Gx PCIe card, setting the filetype of fast.log to pcie, will open a connection over PCIe to a host application caleld tile-pcie-logd, that receives the alert strings and writes them to a file on the host. The file name to open is also passed over the PCIe link. This allows running Suricata on the TILEncore-Gx PCIe card, but have the alerts logged to the host system's file system efficiently. The PCIe API that is used is the Tilera Packet Queue (PQ) API which can access PCIe from User Space, thus avoiding system calls. Created util-logopenfile-tile.c and util-logopen-tile.h for the TILE specific PCIe logging functionality. Using Write() and Close() function pointers in LogFileCtx, which default to standard write and close for files and sockets, but are changed to PCIe write and close functions when a PCIe channel is openned for logging. Moved Logging contex out of tm-modules.h into util-logopenfile.h, where it makes more sense. This required including util-logopenfile.h into a couple of alert-*.c files, which previously were getting the definitions from tm-modules.h. The source and Makefile for tile-pcie-logd are added in contrib/tile-pcie-logd. By default, the file name for fast.log specified in suricata.yaml is used as the filename on the host. An optional argument to tile-pcie-logd, --prefix=, can be added to prepend the supplied file path. For example, is the file in suricata.yaml is specified as "/var/log/fast.log" and --prefix="/tmp", then the file will be written to "/tmp/var/log/fast.log". Check for TILERA_ROOT environment variable before building tile_pcie_logd Building tile_pcie_logd on x86 requires the Tilera MDE for its PCIe libraries and API header files. Configure now checs for TILERA_ROOT before enabling builing tile_pcie_logd in contrib/tile_pcie_logd	12 years ago
Victor Julien	a3b0577a1f	output: add TM_FLAG_LOGAPI_TM thread module flag The TM_FLAG_LOGAPI_TM flag indicates that a module is run by the log api, not by the 'regular' thread module call functions. Set flag in all all Log API users' registration code. Purpose of this flag is in profiling. In profiling output it will be used to list log api thread modules separately.	12 years ago
Victor Julien	c36a8d30e5	threading: add utility to get module id from name Add TmModuleGetIdByName to go from module name to id directly.	12 years ago
Victor Julien	52c3d3ad7c	log api: convert all names to const Instead of strdupping all names w/o a need, use const ptrs.	12 years ago
Anoop Saldanha	34f0897163	check if all packets are processed before disabling detect threads + kill all threads <= detect after FFR + other minor fixes	14 years ago
Anoop Saldanha	ecad4a24fa	live rule support added To reload ruleset during engine runtime, send the USR2 signal to the engine, and the ruleset would be reloaded from the same yaml file supplied at engine startup	14 years ago
Anoop Saldanha	bc6cf43840	#482 - use decode_flag for all decode TMs. Use the flag as a way to retrieve decode TMs from ThreadVars	14 years ago
Victor Julien	effe01ae7b	Add Init and DeInit calls to the thread module API.	14 years ago
William Metcalf	3b3f5816bf	You spin me right round baby, right round like a rotating packet capture right round. Oh, also log file size counters are now uint64_t	14 years ago
Anoop Saldanha	3f1c4efceb	Add new flags var to tm module. TMs can now set flags to identify special properties. Also use these to identify receive TMs	14 years ago
Anoop Saldanha	a844eecb0e	- Updated all runmodes to use synchronization points, right before each thread(slot function) tries to de-init the thread. - Main thread now first disables receive thread(s) before it kills receive and rest of the threads.	14 years ago
Victor Julien	820b0ded82	Add per packet profiling. Per packet profiling uses tick based accounting. It has 2 outputs, a summary and a csv file that contains per packet stats. Stats per packet include: 1) total ticks spent 2) ticks spent per individual thread module 3) "threading overhead" which is simply calculated by subtracting (2) of (1). A number of changes were made to integrate the new code in a clean way: a number of generic enums are now placed in tm-threads-common.h so we can include them from any part of the engine. Code depends on --enable-profiling just like the rule profiling code. New yaml parameters: profiling: # packet profiling packets: # Profiling can be disabled here, but it will still have a # performance impact if compiled in. enabled: yes filename: packet_stats.log append: yes # per packet csv output csv: # Output can be disabled here, but it will still have a # performance impact if compiled in. enabled: no filename: packet_stats.csv Example output of summary stats: IP ver Proto cnt min max avg ------ ----- ------ ------ ---------- ------- IPv4 6 19436 11448 5404365 32993 IPv4 256 4 11511 49968 30575 Per Thread module stats: Thread Module IP ver Proto cnt min max avg ------------------------ ------ ----- ------ ------ ---------- ------- TMM_DECODEPCAPFILE IPv4 6 19434 1242 47889 1770 TMM_DETECT IPv4 6 19436 1107 137241 1504 TMM_ALERTFASTLOG IPv4 6 19436 90 1323 155 TMM_ALERTUNIFIED2ALERT IPv4 6 19436 108 1359 138 TMM_ALERTDEBUGLOG IPv4 6 19436 90 1134 154 TMM_LOGHTTPLOG IPv4 6 19436 414 5392089 7944 TMM_STREAMTCP IPv4 6 19434 828 1299159 19438 The proto 256 is a counter for handling of pseudo/tunnel packets. Example output of csv: pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading 1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337 First line of the file contains labels. 2 example gnuplot scripts added to plot the data.	14 years ago
Victor Julien	b753ecce50	Implement a pkt acq loop infra with support for pcap-file.	14 years ago
William Metcalf	023a0f94a2	first stab at pcap logging no rotating buff etc	15 years ago
Gurvinder Singh	7d0781b349	added support to log dropped packet as netfilter logs while in inline mode	15 years ago
Gurvinder Singh	e5edc6e8e3	add the support to log the fast.log alerts type to syslog	15 years ago
Anoop Saldanha	33f4beb0bc	batching of packets support for cuda b2g mpm. Supported for both 32 and 64 bit platforms	16 years ago
Jason MacLulich	835630efbd	Add initial support for reading packets from a DAG card, we only support reading from a single stream at this time. Use the --dag <dagname> cmd line option to specify from which DAG card to read pkts from. Issue at the moment with pkts being ejected during shutdown -- at the moment we ignore any packets that are not of link type Ethernet.	16 years ago
Victor Julien	4e7df60b2f	Make pcap file mode read multiple packets per 'read'. Update threading model to deal with this.	16 years ago
Jason Ish	a93b2e6b84	Support for reading ERF files.	16 years ago
William Metcalf	ce01927515	Import of GPLv2 Header 050410	16 years ago
Victor Julien	070ed778b8	Libcap-ng support by Gurvinder Singh and myself. Basic support for per thread caps is added, but not activated as it doesn't seem to work yet. Work around for incompatibility between libnet 1.1 and libcap-ng added.	16 years ago
Jason Ish	40f9653c06	Have output plugs use an OutputCtx which is a little more generic than LogFileCtx. The OutputCtx provides a place for module private data to avoi overriding the LogFileCtx.	16 years ago
Anoop Saldanha	41e6735b92	mpm b2g cuda support added	16 years ago
Pablo Rincon	cef12d30b5	Unified output fixes: alert count per module (not per thread), fix timestamps on pcap mode, write all the alerts of a packet, write the log header once also on unified alert	16 years ago
Jason Ish	763fb4a583	Fix threading issues with unified-log. - Only write the header once, on opening, not per thread init. - Track the size in the log file ctx, not per thread.	16 years ago
Pierre Chifflier	4515ae13e4	Add Prelude output plugin Add support for reporting alerts to the Prelude SIEM system, using libprelude to send IDMEF (RFC4765) messages. Each message contains the alert description and reference (using the SID/GID), and a normalized description (assessment, impact, sources etc.) libprelude handles the connection with the manager (collecting component), spooling and sending the event asynchronously. It also offers transport security (using TLS and trusted certificates) and reliability (events are retransmitted if not sent successfully). This modules requires a Prelude profile to work (see man prelude-admin and the Prelude Handbook for help). Signed-off-by: Pierre Chifflier <chifflier@edenwall.com>	16 years ago
Nick Rogness	2b7b78f1bf	Intial IPFW support FreeBSD and OSX	16 years ago
Jason Ish	844c444af1	Use the configuration file to setup alert logging (and http logging). Only setup for the live pcap modes at the moment.	16 years ago
Pablo Rincon	e26833be3f	Changing mutex/spinlocks/conditions naming types	16 years ago
Pablo Rincon	769022f4be	Adding support for Mac OS X, FreeBSD, centrailizing mutex/spins/conditions in a macro API, and some unittests	16 years ago
Pablo Rincon	a64af4adca	Forgot to add this file	16 years ago
Gurvinder Singh	40b8afdd56	support for thread exit constants	16 years ago
Gurvinder Singh	48c94bbf03	initial thread code support	16 years ago
Pablo Rincon Crespo	a84cc38bc9	Preparing multithreading support for alert modules and logfilectx	16 years ago
William Metcalf	04b0f177fc	native PF_RING support with fixes	16 years ago
Breno Silva	a5e386ce52	Unified2 Signed-off-by: Breno Silva <breno.silva@gmail.com>	16 years ago
Victor Julien	a39108843e	Small tm module API rename to reflect that Init/Deinit/ExitPrintStats are per thread calls.	16 years ago
Brian Rectanus	fa5939ca91	64 bit cleanup part2	16 years ago
Victor Julien	689bbfdc45	Rename all structure definitions in the "typedef struct _SomeStruct" format to "typedef struct SomeStruct_" to make the Doxygen output more useful. Remove the Trie multi pattern matcher code. It wasn't used anymore.	16 years ago
Victor Julien	9c7f5afa79	Large update: pcap support, threading fixes, initial stream tracking, time handling, pool support, runmodes, decoders added, autojunk update.	16 years ago
Victor Julien	5df5b35e90	Put all globals in the detection engine ctx. Add HashList type, a hash that also stores the items ina list to it can be traversed. Many cleanups.	16 years ago
Victor Julien	4c4862d838	Improve logging, add alert-output module, at module exit stats, add HTTP POST uri capture.	16 years ago
Victor Julien	edf8650a7e	Tunnel update.	16 years ago
William Metcalf	0ffa1c2465	updates for configure.in, added reject code, some decode stuff for tcp	16 years ago
Victor Julien	05fd319f6c	Add log-httplog module that logs http request uri's, hosts and useragents to a per line text format.	16 years ago
Victor Julien	6c1f2071be	Add unittest registration to the threading modules api.	16 years ago
Victor Julien	bab4b62376	Initial add of the files.	16 years ago

48 Commits (684d7875677a0f9ec4cf3bd71afe81bdfd6f3110)