Commit Graph

307 Commits (1ccb93ab50f890e077d4ea894f9c9a75c67a7092)

Author SHA1 Message Date
Anoop Saldanha eea5ab4a7a Support for app layer decoder events added + app_layer_event keyword added 14 years ago
Mike Pomraning 914b10a8e6 Touch up Makefile for SCConfLogOpenGeneric. 14 years ago
Victor Julien 4cbaeb408c Add functions to determine whether a path is absolute or relative. 14 years ago
Victor Julien 7adac3048d file-data: create initial keyword registration. 14 years ago
Victor Julien 89f83e714c Introduce http_server_body keyword.
The http_server_body content modifier modifies the previous content to inspect
the normalized (dechunked, unzipped) http_server_body. The workings are similar
to http_client_body. Additionally, a new pcre flag was introduced "/S".

To facilitate this change the signature flags field was changed to be 64 bit.
14 years ago
Anoop Saldanha e0c13434ef bug 333 - support new Size Parsing API. Update various conf params inside the engine to use this API to parse sizes in the format xxx <-just the no represents bytes, xxxkb <- kilobytes, xxxmb <- megabytes, xxxgb <- gigabytes, where xxx is a \d+ 14 years ago
Eileen Donlon dbdf2d888f Enable/disable core dump in config (feature 319) 14 years ago
Victor Julien e1022ee5ae file-extraction: Disconnect file handling from flow and move into the app layer state. 14 years ago
Victor Julien 27645f64c6 Remove unused util-filetype.[ch] from Makefile.am. 14 years ago
Victor Julien 5945e652d6 Initial implementation of filemagic keyword. 14 years ago
Victor Julien f4a6f4b293 Add libmagic detection, linking and a basic API. 14 years ago
Victor Julien 23e01d23d3 Implement filestore keyword, including a way for the stateful detection engine to conclude that a file will never have to be stored. 14 years ago
Victor Julien 1eef36b011 Initial checkin of a log-file module, that can write files extracted from flows to disk. 14 years ago
Victor Julien a0ee6ade3e Improve HTTP multipart parsing, add streaming parsing for files. 14 years ago
Pablo Rincon 6d60b3a747 filename and fileext keywords 14 years ago
Anoop Saldanha 2b356dadff Support for tos keyword added 14 years ago
Eric Leblond 391d813c82 Remove unified1 output module. 14 years ago
Eric Leblond 27f1d88374 Add pcap-info alert format.
This patch adds a new alert format called pcap-info. It aims at
providing an easy to parse one-line per-alert format containing
the packet id in the parsed pcap for each alert. This permit to
add information inside the pcap parser.

This format is made to be used with suriwire which is a plugin for
wireshark. Its target is to enable the display of suricata results
inside wireshark.

This format doesn't use append mode per default because a clean file
is needed to operate with wireshark.

The format is a list of values separated by ':':
  Packet number:GID of matching signature:SID of signature:REV of signature:Flow:To Server:To Client:0:0:Message of signature
The two zero are not yet used values. Candidate for usage is the
part of the packet that matched the signature.
14 years ago
Eric Leblond 8bf0897b3c Add factorisation function for runmode.
This patch adds a function which will be used to factorise the
Auto runmode between the different IDS mode.
14 years ago
Eric Leblond de59c9f4b1 Add and use utility functions for checksum computing. 14 years ago
Eric Leblond a85dc9b0e2 Add support for replace keyword.
This patch adds support for the replace keyword. It is used with
content to change selected part of the payload. The major point
with this patch is that having a replace keyword made necessary
to avoid all stream level check because we need to access to the
could-be-modified packet payload.

One of the main difficulty is to handle complex signature. If there is
other content check, we must do the substitution when we're sure all
match are valid. The patch adds an attribute to the thread context
variable to be able to deal with recursivity of the match function.

Replace is only activated in IPS mode and apply only to raw match.
14 years ago
Anoop Saldanha b6ba944e6d Rearrange flow manager functions into flow-manager.[ch]. Some other minor changes/updates 14 years ago
Anoop Saldanha 7c729d2d53 some more code cleanup + comments added 14 years ago
Eric Leblond 2ac8755382 Rename detect-decode-event to detect-engine-event
This patch does a simple renaming of detect-decode-event file to
the more global detect-engine-event name.
14 years ago
Eric Leblond 871b21892a factorize pcap live device function
They are not specific to pcap and could thus be used in other module.
14 years ago
Eric Leblond c45d898572 af-packet: basic support for AF_PACKET socket
This patch provides basic support for AF_PACKET socket. It is
completed by a subsequent patches prodiding extended features
and bugfixes.
14 years ago
Victor Julien fca541f40e Add per app layer parser profiling
Per packet per app layer parser profiling. Example summary output:

Per App layer parser stats:

App Layer              IP ver   Proto   cnt        min      max          avg
--------------------   ------   -----   ------     ------   ----------   -------
ALPROTO_HTTP            IPv4       6    163394        126     38560320     42814
ALPROTO_FTP             IPv4       6       644        117        26100      2566
ALPROTO_TLS             IPv4       6       670        117         7137       799
ALPROTO_SMB             IPv4       6    114794        126       225270       957
ALPROTO_DCERPC          IPv4       6      5207        126        25596      1266

Also added to the csv out.

In the csv out there is a new column "stream (no app)" that removes the
app layer parsers from the stream tracking. So raw stream engine performance
becomes visible.
14 years ago
Victor Julien 820b0ded82 Add per packet profiling.
Per packet profiling uses tick based accounting. It has 2 outputs, a summary
and a csv file that contains per packet stats.

Stats per packet include:
 1) total ticks spent
 2) ticks spent per individual thread module
 3) "threading overhead" which is simply calculated by subtracting (2) of (1).

A number of changes were made to integrate the new code in a clean way:
a number of generic enums are now placed in tm-threads-common.h so we can
include them from any part of the engine.

Code depends on --enable-profiling just like the rule profiling code.

New yaml parameters:

profiling:
  # packet profiling
  packets:

    # Profiling can be disabled here, but it will still have a
    # performance impact if compiled in.
    enabled: yes
    filename: packet_stats.log
    append: yes

    # per packet csv output
    csv:

      # Output can be disabled here, but it will still have a
      # performance impact if compiled in.
      enabled: no
      filename: packet_stats.csv

Example output of summary stats:

IP ver   Proto   cnt        min      max          avg
------   -----   ------     ------   ----------   -------
 IPv4       6     19436      11448      5404365     32993
 IPv4     256         4      11511        49968     30575

Per Thread module stats:

Thread Module              IP ver   Proto   cnt        min      max          avg
------------------------   ------   -----   ------     ------   ----------   -------
TMM_DECODEPCAPFILE          IPv4       6     19434       1242        47889      1770
TMM_DETECT                  IPv4       6     19436       1107       137241      1504
TMM_ALERTFASTLOG            IPv4       6     19436         90         1323       155
TMM_ALERTUNIFIED2ALERT      IPv4       6     19436        108         1359       138
TMM_ALERTDEBUGLOG           IPv4       6     19436         90         1134       154
TMM_LOGHTTPLOG              IPv4       6     19436        414      5392089      7944
TMM_STREAMTCP               IPv4       6     19434        828      1299159     19438

The proto 256 is a counter for handling of pseudo/tunnel packets.

Example output of csv:

pcap_cnt,ipver,ipproto,total,TMM_DECODENFQ,TMM_VERDICTNFQ,TMM_RECEIVENFQ,TMM_RECEIVEPCAP,TMM_RECEIVEPCAPFILE,TMM_DECODEPCAP,TMM_DECODEPCAPFILE,TMM_RECEIVEPFRING,TMM_DECODEPFRING,TMM_DETECT,TMM_ALERTFASTLOG,TMM_ALERTFASTLOG4,TMM_ALERTFASTLOG6,TMM_ALERTUNIFIEDLOG,TMM_ALERTUNIFIEDALERT,TMM_ALERTUNIFIED2ALERT,TMM_ALERTPRELUDE,TMM_ALERTDEBUGLOG,TMM_ALERTSYSLOG,TMM_LOGDROPLOG,TMM_ALERTSYSLOG4,TMM_ALERTSYSLOG6,TMM_RESPONDREJECT,TMM_LOGHTTPLOG,TMM_LOGHTTPLOG4,TMM_LOGHTTPLOG6,TMM_PCAPLOG,TMM_STREAMTCP,TMM_DECODEIPFW,TMM_VERDICTIPFW,TMM_RECEIVEIPFW,TMM_RECEIVEERFFILE,TMM_DECODEERFFILE,TMM_RECEIVEERFDAG,TMM_DECODEERFDAG,threading
1,4,6,172008,0,0,0,0,0,0,47889,0,0,48582,1323,0,0,0,0,1359,0,1134,0,0,0,0,0,8028,0,0,0,49356,0,0,0,0,0,0,0,14337

First line of the file contains labels.

2 example gnuplot scripts added to plot the data.
14 years ago
Eric Leblond a0b4068041 autotools: fix duplicate check command in Makefile.
It seems that check target can not be used in Makefile.am. Using
check-am fix a make failure.
14 years ago
Anoop Saldanha 576ec7da66 smtp parser support 15 years ago
Martin Beyer 2f1262b446 fixed cuda build: portability issues and nvcc version check 15 years ago
Martin Beyer 49d66430bc build cuda modules with make 15 years ago
Anoop Saldanha 35f3eafa5e byte extract added to the engine. Detection support added for packet payload, uri and dce detection engines 15 years ago
Victor Julien 7e128176d2 Add Vector datatype for SSE operations. 15 years ago
Victor Julien d0374ced38 Implement SACK in the stream engine. 15 years ago
Anoop Saldanha 966119b6aa support for http_raw_uri keyword + mpm engine 15 years ago
Anoop Saldanha 6fceeda8c5 move erf dag runmode into its own file runmode-erf-dag.[ch] 15 years ago
Anoop Saldanha f51cf34210 move erf file runmode into its own file runmode-erf-file.[ch] 15 years ago
Anoop Saldanha 86eabbc2f5 move ipfw runmode into its own file runmode-ipfw.[ch] 15 years ago
Anoop Saldanha 036015d6b9 move nfq runmode into its own file runmode-nfq.[ch] 15 years ago
Anoop Saldanha 9affa39b29 move pfring runmode into its own file runmode-pfring.[ch] 15 years ago
Anoop Saldanha e7ac1d7c4c move pcap file runmode into its own file runmode-pcap-file.[ch] 15 years ago
Anoop Saldanha f6af567ce0 move pcap live runmode into its own file runmode-pcap.[ch] 15 years ago
Eric Leblond 9beebf621a Add support for 'nfq_set_mark' keyword
This patch introduces 'nfq_set_mark' which is new rules option. If a packet
matches a rule using nfq_set_mark in NFQ mode, it is marked with the mark/mask
specified in the option during the verdict.
It is thus possible to trigger different behaviour on the packet inside
Linux/Netfilter.
15 years ago
William Metcalf 023a0f94a2 first stab at pcap logging no rotating buff etc 15 years ago
Anoop Saldanha c105a739e9 support for ssl_state keyword added 15 years ago
Anoop Saldanha 4c570777c4 delete files app-layer-tls.[ch] 15 years ago
Eric Leblond e1d966eaf6 Makefile: add sctp files to build
This patch simply adds decode-sctp files to the compilation.
15 years ago
Gurvinder Singh 7d0781b349 added support to log dropped packet as netfilter logs while in inline mode 15 years ago
Gurvinder Singh 8f8b1212af support for ssl_version keyword 15 years ago
Victor Julien 35b938a8db Don't pass config to unittests run in make check. 15 years ago
Eric Leblond 0044bb221b Add suricata unittests to 'make check'
This patch adds a run of suricata's unittests to 'make check'
15 years ago
Gurvinder Singh e5edc6e8e3 add the support to log the fast.log alerts type to syslog 15 years ago
Eric Leblond 37ee483b75 Add affinity util function and related files
This patch adds two new files which implement advanced affinity
settings.

Signed-off-by: Eric Leblond <eric@regit.org>
15 years ago
Victor Julien 3a774165fa Initial version of a inline raw reassembly function that reassembles in a sliding window. Introduce new unittest helpers for stream reassembly. 15 years ago
Victor Julien 2849d2b1d3 Initial code for stream 'inline' mode: packets that are (partly) overlapping with already accepted packets (meaning in the streams seg list) are rewritten to make sure they contain the exact same data. 15 years ago
Anoop Saldanha c9897a44a4 fast pattern support for http_cookie. Also support relative modifiers 15 years ago
Anoop Saldanha bbbedaf963 fast pattern support for http_method. Also support relative modifiers 15 years ago
Eric Leblond 3eada85ff8 Add interface setting discovery via ioctl
This patch adds support for MTU discovery of link following idea
of go.ph1g. It also adds some function to give a approximation of
link header length.
15 years ago
Anoop Saldanha eecf2d7e13 Add the makefile.am addition that I forgot to add in the previous commit for http_raw_header 15 years ago
Anoop Saldanha c61c68fd36 mpm and fast pattern support for http_header. Also support relative modifiers for http_header 15 years ago
Anoop Saldanha 5c6a65dc58 support relative modifiers for http_client_body. Introduce body processing engine in detect-engine-hcbd.[ch] 15 years ago
Gurvinder Singh b7da115e6d support for http_stat_code keyword has been added to detection module 15 years ago
Gurvinder Singh 1deae70cf7 added http_stat_msg keyword support for detection module 15 years ago
Anoop Saldanha 88d94b136d Support for reference.config file 15 years ago
Anoop Saldanha 658ff5753d aho-corasick for the cpu. We have 2 versions of ac. The first MPM_AC uses the delta table and the secone one MPM_AC_GFBS uses the goto-failure table 15 years ago
Gurvinder Singh 3eab715153 support for printing protocol names for known protocol 15 years ago
Victor Julien 1859ed54c7 Add memcmp api with a plain memcmp function and a SSE3 accelerated memcmp. 15 years ago
Victor Julien 87f88867f4 Further improve B2gc. Add B2gm. Improve memory layout. 15 years ago
Victor Julien 9dfbab42f8 WIP B2gc 15 years ago
Pablo Rincon 9d7baa7a9f Adding ssh app layer module with two new keywords: ssh.protoversion and ssh.softwareversion 15 years ago
Anoop Saldanha 33f4beb0bc batching of packets support for cuda b2g mpm. Supported for both 32 and 64 bit platforms 16 years ago
Victor Julien e685579231 Add optional structure validation code. 16 years ago
Pablo Rincon eed0ef6e69 Adding tag keyword support 16 years ago
Kirby Kuehl c3b9305259 dcerpc udp support 16 years ago
Anoop Saldanha 45ea0d914e dce stub content keywords support using dcepayload.c support for all dce related content keywords 16 years ago
Jason MacLulich 835630efbd Add initial support for reading packets from a DAG card, we only support reading from a single stream at this time.
Use the --dag <dagname> cmd line option to specify from which DAG card to read pkts
from.

Issue at the moment with pkts being ejected during shutdown -- at the moment we
ignore any packets that are not of link type Ethernet.
16 years ago
Ondrej Slanina 6bf7d76005 added possibility to run suricata as WIN32 service 16 years ago
Victor Julien 0140a14a15 Introduce atomic operations API that supports GCC's atomic operations and a fallback using (spin)locks. Convert ringbuffer api to use the new atomic api. 16 years ago
Gerardo Iglesias Galvan 55dfa36963 Add support for http_uri keyword 16 years ago
Victor Julien a48a767efc Lockfree ringbuffer wip. 16 years ago
Jason Ish a93b2e6b84 Support for reading ERF files. 16 years ago
Gurvinder Singh 5fe1dc1d24 support for sslv2/sslv3 their unit tests and better stream no reassembly flag handling 16 years ago
Victor Julien b8641f300d Rename asn1 files, fix an invalid free, fix improper init of vars in one unittest. 16 years ago
Pablo Rincon 3fa3229e01 ASN1 decoder and keyword implementation 16 years ago
Victor Julien 70b32f7380 First stab at creating a stateful detection engine.
Stateful detection for app layer detection keywords, except uricontent. Stores it's partial results in the flow structure. Other modifications:

- Generalize transaction tracking, logging and inspection.
- Adapt http and dcerpc to use the new transaction handling.
- Stream engine now always notifies app layer of a stream eof.

This commit fixes bug #124.
16 years ago
Jason Ish 18e5ac8cde Basic rule profiling even though the results may be skewed by a bad rule in a grouping of rules. 16 years ago
Pablo Rincon e18e2ec998 Changing threshold logic 16 years ago
Pablo Rincon 1238668961 Adding actions order and suport for rule action "pass" 16 years ago
Victor Julien 070ed778b8 Libcap-ng support by Gurvinder Singh and myself. Basic support for per thread caps is added, but not activated as it doesn't seem to work yet. Work around for incompatibility between libnet 1.1 and libcap-ng added. 16 years ago
Pablo Rincon ab02ab9ead adding http_header keyword support 16 years ago
Victor Julien 54aa1790f3 Remove Makefile.am reference to non-existing file. 16 years ago
Pablo Rincon 9803def006 Adding pidfile support (thanks to Steve Grubb for the patch) 16 years ago
Anoop Saldanha 97d49d8f5e support for http_client_body keyword 16 years ago
Breno Silva 67f2026279 Global Threshold config 16 years ago
Victor Julien 08600df6b1 Small uri cleanups. 16 years ago
Pablo Rincon b708d7f65d Adding Uricontent inspection with spm. Modifiers for uricontent are now supported 16 years ago
Gerardo Iglesias Galvan ef2ae76c42 Add support for detection_filter keyword 16 years ago
Victor Julien bef70a04ce First stage of detect engine redesign: equal patterns share id's, search phase no longer used, new match verification phase. 16 years ago
Pablo Rincon 25a3a5c6d8 Adding mem wrapper to debug runtime alloc()/free() functions. Fixing some memory leaks. 16 years ago
William Metcalf 260d0d7673 Steve Grub fixes... Thanx Steve! 16 years ago
William Metcalf 0fe4373b67 Rolled back to 0.2.x branch renamed htp to libhtp 16 years ago
William Metcalf f7111f3847 import of integrated htp lib and small libnet fixes 16 years ago
Jan Jezek fe6a72befc Code is now compilable on the Win32 platform 16 years ago
Anoop Saldanha 41e6735b92 mpm b2g cuda support added 16 years ago
Anoop Saldanha 84df26d3fd cuda interface 16 years ago
Breno Silva b02bb6b6b4 VLAN Support 16 years ago
Breno Silva 7e299834d2 FragOffset Rule Keyword 16 years ago
Pierre Chifflier 4515ae13e4 Add Prelude output plugin
Add support for reporting alerts to the Prelude SIEM system, using
libprelude to send IDMEF (RFC4765) messages.

Each message contains the alert description and reference (using
the SID/GID), and a normalized description (assessment, impact,
sources etc.)

libprelude handles the connection with the manager (collecting component),
spooling and sending the event asynchronously. It also offers transport
security (using TLS and trusted certificates) and reliability (events
are retransmitted if not sent successfully).

This modules requires a Prelude profile to work (see man prelude-admin
and the Prelude Handbook for help).

Signed-off-by: Pierre Chifflier <chifflier@edenwall.com>
16 years ago
Nick Rogness 2b7b78f1bf Intial IPFW support FreeBSD and OSX 16 years ago
Pablo Rincon 260e581929 First version of the reputation API 16 years ago
Pablo Rincon 17cd010b0c Detect the number of CPUs configured and online. Printing a small summary at the startup 16 years ago
Victor Julien d446b85237 Remove obsolete files. 16 years ago
Steve Grubb f853da7940 Get make distcheck working
Hello,

Below is a patch that gets "make distcheck" working. Its against the
current code in git. The project version was set to 0.1 in configure,
I changed that to 0.8.1 just so its actually relevant. You might want
to set that to something else.

After checking this patch, I find that there are several source code
files in src/ that are not getting compiled:

-app-layer-detect.c
-app-layer-detect.h
-app-layer-http.c
-reputation.h

Are these new or abandoned? Anyways...here's the patch.

-Steve
16 years ago
Victor Julien 0d34990d7f Add OpenBSD's strlcpy and strlcat and replace all strcat/strcpy/strncat/strncpy by those calls. 16 years ago
Gurvinder Singh f6b0c481b0 urilen support for engine 16 years ago
Pablo Rincon 705471e4ee Adding single pattern matcher algorithms. If you cannot store a context for the patterns, use SpmSearch() macro. Adding unittests and stats 16 years ago
Pablo Rincon 673afeb4d3 fmemopen wrapper added (fix compilation problems on macosx and freebsd) 16 years ago
Breno Silva 1d055b0e09 ICMP Seq Rule Keyword 16 years ago
Eric Leblond 6cf00d6204 Fix typo in Makefile.am
This patch fixes a typo in Makefile.am which was preventing
'make tags' from working.
16 years ago
William Metcalf 8a64321340 raw pcap support additionl ipv4/6 validation 16 years ago
Pablo Rincon b6a3395c08 Adding unittest helper functions for building generic packets, checking arrays of expected match results, perform generic tests, etc. Look at util-unittest-helper.c and detect-ipproto.c for references 16 years ago
Jason Ish e204d07717 Have output modules register themselves so run mode configurator becomes aware of them for purposes of being configured from the config file. 16 years ago
Brian Rectanus c22d42693a Added http_method rule keyword. 16 years ago
Anoop Saldanha f684989f98 dce_iface, dce_opnum, dce_stub_data keyword support 16 years ago
Anoop Saldanha bc4df59414 Support for Classtype keyword and Classification Config file 16 years ago
Victor Julien f0be69dcd0 Fixup smb/smb2/dcerpc wrt loops, debug printing, style. 16 years ago
Victor Julien d5c732f1f9 Add tag keyword stub 16 years ago
Victor Julien 6beee776ca Move rand seed code into util-random 16 years ago
Pablo Rincon f2f9b83280 Adding FTP app layer parser and ftpbounce detection at L7 16 years ago
Gerardo Iglesias Galvan 7e87f373b9 Add icmp_id keyword support 16 years ago
Victor Julien 493715c0d2 Implement alert sid storage in the flow so we can check previous alerts in the flow. 16 years ago
Gerardo Iglesias Galvan e917065e26 Add support for daemon, checking for valid combination of modes 16 years ago
Breno Silva 69eb869cc9 Threshold Rule 16 years ago
Victor Julien ecf86f9c23 Rename to Suricata. 16 years ago
Gurvinder Singh a0f184866c http_cookie keywork support 16 years ago
Gurvinder Singh fc2f7f29fa app layer htp error handling and fixes for memory leaks and segv 16 years ago
Pablo Rincon 1ad6d75dfe Added rpc keyword support at packet level 16 years ago
Pablo Rincon a8d7b71490 First version of flowints 16 years ago
Gerardo Iglesias 991d421394 Changed printf's to logging API functions 16 years ago
Gurvinder Singh 07f7ba55b8 initial support for HTP module init 16 years ago
Victor Julien 2cfa284999 Fix app layer detect to actually work. 16 years ago
Victor Julien f1f7df0766 First iteration of doing app layer detection. 16 years ago
Anoop Saldanha 6ca5dbc9e9 Support fast_pattern modifier keyword for content 16 years ago
Anoop Saldanha dc44700ce5 Support vars lookup from conf file. Current patch support address and port group vars lookup 16 years ago
Anoop Saldanha 7dbc117b37 Host OS Table API. Modifications also make to the radix tree to handle netblocks 16 years ago
Gurvinder Singh cacbf31aad support for ttl keyword 16 years ago
Kirby Kuehl ecaa701bdf smb and dcerpc work 16 years ago
Brian Rectanus ed30067bd7 Ack/Seq Keywords 16 years ago
Jason Ish e0b9e85230 Break out checksum fixup code to make the license separation more clear. 16 years ago