Commit Graph

197 Commits (7dbae32eda0118fefccee25c53134242d0df8d76)

Author SHA1 Message Date
Victor Julien 83dc703d1f defrag: add various counters 2 years ago
Victor Julien fc05d253d2 defrag: add defrag.mgr.tracker_timeout counter
Updated by flow manager.
2 years ago
Victor Julien 76e05c72f6 eve/schema: reformat 2 years ago
Jason Ish 10e6028175 lua: track memory limit exceede errors
Update the Lua allocated to set a code on memory allocation limit
exceeded errors so an appropriate error message can be logged and a
state incremented.

Fixes the tracking of the allocated size by using the difference
between original size, and new size and toss in some debug
validations.
2 years ago
Jason Ish 5a1cba72f0 lua: add logging and counter for instruction limit being exceeded 2 years ago
Jason Ish c8fa454cb2 lua: add blocked functions as a special log type plus stat
Distinguish between a generic Lua script error and an error created by a
function being blocked, so each is logged once respective of each other.

Also add a stat that is incremented when a script fails due to a
blocked function.

NOTE: This does not catch calls to functions that are blocked by not
having the library loaded, such as "io.open", as they are blocked by
not even loading the "io" library.
2 years ago
Shivani Bhardwaj f073cf2350 eve/schema: add tls.subjectaltname fields
Feature 5234
2 years ago
Jason Ish 224f55ba21 detect/lua: don't treat a crashed script as no match
If a rule script crashed, the return value was treated as a no
match. This would make a negation of the rule match and alert.

Instead cleanup and exit early if the rule script crashed and don't
run negation logic.

A stat, detect.lua.errors has been added to count how many times a
script crashes.

Also consolidates the running of the Lua script and return value
handling to a common function.

Bug: #6940
2 years ago
Philippe Antoine 2c305ba37e pop3: protocol detection
Ticket: #6366
2 years ago
Giuseppe Longo 01586d884d output-json/arp: implement logger
This adds a logger for ARP, disabled by default.

Ticket #6827
2 years ago
Giuseppe Longo 5219a5da5f decode/arp: implement decoder
This adds a decoder for ARP.

Ticket #6827
2 years ago
Shivani Bhardwaj 329ac61961 eve/stats: add description for ips
Ticket 6434
2 years ago
Shivani Bhardwaj 861ffff972 eve/stats: add description for transactions
Ticket 6434
2 years ago
Giuseppe Longo bff790b6ac rust/sdp: implement logger
This implements a logger for the SDP protocol.
Given that SDP is encapsulated within other protocols (such as SIP),
enabling it separately is not necessary.

Ticket #6627
2 years ago
Philippe Antoine 0291d37009 websocket: configurable logging of payload in alerts 2 years ago
Philippe Antoine 44b6aa5e4b app-layer: websockets protocol support
Ticket: 2695
2 years ago
Sascha Steinbiss 120313f4da ja4: implement for TLS and QUIC
Ticket: OISF#6379
2 years ago
Jeff Lucovsky 2dfa4cecb5 stats: Memcap pressure max relocation
This commit moves the memcap pressure/pressure_max stats from the global
stats namespace into the memcap namespace.

With per-thread stats, they will be within the flow-manager's values.

Issue: 6398
2 years ago
Juliana Fajardini caf590d51f stream/midstream: add counter for exception policy
Add stats counters for when there is an exception policy applied in case
of a session picked up midstream.

Task #5816
2 years ago
Juliana Fajardini fd9a20ffcf stream/reassemble: add exception policy counters
Add stats counters for exception policies applied in case of memcap hit
during stream reassembly.

Task #5816
2 years ago
Juliana Fajardini 2dee3772bf stream/tcp: add ssnmemcap exception policy counter
Add stats counters for exception policies applied in case a stream
session memcap is hit.

Task #5816
2 years ago
Juliana Fajardini a71ace8575 applayer: add stats counters for exception errors
Add stats counters for exception policy are applied for app-layer errors

Part of
Task #5816
2 years ago
Juliana Fajardini 485c0e1d9a defrag: add exception policy memcap stats counters
Add defrag memcap stats counter.

Task #5816
2 years ago
Juliana Fajardini 657419b53e decode/flow: add exception policy stats counters
We will register stats counters for all policies, even though for now
Suri only uses one possible configuration policy at a time. The idea is
that this could change in the near future, so we want to have this
ready.

Task #5816
2 years ago
Juliana Fajardini ce001d8eae schema: apply clang formatting changes 2 years ago
Arne Welzel f9cf87a003 schema: Add stats.capture and in_iface properties
New suricata-verify test listens on loopback interface, resulting
in the capture and in_iface fields in the stats and event objects.
2 years ago
Jason Ish c2ecae9b82 schema: add flow.wrong_thread 2 years ago
Giuseppe Longo c9d309219e rust/sip: register parser for tcp
This patch lets the parser to work over tcp protocol, taking care of handling
data before calling the request/response parsers.

Ticket #3351.
2 years ago
Hadiqa Alamdar Bukhari 6c193b1a3d dns: add missing dns keywords to schema.json
Found and added missing dns fields in schema.json after manual code review.
Added description to these newly added dns fields.
Feature #5642
2 years ago
Shivani Bhardwaj 487ba82fb9 eve/stats: add description for applayer flows
Ticket 6434
2 years ago
Shivani Bhardwaj 8817514bea eve/stats: add description for expectations
Ticket 6434
2 years ago
Shivani Bhardwaj 1816e98ef0 eve/stats: add description for applayer errors
Ticket 6434
2 years ago
Shivani Bhardwaj 5a1a32ba5b eve/stats: add description for common fields
Ticket 6434
2 years ago
Jason Ish 90ae3a223f eve/schema: allow authorities in dns.answers in alert
Factor out dns.authorities to a definition.
2 years ago
Jason Ish b453eea150 stats: add rules skipped
Rule skipped is a count of the number of rules that are skipped due to
missing requirements.

Feature: #6637
2 years ago
Philippe Antoine f714678d72 schema: adds missing modbus field
./stats/app_layer/error/modbus
2 years ago
Juliana Fajardini 467c3f2c64 schema: apply clang formatting changes 2 years ago
Juliana Fajardini 30ac77ce65 pgsql: add cancel request message
A CanceldRequest can occur after any query request, and is sent over a
new connection, leading to a new flow. It won't take any reply, but, if
processed by the backend, will lead to an ErrorResponse.

Task #6577
2 years ago
Philippe Antoine 8c5310aefd doc: quic in eve/schema
Ticket: #6076
2 years ago
Jeff Lucovsky 904f0ddeee stats: Track stream reassembly drops
Issue: 6235
3 years ago
Yatin Kanetkar b67ff4badf dhcp: Log Vendor Client Identifier (dhcp option 60)
* Log vendor client identifier (dhcp option 60) if extended dhcp
logging is turned on. This required the `vendor_client_identifier` to
be added to the json schema. Validation done using an SV Test
* Added `requested_ip` to the json schema as well, since it was
missed. My SV test failed without it.

Feature #4587
3 years ago
Jason Ish 3802a51552 eve/schema: add host
The "host" field is added to EVE events if the "sensor-name" field is
configured in suricata.yaml.
3 years ago
Jeff Lucovsky 424f12d1b3 schema: Add memcap pressure values
Issue: 6094

This commit extends the EVE schema with memcap_pressure values; these
are included in the stat event type records.
3 years ago
Philippe Antoine b12a35c3cf output: add storing boolean for files
When filestore keyword is triggered, the file is not yet stored,
when the alert is generated, but only marked for storing.

Ticket: 4881
3 years ago
Philippe Antoine f35052941d jsonschema: add missing field .files[].file_id 3 years ago
Victor Julien 1f9767a9cb stats: add drop reason counters
{
  "accepted": 296185,
  "blocked": 162,
  "rejected": 0,
  "replaced": 0,
  "drop_reason": {
    "decode_error": 0,
    "defrag_error": 0,
    "defrag_memcap": 0,
    "flow_memcap": 0,
    "flow_drop": 94,
    "applayer_error": 0,
    "applayer_memcap": 0,
    "rules": 3,
    "threshold_detection_filter": 0,
    "stream_error": 63,
    "stream_memcap": 0,
    "stream_midstream": 2,
    "nfq_error": 0,
    "tunnel_packet_drop": 0
  }
}

Ticket: #6230.
3 years ago
Victor Julien 735c37c668 eve/schema: add ips capture stats 3 years ago
Juliana Fajardini 0437173848 output/drop: add verdict field
Related to
Bug #5464
3 years ago
Juliana Fajardini 53b8defd79 output/alert: add verdict field
Related to
Bug #5464
3 years ago
Philippe Antoine 4f4651e360 output/file: http2 metdata is logged in http object
as is done for http2 events and alerts.
The http.version integer can help to determine if this is HTTP2

Ticket: #6165
3 years ago
Juliana Fajardini 05417407b3 schema: add missing flow event property: emergency 3 years ago
Jeff Lucovsky 9dc68ac59a json/schema: Add additional VLAN layer stat
Issue: 2816

This commit extends the JSON schema with the additional VLAN stat for
tracking VLAN encapsulated packets with 3 levels.
3 years ago
Eric Leblond a73c9b0e40 output: target keys have port
Update JSON schema to support signature with target keyword
3 years ago
Victor Julien a8057eeed8 eve/schema: spelling 3 years ago
Philippe Antoine 416a780f69 jsonschema: do not enforce keys for alert metadata
As this is a free field and can have any key based on a rule
3 years ago
Jason Ish 3a44197183 schema: add "message_id" to email 3 years ago
Jason Ish bf079c9214 schema: fix optional
"optional" is not part of jsonschema. Instead an array named "required"
is used to list all field names that are required.
3 years ago
Jason Ish 49ba378d38 schema: fix engines section
The definition of items is an object, not an array.
3 years ago
Lancer Cheng 08b17e9778 eve: add version and warning in ntlmssp
Bug OISF#5783
3 years ago
Victor Julien 7e6154a26f stream: add counter for acks for unseen data
This is another indicator for packet loss or strange captures.
3 years ago
Victor Julien 83a16a7a89 eve/stream: per packet stream engine logging
Debug facility to get a per packet view into the stream engine's state.

Logs after a packet has been processed in the stream engine, so the view
into the state includes the updates based on the current packet.

Marked as experimental so it can be changed w/o notice.

Bug: #5876.
3 years ago
Victor Julien 66ed3ae6e4 flow/mgr: remove flows_timeout_inuse counter 3 years ago
Jason Ish 59d9a51bad eve: remove dcerpc.interface from schema
Looks like this was due to an error in the dcerpc logging where the
interfaces should have been logged to the "interfaces" array that was
already defined.

Issue: 5814
3 years ago
Jason Ish ef48c5064f schema: add regular expression for tls date format 3 years ago
Jeff Lucovsky c1c67536b6 decode/stat: Add decode counters for unknown/arp
Issue: 5761

This commit adds statistics for ARP and unknown ethertype packets for
diagnostic purposes.
3 years ago
Shivani Bhardwaj 8e3acf1695 eve/schema: add udp.len_invalid 3 years ago
Jason Ish c98c49d4ba dns: parse and alert on invalid opcodes
Accept DNS messages with an invalid opcode that are otherwise
valid. Such DNS message will create a parser event.

This is a change of behavior, previously an invalid opcode would cause
the DNS message to not be detected or parsed as DNS.

Issue: #5444
3 years ago
Victor Julien 96dfd65b96 eve: log max regions 3 years ago
Jeff Lucovsky f8474344cd log: Add module and subsystem identifiers to log
Issue: 2497

This changeset provides subsystem and module identifiers in the log when
the log format string contains "%S". By convention, the log format
surrounds "%S" with brackets.

The subsystem name is generally the same as the thread name. The module
name is derived from the source code module name and usually consists of
the first one or 2 segments of the name using the dash character as the
segment delimiter.
3 years ago
Victor Julien 62a451a9ab eve/schema: bittorrent format fixup 3 years ago
Philippe Antoine 37af957d83 eve/schema: check that each array has at least one element
Ticket: #5167
3 years ago
Juliana Fajardini 84f9ea7254 eve/schema: pgsql - allow flexible parameters list
Pgsql's parameters - for message types like StartupMessage and
ParameterStatus, for instance, don't have a finite, definitive set, as
per their documentation. Our json schema was allow expecting a fixed set
of parameters, though, resulting in SV tests failing if different, valid
parameters appeared.

Bug #5579
4 years ago
Jason Ish e3e7d007b2 eve/schema: bittorrent-dht updates
Some values that were previously strings are now parsed down into
objects.
4 years ago
Jason Ish 0d3cfbbe3f bittorrent-dht/eve: log as bittorrent_dht 4 years ago
Jason Ish 66fc92276a eve-schema: add bittorrent-dht 4 years ago
Eric Leblond 27cdfec28a eve/schema: update following flow changes 4 years ago
Victor Julien 38fdfd8718 eve/schema: flow/stream updates 4 years ago
Victor Julien 308fe31cb5 eve/schema: add tls client logging 4 years ago
Victor Julien 036686e21c etc/schema: clang (re)format 4 years ago
Philippe Antoine b0ce55c9df flow: finish to remove obsolete counters
As was begun in b3599507f4

Ticket: #5317
4 years ago
Eric Leblond 2cc9152fc9 rust/smb: log uuid of interface in dcerpc
When doing a DCERPC request, we can use the context id to log the
interface that is used. Doing that we can see in one single event
what is the DCERPC interface and opnum that are used. This allows
to have all the information needed to resolve the request to a
function call.

Feature #5413.
4 years ago
Philippe Antoine e94920b49f smb: do not use tree id to match create request and response
As an SMB2 async response does not have a tree id, even if
the request has it.

Per spec, MessageId should be enough to identifiy a message request
and response uniquely across all messages that are sent on the same
SMB2 Protocol transport connection.
So, the tree id is redundant anyways.

Ticket: #5508
4 years ago
Shivani Bhardwaj 14561ffe72 eve/schema: add smtp url bool fields 4 years ago
Philippe Antoine 64b2385c64 krb: log for ticket encryption
Also logs if the ticket encryption is weak.
It is different from the encryption used for the rest of the
packet, and this allows to detect kerberoasting attack.

Ticket: #5442
4 years ago
Philippe Antoine 896f0d91ce quic: complete schema.json
adding ja3 and extension fields
4 years ago
Victor Julien 929faae6d4 eve/schema: add drop.udplen, email fields 4 years ago
Victor Julien 3617be326c eve/schema: add pcap_filename field 4 years ago
Victor Julien fc566037b4 eve/schema: add new flow fields 4 years ago
Victor Julien 2ba9da4815 eve/schema: add missing magic from files array 4 years ago
Victor Julien 2a7349406c eve/schema: add missing capture_file field 4 years ago
Victor Julien 42adaf5627 eve/schema: add missing http fields 4 years ago
Victor Julien d58f9e54d0 eve/schema: add missing alert fields 4 years ago
Victor Julien 2abce12b5b eve/schema: add missing smb fields 4 years ago
Victor Julien b24e1f1e46 eve/schema: add missing drop fields for ipv6 4 years ago
Victor Julien 6ad5d6a148 eve/schema: add profiling detect fields 4 years ago
Victor Julien 0035673208 eve/drop: log drop reason
Ticket: #5202.
4 years ago
Philippe Antoine 284ad462fc output: adds schema.json
Ticket: #1369
4 years ago