Commit Graph

16946 Commits (61d47da3598cbdf245c8478c073063e7ec68d6ef)
 

Author SHA1 Message Date
Juliana Fajardini 61d47da359 lua: remove old lua flow calls
Moving forward, the flowlib is to be used.

Task #7489
8 months ago
Juliana Fajardini b03d1aed81 lua: add initial suricata.flow lib
Methods:
`get` creates the flow object.
`id`  returns the flow id.
`has_alerts` returns a boolean indicating if the flow triggered alerts.
`app_layer_proto` returns various app-layer related fields as 5 strings:
  alproto, alproto_ts, alproto_tc, alproto_orig, alproto_expect.
`stats` returns cnts for bytes and packets to sever and to client, as 4
  numbers.
`tuple` -- returns various fields: srcip, dstip, proto, sp, dp.
`timestamps` returns time as 4 numbers: seconds and microseconds, for
  first and last packet of the flow.
`timestring_legacy` returns the first packet from the flow's timestring
  as a string (like fastlog).
`timestring_iso8601` returns the first packet from the flow's
  timestring as an iso8601 compat timestring (like eve).

Example:

```
name = "lua-scflowstats.log"

local flow = require("suricata.flow")

function init(args)
    local needs = {}
    needs["type"] = "flow"
    return needs
end

function setup(args)
    filename = SCLogPath() .. "/" .. name
    file = assert(io.open(filename, "a"))
    SCLogInfo("lua SCFlowStats Log Filename " .. filename)
end

function log(args)
    local f = flow.get()
    timestring = f:timestring_legacy()
    tscnt, tsbytes, tccnt, tcbytes = f:stats()

   file:write ("[**] " .. timestring .. "\nSCFlowStats is\nPacket count to server:  " .. tscnt .. "\nByte count to server: " .. tsbytes .. "\nPacket count to client: " .. tccnt .. "\nByte count to client: " .. tcbytes .. "\n[**]")
    file:flush()
end

function deinit(args)
    file:close(file)
end
```

Task #7489
8 months ago
Juliana Fajardini 5e86c662ef lua: fix typos 8 months ago
Jason Ish 29db2e4bbe lua/dnp3: convert done and complete to boolean
These were integers, 1 of true, 0 for false. Convert to boolean as
that is how they are in eve, and Lua has boolean types.

Ticket: #7601
8 months ago
Jason Ish c69dfc5792 lua/dnp3: add is_request boolean
A DNP3 message can be a request or response, but not both. This is how
the transaction is structured.

Instead of having 2 values, "has_request" and "has_response", just
provide one field, "is_request" as a boolean.

Ticket: #7601
8 months ago
Jason Ish 9b7cab1f21 lua: convert dnp3 to suricata.dnp3 lib
This is an initial 1:1 conversion which is rather simple, as DNP3 only
had one function which converted the whole transaction to a DNP3
table.

Ticket: #7601
8 months ago
Jason Ish 4b8ef41df8 lua: remove buffer type dns.rrname, not needed
DNS is transaction based, Lua scripts need only express interest in
the request or the response.
8 months ago
Jason Ish 7b3763ecad lua: allow for real booleans in "needs" expression
In addition to

    function init (args)
        local needs = {}
        needs["dnp3"] = tostring(true)
        return needs
    end

allow for

    function init (args)
        return {dnp3=true}
    end

with the idea that the former will be completely removed for 8.0.

This works with all existing SV tests and as a string value, any value
is considered "truthy".
8 months ago
Jason Ish 640e4b343d script/dnp3_gen.py: use current clang style 8 months ago
Jason Ish bb7089df93 script/dnp3-gen.py: update for newer versions of Python
More recent yaml loaders require the loader as an argument.
8 months ago
Alice Akaki 7ba4ebdc2c detect: add email.cc keyword
email.cc matches on MIME EMAIL Carbon Copy
This keyword maps to the EVE field email.cc[]
It is a sticky buffer
Supports prefiltering

Ticket: #7588
8 months ago
Alice Akaki 9c3c6cf4cc detect-email.c: don't return NULL for empty buffer
Just return NULL if tx->mime_state is NULL or if SCDetectMimeEmailGetData return 0

Fixes:
09db7c7 ("detect: add mime email.subject keyword")
90aab0d ("detect: add email.from")
8 months ago
Alice Akaki 9e7d23d73f doc: add keywords to the multi-buffer-matching list 8 months ago
Jason Ish 77b94b8713 rust: fixes for breaking change on deranged crate
Deranged v0.4.1 (a dependency of the time crate) has implemented
PartialOrd for some integer types that conflict with the
implementation in the standard library creating an ambiguity as such
implementation are global. For more info see
https://github.com/jhpratt/deranged/issues/18.

To fix, use "::from" directly, instead of using .into() which is where
we run into amgibuity.
8 months ago
Jason Ish 1a47fdfd46 doc/userguide: group af-packet upgrade notes together
Also fix the rendering of the sip nest list.
8 months ago
Jason Ish 080d48ba29 doc/userguide: upgrade note about defrag now off for inline use
Ticket: #7617
8 months ago
Jason Ish 07d98bf75a af-packet: don't default to defrag when inline
Don't default to defrag true when inline. It can still be enabled by
setting defrag to true however it is not recommended.

Ticket: #7617
8 months ago
Jason Ish 8fe526006d doc/userguide: upgrade note about tpacket-v3 default for ids
Ticket: #4798
8 months ago
Binghui Niu 854d0bbcc0 applayer: fix alp_ctx indexing in tests
Fix problem that some app-layer unittests regist failed.
8 months ago
Alice Akaki 5d6a072e35 detect: add email.to keyword
email.to matches on MIME EMAIL TO
This keyword maps to the EVE field email.to[]
It is a sticky buffer
Supports prefiltering

Ticket: #7596
8 months ago
Philippe Antoine 09aed7e243 fuzz: set flow flags as in Suricata
Fixes: d8ddef4c14 ("detect: delay tx cleanup in some edge case")
8 months ago
Alice Akaki 09db7c7ac1 detect: add mime email.subject keyword
email.subject matches on MIME EMAIL SUBJECT
This keyword maps to the EVE field email.subject
It is a sticky buffer
Supports prefiltering

Ticket: #7595
8 months ago
Jason Ish 7fdb08b7c7 af-packet: use tpacket v3 by default for ids mode
If "tpacket-v3" is not present in the configuration file, and we are
in IDS mode, default to "tpacket-v3".

Required moving the check for tpacket-v3 after the copy-mode
configuration, so the warning about tpacket-v3 in active modes has
been moved as well.

Ticket: #4798
8 months ago
Jason Ish 374762d202 af-packet: remove use-mmap option
This option is obsolete and was not used in 7.0 as tpacket-v1 support
was removed (see ticket #4796).
8 months ago
Jason Ish 51f7b5924d af-packet: remove build conditional for tpacket-v3
All kernels on supported distrubtions should now support tpacket-v3,
so only enable af-packet if v2 and v3 are available.
8 months ago
Juliana Fajardini c595bfed87 pgsql/parser: use fn for length parsing...
... there was still one parser missing this conversion.
8 months ago
Juliana Fajardini 0d2bea9bab util/exception: fix coverity warning
CID 1644862:  Control flow issues  (UNREACHABLE)

In ExceptionPolicyTargetFlagToString, a statement cannot be reached
(line 113).
8 months ago
Juliana Fajardini a9b2a62ee4 userguide/exceptions: clarify when stats are logged
The stats for exception policies are only logged/ present when any of
the exception policies are enabled (which means any value other than
"auto" or "ignore" in IDS mode, or "ignore" in IPS mode).

This wasn't clearly stated in the docs.
8 months ago
Juliana Fajardini 08e928988f flow/output: log triggered exception policies
To accompany the Exception Policy stats, also add information about any
Exception Policy triggered and for which target to the flow log event.

Task #6215
8 months ago
Victor Julien 45500acdc4 tls: implement alert parser
Fatal alerts set the tx state to 'finished'.

Add event for malformed alerts.
8 months ago
Jason Ish 7568b8020d dns: stop renaming DNSTransaction to RSDNSTransaction
Not needed anymore as there is no DNSTransaction in the C src to
conflict.
8 months ago
Jason Ish a6b116bcbe lua: document new suricata.dns lua library
Ticket: #7602
8 months ago
Jason Ish 1206c1c5af lua: convert dns function into suricata.dns lib
Notable changes from the previous API:
- rcode will return the rcode as an integer
- rcode_string will return the string representation

Also fixes an issue where an rcode of 0 was returned as nil.

Ticket: #7602
8 months ago
Alice Akaki 90aab0d62f detect: add email.from
email.from matches on MIME EMAIL FROM
This keyword maps to the EVE field email.from
It is a sticky buffer
Supports prefiltering

Ticket: #7592
8 months ago
Alice Akaki 90cf59ac71 rustfmt: rust/src/mime/smtp.rs 8 months ago
Joyce Yu 1c50de4ad0 Eve: use mac addresses from flow for flow timeout
Ethernet metadata is missing for events triggered on flow timeout
pseudopackets. Use the first set of mac addresses stored with the
flow to fill in the ether field.

Ticket: #5486
8 months ago
Joyce Yu dac0d6371e Doc: update eve-json-output ethernet description
Document getting mac addresses from flow when flow timeout.
8 months ago
Victor Julien 749ffbd06a af-packet: use actual snaplen in bpf
Avoids setting a 0 snaplen in BPF, leading to an error.

Fixes: b8b6ed550a ("af-packet: delay setting default-packet-size for af-packet")

Ticket: #7618.
8 months ago
Victor Julien c6fdf99cec datasets: work around scan-build warning
datasets.c:493:27: warning: Dereference of null pointer [core.NullDereference]
  493 |     DEBUG_VALIDATE_BUG_ON(set->hash->config.hash_size != hashsize);
      |                           ^~~~~~~~~~~~~~~~~~~~~~~~~~~
./util-validate.h:95:44: note: expanded from macro 'DEBUG_VALIDATE_BUG_ON'
   95 | #define DEBUG_VALIDATE_BUG_ON(exp) BUG_ON((exp))
      |                                            ^~~
./suricata-common.h:307:36: note: expanded from macro 'BUG_ON'
  307 |         #define BUG_ON(x) assert(!(x))
      |                                    ^
/usr/include/assert.h:109:7: note: expanded from macro 'assert'
  109 |     ((expr)                                                             \
      |       ^~~~
1 warning generated.
8 months ago
Shivani Bhardwaj be372ce39d doc: explain priority port setting
Ticket 7329
8 months ago
Shivani Bhardwaj 040c694256 doc: format and align suricata.yaml section 8 months ago
Juliana Fajardini 3985b24e1b upgrade: list inspection recursion default limit
As the yaml indicated before that if no value was specified there were
no limits, and now there will be one.
8 months ago
Juliana Fajardini e1f9e66af0 doc/upgrade: add datasets hash size limit note 8 months ago
Philippe Antoine a7713db709 detect: add configurable limits for datasets
Ticket: 7615

Avoids signatures setting extreme hash sizes, which would lead to very
high memory use.

Default to allowing:
- 65536 per dataset
- 16777216 total

To override these built-in defaults:

```yaml
datasets:
  # Limits for per rule dataset instances to avoid rules using too many
  # resources.
  limits:
    # Max value for per dataset `hashsize` setting
    #single-hashsize: 65536
    # Max combined hashsize values for all datasets.
    #total-hashsizes: 16777216
```
8 months ago
Victor Julien d32a39ca4b datasets: improve default hashsize handling
Make hashsize default local to dataset code, instead of relying on the
thash code.

Use the same default value as before.
8 months ago
Jason Ish c6d18fc871 doc/userguide: af-packet upgrade notes
Add note about increased block size and how to change it back to old
defaults if needed.

Ticket: #7458
8 months ago
Jason Ish b8b6ed550a af-packet: delay setting default-packet-size for af-packet
AF_PACKET needs more information about its configuration before we can
set the default packet size, so on startup, leave unset in suricata.c
if in AF_PACKET mode.

If defrag is enabled, use a default packet size of 9k for tpacket-v2.
This can still lead to truncation events, then the user can increase
their 'default-packet-size'.

Tpacket-v3 does not need an increased packet size as it will handle
any size of packet that is smaller than the configured block size
which now has a default of 128k.

9k for the snap is somewhat arbitrary but is large enough for the
common 9000 jumbo frame plus some extra headers including tpacket
headers.

Ticket: #7458
8 months ago
Jason Ish 8c7ac89791 af-packet: warn that tpacket-v3 is better for non-inline usage
Ticket: #7458
8 months ago
Jason Ish d78f2c9a4e af-packet: add event for packets truncated by af-packet
Ticket: #7458
8 months ago
Jason Ish 9f96975d55 af-packet: warn if v3 block size is not large enough for defrag
If using tpacket-v3 and defrag, warn if the block size is not large
enough for a fully defragmented packet.

Ticket: #7458
8 months ago