Commit Graph

18611 Commits (0cdc77b7078e3b348ebd2d53a03a0baadc2c76e8)
 

Author SHA1 Message Date
Lukas Sismis 0cdc77b707 hs: use binary mode for cache file I/O
HSSaveCache wrote serialized Hyperscan databases using text mode ("w")
while HSReadStream already read them with binary mode ("rb").
Matched file reading modes to the binary format and simplified
write-size check.

Ticket: 8325
2 months ago
Lukas Sismis d754b28717 hs: remove redundant file handle in HSLoadCache
HSLoadCache opened the cache file but never used the resulting handle
for reading. The actual read was done by HSReadStream which opened
the same file independently.

Removed the unused fopen/fclose pair and flattened the control flow.

Ticket: 8325
2 months ago
Victor Julien 569ba3d26f hs/cache: cleaner and more detailed output
Reduce logging level of a minor informational message.

Split tracking of pruning by age and by version and log those
separately, where the logging only appears if something has been
removed.

Ticket: 8322
2 months ago
Jason Ish 66473f6ad7 doc: add more public datasets
Add:
- UNSW-NB15
- CIC-IDS2017
- CSE-CIC-IDS2018
- CIC-DDoS2019

Update formatting for better display.
2 months ago
Victor Julien e69b732d1c af-xdp: address compile warnings
runmode-af-xdp.c:119:52: error: implicit conversion loses integer precision: 'const int' to 'uint16_t' (aka 'unsigned short') [-Werror,-Wimplicit-int-conversion]
  119 |         aconf->threads = (nr_cores <= nr_queues) ? nr_cores : nr_queues;
      |                        ~                           ^~~~~~~~
runmode-af-xdp.c:119:63: error: implicit conversion loses integer precision: 'const int' to 'uint16_t' (aka 'unsigned short') [-Werror,-Wimplicit-int-conversion]
  119 |         aconf->threads = (nr_cores <= nr_queues) ? nr_cores : nr_queues;
      |                        ~                                      ^~~~~~~~~
runmode-af-xdp.c:135:26: error: implicit conversion loses integer precision: 'const int' to 'uint16_t' (aka 'unsigned short') [-Werror,-Wimplicit-int-conversion]
  135 |         aconf->threads = nr_queues;
      |                        ~ ^~~~~~~~~
runmode-af-xdp.c:268:41: error: implicit conversion loses integer precision: 'intmax_t' (aka 'long') to 'uint32_t' (aka 'unsigned int') [-Werror,-Wshorten-64-to-32]
  268 |                 aconf->busy_poll_time = conf_val_int;
      |                                       ~ ^~~~~~~~~~~~
runmode-af-xdp.c:275:43: error: implicit conversion loses integer precision: 'intmax_t' (aka 'long') to 'uint32_t' (aka 'unsigned int') [-Werror,-Wshorten-64-to-32]
  275 |                 aconf->busy_poll_budget = conf_val_int;
      |                                         ~ ^~~~~~~~~~~~
runmode-af-xdp.c:282:40: error: implicit conversion loses integer precision: 'intmax_t' (aka 'long') to 'uint32_t' (aka 'unsigned int') [-Werror,-Wshorten-64-to-32]
  282 |             aconf->gro_flush_timeout = conf_val_int;
      |                                      ~ ^~~~~~~~~~~~
runmode-af-xdp.c:287:43: error: implicit conversion loses integer precision: 'intmax_t' (aka 'long') to 'uint32_t' (aka 'unsigned int') [-Werror,-Wshorten-64-to-32]
  287 |             aconf->napi_defer_hard_irqs = conf_val_int;
      |                                         ~ ^~~~~~~~~~~~
7 errors generated.

source-af-xdp.c:426:24: error: implicit conversion loses integer precision: 'int' to 'uint8_t' (aka 'unsigned char') [-Werror,-Wimplicit-int-conversion]
  426 |     ptv->afxdp_state = state;
      |                      ~ ^~~~~
source-af-xdp.c:642:44: error: implicit conversion loses integer precision: 'uint32_t' (aka 'unsigned int') to '__u16' (aka 'unsigned short') [-Werror,-Wimplicit-int-conversion]
  642 |     ptv->xsk.cfg.bind_flags = afxdpconfig->bind_flags;
      |                             ~ ~~~~~~~~~~~~~^~~~~~~~~~
2 errors generated.
2 months ago
Victor Julien edc19a15ad detect/parse: convert unittest to fail/pass API 2 months ago
Victor Julien 02f6340972 github-ci: update cov builds
Use --skip-branches to avoid countless partial coverage in unittests.

Update from 22.04 to 24.04 and llvm 15 to 19 to support --skip-branches.
2 months ago
Jason Ish ccd1df99fd github-ci: add coveralls coverage uploads 2 months ago
Jason Ish accaf68615 github-ci: fix cov-ut profile file collision 2 months ago
Jason Ish 65e87949c8 github-ci: ignore /github from coverage output 2 months ago
Jason Ish 8c99759dfe github-ci: ignore /usr and /rustc in coverage 2 months ago
Jason Ish d3b6bae140 github-ci: upload lcov coverage artifacts 2 months ago
Victor Julien e991375b04 github-ci: for codecov netns check, account for cargo build dir
It's not in /root/ like with container based builds.
2 months ago
Jason Ish e6513b5117 codecov: upload lcov reports to codecov
Seems to fix our issues with codecov.io.
2 months ago
Philippe Antoine 71b59f6dbe rust/ffi: move IPPROTO_TCP to ffi
and reexport in suricata

Allows to reduce dependencies to full suricata crate

Ticket: 7666
2 months ago
Philippe Antoine 2a90cef318 plugin: reduce dependency on suricata crate
Ticket: 7666

The end goal is to remove all dependencies on suricata and just
use suricata_sys or suricata_ffi

For now, make usage of what is already available
2 months ago
Juliana Fajardini 926fde859e userguide: fix Deprecations, warn about syslog
The Deprecation note on Syslog was partly conveying the opposite
message, and we were missing a deprecation notice on the Syslog output
section.
2 months ago
Philippe Antoine d5ea973791 detect/ike: move ike.ike.chosen_sa_attribute keyword to rust
Ticket: 8310

And increases expressivity on the way by supporting other modes
than equality
2 months ago
Philippe Antoine 55333a6ca0 detect/ike: move ike.vendor keyword to rust
Ticket: 8310
2 months ago
Philippe Antoine 14edd66494 detect/ike: move ike.nonce_payload keyword to rust
Ticket: 8310
2 months ago
Philippe Antoine d1213e4440 detect/ike: move ike.key_exchange_payload keyword to rust
Ticket: 8310
2 months ago
Philippe Antoine f23169c924 detect/ike: move ike.nonce_payload_length keyword to rust
Ticket: 8310
2 months ago
Philippe Antoine e85bf5be4d detect/ike: move ike.key_exchange_payload_length keyword to rust
Ticket: 8310
2 months ago
Philippe Antoine 841fe44f34 detect/ike: move ike.exchtype keyword to rust
Ticket: 8310
2 months ago
Philippe Antoine 23032ebc65 detect/ike: move ike spi keywords to rust
Ticket: 8310
2 months ago
Victor Julien 28ba93e60c ndpi: minor optimization
Check protocol before doing more expensive work.
2 months ago
Victor Julien 278da4bc6a npdi: fix null pointer deref in detect keywords
Check for the existence of a flow before calling FlowGetStorageById() on it.

Ticket: #8308.
2 months ago
Juliana Fajardini 723aa21e6f misc: update copyright years
For files edited during

Task #7885
2 months ago
Juliana Fajardini dc4d805908 exceptions: s/DEBUG/QA_SIMULATION build flag
Task #7885
2 months ago
Juliana Fajardini 5edb5a5d1a build-scopes: add QA_SIMULATION mode
For qa-simulation scenarios, especially for testing purposes.

Task #7885
2 months ago
Victor Julien e7c9be07e6 af-xdp: add missing unlock in error path
CID 1638288: (#1 of 1): Missing unlock (LOCK)
4. missing_unlock: Returning without unlocking xsk_protect.queue_protect.
2 months ago
Victor Julien b84ae80146 affinity: address coverity warning
CID 1667318: (#1 of 1): Structurally dead code (UNREACHABLE)
unreachable: This code cannot be reached: hwloc_obj_t non_io_ancestor....
2 months ago
Sven Cuyt 1246f1cf8c doc/userguide: add link to differences-from-snort
Ticket: #8031.
2 months ago
Sven Cuyt a0cc534471 doc/userguide: fix table showing example payload
Ticket: #8031.
2 months ago
Sven Cuyt 2d662538e5 doc/userguide: add example with non-zero offset for relative isdataat
Ticket: #8031.
2 months ago
Victor Julien 7c5285e5e6 doc/userguide: explain isdataat absolute vs relative difference
Ticket: #8031.
2 months ago
dependabot[bot] fe5c2250e1 github-actions: bump github/codeql-action from 4.32.0 to 4.32.3
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 4.32.0 to 4.32.3.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Commits](https://github.com/github/codeql-action/compare/v4.32.0...v4.32.3)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-version: 4.32.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2 months ago
Philippe Antoine b37b2b6bd6 decode: use compact uint8_t instead of enum in struct
to save memory

also move it to the first cacheline, as there is a 2 byte gap there
and this field should also not change after it has been set.
2 months ago
Philippe Antoine 9807fe4326 decode/xvlan: treat as its own tunnel
Ticket: 7717

Allows for instance to process/log ARP packets over VXLAN.

That means we need to decode the ethernet layer above vxlan
instead of skipping it as part of the vxlan, even if the vxlan
decoder still checks the ethernet layer to avoid FPs.
2 months ago
Philippe Antoine efb6daa1ae decode: use PacketIsTunnelChild
Instead of directly accessing the field

Will allow PacketTunnelType to hold the precise tunnel type like
DECODE_TUNNEL_ERSPANII with a modification of PacketIsTunnelChild
2 months ago
Philippe Antoine dc29ab46ed ebpf: check maps compatibility
ebpf program does not handle 3 layers of vlan
2 months ago
Philippe Antoine d378610e93 stream: non infinite default value for reassembly.depth
So that empty config have a safer default.

Ticket: 8290
2 months ago
Victor Julien db2e97ab5f detect/iprep: mark as supported for firewall mode
Ticket: #8285.
2 months ago
Ofer Dagan e1f9d8a067 pcap: refactor delete-when-done to support non-alerts
Refactor pcap file deletion to use a single delete-when-done option
with three values instead of separate boolean options:
- false (default): No deletion
- true: Always delete files
- "non-alerts": Delete only files with no alerts

Also account for alerts produced by pseudo packets (flow timeout / shutdown flush):
- Introduce small capture hooks and invoke on pseudo-packet creation so the
  capture layer can retain references and observe alerts emitted after the last
  live packet
- Call the hook from both TmThreadDisableReceiveThreads and TmThreadDrainPacketThreads

Key changes:
- Replace should_delete/delete_non_alerts_only bools with enum
- Move alert counter from global to per-file PcapFileFileVars
- Relocate alert counting from PacketAlertFinalize to pcap module
- Ensure thread safety for both single and continuous pcap modes
- Add unit tests for configuration parsing and pseudo-packet alert path

The --pcap-file-delete command line option overrides YAML config
and forces "always delete" mode for backward compatibility.

Documentation updated to reflect the new three-value configuration.

Fixes OISF#7786
2 months ago
Philippe Antoine 539e4ee665 detect/transforms: add zlib_deflate transform
Ticket: 7846
2 months ago
Philippe Antoine dbea660729 detect/transforms: add gunzip transform
Ticket: 7846
2 months ago
Philippe Antoine 751ef1fe9b detect: add debug validation about nlists
To avoid stack overflow

A single signature should not have more than 65k buffers

Ticket: 8001
2 months ago
Philippe Antoine f4d83f29a9 detect: allocate sids_array on the heap
To be able to load 1M signatures and not have a stack overflow

Ticket: 8001
2 months ago
Philippe Antoine 256c6a99b8 datasets: adds safety check before stack allocation
Even if current caller does not provide too big data.

Ticket: 8001
2 months ago
Victor Julien 336a9d05ca krb5: fix TCP record parsing
A logic error in multi-record parsing meant only the first record was parsed.

Bug: #8278.
2 months ago