Commit Graph

6206 Commits (019f85644251fc56ae12122ec0fc6279017d96b9)
 

Author SHA1 Message Date
Victor Julien e19c41a807 multi-detect: hash lookup for tenants
Use hash for storing and looking up det_ctxs.
10 years ago
Victor Julien 722c56dbf3 detect: clean up thread free code
Introduce DetectEngineThreadCtxFree that doesn't need a 'ThreadVars'
pointer.
10 years ago
Victor Julien 4f77e8967b multi-detect: make threshold prefix aware
Make threshold loading prefix aware, so it can be part of tenant
configuration.

If the setting is missing from the tenant, the global setting is tried
and if that too is missing, the global default is used.

Note: currently per host thresholds are tracked globally and NOT per
tenant.
10 years ago
Victor Julien a6e3cec9e5 multi-detect: make reference prefix aware
Make reference loading prefix aware, so it can be part of tenant
configuration.

If the setting is missing from the tenant, the global setting is tried
and if that too is missing, the global default is used.
10 years ago
Victor Julien 5fff250749 multi-detect: make classification prefix aware
Make classification loading prefix aware, so it can be part of tenant
configuration.

If the setting is missing from the tenant, the global setting is tried
and if that too is missing, the global default is used.
10 years ago
Victor Julien 4d0975eeac multi-detect: implement reload tenant in suricatasc 10 years ago
Victor Julien d8181802d3 multi-detect: add reload-tenant command
Allow for a tenant to be reloaded. The command is the same as the
register-tenant command, so with a yaml and tenant-id as argument.
However this replaces an existing tenant.
10 years ago
Victor Julien 8673801ea3 multi-detect: add tenant id to alert json output
Add a integer field "tenant_id" to the JSON alert output.
10 years ago
Victor Julien f4c9915066 multi-detect: store tenant id in packet
Store tenant id in the packet so that the output API's can log it.
10 years ago
Victor Julien 642c267dc4 multi-detect: refuse to add duplicate tenant
Generate error if tentant to be added is already loaded.
10 years ago
Victor Julien 4e0683c3f5 multi-detect: cleanup, reuse tenant loading code
Reuse tenant loading from YAML code for Unix Socket.
10 years ago
Victor Julien 646eb4c2a8 multi-detect: load tenants from yaml file
Load tenants and mappings from the suricata.yaml when available.
10 years ago
Victor Julien a20e43f97c suricatasc: add unregister-tenant-handler 10 years ago
Victor Julien a21e88d955 suricatasc: add register-tenant-handler command
Arguments:
- tenant id (int)
- name of handler (string)
- traffic id related to handler (int, optional)

Examples:
- register-tenant-handler 1 vlan 1111
- register-tenant-handler 8 pcap
10 years ago
Victor Julien 216638c342 multi-detect: implement unregister-tenant-handler
Remove a tenant handler from the list and apply it.
10 years ago
Victor Julien 93f856a1b3 detect: don't error out on no de_ctx
This can happen on a multi-detect setup with no registered
engines yet.
10 years ago
Victor Julien b6f290fac7 multi-detect: set selector from yaml
Yaml setting is: multi-detect.selector

Implement 'vlan' and 'direct'.
10 years ago
Victor Julien c72b7f83b8 multi-detect: error on start if no selector registered
Force user to select the method at startup.
10 years ago
Victor Julien 1127ad66b4 multi-detect: register counters on 'master' det_ctx
Otherwise counters are only registered after the stats api is
already fixed.
10 years ago
Victor Julien 7c581c0ffc multi-detect: allow start up with 0 tenants 10 years ago
Victor Julien 6d92e8d220 unix-socket: implement register-tenant-handler
Register tenant handlers/selectors based on what the unix command
"register-tenant-handler" tells.

Check traffic id before adding it. No duplicated registrations for
a traffic id are allowed.
10 years ago
Victor Julien 1893c5edb1 multi-detect: initial selectors for tenants
The Detection Thread has the TenantGetId pointer which allows it
to select a tenant id based on the packet.
10 years ago
Victor Julien 0ff6d3dcfd detect: select detect engine at Detect entry
Limited to Pcap only currently.
10 years ago
Victor Julien bc7e9be5c6 unix-socket: allow tenant id with pcap-file
Register the tenant id that the pcap-file optionally got.
10 years ago
Victor Julien bea66156b9 pcap-file: set tenant-id if available
Set the id to each packet's 'pcap_v' structure.
10 years ago
Victor Julien 127d8183d7 suricatasc: allow for tenant id in pcap-file
Allow for an optional 'tenant id' argument to pcap-file. This will
allow us to force the pcap to be inspected by this tenant.

If ommited it's 0, which means it's disabled.
10 years ago
Victor Julien 98d265f40b detect: use multi tenant thread init if MT enabled 10 years ago
Victor Julien b653479815 detect: make multi tenancy a global switch
At start up we will set this flag based on "multi-detect.enabled".
10 years ago
Victor Julien 3f12bdd4f8 tenants: apply added/removed tenant
Apply to the engine.
10 years ago
Victor Julien def2b58725 detect: initial MT lookup logic
In the DetectEngineThreadCtx, store another DetectEngineThreadCtx per
tenant.

Currently it's just a simple array indexed by the tenant id.
10 years ago
Victor Julien 147a6d2bfd multi-detect: (un)register-tenant unix socket commands
Make available to live mode and unix socket mode.

register-tenant:
    Loads a new YAML, does basic validation.
    Loads a new detection engine
    Loads rules
    Add new de_ctx to master store and stores tenant id in the de_ctx so
        we can look it up by tenant id later.

unregister-tenant:
    Gets the de_ctx, moves it to the freelist
    Removes config

Introduce DetectEngineGetByTenantId, which gets a reference to the
detect engine by tenant id.
10 years ago
Victor Julien 3aa58f25ad eve alert: fix stream payload printing 10 years ago
Victor Julien 2ef0ebb24b detect: fix pass transaction handling
If a flow was 'pass'd, it means that no packet of it will flow be handled
by the detection engine. A side effect of this was that the per flow
inspect_id would never be moved forward. This in turn lead to a situation
where transactions wouldn't be freed.

This patch addresses this case by incrementing the inspect_id anyway for
the pass case.
10 years ago
Victor Julien 06ee2bc87e detect: set flow noinspect on pass in applayer/stream
If a pass rule matches in the reassembled stream and/or in the
app-layer state, it means the rest of the flow should not be
inspected.
10 years ago
Victor Julien bbc9874b81 http: destroy htp_tx_t even if incomplete 10 years ago
Victor Julien 814f0b4094 detect: make http prefilter use disrupt flags 10 years ago
Victor Julien b592f98727 detect: optimize http prefilter handing 10 years ago
Victor Julien 6946e0be55 detect: pass flags to inspect_id update logic 10 years ago
Victor Julien c087708fa9 output-tx: use disrupt flags 10 years ago
Victor Julien 00ef789ffc app-layer: pass full flags around in tx handling 10 years ago
Victor Julien ea571add73 app-layer: disruption flags
Stream GAPs and stream reassembly depth are tracked per direction. In
many cases they will happen in one direction, but not in the other.

Example:
HTTP requests a generally smaller than responses. So on the response
side we may hit the depth limit, but not on the request side.

The asynchronious 'disruption' has a side effect in the transaction
engine. The 'progress' tracking would never mark such transactions
as complete, and thus some inspection and logging wouldn't happen
until the very last moment: when EOF's are passed around.

Especially in proxy environments with _very_ many transactions in a
single TCP connection, this could lead to serious resource issues. The
EOF handling would suddenly have to handle thousands or more
transactions. These transactions would have been stored for a long time.

This patch introduces the concept of disruption flags. Flags passed to
the tx progress logic that are and indication of disruptions in the
traffic or the traffic handling. The idea is that the progress is
marked as complete on disruption, even if a tx is not complete. This
allows the detection and logging engines to process the tx after which
it can be cleaned up.
10 years ago
Victor Julien 8125e04b39 detect: clean up flag usage 10 years ago
Victor Julien 6e2c90a83b detect: constify some DetectMpmPrefilter args 10 years ago
Victor Julien c419f33f44 dns: fix state progress handling 10 years ago
Victor Julien b0f5f7ee97 app-layer: fix args to state progress calls 10 years ago
Victor Julien 62fa9f09d4 Sync alversion/appversion types
The app layer state 'version' field is incremented with each update
to the state. It is used by the detection engine to see if the current
version of the state has already been inspected. Since app layer and
detect always run closely together there is no need for a big number
here. The detect code really only checks for equal/not-equal, so wrap
arounds are not an issue.
10 years ago
Victor Julien 11d3f5f67a http: harden tx inspection code 10 years ago
Victor Julien be2849044b flow/stream: xfer noinspect flags to pseudo pkts
Set noinspection flags for payloads and packets on flow and stream
pseudo packets. Without these, the pseudo packets could trigger
inspection even though this was disabled for a flow.
10 years ago
Victor Julien a0899cbe85 detect: optimize Signature layout 10 years ago
Victor Julien baadcab1b8 detect: default to u32 for SigIntId 10 years ago