Commit Graph

17933 Commits (fbf75f2e7dd1ab5a7da2ac9a071ee4db80df703c)
 

Author SHA1 Message Date
Eric Leblond ab5070c0e3 detect/engine: put datajson related code in a func 5 months ago
Eric Leblond e600f40f74 datasets: use SCLogError for errors 5 months ago
Eric Leblond 337472449a detect/engine: remove unused keyword 5 months ago
Eric Leblond cd07db83dc datasets: comment to specify usage of field 5 months ago
Eric Leblond be9e13c6b1 decode: use pointer for PacketContextData
By adding a pointer instead of the structure in the alert info
we spare some memory.
5 months ago
Eric Leblond 92f4b36e72 decode: rename ExtraDataJsonList 5 months ago
Eric Leblond b285894f69 datasets: comments about format 5 months ago
Eric Leblond 449d6fe607 datasets: rename function
DatasetCreateOrGet was in fact doing get or create.
5 months ago
Eric Leblond e0c5302df1 datasets: fix error handling in StringSet 5 months ago
Eric Leblond 9b6047ab3e datasets: factorize set operation 5 months ago
Eric Leblond 3880b8de22 datasjson: fix include barrier name 5 months ago
Eric Leblond be0f5dac0d detect/pcre: fix copyright date 5 months ago
Eric Leblond 43743b0fd3 datajson: key length fix
When using type string, we should not limit the size of the string.

This patch also fixes some implicit cast issues.
5 months ago
Eric Leblond 43a1ef45ca datajson: value_key is necessary in json format 5 months ago
Eric Leblond 6236574b9c doc/userguide: enrichment_key is now context_key 5 months ago
Eric Leblond 62e90206c6 datajson: rename enrichment_key to context_key 5 months ago
Eric Leblond 3e48342248 datajson: factorize code
Code was duplicated and a build error was reported so factorizing
was the best way to clean it.
5 months ago
Eric Leblond df99d29ee4 datajson: add sanity check on length
Also cast to avoid compilation error.
5 months ago
Eric Leblond cfa7a747c2 datajson: use systematic allocation 5 months ago
Eric Leblond d84d7431e4 datajson: more explicit memory handling 5 months ago
Eric Leblond 20a0575d96 doc/userguide: fix some typos
Suggestions from Juliana.

Co-authored-by: Juliana Fajardini Reichow <jufajardini@gmail.com>
5 months ago
Eric Leblond 23f643a4a7 eve/schema: fix ordering 5 months ago
Eric Leblond fed7ff1e76 datasets: remove comment about unused return value 5 months ago
Eric Leblond 79c8f431d0 datajson: simplify memory handling
DatajsonAdd function is now responsible of the handling of the mem
of datajsontype passed as argument.
5 months ago
Eric Leblond 606f7ba656 datajson: use wrapper 5 months ago
Eric Leblond 7e7bd92432 datajson: fix potential leak in error handling 5 months ago
Eric Leblond 4a420085b7 datajson: remove unnecessary abstraction 5 months ago
Eric Leblond 4cadeb17e1 github: upload scan-build artifacts 5 months ago
Eric Leblond d57e8731bd datajson: use more broadly supported formatter 5 months ago
Eric Leblond 7dc683e5d9 datajson: fix formatting 5 months ago
Eric Leblond c15bcbe9d6 datasets: factorize dataset creation
Factorize DatasetGet and DatajsonGet to only have the difference
between the two in the respective function.
5 months ago
Eric Leblond 28742871d2 datasets: separate DatasetGet in 2 functions
This will be used to factorize the code with datajson.
5 months ago
Eric Leblond b766bd9438 datasets/context: rename the datajson files
They are is renamed to datasets-context-json.* so we see that
it is about context and not about a new datasets type.
5 months ago
Eric Leblond 40c545f8d9 doc/userguide: jsonline is now standard ndjson 5 months ago
Eric Leblond f7dc2a71db datajson: rename jsonline to ndjson 5 months ago
Eric Leblond f724c75cc9 doc/userguide: improve datajson doc 5 months ago
Eric Leblond b03d4f8e1a datajson: output context to "context"
Using `alert.extra` was not really reflecting the nature of what
was added. So renaming it to `alert.context`.
5 months ago
Eric Leblond 0bc21eec48 datajson: fix thread safety violation 5 months ago
Eric Leblond 40f86571d9 datajson: reduce size length 5 months ago
Eric Leblond b8e2791482 datajson: fix string format in error message 5 months ago
Eric Leblond 61ac7b46c1 eve/schema: remove reference to datajson 5 months ago
Eric Leblond a652eee508 doc/userguide: remove left over datajson reference 5 months ago
Eric Leblond 7d28758a54 doc/userguide: improve datajson doc
Patch adds ``remove_key`` option and clarifies the text.
5 months ago
Eric Leblond 3dde17fb75 datajson: add remove_key option to dataset
This option allows to remove the key corresponding to the match
value from the JSON object before creating the JSON object that
will be added to the `extra` data.

For example, matching on the following JSON on the `ip` key:

```json
{"ip": "10.16.1.11", "test": "success", "context":3}
```

with a match like:

```
dataset:isset,src_ip,type ip,load src.lst,format jsonline,enrichment_key src_ip,value_key ip;
```

will produce the following:

```json
"extra": {
  "src_ip": {
    "ip": "10.16.1.11",
    "test": "success",
    "context": 3
  }
```

if we add the `remove_key` option to the match:

```
dataset:isset,src_ip,type ip,load src.lst,format jsonline,enrichment_key src_ip,value_key ip, remove_key;
```

it will produce the following:

```json
"extra": {
  "src_ip": {
    "test": "success",
    "context": 3
  }
```

The option is set to false by default.

Ticket: #7372
5 months ago
Eric Leblond 0ae88a408a doc/userguide: basic doc for jsonline format 5 months ago
Eric Leblond 3223d8fe59 datajson: implement jsonline format
This format allows to use a one valid JSON object per line in the
data file.

Ticket: #7372
5 months ago
Eric Leblond 8058964449 datajson: prepare jsonline format
There is just a change in the iterator to go from json to jsonline
so let's factorize the parsing functions.

Ticket: #7372
5 months ago
Eric Leblond 9873c5d2e1 doc/userguide: add dataset with json 5 months ago
Eric Leblond e2d8217934 eve/schema: document datajson output 5 months ago
Eric Leblond 0e88e36020 eve/schema: pktvars is a container
It can contain any vars so need addition properties.
5 months ago