Add a command to extract the accounting data from a live
flow using the unix socket. It takes the flow_id as param
and return the volume of data seen on the flow as well as
its age.
Instead of using distutils/setuptools for installing the Python code,
just install it into our own Python directory.
Distutils is being removed from Python, and setuptools doesn't work well
when trying to install into your own location. For our usage its just
simpler to install with make.
In addition to removing the configure check for distutils, also remove
the check for pyyaml. This lets the user install pyyaml after Suricata
is installed, and Suricata-Update does handle this case gracefully.
Issue: #5313
Many places were still referencing the old Suricata page.
Used git grep with replace to update them. Checked that new links work.
Left old references when they were only documentation examples (for
output or unittests).
Task#4915
Up until now, suricatactl would delete any directory that is provided as
an argument on command line. This patch adds a basic test for the
directories `tmp`, `00` and `ff` in order to justify that the provided
directory is actually a filestore directory.
Additionally, some code has been broken up and made more readable and
pythonic.
Closes redmine ticket #2843
Require distutils to install the Python tools. Update the logic
to only install suricatactl (and suricatasc) if Python and
distutils are found. Suricata-Update will only be installed if
bundled, and python-distutils and python-yaml are found.
So far the suricatactl parser was unclear about the options to use and
did not well display the required and optional param difference. Fix
that to make it legible for any user.
Before
```
└─ $ ▶ ./bin/suricatactl filestore -h
usage: suricatactl filestore [-h] {prune} ...
positional arguments:
{prune}
optional arguments:
-h, --help show this help message and exit
└─ $ ▶ ./bin/suricatactl filestore prune -h
usage: suricatactl filestore prune [-h] [-d DIRECTORY] [--age AGE] [-n] [-v]
[-q]
optional arguments:
-h, --help show this help message and exit
-d DIRECTORY, --directory DIRECTORY
filestore directory
--age AGE prune files older than age
-n, --dry-run only print what would happen
-v, --verbose increase verbosity
-q, --quiet be quiet, log warnings and errors only
```
After
```
└─ $ ▶ ./bin/suricatactl filestore -h
usage: suricatactl filestore [-h] {prune} ...
positional arguments:
{prune} sub-command help
prune Remove files in specified directory older than specified age
optional arguments:
-h, --help show this help message and exit
└─ $ ▶ ./bin/suricatactl filestore prune -h
usage: suricatactl filestore prune [-h] -d DIRECTORY [--age AGE] [-n] [-v]
[-q]
optional arguments:
-h, --help show this help message and exit
-n, --dry-run only print what would happen
-v, --verbose increase verbosity
-q, --quiet be quiet, log warnings and errors only
required arguments:
-d DIRECTORY, --directory DIRECTORY
filestore directory
--age AGE prune files older than age, units: s, m, h, d
```
Pylint is a tool to make sure we do not regress the support for Python
3. The following conventions, warnings, errors, refactors have been
fixed.
W0301: Unnecessary semicolon (unnecessary-semicolon)
C0303: Trailing whitespace (trailing-whitespace)
W1401: Anomalous backslash in string
C0103: Variable name doesn't conform to snake_case naming style
R1705: Unnecessary "elif" after "return"
W1201: Specify string format arguments as logging function parameters
W0611: Unused import
R1710: Either all return statements in a function should return an expression, or none of them should
W0612: Unused variable
C0103: Method name doesn't conform to snake_case naming style
R0201: Method could be a function
Call to suricatactl was failing with Python3 with the following error:
```
Traceback (most recent call last):
File "bin/suricatactl", line 40, in <module>
sys.exit(main())
File "./suricata/ctl/main.py", line 50, in main
args.func(args)
AttributeError: 'Namespace' object has no attribute 'func'
```
Fix this by making it run with Py3 just like it does with Py2.
Closes redmine ticket #2793
This commit addresses the following three cases:
1. Do not use maxsplit keyword arg
maxsplit argument to the split command was not a part of Python 2
and using it with Python 2 causes the following failure:
```
TypeError: split() takes no keyword arguments
```
Avoid this by eliminating all the named arguments from split.
2. Fix failure on extra arguments
Up until now, suricatasc fails if any command which is not supposed to
take args is given args.
Fix this by ignoring any extra params.
Closes redmine ticket #2813
3. Fix failure on different type of args
If a command was given a string argument where it expected an int, it
would fail and the process would exit.
Fix this by handling the exception caused in such cases.
Closes redmine ticket #2812
Up until now, suricatasc gives a message as follows in case a command is
missing arguments:
```
>>> list-hostbit
Arguments to command 'list-hostbit' is missing
```
Fix this up and provide a better message:
```
>>> list-hostbit
Missing arguments: expected 1
>>> pcap-file-continuous
Missing arguments: expected at least 2
```
Since all of the commands were following the same procedure, namely,
split the input extract the arguments, throw the error if required
argument is missing else send the command over to suricata, put all of
this in one compact function alongwith a dictionary for specifications
for different commands, the name of the argument, the type and if it is
required or not.
Following fixups come with this commit:
- Code becomes really cozy
- Split errors on a few commands are well handled
- No redundant code
- More readability
References redmine ticket #2793
Pylint is a tool to make sure we do not regress the support for Python
3. The following conventions, warnings, errors, refactors have been
fixed.
C0326: Exactly one space required around assignment
C0326: No space allowed around keyword argument assignment
C0325: Unnecessary parens after 'if' keyword
W0301: Unnecessary semicolon
W0702: No exception type(s) specified
W0231: __init__ method from base class 'Exception' is not called
W0107: Unnecessary pass statement
C0121: Comparison to None should be 'expr is not None'
E0602: Undefined variable 'raw_input'
W0201: Attribute 'socket' defined outside __init__
W0611: Unused import
https://redmine.openinfosecfoundation.org/issues/2417
Add option to have pcap files deleted after they have been processed.
This option combines well with pcap file continuous and streaming
files to a directory being processed.
Autoconf/automake and python setup.py don't play that well
together with out of tree builds.
Makes suricatasc not an autoconf input file, instead use the
defaults module that is already being created.
In the case of an out of tree build, copy the generated defaults.py
to the build directory manually.
When we install to a non-standard prefix, the Python modules
are not in the standard location requiring the PYTHONPATH
to be fixed up.
This wa a pre-existing issue with suricatasc, and not due to
the move into the python directory.
Will be built and installed as part of the Python code used
for suricatactl, which is intended to be the generic place
for all Python utility code that gets installed with Suricata.
No change to suricatasc code.
Use a new directory, Python to host the Suricata python modules.
One entry point is suricatactl, a control script for
miscalleneous tasks. Currently onl filestore pruning
is implemented.
Jason Ish
Victor Julien
Shivani Bhardwaj
Eric Leblond
Eric Leblond
Jeff Lucovsky
Juliana Fajardini
jason taylor
Victor Julien
Philippe Antoine
Shivani Bhardwaj
Jason Ish
Danny Browning