aiden
/
suricata
mirror of https://github.com/OISF/suricata
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

doc: update http keywords documentation

Browse Source
pull/4436/head
Jason Williams 5 years ago committed by Victor Julien
parent 4a2918e6b5
commit 55a36c79ff
1 changed files with 145 additions and 140 deletions
Show Stats Download Patch File Download Diff File

285
doc/userguide/rules/http-keywords.rst
Unescape Escape View File

@ -12,40 +12,40 @@ All HTTP keywords are modifiers. Note the difference between content modifiers
and sticky buffers. See :ref:`rules-modifiers` for more information. As a
refresher:
* **'content modifiers'** look back in the rule, e.g.::
* **'sticky buffers'** are placed first and all keywords following it apply to that buffer, for instance::
alert http any any -> any any (content:"index.php"; http_uri; sid:1;)
alert http any any -> any any (http.response_line; content:"403 Forbidden"; sid:1;)
* **'sticky buffers'** are placed first and all keywords following it apply to that buffer, for instance::
* **'content modifiers'** look back in the rule, e.g.::
alert http any any -> any any (http_response_line; content:"403 Forbidden"; sid:1;)
alert http any any -> any any (content:"index.php"; http_uri; sid:1;)
The following **request** keywords are available:
============================== ======================== ==================
Keyword Sticky or Modifier Direction
============================== ======================== ==================
http_uri Modifier Request
http_raw_uri Modifier Request
http_method Modifier Request
http_request_line Sticky Buffer Request
http_client_body Modifier Request
http_header Modifier Both
http_raw_header Modifier Both
http_cookie Modifier Both
http_user_agent Modifier Request
http_host Modifier Request
http_raw_host Modifier Request
http_accept Sticky Buffer Request
http_accept_lang Sticky Buffer Request
http_accept_enc Sticky Buffer Request
http_referer Sticky Buffer Request
http_connection Sticky Buffer Request
http_content_type Sticky Buffer Both
http_content_len Sticky Buffer Both
http_start Sticky Buffer Both
http_protocol Sticky Buffer Both
http_header_names Sticky Buffer Both
http.uri Sticky Buffer Request
http.uri.raw Sticky Buffer Request
http.method Sticky Buffer Request
http.request_line Sticky Buffer Request
http.request_body Sticky Buffer Request
http.header Sticky Buffer Both
http.header.raw Sticky Buffer Both
http.cookie Sticky Buffer Both
http.user_agent Sticky Buffer Request
http.host Sticky Buffer Request
http.host.raw Sticky Buffer Request
http.accept Sticky Buffer Request
http.accept_lang Sticky Buffer Request
http.accept_enc Sticky Buffer Request
http.referer Sticky Buffer Request
http.connection Sticky Buffer Request
http.content_type Sticky Buffer Both
http.content_len Sticky Buffer Both
http.start Sticky Buffer Both
http.protocol Sticky Buffer Both
http.header_names Sticky Buffer Both
============================== ======================== ==================
The following **response** keywords are available:
@ -53,21 +53,21 @@ The following **response** keywords are available:
============================== ======================== ==================
Keyword Sticky or Modifier Direction
============================== ======================== ==================
http_stat_msg Modifier Response
http_stat_code Modifier Response
http_response_line Sticky Buffer Response
http_header Modifier Both
http_raw_header Modifier Both
http_cookie Modifier Both
http_server_body Modifier Response
http.server Modifier Response
http.location Modifier Response
http.stat_msg Sticky Buffer Response
http.stat_code Sticky Buffer Response
http.response_line Sticky Buffer Response
http.header Sticky Buffer Both
http.header.raw Sticky Buffer Both
http.cookie Sticky Buffer Both
http.server_body Sticky Buffer Response
http.server Sticky Buffer Response
http.location Sticky Buffer Response
file_data Sticky Buffer Response
http_content_type Sticky Buffer Both
http_content_len Sticky Buffer Both
http_start Sticky Buffer Both
http_protocol Sticky Buffer Both
http_header_names Sticky Buffer Both
http.content_type Sticky Buffer Both
http.content_len Sticky Buffer Both
http.start Sticky Buffer Both
http.protocol Sticky Buffer Both
http.header_names Sticky Buffer Both
============================== ======================== ==================
HTTP Primer
@ -114,8 +114,8 @@ Request:
.. image:: http-keywords/request2.png
Although cookies are sent in an HTTP header, you can not match on them
with the ``http_header`` keyword. Cookies are matched with their own
keyword, namely ``http_cookie``.
with the ``http.header`` keyword. Cookies are matched with their own
keyword, namely ``http.cookie``.
Each part of the table belongs to a so-called *buffer*. The HTTP
method belongs to the method buffer, HTTP headers to the header buffer
@ -128,10 +128,10 @@ relative modifiers, so they may only be used within the same
buffer. You can not relate content matches against different buffers
with relative modifiers.
http_method
http.method
-----------
With the ``http_method`` content modifier, it is possible to match
With the ``http.method`` content modifier, it is possible to match
specifically and only on the HTTP method buffer. The keyword can be
used in combination with all previously mentioned content modifiers
such as: ``depth``, ``distance``, ``offset``, ``nocase`` and ``within``.
@ -153,28 +153,28 @@ Example of the purpose of method:
.. _rules-http-uri-normalization:
http_uri and http_raw_uri
http.uri and http.uri.raw
-------------------------
With the ``http_uri`` and the ``http_raw_uri`` content modifiers, it
With the ``http.uri`` and the ``http.uri.raw`` content modifiers, it
is possible to match specifically and only on the request URI
buffer. The keyword can be used in combination with all previously
mentioned content modifiers like ``depth``, ``distance``, ``offset``,
``nocase`` and ``within``.
The uri has two appearances in Suricata: the raw_uri and the
The uri has two appearances in Suricata: the uri.raw and the
normalized uri. The space for example can be indicated with the
heximal notation %20. To convert this notation in a space, means
normalizing it. It is possible though to match specific on the
characters %20 in a uri. This means matching on the raw_uri. The
raw_uri and the normalized uri are separate buffers. So, the raw_uri
inspects the raw_uri buffer and can not inspect the normalized buffer.
characters %20 in a uri. This means matching on the uri.raw. The
uri.raw and the normalized uri are separate buffers. So, the uri.raw
inspects the uri.raw buffer and can not inspect the normalized buffer.
Example of the URI in a HTTP request:
.. image:: http-keywords/uri1.png
Example of the purpose of ``http_uri``:
Example of the purpose of ``http.uri``:
.. image:: http-keywords/uri.png
@ -182,7 +182,7 @@ uricontent
----------
The ``uricontent`` keyword has the exact same effect as the
``http_uri`` content modifier. ``uricontent`` is a deprecated
``http.uri`` content modifier. ``uricontent`` is a deprecated
(although still supported) way to match specifically and only on the
request URI buffer.
@ -192,14 +192,14 @@ Example of ``uricontent``:
alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"ET TROJAN Possible Vundo Trojan Variant reporting to Controller"; flow:established,to_server; content:"POST "; depth:5; :example-rule-emphasis:`uricontent:"/frame.html?";` urilen: > 80; classtype:trojan-activity; reference:url,doc.emergingthreats.net/2009173; reference:url,www.emergingthreats.net/cgi-bin/cvsweb.cgi/sigs/VIRUS/TROJAN_Vundo; sid:2009173; rev:2;)
The difference between ``http_uri`` and ``uricontent`` is the syntax:
The difference between ``http.uri`` and ``uricontent`` is the syntax:
.. image:: http-keywords/uricontent1.png
.. image:: http-keywords/http_uri.png
When authoring new rules, it is recommended that the ``http_uri``
content modifier be used rather than the deprecated ``uricontent``
When authoring new rules, it is recommended that the ``http.uri``
content sticky buffer be used rather than the deprecated ``uricontent``
keyword.
urilen
@ -233,16 +233,16 @@ Example of ``urilen`` in a signature:
You can also append ``norm`` or ``raw`` to define what sort of buffer you want
to use (normalized or raw buffer).
http_protocol
http.protocol
-------------
The ``http_protocol`` inspects the protocol field from the HTTP request or
The ``http.protocol`` inspects the protocol field from the HTTP request or
response line. If the request line is 'GET / HTTP/1.0\r\n', then this buffer
will contain 'HTTP/1.0'.
Example::
alert http any any -> any any (flow:to_server; http_protocol; content:"HTTP/1.0"; sid:1;)
alert http any any -> any any (flow:to_server; http.protocol; content:"HTTP/1.0"; sid:1;)
``http.protocol`` replaces the previous keyword name: ```http_protocol``. You may continue
+to use the previous name, but it's recommended that rules be converted to use
@ -253,23 +253,23 @@ Example::
alert http any any -> any any (flow:to_server; http.protocol; content:"HTTP/1.0"; sid:1;)
http_request_line
http.request_line
-----------------
The ``http_request_line`` forces the whole HTTP request line to be inspected.
The ``http.request_line`` forces the whole HTTP request line to be inspected.
Example::
alert http any any -> any any (http_request_line; content:"GET / HTTP/1.0"; sid:1;)
alert http any any -> any any (http.request_line; content:"GET / HTTP/1.0"; sid:1;)
http_header and http_raw_header
http.header and http.header.raw
-------------------------------
With the ``http_header`` content modifier, it is possible to match
With the ``http.header`` content modifier, it is possible to match
specifically and only on the HTTP header buffer. This contains all of
the extracted headers in a single buffer, except for those indicated
in the documentation that are not able to match by this buffer and
have their own content modifier (e.g. ``http_cookie``). The modifier
have their own content modifier (e.g. ``http.cookie``). The modifier
can be used in combination with all previously mentioned content
modifiers, like ``depth``, ``distance``, ``offset``, ``nocase`` and
``within``.
@ -277,20 +277,20 @@ modifiers, like ``depth``, ``distance``, ``offset``, ``nocase`` and
**Note**: the header buffer is *normalized*. Any trailing
whitespace and tab characters are removed. See:
https://lists.openinfosecfoundation.org/pipermail/oisf-users/2011-October/000935.html.
To avoid that, use the ``http_raw_header`` keyword.
To avoid that, use the ``http.header.raw`` keyword.
Example of a header in a HTTP request:
.. image:: http-keywords/header.png
Example of the purpose of ``http_header``:
Example of the purpose of ``http.header``:
.. image:: http-keywords/header1.png
http_cookie
http.cookie
-----------
With the ``http_cookie`` content modifier, it is possible to match
With the ``http.cookie`` content modifier, it is possible to match
specifically and only on the cookie buffer. The keyword can be used in
combination with all previously mentioned content modifiers like
``depth``, ``distance``, ``offset``, ``nocase`` and ``within``.
@ -303,14 +303,14 @@ Example of a cookie in a HTTP request:
.. image:: http-keywords/cookie.png
Example of the purpose of ``http_cookie``:
Example of the purpose of ``http.cookie``:
.. image:: http-keywords/cookie1.png
http_user_agent
http.user_agent
---------------
The ``http_user_agent`` content modifier is part of the HTTP request
The ``http.user_agent`` content modifier is part of the HTTP request
header. It makes it possible to match specifically on the value of the
User-Agent header. It is normalized in the sense that it does not
include the _"User-Agent: "_ header name and separator, nor does it
@ -321,30 +321,30 @@ modifiers like ``depth``, ``distance``, ``offset``, ``nocase`` and
buffer when using the ``/V`` modifier.
Normalization: leading spaces **are not** part of this buffer. So
"User-Agent: \r\n" will result in an empty ``http_user_agent`` buffer.
"User-Agent: \r\n" will result in an empty ``http.user_agent`` buffer.
Example of the User-Agent header in a HTTP request:
.. image:: http-keywords/user_agent.png
Example of the purpose of ``http_user_agent``:
Example of the purpose of ``http.user_agent``:
.. image:: http-keywords/user_agent_match.png
Notes
~~~~~
- The ``http_user_agent`` buffer will NOT include the header name,
- The ``http.user_agent`` buffer will NOT include the header name,
colon, or leading whitespace. i.e. it will not include
"User-Agent: ".
- The ``http_user_agent`` buffer does not include a CRLF (0x0D
- The ``http.user_agent`` buffer does not include a CRLF (0x0D
0x0A) at the end. If you want to match the end of the buffer, use a
relative ``isdataat`` or a PCRE (although PCRE will be worse on
performance).
- If a request contains multiple "User-Agent" headers, the values will
be concatenated in the ``http_user_agent`` buffer, in the order
be concatenated in the ``http.user_agent`` buffer, in the order
seen from top to bottom, with a comma and space (", ") between each
of them.
@ -354,19 +354,19 @@ Notes
User-Agent: SuriTester/0.8
User-Agent: GGGG
``http_user_agent`` buffer contents::
``http.user_agent`` buffer contents::
SuriTester/0.8, GGGG
- Corresponding PCRE modifier: ``V``
- Using the ``http_user_agent`` buffer is more efficient when it
comes to performance than using the ``http_header`` buffer (~10%
- Using the ``http.user_agent`` buffer is more efficient when it
comes to performance than using the ``http.header`` buffer (~10%
better).
- `https://blog.inliniac.net/2012/07/09/suricata-http\_user\_agent-vs-http\_header/ <https://blog.inliniac.net/2012/07/09/suricata-http_user_agent-vs-http_header/>`_
http_accept
http.accept
-----------
Sticky buffer to match on the HTTP Accept header. Only contains the header
@ -374,9 +374,9 @@ value. The \\r\\n after the header are not part of the buffer.
Example::
alert http any any -> any any (http_accept; content:"image/gif"; sid:1;)
alert http any any -> any any (http.accept; content:"image/gif"; sid:1;)
http_accept_enc
http.accept_enc
---------------
Sticky buffer to match on the HTTP Accept-Encoding header. Only contains the
@ -384,10 +384,10 @@ header value. The \\r\\n after the header are not part of the buffer.
Example::
alert http any any -> any any (http_accept_enc; content:"gzip"; sid:1;)
alert http any any -> any any (http.accept_enc; content:"gzip"; sid:1;)
http_accept_lang
http.accept_lang
----------------
Sticky buffer to match on the HTTP Accept-Language header. Only contains the
@ -395,10 +395,10 @@ header value. The \\r\\n after the header are not part of the buffer.
Example::
alert http any any -> any any (http_accept_lang; content:"en-us"; sid:1;)
alert http any any -> any any (http.accept_lang; content:"en-us"; sid:1;)
http_connection
http.connection
---------------
Sticky buffer to match on the HTTP Connection header. Only contains the
@ -406,10 +406,10 @@ header value. The \\r\\n after the header are not part of the buffer.
Example::
alert http any any -> any any (http_connection; content:"keep-alive"; sid:1;)
alert http any any -> any any (http.connection; content:"keep-alive"; sid:1;)
http_content_type
http.content_type
-----------------
Sticky buffer to match on the HTTP Content-Type headers. Only contains the
@ -420,13 +420,13 @@ Use flow:to_server or flow:to_client to force inspection of request or response.
Examples::
alert http any any -> any any (flow:to_server; \
http_content_type; content:"x-www-form-urlencoded"; sid:1;)
http.content_type; content:"x-www-form-urlencoded"; sid:1;)
alert http any any -> any any (flow:to_client; \
http_content_type; content:"text/javascript"; sid:2;)
http.content_type; content:"text/javascript"; sid:2;)
http_content_len
http.content_len
----------------
Sticky buffer to match on the HTTP Content-Length headers. Only contains the
@ -437,19 +437,19 @@ Use flow:to_server or flow:to_client to force inspection of request or response.
Examples::
alert http any any -> any any (flow:to_server; \
http_content_len; content:"666"; sid:1;)
http.content_len; content:"666"; sid:1;)
alert http any any -> any any (flow:to_client; \
http_content_len; content:"555"; sid:2;)
http.content_len; content:"555"; sid:2;)
To do a numeric inspection of the content length, ``byte_test`` can be used.
Example, match if C-L is equal to or bigger than 8079::
alert http any any -> any any (flow:to_client; \
http_content_len; byte_test:0,>=,8079,0,string,dec; sid:3;)
http.content_len; byte_test:0,>=,8079,0,string,dec; sid:3;)
http_referer
http.referer
---------------
Sticky buffer to match on the HTTP Referer header. Only contains the
@ -457,9 +457,9 @@ header value. The \\r\\n after the header are not part of the buffer.
Example::
alert http any any -> any any (http_referer; content:".php"; sid:1;)
alert http any any -> any any (http.referer; content:".php"; sid:1;)
http_start
http.start
----------
Inspect the start of a HTTP request or response. This will contain the
@ -468,12 +468,12 @@ or flow:to_client to force inspection of request or response.
Example::
alert http any any -> any any (http_start; content:"HTTP/1.1|0d 0a|User-Agent"; sid:1;)
alert http any any -> any any (http.start; content:"HTTP/1.1|0d 0a|User-Agent"; sid:1;)
The buffer contains the normalized headers and is terminated by an extra
\\r\\n to indicate the end of the headers.
http_header_names
http.header_names
-----------------
Inspect a buffer only containing the names of the HTTP headers. Useful
@ -488,37 +488,37 @@ Example buffer::
Example rule::
alert http any any -> any any (http_header_names; content:"|0d 0a|Host|0d 0a|"; sid:1;)
alert http any any -> any any (http.header_names; content:"|0d 0a|Host|0d 0a|"; sid:1;)
Example to make sure *only* Host is present::
alert http any any -> any any (http_header_names; \
alert http any any -> any any (http.header_names; \
content:"|0d 0a|Host|0d 0a 0d 0a|"; sid:1;)
Example to make sure *User-Agent* is directly after *Host*::
alert http any any -> any any (http_header_names; \
alert http any any -> any any (http.header_names; \
content:"|0d 0a|Host|0d 0a|User-Agent|0d 0a|"; sid:1;)
Example to make sure *User-Agent* is after *Host*, but not necessarily directly after::
alert http any any -> any any (http_header_names; \
alert http any any -> any any (http.header_names; \
content:"|0d 0a|Host|0d 0a|"; content:"|0a 0d|User-Agent|0d 0a|"; \
distance:-2; sid:1;)
http_client_body
----------------
http.request_body
-----------------
With the ``http_client_body`` content modifier, it is possible to
With the ``http.request_body`` content modifier, it is possible to
match specifically and only on the HTTP request body. The keyword can
be used in combination with all previously mentioned content modifiers
like ``distance``, ``offset``, ``nocase``, ``within``, etc.
Example of ``http_client_body`` in a HTTP request:
Example of ``http.request_body`` in a HTTP request:
.. image:: http-keywords/client_body.png
Example of the purpose of ``http_client_body``:
Example of the purpose of ``http.client_body``:
.. image:: http-keywords/client_body1.png
@ -527,52 +527,56 @@ in the :ref:`libhtp configuration section
<suricata-yaml-configure-libhtp>` via the ``request-body-limit``
setting.
http_stat_code
``http.request_body`` replaces the previous keyword name: ```http_client_body``. You may continue
+to use the previous name, but it's recommended that rules be converted to use
+the new name.
http.stat_code
--------------
With the ``http_stat_code`` content modifier, it is possible to match
With the ``http.stat_code`` content modifier, it is possible to match
specifically and only on the HTTP status code buffer. The keyword can
be used in combination with all previously mentioned content modifiers
like ``distance``, ``offset``, ``nocase``, ``within``, etc.
Example of ``http_stat_code`` in a HTTP response:
Example of ``http.stat_code`` in a HTTP response:
.. image:: http-keywords/stat_code.png
Example of the purpose of ``http_stat_code``:
Example of the purpose of ``http.stat_code``:
.. image:: http-keywords/stat-code1.png
http_stat_msg
http.stat_msg
-------------
With the ``http_stat_msg`` content modifier, it is possible to match
With the ``http.stat_msg`` content modifier, it is possible to match
specifically and only on the HTTP status message buffer. The keyword
can be used in combination with all previously mentioned content
modifiers like ``depth``, ``distance``, ``offset``, ``nocase`` and
``within``.
Example of ``http_stat_msg`` in a HTTP response:
Example of ``http.stat_msg`` in a HTTP response:
.. image:: http-keywords/stat_msg.png
Example of the purpose of ``http_stat_msg``:
Example of the purpose of ``http.stat_msg``:
.. image:: http-keywords/stat_msg_1.png
http_response_line
http.response_line
------------------
The ``http_response_line`` forces the whole HTTP response line to be inspected.
The ``http.response_line`` forces the whole HTTP response line to be inspected.
Example::
alert http any any -> any any (http_response_line; content:"HTTP/1.0 200 OK"; sid:1;)
alert http any any -> any any (http.response_line; content:"HTTP/1.0 200 OK"; sid:1;)
http_server_body
----------------
http.response_body
------------------
With the ``http_server_body`` content modifier, it is possible to
With the ``http.response_body`` content modifier, it is possible to
match specifically and only on the HTTP response body. The keyword can
be used in combination with all previously mentioned content modifiers
like ``distance``, ``offset``, ``nocase``, ``within``, etc.
@ -585,15 +589,15 @@ setting.
Notes
~~~~~
- Using ``http_server_body`` is similar to having content matches
- Using ``http.response_body`` is similar to having content matches
that come after ``file_data`` except that it doesn't permanently
(unless reset) set the detection pointer to the beginning of the
server response body. i.e. it is not a sticky buffer.
- ``http_server_body`` will match on gzip decoded data just like
- ``http.response_body`` will match on gzip decoded data just like
``file_data`` does.
- Since ``http_server_body`` matches on a server response, it
- Since ``http.response_body`` matches on a server response, it
can't be used with the ``to_server`` or ``from_client`` flow
directives.
@ -601,6 +605,10 @@ Notes
- further notes at the ``file_data`` section below.
``http.response_body`` replaces the previous keyword name: ```http_server_body``. You may continue
+to use the previous name, but it's recommended that rules be converted to use
+the new name.
http.server
-----------
@ -624,12 +632,12 @@ Example::
http.location; content:"http://www.google.com"; sid:1;)
http_host and http_raw_host
http.host and http.host.raw
---------------------------
With the ``http_host`` content modifier, it is possible to
With the ``http.host`` content modifier, it is possible to
match specifically and only the normalized hostname.
The ``http_raw_host`` inspects the raw hostname.
The ``http.host.raw`` inspects the raw hostname.
The keyword can be used in combination with most of the content modifiers
like ``distance``, ``offset``, ``within``, etc.
@ -640,30 +648,30 @@ to specify a lowercase pattern.
Notes
~~~~~
- The ``http_host`` and ``http_raw_host`` buffers are populated
- The ``http.host`` and ``http.host.raw`` buffers are populated
from either the URI (if the full URI is present in the request like
in a proxy request) or the HTTP Host header. If both are present, the
URI is used.
- The ``http_host`` and ``http_raw_host`` buffers will NOT
- The ``http.host`` and ``http.host.raw`` buffers will NOT
include the header name, colon, or leading whitespace if populated
from the Host header. i.e. they will not include "Host: ".
- The ``http_host`` and ``http_raw_host`` buffers do not
- The ``http.host`` and ``http.host.raw`` buffers do not
include a CRLF (0x0D 0x0A) at the end. If you want to match the end
of the buffer, use a relative 'isdataat' or a PCRE (although PCRE
will be worse on performance).
- The ``http_host`` buffer is normalized to be all lower case.
- The ``http.host`` buffer is normalized to be all lower case.
- The content match that ``http_host`` applies to must be all lower
- The content match that ``http.host`` applies to must be all lower
case or have the ``nocase`` flag set.
- ``http_raw_host`` matches the unnormalized buffer so matching
- ``http.host.raw`` matches the unnormalized buffer so matching
will be case-sensitive (unless ``nocase`` is set).
- If a request contains multiple "Host" headers, the values will be
concatenated in the ``http_host`` and ``http_raw_host``
concatenated in the ``http.host`` and ``http.host.raw``
buffers, in the order seen from top to bottom, with a comma and space
(", ") between each of them.
@ -674,11 +682,11 @@ Notes
Accept: */*
Host: efg.net
``http_host`` buffer contents::
``http.host`` buffer contents::
abc.com, efg.net
``http_raw_host`` buffer contents::
``http.host.raw`` buffer contents::
ABC.com, efg.net
@ -689,10 +697,7 @@ file_data
---------
With ``file_data``, the HTTP response body is inspected, just like
with ``http_server_body``. The ``file_data`` keyword works a bit
differently from the normal content modifiers; when used in a rule,
all content matches following it in the rule are affected (modified)
by it.
with ``http.response_body``. The ``file_data`` keyword is a sticky buffer.
Example::

Write Preview
Loading…
Cancel
Save