From fea037fda88d6548136f28a071f9c878a56634bc Mon Sep 17 00:00:00 2001 From: Ralph Broenink Date: Sat, 14 Oct 2017 11:57:00 +0200 Subject: [PATCH] doc: Moved explanation of normalized buffers to rules introduction --- doc/userguide/rules/http-keywords.rst | 11 ++++++++--- doc/userguide/rules/http-uri-normalization.rst | 10 ---------- doc/userguide/rules/index.rst | 1 - doc/userguide/rules/intro.rst | 17 +++++++++++++++++ doc/userguide/rules/normalized-buffers.rst | 18 ------------------ doc/userguide/rules/payload-keywords.rst | 2 +- doc/userguide/rules/pcre.rst | 5 +++-- 7 files changed, 29 insertions(+), 35 deletions(-) delete mode 100644 doc/userguide/rules/http-uri-normalization.rst delete mode 100644 doc/userguide/rules/normalized-buffers.rst diff --git a/doc/userguide/rules/http-keywords.rst b/doc/userguide/rules/http-keywords.rst index 413a7d6df0..784b212e1b 100644 --- a/doc/userguide/rules/http-keywords.rst +++ b/doc/userguide/rules/http-keywords.rst @@ -151,6 +151,7 @@ Example of the purpose of method: .. image:: http-keywords/method1.png +.. _rules-http-uri-normalization: http_uri and http_raw_uri ------------------------- @@ -161,9 +162,13 @@ buffer. The keyword can be used in combination with all previously mentioned content modifiers like ``depth``, ``distance``, ``offset``, ``nocase`` and ``within``. -To learn more about the difference between ``http_uri`` and -``http_raw_uri``, please read the information about -:doc:`http-uri-normalization`. +The uri has two appearances in Suricata: the raw_uri and the +normalized uri. The space for example can be indicated with the +heximal notation %20. To convert this notation in a space, means +normalizing it. It is possible though to match specific on the +characters %20 in a uri. This means matching on the raw_uri. The +raw_uri and the normalized uri are separate buffers. So, the raw_uri +inspects the raw_uri buffer and can not inspect the normalized buffer. Example of the URI in a HTTP request: diff --git a/doc/userguide/rules/http-uri-normalization.rst b/doc/userguide/rules/http-uri-normalization.rst deleted file mode 100644 index 20d9b14bc6..0000000000 --- a/doc/userguide/rules/http-uri-normalization.rst +++ /dev/null @@ -1,10 +0,0 @@ -HTTP-uri normalization -====================== - -The uri has two appearances in Suricata: the raw_uri and the -normalized uri. The space for example can be indicated with the -heximal notation %20. To convert this notation in a space, means -normalizing it. It is possible though to match specific on the -characters %20 in a uri. This means matching on the raw_uri. The -raw_uri and the normalized uri are separate buffers. So, the raw_uri -inspects the raw_uri buffer and can not inspect the normalized buffer. diff --git a/doc/userguide/rules/index.rst b/doc/userguide/rules/index.rst index 9bf9b88fe6..d741c8a831 100644 --- a/doc/userguide/rules/index.rst +++ b/doc/userguide/rules/index.rst @@ -21,5 +21,4 @@ Suricata Rules enip-keyword app-layer rule-lua-scripting - normalized-buffers differences-from-snort diff --git a/doc/userguide/rules/intro.rst b/doc/userguide/rules/intro.rst index 6b59758b73..398632e56f 100644 --- a/doc/userguide/rules/intro.rst +++ b/doc/userguide/rules/intro.rst @@ -249,3 +249,20 @@ Some keywords function act as modifiers. There are two types of modifiers. alert http any any -> any any (http_response_line; content:"403 Forbidden"; sid:1;) In the above example the pattern '403 Forbidden' is inspected against the HTTP response line because it follows the ``http_response_line`` keyword. + +.. _rules-normalized-buffers: + +Normalized Buffers +~~~~~~~~~~~~~~~~~~ +A packet consists of raw data. HTTP and reassembly make a copy of +those kinds of packets data. They erase anomalous content, combine +packets etcetera. What remains is a called the 'normalized buffer': + +.. image:: normalized-buffers/normalization1.png + +Because the data is being normalized, it is not what it used to be; it +is an interpretation. Normalized buffers are: all HTTP-keywords, +reassembled streams, TLS-, SSL-, SSH-, FTP- and dcerpc-buffers. + +Note that there are some exceptions, e.g. the ``http_raw_uri`` keyword. +See :ref:`rules-http-uri-normalization` for more information. diff --git a/doc/userguide/rules/normalized-buffers.rst b/doc/userguide/rules/normalized-buffers.rst deleted file mode 100644 index a3a282a95f..0000000000 --- a/doc/userguide/rules/normalized-buffers.rst +++ /dev/null @@ -1,18 +0,0 @@ -Normalized Buffers -================== - -.. toctree:: - - http-uri-normalization - -A packet consists of raw data. HTTP and reassembly make a copy of -those kinds of packets data. They erase anomalous content, combine -packets etcetera. What remains is a called the 'normalized buffer'. - -Example: - -.. image:: normalized-buffers/normalization1.png - -Because the data is being normalized, it is not what it used to be; it -is an interpretation. Normalized buffers are: all HTTP-keywords, -reassembled streams, TLS-, SSL-, SSH-, FTP- and dcerpc-buffers. diff --git a/doc/userguide/rules/payload-keywords.rst b/doc/userguide/rules/payload-keywords.rst index 4696936339..ab845bf23b 100644 --- a/doc/userguide/rules/payload-keywords.rst +++ b/doc/userguide/rules/payload-keywords.rst @@ -293,7 +293,7 @@ another ('def'), see example: The replace modifier has to contain as many characters as the content it replaces. It can only be used with individual packets. It will not -work for :doc:`normalized-buffers` like HTTP uri or a content match in +work for :ref:`rules-normalized-buffers` like HTTP uri or a content match in the reassembled stream. The checksums will be recalculated by Suricata and changed after the diff --git a/doc/userguide/rules/pcre.rst b/doc/userguide/rules/pcre.rst index 9c0f4035c8..88bfc363f8 100644 --- a/doc/userguide/rules/pcre.rst +++ b/doc/userguide/rules/pcre.rst @@ -67,7 +67,7 @@ Suricata has its own specific pcre modifiers. These are: uri_buffer just like uricontent and content combined with http_uri.U can be combined with /R. Note that R is relative to the previous match so both matches have to be in the HTTP-uri buffer. Read more - about :doc:`http-uri-normalization`. + about :ref:`HTTP URI Normalization `. .. image:: pcre/pcre3.png @@ -80,7 +80,8 @@ Suricata has its own specific pcre modifiers. These are: * ``I``: Makes pcre match on the HTTP-raw-uri. It matches on the same buffer as http_raw_uri. I can be combined with /R. Note that R is relative to the previous match so both matches have to be in the - HTTP-raw-uri buffer. Read more about :doc:`http-uri-normalization`. + HTTP-raw-uri buffer. Read more about + :ref:`HTTP URI Normalization `. * ``P``: Makes pcre match on the HTTP- request-body. So, it matches on the same buffer as http_client_body. P can be combined with /R. Note