Share memory space for IPV4Vars and (IPV6Vars, IPV6ExtHdrs), since a
packet can only be either IPv4 or IPv6, but not both.
Share memory for TCPVars, UDPVars, ICMPV4Vars and ICMPV6Vars, since a
packet can only be only of these.
Then move other structure members around to remove holes reported by pahole.
This reduces the size of the Packet structure from 2944 bytes (46 cachelines)
down to 1976 (31 cachelines), a 33% reduction.
Strip the 'proxy' parts from the normalized uri as inspected by http_uri,
urilen, pcre /U and others.
In a request line like:
GET http://suricata-ids.org/blah/ HTTP/1.1
the normalized URI will now be:
/blah/
This doesn't affect http_raw_uri. So matching the hostname, etc is still
possible through this keyword.
Additionally, a new per HTTP 'personality' option was added to change
this behavior: "uri-include-all":
uri-include-all: <true|false>
Include all parts of the URI. By default the
'scheme', username/password, hostname and port
are excluded. Setting this option to true adds
all of them to the normalized uri as inspected
by http_uri, urilen, pcre with /U and the other
keywords that inspect the normalized uri.
Note that this does not affect http_raw_uri.
So adding uri-include-all:true to all personalities in the yaml will
restore the old default behavior.
Ticket 1008.
Move the allocation of the mPipe ingress queue from a loop over
the number of workers in the main init function to being done inside
each worker thread. This allows allocating the memory locally on the
worker's CPU without needing to figure out ahead of time where that thread
will be running. This fixes one case of static mapping of workers to CPUs.
Use __thread to hold the queue rather than a global tables of queues.
pcre_study returning NULL is not necessarily an error, from the man page
pcre_study(3):
"If the function returns NULL, either it could not find any additional
information, or there was an error. You can tell the difference by
looking at the error value. It is NULL in first case."
Older libpcre versions would return NULL, causing errors.
Keep a separate checksum for IPV4, since a packet can have both an IPV4
checksum and a TCPV4 checksum, or IPV4 and UDPV4 checksum.
This will allow future sharing of more values.
Use PACKET_RESET_CHECKSUMS() in Unit Tests in place of setting the
individual checksum values.
When http and/or tls logging is disabled, the app layer would still
be flagged as logging. This caused transactions not to be freed until
the end of the flow as the logged tx id would never increment.
This fix postpones the setting of the app layer parser "logger"
flag to the point where we know the logger is enabled.
When logging is disabled, the app layer would still be flagged
as logging. This caused transactions not to be freed until the
end of the flow as the logged tx id would never increment.
This fix postpones the setting of the app layer parser "logger"
flag to the point where we know the logger is enabled.
In the case where DNS requests are sent over the same flow w/o a
reply being received, we now set an event in the flow and refuse
to add more transactions to the state. This protects the DNS
handling from getting overloaded slowing down everything.
A new option to configure this behaviour was added:
app-layer:
protocols:
dnsudp:
enabled: yes
detection-ports:
udp:
toserver: 53
request-flood: 750
The request-flood parameter can be 0 (disabling this feature) or a
positive integer. It defaults to 500.
This means that if 500 unreplied requests are seen in a row an event
is set. Rule 2240007 was added to dns-events.rules to match on this.
Packets that are rejected by the stream engine are not considered
part of an established tcp session. By allowing them to inspect
an smsg, some smsgs would not be properly inspected.
Copied SigTest26TCPV4Keyword and added check for invalid IPV4 checksums.
Created new SigTest26TCPV4AndIPV4Keyword test with a new packet with valid
IPV4 checksums.
When multiple segments were put into a smsg, the seq would be updated
each time a segment was added. Because of this, the seq wasn't pointing
to the start of the data.
This caused some false negatives when the fast_pattern was in the raw
stream, but another part of the inspection was in the state. Because of
the wrong seq, the inspection of the smsg could be delayed. This in turn,
could make the inspection engine consider a TX inspected, even if it wasn't
fully yet.
When installing the rules to tell mPIPE to send packet to Suricata,
give it a higher priority than the default used by Linux. This way if
Linux also tells mPIPE to send it packets, Suricata will get them
instead, as long as Suricata is running.
Libhtp decodes the + character in the query string to a space by default.
Suricata rules (e.g. etpro sid 2806767) are expecting to see the space in
the http_uri buffer.
Added an option per htp config to reenable this default behavior:
query-plusspace-decode: yes
Bug #1035.
Now that we call stream reassembly directly from proto detection, we will
need to check if reassembly has been disabled inside the stream reassembly
callback.
This prevents any calls to bypass and re-enter proto detection, despite
having reassembly disabled.
Make sure we register the detect.alerts counter before packet runtime starts
even in delayed detect mode. The registration of new counters at packet
runtime is not supported by the counters api and might lead to crashes as there
is no proper locking to allow for this operation.
This changes how delayed detect works a bit. Now we call the ThreadInit
callback twice. The first call will only register the counter. The 2nd call
will do all the other setup. This way the counter is registered before the
counters api starts operating in the packet runtime.
Fixes the segv reported in ticket #1018.
This patch is a result of applying the following coccinelle
transformation to suricata sources:
@istested@
identifier x;
statement S1;
identifier func =~ "(SCMalloc|SCStrdup|SCCalloc|SCMallocAligned|SCRealloc)";
@@
x = func(...)
... when != x
- if (x == NULL) S1
+ if (unlikely(x == NULL)) S1
Provide support for icmvp4 and icmpv6 as well. You can now use
alert icmpv4 and
alert icmpv6 as well, apart from the existing
alert icmp, which created a rule that applied to both icmpv4 and icmpv6.
cppcheck:
[detect-fast-pattern.c:1183] -> [detect-fast-pattern.c:1183]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1217] -> [detect-fast-pattern.c:1217]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1449] -> [detect-fast-pattern.c:1449]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1479] -> [detect-fast-pattern.c:1479]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1509] -> [detect-fast-pattern.c:1509]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1539] -> [detect-fast-pattern.c:1539]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1570] -> [detect-fast-pattern.c:1570]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1686] -> [detect-fast-pattern.c:1686]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1716] -> [detect-fast-pattern.c:1716]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1746] -> [detect-fast-pattern.c:1746]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1776] -> [detect-fast-pattern.c:1776]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1806] -> [detect-fast-pattern.c:1806]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1836] -> [detect-fast-pattern.c:1836]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1866] -> [detect-fast-pattern.c:1866]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1896] -> [detect-fast-pattern.c:1896]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:1926] -> [detect-fast-pattern.c:1926]: (style) Same expression on both sides of '&'.
[detect-fast-pattern.c:2022] -> [detect-fast-pattern.c:2022]: (style) Same expression on both sides of '&'.
cppcheck said:
[decode-pppoe.c:58] -> [decode-pppoe.c:60]: (performance, inconclusive) Variable 'pppoedh' is reassigned a value before the old one has been used if variable is no semaphore variable.
Take VLAN IDs into account when re-assembling fragments.
Prevents fragments that would otherwise match, but on different
VLANs from being reassembled with each other.
Fix bug #995. Tag time setting was initialized using "usec" field
instead of "sec" field. This led to immediate timing out of tag.
Added proper matching unittests for all tagging types.
Bug #995.
We don't support jabber protocol detection atm. Disable the code check
inside suricata to check if jabber protocol detection is enabled in the
yaml file.
Also updated an error log message for app layer.
In case of recursive call to protocol detection from within protocol
detection, and the recursively invoked stream still hasn't been ack'ed
yet, protocol detection doesn't take place. In such cases we will end up
still calling the app layer with the wrong direction data. Introduce a
check to not call app layer with wrong direction data.
When sockets are re-used reset all relevant vars correctly.
This commit fixes a bug where we were not reseting app proto detection
vars.
While fixing #989, we discovered some other bugs which have also been
fixed, or rather some features which are now updated. One of the feature
update being if we recieve wrong direction data first, we don't reset the
protocol values for the flow. We let the flow retain the detected
values.
Unittests have been modified to accomodate the above change.
Use the tx id stored for each alert to find the correct XFF address
to add to the extra-data field.
In overwrite mode we still only grab the first available XFF addr,
as this address is set in the header preceeding the individual alerts.
Issue #904.
When generating an alert and storing it in the packet, store the tx_id
as well. This way the output modules can log the tx_id and access the
proper tx for logging.
Issue #904.
Some events we retrieved from error messages are flag now, so check
those. Not all can be converted though. These are no longer set:
HTTP_DECODER_EVENT_INVALID_TRANSFER_ENCODING_VALUE_IN_RESPONSE
HTTP_DECODER_EVENT_INVALID_AUTHORITY_PORT
Part of Bug #982.
Ken Steele
Victor Julien
Eric Leblond
Anoop Saldanha
Nelson Escobar
Jason Ish