In TCP, large gaps in the data could lead to an extremely poor utilization
of the streaming buffer memory. This was caused by the implementation using
a single continues memory allocation from the "stream offset" to the
current data. If a 100 byte segment was inserted for ISN + 20MiB, we would
allocate 20MiB, even if only 100 bytes were actually used.
This patch addresses the issue by implementing a list of memory regions.
The StreamingBuffer structure holds a static "main" region, which can be
extended in the form of a simple list of regions.
[ main region ] [ gap ] [ aux region ]
[ sbb ] [ sbb ]
On insert, find the correct region and see if the new data fits. If it
doesn't, see if we can expand the current region, or than we need to add
a new region. If expanding the current region means we overlap or get
too close to the next region, we merge them.
On sliding, we free any regions that slide out of window and consolidate
auxilary regions into main where needed.
Bug: #4580.
After a gap in a file transaction, the file tracker is truncated. However
this did not clear any stored out of order chunks from memory or stop more
chunks to be stored, leading to accumulation of a large number of chunks.
This patches fixes this be clearing the stored chunks on trunc. It also
makes sure no more chunks are stored in the tracker after the trunc.
Bug: #5781.
In the case port_line is first allocated and port_line_len is set,
Then a second request reaches memcap and frees port_line,
port_line_len should also be reset, because both will get used
by the response parsing.
Ticket: #5701
Tested on Fedora 37 with clang 15.
util-strlcatu.c:45:8: error: a function definition without a prototype is deprecated in all versions of C and is not supported in C2x [-Werror,-Wdeprecated-non-prototype]
size_t strlcat(dst, src, siz)
^
1 error generated.
Tested on Fedora 37 with clang 15.
datasets.c:852:9: error: variable 'n' set but not used [-Werror,-Wunused-but-set-variable]
int n = 0;
^
1 error generated.
Tested on Fedora 37 with clang 15.
app-layer.c:1055:27: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
void AppLayerSetupCounters()
^
void
app-layer.c:1176:29: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
void AppLayerDeSetupCounters()
^
void
2 errors generated.
Indexing of Signature::init_data::smlists would fail for a rule that
used a frame and fast_pattern w/o content, as the array would only be
expanded when adding a content.
Adding a check to see if there list id is in bounds
is an implicit check for the "no content" case.
Ticket: #5530
In the case of a gap, or in the case of a flow where one side
is recognized, but the other is not before the end, we still
increase the counters to have consistency between
jq 'select(.event_type=="flow" and .app_proto=="ftp") | .app_proto' log/eve.json | wc -l
jq 'select(.event_type=="stats") | .stats."app_layer".flow.ftp' log/eve.json
Ticket: #5769
As introduced by commit f848e34bcc
DoInsertSegment can now return `-EINVAL` and there was no
generic handling of error values to return the tcp segment
to its pool.
Ticket: #5777
Issue: 5198
This commit modifies the threaded logging support to use the hash table
for handling thread/slot mappings. As a result, it's no longer necessary
to provide the thread id when ensuring the log output exists.
Issue: 5198
This commit adds a hash table to manage thread id to slot mappings. This
ensures that each thread will have its own slot (file output device)
Victor Julien
Philippe Antoine
jason taylor
dependabot[bot]
Jason Ish
Haleema Khan
Jeff Lucovsky
Juliana Fajardini