TmThreadCreate copy string provided as name for threads to
avoid any issue is a non allocated string is used.
This patch also introduce TmThreadSetGroupName function. This
function is used to be sure we have an allocation when
assigning the thread group name. This way we can free allocated
memory at exit.
Both code changes have required some fixes in different parts of
the code to be in sync with the new API.
Good point about these changes is that it fixes an inconsistency
were some names were not allocated and some were.
This fixes:
72 bytes in 3 blocks are definitely lost in loss record 153 of 316
at 0x4C29C0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
by 0x9AF041: SCRadixCreateRadixTree (util-radix-tree.c:430)
by 0x50FF5D: DefragPolicyLoadFromConfig (defrag-config.c:138)
by 0x5129F5: DefragInit (defrag.c:962)
by 0x87ECFD: UnixSocketPcapFilesCheck (runmode-unix-socket.c:386)
by 0x90FEC0: UnixCommandBackgroundTasks (unix-manager.c:430)
by 0x913C6D: UnixManager (unix-manager.c:980)
by 0x9072F3: TmThreadsManagement (tm-threads.c:602)
by 0x68DE283: start_thread (pthread_create.c:333)
by 0x80A6A4C: clone (in /lib/x86_64-linux-gnu/libc-2.21.so)
This fixes:
16 bytes in 2 blocks are definitely lost in loss record 69 of 319
at 0x4C29C0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
by 0x85955D: OutputStatsLogThreadInit (output-stats.c:118)
by 0x4CAE13: StatsMgmtThread (counters.c:352)
by 0x68DE283: start_thread (pthread_create.c:333)
by 0x80A6A4C: clone (in /lib/x86_64-linux-gnu/libc-2.21.so)
This fixes:
96 bytes in 4 blocks are definitely lost in loss record 178 of 320
at 0x4C29C0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
by 0x8FF88C: TmqhOutputFlowSetupCtx (tmqh-flow.c:163)
by 0x908D5F: TmThreadCreate (tm-threads.c:1097)
by 0x909A62: TmThreadCreatePacketHandler (tm-threads.c:1156)
by 0x8790CF: RunModeFilePcapAutoFp (runmode-pcap-file.c:188)
by 0x8837CB: RunModeDispatch (runmodes.c:372)
by 0x87F2A5: UnixSocketPcapFilesCheck (runmode-unix-socket.c:393)
by 0x910460: UnixCommandBackgroundTasks (unix-manager.c:430)
by 0x91420D: UnixManager (unix-manager.c:980)
by 0x907853: TmThreadsManagement (tm-threads.c:602)
by 0x68DE283: start_thread (pthread_create.c:333)
by 0x80A6A4C: clone (in /lib/x86_64-linux-gnu/libc-2.21.so)
Fix cleaning of threads where mutex and condition where not freed.
This fixes:
352 (192 direct, 160 indirect) bytes in 4 blocks are definitely lost in loss record 301 of 327
at 0x4C29C0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
by 0x909404: TmThreadInitMC (tm-threads.c:1764)
by 0x908DE7: TmThreadCreate (tm-threads.c:1120)
by 0x90A326: TmThreadCreateMgmtThread (tm-threads.c:1183)
by 0x4CA0AD: StatsSpawnThreads (counters.c:856)
by 0x87F254: UnixSocketPcapFilesCheck (runmode-unix-socket.c:396)
by 0x910330: UnixCommandBackgroundTasks (unix-manager.c:430)
by 0x9140DD: UnixManager (unix-manager.c:980)
by 0x9077F3: TmThreadsManagement (tm-threads.c:600)
by 0x68DE283: start_thread (pthread_create.c:333)
by 0x80A6A4C: clone (in /lib/x86_64-linux-gnu/libc-2.21.so)
Queue name was sometimes allocated and sometimes not. This
patch updates the behavior of creation function so it is
always allocated. This way we can free it at exit and fix
memory leak.
This fixes:
900 bytes in 110 blocks are definitely lost in loss record 322 of 329
at 0x4C29C0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
by 0x803E0A9: strdup (in /lib/x86_64-linux-gnu/libc-2.21.so)
by 0x90090E: StoreQueueId (tmqh-flow.c:112)
by 0x8FFEA8: TmqhOutputFlowSetupCtx (tmqh-flow.c:180)
by 0x908C7F: TmThreadCreate (tm-threads.c:1095)
by 0x909982: TmThreadCreatePacketHandler (tm-threads.c:1154)
by 0x87906F: RunModeFilePcapAutoFp (runmode-pcap-file.c:188)
by 0x88376B: RunModeDispatch (runmodes.c:372)
by 0x87F245: UnixSocketPcapFilesCheck (runmode-unix-socket.c:393)
by 0x9102B0: UnixCommandBackgroundTasks (unix-manager.c:430)
by 0x91405D: UnixManager (unix-manager.c:980)
by 0x907773: TmThreadsManagement (tm-threads.c:600)
tm-queue: fix logic WIP
This function is causing a memleak because it is necessary to clean
up after usage.
This fixes at least:
37 (32 direct, 5 indirect) bytes in 1 blocks are definitely lost in loss record 104 of 394
at 0x4C29C0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
by 0x66D0C7B: ??? (in /usr/lib/x86_64-linux-gnu/libjansson.so.4.7.0)
by 0x943584: LiveDeviceIfaceList (util-device.c:264)
by 0x910889: UnixCommandExecute (unix-manager.c:486)
by 0x910D7E: UnixCommandRun (unix-manager.c:545)
by 0x911193: UnixMain (unix-manager.c:593)
by 0x913CC7: UnixManager (unix-manager.c:961)
by 0x907753: TmThreadsManagement (tm-threads.c:600)
by 0x68DE283: start_thread (pthread_create.c:333)
by 0x80A6A4C: clone (in /lib/x86_64-linux-gnu/libc-2.21.so)
This fixes:
2,595 (1,824 direct, 771 indirect) bytes in 57 blocks are definitely lost in loss record 328 of 332
at 0x4C29C0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
by 0x66D0C7B: ??? (in /usr/lib/x86_64-linux-gnu/libjansson.so.4.7.0)
by 0x911A27: UnixManagerListCommand (unix-manager.c:766)
by 0x9108A9: UnixCommandExecute (unix-manager.c:486)
by 0x910D9E: UnixCommandRun (unix-manager.c:545)
by 0x9111B3: UnixMain (unix-manager.c:593)
by 0x913D27: UnixManager (unix-manager.c:961)
by 0x907773: TmThreadsManagement (tm-threads.c:600)
by 0x68DE283: start_thread (pthread_create.c:333)
by 0x80A6A4C: clone (in /lib/x86_64-linux-gnu/libc-2.21.so)
This fixes:
Direct leak of 31752 byte(s) in 3969 object(s) allocated from:
#0 0x4c396b in malloc (/opt/suricata-asan/bin/suricata+0x4c396b)
#1 0xe385b9 in OutputTxLogThreadInit /home/pmanev/sandnet-qa/stage/oisf/src/output-tx.c:193:34
#2 0x106c255 in TmThreadsSlotPktAcqLoop /home/pmanev/sandnet-qa/stage/oisf/src/tm-threads.c:295:17
#3 0x7fbc9fcb3181 in start_thread /build/eglibc-3GlaMS/eglibc-2.19/nptl/pthread_create.c:312
This fixes:
Direct leak of 31792 byte(s) in 3974 object(s) allocated from:
#0 0x4c396b in malloc (/opt/suricata-asan/bin/suricata+0x4c396b)
#1 0xd86ce2 in OutputFiledataLogThreadInit /home/pmanev/sandnet-qa/stage/oisf/src/output-filedata.c:308:34
#2 0x106c255 in TmThreadsSlotPktAcqLoop /home/pmanev/sandnet-qa/stage/oisf/src/tm-threads.c:295:17
#3 0x7fbc9fcb3181 in start_thread /build/eglibc-3GlaMS/eglibc-2.19/nptl/pthread_create.c:312
This fixes:
Direct leak of 31832 byte(s) in 3979 object(s) allocated from:
#0 0x4c396b in malloc (/opt/suricata-asan/bin/suricata+0x4c396b)
#1 0xe22129 in OutputPacketLogThreadInit /home/pmanev/sandnet-qa/stage/oisf/src/output-packet.c:123:34
#2 0x106c255 in TmThreadsSlotPktAcqLoop /home/pmanev/sandnet-qa/stage/oisf/src/tm-threads.c:295:17
#3 0x7fbc9fcb3181 in start_thread /build/eglibc-3GlaMS/eglibc-2.19/nptl/pthread_create.c:312
This fixes:
Direct leak of 31856 byte(s) in 3982 object(s) allocated from:
#0 0x4c396b in malloc (/opt/suricata-asan/bin/suricata+0x4c396b)
#1 0xd7ff39 in OutputFileLogThreadInit /home/pmanev/sandnet-qa/stage/oisf/src/output-file.c:182:34
#2 0x106c255 in TmThreadsSlotPktAcqLoop /home/pmanev/sandnet-qa/stage/oisf/src/tm-threads.c:295:17
#3 0x7fbc9fcb3181 in start_thread /build/eglibc-3GlaMS/eglibc-2.19/nptl/pthread_create.c:312
Code was unused and was leaking memory.
This fixes:
Direct leak of 614240 byte(s) in 3839 object(s) allocated from:
#0 0x4c396b in malloc (/opt/suricata-asan/bin/suricata+0x4c396b)
#1 0x11bc12e in LogFileNewCtx /home/pmanev/sandnet-qa/stage/oisf/src/util-logopenfile.c:474:27
#2 0xcf7ef2 in LogFilestoreLogInitCtx /home/pmanev/sandnet-qa/stage/oisf/src/log-filestore.c:430:31
#3 0xec3275 in RunModeInitializeOutputs /home/pmanev/sandnet-qa/stage/oisf/src/runmodes.c:763:26
#4 0xeae17f in UnixSocketPcapFilesCheck /home/pmanev/sandnet-qa/stage/oisf/src/runmode-unix-socket.c:391:9
#5 0x109bc37 in UnixCommandBackgroundTasks /home/pmanev/sandnet-qa/stage/oisf/src/unix-manager.c:430:20
#6 0x10a9be2 in UnixManager /home/pmanev/sandnet-qa/stage/oisf/src/unix-manager.c:977:9
#7 0x1075643 in TmThreadsManagement /home/pmanev/sandnet-qa/stage/oisf/src/tm-threads.c:600:9
#8 0x7fbc9fcb3181 in start_thread /build/eglibc-3GlaMS/eglibc-2.19/nptl/pthread_create.c:312
This patch adds a new callback PktAcqBreakLoop() in TmModule to let
packet acquisition modules define "break-loop" functions to terminate
the capture loop. This is useful in case of blocking functions that
need special actions to take place in order to stop the execution.
Implement this for PF_RING
Those lines were included former to prevent possible null pointer
dereference but that won't happen anymore with the rest of the rework
done in the code. The code even results in a control flow issue reported
by coverity scan, so just remove it.
Suricata crashed when variable (either address or port)
referred to itself or if one created a looped chain of
variables. For instance:
HOME_NET: "!$EXTERNAL_NET"
EXTERNAL_NET: "!$HOME_NET"
Or:
Var1: "$Var2"
Var2: "$Var3"
Var3: "$Var1"
This patch fixes the following leak:
Direct leak of 9982880 byte(s) in 2902 object(s) allocated from:
#0 0x4c253b in malloc ??:?
#1 0x10c39ac in MimeDecInitParser /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/util-decode-mime.c:2379
#2 0x6a0f91 in SMTPProcessRequest /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/app-layer-smtp.c:1085
#3 0x697658 in SMTPParse /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/app-layer-smtp.c:1185
#4 0x68fa7a in SMTPParseClientRecord /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/app-layer-smtp.c:1208
#5 0x6561c5 in AppLayerParserParse /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/app-layer-parser.c:908
#6 0x53dc2e in AppLayerHandleTCPData /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/app-layer.c:444
#7 0xf8e0af in DoReassemble /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/stream-tcp-reassemble.c:2635
#8 0xf8c3f8 in StreamTcpReassembleAppLayer /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/stream-tcp-reassemble.c:3028
#9 0xf94267 in StreamTcpReassembleHandleSegmentUpdateACK /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/stream-tcp-reassemble.c:3404
#10 0xf9643d in StreamTcpReassembleHandleSegment /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/stream-tcp-reassemble.c:3432
#11 0xf578b4 in HandleEstablishedPacketToClient /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/stream-tcp.c:2245
#12 0xeea3c7 in StreamTcpPacketStateEstablished /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/stream-tcp.c:2489
#13 0xec1d38 in StreamTcpPacket /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/stream-tcp.c:4568
#14 0xeb0e16 in StreamTcp /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/stream-tcp.c:5064
#15 0xff52a4 in TmThreadsSlotVarRun /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/tm-threads.c:130
#16 0xffdad1 in TmThreadsSlotVar /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/tm-threads.c:474
#17 0x7f7cd678d181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2)
We come to this case when a SMTP session contains at least 2 mails
and then the ending of the first is not correctly detected. In that
case, switching to a new tx seems a good solution. This way we still
have partial logging.
If SMTP session is weird then we may reach a state where a field
like MAIL FROM is seen as duplicated.
Valgrind output is:
30 bytes in 1 blocks are definitely lost in loss record 96 of 399
at 0x4C29C0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
by 0x4A5803: SMTPParseCommandWithParam (app-layer-smtp.c:996)
by 0x4A4DCE: SMTPParseCommandMAILFROM (app-layer-smtp.c:1016)
by 0x4A3F55: SMTPProcessRequest (app-layer-smtp.c:1127)
by 0x4A1F8C: SMTPParse (app-layer-smtp.c:1191)
by 0x493AD7: SMTPParseClientRecord (app-layer-smtp.c:1214)
by 0x4878A6: AppLayerParserParse (app-layer-parser.c:908)
by 0x42384E: AppLayerHandleTCPData (app-layer.c:444)
by 0x8D7EAD: DoReassemble (stream-tcp-reassemble.c:2635)
by 0x8D795F: StreamTcpReassembleAppLayer (stream-tcp-reassemble.c:3028)
by 0x8D8BE0: StreamTcpReassembleHandleSegmentUpdateACK (stream-tcp-reassemble.c:3404)
by 0x8D8F6E: StreamTcpReassembleHandleSegment (stream-tcp-reassemble.c:3432)
When DCERPC was wrapped in SMB it wasn't being initialized or
cleaned up properly. To fix, expose DCERPC initialization and
cleanup functions for use by the SMB application layer.
Redmine ticket:
https://redmine.openinfosecfoundation.org/issues/1708
The stripping of leading and trailing "s has issues with rules like the
ones described in issue 1638 thus resulted in crashing the rule parser.
So for now this is a quick fix which approaches this issue directly by
stripping those "s correctly and handling error cases. It also adds the
skip for leading spaces at the msg keyword and worksaround a possible
null pointer dereference (that should never occur though).
A more general approach should be done in the future.
The check for the return value was wrong, we have 0 for success and 1
(and 2) for the error cases like TM_ECODE_FAILED, so we should quit
unless TM_ECODE_OK (0) is returned for NFQInitThread. This fixes#1870
All variables are initialized thanks to ModbusExtractUint8 or ModbusExtractUint16
function that extracts 8bits or 16bits data from pointer the received input data.
In case of extracting error (because of length), ModbusExtractUint8 or
ModbusExtractUint16 returns an error that is managed by the caller function.
All variables are now initialized to zero when they are declared. It does not
change anything functionnally but it removes Modbus warnings.
Eric Leblond
Victor Julien
cardigliano
Andreas Herz
Alexander Gozman
Jason Ish
DIALLO David