1666 Commits (master)

Author	SHA1	Message	Date
Pierre Chifflier	cf5de0c58e	SMB: use String::from_utf8_lossy in logging functions	7 years ago
Pierre Chifflier	b5529e4ffb	SMB: use kerberos-parser to extract Real and PrincipalName	7 years ago
Victor Julien	0dfb3f0e7f	smb1: extract rename info from TRANS2 ... Exclude TRANS2 from generic TX lookup bypass.	7 years ago
Victor Julien	8eeda113c8	smb1: add parsing for RENAME command	7 years ago
Victor Julien	7b61f2c589	smb2: log renames	7 years ago
Victor Julien	15978d4e85	smb: if filename is missing, use '<unknown>'	7 years ago
Jason Ish	27fd521420	eve/dns/v2: support eve/dns v2 in rust	7 years ago
Jason Ish	57d9574839	rust/json: expose more of jansson to rust	7 years ago
Victor Julien	71742ed52b	smb: share can't be <share_root>	7 years ago
Victor Julien	bc193242ad	smb1: add OPEN_ANDX command name for logging	7 years ago
Victor Julien	32b19fac99	smb2: don't log/track each READ/WRITE/etc	7 years ago
Victor Julien	fb986abe81	smb: log file FID/GUID as fuid	7 years ago
Victor Julien	816bd022a6	smb1: improve non nt-status handling ... Support SRV error, with a couple of codes. Rename statux field to status_code.	7 years ago
Victor Julien	0519807639	smb1: ignore tree_id in session setup	7 years ago
Victor Julien	286c054472	smb: improve nbss/smb record detection	7 years ago
Victor Julien	7ab071a58d	rust/smb: implement minimal record parsing in probing	7 years ago
Victor Julien	283be3cade	smb2: break out ioctl handling	7 years ago
Victor Julien	bf08285602	smb2: parse async records	7 years ago
Victor Julien	5c26020714	smb2: add ioctl transactions to log the funcs	7 years ago
Victor Julien	75265ec376	smb2: map ioctl funcs to names ... List is based on Wireshark's list.	7 years ago
Victor Julien	7cd66516f0	smb: use formal MS names for disposition	7 years ago
Victor Julien	f7ed749d4f	smb: disable debug output	7 years ago
Victor Julien	eed492547c	smb1: extract server guid from negotiate	7 years ago
Victor Julien	6d56edc3de	smb2: log client and server guid from negotiate	7 years ago
Victor Julien	c56f5e11ca	smb2: log share type	7 years ago
Victor Julien	d75ebdb981	smb: log create empty filename as '<share_root>' like Bro does	7 years ago
Victor Julien	fcbeab70a4	smb1: log create 'service' fields	7 years ago
Victor Julien	90e2abaac4	smb1: use generic string parsing for trans	7 years ago
Victor Julien	76917a8732	smb1: generic smb string parse func	7 years ago
Victor Julien	668c747aee	smb1: more exact tree connect record parsing	7 years ago
Victor Julien	0ed00cf104	smb: move common parsing funcs into own file	7 years ago
Victor Julien	1c701dc50e	smb: make string parsing functions public	7 years ago
Victor Julien	1d4aac1d4d	smb1: set event on empty/malformed dialect	7 years ago
Victor Julien	c91242e71c	smb: rename file to filename in output	7 years ago
Victor Julien	caf29e92b3	smb1: parse and log timestamps in CREATE	7 years ago
Victor Julien	0e05ef7369	smb2: parse and log timestamps in CREATE	7 years ago
Victor Julien	28f16e38ac	smb1: disable 'generic tx's for common commands ... Don't create a generic TX for each READ, WRITE, TRANS, TRANS2, except if they cause events to trigger.	7 years ago
Victor Julien	78cd92a933	smb: generic event per trans/read/write for tx events	7 years ago
Victor Julien	05992f1772	smb: fix event handling when no tx is available	7 years ago
Victor Julien	be615c9fbc	smb: small cleanups, fixes and optimizations	7 years ago
Victor Julien	dab055d8c8	smb: update to der-parser 0.5.1	7 years ago
Victor Julien	0d69e7b8c2	smb: remove unused dialects from state	7 years ago
Victor Julien	ad1bc7f473	smb1: minor debug improvment	7 years ago
Victor Julien	a44504a1bf	smb: redo gap catch up handling	7 years ago
Victor Julien	7114d5d25b	smb1: parser cleanups	7 years ago
Victor Julien	d9e43d3e63	smb: cleaner server component parsing	7 years ago
Victor Julien	ecbf10da70	smb2: improve write error handling	7 years ago
Victor Julien	b34392051d	smb3: parse transform records	7 years ago
Victor Julien	894a73ee06	smb2: add missing commands and improve ioctl err handling	7 years ago
Victor Julien	170edf7c44	smb1: improve error handling	7 years ago
Victor Julien	7ceb67138f	smb: add status	7 years ago
Victor Julien	98b926bf72	smb1: implement WRITE_AND_CLOSE	7 years ago
Victor Julien	595557eb8d	smb1: locking andx may have no response	7 years ago
Victor Julien	7dff9b9969	smb/nbss: work around bad traffic	7 years ago
Victor Julien	8bef120898	smb: session setup improvements ... Improve ntlmssp version extraction and logging, make its data structures optional. Extract native os/lm from smb1 ssn setup. Move session setup handling into their own files. Only log auth data for the session setup tx.	7 years ago
Victor Julien	75d7c9d64a	rust/smb: initial support ... Implement SMB app-layer parser for SMB1/2/3. Features: - file extraction - eve logging - existing dce keyword support - smb_share/smb_named_pipe keyword support (stickybuffers) - auth meta data extraction (ntlmssp, kerberos5)	7 years ago
Pierre Chifflier	b69acaadf5	Rust: add 'debug' feature ... The 'debug' feature is enabled if suricata was configured with the --enabled-debug' flag. If enabled, the SCLogDebug format and calls the logging function as usual. Otherwise, this macro is a no-op (similarly to the C code).	7 years ago
Victor Julien	053022931c	rust/json: add array_append_string	7 years ago
Victor Julien	73fac478a2	rust/dns: fix nom verbose error mode	7 years ago
Jason Ish	c411519605	app-layer: remove has events callback - not used	7 years ago
Pierre Chifflier	92b537d028	rust: update 'external' api for app layer changes ... Remove unused HasTxDetectState function and remove state argument from SetTxDetectState. Update NTP code.	7 years ago
Victor Julien	f815027cdf	rust/dns: simplify tx freeing ... Now that we no longer need the state when freeing a TX, we can simply do cleanup from the Drop trait.	7 years ago
Victor Julien	7548944b49	app-layer: remove unused HasTxDetectState call ... Also remove the now useless 'state' argument from the SetTxDetectState calls. For those app-layer parsers that use a state == tx approach, the state pointer is passed as tx. Update app-layer parsers to remove the unused call and update the modified call.	7 years ago
Victor Julien	1c270cae13	nfs: remove old test code	7 years ago
Victor Julien	e96d9c1159	app-layer: add tx iterator API ... Until now, the transaction space is assumed to be terse. Transactions are handled sequentially so the difference between the lowest and highest active tx id's is small. For this reason the logic of walking every id between the 'minimum' and max id made sense. The space might look like: [..........TTTT] Here the looping starts at the first T and loops 4 times. This assumption isn't a great fit though. A protocol like NFS has 2 types of transactions. Long running file transfer transactions and short lived request/reply pairs are causing the id space to be sparse. This leads to a lot of unnecessary looping in various parts of the engine, but most prominently: detection, tx house keeping and tx logging. [.T..T...TTTT.T] Here the looping starts at the first T and loops for every spot, even those where no tx exists anymore. Cases have been observed where the lowest tx id was 2 and the highest was 50k. This lead to a lot of unnecessary looping. This patch add an alternative approach. It allows a protocol to register an iterator function, that simply returns the next transaction until all transactions are returned. To do this it uses a bit of state the caller must keep. The registration is optional. If no iterator is registered the old behaviour will be used.	7 years ago
Victor Julien	6e82df274d	rust: update dependencies	7 years ago
Victor Julien	dfae3297a5	rust: don't gen C headers if Rust isn't enabled	7 years ago
Pascal Delalande	80f2fbac6e	rust/tftp: eve logging with rust	7 years ago
Clement Galland	b9cf49e933	rust/tftp: add tftp parsing and logging ... TFTP parsing and logging written in Rust. Log on eve.json the type of request (read or write), the name of the file and the mode. Example of output: "tftp":{"packet":"read","file":"rfc1350.txt","mode":"octet"}	7 years ago
Victor Julien	e8939335ea	rust/nfs: explicitly handle GAPs from C ... It seems that Rust optimizes this code in such a way that it passes the null ptr along as real data. if buf.as_ptr().is_null() && input_len > 0 {	7 years ago
Victor Julien	2c3c8f8b85	rust/filetracker: if file API return error, trunc file	7 years ago
Victor Julien	d27ed5957f	rust/nfs: fix read reply handling ... READ replies with large data chunks are processed partially to avoid queuing too much data. When the final chunk was received however, the start of the chunk would already tag the transaction as 'done'. The more aggressive tx freeing that was recently merged would cause this tx to be freed before the rest of the in-progress chunk was done. This patch delays the tagging of the tx until the final data has been received.	7 years ago
Victor Julien	3a2e4614d0	rust/file: handle file open errors	7 years ago
Victor Julien	45c5030ff0	rust/file: change return type for FileOpenFileWithId ... Make it int so we can easily check it in Rust. No consumer used the File pointer that was returned before anyway.	7 years ago
Victor Julien	288ddc95ac	rust/core: comment cleanup	7 years ago
Victor Julien	8cda2a4351	rust/nfs: add support for detect_flags API	7 years ago
Victor Julien	98eca55241	rust/dns: implement detect_flags API	7 years ago
Pierre Chifflier	4b6555588f	NTP: ensure parser name is not freed after registration	7 years ago
Pierre Chifflier	ec62eedc87	Rust: remove deprecated functions LoggerFlags::get_logged/set_logged	7 years ago
Pierre Chifflier	5c6868b327	NTP: update logger to use new API	7 years ago
Victor Julien	bca0cd71ae	app-layer: use logger bits to avoid looping ... Avoid looping in transaction output. Update app-layer API to store the bits in one step and retrieve the bits in a single step as well. Update users of the API.	7 years ago
Victor Julien	e1e9ada9df	rust/nfs: improve file close handling	7 years ago
Nick Price	350b5d99ce	rust/nfs: don't panic on malformed NFS traffic ... Instead set events.	7 years ago
Pierre Chifflier	f5b27ae767	Rust: fix probing function prototype: change sign and add Flow	7 years ago
Victor Julien	6c251b8576	rust: add --enable-rust-debug ... Add option to put Rust code in non-'--release' mode, preserving debug symbols. Until now Suricata would have to be compiled with --enable-debug for this.	7 years ago
Eric Leblond	b0a6934431	app-layer-ftp: add ftp-data support ... Use expectation to be able to identify connections that are ftp data. It parses the PASV response, STOR message and the RETR message to provide extraction of files. Implementation in Rust of FTP messages parsing is available. Also this patch changes some var name prefixed by ssh to ftp.	7 years ago
Jason Ish	5a8537fe4a	rust/dns - convert more type values to text ... Issue: https://redmine.openinfosecfoundation.org/issues/2364 Convert more record type and errr code values to text. Remove duplicate type declarations.	7 years ago
Clément Galland	3396747cd6	Dns logger display flags information	7 years ago
Victor Julien	bb65a48edd	rust: require at least libc 0.2.33 ... Required to be higher than 0.2.24 for IPPROTO_UDP. Upgraded to latest version.	7 years ago
Pierre Chifflier	83808bbdad	rust/ntp: convert parser to new registration method ... Converting the NTP parser to the new registration method is a simple, 3-steps process: - change the extern functions to use generic input parameters (functions in all parsers must share common types to be generic) and cast them - declare the Parser structure - remove the C code and call the registration function	7 years ago
Pierre Chifflier	0b07bdf5d9	rust: generate declaration for extern unsafe funcs	7 years ago
Pierre Chifflier	e7c0a53cbf	rust/applayer: add registration iface for parsers ... Add Rust support for the common interface to declare and register all parsers. Add a common structure definition to contain all required elements required for registering a parser, similar to the C interface. This also reduces the risk of incorrectly registering a parser: the compiler prevents omitting required functions from the structure, and functions (even if external) are type-checked. Optional functions are explicitly marked.	7 years ago
Jason Ish	7eead7dfbc	autotools: fix distcheck with rust enabled	7 years ago
Victor Julien	d9e5dfa1f0	rust/file: improve truncation handling	7 years ago
Victor Julien	e023ce9aad	rust/dns: fix new warning in rustc 1.21	7 years ago
Victor Julien	fd38e5e82b	rust/nfs: fix new warnings in rustc 1.21	7 years ago
Pierre Chifflier	e4129c1568	Rust/Lua: cast value to arch-dependant type (fix build on x86, #2197)	8 years ago
Jason Ish	6cfabb7863	autogen: cleanup rust strict warning	8 years ago
Jason Ish	6a4cefb7c5	rust: --enable-rust-strict to turn warnings into errors	8 years ago
Jason Ish	3063851d85	rust/dns/tcp - probe even if payload is short ... As the DNS probe just uses the query portion of a response, don't require there to be as many bytes as specified in the TCP DNS header. This can occur in large responses where probe is called without all the data. Fixes the cases where the app proto is recorded as failed. Fixes issue: https://redmine.openinfosecfoundation.org/issues/2169	8 years ago
Victor Julien	a306ccfd34	rust/nfs: implement events ... Remove lots of panic statements in favor of setting non-fatal events. Bug #2175.	8 years ago
Victor Julien	82bd732f4e	rust/nfs: improve proto detect	8 years ago
Victor Julien	6b4a04510a	rust/nfs: remove debug rec_size check ... Records larger than 40k are perfectly valid. Bug #2162.	8 years ago
Jason Ish	40991cab82	rust/dns: handle multiple txt strings ... Fix handling of TXT records when there are multiple strings in a single TXT record. For now, conform to the C implementation where an answer record is created for each string in a single txt record. Also removes the data_len field from the answer entry. In Rust, the length is available from actual data, which after decoding may actually be different than the encoded data length, so just use the length from the actual data.	8 years ago
Pierre Chifflier	8a0549c42e	NTP: change parse function to return the number of parsed messages	8 years ago
Pierre Chifflier	efe11dc37e	Add NTP parser (rust-experimental)	8 years ago
Pierre Chifflier	4f677fd157	Rust gen-c-headers: keep 'const' attribute	8 years ago
Pierre Chifflier	4fe9292ed8	Autotools: add switch to build experimental Rust parsers	8 years ago
Jason Ish	61d9f4bb0a	rust: make distcheck fixes	8 years ago
Jason Ish	f5a90e26a9	rust: for sclog*, strip nul bytes before logging	8 years ago
Jason Ish	717b826d25	rust: safe string handling in logging ... In logging (SCLog*), safely convert strings to cstrings instead of blindly unwrapping them. Also implement a simple rust logger if the Suricata C context is not available.	8 years ago
Jason Ish	14951e3f00	rust: save cargo and CARGO_HOME to variables ... During configure, substitute the path of cargo, as well as the value of CARGO_HOME as variables. This fixes the case where a user might do: make sudo make install Which will cause the cargo bits to be rebuilt, including re-downloading external crates. By saving these to variables we can be sure that the same values are used during make install as were used during make which prevents the Rust artifacts from being rebuild during "sudo make install".	8 years ago
Victor Julien	7c119cc595	nfs: log number of chunks that xfer'd a file	8 years ago
Victor Julien	e8dae2e093	nfs: add to fileinfo events	8 years ago
Victor Julien	db2d928151	rust/nfs: add (file)handle to log as crc32	8 years ago
Jason Ish	829155b9d5	rust/dns: pass byte arrays directly to rust/json ... Using the json.set_string_from_bytes which will safely convert the bytes printable ascii string before logging.	8 years ago
Jason Ish	96cc503026	rust/lua: use lua_pushlstring for strings ... Lua strings can contain NULLs, and Rust strings are UTF8 which can also contain NULLs. Use pushlstring so a NULL containing string can be pushed.	8 years ago
Jason Ish	6dbc5be4be	rust/json: only output printable characters ... Rust strings are UTF8 and we cannot yet rely on jansson having json_stringn on all supported OS distributions yet so sanitize strings to ascii before printing. Also add set_string_from_bytes which is like set_string, but accepts a byte array as input.	8 years ago
Victor Julien	becf1a2dfe	rust/nfs: fix style warning	8 years ago
Victor Julien	e0c6565e68	nfs: nfs_version keyword ... Store nfs version in tx and add keyword to match on it.	8 years ago
Victor Julien	aff576b524	eve/nfs: log nfs version	8 years ago
Victor Julien	0d79181d78	nfs: rename nfs3 to nfs ... Since the parser now also does nfs2, the name nfs3 became confusing. As it's still in beta, we can rename so this patch renames all 'nfs3' logic to simply 'nfs'.	8 years ago
Victor Julien	28cdf7b628	nfs3: create file tx for read on request ... This is done so that we can add creds to it.	8 years ago
Victor Julien	7e0d9619ac	nfs3: add readdirplus path	8 years ago
Victor Julien	41376da03c	nfs: log more rpc	8 years ago
Victor Julien	9edbb6f235	nfs: split record parsers into different files	8 years ago
Victor Julien	25edac7666	nfs3: fill bytes corner case	8 years ago
Victor Julien	5153271b87	nfs2: basic record parsing and tracking	8 years ago
Victor Julien	c7e10c73f9	nfs3: support NFS over UDP	8 years ago
Victor Julien	d9f87cec3d	nfs3: probing parsers in both directions	8 years ago
Victor Julien	8fe32f943b	nfs3: search for next record if needed after GAP	8 years ago
Victor Julien	58af39131f	rust/nfs: handle GAPs ... In normal records it will try to continue parsing. GAP 'data' will be passed to file api as '0's. New call is used so that the file API does know it is dealing with a GAP. Such files are flagged as truncated at the end of the file and no checksums are calculated.	8 years ago
Victor Julien	a116c16019	nfs3: parse mkdir and rmdir request records	8 years ago
Jason Ish	6bddc4d3e0	python: use python path found during configure ... Also look for Python under more names. For example, on OpenBSD if you just install Python 2, you will only get a python2.7 executable.	8 years ago
Jason Ish	c473c56eed	rust/dns: fix panic on rrnames with bad chars ... Check for erros in the UTF-8 conversion, on error, print the the printable chars as chars, and print non printable chars as \xHEX. Redmine issue: https://redmine.openinfosecfoundation.org/issues/2148	8 years ago
Jason Ish	ecc63481c6	rust/dns: fix tcp message length verification ... And add Rust unit tests to check length validation. Redmine issue 2144: https://redmine.openinfosecfoundation.org/issues/2144	8 years ago
Jason Ish	70808a4f1d	rust/dns: support gaps in TCP DNS	8 years ago
Jason Ish	4bdb722371	rust/dns: fix unit tests on Rust 1.7.0	8 years ago
Jason Ish	2aebfbce94	rust/dns: support txt records	8 years ago
Jason Ish	fafa75035f	rust: don't fail distcheck if cargo-vendor not found ... Allow distcheck to pass if cargo vendor is not found by not failing out. It is not required to successfully build a dist tarball, the Rust sources will just not be vendored in. Also don't fail out make dist if Python is not installed. A build will still be successful is Python is available on the end build system.	8 years ago
Jason Ish	33e09a0002	rust dns: fixup for nom 3.0	8 years ago
Jason Ish	ee83f7a0db	rust: build fixes and nom update ... Update nom to ~3.0. Prefix dependencies with ~, which will allow for newer patch versions only. Minor version updates should get a test before using. Remove Cargo.lock from the repo, but still generate as part of the vendoring process for release builds. This will ensure that all users of a particular distribution tarball will be linking against the same Rust dependencies.	8 years ago
Victor Julien	098aced714	rust/nfs/files: no longer Option/Box	8 years ago
Victor Julien	75a6a13790	rust/nfs: move files into tx type data	8 years ago
Victor Julien	de7e0614fa	rust/nfs: add more record types	8 years ago
Victor Julien	d6592211d0	rust/nfs: NFSv3 parser, logger and detection	8 years ago
Victor Julien	69bf219b39	rust: bindings: improve generator script	8 years ago
Victor Julien	71ddc43d49	rust/core: add file tx API call	8 years ago
Victor Julien	9a1fa5f1f4	rust: filetracker API ... Initial version of a filetracker API that depends on the filecontainer and wraps around the Suricata File API in C. The API expects chunk based transfers where chunks can be out of order.	8 years ago
Victor Julien	a809f090d3	rust: filecontainer API ... Wrapper around Suricata's File and FileContainer API. Built around assumption that a rust owned structure will have a 'SuricataFileContainer' member that is managed by the C-side of things.	8 years ago
Victor Julien	f47fd2c243	rust/json: expose json_boolean	8 years ago
Jason Ish	ba1a67e2cb	rust: dns: add log filtering on rrtype ... While the filtering is still configured in C, the filtering flags are passed into Rust so it can determine if a record should be logged or not.	8 years ago
Jason Ish	c54fc7f98f	rust: use LoggerFlags type to track logged state	8 years ago
Jason Ish	b588b49779	rust: lua support for DNS based Rust ... Uses Rust wrappers around Lua to populate Lua data structures.	8 years ago
Jason Ish	9d687025e2	rust: lua wrapper ... Rust wrapper for working with lua state.	8 years ago
Jason Ish	73388042b2	rust: DNS app-layer. ... A DNS application layer in Rust. This is different than the C based one, as it is partially stateless by not matching up responses to replies.	8 years ago
Jason Ish	9449739dd5	rust: dns: nom DNS parsers	8 years ago
Jason Ish	94032d3ada	rust: wrapper around C logging, and "context" ... Where the context is a struct passed from C with pointers to all the functions that may be called. Instead of referencing C functions directly, wrap them in function pointers so pure Rust unit tests can still run.	8 years ago
Jason Ish	9231b0ae92	rust: generate headers as part of build	8 years ago
Jason Ish	d0880d75ff	rust: c header generator	8 years ago
Jason Ish	e739fa1477	rust: add libjansson wrapper for rust	8 years ago
Jason Ish	f6f126d53d	rust: example of how an app-layer may be initialized ... Also shows basic usage of the configuration API from Rust.	8 years ago
Jason Ish	949b358b80	rust: stub out configuration access functions	8 years ago
Jason Ish	de5bb1f953	rust: stub out logging from rust	8 years ago
Jason Ish	8f81792da5	rust: hook rust into the build ... Rust is currently optional, use the --enable-rust configure argument to enable Rust. By default Rust will be built in release mode. If debug is enabled then it will be built in debug mode. On make dist, "cargo vendor" will be run to make a local copy of Rust dependencies for the distribution archive file. Add autoconf checks to test for the vendored source, and if it exists setup the build to use the vendored code instead of fetching it from the network. Also, as Cargo requires semantic versioning, the Suricata version had to change from 4.0dev to 4.0.0-dev.	8 years ago
Jason Ish	cf0b9dd45f	rust: add rust skeleton tree	8 years ago