Super Speedy Syslog Searcher! (s4)
Speedily search and sort many syslog files by datetime.
Super Speedy Syslog Searcher (s4) can search within compressed syslog files
(.gz, .xz) and syslog archives (.tar, .zip).
The first goal of s4 is speedy searching and printing.
Use
Install
text
cargo install s4
Run
For example, print all the messages under /var/log/.
text
s4 /var/log
--help
```text
Super Speedy Syslog Searcher will search syslog files and sort entries by datetime. DateTime filters
may be passed to narrow the search. It aims to be very fast.
USAGE:
s4 [OPTIONS] ...
ARGS:
... Path(s) of syslog files or directories. Directories will be recursed,
remaining on the same filesystem. Symlinks will be followed
OPTIONS:
-a, --dt-after DateTime After filter - print syslog lines with a datetime that
is at or after this datetime. For example, '20200102T123000'
-b, --dt-before DateTime Before filter - print syslog lines with a datetime that
is at or before this datetime. For example, '20200102T123001'
-t, --tz-offset DateTime Timezone offset - for syslines with a datetime that does
not include a timezone, this will be used. For example, '-0800'
'+02:00' (with or without ':'). If passing a value with leading
'-', use the '=' to explicitly set the argument, e.g. '-t=-0800'.
Otherwise the CLI argument parsing will fail. Default is local
system timezone offset. [default: -08:00]
-u, --prepend-utc Prepend DateTime in the UTC Timezone for every line
-l, --prepend-local Prepend DateTime in the Local Timezone for every line
-n, --prepend-filename Prepend file basename to every line
-p, --prepend-filepath Prepend file full path to every line
-w, --prepend-file-align Align column widths of prepended data
-c, --color Choose to print to terminal using colors [default: auto]
[possible values: always, auto, never]
-z, --blocksz Read blocks of this size. May pass decimal or hexadecimal
numbers. Using the default value is recommended [default: 65535]
-s, --summary Print a summary of files processed. Printed to stderr
-h, --help Print help information
-V, --version Print version information
DateTime Filter patterns may be:
"%Y%m%dT%H%M%S"
"%Y%m%dT%H%M%S%z"
"%Y-%m-%d %H:%M:%S"
"%Y-%m-%d %H:%M:%S %z"
"%Y-%m-%dT%H:%M:%S"
"%Y-%m-%dT%H:%M:%S %z"
"%Y/%m/%d %H:%M:%S"
"%Y/%m/%d %H:%M:%S %z"
"%Y%m%d"
"%Y%m%d %z"
"+%s"
Without a timezone offset (%z or %Z), the Datetime Filter is presumed to be the
system timezone.
Pattern "+%s" is Unix epoch timestamp in seconds with a preceding "+".
Ambiguous timezones will be rejected, e.g. "SST".
Prepended datetime, -u or -l, is printed in format "%Y%m%dT%H%M%S%.6f %z:".
DateTime formatting is described at https://docs.rs/chrono/latest/chrono/format/strftime/
DateTimes supported are only of the Gregorian calendar.
DateTimes supported language is English.
```
About
Features
- Prepends datetime and file paths, for easy programmatic parsing or visual
traversal of varying syslog messages
- Supports many varying datetime formats including (WHAT ARE THOSE OFFICIAL ONES?)
- Tested against "in the wild" log files from varying Linux distributions
(see
./logs/)
- Comparable speed as GNU
grep and sort
(see ./tools/compare-grep-sort.sh)
- Handles invalid UTF-8 (prints whatever is found)
Limitations
- Only handles UTF-8 or ASCII encoded log files.
- Cannot handle multi-file
.gz files (multiple "streams")
(TODO describe problem)
- Cannot handle multi-file
.xz files (chooses first file found)
(TODO describe problem)
- Cannot process archive or compressed files within other archive or compressed
files.
e.g. a
.tar file within another .tar file will not be processed, a .gz
file within a .tar file will not be processed.
Hacks
Further Reading
