Super Speedy Syslog Searcher! (s4)
Speedily search and sort many syslog files by datetime.
Super Speedy Syslog Searcher (s4) is a command-line tool to search
and sort syslog files within compressed files (.gz, .xz) and
archives (.tar, .zip).
The first goal of s4 is speedy searching and printing.
Use
Install super_speedy_syslog_searcher
text
cargo install super_speedy_syslog_searcher
Run s4
For example, print all the messages under /var/log/.
text
s4 /var/log
--help
```text
Super Speedy Syslog Searcher will search syslog files and sort entries by datetime. DateTime filters
may be passed to narrow the search. It aims to be very fast.
USAGE:
s4 [OPTIONS] ...
ARGS:
... Path(s) of syslog files or directories. Directories will be recursed,
remaining on the same filesystem. Symlinks will be followed
OPTIONS:
-a, --dt-after DateTime After filter - print syslog lines with a datetime that
is at or after this datetime. For example, '20200102T123000'
-b, --dt-before DateTime Before filter - print syslog lines with a datetime that
is at or before this datetime. For example, '20200102T123001'
-t, --tz-offset DateTime Timezone offset - for syslines with a datetime that does
not include a timezone, this will be used. For example, '-0800'
'+02:00' (with or without ':'). If passing a value with leading
'-', use the '=' to explicitly set the argument, e.g. '-t=-0800'.
Otherwise the CLI argument parsing will fail. Default is local
system timezone offset. [default: -08:00]
-u, --prepend-utc Prepend DateTime in the UTC Timezone for every line
-l, --prepend-local Prepend DateTime in the Local Timezone for every line
-n, --prepend-filename Prepend file basename to every line
-p, --prepend-filepath Prepend file full path to every line
-w, --prepend-file-align Align column widths of prepended data
-c, --color Choose to print to terminal using colors [default: auto]
[possible values: always, auto, never]
-z, --blocksz Read blocks of this size. May pass decimal or hexadecimal
numbers. Using the default value is recommended [default: 65535]
-s, --summary Print a summary of files processed. Printed to stderr
-h, --help Print help information
-V, --version Print version information
DateTime Filter patterns may be:
"%Y%m%dT%H%M%S"
"%Y%m%dT%H%M%S%z"
"%Y-%m-%d %H:%M:%S"
"%Y-%m-%d %H:%M:%S %z"
"%Y-%m-%dT%H:%M:%S"
"%Y-%m-%dT%H:%M:%S %z"
"%Y/%m/%d %H:%M:%S"
"%Y/%m/%d %H:%M:%S %z"
"%Y%m%d"
"%Y%m%d %z"
"+%s"
Without a timezone offset (%z or %Z), the Datetime Filter is presumed to be the
system timezone.
Pattern "+%s" is Unix epoch timestamp in seconds with a preceding "+".
Ambiguous timezones will be rejected, e.g. "SST".
Prepended datetime, -u or -l, is printed in format "%Y%m%dT%H%M%S%.6f %z:".
DateTime formatting is described at https://docs.rs/chrono/latest/chrono/format/strftime/
DateTimes supported are only of the Gregorian calendar.
DateTimes supported language is English.
```
About
Features
- Prepends datetime and file paths, for easy programmatic parsing or visual
traversal of varying syslog messages
- Supports many varying datetime formats including (WHAT ARE THOSE OFFICIAL ONES?)
- Tested against "in the wild" log files from varying Linux distributions
(see
./logs/)
- Comparable speed as GNU
grep and sort
(see ./tools/compare-grep-sort.sh)
- Handles invalid UTF-8 (prints whatever is found)
Limitations
- Only handles UTF-8 or ASCII encoded log files.
- Cannot handle multi-file
.gz files (multiple "streams")
(TODO describe problem)
- Cannot handle multi-file
.xz files (chooses first file found)
(TODO describe problem)
- Cannot process archive or compressed files within other archive or compressed
files.
e.g. a
.tar file within another .tar file will not be processed, a .gz
file within a .tar file will not be processed.
Hacks
Further Reading
