The Molybdenum Replacer

Recursive search and replace CLI application.

Installation

Directly via cargo

• Install rust, this should also install cargo
• Run cargo install molybdenum
• Verify success: mo -h should print its help and version

From source code

• Install rust, this should also install cargo
• Clone the repo: git clone https://github.com/gfannes/molybdenum
• Build and install the app: cargo install --path molybdenum
• Verify success: mo -h should print its help and version

Basic usage

Following commands demonstrate how mo can be used to accomplish different tasks:

• Create list of filenames, only filtering against the pathname itself:
  • mo: When no search pattern is specified, only the filenames are listed
  • mo -l: Explicitly ask to output only the filenames
  • mo -C FOLDER: Use FOLDER as root for searching
  • mo -e hpp -e cpp: Only take files with hpp and cpp extension into account
  • mo -f PART: Keep filenames that match against PART
  • mo -F PART: Keep filenames that do not match against PART
  • mo -0: Use 0x00 to separate filenames. This is handy when using the output with xargs.
  • mo -u -U -a: Take hidden files, folders and binary files into account as well
• Search for a given regex pattern:
  • mo PATTERN: Search for PATTERN in files recursively
  • mo -p PATTERN: Search for PATTERN in files recursively
  • mo -w PATTERN: Search for PATTERN in files recursively, adding word-boundary constraints arround PATTERN
  • mo -s PATTERN: Search for PATTERN, case-sensitive
  • mo -B 10 -A 10 PATTERN: Output a context of 10 additional lines before and after each match
• Replace matches with a given STRING:
  • mo -r naald -n -w needle: Simulate the replacement of the the word needle with the Dutch word naald
  • mo -r naald -w needle: Really replace the word needle with the Dutch word naald

Next to this, mo detects if input comes from a console or redirection, and will act accordingly, as well as for its output: mo can be used to report or make replacements in a piped stream as well.

Yet another tool?

Powerful search can be found without problems, eg, grep, ack, ag, ripgrep or broot.

Tools for replacing recursively in a folder are more difficult to find, although some exist: fart-it. Typically, people use a combination of searching, xargs and a replacement tool like sed or rpl.

I use code searching a lot to investigate a large source code base before attempting a replace. Even with 100k files, search is fast and fairly easy. Recursively replacing text is much more dangerous, especially if it requires the combination of several less frequently used tools; it's difficult to remember a search-xargs-replace combination if not used on a daily basis. On top of this, the search tool used to filter the set of files and perform a dry-run, is not per-se using the same search query as the replace tool. After all, these are different tools. It would be better if a single tool could be used for every-day searching and replacing. This is exactly what The Molybdenum Replacer intends to achieve.

In addition, I find it difficult to use ag and rg to filter against the file path and search for content in the remaining files; the -g option for ag and rg is confusing in my opinion. As such, I would expect ag -g /auro/ -w test to search for the word test in all files that have /auro/ in their path, but that is not what actually happens. It filters with /auro/ and test against the filename (or something that looks like it).

The real reason, of course, is that I had some free time and was looking for a nice project/excuse to learn rust.

Implemented Features

Following features are implemented and usable in the current version:

• Recursive search in a nested folder structure, line-based
  • Fast enough to make it usable. My impression is that it's performance is between ag and ripgrep.
  • Output clear enough for a human to understand when output is a console
• Recursive replacement in a nested folder structure
• Search and replacement from an input stream
  • Output all input when output is redirected
• Support for non-UTF8 filename and content
• Crude detection of binary files

Future Features

Following features might be added sooner or later:

• Improved performance
  • mo is currently single-threaded. To achieve ripgrep-like performance, all CPU's are probably required.
  • When -l is used to only output filenames, mo can stop searching after the first match.
• Use a better name, I just picked something that was still available.
  • Nobody can remember molybdenum, and it is very hard to type. mo is better, but difficult to search on the internet.
• Support for using regex capture groups during replacement
• Better detection of binary files
• Support for filtering against .gitignore files
• Rollback when writing replacement file fails
• Improved testing
  • More and better unit tests
  • Acceptance tests for all common use cases
  • Performance tests and some reporting around this
• Binary releases on github
• Improved color scheme for people using a light background theme
• Support for file type sets
• Allow options without argument to be merged
• Allow input separator to be set explicitly. Now, this is hardcoded 0x0a.

Performance

Count all files in a folder

`` Scenario: count all files Runningmo -l -u -U -a | wc` Elapsed time: 0:00.29 Output: 184675 187146 16959297

Running rg --files -uu -a | wc Elapsed time: 0:00.15 Output: 184675 187146 16589947

Running ag -l -uu -a | wc Elapsed time: 0:00.66 Output: 184675 187146 16589947 ```

Search for word in C++ source code

`` Scenario: search for wordtestin .cpp files in subfoldercorewhere path contains /auro/ Runningmo -C core -e cpp -f /auro/ -w -p test -l | wc` Elapsed time: 0:00.03 Output: 165 165 9008

Running rg -t cpp --files core | rg /auro/ | rg '.cpp$' | tr '\n' '\0' | xargs -0 rg -i -w test -l | wc Elapsed time: 0:00.01 Output: 165 165 9008

Running ag --cpp -l . core | ag /auro/ | ag '.cpp$' | tr '\n' '\0' | xargs -0 ag -w test -l | wc Elapsed time: 0:00.05 Output: 165 165 9008

```

I don't know how I can accomplish this scenario with ag and rg in a single command without relying on xargs and tr.