on - run commands on file event

On top of the Ulriken near Bergen in Norway from my last rl journey.

Inspired from watch, on runs commands repeatedly but on file events likewise to inotifywait.

Contents

• Getting started
• Origin story
  • Rustlings
  • Watch go lint
  • Random go lint output
  • File events
• Implementation
  • Fsnotify
  • Debounce events

Getting started

usage: on [--create] [--write] [--rename] [--remove] [--chmod] <file> <cmd...>

$ on ./main.go date
Tue Nov  5 12:27:53 PM CET 2024
Tue Nov  5 12:27:53 PM CET 2024

$ on --write ./main.go git diff
diff --git a/main.go b/main.go
index e400b66..baa2c16 100644
--- a/main.go
+++ b/main.go
@@ -17,8 +17,7 @@ var (
    remove = flag.Bool("remove", false, "React on remove")
    chmod  = flag.Bool("chmod", false, "React on chmod")

-   verbose = flag.Bool("verbose", false, "Print debug information")
-
+   verbose     = flag.Bool("verbose", false, "Print debug information")
    listenToAll = false
 )

$ on --verbose ./main.go true
ops: []fsnotify.Op{}
received REMOVE        "./main.go"
received CREATE        "./main.go"
received WRITE         "./main.go"
received CHMOD         "./main.go"
received RENAME        "./main.go"
received CREATE        "./main.go"
received WRITE         "./main.go"
received WRITE         "./main.go"
received CHMOD         "./main.go"

Origin story

Rustlings

Rustling is a series of exercise the rust programming language, I completed a while ago. Their approach to run the test suite and give immediate feedback was to just run the suite periodically, shortening the feedback loop and giving low-latency feedback. I underestimated the productivity of short feedback loops like this.

Watch go lint

Random go lint output

File events

Rather than periodically run the linter, I wanted to run it only when the file had changed. Luckily fsnotify provides a watcher to programatically react on file events like create, write, rename and even chmod¹.

With this approach I can get a fresh linter output by saving my code file.

Implementation

Fsnotify

Using fsnotify, the following examples show the use of the file event watcher. It typically watches on directories rather than single files, like on does it with some additional filtering for convenience.

package main

import (
  "fmt"
  "os"

  "github.com/fsnotify/fsnotify"
)

func main() {
  // first argument is the directory,
  // the watcher give us events for
  if len(os.Args) != 2 {
    fmt.Println("usage: watch <dir>")
    return
  }
  defer watcher.Close()

  watcher, err := fsnotify.NewWatcher()
  if err != nil {
    fmt.Println(err)
    return
  }

  // register the watcher
  if err := watcher.Add(os.Args[1]); err != nil {
    fmt.Println(err)
    return
  }
  defer watcher.Remove(os.Args[1])

  // the watcher provides several channels to listen to
  for {
    select {
    // whenver a file event happened,
    // the watcher sends it to the Events channel
    case event, ok := <-watcher.Events:
      if !ok {
        return
      }
      fmt.Println(event)
    // on error the watcher sends it to the Errors channel
    case err, ok := <-watcher.Errors:
      if !ok {
        return
      }
      fmt.Println("error:", err)
    }
  }
}

Which outputs something like the following. The watch command is sent to background in order to print events simultaneously within one terminal.

$ ./watch /tmp &
[1] 1713
$ touch /tmp/test
CREATE        "/tmp/test"
CHMOD         "/tmp/test"
$ echo 123 > /tmp/test
WRITE         "/tmp/test"
$ rm /tmp/test
REMOVE        "/tmp/test"
$ touch /tmp/test
CREATE        "/tmp/test"
CHMOD         "/tmp/test"
$ mv /tmp/test /tmp/test2
RENAME        "/tmp/test"
CREATE        "/tmp/test2" ← "/tmp/test"
$ fg
./watch .
^C

Debounce events

$ ./watch . &
$ dd if=/dev/zero of=test bs=1k count=100 status=progress
WRITE         "./test"
WRITE         "./test"
WRITE         "./test"
WRITE         "./test"
WRITE         "./test"
WRITE         "./test"
WRITE         "./test"
WRITE         "./test"
WRITE         "./test"
WRITE         "./test"
WRITE         "./test"
WRITE         "./test"
100+0 records in
100+0 records out
102400 bytes (102 kB, 100 KiB) copied, 0.000620291 s, 165 MB/s

We can make use of the debounce pattern, as described in the book Cloud Native Go. Frequent events get limit within a specified time frame into a single one as shown in the following diagram.

This concept is implemented in the event loop of on as shown here:

// create new thread-safe map
var debounceMap = sync.Map{}
...
for {
  select {
  case event, _ := <-watcher.Events:
    ...
    // check if event already occured lately and was stored to the map
    // use the file name (event.name) as key
    if t, found := debounceMap.Load(event.Name); found {
      // if there already was an event lately, reset its timer
      timer, ok := t.(*time.Timer)
      timer.Reset(*debounceTimeout)
      continue
    }

    // If the event occured the first time within the current timeframe, just
    // add it with a new timer as value to the map.
    // The provided function gets called when the timer ran out.
    debounceMap.Store(event.Name, time.AfterFunc(*debounceTimeout, func() {
      // remove the event again to end this debounce time frame
      // defer ensures that the command finished before the next timeframe starts 
      defer debounceMap.Delete(event.Name)

      // just run it
      run(command, args)
    }))

  case err, ok := <-watcher.Errors:
  ...
  }
}