< up
2024-11-06

on - run commands on file event

Inspired from watch, on runs commands repeatedly but on file events likewise to inotifywait.

Contents

Getting started

usage:

usage: on [--create] [--write] [--rename] [--remove] [--chmod] <file> <cmd...>

Listen to all events for a file and print date for each event:

$ on ./main.go date
Tue Nov  5 12:27:53 PM CET 2024
Tue Nov  5 12:27:53 PM CET 2024

Run git diff for any write event:

$ on --write ./main.go git diff
diff --git a/main.go b/main.go
index e400b66..baa2c16 100644
--- a/main.go
+++ b/main.go
@@ -17,8 +17,7 @@ var (
    remove = flag.Bool("remove", false, "React on remove")
    chmod  = flag.Bool("chmod", false, "React on chmod")

-   verbose = flag.Bool("verbose", false, "Print debug information")
-
+   verbose     = flag.Bool("verbose", false, "Print debug information")
    listenToAll = false
 )

Debug output with all events:

$ on --verbose ./main.go true
ops: []fsnotify.Op{}
received REMOVE        "./main.go"
received CREATE        "./main.go"
received WRITE         "./main.go"
received CHMOD         "./main.go"
received RENAME        "./main.go"
received CREATE        "./main.go"
received WRITE         "./main.go"
received WRITE         "./main.go"
received CHMOD         "./main.go"

Origin story

Rustlings

Rustling is a rust exercise series, I did a while ago. Their approach to run the test suite and give immediate feedback was to just run the suite periodically, shortening the feedback loop and giving low-latency feedback. I underestimated the productivity of short feedback loops like this.

Watch go lint

I adapted the watch pattern for fixing go linter issues and executed something like watch golangci-lint run.

Random go lint output

While rustlings output is ordered, golangci-lint’s output is randomly sorted. I guess all the linter run in parallel and thus no output order is guaranteed.

File events

Rather than periodically run the linter, I wanted to only run it when the file actually changed. Luckily fsnotify provides a watcher to programatically react on file events like create, write, rename and even chmod1.

With this approach I can get a fresh linter output by saving my code file.

Implementation

Fsnotify

Using fsnotify, the following examples show the use of the file event watcher. It typically watches on directories rather than single files, like on does it with some additional filtering for convenience.

package main

import (
  "fmt"
  "os"

  "github.com/fsnotify/fsnotify"
)

func main() {
  // first argument is the directory the watcher give us events for
  if len(os.Args) != 2 {
    fmt.Println("usage: watch <dir>")
    return
  }
  defer watcher.Close()

  watcher, err := fsnotify.NewWatcher()
  if err != nil {
    fmt.Println(err)
    return
  }

  // register the watcher
  if err := watcher.Add(os.Args[1]); err != nil {
    fmt.Println(err)
    return
  }
  defer watcher.Remove(os.Args[1])

  // the watcher provides several channels to listen to
  for {
    select {
    // whenver a file event happened, the watcher sends it to the Events channel
    case event, ok := <-watcher.Events:
      if !ok {
        return
      }
      fmt.Println(event)
    // on error the watcher sends it to the Errors channel
    case err, ok := <-watcher.Errors:
      if !ok {
        return
      }
      fmt.Println("error:", err)
    }
  }
}

Which outputs something like the following. The watch command is sent to background in order to do some file operations while it prints the events simultaneously within one terminal.

$ ./watch /tmp &
[1] 1713
$ touch /tmp/test
CREATE        "/tmp/test"
CHMOD         "/tmp/test"
$ echo 123 > /tmp/test
WRITE         "/tmp/test"
$ rm /tmp/test
REMOVE        "/tmp/test"
$ touch /tmp/test
CREATE        "/tmp/test"
CHMOD         "/tmp/test"
$ mv /tmp/test /tmp/test2
RENAME        "/tmp/test"
CREATE        "/tmp/test2""/tmp/test"
$ fg
./watch .
^C

Debounce events

When saving a file, there is not necessarily only one event. The count grows with the file size while writing the content to disk gets fragmented. In the following snippet 100kib trigger a bunch of write events:

$ ./watch . &
$ dd if=/dev/zero of=test bs=1k count=100 status=progress
WRITE         "./test"
WRITE         "./test"
WRITE         "./test"
WRITE         "./test"
WRITE         "./test"
WRITE         "./test"
WRITE         "./test"
WRITE         "./test"
WRITE         "./test"
WRITE         "./test"
WRITE         "./test"
WRITE         "./test"
100+0 records in
100+0 records out
102400 bytes (102 kB, 100 KiB) copied, 0.000620291 s, 165 MB/s

I want my linter to run only once per file save. Those events need to be squashed in a way that each save triggers its own command run.

Debounce pattern from the book Cloud Native Go to the rescue! Frequent events get limit within a specified time frame into a single one as shown in the following diagram.

This concept is used within the event loop of on like this, leaving out all unrelated lines:

// create new thread-safe map
var debounceMap = sync.Map{}
...
for {
  select {
  case event, _ := <-watcher.Events:
    ...
    // check if event already occured lately and was stored to the map
    // use the file name (event.name) as key
    if t, found := debounceMap.Load(event.Name); found {
      // if there already was an event lately, reset its timer
      timer, ok := t.(*time.Timer)
      timer.Reset(*debounceTimeout)
      continue
    }

    // If the event occured the first time within the current timeframe, just
    // add it with a new timer as value to the map.
    // The provided function gets called when the timer ran out.
    debounceMap.Store(event.Name, time.AfterFunc(*debounceTimeout, func() {
      // remove the event again to end this debounce time frame
      // defer ensures that the command finished before the next timeframe starts 
      defer debounceMap.Delete(event.Name)

      // just run it
      run(command, args)
    }))

  case err, ok := <-watcher.Errors:
  ...
  }
}

  1. Based on inotify