Skip to content

Commit

Permalink
feat: Synchronous Disk State Check
Browse files Browse the repository at this point in the history 
Some negatives with the current approach (evented FS watcher)

1. Build loops are possible
   If you forget to ignore something, its possible to get stuck in a
   build loop, where the command outputs a file, which triggers a
   rebuild, which outputs a file, which triggers...

2. CPU usage
   Evented watchers aren't free. Larger the project, the more %CPU it
   takes to just watch the FS

3. Wasted work
   After everysave, a build is triggered - which isn't ideal as in most
   cases one will save more than 1 file before tabbing over to the
   browser. This leads to more wasted CPU spent rerunning commands on
   every save.

New approach: Check FS on HTTP request.

Flipped the sequence. Previously:
-> FS Change
-> Pause Proxy
-> Rebuild
-> Unpause Proxy

New sequence:

-> Request comes in
-> Pause Proxy
-> Check for FS changes
-> If changes exist, rebuild
-> Unpause Proxy

This has several advantages:

1. 0% CPU usage while idle
2. No wasted builds

The obvious downside is that now each request needs to check the FS for
changes.

However, with these changes it is possible to chain multiple `tychus`'s
together. When using that feature, a `--wait` flag has been added. Using
that flag will tell tychus to block until the user specified command
finishes. This command will most often be some script or quick series of
commands and not something blocking like a webserver.
  • Loading branch information
@PatKoperwas
PatKoperwas committed Feb 2, 2018
1 parent fc07742 commit 9385bec
Show file tree
Hide file tree
Showing 9 changed files with 370 additions and 306 deletions.
38 changes: 1 addition & 37 deletions Gopkg.lock
View file

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

166 changes: 81 additions & 85 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,15 +1,12 @@
tychus
========

`tychus` is a command line utility to live-reload your application. `tychus`
will watch your filesystem for changes and automatically recompile and restart
code on change.
Tychus is a command line utility for live reloading applications. Tychus serves
your application through a proxy. Anytime the proxy receives an HTTP request, it
will automatically rerun your command if the filesystem has changed.

`tychus` is language agnostic - it can be configured to work with just about
anything: Go, Rust, Ruby, Python, etc.

Should you desire you can use `tychus` as a proxy to your application. It will
pause requests while your application rebuilds & restarts.
anything: Go, Rust, Ruby, Python, scripts & arbitrary commands.


## Installation
Expand All @@ -33,137 +30,136 @@ Currently isn't supported :(

## Usage

Usage is simple, `tychus` and then your command. On a filesystem change that
Usage is simple, `tychus` and then your command. That will start a proxy on port
`4000`. When an HTTP request comes in and the filesystem has changed, your
command will be rerun.

```
// Go
tychus go run main.go
// Rust
tychus cargo run
// Ruby
tychus ruby myapp.rb
// Shell Commands
tychus ls
```

Need to pass flags? Stick the command in quotes

```
tychus "ruby myapp.rb -e development"
```

Complicated command? Stick it in quotes

```
tychus "go build -o my-bin && echo 'Built Binary' && ./my-bin"
```


## Options
Tychus has a few options. In most cases the defaults should be sufficient. See
below for a few examples.

```yaml
-a, --app-port int port your application runs on, overwritten by ENV['PORT'] (default 3000)
-p, --proxy-port int proxy port (default 4000)
-w, --watch stringSlice comma separated list of extensions that will trigger a reload. If not set, will reload on any file change.
-x, --ignore stringSlice comma separated list of directories to ignore file changes in. (default [node_modules,log,tmp,vendor])
--wait Wait for command to finish before proxying a request.
-t, --timeout int timeout for proxied requests (default 10)


-h, --help help for tychus
--debug print debug output
--no-proxy will not start proxy if set
--version version for tychus
```

Note: If you do not specify any file extensions in `--watch`, Tychus will
trigger a reload on any file change, except for files inside directories listed
in `--ignore`

Note: Tychus will not watch any hidden directories (those beginning with `.`).
Note: Tychus will not look for file system changes any hidden directories
(those beginning with `.`).

## Examples

### Sinatra
By default, Sinatra runs on port `4567`. Only want to watch `ruby` and
`erb` files. Default ignore list is sufficient. The following are equivalent.
**Example: Web Servers**

```
tychus ruby myapp.rb -w .rb,.erb -a 4567
tychus ruby myapp.rb --watch=.rb,.erb --app-port=4567
// Go - Hello World Server
$ tychus go run main.go
[tychus] Proxing requests on port 4000 to 3000
[Go App] App Starting
// Make a request
$ curl localhost:4000
Hello World
$ curl localhost:4000
Hello World
// Save a file, next request will restart your webapp
$ curl localhost:4000
[Go App] App Starting
Hello World
```

Visit http://localhost:4000 (4000 is the default proxy host) and to view your
app.
This can work with any webserver:

```
// Rust
tychus cargo run
### Foreman / Procfile
Similar to the previous example, except this time running inside of
[foreman](https://github.com/ddollar/foreman) (or someother Procfile runner).
// Ruby
tychus ruby myapp.rb
```

Need to pass flags? Stick the command in quotes

```
# Procfile
web: tychus "rackup -p $PORT -s puma" -w rb,erb
tychus "ruby myapp.rb -e development"
```

Note: If you need to pass flags to your command (like `-p` & `-s` in this case),
wrap your entire command in quotes.
Complicated command? Stick it in quotes

We don't need to explicitly add a `-a $PORT` flag, because `tychus` will
automatically pick up the $PORT and automatically set `app-port` for you.
```
tychus "go build -o my-bin && echo 'Built Binary' && ./my-bin"
```

**Example: Scripts + Commands**

### Kitchen Sink Example
Running a Go program, separate build and run steps, with some logging thrown in,
only watching `.go` files, running a server on port `5000`, running proxy on
`8080`, ignoring just `tmp` and `vendor`, with a timeout of 5 seconds.
Scenario: You have a webserver running on port `3005`, and it serves static
files from the `/public` directory. In the `/docs` folder are some markdown
files. Should they change, you want them rebuilt and placed into the `public`
directory so the server can pick them up.

```
tychus "echo 'Building...' && go build -o tmp/my-bin && echo 'Built' && ./tmp/my-bin some args -e development" --app-port=5000 --proxy-port=8080 --watch=.go --ignore=tmp,vendor --timeout=5
tychus "multimarkdown docs/index.md > public/index.html" --wait --app-port=3005
```

# Or, using short flags
Now, when you make a request to the proxy on `localhost:4000`, `tychus` will
pause the request (that's what the `--wait` flag is for) until `multimarkdown`
finishes. Then it will forward the request to the server on port `3005`.
`multimarkdown` will only be run if the filesystem has changed.

tychus "echo 'Building...' && go build -o tmp/my-bin && echo 'Built' && ./tmp/my-bin some args -e development" -a 5000 -p 8080 -w .go -x tmp,vendor -t 5
```

## Whats the point of the proxy?
Consider the following situations:
**Advanced Example: Reload Scripts and Webserver**

**1. There is a gap between program starting and server accepting requests.**
Like the scenario above, but you also want your server to autoreload as files
change. You can chain `tychus` together, by setting the `app-port` equal to the
`proxy-port` of the previous `tychus`. An example:

```ruby
# myapp.rb
sleep 5
require "sinatra"
The first instance of `tychus` will run a Go webserver that serves assets out of
`public/`. We only want it to restart when the `app` folder changes, so ignore
`docs` and `public` directories.

get "/"
"Hello World"
end
```
$ tychus go run main.go --app-port=3000 --proxy-port=4000 --ignore=docs,public
After your application restarts, any requests that get sent to it within 5
seconds will return an error / show you the "Site can't be reached page".
[tychus] Proxing requests on port 4000 to 3000
...
...
```

Really puts a damper on the save, alt+tab, refresh workflow.
In order to serve upto date docs, `multimarkdown` needs to be invoked to
transform markdown into servable html. So we start another `tychus` process to
and point its app-port to server's proxy port.

By going through the proxy, when you hit refresh, your request will wait until
the server is actually ready to accept and send you back a response. So save,
alt+tab to browser hit refresh. Page will wait the 5 seconds until the server is
ready. Then it will forward the request.
```
$ tychus "multimarkdown docs/index.md > public/index.html" --wait --app-port=4000 --proxy-port=4001
```

**2. Your code has a compile step.**
Now, there is a proxy running on `4001` pointing at a proxy on `4000` pointing
at a webserver on `3000`. If you save `docs/index.html`, and then make a request
to `localhost:4001`, that will pause the request while `multimarkdown` runs.
Once it is finished, the requests gets forwarded to `localhost:4000`, which in
turn forwards it our websever on `3000`. The request gets sent all the way back,
with the correctly updated html!

While your code is still compiling you alt+tab to the browser and hit refresh...
and you are potentially served old code. Avoid that by going through a proxy.
Had our server code been modified in the `app/` folder, then after
`multimarkdown` finished, and the request got passed on to `4000`, that would
have also triggered a restart of our websever.

**Other Proxy Goodies**

**Error messages**

If you make a syntax error, or your program won't build for some reason, the
output will be displayed in the webpage. Handy for the times you can't see you
server (its in another pane / tab / tmux split).
If you make a syntax error, or your program won't build for some reason, stderr
will be returned by the proxy. Handy for the times you can't see you server (its
in another pane / tab / tmux split).
43 changes: 13 additions & 30 deletions cmd/root.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,27 +20,15 @@ var ignored []string
var noProxy bool
var proxyPort int
var timeout int
var watch []string
var wait bool

var rootCmd = &cobra.Command{
Use: "tychus",
Short: "Starts and reloads your application as you make changes to source files.",
Long: `tychus is a command line utility to live-reload your application. tychus will
watch your filesystem for changes and automatically recompile and restart code
on change.
Example:
tychus go run main.go -w .go
tychus ruby myapp.rb --app-port=4567 --proxy-port=4000 --watch .rb,.erb --ignore node_modules
Example: No Proxy
tychus ls --no-proxy
Example: Flags - use quotes
tychus "ruby myapp.rb -p 5000 -e development" -a 5000 -p 4000 -w .rb,.erb
Example: Multiple Commands - use quotes
tychus "go build -o my-bin && echo 'Done Building' && ./my-bin"
Short: "Live reload utility + proxy",
Long: `Tychus is a command line utility for live reloading applications.
Tychus serves your application through a proxy. Anytime the proxy receives
an HTTP request will automatically rerun your command if the filesystem has
changed.
`,
Args: cobra.MinimumNArgs(1),
Run: func(cmd *cobra.Command, args []string) {
Expand All @@ -63,7 +51,7 @@ func init() {
rootCmd.Flags().BoolVar(&noProxy, "no-proxy", false, "will not start proxy if set")
rootCmd.Flags().IntVarP(&proxyPort, "proxy-port", "p", 4000, "proxy port")
rootCmd.Flags().IntVarP(&timeout, "timeout", "t", 10, "timeout for proxied requests")
rootCmd.Flags().StringSliceVarP(&watch, "watch", "w", []string{}, "comma separated list of extensions that will trigger a reload. If not set, will reload on any file change.")
rootCmd.Flags().BoolVar(&wait, "wait", false, "Wait for command to finish before proxying a request")
}

func start(args []string) {
Expand All @@ -79,16 +67,6 @@ func start(args []string) {
syscall.SIGQUIT,
)

// Clean up watched file extensions
for i, ext := range watch {
ext = strings.TrimSpace(ext)
if !strings.HasPrefix(ext, ".") {
ext = "." + ext
}

watch[i] = ext
}

// If PORT is set, use that instead of AppPort. For things like foreman
// where ports are automatically assigned.
envPort, ok := os.LookupEnv("PORT")
Expand All @@ -98,15 +76,20 @@ func start(args []string) {
}
}

// Clean up ignored dirs.
for i, dir := range ignored {
ignored[i] = strings.TrimRight(strings.TrimSpace(dir), "/")
}

// Create a configuration
c := &tychus.Configuration{
Extensions: watch,
Ignore: ignored,
ProxyEnabled: !noProxy,
ProxyPort: proxyPort,
AppPort: appPort,
Timeout: timeout,
Logger: tychus.NewLogger(debug),
Wait: wait,
}

// Run tychus
Expand Down
14 changes: 7 additions & 7 deletions tychus/configuration.go
View file
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
package tychus

type Configuration struct {
Extensions []string `yaml:"extensions"`
Ignore []string `yaml:"ignore"`
ProxyEnabled bool `yaml:"proxy_enabled"`
ProxyPort int `yaml:"proxy_port"`
AppPort int `yaml:"app_port"`
Timeout int `yaml:"timeout"`
Logger Logger `yaml:"-"`
AppPort int
Ignore []string
Logger Logger
ProxyEnabled bool
ProxyPort int
Timeout int
Wait bool
}
Loading

0 comments on commit 9385bec

Please sign in to comment.