Update readme

README.md

@@ -4,98 +4,159 @@
 [![Hex.pm](https://img.shields.io/hexpm/v/ex_cmd.svg)](https://hex.pm/packages/ex_cmd)
 [![docs](https://img.shields.io/badge/docs-hexpm-blue.svg)](https://hexdocs.pm/ex_cmd/)
 
+ExCmd is an Elixir library for running and communicating with external programs using a back-pressure mechanism. It provides a robust alternative to Elixir's built-in [Port](https://hexdocs.pm/elixir/Port.html) with improved memory management through demand-driven I/O.
 
-ExCmd is an Elixir library to run and communicate with external
-programs with back-pressure mechanism. It makes use os backed stdio
-buffer for this.
+## The Port I/O Problem
 
-Communication with external program using
-[Port](https://hexdocs.pm/elixir/Port.html) is not demand driven. So
-it is easy to run into memory issues when the size of the data we are
-writing or reading from the external program is large. ExCmd tries to
-solve this problem by making better use of os backed stdio buffers
-and providing demand-driven interface to write and read from external
-program. It can be used to stream data through an external
-program. For example, streaming a video through `ffmpeg` to serve a
-web request.
+When using Elixir's built-in [Port](https://hexdocs.pm/elixir/Port.html), running external programs that generate large amounts of output (like streaming video using `ffmpeg`) can quickly lead to memory issues. This happens because Port I/O is not demand-driven - it consumes output from stdout as soon as it's available and sends it to the process mailbox. Since BEAM process mailboxes are unbounded, the output accumulates there waiting to be received.
 
-Getting audio out of a video stream is as simple as
+### Memory Usage Comparison
 
-``` elixir
-ExCmd.stream!(~w(ffmpeg -i pipe:0 -f mp3 pipe:1), input: File.stream!("music_video.mkv", [], 65336))
-|> Stream.into(File.stream!("music.mp3"))
-|> Stream.run()
+Let's look at how ExCmd handles memory compared to Port when processing large streams:
+
+Using Port (memory grows unbounded):
+```elixir
+Port.open({:spawn_executable, "/bin/cat"}, [{:args, ["/dev/random"]}, {:line, 10}, :binary, :use_stdio])
 ```
 
-### Major Features
+![Port memory consumption](./images/port.png)
+
+Using ExCmd (memory remains stable):
+```elixir
+ExCmd.stream!(~w(cat /dev/random))
+|> Enum.each(fn data ->
+  IO.puts(IO.iodata_length(data))
+end)
+```
+
+![ExCmd memory consumption](./images/ex_cmd.png)
+
+ExCmd solves this by implementing:
+- Demand-driven I/O with proper back-pressure
+- Efficient use of OS-backed stdio buffers
+- Stream-based API that integrates with Elixir's ecosystem
+
+## Key Features
+
+- **Back-pressure Support**: Controls data flow between your application and external programs
+- **Stream Abstraction**: Seamless integration with Elixir's Stream API
+- **Memory Efficient**: Demand-driven I/O prevents memory issues with large data transfers
+- **Cross-platform**: Pre-built binaries for MacOS, Windows, and Linux
+- **Process Management**: Proper program termination with no zombie processes
+- **Selective I/O Control**: Ability to close stdin while keeping stdout open
+- **No Dependencies**: No separate middleware or shim installation required
+
+## Installation
+
+Add `ex_cmd` to your list of dependencies in `mix.exs`:
 
-* Unlike beam ports, ExCmd puts back pressure on the external program
-* Stream abstraction
-* No separate shim installation required
-* Ships pre-built binaries for MacOS, Windows, Linux
-* Proper program termination. No more zombie process
-* Ability to close stdin and wait for output (with ports one can not selectively close stdin)
+```elixir
+def deps do
+  [
+    {:ex_cmd, "~> x.x.x"}
+  ]
+end
+```
 
+## Quick Start Examples
 
-## Examples
+### Basic Command Execution
 
 ```elixir
+# Simple command execution
+ExCmd.stream!(~w(echo Hello))
+|> Enum.into("")
+# => "Hello\n"
+
+# Get your IP address
 ExCmd.stream!(~w(curl ifconfig.co))
 |> Enum.into("")
 ```
 
-Binary as input
+### Working with Input Streams
+
+```elixir
+# String input
+ExCmd.stream!(~w(cat), input: "Hello World")
+|> Enum.into("")
+# => "Hello World"
+
+# List of strings
+ExCmd.stream!(~w(cat), input: ["Hello", " ", "World"])
+|> Enum.into("")
+# => "Hello World"
 
-  ```elixir
-  ExCmd.stream!(~w(cat), input: "Hello World")
-  |> Enum.into("")
-  # => "Hello World"
-  ```
+# Binary data
+ExCmd.stream!(~w(base64), input: <<1, 2, 3, 4, 5>>)
+|> Enum.into("")
+# => "AQIDBAU=\n"
 
-  ```elixir
-  ExCmd.stream!(~w(base64), input: <<1, 2, 3, 4, 5>>)
-  |> Enum.into("")
-  # => "AQIDBAU=\n"
-  ```
+# IOData
+ExCmd.stream!(~w(base64), input: [<<1, 2>>, [3], [<<4, 5>>]])
+|> Enum.into("")
+# => "AQIDBAU=\n"
+```
 
-List of binary as input
+### Media Processing Examples
 
-  ```elixir
-  ExCmd.stream!(~w(cat), input: ["Hello ", "World"])
-  |> Enum.into("")
-  # => "Hello World"
-  ```
+```elixir
+# Extract audio from video with controlled memory usage
+ExCmd.stream!(~w(ffmpeg -i pipe:0 -f mp3 pipe:1),
+  input: File.stream!("music_video.mkv", [], 65536))
+|> Stream.into(File.stream!("music.mp3"))
+|> Stream.run()
+
+# Process video streams efficiently
+ExCmd.stream!(~w(ffmpeg -i pipe:0 -c:v libx264 -f mp4 pipe:1),
+  input: File.stream!("input.mp4", [], 65536),
+  max_chunk_size: 65536)
+|> Stream.into(File.stream!("output.mp4"))
+|> Stream.run()
+```
 
-iodata as input
+### Error Handling
 
-  ```elixir
-  ExCmd.stream!(~w(base64), input: [<<1, 2,>>, [3], [<<4, 5>>]])
-  |> Enum.into("")
-  # => "AQIDBAU=\n"
-  ```
+```elixir
+# stream!/2 raises on non-zero exit status
+ExCmd.stream!(["sh", "-c", "exit 10"])
+|> Enum.to_list()
+# => ** (ExCmd.Stream.AbnormalExit) program exited with exit status: 10
+
+# stream/2 returns exit status as last element
+ExCmd.stream(["sh", "-c", "echo 'foo' && exit 10"])
+|> Enum.to_list()
+# => ["foo\n", {:exit, {:status, 10}}]
+```
 
-If you want pipes and globs, you can spawn shell process and pass your
-pipeline as argument
+### Advanced Features
 
 ```elixir
-cmd = "echo 'foo bar' | base64"
-ExCmd.stream!(["sh", "-c", cmd])
+# Redirect stderr to stdout
+ExCmd.stream!(["sh", "-c", "echo foo; echo bar >&2"],
+  stderr: :redirect_to_stdout)
 |> Enum.into("")
-# => "Zm9vIGJhcgo=\n"
+# => "foo\nbar\n"
 ```
 
-Read [stream documentation](file:///Users/akash/repo/elixir/ex_cmd/doc/ExCmd.html#stream!/2) for information
-about parameters.
+## Why Choose ExCmd?
 
-**Check out [Exile](https://github.com/akash-akya/exile) which is an
-alternative solution based on NIF without middleware overhead**
+ExCmd is designed to handle common challenges when working with external programs:
 
-## Installation
+1. **Memory Safety**: Unlike Port, ExCmd prevents memory bloat through demand-driven I/O
+2. **Back-pressure**: Built-in flow control prevents overwhelming either side
+3. **Pipe Control**: Supports selective pipe closure (e.g., closing stdin while reading stdout)
+4. **Process Management**: Ensures proper cleanup and no zombie processes
+5. **Simple Integration**: No additional middleware or setup required
 
-```elixir
-def deps do
-  [
-    {:ex_cmd, "~> x.x.x"}
-  ]
-end
-```
+## Alternatives
+
+- For NIF-based solutions without middleware overhead, consider [Exile](https://github.com/akash-akya/exile)
+- For simple command execution without streaming, Elixir's built-in Port might be sufficient
+
+## Documentation
+
+Detailed documentation is available at [HexDocs](https://hexdocs.pm/ex_cmd/).
+
+## License
+
+See [LICENSE](LICENSE) file for details.