Docs: Local install edits (#2220)

* Update README.md * Update README.md * Update README.md * Update tanka.md * content edits * Create install-for-manual-build.md * Update README.md * Update helm.md * Update local.md * Create get-logs-into-loki.md * Update local.md * Update get-logs-into-loki.md * Update get-logs-into-loki.md * Update local.md * Update get-logs-into-loki.md * Update README.md * Update README.md * Update installation.md * Update README.md * Update docs/getting-started/get-logs-into-loki.md Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com> * Update docs/installation/local.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * Update docs/installation/local.md Co-authored-by: Owen Diehl <ow.diehl@gmail.com> * applied edits * Update docs/getting-started/get-logs-into-loki.md Co-authored-by: Ed Welch <edward.welch@grafana.com> * Update get-logs-into-loki.md * Update docs/getting-started/get-logs-into-loki.md Co-authored-by: Ed Welch <edward.welch@grafana.com> * Update docs/getting-started/get-logs-into-loki.md Co-authored-by: Ed Welch <edward.welch@grafana.com> * Update docs/installation/README.md Co-authored-by: Ed Welch <edward.welch@grafana.com> * applied edits * Update local.md Co-authored-by: Cyril Tovena <cyril.tovena@gmail.com> Co-authored-by: Owen Diehl <ow.diehl@gmail.com> Co-authored-by: Ed Welch <edward.welch@grafana.com>
grafana · Jun 22, 2020 · 687662e · 687662e
1 parent 15378cf
commit 687662e
Show file tree

Hide file tree

Showing 10 changed files with 208 additions and 87 deletions.
diff --git a/docs/clients/README.md b/docs/clients/README.md
@@ -1,13 +1,13 @@
-# Loki Clients
+# Loki clients
 
 Loki supports the following official clients for sending logs:
 
-1. [Promtail](./promtail/README.md)
-2. [Docker Driver](./docker-driver/README.md)
-3. [Fluentd](./fluentd/README.md)
-4. [Fluent Bit](../../cmd/fluent-bit/README.md)
+- [Promtail](./promtail/README.md)
+- [Docker Driver](./docker-driver/README.md)
+- [Fluentd](./fluentd/README.md)
+- [Fluent Bit](../../cmd/fluent-bit/README.md)
 
-## Picking a Client
+## Picking a client
 
 While all clients can be used simultaneously to cover multiple use cases, which
 client is initially picked to send logs depends on your use case.
@@ -41,9 +41,9 @@ and you already have configured `Parser` and `Filter` plugins.
 Fluentd also works well for extracting metrics from logs when using its
 Prometheus plugin.
 
-# Unofficial Clients
+# Unofficial clients
 
-Please note that the Loki API is not stable yet and breaking changes may occur
+Please note that the Loki API is not stable yet, so breaking changes might occur
 when using or writing a third-party client.
 
 - [promtail-client](https://github.com/afiskon/promtail-client) (Go)

diff --git a/docs/clients/promtail/README.md b/docs/clients/promtail/README.md
@@ -6,14 +6,14 @@ deployed to every machine that has applications needed to be monitored.
 
 It primarily:
 
-1. Discovers targets
-2. Attaches labels to log streams
-3. Pushes them to the Loki instance.
+- Discovers targets
+- Attaches labels to log streams
+- Pushes them to the Loki instance.
 
 Currently, Promtail can tail logs from two sources: local log files and the
 systemd journal (on AMD64 machines only).
 
-## Log File Discovery
+## Log file discovery
 
 Before Promtail can ship any data from log files to Loki, it needs to find out
 information about its environment. Specifically, this means discovering
@@ -32,12 +32,12 @@ Just like Prometheus, `promtail` is configured using a `scrape_configs` stanza.
 drop, and the final metadata to attach to the log line. Refer to the docs for
 [configuring Promtail](configuration.md) for more details.
 
-## Receiving Logs From Syslog
+## Receiving logs From Syslog
 
 When the [Syslog Target](./scraping.md#syslog-target) is being used, logs
 can be written with the syslog protocol to the configured port.
 
-## Labeling and Parsing
+## Labeling and parsing
 
 During service discovery, metadata is determined (pod name, filename, etc.) that
 may be attached to the log line as a label for easier identification when

diff --git a/docs/clients/promtail/installation.md b/docs/clients/promtail/installation.md
@@ -1,4 +1,4 @@
-# Installing Promtail
+# Install Promtail
 
 Promtail is distributed as a [binary](#binary), [Docker container](#docker), and
 [Helm chart](#helm).
@@ -12,7 +12,7 @@ Every release includes binaries for Promtail which can be found on the
 
 ```bash
 # modify tag to most recent version
-$ docker pull grafana/promtail:1.5.0
+docker pull grafana/promtail:1.5.0
 ```
 
 ## Helm
@@ -23,13 +23,13 @@ Make sure that Helm is
 Then you can add Loki's chart repository to Helm:
 
 ```bash
-$ helm repo add loki https://grafana.github.io/loki/charts
+helm repo add loki https://grafana.github.io/loki/charts
 ```
 
 And the chart repository can be updated by running:
 
 ```bash
-$ helm repo update
+helm repo update
 ```
 
 Finally, Promtail can be deployed with:
@@ -40,7 +40,7 @@ $ helm upgrade --install promtail loki/promtail --set "loki.serviceName=loki"
 
 ## Kubernetes
 
-### DaemonSet (Recommended)
+### DaemonSet (recommended)
 
 A `DaemonSet` will deploy `promtail` on every node within a Kubernetes cluster.
 

diff --git a/docs/getting-started/get-logs-into-loki.md b/docs/getting-started/get-logs-into-loki.md
@@ -0,0 +1,74 @@
+# Get logs into Loki
+
+After you [install and run Loki](./installation/local.md), you probably want to get logs from other applications into it.
+
+To get application logs into Loki, you need to edit the [Promtail](./clients/promtail/README.md) config file.
+
+Detailed information about configuring Promtail is available in [Promtail configuration](./clients/promtail/configuration.md).
+
+The following instructions should help you get started.
+
+1. If you haven't already, download a Promtail configuration file. Keep track of where it is, because you will need to cite it when you run the binary.
+
+```
+wget https://github.com/grafana/loki/blob/master/cmd/promtail/promtail-local-config.yaml
+```
+
+2. Open the config file in the text editor of your choice. It should look similar to this:
+
+```
+server:
+  http_listen_port: 9080
+  grpc_listen_port: 0
+
+positions:
+  filename: /tmp/positions.yaml
+
+clients:
+  - url: http://loki:3100/loki/api/v1/push
+
+scrape_configs:
+- job_name: system
+  static_configs:
+  - targets:
+      - localhost
+    labels:
+      job: varlogs
+      __path__: /var/log/*log
+```
+
+   The seven lines under `scrape_configs` are what send the logs that Loki generates to Loki, which then outputs them in the command line and http://localhost:3100/metrics.
+
+3. Copy the seven lines under `scrape_configs`, and then paste them under the original job (you can also just edit the original seven lines). 
+
+   Below is an example that sends logs from a default Grafana installation to Loki. We updated the following fields:
+   - job_name - This differentiates the logs collected from other log groups.
+   - targets - Optional for static_configs, however is often defined because in older versions of Promtail it was not optional. This was an artifact from directly using the Prometheus service discovery code which required this entry.
+   - labels - Static label to apply to every log line scraped by this definition. Good examples would be environment name, job name, or app name.
+   - __path__ - The path to where the logs are stored that I want Loki to consume.
+
+```
+- job_name: grafana
+  static_configs:
+  - targets:
+      - grafana
+    labels:
+      job: grafana
+      __path__: "C:/Program Files/GrafanaLabs/grafana/data/log/grafana.log"
+```
+
+4. Enter the following command to run Promtail. Examples below assume you have put the config file in the same directory as the binary.
+
+**Windows**
+
+```
+`.\promtail-windows-amd64.exe --config.file=promtail-local-config.yaml`
+```
+
+**Linux**
+
+```
+./promtail-linux-amd64 -config.file=promtail-local-config.yaml
+```
+
+You should now see your application logs. If you are using Grafana, you might need to refresh your instance in order to see the logs.
diff --git a/docs/installation/README.md b/docs/installation/README.md
@@ -1,6 +1,21 @@
-# Installing Loki
+# Install Loki
 
-1. [Installing using Tanka (recommended)](./tanka.md)
-2. [Installing through Helm](./helm.md)
-3. [Installing through Docker or Docker Compose](./docker.md)
-4. [Installing locally](./local.md)
+## Installation methods
+
+Instructions for different methods of installing Loki and Promtail.
+
+- [Install using Tanka (recommended)](./tanka.md)
+- [Install through Helm](./helm.md)
+- [Install through Docker or Docker Compose](./docker.md)
+- [Install and run locally](./local.md)
+- [Install from source](./install-from-source.md)
+
+## General process
+
+In order to run Loki, you must:
+
+1. Download and install both Loki and Promtail.
+1. Download config files for both programs.
+1. Start Loki.
+1. Update the Promtail config file to get your logs into Loki.
+1. Start Promtail.
diff --git a/docs/installation/docker.md b/docs/installation/docker.md
@@ -1,7 +1,7 @@
-# Installing Loki with Docker or Docker Compose
+# Install Loki with Docker or Docker Compose
 
-You can install Loki with Docker or Docker Compose for evaluating, testing, or developing Loki.
-For production, we recommend Tanka or Helm.
+You can install Loki and Promtail with Docker or Docker Compose if you are evaluating, testing, or developing Loki.
+For production, we recommend installing with Tanka or Helm.
 
 ## Prerequisites
 
@@ -25,8 +25,7 @@ When finished, `loki-config.yaml` and `promtail-config.yaml` are downloaded in t
 
 Navigate to http://localhost:3100/metrics to view the metrics and http://localhost:3100/ready for readiness.
 
-As of v1.5.0, image is configured to run by default as user loki with  UID `10001` and GID `10001`. You can use a different user, specially if you are using bind mounts, by specifying uid with docker run command
-by specifying `--user=UID` with numeric UID suited to your needs.
+As of v1.5.0, image is configured to run by default as user loki with  UID `10001` and GID `10001`. You can use a different user, specially if you are using bind mounts, by specifying the UID with a `docker run` command and using `--user=UID` with numeric UID suited to your needs.
 
 **Windows**
 
@@ -46,7 +45,9 @@ Navigate to http://localhost:3100/metrics to view the output.
 
 ## Install with Docker Compose
 
+Run the following commands in your command line. They work for Windows or Linux systems.
+
 ```bash
-$ wget https://raw.githubusercontent.com/grafana/loki/v1.5.0/production/docker-compose.yaml -O docker-compose.yaml
-$ docker-compose -f docker-compose.yaml up
+wget https://raw.githubusercontent.com/grafana/loki/v1.5.0/production/docker-compose.yaml -O docker-compose.yaml
+docker-compose -f docker-compose.yaml up
 ```
diff --git a/docs/installation/helm.md b/docs/installation/helm.md
@@ -1,72 +1,73 @@
-# Installing Loki with Helm
+# Install Loki with Helm
 
 ## Prerequisites
 
 Make sure you have Helm [installed](https://helm.sh/docs/using_helm/#installing-helm) and
-[deployed](https://helm.sh/docs/using_helm/#installing-tiller) to your cluster. Then add
-[Loki's chart repository](https://github.com/grafana/loki/tree/master/production/helm/loki) to Helm:
+[deployed](https://helm.sh/docs/using_helm/#installing-tiller) to your cluster. 
+
+Add [Loki's chart repository](https://github.com/grafana/loki/tree/master/production/helm/loki) to Helm:
 
 ```bash
-$ helm repo add loki https://grafana.github.io/loki/charts
+helm repo add loki https://grafana.github.io/loki/charts
 ```
 
-You can update the chart repository by running:
+To update the chart repository, run:
 
 ```bash
-$ helm repo update
+helm repo update
 ```
 
 ## Deploy Loki to your cluster
 
 ### Deploy with default config
 
 ```bash
-$ helm upgrade --install loki loki/loki-stack
+helm upgrade --install loki loki/loki-stack
 ```
 
 ### Deploy in a custom namespace
 
 ```bash
-$ helm upgrade --install loki --namespace=loki loki/loki
+helm upgrade --install loki --namespace=loki loki/loki
 ```
 
 ### Deploy with custom config
 
 ```bash
-$ helm upgrade --install loki loki/loki --set "key1=val1,key2=val2,..."
+helm upgrade --install loki loki/loki --set "key1=val1,key2=val2,..."
 ```
 
 ### Deploy Loki Stack (Loki, Promtail, Grafana, Prometheus)
 
 ```bash
-$ helm upgrade --install loki loki/loki-stack  --set grafana.enabled=true,prometheus.enabled=true,prometheus.alertmanager.persistentVolume.enabled=false,prometheus.server.persistentVolume.enabled=false
+helm upgrade --install loki loki/loki-stack  --set grafana.enabled=true,prometheus.enabled=true,prometheus.alertmanager.persistentVolume.enabled=false,prometheus.server.persistentVolume.enabled=false
 ```
 
 ### Deploy Loki Stack (Loki, Fluent Bit, Grafana, Prometheus)
 
 ```bash
-$ helm upgrade --install loki loki/loki-stack \
-    --set fluent-bit.enabled=true,promtail.enabled=false,grafana.enabled=true,prometheus.enabled=true,prometheus.alertmanager.persistentVolume.enabled=false,prometheus.server.persistentVolume.enabled=false
+helm upgrade --install loki loki/loki-stack \
+  --set fluent-bit.enabled=true,promtail.enabled=false,grafana.enabled=true,prometheus.enabled=true,prometheus.alertmanager.persistentVolume.enabled=false,prometheus.server.persistentVolume.enabled=false
 ```
 
 ## Deploy Grafana to your cluster
 
 To install Grafana on your cluster with Helm, use the following command:
 
 ```bash
-$ helm install stable/grafana -n loki-grafana
+helm install stable/grafana -n loki-grafana
 ```
 
 To get the admin password for the Grafana pod, run the following command:
 
 ```bash
-$ kubectl get secret --namespace <YOUR-NAMESPACE> loki-grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo
+kubectl get secret --namespace <YOUR-NAMESPACE> loki-grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo
 ```
 
 To access the Grafana UI, run the following command:
 
 ```bash
-$ kubectl port-forward --namespace <YOUR-NAMESPACE> service/loki-grafana 3000:80
+kubectl port-forward --namespace <YOUR-NAMESPACE> service/loki-grafana 3000:80
 ```
 
 Navigate to `http://localhost:3000` and login with `admin` and the password
@@ -79,8 +80,7 @@ If Loki and Promtail are deployed on different clusters you can add an Ingress
 in front of Loki. By adding a certificate you create an HTTPS endpoint. For
 extra security you can also enable Basic Authentication on the Ingress.
 
-In Promtail, set the following values to communicate using HTTPS and basic
-authentication:
+In Promtail, set the following values to communicate using HTTPS and basic authentication:
 
 ```yaml
 loki:

diff --git a/docs/installation/install-from-source.md b/docs/installation/install-from-source.md
@@ -0,0 +1,26 @@
+# Build from source
+
+In order to build Loki manually, you need to clone the GitHub repo and then `make Loki`.
+
+## Prerequisites
+
+- Go 1.13 or later
+- Make
+- Docker (for updating protobuf files and yacc files)
+
+## Build manually on your local system
+
+Clone Loki to `$GOPATH/src/github.com/grafana/loki`:
+
+```bash
+git clone https://github.com/grafana/loki $GOPATH/src/github.com/grafana/loki
+```
+
+Then change into that directory and run `make loki`:
+
+```bash
+cd $GOPATH/src/github.com/grafana/loki
+make loki
+```
+
+A file at ./cmd/loki/loki will be created and is the final built binary.