Skip to content

Commit 4e0877a

Browse files
NoahMaizelsNoahMaizels
authored
Merge pull request #679 from ethersphere/improve-logging-section
Improve logging section
2 parents 0c8f128 + da9b661 commit 4e0877a

File tree

1 file changed

+140
-128
lines changed

1 file changed

+140
-128
lines changed

‎docs/bee/working-with-bee/logs-and-files.md‎

Lines changed: 140 additions & 128 deletions
Original file line numberDiff line numberDiff line change
@@ -1,172 +1,184 @@
1-
---
2-
title: Logs and Files
3-
id: logs-and-files
4-
---
51

6-
### Linux
2+
# Logging in Bee
73

8-
If you have installed Bee on Linux using a package manager you will now be able to manage your Bee service using `systemctl`.
4+
This section introduces logging in Bee, detailing log locations, exporting logs, setting the general verbosity level, and setting fine-grained verbosity for individual loggers.
95

6+
## Log Locations
7+
8+
:::warning
9+
Bee logs by default can be quite verbose, and over time may occupy disk space in the gigabytes range. You may wish to practice log rotation to prevent excessive disk utilization.
10+
:::
11+
12+
### **Linux (Package Manager Installation)**
13+
When installed via a package manager (e.g., `APT`, `RPM`), Bee runs as a **systemd service**, and logs are managed by the system journal, **journalctl**.
14+
15+
View logs using:
1016
```bash
11-
systemctl status bee
17+
journalctl --lines=100 --follow --unit bee
1218
```
1319

20+
Export all logs as JSON:
21+
```bash
22+
journalctl --unit bee --output=json > bee-logs.json
1423
```
15-
● bee.service - Bee - Ethereum Swarm node
16-
Loaded: loaded (/lib/systemd/system/bee.service; enabled; vendor preset: enabled)
17-
Active: active (running) since Fri 2020-11-20 23:50:15 GMT; 6s ago
24+
25+
Export logs for a specific time range:
26+
```bash
27+
journalctl --since "1 hour ago" --output=json --unit bee > bee-logs.json
1828
```
1929

20-
Logs are available using the `journalctl` command:
30+
Learn more about `journalctl` usage and filtering logs in this [tutorial](https://www.digitalocean.com/community/tutorials/how-to-use-journalctl-to-view-and-manipulate-systemd-logs) from DigitalOcean.
2131

32+
33+
### **macOS (Homebrew Installation)**
34+
35+
For a Homebrew installation on macOS, logs are saved to:
2236
```bash
23-
journalctl --lines=100 --follow --unit bee
37+
/usr/local/var/log/swarm-bee/bee.log
38+
```
39+
40+
View logs in real time:
41+
```bash
42+
tail -f /usr/local/var/log/swarm-bee/bee.log
2443
```
2544

26-
```text
27-
INFO[2021-02-09T18:55:11Z] swarm public key 03379f7aa673b7f03737064fd23ba1453619924a4602e70bbccc133ba67d0968bd
28-
DEBU[2021-02-09T18:55:11Z] using existing libp2p key
29-
DEBU[2021-02-09T18:55:11Z] using existing pss key
30-
INFO[2021-02-09T18:55:11Z] pss public key 03bae655ce94431e1f2c2de8d017f88c8c5c293ef0057379223084aba9e318596e
31-
INFO[2021-02-09T18:55:11Z] using ethereum address 99c9e7868d22244106a5ffbc2f5d6b7c88e2c85a
32-
INFO[2021-02-09T18:55:14Z] using default factory address for chain id 5: f0277caffea72734853b834afc9892461ea18474
33-
INFO[2021-02-09T18:55:14Z] no chequebook found, deploying new one.
34-
WARN[2021-02-09T18:55:15Z] cannot continue until there is sufficient ETH (for Gas) and at least 10 BZZ available on 99c9e7868d22244106a5ffbc2f5d6b7c88e2c85a
45+
46+
### **Docker**
47+
48+
Docker saves **stdout** and **stderr** output as JSON files by default. The logs are stored under:
49+
50+
```
51+
/var/lib/docker/containers/<container-id>/<container-id>-json.log
3552
```
3653

37-
### MacOS
54+
View logs in real time:
55+
```bash
56+
docker logs -f <container-name>
57+
```
3858

39-
Services are managed using Homebrew services.
59+
Export logs to a file:
60+
```bash
61+
docker logs <container-name> > bee-logs.json
62+
```
4063

64+
Export logs for a specific time range:
4165
```bash
42-
brew services restart swarm-bee
66+
docker logs --since "30m" <container-name> > bee-logs.json
4367
```
4468

45-
Logs are available at `/usr/local/var/log/swarm-bee/bee.log`
69+
See [Docker documentation](https://docs.docker.com/reference/cli/docker/container/logs/) for additional options.
70+
71+
72+
### **Shell Script**
73+
74+
For a shell script-installed Bee started using `bee start`, logs are sent to **stdout** and **stderr** by default, which means they will appear in the terminal. They are **not saved to disk by default**.
75+
76+
To save logs to a file, redirect **stdout** and **stderr**:
4677

4778
```bash
48-
tail -f /usr/local/var/log/swarm-bee/bee.log
79+
bee start --password <password> > bee.log 2>&1 &
80+
```
81+
82+
View recent logs and follow for updates:
83+
```bash
84+
tail -f bee.log
4985
```
5086

51-
## Data Locations
87+
## Logging Levels
88+
89+
Bee supports the following log levels:
90+
91+
| Level | Description |
92+
|-------------|------------------------------------|
93+
| `0=silent` | No logs. |
94+
| `1=error` | Critical errors only. |
95+
| `2=warn` | Warnings and errors. |
96+
| `3=info` | General operational logs (default).|
97+
| `4=debug` | Detailed diagnostic logs. |
98+
| `5=trace` | Highly granular logs for debugging.|
99+
100+
### Behavior of Log Levels
101+
102+
Log levels are cumulative: setting a higher verbosity includes all lower levels.
103+
For example, `debug` will output logs at `debug`, `info`, `warn`, and `error` levels.
104+
52105

53-
### Bee
106+
## Setting Verbosity
54107

55-
Configuration files are stored in `/etc/bee/`. State, chunks and other data are stored in `/var/lib/bee/`
108+
The general verbosity level can be set using the `verbosity` configuration option in order to display all log messages up to the selected level of verbosity.
109+
110+
### **YAML Config File**
111+
Set the `verbosity` parameter in `config.yaml`:
112+
113+
```yaml
114+
# Log verbosity: 0=silent, 1=error, 2=warn, 3=info, 4=debug, 5=trace
115+
verbosity: debug
116+
```
117+
118+
### **Command Line Flag**
119+
Specify verbosity when starting Bee:
120+
121+
```bash
122+
bee start --verbosity debug
123+
```
56124

57-
## Logging Guidelines
125+
### **Environment Variable**
126+
Set `BEE_VERBOSITY` before starting Bee:
58127

59-
The Bee logs provide a robust information on the workings of a Bee node which are useful both to node operators and to Bee developers. Log messages have four levels described below, and logs can be adjusted for verbosity and granularity to suit the needs of the user.
128+
```bash
129+
export BEE_VERBOSITY=debug
130+
bee start
131+
```
60132

61-
### Log Levels
62133

63-
The log messages are divided into four basic levels:
134+
## Fine-Grained Logging Control
64135

65-
- `Error` - Errors in the node. Although the node operation may continue, the error indicates that it should be addressed.
66-
- `Warning` - Warnings should be checked in case the problem recurs to avoid potential damage.
67-
- `Info` - Informational messages useful for node operators that do not indicate any fault or error.
68-
- `Debug` - Information concerning program logic decisions, diagnostic information, internal state, etc. which is primarily useful for developers.
136+
Bee allows fine-grained control of logging levels for specific subsystems using the **`/loggers` API endpoint**. This enables adjustments without restarting the node.
69137

70-
There is a notion of `V-level` attached to the `Debug` level. `V-levels` provide a simple way of changing the verbosity of debug messages. `V-levels` provide a way for a given package to distinguish the relative importance or verbosity of a given log message. Then, if a particular logger or package logs too many messages, the package can simply change the `V` level for that logger.
138+
### **1. Retrieving Loggers List**
71139

72-
### Logging API usage
140+
Retrieve the list of active loggers and their verbosity levels:
141+
```bash
142+
curl http://localhost:1633/loggers | jq
143+
```
73144

74-
In the current Bee code base, it is possible to change the granularity of logging for some services on the fly without the need to restart the node. These services and their corresponding loggers can be found using the `/loggers` endpoint. Example of the output:
145+
The list of loggers includes detailed entries for each subsystem. Below is an example for the `node/api` logger:
75146

76147
```json
77148
{
78-
"tree": {
79-
"node": {
80-
"/": {
81-
"api": {
82-
"+": [
83-
"info|node/api[0][]>>824634933256"
84-
]
85-
},
86-
"batchstore": {
87-
"+": [
88-
"info|node/batchstore[0][]>>824634933256"
89-
]
90-
},
91-
"leveldb": {
92-
"+": [
93-
"info|node/leveldb[0][]>>824634933256"
94-
]
95-
},
96-
"pseudosettle": {
97-
"+": [
98-
"info|node/pseudosettle[0][]>>824634933256"
99-
]
100-
},
101-
"pss": {
102-
"+": [
103-
"info|node/pss[0][]>>824634933256"
104-
]
105-
},
106-
"storer": {
107-
"+": [
108-
"info|node/storer[0][]>>824634933256"
109-
]
110-
}
111-
},
112-
"+": [
113-
"info|node[0][]>>824634933256"
114-
]
115-
}
116-
},
117-
"loggers": [
118-
{
119-
"logger": "node/api",
120-
"verbosity": "info",
121-
"subsystem": "node/api[0][]>>824634933256",
122-
"id": "bm9kZS9hcGlbMF1bXT4-ODI0NjM0OTMzMjU2"
123-
},
124-
{
125-
"logger": "node/storer",
126-
"verbosity": "info",
127-
"subsystem": "node/storer[0][]>>824634933256",
128-
"id": "bm9kZS9zdG9yZXJbMF1bXT4-ODI0NjM0OTMzMjU2"
129-
},
130-
{
131-
"logger": "node/pss",
132-
"verbosity": "info",
133-
"subsystem": "node/pss[0][]>>824634933256",
134-
"id": "bm9kZS9wc3NbMF1bXT4-ODI0NjM0OTMzMjU2"
135-
},
136-
{
137-
"logger": "node/pseudosettle",
138-
"verbosity": "info",
139-
"subsystem": "node/pseudosettle[0][]>>824634933256",
140-
"id": "bm9kZS9wc2V1ZG9zZXR0bGVbMF1bXT4-ODI0NjM0OTMzMjU2"
141-
},
142-
{
143-
"logger": "node",
144-
"verbosity": "info",
145-
"subsystem": "node[0][]>>824634933256",
146-
"id": "bm9kZVswXVtdPj44MjQ2MzQ5MzMyNTY="
147-
},
148-
{
149-
"logger": "node/leveldb",
150-
"verbosity": "info",
151-
"subsystem": "node/leveldb[0][]>>824634933256",
152-
"id": "bm9kZS9sZXZlbGRiWzBdW10-PjgyNDYzNDkzMzI1Ng=="
153-
},
154-
{
155-
"logger": "node/batchstore",
156-
"verbosity": "info",
157-
"subsystem": "node/batchstore[0][]>>824634933256",
158-
"id": "bm9kZS9iYXRjaHN0b3JlWzBdW10-PjgyNDYzNDkzMzI1Ng=="
159-
}
160-
]
149+
"logger": "node/api",
150+
"verbosity": "info",
151+
"subsystem": "node/api[0][]>>824634474528",
152+
"id": "bm9kZS9hcGlbMF1bXT4-ODI0NjM0NDc0NTI4"
161153
}
162154
```
163155

164-
The recorders come in two versions. The first is the tree version and the second is the flattened version. The `subsystem` field is the unique identifier of the logger. The `id` field is a version of the `subsystem` field encoded in base 64 for easier reference to a particular logger. The node name of the version tree is composed of the subsystem with the log level prefix and delimited by the `|` character. The number in the first square bracket indicates the logger's V-level.
156+
- **`id`**: The Base64-encoded identifier used to adjust the logger’s verbosity.
157+
- **`verbosity`**: The current log level.
158+
159+
160+
### **2. Adjusting Logger Verbosity**
161+
162+
You can dynamically adjust the log level for any logger without restarting Bee.
163+
164+
**Syntax**:
165+
```bash
166+
curl -X PUT http://localhost:1633/loggers/<id>/<verbosity>
167+
```
168+
169+
- **`<id>`**: The Base64-encoded logger name retrieved from `/loggers`.
170+
- **`<verbosity>`**: Desired log level (`none`, `error`, `warn`, `info`, `debug`, `trace`).
171+
172+
**Example**: Set `node/api` to `debug`:
173+
```bash
174+
curl -X PUT http://localhost:1633/loggers/bm9kZS9hcGlbMF1bXT4-ODI0NjM0NDc0NTI4/debug
175+
```
165176

166-
The logger endpoint uses HTTP PUT requests to modify the verbosity of the logger(s). The request must have the following parameters `/loggers/{subsystem}/{verbosity}`. The `{subsytem}` parameter is the base64 version of the subsytem field or regular expression corresponding to multiple subsystems. Since the loggers are arranged in tree structure, it is possible to turn on/off or change the logging level of the entire tree or just its branches with a single command. The verbosity can be one of `none`, `error`, `warning`, `info`, `debug` or a number in the range `1` to `1<<<31 - 1` to enable the verbosity of a particular V-level, if available for a given logger. A value of `all` will enable the highest verbosity of V-level.
177+
### Log Level Behavior Note
167178

168-
Examples:
179+
Log levels are cumulative. When a logger is set to a specific level, it will include all log messages at that level and below.
169180

170-
`curl -XPUT http://localhost:1633/loggers/bm9kZS8q/none` - will disable all loggers; `bm9kZS8q` is base64 encoded `node/*` regular expression.
181+
For example:
182+
- Setting a logger to `info` will show logs at `info`, `warn`, and `error`.
183+
- Logs at higher levels (`debug` and `trace`) will **not** be displayed.
171184

172-
`curl -XPUT http://localhost:1633/loggers/bm9kZS9hcGlbMV1bXT4-ODI0NjM0OTMzMjU2/error` - will set the verbosity of the logger with the subsystem `node/api[1][]>>824634933256` to `error`.

0 commit comments

Comments
 (0)