Skip to content

Commit 037070f

Browse files
jl3953jl3953
committedJun 16, 2021
initial commit
0 parents  commit 037070f

35 files changed

+2979
-0
lines changed
 

‎.gitignore

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
scratch/*
2+
src/__pycache__/*
3+
*.pyc
4+
config/2021*_*.ini
5+
src/trial_*_*.py

‎README.md

+84
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
# Hotnode incorporation checklist
2+
1) Did you change the hotnode address (`node-X:50051`) in cockroachdb
3+
`pkg/server/server.go`?
4+
5+
2) Does that address match the one in your `trial_<config_object_name>.py`
6+
in `self.hot_node`?
7+
8+
3) Did you change the hotkey threshold to be what you want in CRDB
9+
`pkg/sql/conn_executor_prepare.go` in `isHotkey(key byte[])`?
10+
11+
4) Does that threshold match the one in `trial_<config_object_name>.py`
12+
in `self.hot_node_threshold`?
13+
14+
# How to implement a new server-client test script on branch async_redo
15+
16+
1) Copy `src/async_config_object` and name it
17+
`trial_<whatever_you_want>.py`. The `.gitignore` will
18+
ignore it.
19+
- Make sure you populate the fields under the `#default`
20+
comment.
21+
- Make sure all methods are correctly written,
22+
especially the part on populating server and client nodes.
23+
For example, are the nodes regioned? Does it matter?
24+
- Make sure `generate_all_config_files()` method is implemented.
25+
- Make sure the concurrency of the clients is always called
26+
"concurrency" (`self.concurrency` in `ConfigObject`)
27+
28+
2) Change the fields you need to.
29+
30+
3) Change `config/async_lt.ini` for latency-throughput.
31+
32+
4) Implement to the interface of `src/async_server.py`.
33+
- When implementing `aggregate_raw_logs()` function,
34+
make sure to use the keys
35+
- throughput: `"ops/sec(cum)"`
36+
- p50: `"p50(ms)"`
37+
- p99: `"p99(ms)"`
38+
or the latency throughput graphs won't gnuplot at all
39+
- Make sure gnuplot is installed (`apt install gnuplot-x11`)
40+
41+
5) Whatever you name your implementation from the previous
42+
step, change the line `import async_server` in `src/async_main.py`
43+
to `import <whatever_you_implemented> as async_server`
44+
45+
6) Configure/implement the swath of functions at the
46+
head of `src/async_main.py` to match your needs.
47+
- Make sure the directory is correct. It's set to
48+
`thermopylae_tests/scratch/db_{datetime}` right now.
49+
50+
7) From `~/thermopylae_tests` directory, run `python3 src/async_main.py`
51+
52+
# How to implement new `async_determine_stable_interval.py` on branch async_redo
53+
1) Code to the interface of `src/async_server.py`
54+
2) Replace `import async_server` with `import <whatever_you_coded> as async_server`
55+
3) From `~/thermopylae_tests` dir, run `python3 src/async_determine_stable_interval.py
56+
--duration 30s --csv_location scratch/stabilizer --graph_location scratch/stabilizer`
57+
58+
# How to add your config
59+
1) Make a copy of `src/config_object.py` and name it `trial_<whatever_you_want>.py`.
60+
The `.gitignore` will ignore it in the directory.
61+
2) Change the fields that you need to. Add ones you need.
62+
- You may need to implement new functionality that goes along with any new
63+
fields.
64+
3) Determine what latency throughput files should match it (choose the range and
65+
step_size). See `config/lt.ini` for the default example.
66+
4) In `src/main.py`, add your new `trial_<whatever_you_want>.py` file with the
67+
filepath of the latency throughput file to the configuration section. Remember to
68+
import the config object files in `src/main.py`.
69+
5) Make sure the sqlite database directory is what you want it to be (by default,
70+
it is set to to `/proj/cops-PG0/workspaces/jl87`)
71+
6) IMPORTANT: if any nodes have crashed, make sure to add their ip_enums as *args
72+
in `enumerate_workload_nodes(...)` and `enumerate_warm_nodes(...)`. For example:
73+
`enumerate_workload_nodes(driver_node_ip_enum, num_workload_nodes, 2, 5, 6)`
74+
for node-1, node-4, and node-5.
75+
Don't forget to repeat for `enumerate_warm_nodes(...)`.
76+
7) From the git root, run: `./src/main.py`
77+
78+
### Need to Implement
79+
- Automatic start-up of chosen hotshard node.
80+
- Extracting of the cockroach commit in the copied parameter ini files instead of
81+
just the branch name, which may or may not exist at a further point in time.
82+
83+
### Not Implemented
84+
- Partition affinity

‎config/async_lt.ini

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
[DEFAULT]
2+
concurrency = [16, 56]
3+
step_size = 8

‎config/lt.ini

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
[DEFAULT]
2+
concurrency = [32, 160]
3+
step_size = 8

‎src/README.md

+84
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
# Hotnode incorporation checklist
2+
1) Did you change the hotnode address (`node-X:50051`) in cockroachdb
3+
`pkg/server/server.go`?
4+
5+
2) Does that address match the one in your `trial_<config_object_name>.py`
6+
in `self.hot_node`?
7+
8+
3) Did you change the hotkey threshold to be what you want in CRDB
9+
`pkg/sql/conn_executor_prepare.go` in `isHotkey(key byte[])`?
10+
11+
4) Does that threshold match the one in `trial_<config_object_name>.py`
12+
in `self.hot_node_threshold`?
13+
14+
# How to implement a new server-client test script on branch async_redo
15+
16+
1) Copy `src/async_config_object` and name it
17+
`trial_<whatever_you_want>.py`. The `.gitignore` will
18+
ignore it.
19+
- Make sure you populate the fields under the `#default`
20+
comment.
21+
- Make sure all methods are correctly written,
22+
especially the part on populating server and client nodes.
23+
For example, are the nodes regioned? Does it matter?
24+
- Make sure `generate_all_config_files()` method is implemented.
25+
- Make sure the concurrency of the clients is always called
26+
"concurrency" (`self.concurrency` in `ConfigObject`)
27+
28+
2) Change the fields you need to.
29+
30+
3) Change `config/async_lt.ini` for latency-throughput.
31+
32+
4) Implement to the interface of `src/async_server.py`.
33+
- When implementing `aggregate_raw_logs()` function,
34+
make sure to use the keys
35+
- throughput: `"ops/sec(cum)"`
36+
- p50: `"p50(ms)"`
37+
- p99: `"p99(ms)"`
38+
or the latency throughput graphs won't gnuplot at all
39+
- Make sure gnuplot is installed (`apt install gnuplot-x11`)
40+
41+
5) Whatever you name your implementation from the previous
42+
step, change the line `import async_server` in `src/async_main.py`
43+
to `import <whatever_you_implemented> as async_server`
44+
45+
6) Configure/implement the swath of functions at the
46+
head of `src/async_main.py` to match your needs.
47+
- Make sure the directory is correct. It's set to
48+
`thermopylae_tests/scratch/db_{datetime}` right now.
49+
50+
7) From `~/thermopylae_tests` directory, run `python3 src/async_main.py`
51+
52+
# How to implement new `async_determine_stable_interval.py` on branch async_redo
53+
1) Code to the interface of `src/async_server.py`
54+
2) Replace `import async_server` with `import <whatever_you_coded> as async_server`
55+
3) From `~/thermopylae_tests` dir, run `python3 src/async_determine_stable_interval.py
56+
--duration 30s --csv_location scratch/stabilizer --graph_location scratch/stabilizer`
57+
58+
# How to add your config
59+
1) Make a copy of `src/config_object.py` and name it `trial_<whatever_you_want>.py`.
60+
The `.gitignore` will ignore it in the directory.
61+
2) Change the fields that you need to. Add ones you need.
62+
- You may need to implement new functionality that goes along with any new
63+
fields.
64+
3) Determine what latency throughput files should match it (choose the range and
65+
step_size). See `config/lt.ini` for the default example.
66+
4) In `src/main.py`, add your new `trial_<whatever_you_want>.py` file with the
67+
filepath of the latency throughput file to the configuration section. Remember to
68+
import the config object files in `src/main.py`.
69+
5) Make sure the sqlite database directory is what you want it to be (by default,
70+
it is set to to `/proj/cops-PG0/workspaces/jl87`)
71+
6) IMPORTANT: if any nodes have crashed, make sure to add their ip_enums as *args
72+
in `enumerate_workload_nodes(...)` and `enumerate_warm_nodes(...)`. For example:
73+
`enumerate_workload_nodes(driver_node_ip_enum, num_workload_nodes, 2, 5, 6)`
74+
for node-1, node-4, and node-5.
75+
Don't forget to repeat for `enumerate_warm_nodes(...)`.
76+
7) From the git root, run: `./src/main.py`
77+
78+
### Need to Implement
79+
- Automatic start-up of chosen hotshard node.
80+
- Extracting of the cockroach commit in the copied parameter ini files instead of
81+
just the branch name, which may or may not exist at a further point in time.
82+
83+
### Not Implemented
84+
- Partition affinity

‎src/async_config_object.py

+88
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
import itertools
2+
3+
import config_io
4+
import config_object as co
5+
import node
6+
7+
8+
class ConfigObject:
9+
10+
def __init__(self):
11+
12+
# default YOU MUST POPULATE THESE FIELDS
13+
self.trials = [i for i in range(1)]
14+
self.logs_dir = ["test"]
15+
self.store_dir = ["async_server"]
16+
17+
# server
18+
self.server_concurrency = [1]
19+
self.server_commit_branch = ["async"]
20+
self.server_node_ip_enum = [2] # 196.168.1.???
21+
# self.server_node = [some Node object]
22+
23+
# client
24+
self.client_commit_branch = ["async"]
25+
self.num_workload_nodes = [2]
26+
self.concurrency = [10] # YOU MUST CALL CLIENT CONCURRENCY "CONCURRENCY"
27+
self.driver_node_ip_enum = [i + 1 for i in self.server_node_ip_enum] # 192.168.1.???
28+
self.duration = [3] # duration of trial in seconds
29+
# self.workload_nodes [some Node objects]
30+
31+
# workload
32+
self.batch = [1] # keys per rpc
33+
self.read_percent = [95]
34+
35+
def generate_config_combinations(self):
36+
"""Generates the trial configuration parameters for a single run,
37+
lists all in a list of dicts.
38+
39+
:return: a list of dictionaries of combinations
40+
"""
41+
42+
temp_dict = vars(self)
43+
all_field_values = list(temp_dict.values())
44+
values_combinations = list(itertools.product(*all_field_values))
45+
46+
combinations = []
47+
for combo in values_combinations:
48+
config_dict = dict(zip(temp_dict.keys(), combo))
49+
combinations.append(config_dict)
50+
51+
# populating node information. MAKE SURE THIS PART IS CORRECTLY WRITTEN
52+
for config_dict in combinations:
53+
driver_node_ip_enum = config_dict["driver_node_ip_enum"]
54+
num_workload_nodes = config_dict["num_workload_nodes"]
55+
workload_nodes, _ = co.ConfigObject.enumerate_workload_nodes(
56+
driver_node_ip_enum, num_workload_nodes)
57+
config_dict["workload_nodes"] = [vars(n) for n in workload_nodes]
58+
59+
server_node_ip_enum = config_dict["server_node_ip_enum"]
60+
server_node = ConfigObject.create_server_node(server_node_ip_enum)
61+
config_dict["server_node"] = vars(server_node)
62+
63+
return combinations
64+
65+
@staticmethod
66+
def create_server_node(server_node_ip_enum):
67+
return node.Node(server_node_ip_enum)
68+
69+
def generate_all_config_files(self):
70+
"""Generates all configuration files with different combinations of parameters.
71+
:return:
72+
"""
73+
ini_fpaths = []
74+
config_combos = self.generate_config_combinations()
75+
for config_dict in config_combos:
76+
ini_fpath = co.ConfigObject.generate_ini_filename(suffix=config_dict["logs_dir"])
77+
ini_fpaths.append(config_io.write_config_to_file(config_dict, ini_fpath))
78+
79+
return ini_fpaths
80+
81+
82+
def main():
83+
config_object = ConfigObject()
84+
print(config_object.generate_config_combinations())
85+
86+
87+
if __name__ == "__main__":
88+
main()

‎src/async_determine_stable_interval.py

+70
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
import argparse
2+
import os
3+
import sys
4+
5+
import async_config_object
6+
import async_server
7+
8+
CONFIG_OBJ_LIST = [
9+
async_config_object.ConfigObject(),
10+
]
11+
12+
13+
def main():
14+
parser = argparse.ArgumentParser(description="Determine a stable interval")
15+
parser.add_argument("--duration", type=int, default=20 * 60,
16+
help="Duration (s) that to run test for")
17+
parser.add_argument("--csv_location", type=str,
18+
default=os.path.join(os.getcwd(), "scratch/stabilizer"),
19+
help="location of resulting csv file")
20+
parser.add_argument("--graph_location", type=str,
21+
default=os.path.join(os.getcwd(), "scratch/stabilizer"),
22+
help="location of resulting graph")
23+
args = parser.parse_args()
24+
25+
# Generate configurations
26+
configs = []
27+
for config_object in CONFIG_OBJ_LIST:
28+
configs += config_object.generate_config_combinations()
29+
30+
# Run each configuration
31+
for config in configs:
32+
33+
# clean up all experiments first
34+
for node in config["workload_nodes"] + [config["server_node"]]:
35+
async_server.kill(node)
36+
37+
# server
38+
async_server.build_server(config["server_node"],
39+
config["server_commit_branch"])
40+
_ = async_server.run_server(config["server_node"],
41+
config["server_concurrency"])
42+
43+
# clients
44+
if not os.path.exists(args.csv_location):
45+
os.mkdir(args.csv_location)
46+
for client in config["workload_nodes"]:
47+
async_server.build_client(client, config["server_commit_branch"])
48+
logfiles = async_server.run_clients(config["workload_nodes"],
49+
config["server_node"],
50+
args.duration,
51+
config["concurrency"],
52+
config["batch"],
53+
config["read_percent"],
54+
args.csv_location)
55+
56+
# graph
57+
dat_file = async_server.parse_raw_logfiles(logfiles, args.csv_location)
58+
if not os.path.exists(args.graph_location):
59+
os.mkdir(args.graph_location)
60+
async_server.graph(dat_file, args.graph_location)
61+
62+
# clean up again
63+
for node in config["workload_nodes"] + [config["server_node"]]:
64+
async_server.kill(node)
65+
66+
return 0
67+
68+
69+
if __name__ == "__main__":
70+
sys.exit(main())

0 commit comments

Comments
 (0)