python3
- https://www.w3schools.com/python/python_operators.asp
- Python tutorial: https://www.w3schools.com/python/default.asp
/usr/bin/env: python: No such file or directory
sudo apt install python-is-python3
# or
sudo ln -s /usr/bin/python3 /usr/bin/python
import inspect
>>> print(inspect.stack())
[FrameInfo(frame=<frame at 0x7f8462b852d0, file '<stdin>', line 1, code <module>>, filename='<stdin>', lineno=1, function='<module>', code_context=None, index=None, positions=Positions(lineno=1, end_lineno=1, col_offset=6, end_col_offset=21))]https://docs.python.org/3/library/inspect.html#inspect.stack
https://docs.python.org/3/howto/regex.html
The image below: https://docs.python.org/3/howto/regex.html#performing-matches
Always use the 'search' method (typically).
import regex as re
# a line containing numbers
re.search("[0-9]+", line)
# a line containing a keyword anywhere
re.search("blk", line)Further reference: https://www.cyberciti.biz/faq/grep-regular-expressions/
https://www.w3schools.com/python/ref_set_difference.asp
unique = set1.keys() - set1.keys()
return len(c) == 0>>> tuple_list = [('a', 1), ('b', 2), ('c', 3)]
>>> d = dict(tuple_list)
>>> print(d)
{'a': 1, 'b': 2, 'c': 3}
# or
>>> h = {k:v for k,v in tuple_list}
>>> print(h)
{'a': 1, 'b': 2, 'c': 3}
>>> # Returns None, if does not exist
value = d.get(key)
# can provide a default value
value = d.get(key, "empty")https://sparkbyexamples.com/python/python-get-the-last-n-elements-of-a-list/ https://www.geeksforgeeks.org/how-to-get-first-n-items-from-a-list-in-python/
>>> list = [1, 2, 3, 4, 5, 6, 8, 9 , 10]
# first 3
>>> list[:3]
[1, 2, 3]
# except the last three
>>> list[:-3]
[1, 2, 3, 4, 5, 6]
# the last three
>>> list[-3:]
[8, 9, 10]
>>> f = open("demofile.txt", "r")
print(f.read())
import platform
print('Uname:', platform.uname())
print()
print('Machine :', platform.machine())
print('Node :', platform.node())
print('Processor :', platform.processor())
print('Release :', platform.release())
print('System :', platform.system())
print('Version :', platform.version())
print('Platform :', platform.platform())
(major, minor, patch) = platform.release().strip().split('.')[:3]
if int(major) != 6 or int(minor) != 9:
term_size = os.get_terminal_size()
print('#' * int(term_size.columns/2))
print("WARNING: This code was written for the 6.9 kernel version, so please check the kprobe signatures in case you get bogus values.")
print('#' * int(term_size.columns/2))import os
term_size = os.get_terminal_size()
print('_' * int(term_size.columns/2))from parse import parse as p
dev_name = p("/dev/{}", ("/dev/loop0").strip(), case_sensitive=True)May be you will get : ModuleNotFoundError: No module named 'parse'
pip install parse
L = ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h']
for (idx, i) in zip(range(0, len(L)), L):
print(i, e)https://stackoverflow.com/questions/6076270/lambda-function-in-list-comprehensions https://stackoverflow.com/questions/10712002/create-an-empty-list-with-certain-size-in-python
This inits a list with the sequence of (0, 10) and then applies the lambda function which makes it zero (or any constant that we want)
# Make a null list
xs = [None] * 10
# set all to zero
[(lambda x: 0)(x) for x in [None] * 10]
# or an int list then square them
last_values = [(lambda x: x*x)(x) for x in range(10)]
# o rjust directly create a list of all Zeros ;) like here with 256
prev = [0] * 256https://docs.python.org/3/library/string.html#formatspec
'{:<30}'.format('left aligned')
'left aligned '
'{:>30}'.format('right aligned')
' right aligned'
'{:^30}'.format('centered')
' centered '
'{:*^30}'.format('centered') # use '*' as a fill char
'***********centered***********'I think {1} just meant the second argument in the parsing! Positional argument access.
'{0}, {1}, {2}'.format('a', 'b', 'c')
'a, b, c'
'{}, {}, {}'.format('a', 'b', 'c') # 3.1+ only
'a, b, c'
'{2}, {1}, {0}'.format('a', 'b', 'c')
'c, b, a'
'{2}, {1}, {0}'.format(*'abc') # unpacking argument sequence
'c, b, a'
'{0}{1}{0}'.format('abra', 'cad') # arguments' indices can be repeated
'abracadabra'Special direct invocation guard
What does if name == "main": do?
https://stackoverflow.com/questions/4256107/running-bash-commands-in-python
cmd = '''while read -r x;
do ping -c 3 "$x" | grep 'min/avg/max'
done <hosts.txt'''
# Trivial but horrible
results = subprocess.run(
cmd, shell=True, universal_newlines=True, check=True)
print(results.stdout)
# Reimplement with shell=False
with open('hosts.txt') as hosts:
for host in hosts:
host = host.rstrip('\n') # drop newline
ping = subprocess.run(
['ping', '-c', '3', host],
text=True,
stdout=subprocess.PIPE,
check=True)
for line in ping.stdout.split('\n'):
if 'min/avg/max' in line:
print('{}: {}'.format(host, line))Used in the /tools/workqueue scripts from https://www.kernel.org/doc/html/next/core-api/workqueue.html#examining-configuration and https://github.com/iovisor/bcc/blob/master/tools/wqlat_example.txt
sudo apt-get install python3-drgn
https://drgn.readthedocs.io/en/latest/ https://github.com/osandov/drgn
drgn was developed at Meta for debugging the Linux kernel (as an alternative to the crash utility), but it can also debug userspace programs written in C. C++ support is in progress.
See the difference between {} and []
>>> result:dict[str, int] = {}
>>> type(result)
<class 'dict'>
>>> result:dict[str, int] = []
>>> type(result)
<class 'list'>
>>> https://stackoverflow.com/questions/35988/c-like-structures-in-python
@dataclass
class Point:
x: float
y: float
z: float = 0.0
p = Point(1.5, 2.5)
print(p) # Point(x=1.5, y=2.5, z=0.0)immutable:
from typing import NamedTuple
class User(NamedTuple):
name: str
class MyStruct(NamedTuple):
foo: str
bar: int
baz: list
qux: User
my_item = MyStruct('foo', 0, ['baz'], User('peter'))
print(my_item) # MyStruct(foo='foo', bar=0, baz=['baz'], qux=User(name='peter'))https://stackoverflow.com/questions/17777219/zip-variable-empty-after-first-use
hence after the first use the iterator is empty.
>>> from datetime import datetime
>>> datetime.today().strftime('%Y-%m-%d')
'2021-01-26'
>>> datetime.today().strftime('%Y-%m-%d %H:%M:%S')
'2021-01-26 16:50:03'- https://realpython.com/python-f-strings/
- https://realpython.com/python-print/
- https://docs.python.org/3/tutorial/introduction.html
Following the instructions from https://github.com/stonet-research/icpe24_io_scheduler_study_artifact/blob/main/fig-1-samsung-baseline/qd_iops_vary_bs.py then ran into the issues externally managed issue error: externally-managed-environment
error: externally-managed-environment
× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
python3-xyz, where xyz is the package you are trying to
install.
If you wish to install a non-Debian-packaged Python package,
create a virtual environment using python3 -m venv path/to/venv.
Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
sure you have python3-full installed.
If you wish to install a non-Debian packaged Python application,
it may be easiest to use pipx install xyz, which will manage a
virtual environment for you. Make sure you have pipx installed.
See /usr/share/doc/python3.11/README.venv for more information.
note: If you believe this is a mistake, please contact your
Python installation or OS distribution provider. You can override
this, at the risk of breaking your Python installation or OS,
by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.Solution is to setup the venv and use that, see this. Sequence of commands:
python3 -m venv .venv
source .venv/bin/activate
python3 -m pip install -r requirements.txtsudo access issue https://discuss.python.org/t/using-sudo-inside-venv/29148/2 (take the full path)
(.venv) $ which python3
/home/atr/src/.venv/bin/python3
(.venv) $ sudo which python3
/usr/bin/python3
(.venv) $ sudo .venv/bin/python3 test.py -r --fio ~/src/fio/fio --dev /dev/nvme0n1Regarding how to use Python for fio and plotting, I have a couple of general pointers.
I make use of venv, https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/. This ensures pip3 only installs packages locally. For each Python project I first setup venv within the top-level of the repository:
python3 -m venv .venv
echo .venv >> .gitignore # Should be in the ignore to prevent pushing a vm to gitWhenever, you then want to work on the project, you first type (can be automated with direnv):
source ./venv/bin/activateYou can move your locally installed packages to a dependency file with:
pip3 freeze > requirements.txtAnd install them with:
pip3 install -r requirements.txtAll other "suggested" alternatives like poetry and pipenv are slow, made for production software and break every single month.
pip3 install matplotlib numpy notebookMatplotlib is the standard for plotting (https://matplotlib.org/, a good addition is Seaborne) Numpy installs the standard numerics/data structures library used in data science (https://numpy.org/, a friendlier interface on top is https://pandas.pydata.org/). Notebook installs Jupyter (https://docs.jupyter.org/en/latest/)
import json
with open('fio_data.json') as f:
d = json.load(f)
print(d)The default guide is alright: https://matplotlib.org/stable/users/explain/quick_start.html
If notebooks or jupyter lab is installed, you can type in the terminal:
jupyter notebookIt should automatically open your browser.