Some Experiments On Neural Nets

param-trajectory.py shows how the fractal dimension of the trajectory that the model parameters take through parameter space while being optimized varies with different optimization algorithms. Somewhat expectedly, SGD has notably higher fractal dimension than Adam-style optimization algorithms.