Skip to content

Latest commit

 

History

History
188 lines (124 loc) · 7.69 KB

README.md

File metadata and controls

188 lines (124 loc) · 7.69 KB
Raw

1. Introduction

Instructor: Alexey Grigorev

1.1 Introduction

1.2 Environment preparation

1.2.1 GitHub Codespaces

  • Note: One potential issue at approximately the 11:49 minute mark that you might encounter is being able to connect to the ec2 instance using bash terminal but still being unable to connect using VSCODE remote-ssh extension. I was able to resolve it after a bit of trouble shooting. The permissions in powershell and command prompt for the config and .ssh directory had to be modified on windows the local machine using admin rights (won't allow connection if ssh keys and the directories they are in are too open and won't allow connection if not enough permissions granted) until VS Code remote connection extension was finally happy with the settings enough to allow a connection. If you are not able to get it to work with modifications to the directories and files in your .ssh folder, another approach might be to delete the entire .ssh directory and keys and generate an entire new directory and ssh keys altogether from scratch or from backups, in additional to a clean install and full update of VSCode.

OR

1.2.2 VM in AWS

Note: You don't have to rent an instance in the cloud. You can follow the same instructions for setting up your local environment.

Code:

Recommended development environment: Linux

Step 1: Download and install the Anaconda distribution of Python

wget https://repo.anaconda.com/archive/Anaconda3-2022.05-Linux-x86_64.sh
bash Anaconda3-2022.05-Linux-x86_64.sh

Step 2: Update existing packages

sudo apt update

Step 3: Install Docker

sudo apt install docker.io

To run docker without sudo:

sudo groupadd docker
sudo usermod -aG docker $USER

Step 4: Install Docker Compose

Install docker-compose in a separate directory

mkdir soft
cd soft

To get the latest release of Docker Compose, go to https://github.com/docker/compose and download the release for your OS.

wget https://github.com/docker/compose/releases/download/v2.5.0/docker-compose-linux-x86_64 -O docker-compose

Make it executable

chmod +x docker-compose

Add to the soft directory to PATH. Open the .bashrc file with nano:

nano ~/.bashrc

In .bashrc, add the following line:

export PATH="${HOME}/soft:${PATH}"

Save it and run the following to make sure the changes are applied:

source ~/.bashrc

Step 5: Run Docker

docker run hello-world

If you get docker: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/containers/create": dial unix /var/run/docker.sock: connect: permission denied. error, restart your VM instance.

Note: If you get It is required that your private key files are NOT accessible by others. This private key will be ignored. error, you should change permits on the downloaded file to protect your private key:

chmod 400 name-of-your-private-key-file.pem

1.3 (Optional) Training a ride duration prediction model

Note: The NYC taxi data is now in parquet format, not CSV. Here's a video that explains how to read parquet data.

Links:

1.4 Course overview

1.5 MLOps maturity model

Links:

1.6 Homework

More information here.

Notes

Did you take notes? Add them here: