Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add realistic example #38

Merged
merged 24 commits into from
Oct 2, 2023
Merged
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
2c490d9
Add example image to python prediction
ElliottKasoar Sep 20, 2023
1c9843b
Use binary format for saved tensor
ElliottKasoar Sep 22, 2023
d238893
Improve python prediction output
ElliottKasoar Sep 22, 2023
17b4f61
Use python output for fortran inference
ElliottKasoar Sep 22, 2023
48ee015
Predict and output probabilities
ElliottKasoar Sep 22, 2023
6795238
Remove unnecessary loop
ElliottKasoar Sep 25, 2023
54721f7
Add comments and type hinting
ElliottKasoar Sep 25, 2023
810194a
Print category label
ElliottKasoar Sep 25, 2023
a4cf939
Add binary and TorchScript model to gitignore
ElliottKasoar Sep 25, 2023
3b46b03
Add example data
ElliottKasoar Sep 25, 2023
8e7a4ed
Update readme with expected outputs
ElliottKasoar Sep 25, 2023
d764a26
Add sp and dp options
ElliottKasoar Sep 25, 2023
0ca6748
Define C working precision
ElliottKasoar Sep 25, 2023
76db896
Use built in functions for softmax
ElliottKasoar Sep 25, 2023
eecdfc4
Transpose and use built in reshape
ElliottKasoar Sep 26, 2023
10297c5
Move downloads to readme
ElliottKasoar Sep 26, 2023
f88b3d9
Add tranpose explanation to readme
ElliottKasoar Sep 26, 2023
44d959b
Update data download description
ElliottKasoar Sep 28, 2023
5e33767
Update python inference with new example
ElliottKasoar Sep 28, 2023
46bbc78
Tidy code
ElliottKasoar Sep 28, 2023
1dd9372
Update example description
ElliottKasoar Sep 28, 2023
74aeead
Add assertions for inference outputs
ElliottKasoar Sep 28, 2023
3a09be0
Add warning about assertions for different images
ElliottKasoar Sep 28, 2023
bf2a074
Tidy Fortran variable attributes
ElliottKasoar Sep 29, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -183,3 +183,9 @@ cython_debug/
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/

# Data binary
*.dat

# TorchScript model
*.pt
TomMelt marked this conversation as resolved.
Show resolved Hide resolved
69 changes: 67 additions & 2 deletions examples/2_ResNet18/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# Example 2 - ResNet-18

This example provides a simple but complete demonstration of how to use the library.
This example provides a more realistic demonstration of how to use the library, using ResNet-18 to classify an image.

As the input to this model is four-dimensional (batch size, colour, x, y), care must be taken dealing with the data array in Python and Fortran. See [When to transpose arrays](#when-to-transpose-arrays) for more details.

## Description

Expand Down Expand Up @@ -34,10 +36,22 @@ pip install -r requirements.txt
```

You can check that everything is working by running `resnet18.py`:

```
python3 resnet18.py
```
it should produce the result `tensor([[623, 499, 596, 111, 813]])`.

When using single precision, it should produce the result:

```
Top 5 results:

Samoyed (id=258): probability = 0.8846225142478943
Arctic fox (id=279): probability = 0.045805174857378006
white wolf (id=270): probability = 0.0442761555314064
Pomeranian (id=259): probability = 0.005621383432298899
Great Pyrenees (id=257): probability = 0.004652013536542654
```

To save the pretrained ResNet-18 model to TorchScript run the modified version of the
`pt2ts.py` tool :
Expand Down Expand Up @@ -67,6 +81,15 @@ executable with an argument of the saved model file:
./resnet_infer_fortran ../saved_resnet18_model_cpu.pt
```

This should produce the same top result:

```
Top result

Samoyed (id= 259 ), : probability = 0.884623706
```


Alternatively we can use `make`, instead of cmake, with the included Makefile.
However, to do this you will need to modify `Makefile` to link to and include your
installation of FTorch as described in the main documentation. Also check that the compiler is the same as the one you built the Library with.
Expand All @@ -82,3 +105,45 @@ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:</path/to/library/installation>/lib64
To explore the functionalities of this model:

- Try saving the model through tracing rather than scripting by modifying `pt2ts.py`
- Try changing the input image. For example, running the following code will download an alternative image of a dog from [pytorch/vision](https://github.com/pytorch/vision/tree/v0.15.2/gallery/assets), saving it in the `data` directory:

```python
import urllib

url, filename = (
"https://github.com/pytorch/vision/raw/v0.15.1/gallery/assets/dog1.jpg",
"data/dog2.jpg",
)
urllib.request.urlretrieve(url, filename)
```

`image_filename` in resnet18.py and resnet_infer_python.py must then be modified to match the name of this new file.

Note that the ImageNet labels can be downloaded and accessed similarly, using:

```python
import urllib

url, filename = (
"https://raw.githubusercontent.com/pytorch/hub/e55b003/imagenet_classes.txt",
"imagenet_classes.txt",
)
data = urllib.request.urlopen(url)
categories = [s.strip().decode("utf-8") for s in data]
```

## When to transpose arrays?

In this example, it is expected that the shape and indices of `in_data` in resnet_infer_fortran.f90 match that of `input_batch` in resnet18.py, i.e. `in_data(i, j, k, l) == input_batch[i, j, k, l]`.

Since C is row-major (rows are contiguous in memory), whereas Fortran is column-major (columns are contiguous), it is therefore necessary to perform a transpose when converting from the NumPy array to the Fortran array to ensure that their indices are consistent.

In this example code, the NumPy array is transposed before being flattened and saved to binary, allowing Fortran to `reshape` the flatted array into the correct order.

An alternative would be to save the NumPy array with its original shape, but perform a transpose during or after reading the data into Fortran, e.g. using:

```
in_data = reshape(flat_data, shape(in_data), order=(4,3,2,1))
```

For more general use, it should be noted that the function used to create the input tensor from `input_batch`, `torch_tensor_from_blob`, performs a further transpose, which is required to allow the tensor to interact correctly with the model.
Loading