[Programming question]: Running notebooks sequentially

[haesleinhuepf]

I'd like to have Python code that runs all notebooks in a folder (and sub-folders) and prints out which notebooks run successfully, and which notebooks had an error while execution.

git-bob think about how to achieve this

[github-actions]

^{This message was generated by git-bob (version: 0.19.1, model: claude-3-5-sonnet-20241022, log), an experimental AI-based assistant. It can make mistakes and has limitations. Check its messages carefully.}

Hi,

Here's a solution using nbconvert and nbformat to execute notebooks and track their status:

import os
import nbformat
from nbconvert.preprocessors import ExecutePreprocessor
from pathlib import Path

def run_notebooks(folder_path):
    # Find all .ipynb files
    notebook_files = list(Path(folder_path).rglob("*.ipynb"))
    
    success = []
    failed = []
    
    # Setup notebook execution
    ep = ExecutePreprocessor(timeout=600)
    
    for nb_path in notebook_files:
        try:
            # Read notebook
            with open(nb_path) as f:
                nb = nbformat.read(f, as_version=4)
            
            # Execute notebook
            ep.preprocess(nb)
            success.append(str(nb_path))
            
        except Exception as e:
            failed.append((str(nb_path), str(e)))
    
    # Print results
    print("\nSuccessfully executed notebooks:")
    for nb in success:
        print(f"✓ {nb}")
        
    print("\nFailed notebooks:")
    for nb, error in failed:
        print(f"✗ {nb}")
        print(f"  Error: {error}\n")

# Example usage
run_notebooks("./notebooks")

You can just change the path in the last line to point to your notebooks folder. The script will:

• Find all .ipynb files recursively
• Execute them one by one
• Print success/failure status with error messages if any

Let me know if you need any adjustments!

Best,
git-bob