feat(sudoku): implement environment

.pre-commit-config.yaml

@@ -74,3 +74,4 @@ repos:
         args:
           - --license-filepath=license_header.txt
           - --comment-style=#
+        exclude: .npy$

README.md

@@ -29,7 +29,7 @@
   <img src="docs/env_anim/cvrp.gif" alt="CVRP" width="30%" />
   <img src="docs/env_anim/rubiks_cube.gif" alt="RubiksCube" width="30%" />
   <img src="docs/env_anim/game_2048.gif" alt="Game2048" width="30%" />
-  <img src="docs/env_anim/minesweeper.gif" alt="Minesweeper" width="30%" />
+  <img src="docs/env_anim/sudoku.gif" alt="Sudoku" width="30%" />
 </p>
 
 
@@ -81,6 +81,7 @@ problems.
 | 🔢 Game2048                              | Logic  | `Game2048-v1`                                        | [code](https://github.com/instadeepai/jumanji/tree/main/jumanji/environments/logic/game_2048/)   | [doc](https://instadeepai.github.io/jumanji/environments/game_2048/)   |
 | 💣 Minesweeper                           | Logic    | `Minesweeper-v0`                                     | [code](https://github.com/instadeepai/jumanji/tree/main/jumanji/environments/logic/minesweeper/) | [doc](https://instadeepai.github.io/jumanji/environments/minesweeper/) |
 | 🎲 RubiksCube                            | Logic    | `RubiksCube-v0`<br/>`RubiksCube-partly-scrambled-v0` | [code](https://github.com/instadeepai/jumanji/tree/main/jumanji/environments/logic/rubiks_cube/) | [doc](https://instadeepai.github.io/jumanji/environments/rubiks_cube/) |
+| ✏️ Sudoku                       | Logic    | `Sudoku-v0` <br/>`Sudoku-very-easy-v0`| [code](https://github.com/instadeepai/jumanji/tree/main/jumanji/environments/logic/sudoku/) | [doc](https://instadeepai.github.io/jumanji/environments/sudoku/) |
 | 📦 BinPack (3D BinPacking Problem)       | Packing  | `BinPack-v1`                                         | [code](https://github.com/instadeepai/jumanji/tree/main/jumanji/environments/packing/bin_pack/)  | [doc](https://instadeepai.github.io/jumanji/environments/bin_pack/)    |
 | 🏭 JobShop (Job Shop Scheduling Problem) | Packing  | `JobShop-v0`                                         | [code](https://github.com/instadeepai/jumanji/tree/main/jumanji/environments/packing/job_shop/)  | [doc](https://instadeepai.github.io/jumanji/environments/job_shop/)    |
 | 🎒 Knapsack                              | Packing  | `Knapsack-v1`                                        | [code](https://github.com/instadeepai/jumanji/tree/main/jumanji/environments/packing/knapsack/)  | [doc](https://instadeepai.github.io/jumanji/environments/knapsack/)    |

docs/api/environments/sudoku.md

@@ -0,0 +1,8 @@
+::: jumanji.environments.logic.sudoku.env.Sudoku
+    selection:
+      members:
+        - __init__
+        - reset
+        - step
+        - observation_spec
+        - action_spec

docs/environments/sudoku.md

@@ -0,0 +1,52 @@
+# Sudoku Environment
+
+<p align="center">
+        <img src="../env_anim/sudoku.gif" width="500"/>
+</p>
+
+We provide here a Jax JIT-able implementation of the
+[Sudoku](https://en.wikipedia.org/wiki/Sudoku) puzzle game.
+
+
+## Observation
+The observation given to the agent consists of:
+
+
+- `board`: jax array (int32) of shape (9,9):
+    empty cells are represented by -1, and filled cells are represented by 0-8.
+- `action_mask`: jax array (bool) of shape (9,9,9):
+    indicates which actions are valid.
+
+
+
+## Action
+The action space is a `MultiDiscreteArray` of integer values representing coordinates of the square
+to explore and the digits to write in the cell, e.g. `[3, 6, 8]` for writing the digit `9` in
+the cell located on the fourth row and seventh column.
+
+
+## Reward
+The reward is `1` at the end of the episode if the board is correctly solved, and `0` in every
+other case.
+
+## Termination
+An episode terminates when there are no more legal actions available, this could happen if the
+board is solved or if the agent finds itself in a dead-end.
+
+## Registered Versions 📖
+- `Sudoku-v0`, the classic [game](https://en.wikipedia.org/wiki/Sudoku) on
+a 9x9 grid, 10000 random puzzles with mixed difficulty are included by default.
+- `Sudoku-very-easy-v0`, the classic [game](https://en.wikipedia.org/wiki/Sudoku) on
+a 9x9 grid, only 1000 very-easy random puzzles (>46 clues) included by default.
+
+
+## Using custom puzzle instances
+If one wants to include its own database of puzzles, the `DatabaseGenerator` can be initialized with any collection of puzzles using the argument `custom_boards`.
+Some references for databases of puzzle of various difficulties:
+ - https://www.kaggle.com/datasets/rohanrao/sudoku
+ - https://www.kaggle.com/datasets/informoney/4-million-sudoku-puzzles-easytohard
+
+
+## Difficulty level as a function of number of clues
+<a href="https://www.researchgate.net/figure/Number-of-clues-for-each-difficulty-level_tbl1_259525699"><img width=400 src="https://www.researchgate.net/profile/Arnab-Maji-2/publication/259525699/figure/tbl1/AS:651874038460436@1532430290710/Number-of-clues-for-each-difficulty-level.png" alt="Number of clues for each difficulty level"/></a>
+Adapted from [An Algorithm for Generating only Desired Permutations for Solving Sudoku Puzzle](https://www.researchgate.net/figure/Number-of-clues-for-each-difficulty-level_tbl1_259525699).

examples/load_checkpoints.ipynb

@@ -42,12 +42,10 @@
   },
   {
    "cell_type": "markdown",
+   "metadata": {},
    "source": [
     "## Load configs"
-   ],
-   "metadata": {
-    "collapsed": false
-   }
+   ]
   },
   {
    "cell_type": "code",
@@ -56,7 +54,9 @@
    "outputs": [
     {
      "data": {
-      "text/plain": "{'agent': 'a2c', 'seed': 0, 'logger': {'type': 'terminal', 'save_checkpoint': False, 'name': '${agent}_${env.name}'}, 'env': {'name': 'connector', 'env_kwargs': None, 'network': {'transformer_num_blocks': 4, 'transformer_num_heads': 8, 'transformer_key_size': 16, 'transformer_mlp_units': [512], 'conv_n_channels': 32}, 'training': {'num_epochs': 500, 'num_learner_steps_per_epoch': 100, 'n_steps': 20, 'total_batch_size': 128}, 'evaluation': {'eval_total_batch_size': 5000, 'greedy_eval_total_batch_size': 5000}, 'a2c': {'normalize_advantage': False, 'discount_factor': 0.99, 'bootstrapping_factor': 0.95, 'l_pg': 1.0, 'l_td': 1.0, 'l_en': 0.01, 'learning_rate': 0.0002}}}"
+      "text/plain": [
+       "{'agent': 'a2c', 'seed': 0, 'logger': {'type': 'terminal', 'save_checkpoint': False, 'name': '${agent}_${env.name}'}, 'env': {'name': 'connector', 'env_kwargs': None, 'network': {'transformer_num_blocks': 4, 'transformer_num_heads': 8, 'transformer_key_size': 16, 'transformer_mlp_units': [512], 'conv_n_channels': 32}, 'training': {'num_epochs': 500, 'num_learner_steps_per_epoch': 100, 'n_steps': 20, 'total_batch_size': 128}, 'evaluation': {'eval_total_batch_size': 5000, 'greedy_eval_total_batch_size': 5000}, 'a2c': {'normalize_advantage': False, 'discount_factor': 0.99, 'bootstrapping_factor': 0.95, 'l_pg': 1.0, 'l_td': 1.0, 'l_en': 0.01, 'learning_rate': 0.0002}}}"
+      ]
      },
      "execution_count": 3,
      "metadata": {},
@@ -71,12 +71,10 @@
   },
   {
    "cell_type": "markdown",
+   "metadata": {},
    "source": [
     "## Load a saved checkpoint"
-   ],
-   "metadata": {
-    "collapsed": false
-   }
+   ]
   },
   {
    "cell_type": "code",
@@ -95,12 +93,10 @@
   },
   {
    "cell_type": "markdown",
+   "metadata": {},
    "source": [
     "## Rollout a few episodes"
-   ],
-   "metadata": {
-    "collapsed": false
-   }
+   ]
   },
   {
    "cell_type": "code",
@@ -130,12 +126,10 @@
   },
   {
    "cell_type": "markdown",
+   "metadata": {},
    "source": [
     "## Save GIF"
-   ],
-   "metadata": {
-    "collapsed": false
-   }
+   ]
   },
   {
    "cell_type": "code",
@@ -152,12 +146,10 @@
   },
   {
    "cell_type": "markdown",
+   "metadata": {},
    "source": [
     "## Save PNG"
-   ],
-   "metadata": {
-    "collapsed": false
-   }
+   ]
   },
   {
    "cell_type": "code",
@@ -166,8 +158,10 @@
    "outputs": [
     {
      "data": {
-      "text/plain": "<Figure size 720x720 with 1 Axes>",
-      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAi0AAAIuCAYAAABzfTjcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAAsTAAALEwEAmpwYAAAO0klEQVR4nO3dwW1kxxWF4XubhLoYgb1xCnYM3IhJOAyFMWEoifGGMdgheKcI+lHQdHkxBLzxQsAbueYUvy+Bdw+6h/rxBJA95ywAgO/dZfUBAAC/h2gBACKIFgAggmgBACKIFgAggmgBACKIFgAggmgBACKIFgAggmgBACKIFgAggmgBACKIFgAggmgBACKIFgAggmgBACKIFgAggmgBACKIFgAggmgBACKIFgAggmgBACKIFgAggmgBACKIFgAggmgBACKIFgAggmgBACKIFgAggmgBACKIFgAgwuPqAy4P/e95r7+svuOMvtSXea+H1XectcOOH0bdfz02iPFxuddxj95xHeP+dhzRG8YY9yN8Q9UeO3bYULXHjjHGL7fb7c8rnt1zzhXP/e8B3fOfs56XHnHS37pe0zdU7bHjfUO8v3VVzb+vPuOc/rlW/3w5q7vjN1TtsWOHDVV77Hjf0CueHV17AMDHIVoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAjL//YQAPC//av++N+W/9fK+bMC3rQAABFECwAQQbQAABFECwAQQbQAABFECwAQQbQAABFECwAQQbQAABFECwAQQbQAABFECwAQQbQAABFECwAQQbQAABFECwAQQbQAABFECwAQQbQAABFECwAQQbQAABFECwAQQbQAABFECwAQ4XH1AQDA//bXmqtP+K540wIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIAROg51/5dg8tD/zbv9bD0iJP6UjXvq684b4cdP4yqX4/VV3wD41J1ZH8Y1zHq7cj+MMYYdYRvqNpjxw4bqvbYMca43263Jf/dXh4t3T1r/v156RFn9c+v8Ruq9tixw4aqPXb0z6+rf76c1d2VvqFqjx07bKjaY8f7hl7xbP97CACIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAg951x7wMPlS91ndjxdquq++ohvYIcdO2yo2mLHdVzr7XhbfcYpY4w6jmP1GaftsGOHDVV77Bhj3G+328OKZ6+Plu65+oazurvSN1TtsWOHDVV77Ojuqp+yN9SnrvppPq8+47RP/brD9yl9Q9UeO9439IpnZ7/hAAA+DNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAER4XH0Auf7xf/glzj9m/7ZrAL4hb1oAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCI8Lj6AHL9OFdfAMBH4k0LABBBtAAAEUQLABBBtAAAEUQLABBBtAAAEUQLABBBtAAAEUQLABBBtAAAEUQLABCh51z7B2Senp6+HMcRHU9jjDqOY/UZp+2wY4cNVZvseBxVv4Vv6EvVvK++4rTrddTbW/ZnscW/idpjx3WM+3G7Pax49vJo6e65+oazurvSN1R93fG5sne81D6fRfqOXTbUT/N59R2nferX+B07bKjaY8enfp1z9opHR7/hAAA+DtECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEToOefSA56enr4cxxEdT2OMOo5j9Rmn/VCjfq3sHeM66njL3lC1x3dqhw3Vl6p5X33FN3CpqvQdO2yo2mJHX+7z/uVhyaNXR0t3z9U3nNXdlb6h6uuOzzWfV99xxkv16y6fRfoOG74fO+zYYUPVHjveN/SKZ0e/4QAAPg7RAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEeFx9AMBW+g/+7ebhvwIezvCmBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiPqw8A2Mqcqy+AbXnTAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEWP63h8YY9+6OjqcxRnX36jNOu9SlXqpfV99xxrju8Vns8J2y4fuxw44dNlTtsWOMcV/17J6L/7hXd8/VN5zV3ZW+oerrjs81n1ffccZL9esun0X6Dhu+Hzvs2GFD1R473jcsKa/oNxwAwMchWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIjwuPqAMca9u6PjaYxR3b36jNMudamX6tfVd5wxrnt8Fjt8p2z4fuywY4cNVXvsGGPcVz2755yrnv31gO65+oazurvSN1R93fG55vPqO854qX7d5bNI39Hdlb2gqqviP4eqjb5P4Ruq9tjxvmFJeUW/4QAAPg7RAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEeFx9wBjj3t3R8TTGqO5efcZpl7rUS/Xr6jvOuF73+Cx2+E6NqspeUDWu1/jPoWqT79MGG6r22DHGuK96ds85Vz376wHdc/UNZ3V3pW+o+rrjc83n1Xec8VL9mr6hao8dL9Wvsyp6Q1e97vJvO33HDhuq9tjxvmFJeUW/4QAAPg7RAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQISecy494Onp6ctxHNHxNMao4zhWn3HapS51r/vqM07p6pq19jv9Leywo6vCF1SN67WOt7fVZ5y2w8+oHTZU7bFjjHG/3W4PK569PFq6e66+4azurln1vPqOs7rqNX3HDhuq9tixzYbwn09V7z+jwnfssKFqjx3vG3rFs6PfcAAAH4doAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAi9Jxz6QFPT09fjuOIjqdLVd1XH/EN7LBjhw1Ve+zYYcO4Xut4e1t9xmljjDqOY/UZp+ywoWqPHWOM++12e1jx7OXR0t1z9Q1ndXelb6jaY8cOG6r22GHD92OHHTtsqNpjx/uGXvHs6DccAMDHIVoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCI8Lj6gDHGL939p9V3nDHGuHd3fADusGOHDVV77LDh+7HDjh02VO2xY4zxy6pn95xz1bMBAH636NoDAD4O0QIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIARPgPmaN27GY/4YUAAAAASUVORK5CYII=\n"
+      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAi0AAAIuCAYAAABzfTjcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAAsTAAALEwEAmpwYAAAO0klEQVR4nO3dwW1kxxWF4XubhLoYgb1xCnYM3IhJOAyFMWEoifGGMdgheKcI+lHQdHkxBLzxQsAbueYUvy+Bdw+6h/rxBJA95ywAgO/dZfUBAAC/h2gBACKIFgAggmgBACKIFgAggmgBACKIFgAggmgBACKIFgAggmgBACKIFgAggmgBACKIFgAggmgBACKIFgAggmgBACKIFgAggmgBACKIFgAggmgBACKIFgAggmgBACKIFgAggmgBACKIFgAggmgBACKIFgAggmgBACKIFgAggmgBACKIFgAgwuPqAy4P/e95r7+svuOMvtSXea+H1XectcOOH0bdfz02iPFxuddxj95xHeP+dhzRG8YY9yN8Q9UeO3bYULXHjjHGL7fb7c8rnt1zzhXP/e8B3fOfs56XHnHS37pe0zdU7bHjfUO8v3VVzb+vPuOc/rlW/3w5q7vjN1TtsWOHDVV77Hjf0CueHV17AMDHIVoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAjL//YQAPC//av++N+W/9fK+bMC3rQAABFECwAQQbQAABFECwAQQbQAABFECwAQQbQAABFECwAQQbQAABFECwAQQbQAABFECwAQQbQAABFECwAQQbQAABFECwAQQbQAABFECwAQQbQAABFECwAQQbQAABFECwAQQbQAABFECwAQ4XH1AQDA//bXmqtP+K540wIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIAROg51/5dg8tD/zbv9bD0iJP6UjXvq684b4cdP4yqX4/VV3wD41J1ZH8Y1zHq7cj+MMYYdYRvqNpjxw4bqvbYMca43263Jf/dXh4t3T1r/v156RFn9c+v8Ruq9tixw4aqPXb0z6+rf76c1d2VvqFqjx07bKjaY8f7hl7xbP97CACIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAg951x7wMPlS91ndjxdquq++ohvYIcdO2yo2mLHdVzr7XhbfcYpY4w6jmP1GaftsGOHDVV77Bhj3G+328OKZ6+Plu65+oazurvSN1TtsWOHDVV77Ojuqp+yN9SnrvppPq8+47RP/brD9yl9Q9UeO9439IpnZ7/hAAA+DNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAER4XH0Auf7xf/glzj9m/7ZrAL4hb1oAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCI8Lj6AHL9OFdfAMBH4k0LABBBtAAAEUQLABBBtAAAEUQLABBBtAAAEUQLABBBtAAAEUQLABBBtAAAEUQLABCh51z7B2Senp6+HMcRHU9jjDqOY/UZp+2wY4cNVZvseBxVv4Vv6EvVvK++4rTrddTbW/ZnscW/idpjx3WM+3G7Pax49vJo6e65+oazurvSN1R93fG5sne81D6fRfqOXTbUT/N59R2nferX+B07bKjaY8enfp1z9opHR7/hAAA+DtECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEQQLQBABNECAEToOefSA56enr4cxxEdT2OMOo5j9Rmn/VCjfq3sHeM66njL3lC1x3dqhw3Vl6p5X33FN3CpqvQdO2yo2mJHX+7z/uVhyaNXR0t3z9U3nNXdlb6h6uuOzzWfV99xxkv16y6fRfoOG74fO+zYYUPVHjveN/SKZ0e/4QAAPg7RAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEeFx9AMBW+g/+7ebhvwIezvCmBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiPqw8A2Mqcqy+AbXnTAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEWP63h8YY9+6OjqcxRnX36jNOu9SlXqpfV99xxrju8Vns8J2y4fuxw44dNlTtsWOMcV/17J6L/7hXd8/VN5zV3ZW+oerrjs81n1ffccZL9esun0X6Dhu+Hzvs2GFD1R473jcsKa/oNxwAwMchWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIggWgCACKIFAIjwuPqAMca9u6PjaYxR3b36jNMudamX6tfVd5wxrnt8Fjt8p2z4fuywY4cNVXvsGGPcVz2755yrnv31gO65+oazurvSN1R93fG55vPqO854qX7d5bNI39Hdlb2gqqviP4eqjb5P4Ruq9tjxvmFJeUW/4QAAPg7RAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEeFx9wBjj3t3R8TTGqO5efcZpl7rUS/Xr6jvOuF73+Cx2+E6NqspeUDWu1/jPoWqT79MGG6r22DHGuK96ds85Vz376wHdc/UNZ3V3pW+o+rrjc83n1Xec8VL9mr6hao8dL9Wvsyp6Q1e97vJvO33HDhuq9tjxvmFJeUW/4QAAPg7RAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQATRAgBEEC0AQISecy494Onp6ctxHNHxNMao4zhWn3HapS51r/vqM07p6pq19jv9Leywo6vCF1SN67WOt7fVZ5y2w8+oHTZU7bFjjHG/3W4PK569PFq6e66+4azurln1vPqOs7rqNX3HDhuq9tixzYbwn09V7z+jwnfssKFqjx3vG3rFs6PfcAAAH4doAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAiiBYAIIJoAQAi9Jxz6QFPT09fjuOIjqdLVd1XH/EN7LBjhw1Ve+zYYcO4Xut4e1t9xmljjDqOY/UZp+ywoWqPHWOM++12e1jx7OXR0t1z9Q1ndXelb6jaY8cOG6r22GHD92OHHTtsqNpjx/uGXvHs6DccAMDHIVoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCIIFoAgAiiBQCI8Lj6gDHGL939p9V3nDHGuHd3fADusGOHDVV77LDh+7HDjh02VO2xY4zxy6pn95xz1bMBAH636NoDAD4O0QIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIARBAtAEAE0QIARPgPmaN27GY/4YUAAAAASUVORK5CYII=\n",
+      "text/plain": [
+       "<Figure size 720x720 with 1 Axes>"
+      ]
      },
      "metadata": {
       "needs_background": "light"
@@ -199,7 +193,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.15"
+   "version": "3.10.9"
   }
  },
  "nbformat": 4,

examples/training.ipynb

@@ -59,7 +59,7 @@
    },
    "outputs": [],
    "source": [
-    "env = \"maze\"  # @param ['bin_pack', 'cleaner', 'connector', 'cvrp', 'game_2048', 'job_shop', 'knapsack', 'maze', 'minesweeper', 'rubiks_cube', 'snake', 'tsp']\n",
+    "env = \"maze\"  # @param ['bin_pack', 'cleaner', 'connector', 'cvrp', 'game_2048', 'job_shop', 'knapsack', 'maze', 'minesweeper', 'rubiks_cube', 'snake', \"sudoku\", 'tsp']\n",
     "agent = \"random\"  # @param ['random', 'a2c']"
    ]
   },
@@ -105,7 +105,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.15"
+   "version": "3.10.9"
   }
  },
  "nbformat": 4,