Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump IREE requirement pins to their latest versions. #493

Merged
merged 2 commits into from
Feb 13, 2025

Conversation

iree-pr-automator[bot]
Copy link
Contributor

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@ScottTodd
Copy link
Member

Some tests are never terminating on this. Can someone fix the tests to respect a timeout and resolve whatever the specific issue is in this case?

https://github.com/iree-org/iree-turbine/actions/runs/13270905301/job/37049808883?pr=493

[gw3] [ 94%] PASSED tests/kernel/wave/wave_sim_test.py::test_igemm_conv[1-1-3-1] 
tests/kernel/wave/wave_sim_test.py::test_igemm_conv[1-1-3-2] 
[gw3] [ 94%] PASSED tests/kernel/wave/wave_sim_test.py::test_igemm_conv[1-1-3-2] 
tests/kernel/wave/wave_sim_test.py::test_igemm_conv[1-1-3-4] 
[gw3] [ 94%] PASSED tests/kernel/wave/wave_sim_test.py::test_igemm_conv[1-1-3-4] 
tests/kernel/wave/wave_sim_test.py::test_igemm_conv[1-1-10-1] 

(logs stop here)

@Hardcode84
Copy link
Contributor

tests/aot/turbine_generate_test.py::test_example_builder seems to hang, but I have no idea what this test is about.

@stellaraccident
Copy link
Collaborator

tests/aot/turbine_generate_test.py::test_example_builder seems to hang, but I have no idea what this test is about.

This test is using the iree.builder to build a really simple model. The feature is not used anywhere yet to my knowledge and can be disabled. However, it is a really simple smoke test (compile a linear layer) and may be exposing a real problem that was somehow missed upstream. If it is an issue with iree.builder, that is fine to disable and tag me to fix. If it is an issue that we are hanging forever compiling a linear model, that would be bad.

@Hardcode84
Copy link
Contributor

Currently test outputs

tests/aot/turbine_generate_test.py::test_example_builder Main pid: 1743201
[2/13] Export turbine model import_stage1
[3/13] Export turbine model import_stage2
[4/13] Export turbine model import_stage0
  [1743201] Compiling with dynamic shapes: {'x1': {0: <class 'iree.turbine.aot.testing.example_builder.batch'>}, 'x2': {0: <class 'iree.turbine.aot.testing.example_builder.batch'>}}
  [1743293] Compiling with dynamic shapes: {}
  [1743290] Compiling with dynamic shapes: {}
[5/13] Export turbine model import_stage0
[6/13] Compiling BuildFile[gen](pipe/import_stage0.mlir) (for cpu-host)
[7/13] Compiling BuildFile[gen](pipe/import_stage0.mlir) (for cpu-host)
[8/13] Compiling BuildFile[gen](pipe/import_stage0.mlir) (for cpu-host)
[9/13] Compiling BuildFile[gen](pipe/import_stage0.mlir) (for cpu-host)
[10/13] Compiling BuildFile[gen](pipe/import_stage1.mlir) (for cpu-host)
[11/13] Compiling BuildFile[gen](pipe/import_stage1.mlir) (for cpu-host)
[12/13] Compiling BuildFile[gen](pipe/import_stage2.mlir) (for cpu-host)
[13/13] Compiling BuildFile[gen](pipe/import_stage2.mlir) (for cpu-host)

And hangs

@stellaraccident
Copy link
Collaborator

Currently test outputs

tests/aot/turbine_generate_test.py::test_example_builder Main pid: 1743201
[2/13] Export turbine model import_stage1
[3/13] Export turbine model import_stage2
[4/13] Export turbine model import_stage0
  [1743201] Compiling with dynamic shapes: {'x1': {0: <class 'iree.turbine.aot.testing.example_builder.batch'>}, 'x2': {0: <class 'iree.turbine.aot.testing.example_builder.batch'>}}
  [1743293] Compiling with dynamic shapes: {}
  [1743290] Compiling with dynamic shapes: {}
[5/13] Export turbine model import_stage0
[6/13] Compiling BuildFile[gen](pipe/import_stage0.mlir) (for cpu-host)
[7/13] Compiling BuildFile[gen](pipe/import_stage0.mlir) (for cpu-host)
[8/13] Compiling BuildFile[gen](pipe/import_stage0.mlir) (for cpu-host)
[9/13] Compiling BuildFile[gen](pipe/import_stage0.mlir) (for cpu-host)
[10/13] Compiling BuildFile[gen](pipe/import_stage1.mlir) (for cpu-host)
[11/13] Compiling BuildFile[gen](pipe/import_stage1.mlir) (for cpu-host)
[12/13] Compiling BuildFile[gen](pipe/import_stage2.mlir) (for cpu-host)
[13/13] Compiling BuildFile[gen](pipe/import_stage2.mlir) (for cpu-host)

And hangs

This looks like a problem with the iree.build framework vs a compiler problem. Go ahead and disable and file an issue.

@Hardcode84
Copy link
Contributor

Here is the part of stack trace

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <iree.build.main.CliMain object at 0x7fefca537dd0>

    def build_command(self):
        executor = self._create_executor()

        if not self.args.action_path:
            # Default to all.
            build_actions = list(executor.entrypoints)
        else:
            # Look up each requested and add it.
            build_actions = []
            for action_path in self.args.action_path:
                try:
                    build_actions.append(executor.all[action_path])
                except KeyError:
                    all_paths = "\n".join(executor.all.keys())
                    print(
                        f"ERROR: Action '{action_path}' not found. Available: \n{all_paths}",
                        file=self.stderr,
                    )
                    self.abort()
>       executor.build(*build_actions)

../../miniforge3/envs/turbine-env/lib/python3.11/site-packages/iree/build/main.py:240:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <iree.build.executor.Executor object at 0x7fefc2299990>, initial_deps = (BuildEntrypoint(path='pipe'),)
scheduler = <iree.build.executor.Scheduler object at 0x7fefca555850>, success = False, started_reporter = True

    def build(self, *initial_deps: "BuildDependency") -> bool:
        """Transitively builds the given deps."""
        scheduler = Scheduler(reporter=self.reporter)
        success = False
        started_reporter = False
        try:
            for d in initial_deps:
                scheduler.add_initial_dep(d)
                self.reporter.start_graph(set(scheduler.producer_graph.keys()))
                started_reporter = True
                scheduler.build()
        except KeyboardInterrupt:
            raise
        except:
            # This catches truly unhandled exceptions (not just build action failures,
            # which are noted in the graph). Eagerly print the exception so that it
            # doesn't get swallowed waiting for shutdown.
            self.reporter.reset_display()
            print(
                "Unhandled exception during build. Waiting for background tasks to complete...",
                file=self.stderr,
            )
            traceback.print_exc(file=self.stderr)
        finally:
>           scheduler.shutdown()

../../miniforge3/envs/turbine-env/lib/python3.11/site-packages/iree/build/executor.py:203:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <iree.build.executor.Scheduler object at 0x7fefca555850>

    def shutdown(self):
        self.thread_pool_executor.shutdown(cancel_futures=True)
>       self.process_pool_executor.shutdown(cancel_futures=True)

../../miniforge3/envs/turbine-env/lib/python3.11/site-packages/iree/build/executor.py:565:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <concurrent.futures.process.ProcessPoolExecutor object at 0x7fefc24a97d0>, wait = True

    def shutdown(self, wait=True, *, cancel_futures=False):
        with self._shutdown_lock:
            self._cancel_pending_futures = cancel_futures
            self._shutdown_thread = True
            if self._executor_manager_thread_wakeup is not None:
                # Wake up queue management thread
                self._executor_manager_thread_wakeup.wakeup()

        if self._executor_manager_thread is not None and wait:
>           self._executor_manager_thread.join()

../../miniforge3/envs/turbine-env/lib/python3.11/concurrent/futures/process.py:851:

Seems to be hanging in scheduler.shutdown()

Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>
@Hardcode84
Copy link
Contributor

Hardcode84 commented Feb 13, 2025

Skipped 2 hanging tests

@Hardcode84 Hardcode84 merged commit 4ef04d7 into main Feb 13, 2025
9 of 10 checks passed
@Hardcode84 Hardcode84 deleted the integrates/iree branch February 13, 2025 17:16
xintin pushed a commit to xintin/iree-turbine that referenced this pull request Feb 14, 2025
Diff:
iree-org/iree@iree-3.2.0rc20250205...iree-3.3.0rc20250211

Auto-generated by GitHub Actions using
[`.github/workflows/update_iree_requirement_pins.yml`](https://github.com/iree-org/iree-turbine/blob/main/.github/workflows/update_iree_requirement_pins.yml).

---------

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>
Co-authored-by: iree-pr-automator[bot] <iree-github-actions-bot@google.com>
Co-authored-by: Ivan Butygin <ivan.butygin@gmail.com>
Signed-off-by: xintin <vermagaurav9408@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants