Last active
March 31, 2026 02:24
-
-
Save apivovarov/f179446dac26ea6df64bcafc5725d345 to your computer and use it in GitHub Desktop.
executor->GetOrNullResource<se::DeviceInterconnectResource>() causes SIGILL in tests
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [ RUN ] RaggedAllToAllTest/RaggedAllToAllTest.RaggedAllToAll_2GPUs_CommandBuffer/sync_one_shot_with_multi_gpu_barrier_with_nccl | |
| *** SIGILL (UD1@0x7f9c00625cc9), see go/stacktraces#debugging-ubsan | |
| received by PID 102457 (TID 104591) on cpu 145; stack trace: *** | |
| E0330 19:21:38.081043 104590 process_state.cc:1066] RAW: Signal 4 raised at PC: 0x7f9c00625cc9 while already in FailureSignalHandler! | |
| E0330 19:21:38.081811 104590 process_state.cc:1070] RAW: tid: 104590 raised new signal (old_tid: 104591) | |
| PC: @ 0x7f9c00625cc9 (unknown) xla::gpu::RaggedAllToAllThunk::RunCollective() | |
| @ 0x7f92d4008f08 1904 FailureSignalHandler() | |
| @ 0x7f9d30337c60 (unknown) (unknown) | |
| @ 0x7f9bf5215d8c 288 xla::gpu::CollectiveThunk::ExecuteOnStream() | |
| @ 0x7f9bfa60d618 464 xla::gpu::ThunkExecutor::ExecuteOnStream() | |
| @ 0x7f9bfca0909c 48 xla::gpu::SequentialThunk::ExecuteOnStream() | |
| @ 0x7f9c150119fa 304 xla::gpu::CommandBufferThunk::ExecuteOnStream() | |
| @ 0x7f9bfa60d618 464 xla::gpu::ThunkExecutor::ExecuteOnStream() | |
| @ 0x7f9c16e5b914 1504 xla::gpu::(anonymous namespace)::ExecuteThunksImpl() | |
| @ 0x7f9c16e587c6 416 xla::gpu::GpuExecutable::ExecuteThunks() | |
| @ 0x7f9c16e57291 2016 xla::gpu::GpuExecutable::ExecuteAsyncOnStreamImpl() | |
| @ 0x7f9c16e55f6d 80 xla::gpu::GpuExecutable::ExecuteAsyncOnStream() | |
| @ 0x7f9980410826 1008 xla::Executable::ExecuteAsyncOnStreamWrapper() | |
| @ 0x7f9d0da1427c 2512 xla::LocalExecutable::RunAsync() | |
| @ 0x7f9d0da14e97 288 xla::LocalExecutable::RunAsync() | |
| @ 0x7f9d17cbc671 3296 xla::TfrtGpuExecutable::ExecuteHelper()::$_0::operator()() | |
| @ 0x7f9d17cbbd20 64 absl::internal_any_invocable::RemoteInvoker<>() | |
| @ 0x7f9d17c95313 32 std::__u::__function::__policy_func<>::__call_func<>() | |
| @ 0x7f99fb81aebb 48 absl::internal_any_invocable::RemoteInvoker<>() | |
| @ 0x7f93312143bf 368 thread::TMWorker() | |
| @ 0x7f9300827296 256 Thread::ThreadBody() | |
| @ 0x7f9d3032f0bf 192 start_thread | |
| @ 0x7f9d2f79067f (unknown) clone | |
| E0330 19:21:38.220636 104591 coredump_hook.cc:274] RAW: Remote crash gathering disabled for unit test. Enable that with --remote_coredump_enabled_for_unit_test flag | |
| E0330 19:21:45.685045 104591 process_state.cc:704] RAW: Raising signal 4 with default behavior | |
| /foo/tools/test/local-runtest.sh: line 209: 102457 Illegal instruction (core dumped) "$@" | |
| -- 2026-03-30 19:21:48 PDT Local runner: Test killed by SIGILL while running on h100 | |
| ================================================================================ |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment