Created
July 31, 2026 19:46
-
-
Save sigmaris/0a6e62533d5c2cfdc72ac78e211dbee4 to your computer and use it in GitHub Desktop.
Testing Ultralytics QNN export
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| (venv) debian@trixie86:~/yolo-exporter$ python3 export.py | |
| Ultralytics 8.4.106 🚀 Python-3.13.5 torch-2.13.0+cu130 CPU (Intel Core i9-9900K 3.60GHz) | |
| WARNING ⚠️ qnn export requires INT8 quantization, enabling it. | |
| WARNING ⚠️ QNN export does not support end2end models, disabling end2end branch. | |
| YOLO26n summary (fused): 146 layers, 2,562,496 parameters, 0 gradients, 5.4 GFLOPs | |
| PyTorch: starting from 'yolo26n.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 84, 8400) (5.3 MB) | |
| ONNX: starting export with onnx 1.22.0 opset 20... | |
| ONNX: slimming with onnxslim 0.1.94... | |
| ONNX: export success ✅ 1.7s, saved as 'yolo26n.onnx' (9.5 MB) | |
| Qualcomm QNN: collecting INT8 calibration images from 'data=coco.yaml' | |
| val: Fast image access ✅ (ping: 0.8±0.5 ms, read: 206.2±87.2 MB/s, size: 117.3 KB) | |
| val: Scanning /home/debian/yolo-exporter/datasets/coco/labels/val2017.cache... 4952 images, 48 backgrounds, 0 corrupt: 100% ━━━━━━━━━━━━ 5000/5000 1.3Git/s 0.0s | |
| Qualcomm QNN: starting W8A16 quantization and export with ONNX Runtime QNN (HTP arch 68)... | |
| WARNING:root:The original model opset version is 20, which does not support 16-bit integer quantization natively. Please update the model to opset >= 21. Automatically update the model to opset 21. Please verify the quantized model. | |
| # LIBNATIVE: No alternate configuration present in libnative_tls.conf or from LIBNATIVE_CONFIG environment variable - using default thread-local behavior. | |
| Starting stage: Graph Preparation Initializing | |
| Completed stage: Graph Preparation Initializing (2158 us) | |
| Starting stage: Graph Optimizations | |
| Completed stage: Graph Optimizations (434272 us) | |
| Starting stage: Post Graph Optimization | |
| Completed stage: Post Graph Optimization (24972 us) | |
| Starting stage: Graph Sequencing for Target | |
| [##################################################] 100% | |
| Completed stage: Graph Sequencing for Target (1006695 us) | |
| Starting stage: VTCM Allocation | |
| Completed stage: VTCM Allocation (48554 us) | |
| Starting stage: Parallelization Optimization | |
| Completed stage: Parallelization Optimization (12385 us) | |
| Starting stage: Finalizing Graph Sequence | |
| ====== DDR bandwidth summary ====== | |
| spill_bytes=3932160 | |
| fill_bytes=4341760 | |
| write_total_bytes=5345280 | |
| read_total_bytes=10246144 | |
| Completed stage: Finalizing Graph Sequence (13974 us) | |
| Starting stage: Completion | |
| Completed stage: Completion (586 us) | |
| Qualcomm QNN: export success ✅ 676.3s, saved as 'yolo26n_qnn.onnx' (3.5 MB) | |
| Export complete (676.7s) | |
| Results saved to /home/debian/yolo-exporter/yolo26n_qnn.onnx | |
| Predict: yolo predict task=detect model=yolo26n_qnn.onnx imgsz=640 | |
| Validate: yolo val task=detect model=yolo26n_qnn.onnx imgsz=640 data=/home/lq/codes/ultralytics/ultralytics/cfg/datasets/coco.yaml | |
| Visualize: https://netron.app |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| (.venv) root@q6a:~/yolo-exporter# python3 test_qnn_context_binary.py | |
| 2026-07-31 19:46:31.827695941 [W:onnxruntime:Default, device_discovery.cc:285 GetGpuDevices] Failed to detect devices under "/sys/class/drm/card1": device_discovery.cc:94 ReadFileContents Failed to open file: "/sys/class/drm/card1/device/vendor" | |
| 2026-07-31 19:46:31.848527743 [W:onnxruntime:, qnn_execution_provider.cc:666 QnnEp] File mapped weights feature is only available on Windows arm64 devices for QNN API versions >= 2.32. Feature will be disabled by default | |
| 2026-07-31 19:46:31.855612661 [W:onnxruntime:Default, qnn_backend_manager.cc:326 operator()] Setting ADSP_LIBRARY_PATH = /root/.venv/lib/python3.13/site-packages/onnxruntime_qnn | |
| /local/mnt/workspace/mlg_user_admin/QAIRT/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc | |
| 2026-07-31 19:46:32.027578321 [E:onnxruntime:, onnx_ctx_model_helper.cc:309 LoadQnnCtxFromOnnxGraph] Failed to load from EpContext model. Failed to create context from binary. Error: QNN_CONTEXT_ERROR_CREATE_FROM_BINARY: Failure to create context from binary | |
| Traceback (most recent call last): | |
| File "/root/yolo-exporter/test_qnn_context_binary.py", line 11, in <module> | |
| session = ort.InferenceSession("yolo26n_qnn.onnx", sess_options=options) | |
| File "/root/.venv/lib/python3.13/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 534, in __init__ | |
| self._create_inference_session(providers, provider_options, disabled_optimizers) | |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
| File "/root/.venv/lib/python3.13/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 640, in _create_inference_session | |
| sess.initialize_session(providers, provider_options, disabled_optimizers) | |
| ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
| onnxruntime.capi.onnxruntime_pybind11_state.InvalidGraph: [ONNXRuntimeError] : 10 : INVALID_GRAPH : Failed to load from EpContext model. Failed to create context from binary. Error: QNN_CONTEXT_ERROR_CREATE_FROM_BINARY: Failure to create context from binary | |
| /local/mnt/workspace/mlg_user_admin/QAIRT/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| (.venv) root@q6a:~/yolo-exporter# yolo predict task=detect model=yolo26n_qnn.onnx source=../onnx-models/images/bus.jpg | |
| Ultralytics 8.4.106 🚀 Python-3.13.14 torch-2.13.0+cu130 CPU (aarch64) | |
| 2026-07-31 19:39:36.630788127 [W:onnxruntime:Default, device_discovery.cc:285 GetGpuDevices] Failed to detect devices under "/sys/class/drm/card1": device_discovery.cc:94 ReadFileContents Failed to open file: "/sys/class/drm/card1/device/vendor" | |
| Loading yolo26n_qnn.onnx for Qualcomm QNN inference... | |
| 2026-07-31 19:39:36.680986030 [W:onnxruntime:, qnn_execution_provider.cc:666 QnnEp] File mapped weights feature is only available on Windows arm64 devices for QNN API versions >= 2.32. Feature will be disabled by default | |
| 2026-07-31 19:39:36.687202124 [W:onnxruntime:Default, qnn_backend_manager.cc:326 operator()] Setting ADSP_LIBRARY_PATH = /root/.venv/lib/python3.13/site-packages/onnxruntime_qnn | |
| /local/mnt/workspace/mlg_user_admin/QAIRT/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc | |
| 2026-07-31 19:39:36.847092005 [W:onnxruntime:, qnn_backend_manager.cc:1939 SetupBackend] Unable to get platform info: Failed to get HTP arch. | |
| 2026-07-31 19:39:36.853364607 [E:onnxruntime:, onnx_ctx_model_helper.cc:309 LoadQnnCtxFromOnnxGraph] Failed to load from EpContext model. Failed to create context from binary. Error: QNN_CONTEXT_ERROR_CREATE_FROM_BINARY: Failure to create context from binary | |
| Traceback (most recent call last): | |
| File "/root/.venv/bin/yolo", line 6, in <module> | |
| sys.exit(entrypoint()) | |
| ~~~~~~~~~~^^ | |
| File "/root/.venv/lib/python3.13/site-packages/ultralytics/cfg/__init__.py", line 1119, in entrypoint | |
| getattr(model, mode)(**overrides) # default args from model | |
| ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^ | |
| File "/root/.venv/lib/python3.13/site-packages/ultralytics/engine/model.py", line 536, in predict | |
| self.predictor.setup_model(model=self.model, verbose=is_cli) | |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
| File "/root/.venv/lib/python3.13/site-packages/ultralytics/engine/predictor.py", line 410, in setup_model | |
| self.model = AutoBackend( | |
| ~~~~~~~~~~~^ | |
| model=model or self.args.model, | |
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
| ...<5 lines>... | |
| verbose=verbose, | |
| ^^^^^^^^^^^^^^^^ | |
| ) | |
| ^ | |
| File "/root/.venv/lib/python3.13/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context | |
| return func(*args, **kwargs) | |
| File "/root/.venv/lib/python3.13/site-packages/ultralytics/nn/autobackend.py", line 223, in __init__ | |
| self.backend = self._BACKEND_MAP[format](model, **backend_kwargs) | |
| ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^ | |
| File "/root/.venv/lib/python3.13/site-packages/ultralytics/nn/backends/base.py", line 83, in __init__ | |
| self.load_model(weight) | |
| ~~~~~~~~~~~~~~~^^^^^^^^ | |
| File "/root/.venv/lib/python3.13/site-packages/ultralytics/nn/backends/qnn.py", line 55, in load_model | |
| self.session = onnxruntime.InferenceSession(str(onnx_file), sess_options=options) | |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
| File "/root/.venv/lib/python3.13/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 534, in __init__ | |
| self._create_inference_session(providers, provider_options, disabled_optimizers) | |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
| File "/root/.venv/lib/python3.13/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 640, in _create_inference_session | |
| sess.initialize_session(providers, provider_options, disabled_optimizers) | |
| ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
| onnxruntime.capi.onnxruntime_pybind11_state.InvalidGraph: [ONNXRuntimeError] : 10 : INVALID_GRAPH : Failed to load from EpContext model. Failed to create context from binary. Error: QNN_CONTEXT_ERROR_CREATE_FROM_BINARY: Failure to create context from binary | |
| /local/mnt/workspace/mlg_user_admin/QAIRT/SNPE_SRC/avante-tools/prebuilt/dsp/hexagon-sdk-5.5.5/ipc/fastrpc/rpcmem/src/rpcmem_android.c:42:dummy call to rpcmem_deinit, rpcmem APIs will be used from libxdsprpc |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment