Your AI chip may be waiting on a detour

Follow the route the model actually takes

A device can contain an AI accelerator without every part of your model running there. Google and Arm's October 20 Model Explorer walkthrough is useful because it makes the route visible. It describes adapters for compiler representations and shows examples of inspecting exported graphs, checking delegation, and attaching execution traces to individual nodes.

One reported example found unwanted transpose operations after an export change. Another used a side-by-side graph view to compare CPU and NPU execution structure. A profiling example connected simulator cycle data to a costly node. These are examples from the developers' write-up, not performance measurements made by Signal Scroll.

The practical lesson is to separate the model you intended to deploy from the artifact that actually executes. Between the two, conversion and compilation can change the structure. A hardware specification alone does not tell you whether that path is efficient for your particular model.

This article proposes a debugging notebook for a small on-device AI project. It is a methods explainer rooted in the new walkthrough, not a claim that we ran a board or reproduced Arm's results. You can begin by reading a documented example before installing anything.

Imagine a hypothetical image-classification demo that feels slower after a dependency update. The tempting response is to look for a larger accelerator or a smaller model. A better first question is what changed in the execution route. Did the exported artifact change? Did the device assignment change? Did the measurement include different work? A good investigation starts by making those possibilities distinguishable.

Source: Google Developers: Visual graph debugging with Model Explorer ↗ · Google AI Edge: Model Explorer features and local use ↗ · Arm: TOSA adapter for Model Explorer ↗ · Arm: PTE adapter for Model Explorer ↗ · HUANUO: HNDS6 dual monitor mount documentation ↗

Keep three artifacts separate

For your proposed notebook, identify the source model, the exported representation, and the runtime measurement. Give each a clear filename or identifier and record how it was produced. This is our organizing method. The purpose is to prevent a screenshot from one version being paired with timing from another.

The source model describes the starting point. Record the exact model version and intended input. The exported artifact is the thing produced by your conversion workflow. Record the exporter, relevant settings, and dependency versions. The runtime measurement describes what happened when that artifact was executed under specified conditions.

Do not call two runs comparable merely because the model name matches. In the hypothetical regression, the application might have changed image preparation while the model itself stayed untouched. Or a compiler update might have altered the exported structure. A useful note makes those differences visible before anyone debates hardware.

Start with one known input and a stable expected output. Keep it non-sensitive and suitable for your learning environment. If the output changes after conversion, investigate correctness before celebrating a timing improvement. Faster wrong answers do not satisfy the same job.

For a beginner, these records can fit on one page. The page should answer what file you looked at, what settings produced it, and what timing belongs to it. Avoid filling it with information that nobody will use. Add detail when it resolves an actual ambiguity.

This discipline also helps when asking for support. Another developer can inspect a specific artifact and question instead of trying to reconstruct a vague report that the AI feels slow. The best debugging note makes a small problem reproducible enough to discuss without exposing private application data.

Use the right viewer for the artifact

Google's Model Explorer documentation describes search, side-by-side views, node-specific data overlays, and inspection of tensor metadata. The local tool runs in a browser window, and the documentation says local usage keeps its data local. That description is not a blanket claim about every notebook, extension, or workflow you might connect to it.

The new walkthrough links separate adapters because file formats matter. Arm's TOSA adapter repository specifies its supported Python range and setup. The PTE adapter documents visualization for particular ExecuTorch targets and lists platform requirements. Check the adapter for the artifact you actually have rather than installing every related package.

For a proposed first session, use an isolated development environment and the current official instructions. Note the versions you installed. Open a documented sample or an artifact you are allowed to inspect. If the viewer cannot parse it, preserve the error and check the format and version assumptions before editing the model.

A viewer failure is not evidence that the model is bad. It may simply mean the artifact and adapter do not match. Conversely, a beautifully rendered graph is not proof that execution is correct or fast. The graph is a source of questions and evidence about structure.

Write down one question before exploring. For example: did this export add layout-changing operations around a particular block? Search for the relevant nodes and inspect their relationships. Beginning with a question makes the view more useful than wandering through thousands of boxes.

You can also learn from the official screenshots without running the tool. Describe which structural difference the screenshot supports and which performance claim would require additional measurements. That is a worthwhile exercise for anyone trying to understand what on-device AI tooling actually tells them.

Compare structure before reaching for a stopwatch

In our hypothetical regression, save a baseline export and a changed export produced from the same source model. Compare them in the viewer using a written question. Look for additions, removals, changed shapes, and boundaries between execution groups. Do not assume that fewer visible boxes automatically means less work; grouping can change how the graph is displayed.

A useful comparison note has three parts: observed difference, possible consequence, and next check. For example, you might observe an additional layout-changing operation. The possible consequence is extra work or a different execution path. The next check is to establish whether it actually runs in your measured workload and how much time it contributes.

Keep observation and inference in separate sentences. The graph contains an extra node is narrower than the extra node caused the slowdown. That causal claim needs more evidence. This is the same discipline that makes a good bug report: show what changed, explain the hypothesis, and identify the test that could reject it.

Also compare outputs. A compiler transformation may legitimately change structure while preserving intended behavior. The point of inspection is not to demand identical graphs. It is to understand whether a change is expected and whether the result still meets the application's requirements.

For the first investigation, resist changing multiple settings at once. If you change the model, exporter, input, and runtime together, you make attribution much harder. Choose one suspect difference and hold the rest as stable as your environment allows.

End the comparison with a decision about what to measure next. If you found no relevant structural difference, that is still progress. It redirects attention toward runtime conditions or application stages rather than encouraging you to invent a graph explanation because the viewer is open.

Ask where the work runs, then ask what it costs

Delegation is the assignment of model work to a backend or accelerator. In the proposed notebook, record the intended target and the evidence that supports the observed assignment. Do not infer complete accelerator use merely because the device includes an NPU or the application advertises on-device AI.

If a graph is partitioned, identify the boundaries you can inspect and ask what they imply for your workload. The developer walkthrough shows how a side-by-side view can reveal different delegation structures. For your own project, treat that as a technique to investigate, not a prediction that your graph will look the same.

Then connect structure to measured cost. A node can look complicated and contribute little to end-to-end latency; an ordinary-looking transformation can matter more. Use a trace or other appropriate measurement from your actual environment when available. If you only have a simulator result, label it as such rather than reporting it as device timing.

Here is a hypothetical arithmetic exercise. Suppose a controlled run takes 120 milliseconds, and a particular stage accounts for 12 milliseconds. Even removing that stage completely would leave 108 milliseconds if everything else stayed equal. Those invented numbers are for reasoning, not a benchmark. They help you judge whether a proposed optimization can plausibly solve your problem.

Now consider the user-facing path. Loading an input, preparing it, running inference, and displaying a result may have separate costs. Record which interval you measured. An inference-only improvement does not automatically produce the same percentage improvement in the full application.

The goal is a causal story small enough to challenge. This artifact contains this structure, this workload exercises it, this measurement indicates a cost, and this change is intended to reduce that cost while preserving output. Each link deserves evidence.

Make the comparison comfortable without making it a shopping project

Model Explorer's split view works in a single tab. Begin there, or use ordinary window tiling to keep the graph beside your notes. The important comparison is between artifacts, not between expensive desk setups.

If you already use two compatible displays, a HUANUO FlowLift dual monitor desk mount (affiliate link) is an optional way to arrange a graph and profiling notes; check the exact model's display weight, VESA and desk mounting requirements in HUANUO's documentation first. No mount or second monitor is needed for the debugging method.

As an Amazon Associate I earn from qualifying purchases. This optional recommendation is not a claim of hands-on testing.

We have not tested that mount, established fit for your desk, or measured any productivity benefit. The manufacturer documentation is a place to verify requirements, not evidence that a particular reader's setup is suitable. Keep the hardware decision separate from the AI performance question.

For a useful working layout, place the question and artifact identifiers where you can see them while exploring. Keep the baseline and changed versions clearly named. A misplaced screenshot can make a careful investigation tell the wrong story later.

Use annotations sparingly. Mark the node or boundary relevant to your hypothesis, and write what the annotation means in the accompanying note. A screenshot full of colors can be impressive while explaining very little. Someone reading it tomorrow should be able to identify the one observation you wanted them to inspect.

If your screen is small, inspect one region at a time and save a concise written comparison. The method remains the same. You need a stable reference, a clear question, and a record of what you observed. Extra display space is optional; clarity is the actual requirement.

Finish with a regression story another person can check

A good outcome is not necessarily a dramatic speedup. It may be a precise explanation of why a change is harmless, why a suspected bottleneck is too small to matter, or why the measurement needs to be redesigned. Those are useful results because they stop unproductive optimization.

For the hypothetical demo, write the final note in a compact sequence. State the user-visible problem. Identify the baseline and changed artifacts. Describe the observed structural difference. Report the measured interval and conditions. Explain the proposed fix and the correctness check. List what remains uncertain.

If you make a change, rerun the relevant comparison under the same conditions and preserve the earlier result. Do not report a single favorable run as a universal improvement. Note whether the change affected startup, repeated inference, or the full application. Those distinctions help a collaborator decide whether the improvement matters for their use.

Also record when you decide not to optimize. A small suspected cost may not justify introducing a fragile dependency or a more complicated export process. A readable graph can help you make that trade-off rather than encouraging optimization for its own sake.

For learners, the best first project is a public sample with a question you can answer. Can you identify inputs and outputs? Can you explain a delegation boundary? Can you distinguish a structural observation from a timing result? You do not need to own a new AI board to practice those skills.

The new Google and Arm walkthrough makes an important part of edge AI more inspectable. The durable habit is to follow the actual execution route before blaming the chip. A model name, an accelerator label, and a stopwatch each tell only part of the story. Put them together with traceable artifacts, and your next performance question becomes much easier to answer.