Where remembered composition loses accuracy

The question

When a remembered image is rearranged, how much accuracy is lost by substituting learned patterns, and how much more is lost when those patterns must be recovered from a VSA bundle?

Why this experiment?

A nearby visual pattern is not necessarily the right replacement for a requested change. These comparisons test whether selection helps the resulting operation.

How it works

Supply two feature positions to exchange. Compare how candidate patterns are selected, then decode the edited bundle against an observer-only clean-feature exchange. This is a local feature operation, not a command to move a complete object.

What happened

On the 64 previously unevaluated requests, clean-selected patterns reduced mean regional squared RGB error by 46.3% versus leaving memory unchanged: 0.000382 versus 0.000711. Bundle-selected patterns reduced it by 12.6%, to 0.000621. Removing selection noise therefore improved average fidelity, but the remaining pattern substitution still lost useful distinctions.

Clean selection improved 28 requests, worsened 19 and left 17 unchanged. Bundle selection improved 21, worsened 33 and left 10 unchanged. Neither met the predefined reliability benchmark. Eleven clean-selected edits and 21 bundle-selected edits at least doubled the error. Large absolute gains on some requests can coexist with large relative losses on smaller intended changes.

The paired comparison supported an additional error cost from bundle selection. The separate reused panel showed the same direction: 78.2% lower mean error with clean selection and 48.6% lower with bundle selection, but neither met its benchmark. Reused and previously unevaluated photographs were not pooled.

A later inspection found that 17 clean-selected requests mapped both feature positions to the same learned pattern, erasing the distinction needed for the exchange. Bundle recovery chose the clean features’ nearest pattern in only 38 of 128 constituent selections. These observations identify problems in both nearest-pattern substitution and recovery; they do not show that adding more patterns alone would solve them.

Independent numerical and preservation checks passed, including 576 pixel and edge measurements. Pixels outside the selected regional core, the retained state and the codec remained exact during readback. Automatic constituent substitution remains disabled.

Limits and next question

These are supplied feature-position exchanges with overlapping pixel receptive fields. The exact-feature reference is not an edited ground-truth photograph or evidence of object movement. The learned bank contains patterns from five earlier photographs.

Clean selection receives information unavailable to the operative memory path. Its nearest-cosine pattern is not guaranteed to be the best decoded substitute. The conclusion applies to this bank and selection rule; it does not rule out other ways to compose VSA representations.

Four requests share each source photograph. Uncertainty was estimated by resampling whole photographs, and the recovery-penalty interval is wide. The comparison distinguishes selection paths without establishing an additive causal decomposition of every source of error.

Next question. Which distinctions needed for an operation survive pattern acquisition and nearest-pattern substitution? Keep clean-feature selection as a diagnostic control while investigating the vocabulary or its selection objective, then retest bundle recovery.