run record~/research/cliste/runsAll Cliste runs

Type a section name and press Enter. Use “help” for the list.

Figures: softened for dark viewingDisplay only; image files are unchanged.
← All Cliste runs

Cliste experiment

Do recurring visual arrangements help find remembered experience?

Execution: completedScientific verdict: Conclusion reported in the findings and result narrative

The question

Acquire recurring visual pieces and arrangements from a larger stream of writing images and photographs, then compare their usefulness as episodic memory grows.

Implementation and comparison details

Keep visual occurrences and relative arrangements in the maintained native mind. Compare direct local-vector recall, acquired pieces, intact arrangements and disrupted arrangements at growing exposure counts.

What happened

Completed 2,048 encounters and final restart after repairing checkpoint memory use. On 282 held-out queries, arrangements score 38.7%, individual pieces 38.3%, direct recall 43.6%, and disrupted arrangements 15.2%. There is no established arrangement advantage over pieces. Shared-pattern matching handles grid-aligned relocation well, but fails many smaller shifts. 883 acquired arrangement handles recur across distinct sources.

Results and visual artifacts

Open a figure for its input, comparison, and result. Click an image to enlarge it.

How to read these pictures: the recall figures ask which previously stored image matches a changed or partially hidden view. Each example shows the query, the expected complete image, and the memories returned by three matching methods. The complete images were stored before the query; returning a whole picture is retrieval, not generation of the hidden part. Sentences and photographs are separate visual examples, not caption–photo pairs. The storage-quality figures ask a different question: how much detail survives encoding, storing and decoding a known image, without searching for it. Neither kind of figure shows discovered letters or meanings. Open a figure for the case-by-case explanation and larger images.

Which stored image matches this changed view? Final examples
Which stored image matches this changed view? Final examples
The test first stored 48 candidate images: 24 originals and a deliberately rearranged copy of each. It then presented changed views and asked each method to select a memory. Direct matching compares visual vectors; pieces match acquired local patterns; arrangements also use their relative positions. Returned images are decoded from the chosen memories. In the first two examples, all three methods retrieve the complete sentence or shoreline photograph from a partially hidden query. In the third, a one-pixel movement makes pieces and arrangements select an unrelated, deliberately reversed sentence; direct matching succeeds. In the fourth, moving the dog photograph 16 pixels right and eight down makes direct matching choose its swapped-halves copy; pieces and arrangements find the intact original. Gray covers unavailable input. Orange outlines mark up to three contributing pattern locations in the returned memory, not word or object boundaries. These four selected illustrations are not a representative sample: across all 282 queries, direct matching scored 43.6%, pieces 38.3%, and arrangements 38.7%.

Original recorded figure ↗

What detail survives storage? Four final test images
What detail survives storage? Four final test images
Each row shows an original image, its decoded stored vector, and magnified pixel differences. This check directly fetches the known correct memory; it does not ask Cliste to find it. The sentence pictures are inputs made of pixels, not written responses or captions for the photograph. Text remains recognizable to a human but is visibly softened and distorted. The difference image shows four times the absolute RGB difference, clipped at full brightness: dark means close agreement, bright means a larger mismatch. The scale is fixed across all rows. These examples test the frozen encoder, 8-bit vector storage and decoder; they do not show what the acquired pattern vocabulary contains or whether it improves recall.

Original recorded figure ↗

Which stored image matches this changed view? After 1,024 encounters
Which stored image matches this changed view? After 1,024 encounters
After 1,024 learning encounters, each method searches the same separate pool of 32 reference memories: 16 originals and 16 rearranged alternatives. Each case shows the query, the expected original, and the images selected by direct vector matching, acquired pieces, and acquired arrangements. A full recalled image was stored whole beforehand. Gray areas are hidden from the query; orange boxes mark contributing pattern locations in the selected memory. Deliberately reordered writing and swapped photograph halves are candidate images supplied by the experiment, not text or edits generated during recall. The cases illustrate selected successes and failures rather than the average outcome.

Original recorded figure ↗

What detail survives storage? Reference images after 1,024 encounters
What detail survives storage? Reference images after 1,024 encounters
Original pixels, decoded stored pixels, and fourfold magnified pixel differences for the same four reference images. This directly reads the known correct memory and does not test which memory is selected. Dark error pixels indicate close agreement, and bright pixels indicate larger differences; all rows use the same clipped scale. The visual encoder and decoder remain fixed throughout acquisition. The figure checks their storage/readback path, not whether the additional acquired patterns improve recognition or reveal letters and meanings.

Original recorded figure ↗

Which stored image matches this changed view? After 2,048 encounters
Which stored image matches this changed view? After 2,048 encounters
This is the last evaluation on the unchanged reference panel: 16 original images plus their 16 rearranged alternatives. It is separate from the final test on 24 different sources. Each case shows a changed or partially hidden input, the original image that should be found, and the memories returned by direct vector matching, individual pieces, and arrangements. Gray marks hidden input; orange outlines mark contributing pattern locations in returned memories. The outputs are selections from previously stored complete images, not generated completions or written answers. These selected examples should be read alongside all 60 reference queries: arrangements scored 60.8%, pieces 63.3%, and direct matching 75.0%.

Original recorded figure ↗

What detail survives storage? Reference images after 2,048 encounters
What detail survives storage? Reference images after 2,048 encounters
These are the same four reference sources used in the earlier storage checks. Each row shows original pixels, the known correct stored image decoded back to pixels, and its magnified error. No search is performed, and the encoder and decoder weights have not been trained during these 2,048 encounters. Bright error pixels mean larger absolute RGB differences, multiplied by four and clipped; dark means closer agreement. This checks image fidelity through vector storage, independently of whether acquired pieces or arrangements help choose the correct memory.

Original recorded figure ↗

Which stored image matches this changed view? After 256 encounters
Which stored image matches this changed view? After 256 encounters
The learned vocabulary now comes from 256 encounters. The evaluation again uses the same 16 reference images and their 16 deliberately rearranged copies; it does not search all 256 acquired episodes in this figure. Each example shows what is presented now, the complete original that should be selected, and the returned memories from direct vector matching, individual pieces, and arrangements. Gray hides unavailable input. Orange outlines show contributing pattern locations in a selected memory. Full images are retrieved from storage, not completed or generated. These selected examples include partial views and cases where arrangements lose or gain against direct matching; consult the full query results for their frequency.

Original recorded figure ↗

What detail survives storage? Reference images after 256 encounters
What detail survives storage? Reference images after 256 encounters
The same four reference images are shown as original pixels, decoded stored vectors, and magnified pixel differences. The known correct image is fetched directly; there is no recall search in this figure. Encoding and decoding weights have not changed with the additional encounters, so this is not a training-progress image. Bright error pixels indicate larger absolute RGB differences, multiplied by four and clipped; dark pixels indicate close agreement. The fixed scale makes the storage distortion visible, including softer writing.

Original recorded figure ↗

Which stored image matches this changed view? After 64 encounters
Which stored image matches this changed view? After 64 encounters
After 64 learning encounters, the test separately stores 16 reference images and a rearranged copy of each, giving 32 candidate memories. Each card shows a changed or partially hidden query, the expected complete image, and the images selected by direct vector matching, acquired pieces, and acquired arrangements. The complete picture returned was already in memory; no missing content is generated. Writing is visual input, not a captioning task or a written response. Gray marks hidden input. Orange outlines show contributing pattern locations in the returned memory, not detected letters or objects. The displayed cases follow the original selection rule: a partial writing image, a partial photograph, and the least and most favorable comparisons for arrangements against direct matching, with duplicates removed. At this checkpoint, no query favors arrangements over direct matching. The examples illustrate behavior; the full query panel supplies the overall scores.

Original recorded figure ↗

What detail survives storage? Reference images after 64 encounters
What detail survives storage? Reference images after 64 encounters
Read each row as original image, decoded stored image, and magnified pixel differences. The correct memory is fetched directly without a search. These four reference sources are the same ones shown at later learning checkpoints, and the encoder and decoder stay fixed. The writing is an input image, not language generated by Cliste. Bright areas in the last column mark larger RGB differences, multiplied by four and clipped; dark means close agreement. This is a storage-quality check, separate from the recall comparisons and from learning recurring patterns.

Original recorded figure ↗

Limits and next question

Writing enters as pixels; no letters, word boundaries or meanings are demonstrated. The codec stays fixed. Position robustness is limited by eight-pixel sampling alignment. Local and disrupted banks reach capacity. Shared state adds storage; compression is not demonstrated. The original failed attempt and checkpoint-only repair remain visible.

Next question. Which part of perception or matching causes the sharp loss under small shifts, and can it be improved while retaining the demonstrated shared-pattern recall?

Inspect the evidence

Download the methods and measurements for this experiment. Large training and saved-state files are not included.