4. The full source code and the models used

Purpose of this post

This post publishes all the source code used in the tile sliding puzzle AI experiments. The code is loaded only when you open a file.

Models used

• DQN: Stable-Baselines3. The observation is the 4×4 tile numbers, the actions are the four moves of the empty cell (up, down, left, right), and the reward is the negative sum of each tile's remaining Manhattan distance to its place. The endless repetition in post 1 came from this reward.
• Imitation learning (BC): the DQN policy was first trained on records solved by a person and on solutions generated by a C++ teacher (A*)
• Planner and Executor: two PyTorch CNNs (convolutions with 32→64 channels). The Planner picks the next tile to place, and the Executor picks the single move that moves that tile. The data was augmented with DAgger, which relabels positions where the policy was wrong with the teacher's answer.
• World model: simulates upcoming moves with the C++ engine and picks moves by the score of a ValueNet that predicts the board's progress (0 to 1).
• VLM: Qwen2.5-VL-3B-Instruct from Hugging Face. We gave it the puzzle board image and the list of possible moves and received the next move as JSON.

Supporting pieces

• C++ teacher (PuzzleTeacher, Planner, Executor): demo generation and the grading reference. In research mode, it does not overwrite the moves made by the agent.
• agent_loop_guard: a safeguard that, when DQN or the VLM repeats the same board, escapes with a move that reduces the Manhattan distance
• AI solver in the web game: not a trained model but a weighted A* search running in the browser (Manhattan distance + linear conflict)

Results of running them again

Even after 40 iterations, imitation learning stalled at 71.1% of moves matching the teacher.

We ran boards shuffled by 80 moves again under the same conditions. None of the trained models finished a single board; only the C++ Executor, which has the loop rule written in code, solved them all.

The Planner/Executor networks matched the teacher 83% of the time when picking the next tile and 40% when picking a single move, and could not even finish the first row. The world model solved all 10 boards, but the move sequence was identical even with the ValueNet switched off. What solved the boards was not the ValueNet but the C++ Executor running inside it.

We gave Qwen2.5-VL-3B the board picture and the possible moves and asked it directly at every move. It could not solve even boards shuffled by only 4 moves. In all 20 games it either went back and forth between two cells or kept rotating a 2×2 block of four cells forever. It knows how to turn a loop but not when to stop.

Folder layout

• src: Qt6/C++ puzzle engine, C API for Python, Planner/Executor/Teacher, Qt screen and TCP training server
• .py files at the root: Gymnasium environment, demo generation, DQN/BC/Planner-Executor/world model training, Qt connection client, evaluation simulations
• web: Next.js web puzzle screen

Server deployment settings, monitoring and automation scripts, training data and model weights, and the puzzle picture files are excluded from the release. Web operation code and configuration such as ads, and debugging tools are also left out, so some imports and the scripts paths in the README point to unpublished files.

Full source