5. Code computes the spots, the LLM only chooses

Method

llm_macro_client.py is a client that attaches to the Qt screen.

1. From the received board, it erases the moving piece so only locked blocks remain.
2. It tries dropping the current piece at every rotation and every x. For a T piece that is 34 spots. Spots with negative x are included too.
3. For each spot it computes lines cleared, new holes, cells stacked over holes, bumpiness, maximum height, landing height and wells, and assigns a score.
4. It places the next piece once more, adds 0.7× its best score and re-ranks.
5. It shows the top 6 to the LLM (Qwen2.5-3B GGUF) along with a text board and has it pick only a number. The LLM cannot make up coordinates.
6. It sets the rotation and movement for the chosen spot and hard-drops the piece.

With --no-llm, it uses the first choice without the LLM.

The heuristic alone

The C API does not provide the next piece, so this test skipped step 4 (looking one piece ahead) and used only the first choice. 20 games with the same seeds, stopped at 1,000 pieces.

• 940 pieces and 370 lines on average
• 16 of 20 games survived to 1,000 pieces. The other 4 ended at 375, 736, 811 and 863 pieces.
• Of the 16 games that reached 1,000, 12 ended with a stack height of 6 cells or less.

With a single scoring formula and no training, it far outperforms DQN (0 lines) and MoE (12 lines).

When the LLM chooses

With the LLM attached, we ran one game with seed 0 up to 150 pieces. It was kept short because the GPU is shared with Gomoku training.

• It picked the first choice 147 of 150 times (98%). The second choice twice and the third once.
• It ended with 55 lines, 0 holes and a height of 7 cells. Using only the heuristic's first choice under the same conditions gives 58 lines.
• Each choice took 1.7 seconds on average. The heuristic alone plays 20 games, more than 18,000 pieces, in 14 seconds.

The prompt says "pick 0 if [0] satisfies the conditions", and the candidates are already sorted by score, so the LLM almost always follows the heuristic. In the current structure, the LLM does not improve the board; it only makes things slower.

Summary

• Low-level DQN: hardly ever sees a line reward, so it is the same as random.
• Macro DQN: the action encoding blocked the leftmost column, and it repeats only two spots in the middle.
• VLM rewards and direct VLM choice: cannot tell boards apart and stack in the middle.
• Candidate computation and scoring: passes 1,000 pieces without training. The LLM does little more than confirm the first choice.

Next, I plan to change the macro actions to the 162 spots where pieces can actually be placed, and retrain the DQN using boards played by the heuristic planner as examples.

The actual code for this step

This is the score of a single spot.

on_hole_w = 3.0 * max(0.2, 1.0 - max_h_global / 16.0)

score = (
    + 1.0 * cleared
    - 4.0 * new_holes
    - on_hole_w * cells_above_hole
    - 0.36 * holes_after
    - 1.00 * max_h            # 0.5 → 1.0 (높이 누적 강하게 회피)
    - 0.30 * bump_eff         # 0.18 → 0.30 (양쪽 우물 패턴 회피)
    - 0.70 * landing_height   # 0.5 → 0.7 (깊게 박기 우선)
    - 0.40 * other_well_sum
    + 0.30 * well_bonus       # 0.5 → 0.3 (테트리스 욕심을 줄임)
)

It tries dropping the current piece at every possible spot.

def enumerate_candidates(board: np.ndarray, ptype: int) -> list[dict]:
    if ptype == 0:
        return []
    heights_before = column_heights(board)
    hole_cols_before = column_hole_count(board, heights_before)
    holes_before = sum(hole_cols_before)

    nrot = _PIECE_NROT[ptype]
    w = board.shape[1]
    cands = []
    for rot in range(nrot):
        cells = PIECE_CELLS[ptype][rot]
        min_dx = min(dx for dx, _ in cells)
        max_dx = max(dx for dx, _ in cells)
        x_lo = -min_dx
        x_hi = w - 1 - max_dx
        for x in range(x_lo, x_hi + 1):
            r = evaluate_placement(
                board,
                ptype,
                rot,
                x,
                heights_before=heights_before,
                hole_cols_before=hole_cols_before,
                holes_before=holes_before,
            )
            if r is not None:
                cands.append(r)
    cands.sort(key=lambda r: r["score"], reverse=True)
    return cands

These are the selection rules given to the LLM.

"Selection priorities (STRICT, in order):\n"
"  1) on_hole == 0  (NEVER stack on a column that already contains buried holes)\n"
"  2) new_holes == 0 (do not create new holes)\n"
"  3) cleared >= 1 (clear lines if possible)\n"
"  4) higher 'look' (1-step lookahead) wins ties\n"
"  5) lower max_h, then lower bump, then lower land_h.\n"
"If [0] already satisfies (1)+(2), just pick 0.\n"
f'Reply EXACTLY as: {{"choice": <integer 0..{len(candidates) - 1}>}}'

The next post is "6. The full source code and the models used".