7. Fixing the Go heuristic

The original heuristic

The Go heuristic started by placing a stone in the centre and kept playing only in the centre, coiling around like a snake. It often won by answering the AI's moves and drawing out the AI's odd reactions at the time. It was closer to a lump of stones than to intelligence.

Fixing it against R5

So we started fixing the heuristic against Mixed R5, the most recently trained model. The heuristic took Black and R5 took White; they played up to move 80 and then the game was counted. We made five fixes in one day.

• Tactics: capture stones that can be captured, save stones in atari and avoid moves that walk into atari.
• Opening: for the first 70 moves, play the 3rd and 4th lines in the corners first and avoid the 1st and 2nd lines and pointless scattering in the centre. The centre coil disappeared here.
• Influence: prefer moves loosely connected to your own stones, such as one-space jumps, knight's moves and two-space extensions.
• Vital points: if there is a stone in atari that can be saved, save it before anything else.
• Mistakes seen in R5 games: penalise moves into a tiger's mouth, futile ladders, filling your own territory and nearby double ataris.

After the fixes the heuristic quickly found its feet, and it soon seemed to play better than R5. Because of the 80-move limit White still won on komi, but it was almost neck and neck. The last version lost 21 of 22 games, but R5's median lead was 5 points, smaller than the 6.5-point komi. In 12 of the 22 games the heuristic was ahead before komi. Without komi, R5 would actually have lost.

Raising the move limit and letting them learn from each other

So we raised the move limit to 150 and ran a 100-game match. In the 78 games that were recorded, the heuristic led 47 wins to 31 losses.

Next, we fed the game contents straight back into the heuristic and R5. From the moves one side played, only captures, moves that saved stones in atari and ladders that really captured were picked and passed to the other side as bonus points. PASS, suicide, tiger's mouths and futile ladders are not passed on, and the winning side's moves count double. After letting them learn from each other this way, 189 games at 150 moves gave the heuristic 125 wins and 64 losses.

We are now running 40 games with the limit raised to 200 moves. The results are not in yet, but surprising changes have started to appear in both the heuristic and R5. Of the 7 games finished so far, the heuristic won 6.

The heuristic became the teacher

Counting the bonus points passed in each direction, far more went from the heuristic to R5. R5 learned captures and ladders from the heuristic, while the heuristic mainly learned from R5 how to save stones in atari.

In effect, the heuristic is acting as a teacher for R5. In a way, a heuristic written as rules is producing better moves than the trained AI.

Every game is a reference count that stops at the move limit and counts only territory and captured stones. The life and death of dead stones was not judged.

The actual code for this step

This part stacks one side's good moves into the other side's table. Black's (the heuristic's) moves go to R5, White's (R5's) moves go to the heuristic, and the winner's moves count double.

def absorb_move(book: dict, actor: str, tags: Iterable[str], winner: Optional[str]) -> None:
    """actor 가 둔 좋은 수를 상대 테이블에 넣는다. actor 는 black|white."""
    tags = [t for t in tags if t in ("capture", "atari_save", "working_chase")]
    if not tags:
        return
    if actor == "white":
        target = "to_heuristic"
        weight = 2 if winner == "white" else 1
    else:
        target = "to_r5"
        weight = 2 if winner == "black" else 1
    _add_tags(book, target, tags, weight)

This part puts the mistakes found in R5 games into the heuristic's score. Tiger's mouths and futile ladders get a heavy penalty, and ladders that really capture get bonus points in proportion to what was learned.

def r5_live_delta(board, r: int, c: int, who: int) -> int:
    cells = _cells(board)
    score = 0
    hogu = hogu_verdict(cells, r, c, who)
    if hogu == "hogu_entry":
        score -= 9000
    elif hogu == "hogu_fill":
        score -= 2000
    if (r, c) in _futile_points(cells, who):
        score -= 14000

    if _puts_opponent_in_atari(cells, r, c, who):
        verdict = chase_verdict(cells, r, c, who)
        if verdict is not None:
            if verdict.kind == "working_chase":
                score += 2500 + verdict.group_size * 80
                score += int(_bonus(load_book(), "to_heuristic", ("working_chase",), 35.0, 1400.0))
            elif verdict.kind == "broken_chase":
                score -= 2000

    if _own_territory_fill(cells, r, c, who):
        score -= 3500
    if _nearby_double_atari(cells, r, c, who):
        score -= 6000
    return score

Model used: Qwen3-VL-8B-Instruct + LoRA adapter (lora_mix_qwen_r5). The full source and a description of the model are in the post "8. The full source code and the models used".