Windows & Games — the Window backend

qirabot.Window binds to a single window (by title regex or HWND): screenshots are its client area, clicks are window-relative, and keys are DirectInput scancodes — the level games poll, which virtual-key automation (pyautogui, AutoHotkey's default send mode) can't reach. Stdlib ctypes only; built into the core package, no extras.

Combined with AI vision for element location, this drives what no DOM/accessibility-based framework can: Unity and Unreal games, custom launchers, legacy native apps.

The quickest check is the CLI — built in, no extras:

qirabot desktop "Open the inventory and list all items" --window-title "Genshin"
qirabot desktop "..." --hwnd 132456

The same thing in Python:

from qirabot import Qirabot, Window

window = Window(title="Genshin")   # literal substring; or Window(hwnd=132456)
bot = Qirabot().bind(window)

result = bot.ai("Open the inventory and list all items")
bot.close()

Window selectors: hwnd= (explicit handle), title= (literal substring — paste the title straight from the taskbar, parentheses and dots are safe), title_re= (a regex, for fuzzy/multi-language matching), or class_name= (exact window class — Unity games expose UnityWndClass, Unreal UnrealWindow; steadier than titles and combinable with title/title_re). If several windows match, resolution fails listing the candidates; when the duplicates are unavoidable — cloud-gaming clients and launcher overlays often share the main window's exact title — add ambiguous="largest" to pick the biggest window. timeout= keeps polling for the window while a game is still starting:

window = Window(title="MyGame · Cloud(Beta)", ambiguous="largest")
window = Window(class_name="UnityWndClass", timeout=180)   # just-launched game

Before the first input is injected, the backend switches the window's input language to US English and closes its IME — an active CJK IME would swallow injected letter keys into its composition window instead of the game. This is per-window state (Win+Space switches it back); pass Window(..., english_ime=False) to leave the IME alone.

Game-grade input

• Keys are scancodes — real hardware-level input, including ctrl/alt/win combos. Characters outside the scancode table are injected as unicode key events.

• Hold a key for a duration — quantified in-game movement:

  bot.press_key(window, "w", duration_seconds=2)      # walk forward 2s
  bot.press_key(window, "shift+w", duration_seconds=1.5)  # sprint

• Modifier-click — atomic alt+click for games, ctrl+click multi-select:

  bot.click(window, "enemy unit", modifier="alt")

• Split press/release primitives — mouse_down / mouse_up / key_down / key_up hold an input across other actions (keep moving while clicking, press-and-hold drags). Any input still held is auto-released at the end of an ai() run and on close().

Mixing deterministic steps with AI

Game UI audits work well as deterministic navigation plus AI verification:

bot.click(window, "the Bag icon")
bot.wait_for(window, "the inventory panel is open")
ok = bot.verify(window, "every item slot shows an icon and a count")
items = bot.extract(window, "list the item names visible in the inventory")

See the full walkthrough in examples/game/, including a custom-tool example where the AI calls your GM backend mid-task (add energy on an out-of-energy popup, then continue the daily-quest loop) — how to register such tools is in AI Tasks & Custom Tools.

Recording the window

On Windows you can record just the window under test, with system audio:

bot = Qirabot(record=True, record_window=True, record_audio=True)

Keep the window visible — gdigrab produces black frames for minimized or GPU-composited (fullscreen-exclusive game) windows; for those, record the full screen instead.

Notes

• Whole-desktop automation (any OS) is the separate pyautogui backend; the Window backend is Windows-specific and single-window by design.
• Coming from Airtest 1.x? connect_device("Windows:///132456") becomes Window(hwnd=132456).