QirabotGUI automation driven by AI vision

Drive browsers, mobile apps, desktops, and games through pixels — no DOM, no selectors. Describe what you want in plain English; the AI sees the screen and acts.

Quick Start

Installation

Demos

GitHub

🌐

Browser automation

Playwright under the hood — bot.open() launches Chromium for you, or attach to a running Chrome over CDP. Headless, persistent profiles, remote pools.

🤖

Android without Appium

Direct over adb — no server, nothing installed on the device. Screenshot, tap, swipe, and type (including Chinese and emoji) on any device adb can see.

📱

iOS without Appium

Talks HTTP straight to WebDriverAgent on a real iPhone. No Appium server, no extra packages — just iproxy and a WDA session.

🎮

Windows windows & games

Bind one window by title or HWND. Input is DirectInput scancodes — the level games actually poll, which virtual-key automation can't reach. Drives Unity, Unreal, and native apps.

🔌

Bolt onto your stack

Already running Playwright, Selenium, Appium, or pyautogui? Pass your page or driver object and inject AI steps into the suite you have — no rewrite.

🧩

One API everywhere

bot.ai() runs whole tasks autonomously; bot.click() / extract() / verify() give you deterministic steps with AI element location. Same calls on every platform.

What is Qirabot?

Qirabot is a Python SDK and CLI for vision-based GUI automation. Instead of querying a DOM or an accessibility tree, it looks at the actual pixels on screen: a multimodal AI model reads the screenshot, finds the element you described in natural language, and returns coordinates for the client to act on. That means it works on surfaces traditional frameworks can't touch — canvas-rendered UIs, native mobile apps, desktop applications, and full 3D games.

The decision engine runs in your own process: screenshots go directly from your machine to the model endpoint you configure on Google Vertex AI, under your own Google Cloud credentials.

python

from qirabot import Qirabot

bot = Qirabot()
page = bot.open("https://www.wikipedia.org")

result = bot.ai(page, "Search for SpaceX and get the first sentence of the article")
print(result.output)

bot.close()

Two ways to drive it:

Autonomous — bot.ai(page, "do the whole task"): the AI sees the screen, decides the next action, and loops until the goal is met.
Deterministic — bot.click(page, "Login button"), bot.extract(...), bot.verify(...): you own the control flow, AI vision locates each element. No XPath, no CSS selectors, nothing to break when the layout shifts.

Every run writes an HTML report with per-step screenshots, and --record captures a video. Start with the Quick Start, or jump to the backend you're automating: Browser, Android, iOS, Windows & Games, Desktop.