{
  "markdown": "# 🎥 Gemini-CLI Vision Extension\n\n> **Webcam + ASL + AI Image + AI Video — all from Gemini-CLI.**  \n> Capture frames, interpret American Sign Language, and transform your webcam feed into AI-generated art or animation — entirely by prompt **or slash command**.\n\n---\n\n## 🧠 What It Does\n\nThe **Gemini-CLI Vision Extension** brings real-time visual reasoning to your terminal.\n\nIt connects your **webcam (or tethered iPhone)** to Gemini’s **Model Context Protocol (MCP)** ecosystem, enabling natural, multimodal interactions such as:\n\n> “Can you take a capture of me using device 0?”  \n> “...and use Banana to transform it into a sketch using a fine-point pen.”  \n> “...and then take that sketch and turn it into a music video.”  \n> “Let’s chat in ASL — I’ll sign my question.”\n\nYou can issue these commands two ways:\n1. **Natural Language (NL)** — just *ask* Gemini in plain English.  \n2. **Slash Commands (structured)** — use `/vision:*` commands for precise control.\n\nYou can even skip the live camera and use **any static image** in your working folder.\n\n---\n\n## ⚙️ Setup\n\n### 1. Install\n\n```bash\ngemini extensions install https://github.com/automateyournetwork/GeminiCLI_Vision_Extension.git\n```\n\n2. Mac Permissions\nIf prompted, allow Camera access to your terminal (System Settings → Privacy & Security → Camera).\nYour iPhone or other tethered cameras may appear as extra devices — and yes, you can use them.\n\n🖥️ Core Commands\nCommand\tDescription\n/vision:devices\tDiscover connected cameras (indexes, resolutions, FPS). Start here.\n/vision:start\tOpen a selected device. You can specify width, height, fps, and backend.\n/vision:status\tShow whether a camera is open and its properties.\n/vision:capture\tCapture a single frame and optionally send it directly to Gemini.\n/vision:burst\tCapture a sequence of frames (for ASL or motion analysis).\n/vision:stop\tRelease the camera safely.\n\nYou can run these directly, or ask naturally:\n\n“List my available cameras.”\n“Open my iPhone camera and take a photo.”\n“Stop the camera.”\n\n🎨 Modalities\n1️⃣ Devices\nRun:\n\n```bash\n/vision:devices\n```\n\nLists all available cameras.\n\n✅ macOS: Build in camera usually device 0; iPhone Camera often shows up as device 1.\n\nExample natural language:\n\n“Show me my connected cameras.”\n\nThen:\n\n```bash\n/vision:start camera_index=0 width=640 height=480 fps=15\n```\n\n2️⃣ Capture\nTake a single frame:\n\n```bash\n/vision:capture\n```\n\nor naturally:\n\n“Can you take a capture of me using device 0?”\n\nYou’ll get a saved image and an @attachment you can reuse in a follow-up turn.\n\n3️⃣ Banana Mode 🍌 (AI Image Generation)\nTransform your webcam capture into AI-generated artwork:\n\n```bash\n/vision:banana \"Turn this into a watercolor portrait\"\n```\n\nNatural language:\n\n“Take a capture and use Banana to transform it into a sketch using a fine-point pen.”\n\nBehind the scenes:\n\nCaptures a frame\n\nSends it to Gemini 2.5 Flash Image\n\nSaves generated images (e.g. banana_001.png)\n\nEmits @attachments for chaining\n\nUse for:\n\nStyle transfers\n\nPoster or thumbnail mockups\n\nCinematic selfies or sketches\n\n4️⃣ Veo Mode 🎬 (AI Video Generation)\nTurn stills or Banana images into short AI videos with Veo 3:\n\n```bash\n/vision:veo \"Animate this sketch into a short music video\"\n```\n\nor\n\n“...and then take that sketch and turn it into a music video.”\n\nUses Banana output (or live capture)\n\nRuns Veo 3.0 / 3.1 for image-conditioned generation\n\nOutputs real .mp4 files\n\nSupports aspect_ratio, resolution, seed, and more\n\n5️⃣ ASL Mode 🤟 (American Sign Language)\nChat in ASL directly through your webcam:\n\n```bash\n/vision:asl\n```\n\nGemini:\n\nCaptures a short burst of frames\n\nTranscribes your signing\n\nResponds naturally in English\n\nUse /vision:asl_veo to go further:\n\nUnderstands your ASL input\n\nGenerates an ASL gloss reply\n\nAnimates a generic avatar replying in ASL using Veo\n\nExample:\n\n“Let’s chat in ASL — I’ll sign my question.”\n\n🔄 Typical Flow\n```bash\n/vision:devices\n/vision:start camera_index=0\n/vision:capture\n/vision:banana \"Make this look like a Pixar movie poster\"\n/vision:veo \"Animate the poster into a trailer opening\"\n/vision:stop\n```\n\nOr conversationally:\n\n“Open my main camera, take a selfie, turn it into a Pixar-style poster, and animate it into a short trailer.”\n\n🧩 Architecture\n```mermaid\nflowchart TD\n    A[Camera Device(s)\\n/webcam/iPhone/static image] -->|/vision:devices| B[Capture]\n    B -->|/vision:capture| C[Banana 🍌 AI Image]\n    C -->|/vision:veo| D[Veo 🎬 AI Video]\n    B -->|/vision:asl| E[ASL 🤟 Interpreter]\n    E -->|/vision:asl_veo| F[Veo Avatar Reply in ASL]\n    B -->|Attachments| G[Gemini Context / Multimodal Chain]\n\n    subgraph Gemini CLI\n    A\n    B\n    C\n    D\n    E\n    F\n    G\n    end\n```\n\nFlow Summary:\n\n/vision:devices — detect cameras\n\n/vision:start — open camera\n\n/vision:capture — grab frame\n\n/vision:banana — AI-stylize image\n\n/vision:veo — animate into video\n\n/vision:asl — communicate via ASL\n\n/vision:asl_veo — reply back in sign language\n\n🧠 Under the Hood\nRuns as an MCP server using FastMCP (no HTTP)\n\nUses OpenCV for frame capture\n\nUses Google Gemini 2.5 for image reasoning (Banana)\n\nUses Veo 3 for AI video generation\n\nUses Gemini Flash multimodal for ASL understanding\n\nSaves all outputs as real files (.jpg, .png, .mp4) — no base64 bloat\n\nGEMINI_API_KEY and GOOGLE_API_KEY (for Veo3) in your environment\n\n🛡️ Safety\nAlways obtain consent before capturing people.\n\nStop your camera when done (/vision:stop).\n\nAvoid personal likeness prompts in Veo.\n\nUse safe, creative prompt phrasing.\n\n💡 Example Prompts\nNatural Language\n\n“Can you take a capture of me using device 0?”\n“Make this capture look like a pencil sketch.”\n“Animate this into a 10-second video.”\n“Let’s chat in ASL.”\n\nSlash Command\n\n```bash\n/vision:capture\n/vision:banana \"Sketch in fine point pen\"\n/vision:veo \"Turn it into a music video\"\n/vision:asl_veo duration_ms=20000 aspect_ratio=\"16:9\" resolution=\"1080p\"\n```\n\n👤 Author\nJohn Capobianco\nHead of Developer Relations — Selector AI\n\n📍 Creator of the Gemini-CLI multimodal suite:\n/talk, /listen, /vision, /computeruse, /packet_buddy, /subnetcalculator, /file_search\n\n“The CLI is dead — long live the multimodal CLI.”\n\n## 🎬 Demo Video  \nCheck out this quick walkthrough showing the **Vision Extension** in action—webcam capture, Banana (image generation), and Veo (video generation).  \n[![Make videos with Gemini-CLI, Nano Banana, and Veo3](https://img.youtube.com/vi/0LIPzVDF4LU/0.jpg)](https://youtu.be/0LIPzVDF4LU?si=upktxT1c-AxY3z2a)  \n\n<https://youtu.be/0LIPzVDF4LU?si=upktxT1c-AxY3z2a>  \n",
  "bytes": 6514,
  "sha": "77ad4cb34005d9ef09d81dffb5429e4355bf147c830a750542996e83cf1b145d",
  "repo_slug": "automateyournetwork/geminicli_vision_extension",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/plg_automateyournetwork_geminicli_vision_ext_b38f7fb0/readme"
}