Enable screenshot/image input for custom Vision models in ZCode

README.md

Enable image/screenshot input for a custom Vision model in ZCode

ZCode may classify an auto-discovered custom OpenAI-compatible model as text-only even when the underlying model and gateway support Vision.

Symptom

After attaching a screenshot, ZCode responds with a message similar to:

I'm unable to view images. The selected model does not support image input.

Root cause

ZCode stores custom-provider metadata in:

~/.zcode/v2/config.json

An auto-discovered model may be recorded with an explicit text-only input modality:

{
  "modalities": {
    "input": ["text"],
    "output": ["text"]
  }
}

Setting only "supportsImages": true is not enough. ZCode uses modalities.input as the effective capability gate, so the working model entry needs both declarations:

{
  "modalities": {
    "input": ["text", "image"],
    "output": ["text"]
  },
  "supportsImages": true
}

Prerequisite: verify the backend

First verify that the model accepts an OpenAI-compatible multimodal request through the same gateway used by ZCode. A valid message uses a content array containing text and image_url parts:

{
  "model": "glm-5.2",
  "messages": [
    {
      "role": "user",
      "content": [
        {"type": "text", "text": "Describe this image."},
        {
          "type": "image_url",
          "image_url": {"url": "data:image/png;base64,..."}
        }
      ]
    }
  ]
}

If this request fails, fix the model/gateway path first. This ZCode patch only corrects client-side capability metadata.

Apply the fix

1. Quit ZCode completely (Cmd-Q on macOS; Quit/Exit on Windows or Linux). Closing only the window is insufficient because ZCode caches model capabilities.
2. Download fix-zcode-custom-model-vision.py from this gist.
3. Run it with the model ID used by the custom provider:

python3 fix-zcode-custom-model-vision.py glm-5.2

If multiple custom providers contain the same model ID, list the candidates:

python3 fix-zcode-custom-model-vision.py glm-5.2

Then rerun with the intended provider UUID:

python3 fix-zcode-custom-model-vision.py glm-5.2 --provider-id PROVIDER_UUID

The script:

• ignores ZCode's built-in providers;
• creates a timestamped backup;
• preserves credentials and unrelated settings;
• atomically updates the configuration;
• verifies the saved result;
• never prints API keys or provider options.

Validate

1. Reopen ZCode.
2. Select the patched custom model.
3. Start a new task/chat.
4. Attach a screenshot.
5. Ask for objectively verifiable OCR, such as:

Transcribe the largest visible heading exactly.

A successful HTTP request alone is not enough—the answer must demonstrate that the model saw the image.

Rollback

The script prints the backup path, for example:

~/.zcode/v2/config.json.before-image-modalities-20260728-170000

Quit ZCode and restore that file over ~/.zcode/v2/config.json.

Persistence caveat

Editing or refreshing the custom provider—or upgrading ZCode—may regenerate the model metadata and restore "input": ["text"]. If screenshot support disappears, inspect the model entry and rerun the patch.

The durable upstream fix is for ZCode to preserve custom input modalities, expose them in the custom-model UI, or import reliable capability metadata during model discovery.

fix-zcode-custom-model-vision.py

#!/usr/bin/env python3
"""Enable image input for a custom model in ZCode's v2 config.

Creates a backup, modifies only the selected custom model entry, writes
atomically, and verifies the result. Provider options and credentials are
never displayed.
"""

from __future__ import annotations

import argparse
import json
import os
from pathlib import Path
import shutil
import sys
import time
from typing import Any


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(
        description=(
            "Add the image input modality to a custom ZCode model definition."
        )
    )
    parser.add_argument(
        "model_id",
        nargs="?",
        default="glm-5.2",
        help="Model ID as configured in ZCode (default: glm-5.2)",
    )
    parser.add_argument(
        "--provider-id",
        help="Custom provider UUID; required only when multiple providers match",
    )
    parser.add_argument(
        "--config",
        type=Path,
        default=Path.home() / ".zcode" / "v2" / "config.json",
        help="ZCode config path (default: ~/.zcode/v2/config.json)",
    )
    return parser.parse_args()


def main() -> int:
    args = parse_args()
    path = args.config.expanduser().resolve()

    if not path.is_file():
        raise SystemExit(f"ZCode configuration not found: {path}")

    try:
        data: dict[str, Any] = json.loads(path.read_text())
    except (OSError, json.JSONDecodeError) as exc:
        raise SystemExit(f"Could not read valid JSON from {path}: {exc}") from exc

    providers = data.get("provider", {})
    if not isinstance(providers, dict):
        raise SystemExit("Unexpected config schema: 'provider' is not an object")

    matches: list[tuple[str, str, str, dict[str, Any]]] = []

    for provider_id, provider in providers.items():
        if provider_id.startswith("builtin:") or not isinstance(provider, dict):
            continue

        models = provider.get("models", {})
        if not isinstance(models, dict):
            continue

        for configured_id, model in models.items():
            if (
                configured_id.lower() == args.model_id.lower()
                and isinstance(model, dict)
            ):
                matches.append(
                    (
                        provider_id,
                        configured_id,
                        str(provider.get("name", "<unnamed>")),
                        model,
                    )
                )

    if args.provider_id:
        matches = [match for match in matches if match[0] == args.provider_id]

    if not matches:
        qualifier = (
            f" in provider {args.provider_id!r}" if args.provider_id else ""
        )
        raise SystemExit(
            f"No custom model entry matching {args.model_id!r}{qualifier}."
        )

    if len(matches) > 1:
        print("Multiple custom provider/model entries matched:")
        for provider_id, configured_id, provider_name, _ in matches:
            print(f"  {provider_id}: {provider_name} / {configured_id}")
        raise SystemExit(
            "Rerun with --provider-id followed by the intended provider UUID."
        )

    provider_id, configured_id, provider_name, model = matches[0]
    before = json.loads(json.dumps(model))

    modalities = model.setdefault("modalities", {})
    if not isinstance(modalities, dict):
        raise SystemExit(
            "Unexpected model schema: 'modalities' is not an object; refusing to edit"
        )

    inputs = modalities.setdefault("input", ["text"])
    if not isinstance(inputs, list) or not all(
        isinstance(value, str) for value in inputs
    ):
        raise SystemExit(
            "Unexpected model schema: 'modalities.input' is not a string list; "
            "refusing to edit"
        )

    new_inputs: list[str] = []
    for modality in ("text", "image", *inputs):
        if modality not in new_inputs:
            new_inputs.append(modality)

    modalities["input"] = new_inputs
    modalities.setdefault("output", ["text"])
    model["supportsImages"] = True

    stamp = time.strftime("%Y%m%d-%H%M%S")
    backup = path.with_name(
        f"{path.name}.before-image-modalities-{stamp}"
    )
    shutil.copy2(path, backup)

    tmp = path.with_name(f".{path.name}.tmp-{os.getpid()}")
    try:
        tmp.write_text(json.dumps(data, indent=2, ensure_ascii=False) + "\n")
        json.loads(tmp.read_text())
        os.replace(tmp, path)
    finally:
        if tmp.exists():
            tmp.unlink()

    verified = json.loads(path.read_text())
    after = verified["provider"][provider_id]["models"][configured_id]

    if "image" not in after.get("modalities", {}).get("input", []):
        raise SystemExit("Verification failed: image input modality is absent")
    if after.get("supportsImages") is not True:
        raise SystemExit("Verification failed: supportsImages is not true")

    print(f"Backup: {backup}")
    print(f"Provider: {provider_name} ({provider_id})")
    print(f"Model: {configured_id}")
    print("Before:")
    print(json.dumps(before, indent=2, ensure_ascii=False))
    print("After:")
    print(json.dumps(after, indent=2, ensure_ascii=False))
    print("Verification: PASS")
    print("\nRestart ZCode and test the model in a new task/chat.")
    return 0


if __name__ == "__main__":
    sys.exit(main())