Ollama Test, bot AI

Ollama Bot AI Architecture.md

AI Chatbot Architecture for IaC Deployment Workflow

This document describes a simple AI chatbot architecture used to assist infrastructure automation workflows, especially for testing sub-domain deployment using Terraform as Infrastructure as Code (IaC).

The main idea is to connect a chat-based interface with an agent runtime, an LLM provider, and a set of infrastructure tools, so the bot can receive a request, reason through the task, and help execute or prepare deployment-related actions.

Architecture Diagram

flowchart LR
    user["Chat Users"] -->|"message"| botApi["Chat Bot API"]
    botApi -->|"webhook"| gateway["Agent Gateway"]
    gateway -->|"request"| agent["Agent Runtime<br/>Tool Registry<br/>Skills / Memory"]
    agent -->|"LLM call"| llm["Ollama Cloud<br/>kimi-k2.7-code<br/>Code-specialized model"]
    llm -->|"response"| agent
    agent -->|"persist config / memory"| profile["Profile Data<br/>Skills<br/>Memory<br/>Cron<br/>Config"]
    agent -->|"read secrets"| secrets["Secrets / Env Vars<br/>BOT_TOKEN<br/>OLLAMA_API_KEY<br/>ALLOWED_USERS"]
    agent -->|"tool execution"| tools["Infrastructure Tools<br/>Terraform<br/>Terminal<br/>File System<br/>Git / Repository"]
    tools -->|"deployment output"| agent
    agent -->|"reply"| gateway
    gateway -->|"response"| botApi
    botApi -->|"reply"| user

Loading

High-Level Flow

1. Users send a message through a chat group or direct chat.
2. The message is received by the Chat Bot API.
3. The API forwards the request to the Agent Gateway through a webhook.
4. The Agent Gateway passes the request into the Agent Runtime.
5. The Agent Runtime uses a code-specialized LLM model through Ollama Cloud.
6. The agent can access registered tools such as terminal, file system, Git, and Terraform.
7. Secrets are loaded from environment variables instead of being hardcoded.
8. Execution results, memory, skills, and configuration can be persisted into profile storage.
9. The final response is returned back to the user through the chat interface.

Main Components

1. Chat Interface

The chat interface acts as the user entry point. It allows users to interact with the AI bot using natural language prompts.

Example use cases:

• Ask the bot to inspect Terraform configuration.
• Ask the bot to prepare a test deployment plan.
• Ask the bot to validate a sub-domain setup.
• Ask the bot to summarize deployment output.

2. Chat Bot API

The Chat Bot API receives messages from users and forwards them to the backend through webhook integration.

Responsibilities:

• Receive incoming messages.
• Forward requests to the Agent Gateway.
• Send replies back to users.
• Restrict access using allowed user configuration.

3. Agent Gateway

The Agent Gateway acts as a transport bridge between the chat platform and the AI agent runtime.

Responsibilities:

• Receive webhook payloads.
• Normalize incoming messages.
• Forward the request to the agent runtime.
• Return the final agent response to the chat platform.

4. Agent Runtime

The Agent Runtime is the core execution layer.

Responsibilities:

• Interpret user requests.
• Call the LLM model.
• Select and execute registered tools.
• Manage skills and memory.
• Coordinate infrastructure-related tasks.

Example tool registry:

• Browser
• Terminal
• File system
• Git / Repository
• Terraform
• Deployment scripts

5. LLM Provider

The chatbot uses Ollama Cloud as the LLM provider with the kimi-k2.7-code model.

The reason for this setup is to keep the workflow:

• Model-agnostic
• Tool-friendly
• Cost-efficient
• Flexible across different IDEs and agent frameworks

6. Secrets and Environment Variables

Sensitive values should be stored in environment variables, not hardcoded in the codebase.

Example variables:

BOT_TOKEN=your_bot_token
OLLAMA_API_KEY=your_ollama_api_key
ALLOWED_USERS=user_id_1,user_id_2,user_id_3

7. Profile Data

The agent can persist configuration, memory, skills, and scheduled tasks into a local profile directory.

Example profile data:

~/.agent/profiles/default

Stored data may include:

• Agent skills
• Memory
• Cron configuration
• Runtime configuration
• Tool preferences

Deployment Workflow Example

sequenceDiagram
    participant User as Chat User
    participant Bot as Chat Bot API
    participant Gateway as Agent Gateway
    participant Agent as Agent Runtime
    participant LLM as Ollama Cloud / kimi-k2.7-code
    participant TF as Terraform

    User->>Bot: Request sub-domain test deployment
    Bot->>Gateway: Forward message via webhook
    Gateway->>Agent: Send normalized request
    Agent->>LLM: Ask for reasoning and deployment steps
    LLM-->>Agent: Return suggested plan
    Agent->>TF: Run or prepare Terraform workflow
    TF-->>Agent: Return plan/apply output
    Agent->>LLM: Summarize result
    LLM-->>Agent: Final response
    Agent-->>Gateway: Send response
    Gateway-->>Bot: Return reply
    Bot-->>User: Show deployment result

Loading

Example Use Case

A user sends a message such as:

Please help test deploy a new sub-domain using Terraform.

The agent may then:

1. Inspect the Terraform configuration.
2. Validate required variables.
3. Check DNS or sub-domain naming rules.
4. Generate or review the Terraform plan.
5. Execute the deployment command if allowed.
6. Summarize the result back to the chat.

Cost-Efficiency Idea

This setup was also tested as an alternative way to use AI more efficiently.

Instead of subscribing individually to multiple AI platforms, a small team can use a shared model provider setup and integrate it into different tools, IDEs, and agent workflows.

Potential benefits:

• Lower monthly cost per person
• More flexible model usage
• Less dependency on a single AI provider
• Easier integration with engineering workflows
• Better fit for infrastructure and automation experiments

Notes

This is an experimental architecture for AI-assisted infrastructure workflows. For production usage, additional safety layers should be added, such as:

• Approval before executing destructive commands
• Role-based access control
• Audit logging
• Command allowlist / denylist
• Secret scanning
• Terraform plan review before apply
• Separate staging and production environments