Generative AI at Work: From Theory to Real Practice

Generative AI has gone from science fiction to a working tool in record time. But between the hype and reality there's a gap. This article is not a theoretical introduction: it's a practical guide for those who already understand the potential and want to implement it without dying in the attempt.

Let's forget the hype. These are the facts you need to know:

When we talk about "implementing AI", there are three fundamentally different architectures. Each has its use cases:

☁️ 1. Cloud API (The Most Common)

You call an API (OpenAI, Anthropic, Google) and receive responses. Simple, fast, but with implications:

When to use it: Rapid prototyping, non-sensitive tasks, when you need the best available model.

🖥️ 2. Local LLM (Self-Hosted)

You run the model on your own infrastructure using tools like Ollama, LMStudio or vLLM.

# Run Llama 3.1 8B locally with Ollama
ollama run llama3.1:8b

# Or with LMStudio (graphical interface)
# Download the GGUF model → Load → Ready

# OpenAI-compatible API
curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "llama3.1:8b", "messages": [{"role": "user", "content": "Hello"}]}'

When to use it: Sensitive data, compliance (GDPR, HIPAA), high request volume, no variable costs.

🎯 3. Custom Models (Fine-Tuning)

You train or fine-tune a model with your own data. The holy grail for specific use cases.

After implementing AI in dozens of projects, this is the framework we use:

// Simplified decision tree

Is the data sensitive?
├─ YES → Do you need frontier-quality responses?
│       ├─ YES → Claude/GPT-4 with Enterprise Agreement
│       └─ NO → Local LLM (Llama 3.1, Mistral)
│
└─ NO → Is the volume high (>10K req/day)?
        ├─ YES → Limited budget?
        │       ├─ YES → Local LLM or DeepSeek
        │       └─ NO → GPT-4o or Claude 3.5
        │
        └─ NO → Is the task very specific?
                ├─ YES → Fine-tuning a small model
                └─ NO → Cloud API (GPT-4o-mini, Claude Haiku)

At Cadences we implement a pattern we call "AI Service with fallback". The idea is simple:

1. Define an abstract interface for AI operations
2. Implement multiple providers (14 in our case)
3. Route based on task type, cost and availability
4. If a provider fails, automatically switch to another

// Simplified AI Service example
interface AIProvider {
  chat(messages: Message[]): Promise<string>
  embed(text: string): Promise<number[]>
}

class AIService {
  private providers: Map<string, AIProvider>
  
  async chat(messages: Message[], options: ChatOptions) {
    // 1. Select provider based on task
    const provider = this.selectProvider(options)
    
    try {
      return await provider.chat(messages)
    } catch (error) {
      // 2. Fallback to alternative provider
      const fallback = this.getFallback(provider)
      return await fallback.chat(messages)
    }
  }
}

Benefits of this pattern:

• Resilience: If OpenAI has an outage, you switch to Claude or Gemini
• Cost optimization: Use cheap models for simple tasks
• Flexibility: Adding a new provider means implementing an interface
• A/B Testing: Compare responses from different models

🤖 1. Project Assistant with Context

The Cadences assistant is not a generic chatbot. It knows your projects, tasks, clients and data. How?

🏷️ 2. Automatic Ticket Classification

When an email or form arrives, we automatically classify it using a fine-tuned model:

This classifier runs on the Cadences Local ML Trainer, without sending data to third parties.

💬 3. Storefront Chatbot

Each Storefront can have a chatbot that knows the business's products and services:

// Chatbot configuration
{
  "provider": "gemini",
  "model": "gemini-2.0-flash-exp",
  "context": {
    "businessName": "Restaurant23",
    "products": [...],  // Full menu
    "faqs": [...],      // Frequently asked questions
    "rules": [
      "Always respond in English",
      "If they ask about reservations, give the phone number",
      "Don't give approximate prices, use real ones"
    ]
  }
}

Generative AI is easy to implement, hard to measure. These are the metrics we recommend:

⚡ Productivity Metrics

• Time saved: Minutes per task before vs. after
• Tasks completed: Work volume processed per day
• Automation rate: % of tasks that don't require human intervention

✅ Quality Metrics

• Accuracy: % of correct responses (you need ground truth)
• Escalation rate: % of cases requiring human review
• User satisfaction: NPS or response ratings

💰 Cost Metrics

• Cost per request: $ spent per API call
• Cost per completed task: Total $ / tasks processed
• ROI: (Value generated - AI Cost) / AI Cost × 100

Generative AI is transformative, but it's not magic. It requires thoughtful architecture, proper model selection, output validation and clear metrics.

The keys to a successful implementation:

1. Start small: One use case, one model, measurable results
2. Multi-provider: Don't depend on a single vendor
3. Local-first when it matters: Privacy is non-negotiable for sensitive data
4. Measure everything: If you can't measure the impact, you can't improve it
5. Iterate fast: The landscape changes every month, stay updated

Generative AI won't replace your job. But someone who knows how to use it well, probably will.