[FEATURE] Ollama Integration - Local LLM Support

[Suyashd999]

Summary

Adds local LLM support via Ollama for privacy-first, offline-capable package management.

Features

• ✅ Auto-detect Ollama installation
• ✅ Smart model selection (prefers code-focused models)
• ✅ Streaming responses
• ✅ Fallback to Claude/OpenAI
• ✅ Works completely offline
• ✅ Zero data sent to cloud

Supported Models

• CodeLlama (7B, 13B, 34B)
• Llama 3.2
• Mistral / Mixtral
• DeepSeek Coder
• Phi-2
• And more...

Files Added

• cortex/providers/ollama_integration.py - Core integration
• tests/test_ollama_integration.py - Test suite
• docs/README_OLLAMA.md - Documentation

Testing

pytest tests/test_ollama_integration.py -v

Bounty: $150 (+ $150 bonus after funding)

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features

  • Added local LLM support through Ollama integration with automatic model discovery and selection
  • Enabled offline and privacy-preserving AI capabilities with model management and configuration options
  • Introduced provider routing with automatic fallback support

• Documentation

  • Added comprehensive integration guide covering setup, configuration, usage examples, troubleshooting, and architecture details

• Tests

  • Added test coverage for LLM provider functionality and integration scenarios

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

📝 CodeRabbit Plan Mode

Generate an implementation plan and prompts that you can use with your favorite coding agent.

• Create Plan

Examples

• Example 1
• Example 2

---

🔗 Similar Issues

Related Issues

• [Kernel Feature] Model Lifecycle Manager - Systemd-Based LLM Service Management #216
• [Kernel Feature] Model Lifecycle Manager - Systemd-Based LLM Service Management #220
• [Kernel Feature] Model Lifecycle Manager - Systemd-Based LLM Service Management #217
• Federated Learning for Package Intelligence #174

🔗 Related PRs

#41 - LLM Router - Multi-Provider Support (Issue #34) [merged]
#192 - Added Kimi K2 provider tests Fixes #40 [merged]
#311 - feat: add parallel LLM call tests and demo scripts [merged]
#318 - fix: Automatically load API keys from .env files at runtime [merged]
#320 - feat: Add parallel LLM calls implementation (#276) [merged]

👤 Suggested Assignees

• mikejmorgan-ai
• Sahilbhatane
• dhvll
• aliraza556

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord or schedule a call!

[Suyashd999]

@mikejmorgan-ai had created a PR on this #287 creating issue tracking for that.

@sujay-d07 will finish this

[Bhav-ikkk]

Can I take this issue?

[Suyashd999]

Can I take this issue?

cortex dev already working on this there are other issues in https://github.com/cortexlinux/cortex/issues, you can pick from the 2nd or 3rd pane

[Bhav-ikkk]

Got it, thanks for the update! I’ll check the project board (panes 2/3) and look for a different issue to pick up. I'll comment on the new issue once I find one suitable.