feat: Add Ollama integration for local LLM support in Cortex Linux

AGENTS.md

@@ -16,12 +16,25 @@ git clone https://github.com/cortexlinux/cortex.git
 cd cortex
 python3 -m venv venv
 source venv/bin/activate
-pip install -e .
 
-# Configure API key
+# Install Cortex with interactive Ollama setup
+# Use python setup.py develop (NOT pip install -e .) for interactive prompts
+python setup.py develop
+
+# The setup will automatically:
+# - Prompt if you want to install Ollama (y/n)
+# - Install Ollama if you choose 'yes'
+# - Let you select an AI model to download
+# - Configure everything for first use
+
+# Note: pip install -e . works but hides interactive prompts
+# Use python setup.py develop for full interactive experience
+
+# Optional: Configure API key for cloud providers (if not using Ollama)
 echo 'ANTHROPIC_API_KEY=your-key-here' > .env
 
 # Verify installation
+cortex --help
 cortex install nginx --dry-run
 ```
 

CHANGELOG.md

@@ -8,22 +8,46 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [Unreleased]
 
 ### Added
+- **🚀 Ollama Integration - Local LLM Support**
+  - Privacy-first local LLM inference via Ollama
+  - Zero-cost, offline-capable package management
+  - No API keys required for basic usage
+  - Auto-installation and setup during `pip install`
+  - Smart model selection (prefers code-focused models)
+  - Streaming response support
+  - Automatic fallback to Claude/OpenAI when local models unavailable
+  - New `OllamaProvider` class in `cortex/providers/ollama_provider.py`
+  - Setup script `scripts/setup_ollama.py` for automated installation
+  - Comprehensive documentation in `docs/OLLAMA_INTEGRATION.md`
+  - Example usage in `examples/ollama_demo.py`
+  - Test suite in `tests/test_ollama_integration.py`
+  - Updated LLM router to prioritize local models
+  - CLI command `cortex-setup-ollama` for manual setup
+
 - Comprehensive code assessment (ASSESSMENT.md)
 - Detailed improvement roadmap (ROADMAP.md)
 - Enhanced contribution guidelines (CONTRIBUTING.md)
 - Professional README with full documentation
 - This CHANGELOG file
 
 ### Changed
-- Updated README with proper installation instructions
-- Standardized Python version requirement to 3.10+
+- **LLM Router** now defaults to Ollama for all tasks (privacy-first)
+- Updated routing priority: Ollama → Claude → Kimi K2 → OpenAI
+- API keys now optional (only needed for cloud fallback)
+- Updated README with Ollama quickstart and features
+- Enhanced sample configuration with LLM provider settings
+- Modified `setup.py` to trigger Ollama installation post-install
+- Updated Python version requirement to 3.10+
 - Improved documentation structure
 
 ### Fixed
 - (Pending) Shell injection vulnerability in coordinator.py
 - (Pending) CI/CD pipeline test directory path
 
 ### Security
+- Enhanced privacy with 100% local LLM processing option
+- Zero data transmission to cloud when using Ollama
+- No API key exposure risk for basic usage
 - (Pending) Added additional dangerous command patterns to sandbox
 
 ---

MANIFEST.in

@@ -2,4 +2,5 @@ include README.md
 include LICENSE
 recursive-include LLM *.py
 recursive-include cortex *.py
+recursive-include scripts *.py
 include LLM/requirements.txt

README.md

@@ -63,13 +63,15 @@ cortex install "tools for video compression"
 
 | Feature | Description |
 |---------|-------------|
-| **Natural Language** | Describe what you need in plain English |
-| **Dry-Run Default** | Preview all commands before execution |
-| **Sandboxed Execution** | Commands run in Firejail isolation |
-| **Full Rollback** | Undo any installation with `cortex rollback` |
-| **Audit Trail** | Complete history in `~/.cortex/history.db` |
-| **Hardware-Aware** | Detects GPU, CPU, memory for optimized packages |
-| **Multi-LLM Support** | Works with Claude, GPT-4, or local Ollama models |
+| **🤖 Natural Language** | Describe what you need in plain English |
+| **🔒 Privacy-First** | Local LLM support via Ollama - no API keys required |
+| **📴 Offline Capable** | Works completely offline with local models |
+| **🆓 Zero Cost** | Free local inference, optional cloud fallback |
+| **🛡️ Sandboxed Execution** | Commands run in Firejail isolation |
+| **⏮️ Full Rollback** | Undo any installation with `cortex rollback` |
+| **📋 Audit Trail** | Complete history in `~/.cortex/history.db` |
+| **🔧 Hardware-Aware** | Detects GPU, CPU, memory for optimized packages |
+| **☁️ Multi-LLM Support** | Ollama (local), Claude, GPT-4, or Kimi K2 |
 
 ---
 
@@ -79,7 +81,7 @@ cortex install "tools for video compression"
 
 - **OS:** Ubuntu 22.04+ / Debian 12+
 - **Python:** 3.10 or higher
-- **API Key:** [Anthropic](https://console.anthropic.com) or [OpenAI](https://platform.openai.com)
+- **API Key (Optional):** [Anthropic](https://console.anthropic.com) or [OpenAI](https://platform.openai.com) for cloud fallback
 
 ### Installation
 
@@ -92,24 +94,37 @@ cd cortex
 python3 -m venv venv
 source venv/bin/activate
 
-# 3. Install Cortex
-pip install -e .
+# 3. Upgrade pip and tools
+pip install --upgrade pip setuptools wheel
 
-# 4. Configure API key
-echo 'ANTHROPIC_API_KEY=your-key-here' > .env
+# 4. Install Cortex with interactive setup
+python setup.py develop
+
+# During installation, you'll be prompted to install Ollama (local AI)
+# Choose 'y' for offline/private AI, or 'n' to use cloud providers
+```
+
+### Optional: Configure Cloud Providers
 
-# 5. Verify installation
-cortex --version
+```bash
+# Only needed if you skipped Ollama or want cloud fallback
+echo 'ANTHROPIC_API_KEY=your-key-here' > .env
+# OR
+echo 'OPENAI_API_KEY=your-key-here' > .env
 ```
 
 ### First Run
 
 ```bash
 # Preview what would be installed (safe, no changes made)
+# Uses local Ollama by default - no API calls!
 cortex install nginx --dry-run
 
 # Actually install
 cortex install nginx --execute
+
+# Check Ollama status and installed models
+ollama list
 ```
 
 ---
@@ -119,7 +134,7 @@ cortex install nginx --execute
 ### Basic Commands
 
 ```bash
-# Install with natural language
+# Install with natural language (uses local LLM)
 cortex install "web server for static sites" --dry-run
 cortex install "image editing software like photoshop" --execute
 
@@ -131,6 +146,11 @@ cortex rollback <installation-id>
 
 # Check system preferences
 cortex check-pref
+
+# Manage local LLM models
+ollama list                    # Show available models
+ollama pull llama3:8b         # Download a model
+cortex-setup-ollama           # Re-run Ollama setup
 ```
 
 ### Command Reference
@@ -143,6 +163,7 @@ cortex check-pref
 | `cortex history` | View all past installations |
 | `cortex rollback <id>` | Undo a specific installation |
 | `cortex check-pref` | Display current preferences |
+| `cortex-setup-ollama` | Setup/reinstall Ollama integration |
 | `cortex --version` | Show version information |
 | `cortex --help` | Display help message |
 
@@ -157,6 +178,39 @@ Cortex stores configuration in `~/.cortex/`:
 └── audit.log        # Detailed audit trail
 ```
 
+### Local LLM Support (Ollama)
+
+**Privacy-First by Default**: Cortex uses local LLMs via Ollama for zero-cost, offline-capable operation.
+
+**Benefits:**
+- ✅ **100% Private**: All processing happens locally
+- ✅ **Completely Offline**: Works without internet after setup
+- ✅ **Zero Cost**: No API fees or subscriptions
+- ✅ **No API Keys**: Get started immediately
+
+**Recommended Models:**
+- `phi3:mini` (1.9GB) - Lightweight, default
+- `llama3:8b` (4.7GB) - Balanced performance
+- `codellama:13b` (9GB) - Code-optimized
+- `deepseek-coder-v2:16b` (10GB+) - Best for system tasks
+
+**Manage Models:**
+```bash
+ollama list                     # Show installed models
+ollama pull llama3:8b          # Download a model
+ollama rm phi3:mini            # Remove a model
+```
+
+**Cloud Fallback:**
+If local models are unavailable, Cortex automatically falls back to cloud providers (if configured):
+```bash
+# Optional: Set cloud API keys for fallback
+export ANTHROPIC_API_KEY=your-claude-key
+export OPENAI_API_KEY=your-openai-key
+```
+
+📖 **[Full Ollama Documentation](docs/OLLAMA_INTEGRATION.md)**
+
 ---
 
 ## Architecture
@@ -176,11 +230,12 @@ Cortex stores configuration in `~/.cortex/`:
                               ▼
 ┌─────────────────────────────────────────────────────────────────┐
 │                      LLM Router                                 │
-│              Claude / GPT-4 / Ollama                            │
+│         Ollama (Local) → Claude → GPT-4 → Kimi K2              │
 │                                                                 │
 │  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐             │
-│  │  Anthropic  │  │   OpenAI    │  │   Ollama    │             │
-│  │   Claude    │  │    GPT-4    │  │   Local     │             │
+│  │   Ollama    │  │  Anthropic  │  │   OpenAI    │             │
+│  │   (Local)   │  │   Claude    │  │    GPT-4    │             │
+│  │  PRIORITY   │  │  Fallback 1 │  │  Fallback 2 │             │
 │  └─────────────┘  └─────────────┘  └─────────────┘             │
 └─────────────────────────────────────────────────────────────────┘
                               │

cortex/_post_install.py

@@ -0,0 +1,51 @@
+#!/usr/bin/env python3
+"""
+Post-install hook for Cortex Linux.
+Automatically runs after pip install to setup Ollama.
+"""
+
+import os
+import sys
+
+
+def run_setup():
+    """Run Ollama setup after installation."""
+    # Skip if in CI or if explicitly disabled
+    if (
+        os.getenv("CI")
+        or os.getenv("GITHUB_ACTIONS")
+        or os.getenv("CORTEX_SKIP_OLLAMA_SETUP") == "1"
+    ):
+        return
+
+    # Check if already ran setup (marker file in user's home)
+    marker_file = os.path.expanduser("~/.cortex/.setup_done")
+    if os.path.exists(marker_file):
+        return
+
+    print("\n" + "=" * 70)
+    print("🚀 Running Cortex post-installation setup...")
+    print("=" * 70 + "\n")
+
+    try:
+        # Import and run the setup function
+        from scripts.setup_ollama import setup_ollama
+
+        setup_ollama()
+
+        # Create marker file to prevent running again
+        os.makedirs(os.path.dirname(marker_file), exist_ok=True)
+        with open(marker_file, "w") as f:
+            f.write("Setup completed\n")
+
+    except Exception as e:
+        print(f"⚠️  Ollama setup encountered an issue: {e}")
+        print("ℹ️  You can run it manually later with: cortex-setup-ollama")
+    finally:
+        print("\n" + "=" * 70)
+        print("💡 TIP: To re-run setup anytime, execute: cortex-setup-ollama")
+        print("=" * 70)
+
+
+if __name__ == "__main__":
+    run_setup()

cortex/first_run_wizard.py

@@ -372,15 +372,15 @@ def _setup_ollama(self) -> StepResult:
                     return StepResult(success=True, data={"api_provider": "none"})
 
         # Pull a small model
-        print("\nPulling llama3.2 model (this may take a few minutes)...")
+        print("\nPulling codellama:7b model (this may take a few minutes)...")
         try:
-            subprocess.run(["ollama", "pull", "llama3.2"], check=True)
+            subprocess.run(["ollama", "pull", "codellama:7b"], check=True)
             print("\n✓ Model ready!")
         except subprocess.CalledProcessError:
-            print("\n⚠ Could not pull model - you can do this later with: ollama pull llama3.2")
+            print("\n⚠ Could not pull model - you can do this later with: ollama pull codellama:7b")
 
         self.config["api_provider"] = "ollama"
-        self.config["ollama_model"] = "llama3.2"
+        self.config["ollama_model"] = "codellama:7b"
 
         return StepResult(success=True, data={"api_provider": "ollama"})
 

cortex/kernel_features/hardware_detect.py

@@ -364,7 +364,7 @@ def recommend_models(total_vram_gb: float, system_ram_gb: float, has_npu: bool)
         recommendations.extend(["llama3.1-8b", "mistral-7b", "qwen2.5-14b", "codellama-13b"])
 
     if available_gb >= 8:
-        recommendations.extend(["llama3.2-3b", "phi-3-mini", "gemma-2b", "qwen2.5-7b-q4"])
+        recommendations.extend(["phi3:mini", "llama3:8b", "gemma:2b", "qwen2.5:7b"])
 
     if available_gb >= 4:
         recommendations.extend(["tinyllama-1.1b", "phi-2", "qwen2.5-1.5b"])