Migrate away from TypeORM

[devin-ai-integration]

TypeORM forces too tight a coupling of API and database storage. Hence, we need to migrate away from it. It also structures some of the tables in a suboptimal way, by creating unnecessary indexes and/or junction tables.

• Seek for alternatives
• Build experimental branches with alternative DB adapters
• Write a lossless migration for existing installations
• Adopt one of the branches

---

Originally from spacebarchat/server issue spacebarchat#750, created by erkinalp, closed as not planned

[traycerai]

Plan

Observations

The Spacebar server is a Discord-like chat platform using TypeORM 0.3.25 with ~60 entities, complex relationships, and custom Snowflake IDs. The codebase has significant TypeORM coupling through static repository methods, query builders, decorators, and custom BaseClass inheritance. Migration files exist for MySQL/MariaDB/PostgreSQL/SQLite, and there are cyclic imports preventing standard TypeORM CLI usage.

Approach

This migration will be executed in phases to minimize risk and ensure data integrity. We'll start by evaluating modern ORM alternatives (Prisma, Drizzle, MikroORM), create a database abstraction layer to decouple TypeORM usage, implement the chosen ORM alongside TypeORM, migrate entities incrementally, and finally remove TypeORM dependencies. The approach prioritizes backward compatibility and lossless data migration.

Reasoning

I analyzed the codebase structure, examined the main Database.ts configuration file, reviewed entity patterns in BaseClass.ts and complex entities like User.ts and Guild.ts, searched for TypeORM usage patterns throughout the API routes, identified query builder usage in gateway operations, examined migration files across different database types, and reviewed database-related scripts to understand the full scope of TypeORM integration.

Proposed File Changes

📄 docs/TYPEORM_MIGRATION_PLAN.md ^(NEW) 🔗

Create comprehensive documentation outlining the migration strategy, timeline, and decision rationale. Include comparison matrix of ORM alternatives (Prisma vs Drizzle vs MikroORM), risk assessment, rollback procedures, and testing strategy. Document the current TypeORM usage patterns identified in the codebase analysis.

📁 src/util/database ^(NEW) 🔗

Create new database abstraction layer directory to house the new ORM implementation and provide a clean interface for database operations.

📁 src/util/database/interfaces ^(NEW) 🔗

Create interfaces directory for database operation contracts that will be implemented by both TypeORM (temporarily) and the new ORM.

📄 src/util/database/interfaces/IRepository.ts ^(NEW) 🔗

Define generic repository interface with methods like findOne, findMany, create, update, delete, increment, decrement that abstract away ORM-specific implementations. This interface will be implemented by both TypeORM and new ORM adapters to ensure seamless transition.

📄 src/util/database/interfaces/IDatabase.ts ^(NEW) 🔗

Define database connection interface with methods for initialization, transaction management, and repository access. This will abstract the database connection layer from the specific ORM implementation.

📁 src/util/database/adapters ^(NEW) 🔗

Create adapters directory to house ORM-specific implementations of the database interfaces.

📄 src/util/database/adapters/TypeORMAdapter.ts ^(NEW) 🔗

References

• 📄 src/util/util/Database.ts ^(MODIFY)

Create TypeORM adapter that implements the IDatabase and IRepository interfaces, wrapping existing TypeORM functionality. This adapter will delegate to the current src/util/util/Database.ts implementation and provide the abstraction layer needed for gradual migration.

📁 evaluation ^(NEW) 🔗

Create evaluation directory for ORM comparison prototypes and benchmarks.

📁 evaluation/prisma-prototype ^(NEW) 🔗

References

• 📄 src/util/entities/User.ts
• 📄 src/util/entities/Guild.ts

Create Prisma evaluation prototype directory with schema.prisma file generated from existing database structure using prisma db pull. Implement key entities (User, Guild, Channel) and test complex queries, relationships, and migration scenarios.

📁 evaluation/drizzle-prototype ^(NEW) 🔗

References

• 📄 src/util/entities/User.ts
• 📄 src/util/entities/Guild.ts

Create Drizzle evaluation prototype directory with schema definitions in TypeScript. Use drizzle-kit introspect to generate initial schema from existing database, then test the same scenarios as Prisma prototype for comparison.

📁 evaluation/mikroorm-prototype ^(NEW) 🔗

References

• 📄 src/util/entities/BaseClass.ts ^(MODIFY)

Create MikroORM evaluation prototype directory. Since MikroORM supports decorators similar to TypeORM, test how easily existing entity definitions can be adapted. Focus on decorator compatibility and migration path complexity.

📄 scripts/evaluate-orms.js ^(NEW) 🔗

Create evaluation script that runs performance benchmarks and feature compatibility tests across all three ORM prototypes. Test query performance, relationship loading, transaction handling, and migration capabilities. Generate comparison report for decision making.

📄 scripts/schema-export.js ^(NEW) 🔗

References

• 📄 src/util/util/Database.ts ^(MODIFY)

Create script to export current database schema and data for migration testing. This script will connect to existing database, export schema definitions, and create test data dumps for validating migration accuracy.

📄 src/util/database/DatabaseService.ts ^(NEW) 🔗

Create main database service that acts as a facade for database operations. This service will initially use TypeORMAdapter but can be switched to the new ORM adapter once migration is complete. Provides methods like getRepository<T>(), transaction(), and initialize().

📄 src/util/entities/BaseClass.ts ^(MODIFY) 🔗

References

• 📄 src/util/database/DatabaseService.ts ^(NEW)

Gradually modify BaseClass to use the new DatabaseService instead of directly extending TypeORM's BaseEntity. Add compatibility methods that delegate to the database service while maintaining the same API surface. This allows entities to work with both TypeORM and the new ORM during transition.

📁 migration-scripts ^(NEW) 🔗

Create directory for migration scripts that will handle the transition from TypeORM to the new ORM.

📄 migration-scripts/validate-migration.js ^(NEW) 🔗

Create comprehensive validation script that compares data integrity between TypeORM and new ORM implementations. Validates that all entities, relationships, and data are correctly migrated without loss. Includes checksums, row counts, and relationship integrity checks.

📄 migration-scripts/migrate-entities.js ^(NEW) 🔗

References

• 📄 src/util/entities

Create script to systematically migrate entity definitions from TypeORM decorators to the chosen new ORM format. Handles conversion of relationships, column types, indexes, and constraints while preserving all existing functionality.

📄 package.json ^(MODIFY) 🔗

Add new ORM dependencies (based on evaluation results), update scripts to include migration and validation commands, and add development dependencies for ORM evaluation tools. Keep TypeORM as dependency during transition period.

📄 src/util/util/Database.ts ^(MODIFY) 🔗

References

• 📄 src/util/database/DatabaseService.ts ^(NEW)

Modify existing Database.ts to work alongside the new database abstraction layer. Add feature flags to switch between TypeORM and new ORM implementations. Maintain backward compatibility while allowing gradual migration of functionality.

📄 scripts/syncronise.js ^(MODIFY) 🔗

References

• 📄 src/util/database/DatabaseService.ts ^(NEW)

Update synchronization script to work with the new database abstraction layer. Add support for the new ORM's schema synchronization while maintaining compatibility with TypeORM during transition period.

📄 tests/database-migration.test.js ^(NEW) 🔗

References

• 📄 src/gateway/opcodes/LazyRequest.ts
• 📄 src/gateway/opcodes/RequestGuildMembers.ts

Create comprehensive test suite for database migration validation. Tests should verify data integrity, relationship consistency, query result equivalence between TypeORM and new ORM, and performance benchmarks. Include tests for complex scenarios like the query builder usage found in gateway operations.

Diagram

sequenceDiagram
    participant Dev as Developer
    participant Eval as ORM Evaluation
    participant Abs as Abstraction Layer
    participant TypeORM as TypeORM (Current)
    participant NewORM as New ORM
    participant DB as Database

    Note over Dev,DB: Phase 1: Evaluation & Planning
    Dev->>Eval: Create ORM prototypes (Prisma/Drizzle/MikroORM)
    Eval->>DB: Test with existing schema
    Eval->>Dev: Performance & compatibility report
    Dev->>Dev: Select optimal ORM

    Note over Dev,DB: Phase 2: Abstraction Layer
    Dev->>Abs: Create IRepository & IDatabase interfaces
    Dev->>Abs: Implement TypeORMAdapter
    Abs->>TypeORM: Delegate existing operations
    TypeORM->>DB: Continue current operations

    Note over Dev,DB: Phase 3: New ORM Integration
    Dev->>NewORM: Implement chosen ORM adapter
    NewORM->>DB: Parallel operations for testing
    Dev->>Abs: Add feature flags for ORM switching
    
    Note over Dev,DB: Phase 4: Entity Migration
    loop For each entity
        Dev->>NewORM: Migrate entity definition
        Dev->>Abs: Switch entity to new ORM
        Abs->>NewORM: Route operations to new ORM
        NewORM->>DB: Execute with new ORM
    end

    Note over Dev,DB: Phase 5: Cleanup
    Dev->>TypeORM: Remove TypeORM dependencies
    Dev->>Abs: Simplify to direct new ORM usage
    NewORM->>DB: All operations via new ORM

Loading

Import In IDE

🤖 Prompt for AI Agents

I have created the following plan after thorough exploration and analysis of the codebase. Follow the below plan verbatim. Trust the files and references. Do not re-verify what's written in the plan. Explore only when absolutely necessary. First implement all the proposed file changes and then I'll review all the changes together at the end.

### Observations

The Spacebar server is a Discord-like chat platform using TypeORM 0.3.25 with ~60 entities, complex relationships, and custom Snowflake IDs. The codebase has significant TypeORM coupling through static repository methods, query builders, decorators, and custom BaseClass inheritance. Migration files exist for MySQL/MariaDB/PostgreSQL/SQLite, and there are cyclic imports preventing standard TypeORM CLI usage.

### Approach

This migration will be executed in phases to minimize risk and ensure data integrity. We'll start by evaluating modern ORM alternatives (Prisma, Drizzle, MikroORM), create a database abstraction layer to decouple TypeORM usage, implement the chosen ORM alongside TypeORM, migrate entities incrementally, and finally remove TypeORM dependencies. The approach prioritizes backward compatibility and lossless data migration.

### Reasoning

I analyzed the codebase structure, examined the main Database.ts configuration file, reviewed entity patterns in BaseClass.ts and complex entities like User.ts and Guild.ts, searched for TypeORM usage patterns throughout the API routes, identified query builder usage in gateway operations, examined migration files across different database types, and reviewed database-related scripts to understand the full scope of TypeORM integration.

## Mermaid Diagram

sequenceDiagram
    participant Dev as Developer
    participant Eval as ORM Evaluation
    participant Abs as Abstraction Layer
    participant TypeORM as TypeORM (Current)
    participant NewORM as New ORM
    participant DB as Database

    Note over Dev,DB: Phase 1: Evaluation & Planning
    Dev->>Eval: Create ORM prototypes (Prisma/Drizzle/MikroORM)
    Eval->>DB: Test with existing schema
    Eval->>Dev: Performance & compatibility report
    Dev->>Dev: Select optimal ORM

    Note over Dev,DB: Phase 2: Abstraction Layer
    Dev->>Abs: Create IRepository & IDatabase interfaces
    Dev->>Abs: Implement TypeORMAdapter
    Abs->>TypeORM: Delegate existing operations
    TypeORM->>DB: Continue current operations

    Note over Dev,DB: Phase 3: New ORM Integration
    Dev->>NewORM: Implement chosen ORM adapter
    NewORM->>DB: Parallel operations for testing
    Dev->>Abs: Add feature flags for ORM switching
    
    Note over Dev,DB: Phase 4: Entity Migration
    loop For each entity
        Dev->>NewORM: Migrate entity definition
        Dev->>Abs: Switch entity to new ORM
        Abs->>NewORM: Route operations to new ORM
        NewORM->>DB: Execute with new ORM
    end

    Note over Dev,DB: Phase 5: Cleanup
    Dev->>TypeORM: Remove TypeORM dependencies
    Dev->>Abs: Simplify to direct new ORM usage
    NewORM->>DB: All operations via new ORM

## Proposed File Changes

### docs/TYPEORM_MIGRATION_PLAN.md(NEW)

Create comprehensive documentation outlining the migration strategy, timeline, and decision rationale. Include comparison matrix of ORM alternatives (Prisma vs Drizzle vs MikroORM), risk assessment, rollback procedures, and testing strategy. Document the current TypeORM usage patterns identified in the codebase analysis.

### src/util/database(NEW)

Create new database abstraction layer directory to house the new ORM implementation and provide a clean interface for database operations.

### src/util/database/interfaces(NEW)

Create interfaces directory for database operation contracts that will be implemented by both TypeORM (temporarily) and the new ORM.

### src/util/database/interfaces/IRepository.ts(NEW)

Define generic repository interface with methods like `findOne`, `findMany`, `create`, `update`, `delete`, `increment`, `decrement` that abstract away ORM-specific implementations. This interface will be implemented by both TypeORM and new ORM adapters to ensure seamless transition.

### src/util/database/interfaces/IDatabase.ts(NEW)

Define database connection interface with methods for initialization, transaction management, and repository access. This will abstract the database connection layer from the specific ORM implementation.

### src/util/database/adapters(NEW)

Create adapters directory to house ORM-specific implementations of the database interfaces.

### src/util/database/adapters/TypeORMAdapter.ts(NEW)

References: 

- src/util/util/Database.ts(MODIFY)

Create TypeORM adapter that implements the IDatabase and IRepository interfaces, wrapping existing TypeORM functionality. This adapter will delegate to the current `src/util/util/Database.ts` implementation and provide the abstraction layer needed for gradual migration.

### evaluation(NEW)

Create evaluation directory for ORM comparison prototypes and benchmarks.

### evaluation/prisma-prototype(NEW)

References: 

- src/util/entities/User.ts
- src/util/entities/Guild.ts

Create Prisma evaluation prototype directory with schema.prisma file generated from existing database structure using `prisma db pull`. Implement key entities (User, Guild, Channel) and test complex queries, relationships, and migration scenarios.

### evaluation/drizzle-prototype(NEW)

References: 

- src/util/entities/User.ts
- src/util/entities/Guild.ts

Create Drizzle evaluation prototype directory with schema definitions in TypeScript. Use `drizzle-kit introspect` to generate initial schema from existing database, then test the same scenarios as Prisma prototype for comparison.

### evaluation/mikroorm-prototype(NEW)

References: 

- src/util/entities/BaseClass.ts(MODIFY)

Create MikroORM evaluation prototype directory. Since MikroORM supports decorators similar to TypeORM, test how easily existing entity definitions can be adapted. Focus on decorator compatibility and migration path complexity.

### scripts/evaluate-orms.js(NEW)

Create evaluation script that runs performance benchmarks and feature compatibility tests across all three ORM prototypes. Test query performance, relationship loading, transaction handling, and migration capabilities. Generate comparison report for decision making.

### scripts/schema-export.js(NEW)

References: 

- src/util/util/Database.ts(MODIFY)

Create script to export current database schema and data for migration testing. This script will connect to existing database, export schema definitions, and create test data dumps for validating migration accuracy.

### src/util/database/DatabaseService.ts(NEW)

Create main database service that acts as a facade for database operations. This service will initially use TypeORMAdapter but can be switched to the new ORM adapter once migration is complete. Provides methods like `getRepository<T>()`, `transaction()`, and `initialize()`.

### src/util/entities/BaseClass.ts(MODIFY)

References: 

- src/util/database/DatabaseService.ts(NEW)

Gradually modify BaseClass to use the new DatabaseService instead of directly extending TypeORM's BaseEntity. Add compatibility methods that delegate to the database service while maintaining the same API surface. This allows entities to work with both TypeORM and the new ORM during transition.

### migration-scripts(NEW)

Create directory for migration scripts that will handle the transition from TypeORM to the new ORM.

### migration-scripts/validate-migration.js(NEW)

Create comprehensive validation script that compares data integrity between TypeORM and new ORM implementations. Validates that all entities, relationships, and data are correctly migrated without loss. Includes checksums, row counts, and relationship integrity checks.

### migration-scripts/migrate-entities.js(NEW)

References: 

- src/util/entities

Create script to systematically migrate entity definitions from TypeORM decorators to the chosen new ORM format. Handles conversion of relationships, column types, indexes, and constraints while preserving all existing functionality.

### package.json(MODIFY)

Add new ORM dependencies (based on evaluation results), update scripts to include migration and validation commands, and add development dependencies for ORM evaluation tools. Keep TypeORM as dependency during transition period.

### src/util/util/Database.ts(MODIFY)

References: 

- src/util/database/DatabaseService.ts(NEW)

Modify existing Database.ts to work alongside the new database abstraction layer. Add feature flags to switch between TypeORM and new ORM implementations. Maintain backward compatibility while allowing gradual migration of functionality.

### scripts/syncronise.js(MODIFY)

References: 

- src/util/database/DatabaseService.ts(NEW)

Update synchronization script to work with the new database abstraction layer. Add support for the new ORM's schema synchronization while maintaining compatibility with TypeORM during transition period.

### tests/database-migration.test.js(NEW)

References: 

- src/gateway/opcodes/LazyRequest.ts
- src/gateway/opcodes/RequestGuildMembers.ts

Create comprehensive test suite for database migration validation. Tests should verify data integrity, relationship consistency, query result equivalence between TypeORM and new ORM, and performance benchmarks. Include tests for complex scenarios like the query builder usage found in gateway operations.

---

Developer Humor

🔄 Why did TypeORM break up with Spacebar?
Because it was creating too many unnecessary tables
And the relationship was getting too tightly coupled!
💔 But don't worry, a new ORM is coming
To make the database queries sing! 🎵
⚡ Less coupling, more performance,
That's the migration dance! 💃

---

Execution Information

Branch: master
Commit: e031e72

---

💡 Tips

Supported Commands (Inside Comments)

• Use @traycerai generate to iterate on the previous version of the implementation plan.

Supported Commands (Inside Description)

• Add @traycerai ignore anywhere in the ticket description to prevent this ticket from being processed.
• Add @traycerai branch:<branch-name> anywhere in the ticket description to specify the target branch for the implementation plan.

Community

• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[erkinalp]

@devin-ai-integration you can do all four steps btw