Skip to content

Feat: Entity-first migrations#3015

Open
sinder38 wants to merge 5 commits intoSeaQL:masterfrom
sinder38:feat/entity-migrations
Open

Feat: Entity-first migrations#3015
sinder38 wants to merge 5 commits intoSeaQL:masterfrom
sinder38:feat/entity-migrations

Conversation

@sinder38
Copy link
Contributor

@sinder38 sinder38 commented Mar 19, 2026

PR Info

Goal

Safe, production ready migration workflow.

PR Todo

CLI

  • Add entity crate generation to sea-orm-cli
  • --preview option for generate ( discovery without creating the migration)
  • fluff ✨
  • interactive (handle renames)
  • emit warning (destructive changes, assumptions)

Core

  • Implement generate_migration
  • Capture DDLs instead of applying them
  • Write output as raw SQL wrapped in db.execute_unprepared with timestamped filename
  • Allow dangerous change discovery
  • Migration summary
  • Warn on "obvious" data migration (some changes like adding a NOT NULL column obviously would require additional input from the developer)
  • Migrate down place holdert
  • Discover possible renames
  • Discover constraints ( would require a quick fix or sea-schema update)

Integration into the rest of codebae

  • Auto-update migration/src/lib.rs to register the new migration
  • Solve repetition issue (2 crates: entity, migration)
  • Solve migration tracker protection issue

Quality

  • Tests
  • Document the new workflow

Out of scope (probably)

  • Generate sea_orm statements instead of raw sql (I want this one myself, but it may take too long)
  • Generate proper down (not very useful)
  • Allow offline migration ( I need to check if dependencies for that already exist)
  • Generate data transition (hardest by far, definitely not doing that)

@sinder38
Copy link
Contributor Author

sinder38 commented Mar 19, 2026

In terms of design:

  1. I created a separate crate for entity-first migrations instead of extending the migrations crate directly, for easier reading. Currently it imports the migrations crate and builds on top, but I feel in the final version they should be merged into one.
  2. Currently CLI functionality is not in sea-orm-cli again, to avoid the mess.
  3. I changed how sync() works. Now it uses discover() for discovery and applies the changes. discover() itself is used by the entity-first migrator for detecting changes.
  4. Also, discover() with dangerous turned on does not protect against deleting the migration tracker table. This is handled later down the line in "entity". I will change that, but I am not sure what the best approach is yet.

Other:

I think the migrations CLI output can be improved slightly in terms of clarity

NOT READY FOR MERGE THOUGH
Please review @Huliiiiii

@tyt2y3
Copy link
Member

tyt2y3 commented Mar 20, 2026

oh wow, looks promising!

@sinder38
Copy link
Contributor Author

This is going to take longer than expected. I underestimated implementing proper change discovery phase

Comment on lines +65 to +80
#[arg(
long,
default_value = "true",
help = "Generate migration file based on Utc time",
conflicts_with = "local_time",
display_order = 1001
)]
universal_time: bool,

#[arg(
long,
help = "Generate migration file based on Local time",
conflicts_with = "universal_time",
display_order = 1002
)]
local_time: bool,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we merge this into a single option, like filename-timezone / migration-timezone?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I thought the same. just copied this part from original migrator.

let url = cli
.database_url
.expect("Environment variable 'DATABASE_URL' not set");
let schema = cli.database_schema.unwrap_or_else(|| "public".to_owned());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use clap's default_value.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes! again, just copied this part from original migrator.

// Future: when --ephemeral is added, build an in-memory db here instead,
// apply existing migrations via the migrator, then pass it to generate_from_db.

if let Err(e) = generate_from_db(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Too many parameters. Use a struct instead.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay

.to_owned(),
)
.await
.expect("Failed to connect to database");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be better to expose the reason to the user.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, same thoughts.
again, just copied this part from original migrator.

let builder = entity_set.register(schema.builder());

println!("Discovering schema changes...");
let raw = builder.discover(&db, dangerous).await?;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

raw…what?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea, was that it was unprocessed changes, will change

{
let _existing = Self::discover_existing_schema(db).await?;

#[allow(unreachable_code)]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why so many #[allow(unreachable_code)]?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copied from original. I just decided not to think about it. I the new discover() I removed them.

let in_entities = self
.entities
.iter()
.any(|e| get_table_name(e.table.get_table_name()) == existing_name);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rename get_table_name to something more semantic.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, but I don't see a problem.

// and generate drop statements for them.
#[allow(unreachable_code)]
if allow_dangerous {
for existing_table in &_existing.tables {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How will this handle multiple schemas?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did't cover multiple schemas. as most of the sea architecture doesn't support them as well

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, leave a TODO comment here so we don't forget to handle this during future refactoring.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't worry I am doing rewrite of discover() since yesterday.

Comment on lines +342 to +348
fn discover_changes(
&self,
db: &C,
db_backend: DbBackend,
existing: &DiscoveredSchema,
created_enums: &mut Vec<Statement>,
) -> Result<(), DbErr> {
let db_backend = db.get_database_backend();

changes: &mut Vec<Statement>,
allow_dangerous: bool,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Too many params.

.table(
self.table
.get_table_name()
.expect("Checked above")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indents too deep.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay, will change

@sinder38
Copy link
Contributor Author

sinder38 commented Mar 21, 2026

Thanks for feedback! I didn't have experience developing CLI in rust before.
The most important question is about the multiple schemas. Am I mandated to support them at this point?
Also on the note of CLI QoL and formatting. I am currently inheriting some of the bad in my opinion formatting from migrator, it it fine to change some parts of it?

I just wanted to say that I mostly wanted to discuss (#3015 (comment)) the integration with existing code and not get exact code review, as this first draft for a reason

@Huliiiiii

@Huliiiiii
Copy link
Member

Oh, I got confused. I thought you wanted me to review the code above.

As for the refactor, I think we can merge it back into the CLI crate first, and then see how it goes. For the more significantly changed parts, feel free to refactor them, since that should make the diff clearer.

For the rest of the architecture, I agree with calling discover inside sync.

I think a better approach is to update the migration crate so that discover either accepts options (or add another method that does) or returns a structured change plan instead of Vec<Statement> to avoid manually inspecting the SQL string.

Ideally, both should be implemented.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Entity first migration-generation

3 participants