When running this ActiveRecord migration on the test (not production) database, large changes in db/structure.sql
appear. This is reproducible. The changes are:
- A small number of the expected changes based on the migration code (as desired).
- A large number of unexpected changes unrelated to the migration code (not desired). All these unexpected changes rearrange the order of the tables, views and materialized views that are not mentioned in the SQL code in the migration. These rearranged items are not associated (e.g., by dependencies) with the items in the SQL code in the migration.
What causes these large unrelated differences, and how to prevent them?
Could they be related to the mismatch between the postgresql versions on the production database server and on the machine where the migrations are executed on the test database?
These unrelated changes make git diff
output much less useful. In this tiny minimal example below, just a few tables are rearranged. But in the real life migration code it changes the order of dozens of tables, views and matviews.
class FixCreateImmuneProductionQualityMetricsViewsWi < ActiveRecord::Migration[5.2]
def up
sql = <<~SQL
DROP VIEW master_foo;
DROP VIEW IF EXISTS bar1;
CREATE OR REPLACE VIEW bar1 AS
SELECT * FROM baz1
UNION ALL
SELECT * FROM baz2;
CREATE OR REPLACE VIEW master_foo AS
SELECT
*,
'bar1' AS bar_type
FROM
bar1
UNION ALL
...
SQL
execute sql
end
def down
sql = <<~SQL
DROP VIEW IF EXISTS master_foo;
DROP VIEW IF EXISTS bar1;
...
SQL
execute sql
end
end
Hardware/software used:
Machine where migrations are executed:
MacBookPro, Apple M1 Max, macOS Sonoma 14.2.1
ruby 2.7.3p183 (2021-04-05 revision 6847ee089d) [arm64-darwin21]
15.3 psql (PostgreSQL) 15.3 (Homebrew)
Production database server:
psql (14.9 (Homebrew), server 13.3 (Ubuntu 13.3-1.pgdg20.04+1))
2
Answers
Yes, this is mostly due to different versions of the database engine you’re using between different machines and/or configurations. Other possible cause is: Rails (Active Record) internal changes for better readability or consistency.
Remember that differences in the order of tables or views in the
structure.sql
(orschema.rb
) file generally don’t impact the functionality of your Rails application. But be careful, if you see other kind of changes, review them carefully (for example, in MySQL I’ve seen changes in the collation or the charset, which can cause actual weird problems in your app).Regarding the "order", this can be forced by using the
:before
and:after
options in your migrations. Example:Those options also work for columns:
No, this is not related to different DB versions, it depends more on the order that different users run the migrations. If you have all your team blow away their DBs, and reload them from a common dump, then they’ll all produce the same output… until someone runs a migration in a different order (which obviously will happen as soon as more than one user creates a new migration).
Unfortunately, there’s no simple/nice way to resolve this.