I am trying to split my django settings into production and development. Th ebiggest question that I have is how to use two different databases for the two environments? How to deal with migrations?
I tried changing the settings for the development server to use a new empty database, however, I can not apply the migrations to create the tables that I already have in the production database.
All the guides on multiple databases focus on the aspect of having different types of data in different databases (such as users database, etc.) but not the way I am looking for.
Could you offer some insights about what the best practices are and how to manage the two databases also in terms of migrations?
EDIT:
Here is what I get when I try to run python manage.py migrate
on the new database:
Traceback (most recent call last):
File "/opt/homebrew/Caskroom/miniforge/base/envs/stokk/lib/python3.9/site-packages/django/db/backends/utils.py", line 85, in _execute
return self.cursor.execute(sql, params)
psycopg2.errors.UndefinedTable: relation "dashboard_posttags" does not exist
LINE 1: ...ags"."tag", "dashboard_posttags"."hex_color" FROM "dashboard...
^
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/....../manage.py", line 22, in <module>
main()
File "/....../manage.py", line 18, in main
execute_from_command_line(sys.argv)
File "/opt/homebrew/Caskroom/miniforge/base/envs/stokk/lib/python3.9/site-packages/django/core/management/__init__.py", line 425, in execute_from_command_line
utility.execute()
File "/opt/homebrew/Caskroom/miniforge/base/envs/stokk/lib/python3.9/site-packages/django/core/management/__init__.py", line 419, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/opt/homebrew/Caskroom/miniforge/base/envs/stokk/lib/python3.9/site-packages/django/core/management/base.py", line 373, in run_from_argv
self.execute(*args, **cmd_options)
File "/opt/homebrew/Caskroom/miniforge/base/envs/stokk/lib/python3.9/site-packages/django/core/management/base.py", line 417, in execute
output = self.handle(*args, **options)
File "/opt/homebrew/Caskroom/miniforge/base/envs/stokk/lib/python3.9/site-packages/django/core/management/base.py", line 90, in wrapped
res = handle_func(*args, **kwargs)
File "/opt/homebrew/Caskroom/miniforge/base/envs/stokk/lib/python3.9/site-packages/django/core/management/commands/migrate.py", line 75, in handle
self.check(databases=[database])
File "/opt/homebrew/Caskroom/miniforge/base/envs/stokk/lib/python3.9/site-packages/django/core/management/base.py", line 438, in check
all_issues = checks.run_checks(
File "/opt/homebrew/Caskroom/miniforge/base/envs/stokk/lib/python3.9/site-packages/django/core/checks/registry.py", line 77, in run_checks
new_errors = check(app_configs=app_configs, databases=databases)
File "/opt/homebrew/Caskroom/miniforge/base/envs/stokk/lib/python3.9/site-packages/django/core/checks/urls.py", line 13, in check_url_config
return check_resolver(resolver)
File "/opt/homebrew/Caskroom/miniforge/base/envs/stokk/lib/python3.9/site-packages/django/core/checks/urls.py", line 23, in check_resolver
return check_method()
File "/opt/homebrew/Caskroom/miniforge/base/envs/stokk/lib/python3.9/site-packages/django/urls/resolvers.py", line 446, in check
for pattern in self.url_patterns:
File "/opt/homebrew/Caskroom/miniforge/base/envs/stokk/lib/python3.9/site-packages/django/utils/functional.py", line 48, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "/opt/homebrew/Caskroom/miniforge/base/envs/stokk/lib/python3.9/site-packages/django/urls/resolvers.py", line 632, in url_patterns
patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
File "/opt/homebrew/Caskroom/miniforge/base/envs/stokk/lib/python3.9/site-packages/django/utils/functional.py", line 48, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "/opt/homebrew/Caskroom/miniforge/base/envs/stokk/lib/python3.9/site-packages/django/urls/resolvers.py", line 625, in urlconf_module
return import_module(self.urlconf_name)
File "/opt/homebrew/Caskroom/miniforge/base/envs/stokk/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 850, in exec_module
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "/....../app/urls.py", line 11, in <module>
from main_platform.views.investor import AccountView, profile, app_home_redirect
File "/....../main_platform/views/investor.py", line 118, in <module>
class PostFilter(django_filters.FilterSet):
File "/....../main_platform/views/investor.py", line 124, in PostFilter
for tag in PostTags.objects.all():
File "/opt/homebrew/Caskroom/miniforge/base/envs/stokk/lib/python3.9/site-packages/django/db/models/query.py", line 280, in __iter__
self._fetch_all()
File "/opt/homebrew/Caskroom/miniforge/base/envs/stokk/lib/python3.9/site-packages/auto_prefetch/__init__.py", line 98, in _fetch_all
super()._fetch_all()
File "/opt/homebrew/Caskroom/miniforge/base/envs/stokk/lib/python3.9/site-packages/django/db/models/query.py", line 1354, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/opt/homebrew/Caskroom/miniforge/base/envs/stokk/lib/python3.9/site-packages/django/db/models/query.py", line 51, in __iter__
results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
File "/opt/homebrew/Caskroom/miniforge/base/envs/stokk/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1202, in execute_sql
cursor.execute(sql, params)
File "/opt/homebrew/Caskroom/miniforge/base/envs/stokk/lib/python3.9/site-packages/django/db/backends/utils.py", line 99, in execute
return super().execute(sql, params)
File "/opt/homebrew/Caskroom/miniforge/base/envs/stokk/lib/python3.9/site-packages/sentry_sdk/integrations/django/__init__.py", line 563, in execute
return real_execute(self, sql, params)
File "/opt/homebrew/Caskroom/miniforge/base/envs/stokk/lib/python3.9/site-packages/django/db/backends/utils.py", line 67, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/opt/homebrew/Caskroom/miniforge/base/envs/stokk/lib/python3.9/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/opt/homebrew/Caskroom/miniforge/base/envs/stokk/lib/python3.9/site-packages/django/db/backends/utils.py", line 85, in _execute
return self.cursor.execute(sql, params)
File "/opt/homebrew/Caskroom/miniforge/base/envs/stokk/lib/python3.9/site-packages/django/db/utils.py", line 90, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/opt/homebrew/Caskroom/miniforge/base/envs/stokk/lib/python3.9/site-packages/django/db/backends/utils.py", line 85, in _execute
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: relation "dashboard_posttags" does not exist
LINE 1: ...ags"."tag", "dashboard_posttags"."hex_color" FROM "dashboard...
2
Answers
Found the solution in this post
Apparently, Django tries to evaluate all queries declared on class level before running the migration. This causes an error because the database is empty, and the query can not run.
To solve this, comment out everything in the main
urls.py
, so that there are no imports and urlpatterns. Then runpython manage.py makemigrations
and after it has completed, revert theurls.py
to the original.If you have a new empty database, you can just run "python manage.py migrate" and all migrations will be executed on the new database. The already done migrations will be stored in a table in that database so that django always "remembers" the migrations state of each individual database. Of course that new database will only have the tables structure – there is not yet any data copied!
Does this answer your question?