Question 116630 in Mongodb Question posted in
The official documentation can be found here.

Mongodb – access data and run jobs using same application, is it a good practice?

I have a spring boot application that I use to access and fetch a MongoDB database, this application use also a JobRuner solotion to run some ETL jobs and write their results in the same Mongodb database, furthermore this application expose some Rest web-services to be consumed by other applications.

My application is working fine, but sometimes it does not return back responses, when I looked for the reason why I did not find anything weird.

Also, in the beginning we had many applications that access the same database, and we wanted to have a single source of truth so we created this application which itself will expose results for them instead of letting them access our database.

I’m wondering if that is because of the jobs it launches, especially that it is often down when the jobs are running (but I’m not sure yet!)

Tags:

2

Answers


  1. 0

    this is design / architectural decision question.

    i) If both( ETL and REST) use the same DB Schema – Accessing the data and running the ETL job on the same database – not a good practice generally. Reasons- when the ETL runs ( may be for minutes or hours?) how do you make sure the data accuracy that are exposed by REST services? During that time – data will not be in consistent state as they might be getting changed by ETL job. So the REST clients might get inaccurate data.

    ii) if ETL and REST are involved with different schema, for small to medium size application this is ‘kind of ok’ but not the best one. But DB performance might be a challenge.

    iii) For small size application this solution should be ok. But make sure the ETL runs only at midnight when the REST accesses are nil or minimum.

    Login or Signup to reply.
  2. 0

    Agree with D.S.
    its a architect level question and individual architect for project should decide if they should be together or running differently.

    1. If I am the architect for any project, I will keep both separate. No one knows what will come next and we have to do some major changes in any of modules either job or Rest api’s.
    2. Also in case I want to do any changes in one of module, I have to take whole applications which will stop accessing API’s as well as job. This may not impact much if it is load balanced, but you never know. Better to stay ahead of known issues rather than complicating them.
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search