skip to Main Content

I’m about to start a new project which should have a rather large database.

the records have roughly up to 10 fields , these datas will have following specifications :

  • Only insert and read records will be performed
  • There will be 8603876000 record stored at the beginning (insert time and performance not important)
  • Every 10 min 500 new record will be added to db
  • Read query performance is so important ( to find relations between fields )

i will be happy if you share your experience and ideas

2

Answers


  1. i think need a database that can handle large data volume and fast read queries.

    MongoDB: A non-relational database that stores data in JSON-like documents. It is considered to be the best database for large amounts of text and the best database for large data. It also supports replication and sharding for high availability and scalability.
    Microsoft Azure: A cloud platform that offers a wide range of database software and management options, including relational and non-relational databases. It allows you to build your own databases or manage existing ones with various tools and features

    Redshift: A cloud-based data warehouse that is optimized for analytics and fast query performance. It can handle petabytes of structured and semi-structured data using columnar storage and massively parallel processing

    BigQuery: A serverless data warehouse that runs on Google Cloud Platform. It can process terabytes of data in seconds and petabytes in minutes using SQL-like queries. It also supports streaming data ingestion and machine learning integration

    MySQL: A free and open source relational database that is popular for its ease of use. It can handle large loads of data with multiple requests per second. It also supports replication and partitioning for high availability and scalability.

    i hope this helps you✅❤️

    Login or Signup to reply.
  2. When taking the given problem, I feel considering the following will be helpful for you.

    • Consider using a NoSQL database if your data is non-relational. NoSQL
      databases like MongoDB can be faster than traditional relational
      databases for certain types of queries.
    • Use a load balancer to
      distribute the workload across multiple servers. This can improve
      performance and increase reliability by ensuring that no single
      server is overloaded.
    • And if you need more performance, sometimes it will be good to consider partitioning the database to improve performance. Partitioning involves splitting the database into smaller, more manageable sections.
    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search