I’m about to start a new project which should have a rather large database.
the records have roughly up to 10 fields , these datas will have following specifications :
- Only insert and read records will be performed
- There will be 8603876000 record stored at the beginning (insert time and performance not important)
- Every 10 min 500 new record will be added to db
- Read query performance is so important ( to find relations between fields )
i will be happy if you share your experience and ideas
2
Answers
i think need a database that can handle large data volume and fast read queries.
MongoDB: A non-relational database that stores data in JSON-like documents. It is considered to be the best database for large amounts of text and the best database for large data. It also supports replication and sharding for high availability and scalability.
Microsoft Azure: A cloud platform that offers a wide range of database software and management options, including relational and non-relational databases. It allows you to build your own databases or manage existing ones with various tools and features
Redshift: A cloud-based data warehouse that is optimized for analytics and fast query performance. It can handle petabytes of structured and semi-structured data using columnar storage and massively parallel processing
BigQuery: A serverless data warehouse that runs on Google Cloud Platform. It can process terabytes of data in seconds and petabytes in minutes using SQL-like queries. It also supports streaming data ingestion and machine learning integration
MySQL: A free and open source relational database that is popular for its ease of use. It can handle large loads of data with multiple requests per second. It also supports replication and partitioning for high availability and scalability.
i hope this helps you✅❤️
When taking the given problem, I feel considering the following will be helpful for you.
databases like MongoDB can be faster than traditional relational
databases for certain types of queries.
distribute the workload across multiple servers. This can improve
performance and increase reliability by ensuring that no single
server is overloaded.