Because my database file is so big in mongoDB

2

I have a query with the size of the database in mongoDB. I copied a Postgres table from 3 million records to mongoDB.

The table in Postgres weighs 400Mb. The collection in mongoDB weighs 1.6Gb

Is there something I'm doing wrong or just in mongoDB it's more expensive to save data?

Thanks

    
asked by Gonzalo Fernandez Larravide 19.08.2016 в 20:49
source

2 answers

1

It happens because MongoDB uses pre-allocation of space for each of the files and keeps records ( journals ) of the history of use of the files.

There is a good answer here and I translate just a summary of how it works on MongoDB 3 below. The original answer comes from the information in the MongoDB FAQ

Pre-allocating files

MongoDB tries to avoid fragmentation on your hard drive by pre-allocating a size to each file. This is why the size on your hard drive may be larger than the size that is actually in use by your data.

This is probably the biggest cause of the difference you find. The storage.mmapv1.smallFiles option allows you to reduce the size of these files.

The oplog.rs registry

If the mongod you are using is a member of a replica set, the data folder includes a oplog.rs file. Here is a capped collection , which is a limited data set that is used as a buffer to streamline replication tasks.

Usually this adds about 5% to the volume of your data.

The registration

The data folder contains all the log files ( journal files ) that store changes on disk before applying it to the database.

Empty records

MongoDB maintains a list of empty records while deleting documents and collections. This space can be re-used, but it is not the default behavior.

    
answered by 21.08.2016 в 17:04
0

Depends on storage engine (storage engine) you're using.

By default, from mongoDB version 3.2, the storage engine is Wired Tiger, which generally uses less space because compress the data before writing it to disk. Previously it is MMAPV1.

These are just the defaults. The configuration is found in the mongod.conf file, which is commonly found in the / etc directory. Find the storage section to see what engine you are using:

storage: engine: mmapv1 or storage: engine: wiredTiger

If you are using MMAPV1, it is best to export the database using mongodump and change the configuration to WiredTiger by editing mongod.conf . Once done, restore using mongorestore.

    
answered by 28.08.2017 в 17:26