Monday, 19 August 2013

Hardware for a sharded + redundant MongoDB-environment

Hardware for a sharded + redundant MongoDB-environment

I'm trying to plan a new database-environment from scratch and I'm
wondering how many servers are needed and how much performance they should
provide.
Since I want it to be fast, I'm considering using SSD-memory and loads of
RAM. However, flash-memory is expensive and makes up for the biggest part
of the cost of a server. Thus, the entire system should be set up for
horizontal scaling from the start, so I can add more nodes when I need
more storage/performance.
To get started I'm thinking of using 2 shards, each consisting of a master
and a replica-slave for redundancy. MongoDB-documentation suggests using 1
master and 2 slaves, but I'm afraid that won't be in the available budget,
since each of these servers will be equipped with about 200 GB of RAM and
6x400 GB SSD as Raid 10.
When using shards, it is also suggested using 3 config-servers for
failsafe/high-availability. Same as above, I'm thinking 1 master and 1
slave as a start.
What sort of hardware would you recommend to put the config-servers on?
Should they be somewhat equally performant as the shard-nodes in terms of
cpu/memory/harddisk? Or can I put them on virtualization or on cheaper
hardware?
Does the setup I described even make sense? How about the ratio of RAM vs.
harddisk on the shard-nodes? At the moment it would probably be easier and
cheaper to just put twice the number of discs into 1 shard (1 master, 1
slave) and skip the sharding until I really need it. However (as mentioned
above) - the system should be ready for sharding from the beginning,
because storage-needs can change overnight. Or is it possible to set it
all up, but have it running on 1 shard only for now?
Since I'm only planning to use 2 instead of 3 servers for
high-availability/failsafe I probably need arbiters as well. Do they need
dedicated hardware as well? Or can I use one arbiter in a virtual-mashine
that serves config-servers and shard-nodes? Or is using 3 seperate servers
for redundancy an absolute must in your opinion?
Any insights highly appreciated. Thanks and best regards!

No comments:

Post a Comment