What is a distributed master-less file storage?


Warning: count(): Parameter must be an array or an object that implements Countable in /home/styllloz/public_html/qa-theme/donut-theme/qa-donut-layer.php on line 274
0 like 0 dislike
22 views
All kind time of day!

Does anyone know of highly reliable distributed file storage?

5aedff0059ced662958069.pngSpecifics:

  • master-less, names-less - only one type of nodes. Without a separate metadata nodes, etc. (a good example: cassandra or cockroachdb)
  • recording source is always only one. No concurrent conflicting operations read/write
  • the number of nodes from 2 to 100
  • nodes are weakly potent, can appear and disappear (not instantly - you can count on sync, but the configuration is not fixed)
  • the delay of the write and read are not critical within reasonable limits (i.e. not HFT, but not batch mode)
  • support for redundancy - or rather, the guarantee of data recovery
  • linux only - windows and mac support is not important


Optional:

  • sharding - i.e. linear increase in the available size depending on the number of connected nodes
  • the file APIs, but also their own utilities, too


Researched:

  • gluster with replica - does not allow to add nodes to the number of more replica proof
  • xtreemefs - requires separate nodes for the metadata proof
  • cockroachdb* - requires at least 3 nodes for raft'a proof
  • cassandra* java, quite demanding


* it is clear that to write a wrapper in the database and/or make a piece of writing of scripts for sync case 2-3 days, but it will be the bike.
by | 22 views

3 Answers

0 like 0 dislike
You can look https://minio.io/ a La your S3 private, no masters, works on reading with the loss of half the nodes and for the record, if one lost less than half, knows how to heal. Maybe even one node, but the algorithms of fault begin working with 4 nodes. Perhaps it's not what you meant.
by
0 like 0 dislike
read towards ceph if not something major can be 1gb/s to drive, if direct HL need, without the 10gb/s can not do
by
0 like 0 dislike
https://habr.com/company/yandex/blog/228389/
Elliptics from Yandex. How to use it to create your fault-tolerant storage
by

Related questions

0 like 0 dislike
3 answers
0 like 0 dislike
2 answers
0 like 0 dislike
2 answers
0 like 0 dislike
7 answers
110,608 questions
257,187 answers
0 comments
40,796 users