How to create a cluster of Raspberry Pi?

You can distribute. It is the task of the application.

Collect a cluster of Raspberry Pi is meaningful only for fun. Performance is more appropriate to take the Xeon which is not the newest generation.
How to process the data in a cluster very much depends on the application, and the overall effective solution that will allow you with a dozen Raspberry Pi to make a virtual machine with at least 5x the performance compared to one system, no.

The principle is the same as you described: whence it takes the data, processes it, and sends it somewhere result. More precisely, it is impossible to tell from such a General question. For example, mining with the use of a pool is one example of how a bunch of computers can work on solving the General problem.

In professional data centers fast network, direct memory access and a bunch of other tricks that are even approximately is not available on the Raspberry Pi. I think for training it makes more sense to raise the new path on the desktop and configured between the virtual network.

Perhaps the most effective method in this case, is to run separate instances on each piece of iron that will handle data from one network storage (database)