Understand that many years have passed, but suddenly someone come in handy later :)
In fact, the situation with flowers - it is simple, because it is a discrete space. Roughly speaking, we have RGB - coordinates of a point in three-dimensional space. The point is the area of a sphere of a certain radius. The points that lie on the surface of the sphere equidistant from the starting point to a distance equal to the radius of the sphere. Task is to limit the search to direct the search, i.e., we take a sphere with a radius of 0 - point different from ours - so they are coming, not - increase the radius to 1, there are points - excellent, found no increase the radius further, etc. is Possible, then you can not use direct search and a bisection in a specified interval - there is the creative moment :) As all of this deal to realize (it would work well for the situation above, when the search on a fixed set it is necessary to perform very many times).
1. Create three array [0..255]. Arrays R, G, B. the array Element is the set of indices of elements in the source array of pixels. Ie let the point with color 100, 25, 250 has an index of outcome in the array 200. Then we should make R.add(200); G.add(200); B.add(200);
2. As we fill the arrays in a sequence, each element of the arrays R,G,B will have an ordered set. If, for some reason, it's not that the second step should be to organize each set.
3. Build three two-dimensional UR array[0..255,0..255]. UG. UB. Where UR[i,j] = the intersection of the sets R[i] and R[j]. The beauty is that many have already ordered and their intersection is being built quite quickly. You can read more here https://habrahabr.ru/post/250191/
Our preparatory work has ended. It will take some time, but further search will be carried out very quickly.
So, actually the search itself is the nearest point to the desired simple. For i=0 to 255. the coordinates of the required point r, g, b. Take a set of values UR[r-i, r+i] is concatenated to the URi, so for each array. Then look at the intersection of the sets URi, UGi, UBi - if empty, i++, otherwise any (or many - depending on tasks) of the found elements of the original. Of course, it is necessary to watch, that would not overstep the bounds of arrays, etc., the main thing - the idea - we are looking for the closest points as the points lying on the surface of a sphere centered at the origin, with the smallest radius. But that would not run every time through the array - we have prepared three arrays with the projected points for each axis.
Would be limited to arrays R, G and B, but then each search would have to find a junction is, perhaps, more profitable for a small number of iterations of the search. For a large more profitable to build many intersections and look for them.