Select by color - with swatches

This is kind of a search function.

I'd like to search by color, but without having to know the color before I start searching. In my perfect scenerio I'd be able to see a set of swatches of all colors for fill and another set for outlines, then click a swatch to select all objects, even in groups, of that color. Below or next to each swatch would be the color in its native format, might be rgb, hex, cmyk, or whatever.

Sometimes there are multiple colors that all look very similar, it would also be great to select a set of swatches and merge them to the same color.

it would be great to select 2 swatches and 'switch' the colors with each other.