Search Your Own Images

In last week's post, An NN for Chess Images?, I used a photo from my archive and wondered,

Could a network recognize that this is not a photo? Or (without being told) that it shows dogs. My first step should probably be to make some sort of survey of what software and services are available.

First stop: images.google.com. This simple tool allows search on an image via a number of input paths. The most straightforward are:-

• Copy and paste the URL for an image, and
• Upload an image.

Both of these are accessed by clicking the camera icon in the search box on images.google.com. I fed it the link for the painting/photo used in 'NN for Chess Images', and it gave me three results. Two of these ('Best guess for this image') were pages about Dalmatian dogs (well done, Google!) and the third was a link to the home page for this blog, where the original post is still displayed because it is among the most recent.

The results also included a dozen thumbnails for 'Visually similar images'. None of the thumbnails showed a Dalmatian, but all of them had the same sort of pink background shown in my original photo. (Remember Chess in the Pink, April 2018?) Unfortunately, I can't give a link to the Google results, because it is based on some sort of encryption technique that probably includes details about its origin (i.e. me).

After that little experiment, I fed Google image search the link to the image used in 'Mystery Painting' on eBay (June 2017). This is a painting that pops up occasionally in different places, but where I have been unable to determine its origin. This time the Google results could find nothing more interesting about the image than that it showed a 'picture frame'. Under 'Visually similar images', the dozen thumbnails displayed different paintings in picture frames.

Given a similar image used in my original Mystery Painting (December 2007), Google determined that it showed 'edelen middeleeuwen', which another Google service translates from Dutch to 'nobles middle ages', which I translate to 'nobles from the Middle Age'. Along with a couple of links to Dutch-language pages, the thumbnails showed one more copy of my original image.

The Google image search clearly has limitations on its usefulness. As a final test I used an image which has been sitting in my 'Junk' directory for over a year, not because the image is junk, but because I didn't know what to do with it.

After I uploaded the image, Google declared, 'Best guess for this image: donald trump chess', where the first link (from 'About 25.270.000.000 results'!) was Does Donald Trump play chess? (quora.com). About half of the thumbnails were variations on my uploaded image; the other half were people playing chess, including two showing Garry Kasparov. The Trump image is obviously photoshopped, where a Star Trek 3D chess set has been pasted over whatever Trump was holding in his hands.

According to a database I constructed, my collection of images (mostly) from eBay has about 20.000 examples. Since uploading a small percentage of these one-by-one would be an exercise in tedium, I'll have to find a better solution.