Analyze Your Own Images

In last week's post, Search Your Own Images, I noted 'First stop: images.google.com', fed it a few images, and finished with a test on a special image:-

After I uploaded the image, Google declared, 'Best guess for this image: donald trump chess', where the first link (from 'About 25.270.000.000 results'!) was 'Does Donald Trump play chess?' (quora.com).

Second stop: How Image Recognition Works (upwork.com). After a brief, concise introduction to the technology behind image recognition, the article mentioned,

A number of APIs have recently been developed that aim to allow organizations to glean insights from images without requiring in-house computer vision or machine learning expertise.

This was followed by a list of three specific services:-

• Google Cloud Vision
• IBM Watson Visual Recognition
• Clarif.ai [NB: clarifai.com?]

That first suggestion led me to Vision API - Image Content Analysis (cloud.google.com/vision), which said,

Google Cloud Vision API enables developers to understand the content of an image by encapsulating powerful machine learning models in an easy to use REST API. It quickly classifies images into thousands of categories (e.g., "sailboat", "lion", "Eiffel Tower"), detects individual objects and faces within images, and finds and reads printed words contained within images.

I fed last week's Donald Trump 3D-chess image to 'Try the API' and it returned the following analysis.

The 'Faces' tab is shown in the image -- 'Joy: Very Likely', 'Confidence 94%'. The next tab, 'Labels', told me:-

96% Indoor Games And Sports
96% Chess
94% Games
94% Board Game
81% Tabletop Game
73% Recreation
71% Chessboard

The next tab, 'Web', included (1) 'Web Entities':-

1.0731 Donald Trump
1.0593 Chess
0.7329 United States
0.7046 Three-dimensional chess
0.6912 Brilliant Chess

And (2) 'Pages with Matched Images' with dozens of links, of which the first two were:-

• Trump Is Playing 4D Chess (knowyourmeme.com), and
• Why I Am Absolutely Certain That Trump Is Not Playing 3D Chess (medium.com)

The 'Document' tab drew a green rectangle around the lowest of the three chessboards, with no further explanation. I suppose this is related to the service that 'reads printed words contained within images'. The 'Properties' tab returned 'Dominant Colors' and 'Crop hints', while the 'Safe search' tab was the most curious:-

Adult: Unlikely
Spoof: Very Unlikely
Medical: Very Unlikely
Violence: Very Unlikely
Racy: Possible

I would say, 'Spoof: Very likely' and I have no idea where 'Racy: Possible' comes from. As for the last tab, 'JSON', this returned a text list to import the image's attributes into another process.

Who knew there was so much information in a single photo? Google knew. Google knows everything.