If you’ve always thought of paintings as individual inspirations of artists that can’t be linked to each other, then think again. Researchers at MIT have successfully created a new algorithm aka MosAlc System that uses stylistic connections to indicate a connection between paintings at the Metropolitan Museum of Art (MET) and Amsterdam’s Rijksmuseum. MIT’s Department of Computer Science and Artificial Intelligence Laboratory and Microsoft are the researchers behind this discovery. This connection between paintings is otherwise unnoticed by humans.
This algorithm can find a connection between the paintings of artists who’ve never even met. It uses deep networks to identify paired works from different artists and cultures. The algorithm is known as the “MosAlc System” and is inspired by the “Rembrandt and Velazquez” exhibit at Rijksmuseum. One of the examples the researchers cited was the connection in the exhibit Francisco de Zubarán’s. Researchers claim that Martyrdom of Saint Serapion and Jan Asselijn’s The Threatened Swan have visual similarities in terms of their posture.
How does the MosAlc System Work?
MosAlc System basically provides you with a similar image as an answer to your query. For example, let’s imagine that you ask the algorithm “Which musical instrument is the closest to this painting of a blue and white dress?” To answer your question, the MosAlc System will present to you a blue and white porcelain violin. This violin can be connected to the cultural exchange between the Dutch and the Chinese.
Have you ever heard about Google’s X degrees of separation experiment that connects two images using a series of paintings? The MosAlc System works exactly like this. However, the only difference between the two is that the MosAlc System requires only a single image to find analogous stylistic images. So, when you feed an image to the MosAlc System, it finds other similar images by scanning through various cultures of the world.
The main challenge faced by the MIT researchers was to develop the MosAlc System in a way that it not only identifies similar colors but also themes. Mark Hamilton, the lead author on the research and his colleagues relied on the new K-Nearest Neighbor (KNN) data structure to build the algorithm. KNN puts similar images in the form of a tree-like structure and wanders around until they find the closest result. This feature was used in the MosAlc system and then applied to the open-access collections of the MET and Rijksmuseum.
Although it is still not clear whether the MosAlc System can differentiate between deepfakes and genuine images, it is definitely the start of something bigger. This is a groundbreaking discovery to identify similarities among different cultures of the world. It will certainly help people across the globe view these historical paintings in a new light.
