Sabine Lang & Bjorn Ommer
At Heidelberg University, the collaboration between computer scientists and art historians has resulted in innovative ways to conduct research and new research tools to analyze large image corpora. A web-based interface for image retrieval has been implemented and successfully applied onto art historical data; research questions mainly focused on reception theory and the identification of schools or individual artists. This is of great value to art history, because it allows the evaluation of big data and thus facilitates general judgments about the history of art.
In comparison to other retrieval tools, our approach shows major benefits: Our algorithm is unsupervised, operates on pure visual qualities and does not rely on labeled image data. Other approaches often use a bag-of-words-model, where image elements are ordered similar to words in a glossary, neglecting spatial information and geometry. Although this allows for a faster search, it also leads to the fact that only objects listed in the glossary are found. Our algorithm finds identical and similar image regions, because it is based on a shape-sensitive classifier. The interface itself is easy to operate and allows to search for up to five image regions simultaneously. Results are then presented as an one-dimensional image atlas, where patterns are already visible, and can be evaluated by the user. Based on the user’s feedback, an iterative search process leads to improved results.
The interface has been applied onto a diverse range of image collections, characterized by different genres, materials or subject matters. Tests showed very good results throughout, giving proof that our algorithm is indeed applicable to diverse scenarios. One project included the detailed analysis of the Sachsenspiegel1, a medieval law book, which is attributed to Eike of Repgow (1180-1235). The manuscript exists in four versions and thus provides a basis to study reproduction practices and variations between editions. Approaches included the automatic detection of multiple types of gestures within the digitized manuscripts. The study of legal gestures was not only relevant to art history, but also to the discipline of law. Commonly, it is assumed that legal gestures are codified, especially when illustrated in law books. Our algorithm, however, demonstrated that gestures within the Sachsenspiegel varied, even when referring to one gesture, such as speaking or swearing. Thus, artistic freedom was of greater importance than hitherto assumed. The analysis also allowed statements about the most commonly used gestures. Moreover, two versions for speaking could be determined as well as pointing being the most variant gesture within the Sachsenspiegel.2
The interdisciplinary structure of the group allows for a direct evaluation of findings by an art historian or a computer scientist. This leads to a prompt feedback system and the possibility to direct to future research questions relevant to art history or computer vision. The close proximity and mutual understanding of the other discipline is of great value.
1. The versions of the Sachsenspiegel have been digitized and are available on the website of the University of Heidelberg Library.
2. Schlecht, Joseph, Bernd Carqué, and Björn Ommer. “Detecting gestures in medieval images.” Image Processing (ICIP), 2011 18th IEEE International Conference on. IEEE, 2011.