Image Classifier for Annotations

At the time of the research for The Library is Open, a point of interest for everyone was annotations.

The Library is Open was a research project focused on knowledge production and its systems. The work questioned the shadows and biases cast by knowledge taxonomy; examined digital proprietary tools as impediments to the free access and circulation of knowledge; and developed annotation tools to foster collective interpretation towards existent knowledge.

As a research group, we were reading and annotating texts together and debating the possibilities of sharing these notes. One particular discussion was about what could/should be considered an annotation: folding corners of pages, linking to other contents, highlighting, scribbling, drawing. I was curious if we could train a computer to see all of these traces, so I started prototyping some examples.

Aim: make the computer recognise "clean" pages of books or "annotated" pages of books.

Each set (test and training) had 50 examples of "clean" pages and "annotated" pages, it makes sense to add more in the future. The results were not very accurate. Pages with hand-written text gave better results while highlighting and computer notes were often misinterpreted. It’s useful to try to see what the computer is looking for, understand if the script is breaking the image in parts, and try other scripts.