Title: Confusion matrices for STEM image classification
Legend: The figure shows confusion matrices averaged across a 5-fold validation of images acquired from textbooks. The image on the left shows the classification results across various image types, using a confidence threshold of 0.6 to accept a label. The image on the right shows classification results across chart types, used as a secondary classifier, and with a confidence threshold of 0.85 to accept a proposed label. Overall average accuracy across all image types is 92.2%, and across charts is 86.4%. Varying threshold values can be used to change the algorithm outputs to be more or less conservative.
Citation: N. Ghouse, R. Janakiraman & E. Tekin, manuscript under preparation
About the Lab: Loss of vision can frequently lead to a loss of independence and a reduction in quality of life for an individual. The Tekin lab is interested in harnessing new mobile technologies to provide access to environmental information for persons with vision loss. As mobile devices and wearable technologies proliferate, there are new avenues to interact with the constant mesh of devices and appliances in the environment, opening up new possibilities to improve the independence and self-sufficiency of persons with vision loss.
One of the lab’s main interest is using emerging technologies to improve communication aids for persons with vision and hearing loss, a fast growing segment of the population in developed countries as life expectancies increase. Whereas persons who have hearing loss but good vision can make use of facial cues to improve their speech reception, persons who have combined vision and hearing loss are unable to compensate for the loss of information in communication. The lab is exploring combining audio and video inputs in order to improve speech enhancement algorithms to aid speech reception for persons with such dual sensory loss.