Smartphone microscopy and image analysis for diagnostics
Juliane Pfeil1, Marcus Frohme1, Katja Schulze2
1Division Molecular Biotechnology and Functional Genomics, Technical University of Applied Sciences, Wildau, Germany
2Oculyze GmbH, Wildau, Germany
In the last 10 years, there has been a significant improvement of the optical camera components in mobile devices, as well as an extensive improvement towards their computing power. Both improvements were basic prerequisites to use smartphones not only to make phone calls and browsing, but to also consider them as sensor devices for new applications.
Due to its specific feature of mobility, particularly interesting are insufficiently equipped or remote locations and applications that require immediate analysis. The idea to develop a microscope based on the camera module of a smartphone targeted primarily applications in the medical-diagnostics in order to carry out an on-site and immediate analysis. This so-called point-of-care-testing (POCT) gains not only in developing and emerging countries attention, but also in more rural regions in Germany with a deficient medical infrastructure due to a lack of physicians coming along with an increasing average age of the population (Kassenärztliche Bundesvereinigung 2018a, b).
The first smartphone microscope was presented in 2009 (Breslauer et al., 2009). This very bulky attachment, consisting of various conventional microscope modules, made it possible for the first time to examine cell or tissue samples directly on site. Various research groups subsequently worked on improving the technology in order to not only use the optics of the smartphone, but to also establish automated image analysis, which makes specialist knowledge obsolete. These latest developments allow to examine samples not only in a bright field, but also to establish fluorescence, dark field and phase contrast microscopy. Their application for the diagnosis of various diseases such as tuberculosis, parasite infections and fertility examinations were presented. So far, however, only few solutions are on the market with direct applications for POCT not being available yet. (Pfeil et al., 2018)
In image analysis, with the help of neural networks and Deep Learning (DL) large numbers of different objects in a single picture can be identified. Primarily, these algorithms are used for the detection of common objects, to train self-driving vehicles or to recognize faces. However, their establishment in medical settings, especially in microscopy, is on the advance. The particular difficulty in the recognition of microscopic objects lies not only in the classification of individual objects, but also in achieving a so-called instance segmentation of cells or particles in order to provide the obligatory accuracy for medical-diagnostic applications. (Felsberg, 2017)
Figure 1 exemplifies these differences for microscopic images of blood cells. The easiest approach is a simple classification, i. e. the assignment of a single object or condition to a class. Here one can review if the figure on the left shows a blood sample or something else. However, this statement is not very helpful in a diagnostic context. A localization and classification of all cells, as shown in the middle, opens up the possibility to evaluate each object individually. In order to obtain information about the morphology, the instance segmentation shown on the right is necessary, which is algorithmically the most complex to implement.