Our 3D Skin Cancer Virtual Reality (VR) Patient Immersive Viewing System

We are developing diagnostics & therapeutics for human basal cell carcinoma, squamous cell carcinoma, and melanoma. This skin cancer group comprises the most diagnosed cancers worldwide of all cancer types. This project involves the utilization of a Virtual Reality System for skin cancer screening, three dimensional immersive viewing of the skin, and detailed spatial and molecular analysis of the tumor.

The subject matter for our analysis is the skin tissue tumor viewed directly in real-time from body sites of the cancer patients.

The tissues will be viewed by confocal microscopy in three dimensions on a computer screen and projected into a virtual reality (VR) space.

Image software (e.g., ImageJ) is used to convert confocal microscope image stacks or slices into 3-D volumetric object files (NIfTI) containing the image and metadata.

3D NIfTI files can then be viewed in a fully virtual reality (VR) environment that reconstructs the 3D image of the skin tumor and its microenvironment or that of a control non-cancerous sample.

The view of the tissue can be manipulated to increase or decrease its magnification, adjust brightness & contrast, or to rotate the image to look at different parts of the tumor, using a portable headset, tactile controllers, and a workstation.

The patient or healthcare provider will be able to project cell images from skin tissue into a VR space, and see in 3D detailed images of the internal structure and the spatial distribution of cells & specific molecular markers within the tumor space.

Interactions between multiple cells in the tumor microenvironment could be monitored by the system – e.g., melanoma cell interactions with stroma, the blood supply, and immune cells. The three major skin cancers and the tumor microenvironment that surrounds them can be analyzed in detail: basal cell carcinoma, squamous cell carcinoma, and the melanoma.

Determinations can also be made of the penetration of drugs into the patient's skin and within the tumor space to test for the bioavailability of possible cancer therapeutics within the patients skin and tumor.

The system may be used for clinical viewing by researchers, physicians, other healthcare workers, and by the patients themselves. Recordings of the immersive viewing sessions can also be made to examine changes in the tumor microenvironment with time and/or after specific treatments. As a minimal viable product (MVP), the system can be configured as an add-on to a confocal microscope for Cancer Research. Many cancer centers have one or more of these microscopes already in place.

We are also pursuing the production of a system equipped with all of the necessary enhancements to gain FDA approval as a 510(k) or De Novo medical device. This system also includes software that analyzes the images and signal molecules detected in the patient's tissue using artificial intelligence (AI) and machine learning (ML) software to discriminate between patients at high or low risk. The proposed project is a very important project that has the potential to help millions of patients inflicted with skin cancer.