1. Introduction

In the United States, the total cost for diagnostic medical imaging was estimated to be ＄100 billion in 2006 alone ^[1] and utilization rates of medical imaging are increasing^[2,3,4]. Similarly, the augmented reality (AR)/ virtual reality (VR) industry is large, and is increasing rapidly. The worldwide revenues from the AR and VR markets are projected to increase from ＄5 billion in 2016 to ＄162 billion in 2020 ^[5].

It is forseeable that these two fields of AR/VR and diagnostic medical imaging will merge in the near term since AR/VR provides true depth perception, fly through viewing, an improved human machine interface (HMI) and other augmented reality features that will likely enhance the diagnostic and therapeutic utility of diagnostic imaging ^[6,7,8,9,10]. As these two fields merge, one key component that deserves specific attention is the 3D cursor, since it will provide efficient navigation through structures.

While the concept of a 3D cursor has been discussed in the computer graphics industry for quite some time ^[11], it is infrequently discussed in the medical literature. Furthermore, there is no published review article on the use of a 3D cursor in the medical literature to date. The purpose of this paper is to provide the reader with a systematic review the literature on the use of a 3D cursor in medicine.

---

2. Materials and method

With the consultation of an experienced health sciences librarian, we conducted a systematic review in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines ^[12] in January 2018 with the search term "3D cursor" using PubMed, the Association for Computing Machinery (ACM) Digital Library and the Electrical and Electronics Engineers (IEEE) Digital Library. Two authors (D.D. and R.D.) reviewed the abstracts of all of the search results to determine relevance to medicine. Articles deemed relevant to medical imaging were downloaded and reviewed in their full text format. Finally, the uses of the term "3D cursor" in medicine were discussed and categorized.

---

3. Results

The PubMed search yielded a total of 6 articles ^{[8,9,13,14,15,16]}, all of which had relevance to the medical field. The ACM Digital Library search yielded a total of 16 references ^{[17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32]}, one ^[29] of which had relevance to the medical field. The IEEE Digital Library yielded a total of 19 references ^{[33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51]}, three ^[36,45,48] of which had relevance to the medical field. The citations were imported into EndNote 7 (Thomson Reuters, New York, NY). Thus, a total of 31 references did not have relevance to the medical field and a total of 10 referenced did have relevance to the medical field. These references fell into three categories: 1. hands or hand held controller; 2. non-volume-subtending digital 2D cursor capable of moving in all three axes. 3. volume-subtending digital 3D cursor capable of moving in all 3 axes. See Figure 1.

---

3.1. 3D cursor referring to "hands" or "hand-held controller"

In 1996, the term "3D cursor" was first used in the medical literature to represent a hand-held controller. Wong and colleagues discuss their method of using a hand-held controller as a 3D cursor to select particular volumes of interest ^[16]. Their setup included wireless glasses synchronized to rapidly alternating images for the left eye and right eye displayed on a cathode ray tube with the observer holding the controller in his hand. The 3D cursor is a tangible object used to select particular regions within the volume of interest. See Figure 2.

Fifteen years later, Jerald and Yoganandan developed an interactive medical visualization system, which uses an advanced hand-held controller system ^[29]. The uniqueness of their system is that the viewpoint of the images is controlled by two hand-held controllers with each controller manipulating a volume rendering image. These authors use the terminology "3D cursor" to denote the user's hands holding the hand-held controllers. This yielded a positive result with faster response time in the placement of the 3D object, which was 4.5–4.7 times as fast as a mouse interface and 1.3–1.7 times as fast as a one-handed wand.

Recently, in the 2017 article, "On the utility of 3D hand cursors to explore medical volume datasets with a touchless interface, " Lopes and colleagues discussed the limitations of conventional 2D input devices including mice and keyboards, which hamper 3D analysis ^[13]. They discussed the fact that several attempts with the conventional 2D input devices (i.e., mouse and keyboard) may be needed to achieve the desired orientation needed for 3D analysis. Furthermore, if the user is a surgeon, sterile precautions must be maintained adding an additional challenge in achieving the desired orientation. Lopes and colleagues therefore developed a touchless interface, which could interpret simple hand gestures to facilitate manipulation of 3D medical data wherein each human hand acts as a 3D cursor. In this study, the touchless interface improved spatial awareness and fluent interaction with the 3D volume more than with traditional 2D input devices. See Figure 3.

---

3.2. 3D cursor referring to digital cursor capable of moving in all three axes

In 1995, the term "3D cursor" was first used to denote a cursor that could move in 3D space in the article "Interactive delineation of brain sulci and their merging into functional PET images" by Michel et al. Michel and colleagues developed software tools to assist the neurologist and physiologist in analysis of cerebral images with structural magnetic resonance imaging (MRI) and functional positron emission tomography (PET) imaging ^[45]. Once the two datasets are coregistered, the authors built a 3D trace line for each sulcus of the brain, which helps one follow the brain surface curvature. The 3D cursor successfully provided the position in 3D space to link the surface viewer with the volume.

Five years later, Goodsitt and colleagues evaluated depth perception using a virtual 3D cursor in mammography ^[15]. In this study, three observers wore stereo glasses and estimated the depth of a fibril by adjusting the position of the cross-shaped virtual cursor on the computer system. All three observers were able to accurately estimate the depth of vertically oriented objects, but only one was able to accurately estimate the depth of the horizontally oriented object. The authors felt that this was related to the varying degree of observers' aptitude of stereoscopic visualization. A figure of the 3D cursor was not provided in this article.

It wasn't until the year 2013 until the term "3D cursor" was used in this similar fashion. In this study, Wang and colleagues explored a brain-computer interface (BCI) technology in a subject with tetraplegia with the surgically placed electrode inside of the skull at the sensorimotor cortex of the brain^[14]. Wang and colleagues assessed whether the subject using the BCI was able to move a 2D and a 3D cursor over a period of 30 days. Their 2D cursor is said to move in the x-y plane and their 3D cursor includes movement in the z-direction.

Most recently, in 2015, Eagleson and colleagues used the term "3D cursor" in a similar to the other three groups in their developing a prototype system for pre-operative planning ^[48]. Their NeuroTable system, a haptic device to facilitate control over a 3D cursor, which allowed users to target specific points within the pre-operative 3D scan.

---

3.3. Volume-subtending digital 3D cursor capable of moving in all 3 axes

In 2015, Moreira and colleagues generated a web-based image annotation tool for markup of radiology images in volumetric datasets using axial, frontal and sagittal planes ^[36]. The authors evaluated radiologists' preference of 3 options for marking a 3D region of interest including: pixel joining by similarity; border detection by imaging features; and, spheric shaped 3D cursor. Moreira et al. found that preferred option by the 6 radiologists for the best interface for marking 3D ROIs was the spheric 3D cursor. While not explicitly stated by the authors, this is noted to be a volume-subtending 3D cursor. See Figure 4.

In 2016–2017, Douglas and colleagues designed a 3D cursor in which the 3D cursor itself subtends a volume and is embedded into the volumetric medical imagery ^[8,9]. Similar to the non-volume subtending 3D cursor, this cursor can also be moved anywhere in 3D space. It is denoted in red, but other false colors are possible. This 3D cursor was generated as part of the depth-3-dimensional (D3D) AR/VR imaging system, which provides stereoscopic imaging with depth perception and head tracking. The primary purpose for the creation of this cursor was to improve visualization of certain 3D anatomical structures. See Figures 5–6.

---

4. Discussion

The term "3D cursor" has been used 10 times in the medical literature, but refers to three distinct topics: "hands" or "hand-held controller"; "non-volume subtending cursor moving in 3D space"; and, "volume-subtending 3D cursor that can move in 3D space". The role of 3D cursors may expand in years to come.

Surgeons must be able to reference medical images throughout an operation; thus, operating rooms are equipped with radiology picture archiving communication systems (PACS). However, since surgeons must maintain sterility when operating, they have two options to view patient images: scrub out of the surgery to manipulate and view images; or, have an assistant who is not scrubbed manipulate the PACS to show the desired image. Both of these options take valuable time in the operating room and a better solution is needed. The concept of the surgeon's hands representing the "3D cursor" as highlighted by Lopes et al. ^[13] has a important role in improving care, as this would allow the surgeon to control images while maintaining sterility. In addition to the use of "hands" as a 3D cursor, the use of real-world interface props, which can be manipulated by the user to specify spatial relationships, have specific advantages.

Surgical props can be equipped with orientation trackers; thus, humans may be able to more naturally interface with the virtual world. Such technology of interface props with orientation tracking has been discussed for some time including research in neurosurgery where a head viewing prop, a cutting-plane selection prop, and a trajectory selection prop were used ^[53]. Manufacturers could make "hand-held controllers" sterile as well, which would be advantageous to surgeons and providing an efficient method to view key images throughout an operation. The term "hand-held controllers" type of 3D cursor may also have a role in diagnostic radiology in image manipulation; however, there is an additional challenge that radiologists face.

Medical imaging datasets are extremely large due to the advances in spatial resolution of computed tomography (CT) and magnetic resonance imaging (MRI). As an example, an axial chest CT has a matrix of 512 × 512 pixels with 500+ sub-1mm-thick slices. Thus, there are over 130 million voxels generated in a chest CT. A 3D cursor capable of moving efficiently to any voxel within the dataset would yield improved efficiency in conventional "slice-by-slice" viewing of medical images, especially since a radiologist needs to pass through the volume several times in different window-level settings for a comprehensive review.

Looking into the future of radiology with AR/VR viewing, 3D cursors are expected to play a role. AR/VR D3D viewing offers depth perception, head tracking, joystick fly-through, focal point convergence, false coloring, transparency adjustment, which overall generates improved HMI ^[6]. Several imaging scenarios have been explored including: discerning certain patterns of microcalcifications (branching vs. cluster) ^[9]; breast cancer ^[8]; lung cancer ^[54]; and, brain aneurysm ^[52]. The 3D cursor plays an essential role in not only efficient maneuvering through the human body, but also in highlighting particular areas of concern. Many abnormalities in the human body are subtle, but important. Raising attention to and communication of these through use of a 3D cursor may prove life saving.

---

5. Conclusion

Three distinct concepts of a 3D cursor have been discussed in the medical literature. One concept is the use of an individual's hands or a hand-controller to input commands into the computer, which has the potential key advantage of allowing a surgeon to control an image while maintaining sterility. Another concept is a digital non-volume-subtending 3D cursor, which can move in any of the 3 dimensions, which has the advantage of providing efficient movement through large volumetric datasets. The third concept is a volume-subtending 3D cursor, which has the advantages of both being able to move within a large volumetric dataset efficient and highlight particular volumes of interest.

---

Conflict of interest

Author R.E.D. has a direct financial interest in D3D Technologies. The author D.B.D. is a family member of R.E.D. Authors C.W. and D.G. are employees of DXC Technologies. Authors J.B. and M.W. have no financial interest.

---