{"id":3617,"date":"2018-12-13T10:58:05","date_gmt":"2018-12-13T09:58:05","guid":{"rendered":"https:\/\/blog.zhdk.ch\/masterkvis\/?p=3617"},"modified":"2019-08-07T15:47:36","modified_gmt":"2019-08-07T13:47:36","slug":"visualizing-the-inner-ear-for-robotic-surgery","status":"publish","type":"post","link":"https:\/\/blog.zhdk.ch\/masterkvis\/visualizing-the-inner-ear-for-robotic-surgery\/","title":{"rendered":"Journey to the Inner Ear"},"content":{"rendered":"\n<h4 class=\"wp-block-heading\"><strong>Visual Design of Robot-Assisted Surgery for the Hearing Impaired<\/strong><\/h4>\n\n\n\n<h4 class=\"wp-block-heading\">Catherine Tsai, 2019<\/h4>\n\n\n\n<figure class=\"wp-block-image alignwide\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"706\" src=\"https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/ZHdK-postcard-image-V1-1024x706.jpg\" alt=\"\" class=\"wp-image-4160\" srcset=\"https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/ZHdK-postcard-image-V1-1024x706.jpg 1024w, https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/ZHdK-postcard-image-V1-300x207.jpg 300w, https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/ZHdK-postcard-image-V1-768x529.jpg 768w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption>A tablet for surgical planning: <em>Otoplan<\/em> is a software on a tablet device that allows surgeons to generate 3D ear models from patient CT scans and plan the trajectory of the robot drill. This master\u2018s project optimizes the visual interface for the surgeon user.<\/figcaption><\/figure>\n\n\n\n<p><strong>How can designing audience-specific visualizations improve the user experience of a novel robot-assisted surgical platform? What visual aspects are important to improve accuracy, orientation, and safety for the surgeon user? What visual aspects are important to reduce fear and improve trust for patients about to undergo the surgery?<\/strong><\/p>\n\n\n\n<div class=\"wp-block-media-text alignwide\"><figure class=\"wp-block-media-text__media\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"925\" src=\"https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Cochlear-implant-diagram-1024x925.png\" alt=\"\" class=\"wp-image-4170\" srcset=\"https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Cochlear-implant-diagram-1024x925.png 1024w, https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Cochlear-implant-diagram-300x271.png 300w, https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Cochlear-implant-diagram-768x694.png 768w, https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Cochlear-implant-diagram.png 1641w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure><div class=\"wp-block-media-text__content\">\n<h5 class=\"wp-block-heading\">What is a cochlear implant?<\/h5>\n\n\n\n<p>A sound processor (1) captures sound and converts it into a digital code. The code is sent through a coil (2) to the implant (3). The implant sends an electrical signal through an electrode that must travel through the side of the skull to reach the cochlea, our hearing organ. The hearing nerve (4) transmits the signal as impulses to the brain, which interprets these as&nbsp; sound.&nbsp;<\/p>\n\n\n\n<p class=\"has-small-font-size\">Picture from Cochlear\u00ae.<br><\/p>\n<\/div><\/div>\n\n\n\n<div class=\"wp-block-media-text alignwide has-media-on-the-right\"><figure class=\"wp-block-media-text__media\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"576\" src=\"https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Robotic-cochlear-implantation-2-1024x576.jpg\" alt=\"\" class=\"wp-image-4176\" srcset=\"https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Robotic-cochlear-implantation-2-1024x576.jpg 1024w, https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Robotic-cochlear-implantation-2-300x169.jpg 300w, https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Robotic-cochlear-implantation-2-768x432.jpg 768w, https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Robotic-cochlear-implantation-2-1110x624.jpg 1110w, https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Robotic-cochlear-implantation-2-528x297.jpg 528w, https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Robotic-cochlear-implantation-2.jpg 1280w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure><div class=\"wp-block-media-text__content\">\n<h5 class=\"wp-block-heading\">A robot assists a surgeon&nbsp;<\/h5>\n\n\n\n<p>A robot with an optical tracker and drill assists a head and neck surgeon with difficult parts of cochlear implantation surgery.<\/p>\n\n\n\n<p class=\"has-small-font-size\">Picture from ARTORG Center.<br><\/p>\n<\/div><\/div>\n\n\n\n<h4 class=\"wp-block-heading\">Background<\/h4>\n\n\n\n<p>Five percent of the world is deaf or hard of hearing. The cochlea is the organ that allows us to hear, and thousands of cochlear implantations are performed each year to restore the sense of hearing. Because ear anatomy is difficult and the surgery is technically challenging, 30\u201350% of cases are not successful.&nbsp; With modern technology, surgeons like myself now have a solution \u2013 surgical robots. Robots can overcome human limitation, making the surgery more effective, less invasive, and safer for the patient.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Aims<\/h4>\n\n\n\n<p>Programming the surgical robot requires creating 3D models from patient-specific CT scans. The aim of this project was to explore the visual needs of the various audiences of this novel surgical system.<\/p>\n\n\n\n<h5 class=\"wp-block-heading\" style=\"text-align:center\">From Pixels to 3D Models<\/h5>\n\n\n\n<ul class=\"wp-block-gallery alignfull columns-3 is-cropped wp-block-gallery-1 is-layout-flex wp-block-gallery-is-layout-flex\"><li class=\"blocks-gallery-item\"><figure><a href=\"https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Amira-1024x569.png\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"569\" src=\"https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Amira-1024x569.png\" alt=\"\" data-id=\"4168\" data-link=\"https:\/\/blog.zhdk.ch\/masterkvis\/visualizing-the-inner-ear-for-robotic-surgery\/amira\/\" class=\"wp-image-4168\" srcset=\"https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Amira-1024x569.png 1024w, https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Amira-300x167.png 300w, https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Amira-768x427.png 768w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/a><figcaption>Patient-specific 3D ear and skull models are generated from the pixels of a computed tomography (CT) scan using Amira software.<\/figcaption><\/figure><\/li><li class=\"blocks-gallery-item\"><figure><a href=\"https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Sketchfab-1024x764.png\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"764\" src=\"https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Sketchfab-1024x764.png\" alt=\"\" data-id=\"4178\" data-link=\"https:\/\/blog.zhdk.ch\/masterkvis\/visualizing-the-inner-ear-for-robotic-surgery\/sketchfab\/\" class=\"wp-image-4178\" srcset=\"https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Sketchfab-1024x764.png 1024w, https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Sketchfab-300x224.png 300w, https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Sketchfab-768x573.png 768w, https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Sketchfab.png 1891w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/a><figcaption>Using Cinema 4D, various visual themes are created. Models are shared using Sketchfab to test the ability of the visualizations to improve accuracy of identifying structures and ability to orient structures.<\/figcaption><\/figure><\/li><li class=\"blocks-gallery-item\"><figure><a href=\"https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Engineering-Info-Board-1024x766.png\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"766\" src=\"https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Engineering-Info-Board-1024x766.png\" alt=\"\" data-id=\"4172\" data-link=\"https:\/\/blog.zhdk.ch\/masterkvis\/visualizing-the-inner-ear-for-robotic-surgery\/engineering-info-board\/\" class=\"wp-image-4172\" srcset=\"https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Engineering-Info-Board-1024x766.png 1024w, https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Engineering-Info-Board-300x224.png 300w, https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Engineering-Info-Board-768x575.png 768w, https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Engineering-Info-Board.png 1239w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/a><figcaption>After the final designs are created, engineers at ARTORG Center program them into <em>Otoplan<\/em>.<\/figcaption><\/figure><\/li><\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Methods<\/h4>\n\n\n\n<p>After using Amira to generate 3D models from CT scans, I then used Cinema 4D to experiment with various visual styles. In collaboration with engineers and surgeons, I conducted focus groups, in-person interviews, and surveys to test the surgeon user experience of lifelike compared to the baseline unnaturally-colored visualizations. Quantitative studies were performed test the effect of lifelike visualizations on ability to orient and identify anatomical structures. Interviews and surveys were used to explore how best to use visualizations to guide the patient experience of undergoing robot-assisted surgery.&nbsp;<\/p>\n\n\n\n<div class=\"wp-block-media-text alignfull\"><figure class=\"wp-block-media-text__media\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"622\" src=\"https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Infoboard-tsai-ear-visual-styles-1024x622.png\" alt=\"\" class=\"wp-image-4174\" srcset=\"https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Infoboard-tsai-ear-visual-styles-1024x622.png 1024w, https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Infoboard-tsai-ear-visual-styles-300x182.png 300w, https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Infoboard-tsai-ear-visual-styles-768x467.png 768w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure><div class=\"wp-block-media-text__content\">\n<h5 class=\"wp-block-heading\">Testing various visual styles<\/h5>\n\n\n\n<p>After a web-based search on various visual styles commonly used to portray 3D-generated anatomy from CT scans, I created 5 visual styles of my ear models to test with surgeons: unnaturally-colored, life-like, cadaveric, and stylized. The visual styes were created in Cinema 4D and uploaded to Sketchfab for sharing. Visual preferences were qualitatively tested in focus groups and interviews.<br><\/p>\n<\/div><\/div>\n\n\n\n<h4 class=\"wp-block-heading\">Results and Conclusions<\/h4>\n\n\n\n<p>Lifelike visualizations improve the user experience for surgeons regarding accuracy, orientation, and satisfaction, while graphical visualizations can enhance patient understanding and trust in a robot-assisted surgical system.<\/p>\n\n\n\n<ul class=\"wp-block-gallery alignfull columns-2 is-cropped wp-block-gallery-2 is-layout-flex wp-block-gallery-is-layout-flex\"><li class=\"blocks-gallery-item\"><figure><a href=\"https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Tsai-exhibition-image-ear-2-1024x819.jpg\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"819\" src=\"https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Tsai-exhibition-image-ear-2-1024x819.jpg\" alt=\"\" data-id=\"4192\" data-link=\"https:\/\/blog.zhdk.ch\/masterkvis\/visualizing-the-inner-ear-for-robotic-surgery\/tsai-exhibition-image-ear-2\/\" class=\"wp-image-4192\" srcset=\"https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Tsai-exhibition-image-ear-2-1024x819.jpg 1024w, https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Tsai-exhibition-image-ear-2-300x240.jpg 300w, https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Tsai-exhibition-image-ear-2-768x615.jpg 768w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/a><figcaption>Life-like 3D reconstruction of the cochlea, malleus, incus, stapes, and facial nerves from a patient\u2018s CT scan. Transparency allows us to see the cochlear implant electrode threading into and coiling within the cochlea.<\/figcaption><\/figure><\/li><li class=\"blocks-gallery-item\"><figure><a href=\"https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Tsai-exibition-skull-1024x819.jpg\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"819\" src=\"https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Tsai-exibition-skull-1024x819.jpg\" alt=\"\" data-id=\"4194\" data-link=\"https:\/\/blog.zhdk.ch\/masterkvis\/visualizing-the-inner-ear-for-robotic-surgery\/tsai-exibition-skull\/\" class=\"wp-image-4194\" srcset=\"https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Tsai-exibition-skull-1024x819.jpg 1024w, https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Tsai-exibition-skull-300x240.jpg 300w, https:\/\/blog.zhdk.ch\/masterkvis\/files\/2019\/08\/Tsai-exibition-skull-768x614.jpg 768w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/a><figcaption>Life-like 3D reconstruction of a patient\u2018s skull with cochlear implant and blue fiducials (screws). Fiducials allow the robot to capture the movement of the patient\u2018s head through optical tracking.<\/figcaption><\/figure><\/li><\/ul>\n\n\n\n<p><strong>Master project 2018\u20132019<\/strong><\/p>\n\n\n\n<p><strong>Mentoring<br><\/strong>Fabienne Boldt<br>Simon Schachtli<\/p>\n\n\n\n<p><strong>Cooperation partners<br><\/strong>ARTORG Center for Biomedical Engineering at the University of Bern<br>Inselspital, University Hospital of Bern<br>Mentor: Stefan Weber<\/p>\n\n\n\n<p><strong>Contact<\/strong><br>Email Catherine Tsai for more information: catherine.tsai@zhdk.ch<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Visual Design of Robot-Assisted Surgery for the Hearing Impaired Catherine Tsai, 2019 How can designing audience-specific visualizations improve the user experience of a novel robot-assisted surgical platform? What visual aspects are important to improve accuracy, orientation, and safety for the surgeon user? What visual aspects are important to reduce fear and improve trust for patients [&hellip;]<\/p>\n","protected":false},"author":2389,"featured_media":4160,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[],"class_list":{"0":"post-3617","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-projekte","8":"czr-hentry"},"_links":{"self":[{"href":"https:\/\/blog.zhdk.ch\/masterkvis\/wp-json\/wp\/v2\/posts\/3617","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.zhdk.ch\/masterkvis\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.zhdk.ch\/masterkvis\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.zhdk.ch\/masterkvis\/wp-json\/wp\/v2\/users\/2389"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.zhdk.ch\/masterkvis\/wp-json\/wp\/v2\/comments?post=3617"}],"version-history":[{"count":19,"href":"https:\/\/blog.zhdk.ch\/masterkvis\/wp-json\/wp\/v2\/posts\/3617\/revisions"}],"predecessor-version":[{"id":4284,"href":"https:\/\/blog.zhdk.ch\/masterkvis\/wp-json\/wp\/v2\/posts\/3617\/revisions\/4284"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.zhdk.ch\/masterkvis\/wp-json\/wp\/v2\/media\/4160"}],"wp:attachment":[{"href":"https:\/\/blog.zhdk.ch\/masterkvis\/wp-json\/wp\/v2\/media?parent=3617"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.zhdk.ch\/masterkvis\/wp-json\/wp\/v2\/categories?post=3617"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.zhdk.ch\/masterkvis\/wp-json\/wp\/v2\/tags?post=3617"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}