Quicktime Movie: Visible Human Explorer Display since April 1996 at
Ontario Science Centre's Human Body Exhibition
The Visible Human Project provides the world with thousands of images to help us understand ourselves. Flashbak Imaging has implemented an innovative graphics engine on standard personal computer systems that provides immediate, interactive access to these images. Coupling this with an easy to use interface, the company has provided an opportunity for the general public to explore the largest human anatomy data set ever created at sites such as the Ontario Science Centre.
One of the major forces in the development of a human being's comprehension of the world around him and his place in it, is the complex interaction of the eye and brain. That interaction provides the recognition of structure or the potential of connection between apparently separate entities before the information can be codified into the formal framework of language or numbers. Thus the ability to capture data in image form, to store it, to share it with others, to display it, and to enhance it, is a boon of infinite value to research.
The Visible Human Project provides the world with thousands of images to help us understand ourselves. This paper describes a system that provides immediate, interactive access to these thousands of images and some of the applications that may follow from that capability.
Since this system is based on the world's most popular computer platform, the IBM PC, the data can be available for information and education to wide segment of the population. We believe that data and information gain potential value in direct proportion to their accessibility.
Flashback Imaging's Visible Human Explorer is a digital image library for the thousands of Visible Human images. It is based on a combination of hardware and software that provides rapid display of full resolution digital imagery stored on disk or CD-ROM. Fast access to large data sets for study is still often limited by inefficiencies in image display imposed by current technology and implementation.
Typically, a relatively small subset of an image database is loaded into the Random Access Memory (RAM) of a computer workstation. A user can then scan through these images efficiently because of the fast access to RAM memory. Delays in this process occur when the amount of data overwhelms the available RAM and the user must pause to load new images into RAM. The image review and analysis process becomes tedious. A partial solution to this is to add more RAM but this is expensive and inflexible. We have developed a radically different technique that bypasses RAM.
The unique nature of this implementation is that the size of the database of images to be viewed is not limited by the amount of available RAM. It is only limited by the size of the hard disk upon which the data is stored. Image display and animation (or looping) takes place directly from disk. The computer monitor becomes a window of the hard drive with the ability to view files sequences of unlimited length. The size of individual images does not affect the animation speed. The animation is done with the full quantitative data set. Creation of MPEG, Quicktime or other special animation file formats (and the associated time-loss and data degradation) is not required.
Another very important feature is that the implementation is successful on conventional desktop PCs. Expensive workstations are not necessary. On a standard 486 or Pentium computer system with as little as 4 Megabytes of RAM, the software allows the user to freely scan along the axial, coronal, or sagittal images of the Visible Man or Visible Woman at full spatial resolution. Through mouse control, the user can loop, roam, and pan around in a sequence of images while animation continues. Furthermore a 3D volumetric perspective mode is supported where the user can interactively navigate the entire Visible Man or Visible Woman bodies.
The researcher has access to additional capabilities in the software including contrast enhancement, various noise-reduction and feature enhancement filters, red-green-blue channel combinations, measurements etc.
The Visible Human Data is processed in various ways for different applications and implementations of the Visible Human Explorer. All of the processing discussed has been performed on Intel Pentium-based personal computers with high capacity SCSI hard disk storage.
A sizeable proportion of each of the original digital photographic images of the Visible Woman axial sections contains the blue gelatin used to stabilize the body. The texture and colour of the gelatin varies from section to section and this becomes distracting to the viewer when the sections are scanned quickly. Thus the first step in the processing was to use a blue filter to remove the gelatin; this had little if any effect on the colour of the remaining portions of the imagery.
We then created sections with the same spatial resolution with other orientations by effectively electronically stacking the axial views and resampling them in other directions. First the orthogonal coronal and sagittal sections were created and stored to disk. Then we resampled the data into slightly larger parallelogram-shaped sections to support the 3D volume display.
The images are stored on disk in a spatially uncompressed format. This provides for random access to any portion of an image at the same speed as for the start of an image. The uncompressed format means that numerical analysis can be done on the image displayed, no quantitative information or time is lost in a compression/decompression stage. The fast random access to any point in an image means that a sequence of image can be displayed at the same rate whether each has an associated offset of zero or some other value. This allows, for example, for the rapid display of a sequence of images centred on the spinal cord and allows the eye and brain to concentrate on the structure of the spine itself.
Many of the images have been compressed in colour. Image sequences can be shown in 24, 16, or 8 bit colour. We found that the 16 bit version is virtually indistinguishable from the 24 bit version. With special care, we managed to convert the 24 bit to 8 bit colour with very little colour loss. The 16 bit version requires twice as much storage space as the 8 bit version and halves the rate of display in images/second. As indicated above, the images have been blue-filtered to remove the gelatin. We have found the 8 bit version to be adequate for the applications discussed. If necessary, the software can be configure to display the associated 16 or 24 bit version of an image each time the user stops at a single image while scanning through a sequence.
The core of the full resolution image animation engine is the software to move image data from a storage device to the computer monitor at the full rate of the interface bus. Initially, we relied on the fast SCSI bus and hard disks to get sufficient throughput. Tests on recent Enhanced IDE disks on the PCI bus show a similar throughput of 3 to 4 megabytes per second or 5 to 20 frames per second depending upon screen resolution. The IDE disks are significantly less expensive than SCSI disks of the same capacity. We do expect however, that we can achieve almost twice this bandwidth for even more demanding applications, by using the latest disks and Ultra Wide SCSI or Fibre Channel connections.
The digital CT (Computerized Tomography) and MRI (Magnetic Resonance Imagery) sequence images were processed as well. Both techniques produce grey scale images so no colour compression was done. The MRI output includes three simultaneous outputs (PD, T1 and T2). As an example, this can be combined in an animation sequence with 'PD' displayed in the red and green channels and 'T1' or 'T2' in the blue channel.
The CT data shows responses over only a restricted portion of the full range of the primary 12 bit imagery resolution. The position of the restricted range varies with different areas of the body. Again, conversion to lower level resolution data was done to preserve information and consistency over the whole body during image looping. Additional contrast control is always available under keyboard control. Coronal and sagittal sequences have not yet been created for these data sets.
Features of this approach that open applications to it include:
- rapid access to huge imagery data sets
- immediate access to full quantitative data
- flexible user interface
- economical hardware requirements
We will outline an application already in place and some of the possibilities under consideration for the future.
A primary application is in public displays of the data.
An implementation of the Visible Human Explorer has been on public exhibit in the Ontario Science Centre (Toronto, Ontario, Canada) since April 1996 (figure 1). This implementation is illustrated in the accompanying Quicktime video. The public, who indirectly fund much of the research have a right to be justifiably awed by the work of scientists and researchers and a need to have it presented to them in an intriguing manner. Thousands of visitors to the Science Centre, both adults and children, have been able to interactively scan through over 8000 high resolution images of human anatomy of the Visible Man.
The hardware of the system consists of a standard PC configuration with a large disk, specifically a Pentium 75 MHz, 16 MB DRAM, 2 MB graphics card and 17 inch monitor and the 4 Gbyte disk. The machine is enclosed in a cabinet and the only public interaction is via a two button trackball. The tyranny of the common graphical user interface of tiny click-boxes and scroll bars that are too small for young children to navigate has been eliminated. To fit data into the available disk, the data was subsampled to show every second image in each direction and minimum-loss compressed to 8 bit colour. For data read directly from disk and not filtered or enhanced on the fly, CPU speed has little if any effect on the display rate (Quicktime movie).
With Science Centre staff, we developed a user interface suitable for long term exposure to the lay public. As the user approaches, the monitor will be displaying a sequence of wandering images, in fact a small version of the Visible Man dataset to pique the curiosity as well as to serve as a screen saver. The Science Centre is very hands-on with a culture of "touch it and see what happens" so the system reacts to a motion of the trackball or a pushing a button to display the screen in figure 2.
The next user action will bring up the working display (figure 3). The screen is initially divided into four panels. Three of the panels contain default images from the axial, coronal, and sagittal image sequences. The lower right is the orientation panel. The orientation panel displays what appears to be 3 glass museum cases, each displaying a portion of the Visible Man dataset outlined in a frame and which corresponds to that shown in the other panels. This frame is colour matched to the frame around the corresponding quarter screen display panel. Any motion of the trackball will move the currently active frame in the museum case, bringing up a new portion of the Visible Man data there, but also in the associated display panel at higher resolution. Clicking the right trackball button will loop sequentially through the panels making them, in turn, the currently active one. A click on the left button will replace the screen with a full screen version of the corresponding Visible Man segment. For example, if the user clicks the left button when the top-left panel is active, he will see a full screen version of the sagittal view (figure 4). One line messages along the bottom of the screen are there to guide the user, but most learn without that help through the immediate positive feedback given through the screen display and orientation panel.
Flashback Imaging has also released the Visible Woman Abdomen Version using the same interface as the above Visible Man Full Body Version. By concentrating on a smaller section of the higher resolution Visible Woman body, a 4 GByte disk can hold almost 1400 axial slices, 900 coronal slices and 900 sagittal slices (figure 5, figure 6, figure 7).
In addition to 2D display, the user also can bring up the full screen 3D display where three (axial, coronal and sagittal) parallelogram-shaped sections are merged together to show a three-dimensional cross-section view of the Visible Man or Visible Woman (figure 8). Again, the user can use the same navigation method to freely move around the human body in 3D.
Other applications are being developed. One of the most obvious is to assist in the teaching of biological and medical subjects. Segments of the data set needed for a particular anatomy lecture can be downloaded to a hard disk as needed from a mass storage device. The lecturer can then interactively show the anatomy in detail with digital enhancement as desired. Segments could also be placed on a CD-ROM. The decreasing cost of CD-ROM writers means that small runs of highly specialized collections can be produced. Also, individual medical practitioners can then educate their patients on the anatomy of their individual problem. The core of the Visible Human Explorer is a software engine for displaying huge amounts of data. This is independent of the interface which can be tailored to the different applications.
Other possibilities flow from the goal of economical information distribution. A PC based system cannot reasonably produce the results of a super computer numerical analysis. However, it can be used to illustrate the imagery results of that analysis to scientific seminars, to funding agencies, or to the public. A laptop computer with large enough internal disk, CD-ROM or attached SCSI disk is the only hardware requirement.
One of the most intriguing possibilities under consideration, is that of the Visible Human Explorer as a visible, interactive window into a database of textural, numerical, and additional visible data. Normal human activities develop incredibly efficient hand-eye coordination and also incredibly quick pattern recognition. These qualities, linked with this browsing software, forms the basis of a powerful front end to various datasets. The software provides both fast random and fast sequential access to large datasets. A researcher can scan through the data with images flashing on the screen in front of his eyes. When the eye/brain recognizes a single image of interest out of the flashing sequence, the brain/hand can stop the sequence. If the sequence is not stopped in time it can be run backwards from that point, under control of mouse movement to the image of interest. Information linked with that image, could then be displayed. Here that might be the CT or MRI scan associated with the same section as the photographic image. An extension of that is that the cursor position on that particular image may provide a link to a great deal of information about that voxel location - the muscle type, CT density etc.
Once again, access is direct and interactive, there is no need to previously plan and choose images to load and wait during a lengthy image loading stage before the pixel location can be found. In a reverse linking, the textural database could be searched for the annotation "spinal cord", and the system would automatically prepare a list of images containing the spinal cord. This list would then in turn drive the animation of only those images. Note that animated images do not need to be stored sequentially on the disk to maintain display speed. By linking the database to the images though an look-up-table, the database can be easily expanded, contracted or replaced completely. All that needs to be changed is the table and storage disks as appropriate.
We have described an innovative software engine for access and display of imagery data sets. Its features include:
- the fact that it runs on affordable off-the-shelf hardware, multiplying the possibility of access and distribution for the results of the Visible Human project
- flexible, interactive interfaces with the possibility of additional interfaces for other applications
- the ability to handle data sets of any size, limited only by disk capacity, not computer RAM
- the ability to digitally enhance imagery during animations
The possibilities for this central software engine are limited only by imagination and experience.
The work described here has been undertaken through internal company Research and Development funds. We wish to thank the Toronto Hospital for their collaboration and allowing us access to their licensed data. We thank the National Institute of Health for its insightful support of the Visible Human Project and for granting a data use license to the Toronto Hospital. The encouraging words of Dr. Michael J. Ackerman of the National Library of Medicine and Dr. Victor M. Spitzer of the University of Colorado Health Sciences Center were also of great benefit.
1. National Library of Medicine (U.S.) Board of Regents. Electronic Imaging: Report of the Board of Regents. U.S. Department of Health and Human Services, Public Health Service, National Institutes of Health, 1990. NIH Publication No. 90-2197.
Quicktime Movie: Visible Human Explorer Display since April 1996 at
Ontario Science Centre's Human Body Exhibition
Figure 1: Visible Human Explorer
Display since April 1996
at Ontario Science Centre's
Human Body Exhibition
Figure 2: Introduction Screen
of Visible Human Explorer at
Ontario Science Centre's
Human Body Exhibition
(Visible Man)
Figure 3: Main Screen
of Visible Human Explorer
(Visible Man)
Figure 4: Sagittal View of Visible Man
Figure 5: Axial View of Visible Woman
Figure 6: Coronal View
of Visible Woman Abdomen
Figure 7: Sagittal View of
Visible Woman Abdomen
Figure 8: Three-dimensional Cross-section
View of Visible Woman Abdomen
Further information can be obtained via WWW: http://www.interlog.com/~flashbak or by contacting one of the following persons:
Hao Le, Flashback Imaging, Ontario, Canada (flashbak@interlog.com)
Brian Wannamaker, Sea Scan, Ontario, Canada (seascan@inforamp.com)
David J. Mikulis, Toronto Hospital, Toronto, Canada (mikulis@camtwh.eric.on.ca)
