NLM Home PageVHP Home Page


 

Next: References Previous: Title Page Contents: Conference page

Virtual Body Structures

 

Introduction
Methods
Results
Conclusions
Acknowledgments
References

 

Introduction

1 VBS

 

The system uses a Pentium III 500 MHz machine with 256 MB of RAM and a video card with 64 MB of memory.  This allows the 3D models to be created in minutes.

 

VBS utilizes two different types of data, the Visible Human Male (VHM) images [1], and the VHM classified and segmented dataset, to create on-the-fly realistic 3D virtual body structures. The 1871 axial anatomical slices are the actual bitmap consisting of 1760 by 1024 true color tif file. The second type of data represents the segmentation and classification of the first database with a bit depth of 16, and a voxel size of 0.33 x 0.33 x 1.0 mm. The use of segmented data allows us to achieve full spatial resolution inherent in this data set. The system searches the data and creates high-resolution 3D images of user-selected structures. Besides creating and storing the models in various file formats a user is able to:

 

·        Touch the 3D model with a haptic device

·        Rotate and scale a 3D model

·        Apply a texture map to a 3D model

·        Animate the creation of the model

·        Render the 3D model as a 2D image with lighting and a

background

·        Export the 3D model to different formats such as VRML

·        Zoom in and examine a 3D model

·        Create pieces of a body structure

·        Modify a 3D model, delete and add vertices and polygons

·        Find the number of vertices and faces of the 3D Models

 

The system design takes advantage of the segmented and classified data of the Visible Human Project as well as the advanced manipulation and texture mapping features of the graphics package. 

     

1.1 VBS User Interface

 

The user can select body structures from a slice, anatomical system, or a region of the body, to create 3D models. “Load Slice”, “System”, or “Region” list boxes narrow the list of structures in the entire body.  The “Load Slice” list box allows a user to select structures within the loaded slice. The “System” list box allows the user to select the various types of the visible human’s internal systems, such as circulatory, digestive, and skeletal.  The “Region” list box allows the user to select different parts of the visible human, such as head, abdomen, pelvis, and extremities. Thus, for example, if a user selects Urinary System and Abdomen as Region, VBS allows  left/right kidney and renal pelvis, as well as ureter as choices of structures. However, if one selects Urinary System and Pelvis as Region, VBS allows trigone bladder, lumen and mucosa urethra, bladder, sphincter urethra, or ureter as choices of structures. “Start Slice” and “End Slice” list boxes option makes it possible to partially create the 3D structure(s), Figure 1.

 

For higher accuracy of creating structures the “XY Resolution” and the “Z Resolution” values can be lowered. Lowest resolution of 1 provides the most accurate models and is used for evaluating the anatomical structures. However, for a quick shape analysis a user can set higher XY and Z resolutions. The XY resolution is the number of rows and columns skipped on a slice and the Z resolution is the number of slices skipped when creating a 3D structure. The “Make Solid” box can be modified to enhance the structure visually for a more realistic effect. The created models can be solid or outlined. The “time message” box provide a notification of the time it took for the structure to be built. For example, when bladder, right kidney, and ureter were created at the same time with XY resolution of 1 and Z resolution of 3, the times taken to create the structures were 0.72, 10.33, and 1.72 minutes using 3D outline algorithm. User can select one of the three algorithms: “3D outline”, “3D Horiz EndPts”, or “3D Vert Endpts. Choice of an algorithm can make it possible to conduct important first stage shape analysis to determine the best way to merge the three algorithms into one.

Methods

1.2 VBS Algorithms

 

Once the 3D structure(s) to be created, and an algorithm are chosen, we extract the top and the bottom slices for the structure(s). Next the outline of the structure for the chosen slice is constructed. This process is repeated on all the slices that make up the body structure.  Once all the outlines are created VBS extrudes 3D non-rational uniform B-spline (NURBS) surface making the model solid.

 

Outline Generation

 

The 3D Outline algorithm is by far the most complex and most exact.  All the areas on the same slice are considered for the same body structure.   An outline of the right kidney is shown in Figure 2.  In order to draw a curve representing the outline of the kidney using the Outline algorithm, we begin looking through the slice from the top left corner until the first point of the desired body structure is found.  Once the top, left-most point (the green sphere here) is found, a vertex (X, Y, Z coordinates) is stored, and this point is known as a starting point.  Each row containing the structure has a starting point and ending point. Once the first vertex has been found, the next point we consider has the same X (column) value, while the Y (row) value is equal to the current row minus (down) a XY resolution number of rows in the data file. If the jump down to the next point is within the body structure, X values is reduced by multiples of the XY resolution until we are no longer in the body structure.  At this point the X value is increased by 1 until the point is within the body structure. This point is again stored as a starting point vertex for the current row.  However, if the jump down to the next point is outside the body structure, X values is increased by XY resolution until the body structure is found. If this point does not hit the body structure, then the algorithm knows that it’s finished looking for the starting points and then similarly begins looking for the ending points.  Once all the end points are found, we mark every index in the array between the ending point and starting point as used, so the structure that was just outlined is not used again when searching for other pieces of the structure on the same slice (see Figure 3).

Once we reach the same Y position as the starting vertex, we have finished this outline of the structure.  Now the search for other pieces of the structure begins at the very first vertex.  The rest of the slice is searched for remaining pieces. 

 

The Horizontal Endpoint algorithm works by simply scanning the slice on particular rows in the X direction looking for the body structure.  When the body structure is found, its starting point and ending point on a particular row are recorded. The outline of the body structure on this slice is created using all stored points once all lines are “scanned”.

 

The Vertical Endpoint algorithm works in the exact same way as the Horizontal Endpoint algorithm except in the vertical direction.  The algorithm searches a slice vertically until the left-most point is found.  Then it searches in this column until the end point is found.

 

After all outlines are created, VBS extrudes 3D non-rational uniform B-spline (NURBS) surface models. Now, structures can be easily displayed, modified, and manipulated.

Results

2 RESULTS

 

Creation and manipulation of over 100 different body structures has shown the robustness and stability of the system and the interfaces. Also, the ability to create the body structures using different algorithms has allowed us to conduct experiments to see how all the algorithms could be combined in the future. High resolution of our database allows for highly realistic 3D structures, see Figure5, Figure 6, Figure 7, Figure 8, Figure 9, and Figure 10. These structures can be manipulated, scaled and have texture maps placed on them, and output in several different formats. The standard manipulation capabilities include translucent visualization, interactive rotation, translation, and scaling.  We have also incorporated the ability to create more than one body structure at the same time (e.g. the separate lobes of the lungs along with the different pieces of the heart). This is a real advantage over currently existing systems that create one structure at a time and then superimpose them into the same scene. Optimization of the structures helps reduce the memory required for storage.

 

The table in Figure. 5 shows the time needed to create the right kidney with different algorithms. The size and shape of the kidney is very well known, and can be used to estimate the time required for other body structures. Obviously, larger and more complicated body structures require longer creation times.  The table is followed by several images and screen shots created using the VBS system.

 

Assign Haptics

 

Besides standard manipulation, the 3D virtual structures, created in real-time, now can even be "touched". We have developed a unique graphics-to-haptics PC-based tool, G2H [2], [3], which transforms any graphical virtual environment (created or imported) into a haptic virtual environment, without any additional programming. Created structures are ported to G2H, thus introducing tactile sensory capability. The models are “touched” via a haptic device and the contours of the structures are palpable. Furthermore, the user can dynamically change the “quality of the touch” depending on the tissue type. The parameters such as the stiffness, static and dynamic friction, damping, etc can be changed and these values can be stored for future use, see Figure 11.

 

Visualization

 

While viewing a 2D slice, area of interest can be viewed as a cropped portion with higher resolution with identification of anatomical structure being pointed. Furthermore, any structure on the slice can be segmented in real time and labled with required resolution by the user.  

 

For 3D model visualization, the user may use a 3D VRML viewer.  Incase a user wishes to use haptics, haptic anatomical modeler (HAM) developed in our lab can be downloaded. Basic visualization features of HAM are similar to the VRML viewer. HAM has two important special features. 1) HAM enables 3D graphical models haptic at a click of a mouse, thus making the models palpable, 2) HAM also allows stereoscopic visualization with either interlaced or anaglyph mode. During stereoscopic viewing the user is able to move the scene in and out of the screen, as well as pan the screen in all directions. Furthermore, while in a stereoscopic mode far and near focus can be adjusted and models can be rotated, translated and scaled. The concept of stereoscopic haptic virtual environment was first demonstrated in our earlier paper [4].

 

Texture Mapping

We are able to create realistic 3D volumetric texture mapping using the 2D image slices. We are also able to create an animation of textured bounded volumetric structure slice by slice. The volume can be manipulated.  This facilitates fundamental learning experience that leads one to "build 3D model in mind", providing a validation of computer assisted individual learning. In Figure 12 one can see the textured bounded volume for an eyeball.

3 Web-VBS

 

To make these structures readily accessible, we have designed a Web-based system (Web-VBS) that allows a user at a distant location to download 3D models.  The client server architecture is easy to use as can be seen by the user interface of Web-VBS [5], [6], Figure 13. The user inputs are used to query the server database.

 

A user is able to evaluate and download any axial slice, or slices containing a structure. In the later case, the user can provide a Z resolution, in which every "Zth" slice will be visualized. In general, if a 3D model in the format and resolution required exists, then the model is provided for visualization and manipulation, otherwise, the VBS-engine on the server side can be used to create the model, store it in our database, and send it to the user. See Figure14 and Figure 15 for basic concepts for the software architecture of our system.

 

For each slice and 3D model three different resolutions will be available. A user will be able to choose low, medium or high-resolution both for loading 2D slice or the 3D virtual body structure. For the 2D axial slices the low resolution images are approximately 100 Kbytes, medium resolution images are approximately 175 Kbytes, and high-resolution images are approximately to 375 Kbytes. The size of the slices includes all the resources needed (e.g. Java applets) at the client end to play with the slice. Users will be able to choose the image resolution appropriate for their computer and network resources. High-resolution, for example, will require a fast Internet connection and a computer with enough memory to handle large images while medium resolution will require a good Internet connection, and low resolution should work well for users with modem Internet access with limited computer resources. The user will have the option to save and store the newly created structures.

Conclusions

4 Problems and Open Issues

 

While our system creates realistic structures with rigid objects and soft tissues such as liver or kidneys, more complex structures, such as the intestines or the brain are difficult to produce. These extended soft-tissue structures are too complex on each slice to connect reliably in the fashion required by our algorithm.

 

Another open issue arises when outlines of a structure are mis-aligned from slice to slice. The class that connects all the outlines to make a solid structure then creates a twist in the 3D model, because it tries to align the vertices of different outlines in a smooth fashion. We have several ideas as to how to correct this problem.  They involve using one of the outlines as a mask and deleting the vertices that keep the object from being smooth.

 

Another important issue is that of developing methods for evaluating the accuracy of the slices, segmented data, as well as the 3D models. For example, alignment for some slices is off and this produces twists in 3D structures.  Once the evaluating process is in place, we expect to provide feedback for the data, and thus iteratively improve the accuracy of the data for the re-segmentation process.

           ; 

5 Conclusion

 

We have presented a system for creating realistic 3D body structures from the Visible Human data. Once these body structures have been created, they can be viewed in mono or stereoscopic mode, manipulated, and touched.

 

The structures created by our system are highly detailed, and can be used immediately for teaching anatomy.  We expect that with the use of haptic virtual reality they can be used in surgical training simulations.  We also plan to extend our system to patient specific data.

 

We hope that our web-based system can be used as a initial platform for developing foundation that would make it possible to develop a network based, multi-institutional, collaborative virtual environments for real-time teaching of human anatomy and surgery.

 

Acknowledgements:

Next: References Previous: Title Page Contents: Conference page

Office of High Performance Computing and Communications
Lister Hill National Center for Biomedical Communications
U.S. National Library of Medicine, 8600 Rockville Pike, Bethesda, MD 20894
National Institutes of Health
Department of Health & Human Services
Copyright and Privacy Policy
Last updated: 2 July 2001