Finger Sculpting with Digital Clay


Jarek Rossignac

College of Computing and GVU Center

Georgia Institute of Technology

Atlanta, Georgia, USA,


(In collaboration with Chris Shaw, with students Schwa Gargus, Ignacio Llamas, Byungmoon Kim and with faculty members Mark Allen, Wayne Book, Imme Ebertuphoff, Ari Glezer, John Goldthwaite, Beth Mynatt, David Rosen, and their students and postdocs  who participate in the Digital Clay ITR project sponsored by the NSF.)




The ambitious interdisciplinary Digital Clay project, funded by an NSF/ITR grant, brings together faculty members and students from Mechanical and Electrical Engineering and Computer Science. It is focused on building a computer controlled physical surface capable of changing its shape to reflect changes in a digital 3D model, of sensing the pressure of bare human hands, and of intelligently reacting to this pressure by opposing a force that conveys prescribed stiffness properties and by altering its shape and the associated 3D digital model to support a variety of intuitive shape editing operations. Thus our exploration of the Digital Clay should lead to a new generation of haptic devices for bare-hands touch-based input and output interaction with 3D shapes. We plan to explore applications in Computer Aided Design, Architecture, Art, Medical Training, and assistance to the visually impaired. For instance, Digital Clay will allow two remotely located users to simultaneously sculpt the same object. Each user will have a version of the Digital Clay that has the form of a shared 3D surface model. Finger pressures of one user will alter the local shape, which will alter the computer model, which will be transmitted to the other location, which will alter the other shape. The full scale realization of the Digital Clay will involve a very large number of miniature valves, sensors, hydraulic actuators, and articulated structural elements.


The Finger Sculpting effort, in part supported by a SEED GRANT from Georgia Tech’s GVU Center Jarek Rossignac, Chris Shaw and graduate students Schwa Gargus, Ignacio Llamas, and Byungmoon Kim, all from the College of Computing. It is focused on developing the interface model between the human finger and the surface. Starting from the nature of the contact: How will the surface know that you are push it to create a depression? How will it know that you want to pull it to create a protrusion? How much resistance should it offer? How does it know that you want to stop editing its shape and simply explore it. We have developed models for this contact behavior and have simulated them on a Phantom force-feedback haptic device,  so that we can validate and fine-tune the models before the actual Digital Clay hardware is operational. The next question is: How do you control the shape of the surface? We have explored an intuitive approach which mimics grabbing a different portion of the surface with each hand and then simultaneously dragging and rotating the associated coordinate systems with your hands to impose two sets of simultaneous constraints, each involving 6 degrees of freedom. We have developed a solver that deforms the surface to match all 12 constraints in realtime and offers the intuitive look-and-feel of an elastic material. To validate this solution before the Digital Clay is operational, we have developed the "Twister" environment, which uses two Polhemus trackers allowing the operator to use two hands to simultaneously stretch, twist, and bend the surface of a virtual model of a 3D object. Once a prototype of the Digital Clay becomes available, the operator  will touch the surface with bare hands directly. We plan to use these prototypes and early versions of Digital Clay for assessing the usability and tangible benefits of our two-hand finger sculpting paradigm and of the use of a digitally controlled real surface, rather than of a virtual model of it.



Brief Biography:

Jarek Rossignac is Full Professor in the College of Computing at Georgia Tech. His research is focused on the interactive design, compression, progressive transmission, and visual inspection of complex 3D models and of their animations. He is the co-inventor of the Topological Surgery 3D compression that forms the core of the MPEG-4 standard and the inventor of the Edgebreaker compression scheme. In addition to his contributions to the Digital Clay project presented here, Rossignac is the Principal Investigator of an NSF-CARGO project focused the Multi-scale Topological Analysis of Deforming Surfaces and a co-investigator of an NSF-CARGO project focused on the progressive transmission of compressed engineering and scientific data sets that represent  time-evolving three-dimensional scalar fields. In 1985, Rossignac received a PhD in E.E. from the University of Rochester, New York, for his work on the support of Minkowski and blending operations in CSG-based Solid Modellers. He also holds a Diplome d’Ingenieur from the French Engineering School E.N.S.E.M. and a Maitrise in Mechanical Engineering from the University of Nancy in France. Until 1996, he was Senior Manager and Visualization Strategist at IBM Research, where he managed a research department comprosing projects in 3D graphics, CAD, and VR. He was also responsible for the development and commercialization of IBM's Data Explorer and of the 3D Interaction Accelerator products. At Georgia Tech, from 1996 till 2000, he served as the Director of the GVU Center. He has authored over 80 technical papers and 17 patents, for which he has received 8 Best Paper Awards and 5 Invention Awards. He chaired 20 conferences, workshops, and technical program committees. He has served on the Editorial Boards of 8 journals and on 50 Technical Program Committees. He Guest-Edited 8 special issues of professional journals and gave 16 keynote lectures. He was elected Fellow of the Eurographics Association in 2000 and received the Senior Faculty Research Award from the College of Computing at Georgia Tech in 2002.