Sign to Speech Project - Gesture Recognition

The goal of the Sign to Speech project is to create a system that identifies human gestures in this case the single hand sign language alphabet and phrase “I Love You” utilizing a glove with bend gauge sensors and an accelerometer to capture the electronic characteristics patterns of each letter and the phrase. Both biologist and sociologist are working together to define “gestures” and the “encoded patterns of gestures. An overview of gesture recognition can be found at the following URL http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/COHEN/gesture_overview.html.

A number of control memory and display devices have been designed based upon the following block diagram.

Figure 1: Block Diagram of Architecture for Gestural Control of Memory and Display.

Further research examined servo and robotic systems to conceptualize local control of mechanisms based upon the following block diagram.

Figure 2: Block Diagram of Architecture for Local Control of Actuated Mechanisms.

The issues that arise is the delination between gestures i.e., noting the start and ending points of each gesture. The translation to speech is somewhat straight forward by comparing the captured pattern to the letter or phrase and then performing the display of the text on a console monitor or through Apple’s PlainTalk or Say speech synthesis applications.

References

Apple Computer, Inc. (2008). Apple Speech Recognition. http://www.apple.com/macosx/features/300.html#universalaccess

Thomas Baudel and Michel Beaudouin-Lafon. CHARADE: Remote control of objects using free-hand gestures. Communications of the ACM, 36(7):28-35, July 1993.

Charles J. Cohen, Lynn Conway, and Dan Koditschek. "Dynamical System Representation, Generation, and Recognition of Basic Oscillatory Motion Gestures," 2nd International Conference on Automatic Face- and Gesture-Recognition, Killington, Vermont, October 1996.

Jill Crisman and Charles E. Thorpe. SCARF: A color vision system that tracks roads and intersections. Robotics and Automation, 9(1):49-58, February 1993.

Trevor J. Darrell and Alex P. Penland, Space-time gestures. In IEEE Conference on Vision and Pattern Recognition, NY, NY, June 1993.

E. D. Dickmanns and V. Graefe. Dynamic Monocular machine vision. Machine Vision and Applications, pages 223-240, 1988.

David Kortenkamp, Eric Huber, and R. Peter Bonasso. Recognizing and interpreting gestures on a mobile robot. Proceedings of the Thirteenth National Conference on Artificial Intelligence (AAAI ’96), 1996.

Pattie Maes, Trevor Darrell, Bruce Blumberg, and Alex Pentland. The Alive System: Full-body interaction with autonomous agents. In Computer Animation ’95 Conference, IEEE Press, Geneva, Switzerland, April 1995.

K. V. Mardia, N. M. Ghali, T. J. Hainsworth, M. Howes, and N. Sheehy. Techniques for online gesture recognition on workstations. Image and Vision Computing, 11(5):283-294, June 1993.

Kouichi Murakami and Hitomi Taguchi. Gesture recognition using recurrent neural networks. Journal of the ACM, 1(1):237-242, January 1991.

Alfred A. Rizzi, Louis L. Whitcomb, and D. E. Koditschek. Distributed Real-Time Control of a Spatial Robot Juggler. IEEE Computer, 25(5), May 1992.

Dean Rubine. Specifying gestures by example. Computer Graphics, 25(4):329-337, July 1991.

Thad Starner and Alex Pentland. Visual recognition of American Sigh Language using Hidden Markov Models. IEEE International Symposium on Computer Vision, November 1995.

Jerry Seinfeld/Bill Gates Microsoft Future Ad

Here is the link to Microsoft's Ads with Jerry Seinfeld and Bill Gates.

Shoe Circus Ad
http://www.therawfeed.com/2008/09/microsofts-seinfeld-ad-hits-here-it-is.html

New Family Ad
http://www.youtube.com/watch?v=gBWPf1BWtkw

I'm a PC Ad
http://www.youtube.com/watch?v=kkZdkHylJ3w

I think the symbolism is fitting and quite revealing on how future prediction are perceived by the common person.

The Microsoft Ad campaign moves forward article by Michael Todd on September 21, 2008 in eFluxMedia located at URL:

http://www.efluxmedia.com/news_Microsofts_Ad_Campaign_Moves_Forward_24783.html.

The article notes that Microsoft is indeed concerned about Apple's rise in market share. Also, the "I'm a PC" Ad produces many misleading images including the Space Shuttle environment.

Analysis of Web 2.0 Tools - Google Docs

Google Docs is a collaborative suite of word processing, spreadsheet and presentation applications that are shared in real-time. Each of the application uses the standardized menu and toolbars to minimize the learning curve for the users. Google Docs provides some standard templates such as Resumes, Cover Letters, Expenses, Loan Calculators, Flipbooks and Presentation Design. Of course the user can create a document, spreadsheet or presentation from scratch.

The sharing feature allows the initial creator to invite others as collaborators or viewers. The originator can organize the documents into subfolders. The documents are safe from local hard drive and power outages. Also, the documents can be access from any Internet connection and exported to standard file formats of DOC, XLS, CSV, ODS, ODT, PDF, RTF and HTML.

Viewing access is controlled by who is allowed to collaborate and view selected documents. Additionally, the documents can be published as a web page, posted to a blog, or shared only within the established group(s). This does not account for internal breaches within the Google organization.

The navigation between documents is as simple as selecting another tab in the browser in my case Firefox 3.0.1. The following images below display each of the suite applications.

Figure 1: Google Docs startup page.

Figure 2: Document application.

Figure 3: Spreadsheet application.

Figure 4: Presentation application.

I encouraged all of the capstone teams to employ Google Docs and Groups collaboration applications as well as Google’s gmail to keep all of the projects data, information and knowledge in central repository for ease of sharing and access. My only concern is the security of files on a third party server(s) which are subject to attacks 24x7x365 worldwide. The assessment of the risks involved to the benefits received must be weighted on project-by-project basis. I do not envision corporations or the Department of Defense (DOD) sharing their trade secrets via this security limited medium.

References

http://www.google.com/google-d-s/tour1.html
http://www.google.com/google-d-s/tour2.html
http://www.google.com/google-d-s/tour3.html
http://www.google.com/google-d-s/tour4.html
http://www.google.com/google-d-s/tour5.html
http://www.google.com/google-d-s/tour6.html

Prediction Signing to Speech - Gesture Recognition

The Americans with Disabilities Act (ADA) enforces the laws, regulations and guidelines to ensure the translation of speech to text through close caption or translated to sign language. My future prediction is that in the next 10 to 15 years technology will advance to provide sign language or gesture recognition to speech universal translator so that one does need to learn how to sign to have a conversation hearing impaired person. This technology could further the abilities of the brain injured individuals to communicate.

Some organizations are actively working on gesture recognition such as http://trace.wisc.edu/docs/modality_translation_poster2001/index.htm that people with permanent functional limitations such as hearing, visual and cognitive impairments can effectively communicate with the world around them. A plethora of other research sites are available at http://www.visionbib.com/bibliography/people932.html relating to sign language and ASL recognition.

Analysis Web 2.0 Tools - Podcast Audio

My analysis of creating and uploading an initial podcast as a neophyte was a time consuming event. I began by attempting to create an iMovie video/audio podcast experience only to discover that most host sites preferred mpeg file formats over the mov file format of quick time. This could be based upon my Macintosh which is ancient and prehistoric in computer terms with a 1.8 GHz Power PC G5 running Mac OS X version 10.3.9 and 512MB of RAM. It was originally acquired in 2004 in its heyday by an acquaintance of mine.

The iMovie route video/audio recording experience was a relatively painless process. The pain occurred when I attempted to upload the file to last.fm, splashcast.net, and other hosting services. The next attempt I downloaded and install Audacity and LameLib, which again was a relatively painless process to record an audio track. The access to splashcast.net did not work well with the Mac computer so I ported the Mac files to a PC running Windows OS and the upload to splashcast.net without any issues. This was successful but did not achieve my desired outcome. Finally, I uploaded a single image to splashcast.net and recorded a background audio to the image. This method proved to achieve the desired result. The embedded html code from splashcast.net was copied to my blog site to complete the task event.

I believe the linkage of podcast to blog site will be easier in the near future based upon Dr. Rheingold’s social networking work by directly editing and recording video and audio podcast on the blog site itself.

Second Life - Doctorate Residency in Virtual Reality

After our discussion on Second Life (SL) this evening (Aug 19, 2008 6:00pm MDT) regarding Web 2.0 Tools such as 1 minute podcasts to distribute a thought provoking idea or question via a blog RSS feed, I thought that SL would be a great venue to provide a virtual Doctorate residency. Now at the end of my Doctorate coursework where our cohort traveled three times year to Colorado Springs and now the cohorts travel twice a year. First, I believe the virtual reality world of SL would augment the residencies by providing access and participation through the interface. Second, access to the residencies through SL would accommodate the students who are experiencing life, work and financial issues when scheduled to attend.

I acknowledge that the logistics, quality equipment and technical specifications must meet the SL minimum requirements for the end user but must exceed the SL requirements at the delivery point. Furthermore, each session would need to be recorded and archived for replay and review. I would suggest a pilot residency as early as January 2009 for the DCS/EIS cohorts to entail one session, possibly a Thursday session for those cohorts not required to be onsite. Additionally, the SL venue could provide another future forum method for presenting a Doctorate Candidates’ Dissertation Defense.

What do you think?

Analysis of Web 2.0 - Flickr

This is a link to a simple photos uploaded to Flickr: http://flickr.com/photos/29403741@N04

My analysis of the http://flickr.com site is divided into positive and negative experiences with the site.

Positive Experiences:

Home Page is simple with navigation at the top left utilizing a drop down menu box for the five options. On the right side at the top is the Sign in, Help, Sign out and Search options. The section below greets the user sign in name, which is immediately followed by in red lettering a security notice warning. The warning notifies the user of this Web tool of the phishing technique being employed to acquire user’s credentials.

Then on the left side of the web page body it displays the sites news, advertises for upgrades to its Pro version of the site for a monthly fee, other advertisement and offers tips on how to upload photos from your cell phone. These are all good marketing campaigns. To the direct right of these displays is the option to upload photos, review your photo stream, review everyone’s public uploads, and review your groups.

The bottom portion of the site holds another set of navigation options somewhat different from the navigation at the top of the page. As part of this navigation is the Flickr Blog, About, Terms of Service, Privacy disclaimer, Copyright policy, Report Abuse options.

Negative Experiences:

The duplication of navigation options is not available on all sub-pages for instance under the Organize sub page offer no quick way to exit to the Home page or other pages except for the manipulation of the URL or Back button. Additionally, the duplicate navigation only helps balance the page when minimal information is displayed on a sub-page. The upload of images speed is depended upon the type of Internet connection. I do not recommend dial-up when uploading images due to bandwidth and potential time out anomalies.

Overall the Flickr.com Web 2.0 tool is an adequate for uploading, editing and sharing photos and videos. Being security minded, I prefer not to place my personal information, photos, videos, and/or any other data on an organization’s server that I have no knowledge about, this includes the physical country, address and phone number of the organization.

Picasa by Google is a better intuitive product for Mac, Linux and Windows.

http://picasa.google.com/web/mac_tools.html

http://picasa.google.com/linux/

http://picasa.google.com/

Review of Dr. Rheingold's Social Media Video

Dr. Rheingold's Social Media Video located at the following URL
http://vlog.rheingold.com/index.php/site/video/social-media-classroom-why-use-forums/
is a further extension of original text based platforms of UUNET, BBS, and NetNews which were the forefront of the Blog that added images and static videos to the collaboration forums. Now the Social Media Video will insert the responder’s video as a thread to the original video. The dilemma I foresee with this medium is the lack of being anonymous and unidentified with a video response but the platform still offers the text based response as well. I do perceive that this video medium will be employed by business to capture important visual responses to ensure legal liability.

Analysis of Web 2.0 - Second Life

Second Life is a three dimension (3D) virtual reality world created and developed by its residents since 2003 when it made its public debut. The URL http://secondlife.com contains a plethora information pertain to this virtual world. The structures and items in Second Life are created from basic shapes such as cylinders, cones, triangles, circle, squares, etc. and changed through a simple edit menu to manipulate the basic object into buildings, lamps, chairs, clothing, and flowing hair.

Currently, are Doctorate course is using Second Life to hold interactive classes to discuss topic Futuring and Innovation (Socio-Technical Futuring). These sessions have shown the participants how to move their avatar, create a lamp, chair, fountain objects from a cylinder, and chat using voice and typing features. Other opportunities discovered are the usage of Second Life by businesses and other University to hold virtual conferences.

I can see the value of Second Life to extend the sharing of information through virtual conferencing and collaboration session. I understand the potential for simulating the economic commerce environment to apply business and marketing strategies which could be applied in real life. I plan on continuing my edification after the course and my dissertation is complete.

Here is a photo of our class avatars in Second Life.

Analysis of Web 2.0 - Blog Tool

The Web 2.0 Blog application is an extraordinary social networking tool. Blogging is an effective medium for sharing ones beliefs, ideas, concepts, and perspectives. The blogger must be aware that they will receive both good and bad criticism and alternative viewpoints from a potential global audience, depending upon the blog configuration. The collaboration of thoughts and ideas promote innovative starting points which may mature into a new product or service. The disadvantage of the global brainstorming is the lost of ones intellectual property.

Another downside of Blogging is the potential and realistic economic consequences encountered when writing about an organization the Blog owner is affiliated and associated. The consequences are severe when the Blogger portrays and characterizes an organization in a prejudicial, detrimental and one sided viewpoint. The economic consequence involved the immediate dismissal from employment.

Overall the Web 2.0 Blog tool is an extraordinary freedom of speech instrument that allows global discussion of local and global issues and problems regardless political structure but not necessarily of their consequences.