LVSpeak: Automating VI development through speech

Eyes on VIs is pleased to welcome its first guest blogger, Norm "The Captain" Kirchner! Norm is the first person in history to sacrifice his LabVIEW Champion status for the pleasure of working at National Instruments. (NI Employees cannot be LabVIEW Champions). Norm has been using LabVIEW for over 9 years and he is going to share his "LVSpeak" project, which (I hope you will agree) is pretty darn amazing. Thanks for joining us, Norm!

- Christina

---

Imagine if LabVIEW was able to read your thoughts and react to them. You just think “edit icon” and *pop* the icon editor is opened for you immediately. Imagine if every time you wanted to drop a property node for a control or group of controls, they just showed up on the diagram.

Although this ability is not implemented in LabVIEW yet, we can get darn close by using our voice and a little creative coding.

LVSpeak (LabVIEW Speak) is a very simple concept with a great deal of possibilities. It utilizes the Microsoft Speech API (SAPI) and provides an open and extendable interface to it within LabVIEW. Currently, two extensions of that architecture exist. They are ‘Quick Edit’ and ‘Speech Enabled Quick Drop’.

These extensions of LVSpeak and the LabVIEW development environment give the coder tools to greatly speed the process of code creation and modification upwards of 70%.

Note: Volume on the videos is very low, turn speakers all the way up

Video 1: Code creation

WHY

LabVIEW is an easy to use and intuitive application development environment and programming language, but there are still some basic actions that require multiple clicks and force the user to navigate through a variety menus and options.

LVSpeak was developed to take those minor but time consuming steps and reduce them to a single voice command. At the root of this entire effort is one simple premise, “A good engineer is a lazy engineer”. And until we can program LabVIEW with our minds, turning a 4 step, 3 second action into a ½ second, 1 step voice command, makes me a happy lazy engineer.

At the end of the day, any action that requires you to remove your hand from the mouse (Ctl+I), or needs more than two mouse clicks and navigation deeper than a top level context menu (Label Visible) is warranted to become a Voice Command

HISTORY

When LabVIEW Scripting was still very new in LabVIEW 7.0 and some of it’s functionality was accidentally exposed by NI, it occurred to me that you could combine this scripting ability to use LabVIEW code to write LabVIEW code along with the free Microsoft speech recognition technology and do some creative things. Although an interesting idea, integrating dll’s and ActiveX objects into G was still foreign to me and presented a barrier that caused LVSpeak to sit dormant until NI Week 2008 and the release of Quick Drop.

Almost immediately I recognized a synergy between this great new development tool in LabVIEW and the still undeveloped LVSpeak. This was only confirmed further when I was watching the coding challenge at NI Week 08. During the coding speed challenge, the creator of Quick Drop, Darren Nattinger, was slowed to a crawl when doing simple tasks like creating a constant or typing verbose function names in the Quick Drop window.

At that point I realized that all the components needed, to allow the developer to program as fast as they could imagine the code, were in place.

HOW

The how is actually more simple than I would like to admit.

There are two key components to the Microsoft SAPI that are utilized in LVSpeak

• Grammar List
• 'Speech Recognized’ .NET Callback Event

Within LabVIEW, two components are required to enable speech recognition in any program.

• Load Command List (Grammar)
• Register for speech recognized event

The way everything plays together is quite simple

LVSpeak Core starts and

1. Initializes the Microsoft SAPI
2. Creates a LV User event
3. Registers a callback VI to be run when speech is detected and fire the LV User Event with the command
   

Programs utilizing LVSpeak

1. Register for the Grammar Detected event
   
2. Load their command list into the “Grammar”
3. Catch the fired event and respond accordingly to the string
   

For speech enabled Quick Drop, the grammar list is everything in the function palette, and whenever the LVSpeak event is caught, it takes the detected string and loads it into the text box as if you had just typed it.

Quick Edit follows that same flow. The grammar list is all created Quick Edit commands pulled from an enumeration. When one of those commands is detected, it runs the corresponding code to execute that command.

Video 2: Basic detection

Video 3: Quick Edit Demo


Getting Started

To start developing LabVIEW with your voice you need to download and install

• VI Package Manager (JKISoft.com)
• Microsoft Speech SDK 5.1 (http://tinyurl.com/lzz3t5)
• LVSpeak Core
• LVS Quick Drop Addon

Once all these parts are installed, you should see a new item in your LabVIEW tools menu "Enable LVSpeak"


Select that option, and you should see two floating windows show up.
If you microphone is active, manually drop some controls on your front panel, flip over to the block diagram, select all the controls and say "Label Side" in a relatively monotonic voice.



Summary


If you program LabVIEW on a regular basis, begin paying attention to how long some tasks take and how often you repeat some basic tasks that could be streamlined by a little scripting automation.


Controlling LV with your voice is not just a novel idea.
It is truly a quantum leap forward in how you develop your code and a HUGE performance booster.

So wipe the dust off of your microphone or headset and get ready to take your development process to the next level.

~,~
The Captain Was Here

Norm Kirchner

PS Thank you Christina for your patience with me getting this out to you and providing a great resource for the rest of us LabVIEW nuts out in the world.

VI Makeover Edition - Customizing a Slider

In a previous post, I talked about a VI for an NI Week keynote demo.

Now I'd like to talk about customizing the slider control for that project.

You can watch the video on EyesOnVIs.com or YouTube, or you can read the text version below.

When customizing a control, I like to start with the simplest control of the right type, which usually means controls from the Classic palette. For this demo, I customized a Classic Vertical Pointer Slide. Note that the control editor lets you change the appearance of a control, but not its functionality.

You can edit a control either from the Edit>>Customize Control menu or from the Advanced>>Customize right-click menu.

This will open the control in the Control Editor. From there, you can customize using the normal right-click options (such as setting the Scale style as I did for this demo) or go into "Customize" mode either by clicking the toolbar button to switch to the tweezer icon or by showing the Parts window from the Window menu.

You can find the slider thumb by cycling through the Parts and then right-click on it to see the options for changing its image. For this demo, I used a graphic that I created myself in a graphics program, but there are lots of sources for nice, professional images that you could use for your projects. You can find the graphics I made in the Image Archive on my website, EyesOnVIs.com.

After customizing the thumb, I added some free labels for + and - scale markers, made the text big and bold so it would be easily visible on the keynote projector screens, and colored the increment and decrement arrows transparent.

I hope this inspires you to make your own custom controls and take your LabVIEW UIs to the next level!

LabVIEW Scripting

If "VIs dynamically constructing new VIs" doesn't sound like the premise of a bad summer movie to you, then you might be interested in the recent LabVIEW Scripting post on NI Communities.

To quote Darren Nattinger (who has a lot more experience with Scripting than me), "LabVIEW Scripting is an (until today) internal LabVIEW feature that allows you to write VIs that create, modify, and inspect other VIs."

Important note: "As this is an 'NI Labs' product, NI will not be supporting LabVIEW Scripting through Applications Engineering." That means if you have technical questions, you will need to post them to the API Community instead of going through NI's usual support channels.

In my experience, Scripting is very difficult to work with, but I am really looking forward to seeing what people will do with it. JKI has already announced the JKI Right-click Framework for LabVIEW, which will allow end-users to add features to the LabVIEW editor itself. Pretty cool.

Help Define the New LabVIEW Advanced Course

Scott Romine and Nancy Hollenback are rewriting the LV Advanced I Course, and they want your help!

Join the Designing, Drafting, & Developing the LV Advanced Course group on NI Communities to participate in discussions and polls that will shape the future of the course.

I've joined the group, and I look forward to hearing everyone's input on what techniques every LabVIEW developer should master!

Tree Control Scrollbar Tip

One of my colleagues recently showed me a VI that automatically shows and hides the scrollbars of the tree control, as well as resizing the columns, depending on the text in the cells. Those of you who are familiar with the tree control are probably aware that it doesn't have this capability built-in.

The VI isn't in the palettes but is in the tree folder of vi.lib. It's called Tree_ShowScrollBar.vi.

To see a brief demo of the functionality, check out this video on my website. If you have trouble viewing that video, you can try the YouTube version.