Wx - Weather Extension

1) - What is it?

Wx is a program to augment the capabilities of the HAL 2000 Home Automation system. It provides an interface between HAL and the Ann Arbor, Michigan based Weather Underground Weather Service. The Wikipedia entry for them is at

http://en.wikipedia.org/wiki/Weather_Underground_(weather_service)

For convenience, the service will be referred to as WU in this document.

2) - Why?

HAL incorporates a direct weather station interface. Unfortunately, there are very few weather stations that are supported by HAL, and even for those supported, HAL does very little with that interface.

Essentially, you can ask HAL what the current weather conditions are, and he will speak the output of the weather station. But he cannot make any real-time decisions based on those conditions, or even give you a specific parameter, such as temperature.

3) - How is Wx different?

Wx retrieves the same weather data from a nearby station that is participating in the Weather Underground network. This means you do not need to have a station of your own if there is one nearby. Wx uses HALi to plug the data into HAL as if it were a locally attached station, giving HAL the same capabilities it would have with a local station. Or alternately, place your own weather station on the WU network and have HAL retrieve it's data, without regard as to whether it is supported by HAL or not.

    But wait, there's more.

Wx also creates a number of small text files that can be spoken by HAL. This expands the capabilities beyond merely reading off the weather station data. Now you can ask for specific items, such as temperature or humidity. Merely create rules for HAL that return the requested data.

    But there's still more.

Wx looks for various HAL flags, and if present, sets them when the weather condition it references is true. For example, if HAL has a flag called "WEATHER CLEAR" and the WU report says the skies are clear, this flag will be set to TRUE. There are flags for physical conditions as well as subjective judgments such as being warm, cool, hot, cold and windy.

    Is there anything else?

Yes, Wx also displays the weather on the desktop in real time, and can be set to announce various parameters using HAL's voice when those parameters change.

4) - What is this HALi stuff?

HALi is an interface provided by HAL2000 for reading and writing it's internal data. It uses ActiveX and provides tremendous flexibility in the interface.

5) - How do I get Wx and what does it cost?

Wx is free, and you get it from my web page. It is currently in beta with a limited audience, but once released it will be free for the download.

6) - Wx is a dumb name. Why did you pick it?

The earlier version of this program used AccuWeather and was called simply AccuWeather. There were two problems with that. AccuWeather did not provide a clean interface for getting their data, and they owned the name.

We changed to WU for the data source, since they have a much better interface, and we named it 'Wx' for Weather Extension, since this is an extension to HAL (for Weather).

7) - What do I need to use Wx?

Several things. First you need a computer (ha ha). On that computer you need HAL2000 (or one of it's variants). The computer also needs to be Internet connected.

Beyond these obvious needs, you need Perl.

8) - Perl? What is Perl?

Perl is a language. It was created in 1987, and has become largely the dominate language underpinning the Internet. While a bit dated, and slowly being supplanted by newer languages, it is nonetheless pretty much everywhere, and has a huge following. There are also a great many supporting packages, or modules that can be installed to enhance Perl capabilities. The Wikipedia entry is here: http://en.wikipedia.org/wiki/Perl

9) - Why Perl?

Several reasons.

First, I like Perl.

Second, it is extremely easy to do very complex things in Perl.

Third, it is NOT a Microsoft language. I'm no Microsoft basher, but I do not always like to do everything "the Microsoft way". While a leader in the industry, they do not always have the best solution to a given problem. So I prefer Perl.

Finally, Perl is an interpreted language, and as such, it is very easy to distribute source code. Free software should come with source code.

(10) - How do I get Perl and what does it cost?

Perl is completely free. Just download it from http://www.ActiveState.com

The full link to the current version at this moment is:
http://www.activestate.com/store/download.aspx?prdGUID=81fbce82-6bd5-49bc-a915-08d58c2648ca

Unfortunately, this is not likely to be a permanent link as it changes when new updates are released. You want the Windows .MSI package. Download and run it, and it will set up all you need.

(11) - What are the Packages and how do I get tthem?

Perl Modules, sometimes called Packages, are pre-written, tested and debugged chunks of code, written by some good Samaritan and published to to a public repository. They are universally free. This is one of the great parts about Perl. Almost anything you want to do, someone has already done, and you can grab their code and use it with very little effort. This is unlike many other languages where developers will happily sell you their closed library modules, and availability of free and reusable modules is limited.

With the Perl package linked above comes something called the PPM, the Perl Package Manager. This is a GUI front-end to a large number of archives of Perl modules that are freely available. Simply run the PPM, and navigate to the package needed by name, and 1 click downloads and installs the package.

Some of the packages we use are:
Weather-Underground.pm written by Mina Naguib. It encapsulates a nice interface to WU functions.
Tk.pm written by Nick Ing-Simmons. It provides a set of Graphical User Interface widgets to make nice GUI displays with.
Config-IniFiles.pm written by Jeremy Wadsack. This provides a nicely encapsulated set of code to manage Windows style INI files.
Tie-Handle-CSV.pm written by Daniel B. Boorstein. This package provides easy access to CSV files, which we use to decode the files returned directly by the individual weather stations.

Mechanize.pm written by Andy Lester. Provides for Web Browsing within a Perl Program.

Win32-OLE.pm which provides OLE Automation extensions for accessing HALi ActiveX controls.

All of these packages are readily available via the PPM with automatic installation just a few clicks away.

(12) - Where are these packages?

There are various archives on the Internet. They are called Repostiories. PPM comes by default from ActiveState with several repositories already set up, others you can add. The biggest archive is CPAN, but strangely enough CPAN is not accessible directly from the PPM as a repository. Instead ActiveState keeps a copy of the CPAN in their archive. Or most of it. Don't ask why. It's complicated.

(13) - Back to Why Perl? Isn't this just a case of "when the only tool you have is a Hammer"?

Uh, No. I have Visual C++, Visual Basic, Delphi (Pascal) and others too numerous to mention, and could have written this in any of them. But having used all of those, I find that giving someone else the source code is a pain as re-creating someone else's original programming environment can be tricky. Further, when using fancy GUI oriented developer tools, I find myself seduced into spending hours fiddling with the complex User Interface, hours that could be spent programming. Perl is un-matched for the ease with which one can quickly churn out a moderately complex function.

Further, I am convinced that unless one knows Perl, one is not quite proficient with a computer. Perl is a "down to the bare metal" and "Get things done", no nonsense tool.

This exercise was intended to be educational. To provide a clear and well documented example of precisely how to build a moderately complicated application, with examples and sample code that any nominal non-programmer can figure out. By doing this in Perl, I introduce the student to the language and give them everything they will need, by example, to write their own applications. Yes it's a little daunting at first. Anything worthwhile is. But if you've ever done anything more complex than "Hello World" in, say, Basic, you will quickly find Perl to be both easy and powerful.

(14) - Why make users mess with all this Perl stuff, Why not just give them an EXE?

I could indeed purchase a commercial Perl to EXE compiler and do just that. I may one day. It's actually on my to-do list. But the purpose of this exercise is educational. Get your hands dirty and learn something! Darn it!!

I'm not giving away a weather utility, I'm teaching programming skills.

(15) - Why no installer?

I could easily provide an installer to hide all the bones, hide the Perl, etc. As to why not, see the previous two questions. Get your hands dirty!

(16) - Everything, except Perl itself, some graphics and the required packages is in this one text file? Isn't that odd?

Yes and no. It has been often said that Perl is the ideal language for programs up to about 10,000 lines. Others have put the limit at perhaps 1,000. This program is about 4100 lines, but an awful lot of those are comments. I could probably take them out and trim it down to less than 3000 lines.

However the reason for the sentiment is that many Perl programs are small, short scripts that do one thing and are self-contained. This does not need to be. I could just as easily have put each subroutine in it's own file, and had the program include them all and made it extremely modular.

The reason to make many small individual files rather than one large one is that you can apply a version control system and have many programmers all working simultaneously on the program, each programmer "checking out" files he is working on, and "checking in" the files he is done with, and the Version Control system keeping track of changes so any mistakes can be "rolled back" with ease. This is necessary for large programs with many programmers.

And if you do that, then you can write programs of any size in Perl as easily as any other language, and indeed some very large programs have been written in Perl.

But for now, there is one programmer, me, and no automated version control system. And I intend to distribute the program widely. So one file is convenient. I can give anyone this one file and they can run it anywhere with minimal fuss.

(17) - So what do I need to write Perl code? There's no editor, no development GUI.

What do I do?

Those things are certainly available. ActiveState will sell you fancy GUI tools. But you don't need them. In fact, I think they get in the way. You do not need anything more than Windows Notepad. Although to be fair, notepad is a bit too minimalist. At minimum you need an editor that will tell you what line you are on. Some folks may be familiar with Emacs or Jove, or even Vi. Those all work fine for Perl. But if those names are meaningless to you, consider PCE (Perl Code Editor) from Perlvision.com. It's free for the download, and does most of what you would want or need.

Everyone interested in Perl should own a Camel Book. The name comes from O'Reilly Media (the publisher) and their practice of putting an animal on the cover of every book. When "Programming Perl" was released, it had a Camel on the cover. Hence the bible of the language is known simply as the Camel Book.

There are many, many books about Perl, some worthwhile, some not. But the web has all the documentation and tutorials you really need. You don't even need the Camel book, really. But come on, it's cheap.

(18) - You said something about putting flags and macros in HAL.

What does that entail?

If you are using HAL, you may have experimented with writing HAL Macros, or Rules. But if you are a complete beginner, you may need a few pointers to getting started.

Begin by opening the HAL Automation Setup Wizard.

[ Need detailed description of installing HAL Flags here]

Wx provides you with the underlying components HAL needs to make excellent use of the Weather information. There are some 25 flags that are set to true when various weather conditions occur, including several that are tied to thresholds of temperature and wind speed. This allows you to write rules to cause HAL to, for example, turn a ceiling fan on when it gets hot, or to shut off irrigation when it's windy.

There are also some text files created that HAL can use to speak specific weather conditions. So you can write macros to respond to questions like "What is the outside temperature?" or "What is the Barometric pressure?"

(19) - Exactly how do I get started?>

First start with Perl itself.

Download it and run the .msi installation file.

Then go to Start|Run and type 'cmd'

In the resulting dos box, type 'cd \Perl'

Next, type 'Perl -v'

Perl should respond 'This is Perl,' followed by version numbers and some other information.

Perl itself is now installed and functional.

Next install the particular Packages we need

type 'PPM'.

This should open a GUI application to manage the Packages.

Select View|All Packages

In the search box type 'Config-IniFiles'

The display should narrow to just a couple of packages. Right-Click on Config-IniFiles and select 'Install'

Select 'Action|Install' and let it install the package. It will take a few minutes to download and install it.

Repeat the above steps for the remaining packages:

Win32-OLE

Weather-Underground

Tie-Handle-CSV

WWW-Mechanize

Tk (Note: There's a bunch of these, you want 804.027-r6)

Win32-SAPI5

Note, this last one is the source of some confusion. Not everyone's PPM is finding it. I have included the files in the zip I am distributing, so if you unzip the files into the appropriate directories you won't really need it. But do try to install it and let me know whether you have problems or not.

After Perl is installed, download and install the zip file containing Wx.

Included in the zip file is a file named wx.ini which holds the configuration information. You may edit this to update any information necessary. You will want to set your zip code, your location and the station ID of your nearby WU station. These are all search criteria for Weather Underground. Or you can set everything from the Wx application itself once it's installed.

Wx should install in C:\Perl\Wx, and will place files in C:\HALogs and C:\HAScripts.

Once Wx is installed, place the command 'wperl c:\perl\wx\wx.pl run' in the HAL System Setting Personal Assistant Startup Applications box.

While working with HAL, install any macros you want to use to query conditions, as well as any rules that are to be triggered in response to conditions.

Restart HAL and Wx should start up automatically.

If it does not start, try starting it manually from the dos box using the command 'perl wx.pl'. If it encounters any errors, they will print on the dos box, hopefully telling you about some problem you can easily fix.

(20) - What is in this ZIP file?

The zip file has this FAQ file, plus a huge text file that is the Perl Source code for the application. It also has some icons that the application expects. These files are placed in directories C:\Perl\Wx and C:\Perl\Wx\Icons.

These are all the files really necessary for the program. Everything else is a part of perl, or downloaded from the PPM archive. Unfortunately, there is one minor complication. For reasons that are presently unclear, not all users of the PPM are finding all of the files as they should. The Win32-SAPI5 files are proving a bit of an annoyance.

Therefore, for the moment, until we can resolve the issue, I am distributing the 4 files required to make SAPI work. They are SAPI5.pm and Locale.pm which are placed in the c:\Perl\site\lib\Win32 directory plus a file named .packlist stored in C:\Perl\site\lib\auto\Win32\SAPI5 and SAPI5.html which is stored in \Perl\html\site\lib\Win32.

Simply unpacking the zip file should place these files properly. But rather than rely on the files I am providing, try to install SAPI5 using your own PPM, and only install them if you cannot find SAPI5 in the PPM.

(21) - What's this about SAPI?

SAPI is the Microsoft speech interface provided in most versions of Windows. HAL has it's own different (and better) speech interface using HALi, so for normal use, SAPI is not required. If SAPI gives you trouble and you do not want to use SAPI, simply edit the wx.pl source code and place a '#' in front of the line that says 'use Win32-SAPI5'. Make sure you set all speech options in the INI file to 0 if you are not using HALi. If you install the SAPI files you do not need to do this, as SAPI is not used unless you intentionally do so, but the program will die if the SAPI files are not present.

However, SAPI does provide an alternate speech interface should HALi be not working, unavailable or simply running the program on a non-HAL machine. In this instance, Wx can still be configured to speak using the 'Microsoft Sam' voice.

(22) - How do I configure HAL?

Wx will run on a HAL enabled machine and speak via HAL without any specific configuration setup in HAL. But to get the maximum use from it you may want to add some configuration options to the HAL installation.

Wx interfaces to HAL on multiple levels.

The first and most rudimentary is speech. The program can announce the temperature periodically, announce when conditions change and so on without HAL doing anything.

The second tier of speech interface is to create a number of small text files with weather information in them ready for HAL to speak on demand. They are as follows:

Barometer.txt. - This contains a phrase similar to "The Barometric Pressure is 29.03 inches of Mercury and Falling."

Dewpoint.txt - Similar to the above, contains the dewpoint reading

Humidity.txt - The current humidity

Temperature.txt - The current temperature

Visibility.txt - The visibility in miles

Wind.txt - The wind speed and direction.

Two other files need a little discussion. Weather.txt is a longer file that contains essentially the text of all of the above files, arranged in a nice format for HAL to describe the overall outside weather. This is a good response to the question "What's the weather like outside?"

The remaining file is WeatherLog.CSV. This is a Comma Separated Values file of the data from all weather reports. This is intended to be imported into Excel, or another program, and to be used to produce charts and graphs of the weather data. Use of this file is left to the user.

Using the HAL Automation Setup Wizard to create simple rules to speak the contents of the text files allows HAL to respond nicely when asked questions like "What is the outside temperature?".

The next level of interfacing to HAL is to "pretend" to be a locally attached Davis weather station, using the HALi interface. Under system settings, check the checkbox for 'Weather Station Enabled' and set station type to HALi. Note, this does not yet work with current versions of HAL. A soon to be available beta version should enable this feature.

The next method Wx uses to interface to HAL is to set flags in HAL that HAL can then use as triggers, or as secondary conditions in rules to make decisions based on the weather. When Wx retrieves the weather report, it takes the reported conditions (Clear, Rain, Snow, Cloudy, Overcast, etc) and sticks the word 'WEATHER ' in front of it. It then looks for a flag in HAL by that name. If it finds it, the flag is set true. e.g. If Weather Underground reports the sky is overcast, Wx looks for a flag named 'WEATHER OVERCAST' and if present, sets it true.

If it does not find the flag in HAL, it writes an error message to C:\HALogs\WeatherError.txt. This file can be occasionally reviewed to determine if new flags need to be added to HAL. There are currently 20 such flags defined, and more can be added by simply adding the flag to HAL. No changes are needed to Wx to make use of any new flags.

There are also five extended, or threshold flags. These are named Wx flags, and they are named 'WX COLD', 'WX COOL', 'WX WARM', 'WX HOT', and 'WX WINDY'. These flags are tested against some thresholds in the INI file and if the threshold is exceeded the flag is set. This can be used, for example, to have HAL shut off irrigation systems if it's windy, or turn on an attic fan if it's hot. Like the weather flags, if they are not present in HAL, they are ignored, but it does NOT write an error to the error file.