Shopping Gizmo Firefox Add-on

A mostly, but not fully finshed re-write of the Shopping Helper Firefox Add-on for helping you shop on-line. Alerts you to prices at other sites while you shop. Allows you to keep lists of products and get alerts when their prices change. Allows you to build up your own comparison grids. And much more as detailed below.

This Add-on is the fourth generation of the basic ideas I originally came up with back in 2006 while working for Pronto.com. Since I had done all the development of this on my own time, but it used the company logo, I needed to re-brand it as well as remove any lingering dependencies on Pronto-specific things.

The Shopping Gizmo is the fourth incarnation and evolution that started with a basic, common idea of having a browser add-on that could help you shop by alerting you to prices on other sites. This was the main premise of Pronto.com whehn it started, and there are many variations of these already out there, though none as fully-featured as what the Shopping Gizmo does.

Here's the brief explanation of the evolution.

• Pronto Fox (first generation, 2006) - Prototype I made in my spare time to attempt to reproduce the features of Pronto's main product inside Firefox, since Pronto's official version only worked with Internet Explorer (IE).
• Shopping Messenger (second generation, 2006-2007) - Officially sanctioned and some additional development by Pronto to re-brand ProntoFox as a company offering. It was a featured Firefox add-on hosted on Mozilla's add-ons site and was downloaded over 500,000 times.
• Shopping Helper (third generation, 2008-2012) - A complete re-imagination of the browser add-on with many, many new ideas and features. Created in my spare time after Pronto abandoned the strategy of in-browser add-ons. I got permission to use the Pronto API and logo for it though and offered through Mozilla's add-on site.
• Shopping Gizmo (fourth generation, 2013-?) - Re-branding of Shopping Helper with removal of Pronto.com dependencies, many code and UI improvements and easier ability to integrate with other product APIs.

A section below has the excruciatingly long story behind of the history behind the first two generations.

The bulk of the work of transforming the Shopping Helper into the Shopping Gizmo is done, including most all the coding. Here's the major things that were still needing to be done at the time of this writing:

• The help pages need to be updated.
• The images and screenshots are needed.
• The web site pages needs to be updated.
• Needs to be tested with newer version so Firefox.
• Needs to be submitted for approval to Mozilla's add-on site.
• A long to-do list of new fetaures and enahncements.

The previous version is still available on the Shopping Helper Mozilla Add-on Site, but with each new version of Firefox, there is some chance that something will stop working.

As with all Firefox Add-ons, the bulk of the coding happens in JavaScript. However, there are a lot of other components that are required to make this all happen:

• XUL - for layout of windows and dialogs
• CSS - for styling everything
• XPCOM - for internal component communication
• HTML - for non-XUL visual components
• SQLite - for data storage
• AJAX - for API interactions and page fetching
• XPATH - for extracting data from web pages

The screenshots for this section are all the originals from the Shopping Helper, so do not reflect the rebranded views, but the prior views.

The core feature (what started it all) is to alert the user as they are shopping on one web site about the availability and prices for the same product on other sites. Rather than the user having to hunt around the web, tihs provides automation to do the job for the user. This is the one feature that depends on some API to be able to query for product data. Originally it used Pronto's API, but can now be integrated with any API. The notification is a appearance on the top of the screen of a small banner where the information can be accessed.

As the user browses the web, they can add products they encounter to one of any number of shopping lists they create. Though a regular browser bookmark might seem to achieve the same thing, this is much, much more powerful. First, the products in this list are also continually refreshed in the background, so that if their price or availability changes, the user will be notified. Second, the price history of everything in the list will be tracked. Finally, prices from other stores can be automatically be pulled in for comparison purposes.

There are two ways a product can be added to the list. In order of simplicity to the user they are:

• Automated - The user clicks to add the product on the current page and the extension will extract out the product titles, image and price for the user. This is not always done perfectly, so the user is presented with a confirmation screen and some alternatives in case the chosen ones are wrong.
• Manual - If the automated addition fails, the user can manually selected the product title, image and price by right-clicking on the page. Once all components are selected, the user then adds the product and it will extract the infomration.

The shopping lists that are created are displayed in a grid that shows the prices for each product at each store. Though products are explicitly added by the user, the stores are automatically added based on the browsing behavior of the user and their selected store preferences. Stores can also be manually added and removed from any list so that the grid contains precisely the -products and stores the user wants.

Besides showing the products, stores and prices in the grid, this is also the place where the shopping lists can be managed. This includes adding and removing items, re-ordering the items and accesing detailed product and store information. AT right it shows the context menus and options for managing the shopping lists, products and stores.

For all products in the user's lists, once or twice a day, their prices are automatically refreshed and compared to the previous price. If a product's price changes significantly, the user will be alerted by a status bar notification which also allows them to view the details of the product alerts. Products that go in and out of stock are also tracked where the user has a choice of whether they want to be alerted to those types of events.

For as long as a product in in the user's lists, the price is continually refreshed and tracked for the user. This is done to provide product alerts, but it also keeps the price history so that the user can view how the price has chnaged over time.

For products in the user's lists, extra details are kept about the rpoduct that can be edited by the user. The user can keep notes about the product or alter the display title, or change the bookmark link to a different page.

Often times a user might run into the same product from two different stores and want to add the product twice. If the API being used recognized that they are the same product, they the products will automatically be merged so that the product appears in the list once, but with both stores added so the price from each can be viewed. However, if the API is not aware of the products being the same, the user has the option of manually merging the products to explicitly indicate that they are the same.

The user can select a set of favorite stores which can be used to perform searches constrained to only those stores. They can create multiple sets of favorite stores so they can use different favorites for different types of products. These favorite stores also help automatically opopulate the shopping list grids, so that even if the user does not add a store to a shopping list, if the product is sold by that store (and the API recognizing it), when they add the product, the favorite store will automatically show up for them.

Much of the behavior can be customized so that the user can completely control when they get notified, how frequently prices should get refreshed, whether to open new windows for links, and a lot more.

This was originally written for an internal Pronto.com newsletter (my former employer). It has been edited to remove names and any other confidential information.

The original Pronto download-able price comparison client (circa 2005) was an "IE BHO", which stands for Internet Explorer, Browser Helper Object. This means it was written in C++ and it required all the libraries and interfaces from Internet Explorer. It's not quite an easy integration like a Firefox or Chrome plug-in is, especially if you are trying to get PERL-style regular expressions working (which was required at the time). Naturally, Pronto targeted IE because it was the largest install base for browsers in 2005. Adding Firefox support was to be a secondary effort we would do after the IE version was out. Development of this client was done out of the New York City office by the applications team.

Out in Pronto's Boulder office, the folks on the Search & Data Team (there were 7 of us, myself included) were plugging away at crawling, classifying and normalizing product offers. The entire Boulder office was using various versions of GNU/Linux and in fact, there was only a single computer (an under-powered laptop) in the entire office that ran a Microsoft operating system. This created the unfortunate situation where no one in the Boulder office could actually use the company's main (and only) product. Seeing that everyone joined Pronto because they thought a shopping assistant client was a great and useful idea, a desire to be able to benefit from the fruits of our labors was growing. We were creating all the content that the client would require.

By January of 2006, an officially supported Firefox version was beginning development. On the surface, this sounded like good news for us GNU/Linux users, but in reality, this was not going to solve the problem. What we found out was that the "Firefox Version" was not planned to be a true, self-contained Firefox Add-on, but instead it would be a simple add-on "shell" that launched the Pronto IE BHO application: it would merely pass the page URL from Firefox to the IE BHO as you browsed. The computations, server interactions and the visuals were all still occurring in what was effectively an IE instance. Thus, unless you had IE, you still could not run Pronto's offering: i.e., You could run Firefox, but only on MS Windows. Note that I am not sure if we had any Mac users at Pronto at this point, though there was a slight undercurrent of wanting Mac support in some corners of the NYC office, and the planned Firefox version would not have helped them either.

Trying to solve the problem a different way, I devoted the long MLK weekend of 2006 attempting to build a native, self-contained Firefox Add-on that would interact with the Pronto servers the same way that the IE version did. Since the client API into the back-end data was not actually documented at this time and since no one was really working that weekend, I had to reverse engineer the Pronto client and learn how to build a Firefox plug-in. I also did not use the richer page extraction models which the IE version used, but instead stayed with a simpler URL-based method of detecting product detail pages. Although 3 days was not enough to produce a polished product, I did succeed in getting the add-on working and displaying "toasts" while navigating to product details pages. A "toast" was the word we used to avoid saying "pop-up", which had avery bad connotation at the time. The excitement I felt when the first toast "fired" was among the most exciting Pronto experiences I have had.

Here is the text of the email I sent out to the GNU/Linux crowd announcing the first version of ProntoFox:

	 Reflecting on the glorious corporate generosity handed out on behalf
	 of the late Dr. Martin Luther King Jr., I paused to think about his
	 dreams and ideals.  A world where everyone has equal opportunity,
	 whether to ride the buses, use the restrooms, or to save money when
	 shopping on-line.

	 While I am not quite committed to devote my entire life to fighting
	 the epic, noble battle, I did decide to devote a weekend of my life
	 to narrowing the gap between the haves and have nots, attempting to
	 eradicate as much [platform] discrimination and injustice as
	 possible.
    

At the time, we put all these experimental projects under the guise of "Pronto Labs", which had a very nicely designed logo (by one of the other co-founders) you can see at the right.

Of course, only the folks in the Boulder office really cared at this point, and it was not polished enough to warrant any general announcement, but it was worth offering to the limited audience since it was better than nothing. To be perfectly honest, there was also some worry that those working on the official Firefox version might be offended, since the difference between the two was fairly subtle if you operated in the MS Windows world. We did mention this to a few other top management folks at the time to ensure we kept everything in the right context, and got the blessing to use this as long as we limited it to internal use. From the company perspective, we did not want the rest of the world to get confused about us having two different Firefox Add-ons: brand dilution was the fear.

Over the next few weeks, I invested more of my own time into polishing and extending the features of what I began calling "ProntoFox". This included the nifty little logo you can see at the right, but also added a bunch of features that our other client did not have. By early March it was relatively functional and stable and was being circulated more broadly for internal use. One of the funnier (or maybe sad) moments was that by early April, ProntoFox accounted for 8% of all Pronto traffic, which caused me to have to put out a patch release so that this traffic could be better tracked.

Meanwhile, on the business-front, one of our biz-dev folks had been busy and found some interest in our upcoming (official) Firefox version. Mozilla was gearing up for doing a big splash for the upcoming Firefox 2 release and were looking for a handful of useful add-ons to be recommended to help promote it as a better browsing platform. A shopping assistant add-on would be very attractive to the general public. However, what would not have been appealing to Mozilla was an add-on that deferred the work to an IE BHO. Such a thing was neither cross-platform, nor did it really highlight the usefulness of the Firefox ecosystem. The limitations of our current (official) Firefox Add-on, and its IE dependency, were not understood by the business team, so this had not been explicitly communicated to Mozilla.

The Mozilla partnership had gotten some serious legs before I first got a whiff of it (April 20th, 2006 to be precise). The first order of business was to explain to the biz-dev folks how surprised Mozilla was going to be when they learned what our Firefox add-on was mostly just an IE BHO. This took a little bit of explaining from a few of us. The second order of business was convincing folks that ProntoFox already had most everything we needed and had the potential to salavge this opportunity. In the conversations with Mozilla, they had suggested a bunch of improvements over our existing client, most of which ProntoFox already contained. They had already started on a development plan for these extra features, but we eventually got the point across and the decision was made to "product-ize" ProntoFox so that it could become a recommended Firefox add-on.

Turning my hobby into a product required some actual UI designs and would happily include an actual QA process. Since I was busy on my Pronto-time with other projects, the product-ization development work was given over to folks in the NYC office, though I was generally involved in a consultanting role. By the middle of June 2006 we had the first product-ized version ready for the folks at Mozilla to start reviewing, though it would be some time before they got around to it.

During July, we spent most of the month bugging the folks at Mozilla to review our new add-on. We were promised an extensive review from the Mozilla UI Team as part of the process of becoming a recommended add-on, but the upcoming Firefox 2 release was keeping them all quite busy. It was during this June to July quiet time that the development responsibility for ProntoFox shifted back to the Boulder office, though with me still working on other projects, I wound up being only half of the team. It was actually my daughter that was the other half of this team.

By the first week of August 2006 we had a spiffy new revision, and by the second week we finally got the long anticipated feedback from Mozilla. After incorporating this feedback, the focus shifted to the biz-dev end trying to pin down Mozilla about the details and process for becoming a featured add-on.

By the last week of August, we had a verbal commitment from Mozilla about being a featured add-on. Though the dev work was minimal now, there were a lot of details to wrap up from the external messaging, branding and legal perspectives. Despite my personal attachment to the name "ProntoFox", something with broader appeal was needed. For a while it was going to be "The Pronto Personal Shopper", which was the same name as the IE client. We then decided that the same name for two different things was going to lead to confusion. On Sept. 8th, 2006 we put the first version on Mozilla's add-on site (affectionately referred to as "a. m. o." for addons.mozilla.org). This first version was simply called "Pronto", though we expected it might change.

By late September we were upgrading the extension to work with Firefox 2, and refining the pages and messaging around the add-on. It was in this period that we settled on the final name for the extension: "The Pronto Shopping Messenger". We were starting to get a little press and a modicum of downloads from the a.m.o site and we were preparing a new version to coincide with us becoming a Mozilla-recommended add-on. We put out version 0.9.8 on October 6th, which would interestingly turn out to also be the final version we ever released.

For the next few weeks biz-dev was doing the bulk of the final touches in coordinating with Mozilla for the materials they needed for us to get into their rotation of recommended add-ons. There was great anticipation in these last days leading up to the main event. On Tuesday, October 24th, the Pronto Shopping Messenger went into Mozilla's rotation of recommended add-ons. It was a serious Pronto moment of anticipation and it did not disappoint. Immediately, the download traffic shot up exponentially. It was all very exciting, not just for me personally, but for Pronto to get its first major endorsement and best opportunity for distribution we had seen yet (and free distribution at that).

Before being recommended, the first 6 weeks that our add-on was hosted on a.m.o. we has a total of 1,900 downloads. Less than 24 hours after hitting the recommended page it was downloaded an additional 4,500 times. Merchant leads went from about 50 a day to over 750 on the first day we were featured. We also started to get a bunch of free press from other places due to Mozilla's recommendation. Things settled down a little, but we were sustaining well over 2,000 downloads a day through the first month. ", "

Downloads cracked the 100,000 barrier on December 18th and then accelerated a bit to reach 200,000 right around ProntoFox's first birthday in January. In early February Mozilla did some marketing themselves with us included, and we saw an exponential spike in traffic then. However, the 3rd week of March of 2007 was by far the biggest spike which saw us get 30,000 downloads in a single week. Both of these spikes were results of Mozilla driving more traffic to their recommended add-on pages, usually around a new release. By late March we were at over 300,000 downloads.

Meanwhile at Pronto, a new social-focused shopping project was just getting off the ground. This and other initiatives became the big focus, and though we were chugging away with a lot of downloads, ProntoFox was not generating enough revenue to pay everyone's salary. Downloads continued in the background as Pronto shifted its focus elsewhere. Sometime late in 2007, the back-end API the clients used had been completely left-behind. I am not sure exactly when the clients would have stopped working, but by the end of 2007, and after over 500,000 downloads, the Pronto browser add-on business was all but dead and the original client API might or might not have still been working.

If you go to the Mozilla a.m.o. site today (circa 2012), even though I have disabled the Pronto Shopping Messenger a few years ago, the site reports that there are still over 4,000 active daily users. The client cannot be doing much, but interesting to know that so many people are actively using some older version of Firefox and have not uninstalled this add-on.

On ProntoFox's second birthday, the MLK holiday weekend in 2008, work began on a followup Firefox add-on that did not depend on the old client API (which no longer existed). This used Pronto's newer Partner API (PAPI) and was focused more on new product features than direct price comparisons (shopping lists, price alerts, favorite stores, price histories, etc.) This add-on was the third generation Shopping Helper add-on. In comparison to the success of its ancestor, in the 3.5 years this newer add-on was available, it has only garnered 4,200 downloads and had only about 125 daily active users as of 2012.