How does Area42 deals work?


Machines do all the hard work for you.

For the geek out there, the app is build relatively in a very simple manner. The steps involved in delivering you the relevant deals are:

  1. A simple python script fetches the deals from various deal sites.
  2. All the irrelevant words (stop words) are removed from the title of the deal
  3. Then a script creates a phonetic encoding of all the relevant keywords (like ipod=APK, android=AADS)
  4. Then an inverted index is created in redis in this structure:

            Key: APK
            Value: set(
                {"title" : "Awesome deal on iPod for $10", "link" : ""}, 
                {"title" : "Another awesome deal for iPod", "link" : ""}
            Key: AADS
            Value: set(
                {"title" : "Awesome deal on Android for $5", "link" : ""}, 
                {"title" : "Another awesome deal for Android", "link" : ""}
  5. Finally, based on your likes (keywords submitted by you). A cron job aggregates the data and sends out an email!

Stuff that powers Area42 Deals

  1. Python: The language.
  2. Jellyfish: Jellyfish is the awesome python library which does the magic to construct the inverted index. For now, I am doing two kind of matching. For 1. String comparison:Levenshtein Distance and 2. For Phonetic encoding: Double Metaphone
  3. Redis: All the data aggregated from various data sources is stored in Redis in form of an inverted index
  4. Flask: The website runs on flask. The awesome python web framework!
  5. Fabric: I am using fabric to deploy the code to EC2.
  6. EC2: The app is hosted on Amazon's EC2