Epic PyObjC, Part 3: Browsing, Caching, Indicating

Welcome to the third installment of the Epic PyObjC tutorial series, which aims to be a thorough introduction to both PyObjC and Cocoa on OS X Leopard. After working with Cocoa Bindings and metaweb.py in the previous segment, this time we're going to look at a few common tasks in application development: reading and writing from disk, handling double clicks, and using an NSProgressIndicator to improve the user experience.

If you haven't kept up with the project thus far, you can download the current state of the project here.

Double Click to View in Browser

The first thing we're going to do is make MetaWindow load the relevant page in a browser when we double click on a row in our NSTableView. To accomplish that, we need IBOutlets to both the tableview and the array controller. We need the tableview to call its setDoubleAction_ and setTarget_ methods, and we need the array controller to figure out the currently selected row.

But wait, you might ask, the MWController already has the search results stored in its results field. If we just ask the tableview what row index it has highlighted, then the we don't need to query the array controller! Unfortunately, I would reply with a melancholic smile--intuition to the contrary--that won't quite work in our situation. If you click on one of the column headers in the table view you can rearrange the columns in alphabetical and reverse alphabetical order, but the data in the results field is blissfully unaware of those changes. Instead, they are all being handled by the array controller, and thus only the array controller really knows what data is where.

Onward to the code!

At the top of MWController.py add an import for webbrowser, so that the imports look like this:

import objc, metaweb, webbrowser
from Foundation import *

Now we're going to create a method inMWController that will open a web browser for the entry in the currently selected row.

def open_(self,sender):
    selectedObjs = self.arrayController.selectedObjects()
    if len(selectedObjs) == 0:
        NSLog(u"No selected row!")
        return
<span class="n">row</span> <span class="o">=</span> <span class="n">selectedObjs</span><span class="p">[</span><span class="mf">0</span><span class="p">]</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">row</span><span class="o">.</span><span class="n">has_key</span><span class="p">(</span><span class="s">&#39;id&#39;</span><span class="p">)</span> <span class="ow">or</span> <span class="n">row</span><span class="o">.</span><span class="n">id</span> <span class="o">==</span> <span class="bp">None</span><span class="p">:</span>
    <span class="n">NSLog</span><span class="p">(</span><span class="s">u&quot;Row has no id!&quot;</span><span class="p">)</span>
    <span class="k">return</span>
    
<span class="n">url</span> <span class="o">=</span> <span class="s">u&quot;http://www.freebase.com/view</span><span class="si">%s</span><span class="s">&quot;</span> <span class="o">%</span> <span class="n">row</span><span class="o">.</span><span class="n">id</span>
<span class="n">webbrowser</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="n">url</span><span class="p">)</span>

Although the method looks a bit long, its mostly because of code for handling two bad cases: the method getting called when a row isn't selected, and the selected row being incomplete.

If you haven't done much Cocoa development, it may feel like these method names are being drawn out of thin air, but its actually quite easy to figure them out. In XCode go to the Help menu, then go to Documentation (Shift-Option-Apple-?) and search in the search field on the right. Its always easiest to search by the class you're using, here it would be NSArrayController and NSTableView.

Armed with the trusty Apple Developers Documentation, most of the time you won't even need an internet connection to develop. Except when you click on an item in the XCode Documentation search and it turns that it doesn't exist locally... which may happen to you more often that you like.

Now we simply need to setup the table view to call open_ when double clicked. When developing in Cocoa, the best time to do one-time setup is in a class' awakeFromNib method. awakeFromNib is called when an object serialized in a nib is instantiated. Since we have an instance of MWController in our MainMenu.xib file (which is our application's main nib and thus loaded at launch), that instance will have its awakeFromNib function called as the application launches.

The implementation of the awakeFromNib method forMWController will look like this:

def awakeFromNib(self):
    if self.tableView:
        self.tableView.setTarget_(self)
        self.tableView.setDoubleAction_("open:")

You don't necessarily have to check that self.tableView has been initialized (it almost certainly has been), but as my uncle always says: live optimistically, but code cautiously¹.

Now save everything and Build and Go the application in XCode. Once it launches, search for something and double click on one of the rows. It should open up the article in your default web browser.

That was pretty easy, huh? Perhaps too easy, and you're wondering why I'm bothering to detail it here? Ahh, good question. This segment of the tutorial is trying to replicate the real application development cycle:

1. Have an idea.
2. Add it to your application.
3. Repeat.

Hopefully making it respond to double clicks--as well as the next two fairly quick examples--will give you a feel for the typical PyObjC workflow (the same as the three steps above, but with a Step 1.5: check the Apple Documentation to figure out the methods and classes you'll need).

Caching Results to Disk

Next we're going to add a simple caching layer for results from metaweb.py. It will have two fairly simple cases, which we'll now consider: handling a search with a cached result, and handling a search with an uncached result.

1. Handling a cached result.

   When the search button is clicked, we want to check a manifest of cached items (a dictionary with the search term being the key and a 2-tuple of a filename and a timestamp).

   For a cached result, the manifest will contain the key, and thus we'll be returned a 2-tuple. First, we'll verify that the timestamp isn't more than a day old. If it is that old, we'll discard the cached value and proceed as if there was no cached value.

   Otherwise, if it isn't older than a day, then we'll update the value of MWController's results field with the cached data, which we will load in from disk using the filename in the 2-tuple.

2. Handling an uncached result.

   When we search for an uncached result, we first verify it isn't in the cache, then retrieve it using metaweb.py, then we cache the newly retrieved result. Afterwards we treat it like a cached result.

In order to support those cases we need to create a persistent cache manifest that is saved to disk when the program closes and loaded from disk from the program is opened. Do you remember the applicationDidFinishLaunching_ and applicationWillTerminate_ methods stubs we implemented in MetaWindowAppDelegate? Well, we're finally going to use those.

A brief sidenote on serialization options in PyObjC.

Python's Pickle module is perhaps the easiest data serialization you'll ever encounter. Sure, you can do something vaguely similar in Cocoa by implementing the NSCoder class methods, but its substantially more code to get it working.

However, Pickle doesn't know how to serialize objects that subclass from NSObject. That means it can't serialize any of the Cocoa or Objective C classes. Thats inconvenient, but you can often work around it.

The easiest fix is to convert the object into a native Python class. For example, if you're dealing with an NSMutableDictionary, just convert it into a Python dict and serialize that (and convert it back into an NSMutableDictioanry when you deserialize it).

In the end, though, if you find yourself working with a particularly large or complex class that inherits from NSObject, you may want to go ahead and simply use NSCoder. Eventually you're tricky solution will become sufficiently complex to make it easier to use NSCoder anyway, so you might as well save your future self some time.

Keep in mind there are other solutions to persistent data like CoreData and SQLite which provide a non-traditional alternative to serialization.

Open up MetaWindowAppDelegate.py in XCode. At the top add an import for pickle and os.

import pickle,os
from Foundation import *
from AppKit import *

Now we're going to replace the stub applicationDidFinishLaunching_ method with this:

def applicationDidFinishLaunching_(self, sender):
    try:
        path = self.pathForFile('cache.serialized')
        file = open(path,'r')
        self.cache = pickle.load(file)
        file.close()
    except IOError:
        self.cache = {}

Now when MetaWindow launches it will attempt to load the cache.serialized file from the MetaWindow directory in the ~/Library/Application Support/ folder. If it exists, it will use the contents of that file as its cache, otherwise it will use an empty dictionary as the cache.

We'll also need to make sure that it saves self.cache when the application closes. To do that we modify the applicationWillTerminate_ method.

def applicationWillTerminate_(self,sender):
    path = self.pathForFile('cache.serialized')
    file = open(path,'w')
    pickle.dump(self.cache,file)
    file.close()

Now our cache will persist across application launches and closes. Save and close MetaWindowAppDelegate.py, and open up MWController.py.

The first thing we'll do in MWController, is create a property for retrieving the cache from the application delegate. Things at launch time can happen in a strange order, so its safer to wait until you need the cache to retrieve it from the app delegate instead of asking for it the moment the application starts up.

At the head of MWController.py we're going to add imports for pickle, md5 and datetime, and also want to import everything from AppKit, so it'll look like this:

import objc, metaweb, webbrowser, pickle, md5, datetime
from AppKit import *
from Foundation import *

Add the field _cache to MWController with the value None so it looks like this:

class MWController(NSObject):
    tableView = objc.IBOutlet()
    textField = objc.IBOutlet()
    arrayController = objc.IBOutlet()
    results = []
    _cache = None

Then we need to create the property by adding this code to MWController.

def getCache(self):
    if _cache is None:
        _cache = NSApp.delegate().cache
    return _cache
cache = property(getCache,None,None,"Cache of searches.")

This is our first time usingNSApp, which is a Cocoa shortcut, and refers to the current application. It is equivalent to NSApplication.sharedApplication() in Python or [NSApplication sharedApplication]; in Objective C.

The code snippet we just wrote creates a read only property named cache. Python properties are handy syntactical sugar for using accessors and mutators, and using one here will allow us to lazily retrieve the cache from the app delegate, without cluttering up usage.

A brief sidenote on mutators and accessors in Objective C. When you look at (or create) Objective C code you'll run into more mutators and accessors than you'll see in a lifetime of Python code. It can be an extremely handy pattern, especially in facilitating lazy initialization or dynamic values.

Beyond the general handiness of the pattern, it also occurs frequently in Objective C because an object can have a field and a method with the same name (i.e. fields and methods exist in different namespaces). This is a side effect of the Smalltalk inspired method calling syntax which makes it clear whether you are accessing a field or a method:

[myObject name];
myObject.name;

Unfortunately, that distinction has been somewhat obscured with the advent of Objective C 2.0. With ObjC 2.0, Apple has added properties in a fashion quite similar to Python's properties, and it is no longer possible to determine if the code myObject.name is referring to a field or is camouflague for an accessor/mutator pair.

Supporters of the new syntax argue that it should be an irrelevant distinction if you are properly encapsulating your classes and not relying on implementation details, but the old guard still mourns the loss of elegance, and to this day they keep the grave of Objective C 1.0 well adorned with flowers.

Or at least blog posts.

Now lets create a method for retrieving an object from the cache. If the cached data is too old it should pretend there isn't any stored data, to avoid the data getting stale.

def getCachedSearch(self,searchString):
    if self.cache.has_key(searchString):
        filename,timestamp = self.cache[searchString]
	age = datetime.datetime.now() - timestamp
        if age > datetime.timedelta(days=1):
            return None
        filepath = NSApp.delegate().pathForFile(filename)
        file = open(filepath,'r')
        data = pickle.load(file)
        file.close()
        return data
    return None

Thanks to the functionality in the datetime module its pretty simple to verify the age. Now we need to create a method for caching a new result.

def cacheResultsForSearch(self,searchString,results):
    filename = u"%s.cached" % md5.md5(searchString).hexdigest()[12:]
    filepath = NSApp.delegate().pathForFile(filename)
    file = open(filepath,'w')
    pickle.dump(results,file)
    file.close()
    self.cache[searchString] = (filename,datetime.datetime.now())

Notice that we are saving the results to disk in a separate file, and then storing the name of that file (and when it was created) in the caching manifest. We could simplify this and store the results directly in the cache, but then we'd need to keep all results stored in memory at all times.

The last step in modifying MetaWindow is to update the search_ method a bit.

@objc.IBAction
def search_(self,sender):
    search_value = self.textField.stringValue()
    cached = self.getCachedSearch(search_value)
    if cached is None:
        cached = metaweb.search(search_value)
        self.cacheResultsForSearch(search_value,cached)
    self.results = [ NSDictionary.dictionaryWithDictionary_(x) for x in cached]
    self.arrayController.rearrangeObjects()

Remember that we can't use pickle for Objective C objects, so we are being careful to cache the Python data, and then convert it into NSDictionary objects before using it in our tableview.

Now we have one final tidbit to deal with before we're done with caching.

As it stands, the cached files will never be deleted, even though we don't want to keep them around longer than a day. Lets head back to MetaWindowAppDelegate.py and improve upon the applicationWillTerminate_ method a bit.

First add an import for datetime at the head of the file.

import os,pickle,datetime
from Foundation import *
from AppKit import *

And then go down to the applicationWillTerminate_ method. We'll want to weed out all results which are more than one day old. Playing around at the interpreter we can figure out a solution pretty quickly.

>>> today = datetime.datetime.now()
>>> yesterday = today - datetime.timedelta(days=1)
>>> today < yesterday
False
>>> two_days_ago = datetime.datetime.now() - datetime.timedelta(days=2)
>>> two_days_ago < yesterday
True

So we'll change applicationWillTerminate_ to look like this:

def applicationWillTerminate_(self,sender):
    yesterday = datetime.datetime.now() - datetime.timedelta(days=1)
    for key in self.cache:
        filename,createdTime = self.cache[key]
        if createdTime < yesterday:
            filepath = self.pathForFile(filename)
            os.remove(filepath)
            del self.cache[key]
<span class="n">path</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">pathForFile</span><span class="p">(</span><span class="s">&#39;cache.serialized&#39;</span><span class="p">)</span>
<span class="nb">file</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="n">path</span><span class="p">,</span><span class="s">&#39;w&#39;</span><span class="p">)</span>
<span class="n">pickle</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">cache</span><span class="p">,</span><span class="nb">file</span><span class="p">)</span>
<span class="nb">file</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>

Now we're cleaning up after ourselves and MetaWindow is being a good citizen. Excellent.

Animating an NSProgressIndicator While Searching

The last thing we're going to take care of in this segment is to make the search experience a bit more responsive. As it stands, when you search for uncached results it can take a few seconds to complete, and the search button remains depressed the entire time, which makes it look a bit like the program is freezing.

Instead, it would be much nicer if there was some indication that progress is indeed being made.

Go ahead and add another IBOutlet to MWController, this one named indicator.

class MWController(NSObject):
    tableView = objc.IBOutlet()
    textField = objc.IBOutlet()
    arrayController = objc.IBOutlet()
    indicator = objc.IBOutlet()
    results = []
    _cache = None

And now open up MainMenu.xib in InterfaceBuilder. From the Library window find the Circular Progress Indicator and add it to the Window (Window) window. Resize the textfield so that the progress idicator can fit in the top left corner.

Then click on the Controller object in the MainMenu.xib (English) window, hold down control, and drag from Controller to the new circular progress indicator we just created. Release and then select indicator from the drop down menu to connect the IBOutlet.

Finally, save and go back to the MWController class. Now we're going to modify the search_ method a bit. Specifically, we're going to have it start the indicator's animation before we begin retrieving data from Metaweb, and stop it when the data is retrieved.

@objc.IBAction
def search_(self,sender):
    search_value = self.textField.stringValue()
    cached = self.getCachedSearch(search_value)
    if cached is None:
        self.indicator.startAnimation_(self)
        cached = metaweb.search(search_value)
        self.cacheResultsForSearch(search_value,cached)
        self.indicator.stopAnimation_(self)
    self.results = [ NSDictionary.dictionaryWithDictionary_(x) for x in cached]
    self.arrayController.rearrangeObjects()

Save, build and run the application. It should now animate the indicator while searching, and users will no longer be forced to ponder whether or not the application has frozen.

Wrapping Up Part Three

You can download the current zip of the project, or retrieve the code from the repository on GitHub.

With that we've come to the conclusion of the third segment of this tutorial. The forth--and final--segment will take a look at implementing some drag and drop functionality, as well as giving some advice on other resources to look into and thoughts about continuing to use and enjoy PyObjC.

The fourth segment is now completed, and you can continue on here.