Parameterizing Your Tests

I use Selenium at work to write browser tests. I use the Python bindings with Selenium Remote Control to drive the tests. Now, once you have written a test suite, it is not unreasonable to want to run the same set of tests over different browsers. Now that the browser space is getting interesting again, you'd better test your app on 5 or 6 browsers. So how do I write my tests once, and run them on different browsers? In the Python unittest framework, tests are written as classes that extend unittest.TestCase. So, to parameterize them, you can can subclass the test dynamically and set the browser attribute onto it:

import unittest

class TestPages(unittest.TestCase):
    def testPage(self):
        print("testing page on %s" % self.browser)

TestPagesOnFirefox = type('TestPagesOnFirefox', 
                                 (TestPages,), dict(browser='firefox'))

if __name__ == '__main__':
    unittest.main()

So, we are subclassing TestPages calling it TestPagesOnFirefox and at the same time setting its browser attribute to 'firefox'.

AttributeError: 'TestPages' object has no attribute 'browser'

This is because TestPages does not have the browser property defined. What do we do? Let's erase it:

del TestPages

After that it runs okay and we get the output we want:

testing page on firefox
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

So we can generalize that and expand all the browsers you want to. We write an expand_browsers function like so:

def expand_browsers(gls, browsers=['firefox', 'ie6', 'ie7', 'safari', 'chrome', 'opera']):
    original_test_classes = filter(
        lambda o: isinstance(o, type) and issubclass(o, unittest.TestCase), 
        gls.values())
    
    for cls in original_test_classes:
        for browser in browsers:
            new_class = type('%sOn%s' % (cls.__name__, browser), (cls,), dict(browser=browser))
            gls[new_class.__name__] = new_class
        del gls[cls.__name__]

After you declare all you tests you call the function like so:

expand_browsers(globals())

This goes through the namespace of your module and for each class that is a subclass of unittest.TestCase(You can choose a different subclass as marker here if you want), it subclasses it and specifies the browser property in it. After that, it erases the original testcase class so that it can't be found, like we did above.

Final complete example:

import unittest

class TestPages(unittest.TestCase):
    def testPage(self):
        print("testing page on %s" % self.browser)

class TestMorePages(unittest.TestCase):
    def testPageTwo(self):
        print("testing page two on %s" % self.browser)

def expand_browsers(gls, browsers=['firefox', 'ie6', 'ie7', 'safari', 'chrome', 'opera']):
    original_test_classes = filter(
        lambda o: isinstance(o, type) and issubclass(o, unittest.TestCase), 
        gls.values())
    
    for cls in original_test_classes:
        for browser in browsers:
            new_class = type('%sOn%s' % (cls.__name__, browser), (cls,), dict(browser=browser))
            gls[new_class.__name__] = new_class
        del gls[cls.__name__]

expand_browsers(globals())

if __name__ == '__main__':
    unittest.main()

Output:

.testing page two on firefox
.testing page two on ie6
.testing page two on ie7
.testing page two on opera
.testing page two on safari
.testing page on chrome
.testing page on firefox
.testing page on ie6
.testing page on ie7
.testing page on opera
.testing page on safari
.
----------------------------------------------------------------------
Ran 12 tests in 0.000s

OK

Code examples also on github.