Posts about selenium

I have been on and off the TDD bandwagon and now I understand why. In order to have an enjoyable TDD experience, your tests must 1) be easy to write, 2) easy to debug and 3) finish running in seconds. Easy to write: that's mostly a process of structuring your code to be testable, something that is - although not always easy - at least straightforward. Easy to debug: very much depends on the language and framework(s) you are using. But if your tests run long, you starting waiting around for the tests to finish, and before you know it you are checking twitter and listening to podcasts(from Giles Bowkett). Your productivity go way down, and you get miserable - that is, if you care about your productivity. The hardcore TDD guys are big into using mocks, and the reason they give is the running time of the tests. Because with mocks they don't have to actually run any significant portion of code - the code is very hollow - the tests can run really fast. But, although I like mocks, they don't work everywhere. One example is Selenium. You may say, Selenium is not for TDD! It's for functional testing. Yeah, but why not? Selenium is starting to work well for me. It runs your tests against real browsers, and it can run your tests across different browsers automatically. Since, we have a lot of javascript in our app, I don't think any test suite that doesn't actually test your js against the actual browser gives you any real assurance. The problem with selenium, of course, is that it runs slow. Thus, my challenge, shall be to make my selenium tests run fast enough that I'd no longer have to wait around for my tests to finish(probably under 10 seconds). The first strategy I am going to use is to reuse test fixtures across tests in a fine grain way, so we can save much of the time used for setUp/tearDown.

I am trying out selenium yet again to help our non-existent QA team. I have to admit that I kinda flaked out on unit testing too(I kinda got sick of TDD in rails bogging me down to be honest), which I mean to catch up on, but, for now, I wanted to focus on selenium, since, with an ajax app like ours, too many things could go wrong, it could be either client side or server side or a combination, so intergration testing I think can really buy us a lot. Besides, I really wanted to know if selenium works well or not.

Anyway, this is not a post about selenium, exactly, I'll probably dedicate another to it later, when I have had sufficient experience with it. The problem I faced in this episode is that when I run the app through the selenium-rc proxy server, my flash messages aren't showing up! Yes, the proxy server is muffling my flash messages, what could it be? This first thought was it had to be a cookie issue since the flash message is implemented as a one-time use cookie, but the other cookies worked fine, or I wouldn't have been able to login to the app.

From looking at the cherrypy code, it looks like it's setting 2 cookies, one for auth, the other for the flash message, and it generates 2 Set-Cookie headers. I suspected that this was screwing up the selenium proxy server. I decided to dig into the selenium server code and put in some debug statements.

Aww! Getting the selenium-rc source and then building it with maven 2 is back to the painful slow Java days. Man! Maven, do you really have to download the entire internet just to build the project? Maven just symbolizes all that I hate about Java. It's bloated; it's framework heavy; it forces things on you that you don't need; running a build with it is glacial; build plugins is a big hassel, did I leave out anything? I think Maven may be the worst thing that has happened to the Java community...but I digress.

Basically I tracked it down to the part where the proxy server gets the header fields from the HttpURLConnection object(part of the standard Java API), but it drop the flash message cookie somehow. I suspected it was because that API just doesn't cope with duplicate header names in the response, but that seems strange that this has never come up. Googling found that people have been able to get dupliately named headers using the getHeaderFieldKey(int n) and getHeaderField(int n) methods, which take in a positional parameter. This is what the proxy server was doing, but yet it didn't work.

I used wireshark to look at the packets to confirm my theory, it did, but I also found another interesting thing - there is an empty line between the first Set-Cookie header and the second. I came to me that this is probably what tripped up the HttpURLConnection code, and I was right. I changed the Cookie code in the standard library to use \n instead of \r\n(so it interpreted as an empty line) as the separator and it solved the problem.

But I don't want to just patch a standard python library like that, it's not deployable. Don't know whether I should fix this in selenium-rc or not, in which case someone suggested I use the apache httpclient in place of HttpURLConnection.

Update: It turns out you can use the same trick as I displayed here to patch Python libraries, which is what I did:

    import Cookie

    def make_myoutput():

        org = Cookie.BaseCookie.output

        def myoutput(self, attrs=None, header='Set-Cookie: ', sep='\n'):

            return org(self, attrs, header, sep=sep)

        return myoutput

    Cookie.BaseCookie.output = make_myoutput()