Login

Posts about python

Get Argumentative with Javascript

When I looked at the Google Maps API for Javascript, I couldn't help but notice that all the functions in the API Reference look kinda Pythonic. Functions call be call with a positional argument list like normal, i.e.:
new GMap2(mynode, myopts)
or by name with an object literal:
new GMap2({node: mynode, opts: myopts})
Moreover, arguments can be optional, and they are denoted in the doc with a ? at the end. Finally, the type of each argument is specified.

Here is an example of their API:
GMap2(container:Node, opts?:GMapOptions)

I thought: How cool is this!!?? I want this!! But, as the Maps API is proprietary, I couldn't get hold of their source. So, it was time for a little hacking...

My immediate goals were: argument validation, named argument passing, and optional arguments. Now I have a prototype with these three things working.

Argument Validation
Argument validation in my terms means checking the input arguments to a function when it is called to see if they are valid. The most basic of these checks is whether the number of arguments is correct. Example, we start with a simple add function:
function add(one, other){
  return one + other;
}
Then we enhance it by wrapping in with the $f function:
add = $f(add);
The new enhanced add function now will require the argument list to match up with what is expected:
> add(1,1)
  2
> add(1)
  Error: Argument 'other' of function add(one, other) was not specified.
> add(1,2,3)
  Error: function add(one, other) expected 2 arguments but got 3: (1, 2, 3).

Named Argument Passing
Named argument passing is something that Python programmers enjoy, but in Javascript you can simulate it using map literals. For example, given the add function defined above, you can now also write:
> add({one: 1, other: 2});
  3
The order does not matter because it is a map, so the above is equivalent to:
> add({other: 2, one: 1});
  3
You can also mix positional and named arguments just like in Python:
> add(1, {other: 2});
  3

Optional Arguments
Optional arguments is another great Python feature. It is really the feature that makes named argument passing worthwhile. In Python we have a nice syntax to specify the default value for an optional argument:
def f(x=0):
  return x * 2
In Javascript, not so lucky. I had to resort to this:
f = $f(
  {x: 0},
  function f(x){
    return x * 2;
  }
)
Not so bad? Maybe? I am trying to make it look like a decorator/annotation. So, with this code, you can do:
> f()
  0
Just what you'd expect.

The combination of named argument passing and optional arguments enable very concise code in some situations. Example, let's say you have a function that generates an html input element given a bunch of parameters:
function input(type, name, value, classes){
  return '<input \
    type="' + type + '" \
    name="' + name + '" \
    value="' + value + '" \
    class="' + classes.join(' ') + '">';
}
In this case, type and name are required. Value is required by the spec but we can just default it to ''. Classes is optional and we'll just default it to [] to avoid a check for null.
So the code to do that is:
input = $f({value: '', classes: []}, input)
And now you call call the function in these variety of ways:
> input('text', 'age')
  <input     type="text"     name="age"     value=""     class="">
> input({type: 'text', name: 'age'})
  <input     type="text"     name="age"     value=""     class="">
> input({type: 'text', name: 'age', classes:['text', 'age']})
  <input     type="text"     name="age"     value=""     class="text age">
Nice!

The code for this prototype is as usual on github. The next on my todo list is to implement variable length argument list and extra keyword argument lists. So stay tuned.
Posted by Toby 8 months ago about javascript, programming and python (0 comments)

Prototype Inheritence in Python

One of Toby's favorite pastimes is to take concepts from one language and realize it in another. In this episode: Toby implements Javascript-style prototype inheritence in Python.

One thing that is super cool about Javascript is the fact that objects are just maps. Not only can you add attributes dynamically at any point, you can also add methods dynamically. In Python, you can add methods dynamically, but normally you'd have to add it into the class. But there are cases when you don't want to bother making a class. This inspired me to see how far I can go in making Python behave more like Javascript.

Introducing prototype.py
prototype.py lets you write code in Python with Javascript-style prototype inheritence semantics. If you are not familiar with prototype inheritence, just read on for now. I will link to some resources at the end. Now, let's see how to use this bad boy. First you import the module:
>>> from prototype import *
To create a constructor(we don't really have classes anymore), you write a function with the @constructor decorator:
>>> @constructor
... def Person(this, first, last):
...   this.firstName = first
...   this.lastName = last
...
>>> Person
<constructor 'Person'>
I am going to use this rather than self and camelCase rather than under_scores for prototype-style code, because, well...Dorothy, we are not in Kansas anymore.

Now you can do:
>>> bird = Person('Charlie', 'Parker')
>>> bird.firstName
'Charlie'
>>> bird.lastName
'Parker'
You can add attributes to the object just like in normal Python. But unlike in normal Python, which would raise an AttributeError when trying to access non-existent attributes, in prototype-land it merely returns None:
>>> print bird.age
None
You can dynamically add a method to the instance just by tagging on a function:
>>> def sing(this):
...   print '%s sings!!' % this.lastName
...
>>> bird.sing = sing
>>> bird.sing()
Parker sings!!
This affects only the bird instance. If you want the method to be added to all instances of Person however, you can add it to the prototype of Person.
>>> Person.prototype.sing = sing
>>> monk = Person('Thelonious', 'Monk')
>>> monk.sing()
Monk sings!!
This works because Person.prototype is the prototype of the monk instance. In code, this means:
>>> monk.__proto__ == Person.prototype
True
When the sing attribute is not found on the monk instance itself, it will follow the __proto__ reference and look it up in its prototype, which is also referred to as its parent. We can manipulate the parent link:
>>> monkJr = Person('T.S.', 'Monk')
>>> monkJr.__proto__ = monk
so that now monkJr inherits all of monk's attributes:
>>> monk.hair = 'black and curly'
>>> monkJr.hair
'black and curly'
The following are some other properties demonstrating the prototype inheritence model:
>>> assert monkJr.constructor == monk.constructor == Person
>>> assert Object.prototype.constructor == Object
>>> assert Person.prototype.constructor == Person
>>> assert Person.prototype.__proto__ == Object.prototype
If this seems confusing, remember that in prototype-land, there are two kinds of things: constructors and instances. The prototype of a constructor is nothing more than an instance of the type that the constructor represents. A new instance returned from calling a constructor will have its parent set to the prototype of the constructor.

Things You Can't Do in Javascript, i.e. the 1+1>2 Effect
Python is a more powerful and flexible language than Javascript, so, why am I dumbbing it down to Javascript's level? To show that I am not trying to do that at all, let me point out that there are at least a couple of really cool things you can do with prototype.py that you can't with Javascript, simply due to the fact that it is in Python.

The first thing is properties. With prototype.py, you can add properties to objects like so:
>>> def getName(this):
...   return '%s %s' % (this.firstName, this.lastName)
...
>>> Person.prototype.name = property(getName)
>>> bird.name
'Charlie Parker'
You can also specify setters and deleters in the way you'd expect:
>>> def setName(this, name):
...   first, last = name.split(' ')
...   this.firstName = first
...   this.lastName = last
...
>>> def deleteName(this):
...   del this.firstName
...   del this.lastName
...
>>> Person.prototype.name = property(getName, setName, deleteName)
>>> bird.name = 'Dizzy Gillespie'
>>> bird.name
'Dizzy Gillespie'
>>> del bird.name
>>> bird.name
'None None'

The second thing is named parameters, keyword parameters and optional parameters. Let's just do an example with keyword parameters. Let's say I just wanted a way to easily create ad-hoc objects as key-value pairs, I could write:
>>> @constructor
... def Data(this, **kwargs):
...   for key in kwargs.keys():
...     setattr(this, key, kwargs[key])
...
Which lets me write:
>>> project = Data(project='prototype.py', language='python')
But, I could also define methods and properties in this way:
>>> file = Data(fileName='prototype.py', fileExt=property(getExt), 
                    read=read, write=write)
Sweet!

A Note About the Implementation
The implementation of prototype.py is only about 60 lines of Python at time of writing, would have gone down to 40 without property support. The code can be found here. 

A big discovery that enabled me to do this was the new module. It allowed me to take a function and make it into an instance method by binding it to an object:
method = new.instancemethod(function, object)

Another note is that I made the design decision that for methods, I chose to store the unbound function rather than the bound method in the __dict__ of the object. I would then bind the function on the fly when it's asked for. This made it very easily to inherit methods because you don't have to worry that the method is really bound to the parent rather than the instance in question.

Further Reading About Prototype Inheritence
If you want to learn more about prototype inheritence, try:
Posted by Toby 9 months ago about javascript, programming, prototype and python (2 comments)

Null in 10 Languages

  • Java, Javascript, C#: null
  • Ruby, Smalltalk, Lisp, OCaml: nil
  • Lisp: ()
  • Python: None
  • Haskell: Nothing
  • C: 0
Posted by Toby 10 months ago about c, c#, haskell, java, javascript, lisp, ocaml, programming, python, ruby and smalltalk (0 comments)

Parameterizing Your Tests

I use Selenium at work to write browser tests. I use the Python bindings with Selenium Remote Control to drive the tests. Now, once you have written a test suite, it is not unreasonable to want to run the same set of tests over different browsers. Now that the browser space is getting interesting again, you'd better test your app on 5 or 6 browsers. So how do I write my tests once, and run them on different browsers? In the Python unittest framework, tests are written as classes that extend unittest.TestCase. So, to parameterize them, you can can subclass the test dynamically and set the browser attribute onto it:
import unittest

class TestPages(unittest.TestCase):
    def testPage(self):
        print("testing page on %s" % self.browser)

TestPagesOnFirefox = type('TestPagesOnFirefox', 
                                 (TestPages,), dict(browser='firefox'))

if __name__ == '__main__':
    unittest.main()
So, we are subclassing TestPages calling it TestPagesOnFirefox and at the same time setting its browser attribute to 'firefox'.
We run it and get this error:
AttributeError: 'TestPages' object has no attribute 'browser'
This is because TestPages does not have the browser property defined. What do we do? Let's erase it:
del TestPages
After that it runs okay and we get the output we want:
testing page on firefox
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK
So we can generalize that and expand all the browsers you want to. We write an expand_browsers function like so:
def expand_browsers(gls, browsers=['firefox', 'ie6', 'ie7', 'safari', 'chrome', 'opera']):
    original_test_classes = filter(
        lambda o: isinstance(o, type) and issubclass(o, unittest.TestCase), 
        gls.values())
    
    for cls in original_test_classes:
        for browser in browsers:
            new_class = type('%sOn%s' % (cls.__name__, browser), (cls,), dict(browser=browser))
            gls[new_class.__name__] = new_class
        del gls[cls.__name__]
After you declare all you tests you call the function like so:
expand_browsers(globals())
This goes through the namespace of your module and for each class that is a subclass of unittest.TestCase(You can choose a different subclass as marker here if you want), it subclasses it and specifies the browser property in it. After that, it erases the original testcase class so that it can't be found, like we did above.
Final complete example:
import unittest

class TestPages(unittest.TestCase):
    def testPage(self):
        print("testing page on %s" % self.browser)

class TestMorePages(unittest.TestCase):
    def testPageTwo(self):
        print("testing page two on %s" % self.browser)

def expand_browsers(gls, browsers=['firefox', 'ie6', 'ie7', 'safari', 'chrome', 'opera']):
    original_test_classes = filter(
        lambda o: isinstance(o, type) and issubclass(o, unittest.TestCase), 
        gls.values())
    
    for cls in original_test_classes:
        for browser in browsers:
            new_class = type('%sOn%s' % (cls.__name__, browser), (cls,), dict(browser=browser))
            gls[new_class.__name__] = new_class
        del gls[cls.__name__]

expand_browsers(globals())

if __name__ == '__main__':
    unittest.main()
Output:
.testing page two on firefox
.testing page two on ie6
.testing page two on ie7
.testing page two on opera
.testing page two on safari
.testing page on chrome
.testing page on firefox
.testing page on ie6
.testing page on ie7
.testing page on opera
.testing page on safari
.
----------------------------------------------------------------------
Ran 12 tests in 0.000s

OK

Code examples also on github.
Posted by Toby 12 months ago about programming, python and testing (0 comments)

goodday dot py

Looking at rails and the datejs library, I can't help but feel bummed that Python doesn't have an expressive date library like that. So I started one. I've put up my code on google code. 
Posted by Toby about 1 year ago about dates, programming and python (2 comments)

Auto-mixin in Python

Python has multiple inheritence - a feature that's really handy because it allows the programing idiom called mixins - where you write a a bunch of methods on a "mixin" class and then later on attaching all of those methods onto another class simply by mixin-it-in. In Python, the process of mixin-it-in is simply to add it to the parent list of the class:
class MyObject(MyParent, MyMixin):
   ...
In Ruby, you can automatically mix in a mixin programmatically because the mix-it-in is done within the class definition, and because of its open classes. This is how the rails helpers are automatically found in a directory and mixed into the controllers for example.
How would you do this in Python? Let's say we simplify the problem to this: For a given Controller subclass named ProductController, for example, if there exists a helper class by naming convention: ProductControllerHelper, then we mix it in. Here's my solution:
class Meta(type):
    def __new__(cls, name, bases, dct):
        helper_mixin = globals().get('%sHelper' % name)
        if helper_mixin:
            bases = list(bases)
            bases.append(helper_mixin)
            return type.__new__(cls, name, tuple(bases), dct)
        else:
            return type.__new__(cls, name, bases, dct)
        
class Controller(object):
    __metaclass__ = Meta
        
class ProductControllerHelper(object):
    def method1(self):
        print "method1 invoked"
        
class ProductController(Controller):
    def index(self):
        self.method1()
        
        
if __name__ == '__main__':
    ProductController().index()
Output:
method1 invoked
It worked!

But! We are not done. What if what you want to mix in depends on how you configure the class inside the class definition? In my case, I wanted to mix in a bunch of classes when I write a DSL-like declaration like this:
class MyTestCase(TestCase):
    all = fixture(SeleniumRCServer, BrowserSession, DBData, Login)
    each = fixture(AddDelivery)
    ...
This is for a testing framework I am working on. I have a test case, and I want to specify its test context(fixtures) in the manner shown. Each object in the fixture(..) definition is a mixin. So I want to mix them all into MyTestCase.
This case is harder than the previous case because when __new__ is invoked, the class definition hasn't been processed yet, and therefore you won't have access to the class attributes all and each. The hack I came up with is this:
class TestCaseMetaClass(type):
    def __init__(cls, name, bases, dct):
        if len(bases) == 1 and bases[0] != unittest.TestCase:
            bases = list(bases)
            bases.extend(cls.all)
            bases.extend(cls.each)
            cls.__real__ = type(name, tuple(bases), dct)
        else:
            type.__init__(name, bases, dct)
        
    def __call__(cls, *params, **kws):
        if hasattr(cls, '__real__'):
            return cls.__real__(*params, **kws)
        else:
            return type.__call__(cls, *params, **kws)
            
class TestCase(unittest.TestCase):
    __metaclass__ = TestCaseMetaClass
Basically, I first override __init__ of the metaclass(rather than __new__) in order to record what was declared in all and each, and create a new class on the fly mixing in all the mixins defined in the fixture(..) definitions. We store that class in an attribute __real__ attached to the class. Then we override __call__ for the metaclass to return an instance of the class held in the __real__ attribute instead of the actual class. Oh yes, this is really ugly, but it worked, at least for what I was doing. If you can write to a classes' __bases__ attribute(which defines the parents of the class), then this would be much easier, but I haven't figured out how to do it if it can be done at all. But because of this limitation, it looks unlikely you can do something like Ruby's mixology in Python.

Update: the better solution to the second part is found:
class Meta2(type):
    def __new__(cls, name, bases, dct):
        mixins = []
        all = dct.get('all')
        if all:
            mixins.extend(all)
        each = dct.get('each')
        if each:
            mixins.extend(each)
        bases = list(bases)
        bases.extend(mixins)
        return type.__new__(cls, name, tuple(bases), dct)
        
class TestCase(object):
    __metaclass__ = Meta2
    
class Fixture1(object):
    def method2(self):
        print "method2 invoked"
    
class Fixture2(object):
    def method3(self):
        print "method3 invoked"
        
class BlahBlahTest(TestCase):
    all = [Fixture1]
    each = [Fixture2]
    def blah_test(self):
        self.method2()
        self.method3()
Posted by Toby about 1 year ago about metaprogramming, mixin, programming, python and ruby (0 comments)

REST stuff in GAE

I've been working on getting justtodolist.com to use REST style URLs. It feels like I increasingly post my technical stuff on stackoverflow.com though, as was the case this time. So I'll just post a link to my stackoverflow question.
Posted by Toby about 1 year ago about gae, programming and python (0 comments)

Python Optional Arguments Gotcha

When you use Python optional arguments you should know that the default value you set is static, it is set at the time the function is defined, and does not change after that. If you a mutable type as the default value, you need to be careful.
Ex:

>>> def enlist(n, lst=[]):
...   lst.append(n)
...   return lst
...
>>> enlist(3)
[3]
>>> enlist(4)
[3, 4]
>>> enlist(5)
[3, 4, 5]
>>> enlist(6)
[3, 4, 5, 6]

Everytime you call the enlist function, the list gets bigger, because lst is never reset to the empty list. It is set to the empty list only once, when enlist was defined. After that it references the same instance everytime you don't supply a lst argument.
Posted by Toby about 1 year ago about gotcha, programming and python (0 comments)

Selenium-RC Proxy Server War Story

I am trying out selenium yet again to help our non-existent QA team. I have to admit that I kinda flaked out on unit testing too(I kinda got sick of TDD in rails bogging me down to be honest), which I mean to catch up on, but, for now, I wanted to focus on selenium, since, with an ajax app like ours, too many things could go wrong, it could be either client side or server side or a combination, so intergration testing I think can really buy us a lot. Besides, I really wanted to know if selenium works well or not.

Anyway, this is not a post about selenium, exactly, I'll probably dedicate another to it later, when I have had sufficient experience with it. The problem I faced in this episode is that when I run the app through the selenium-rc proxy server, my flash messages aren't showing up! Yes, the proxy server is muffling my flash messages, what could it be? This first thought was it had to be a cookie issue since the flash message is implemented as a one-time use cookie, but the other cookies worked fine, or I wouldn't have been able to login to the app.

From looking at the cherrypy code, it looks like it's setting 2 cookies, one for auth, the other for the flash message, and it generates 2 Set-Cookie headers. I suspected that this was screwing up the selenium proxy server. I decided to dig into the selenium server code and put in some debug statements.

Aww! Getting the selenium-rc source and then building it with maven 2 is back to the painful slow Java days. Man! Maven, do you really have to download the entire internet just to build the project? Maven just symbolizes all that I hate about Java. It's bloated; it's framework heavy; it forces things on you that you don't need; running a build with it is glacial; build plugins is a big hassel, did I leave out anything? I think Maven may be the worst thing that has happened to the Java community...but I digress.

Basically I tracked it down to the part where the proxy server gets the header fields from the HttpURLConnection object(part of the standard Java API), but it drop the flash message cookie somehow. I suspected it was because that API just doesn't cope with duplicate header names in the response, but that seems strange that this has never come up. Googling found that people have been able to get dupliately named headers using the getHeaderFieldKey(int n) and getHeaderField(int n) methods, which take in a positional parameter. This is what the proxy server was doing, but yet it didn't work.

I used wireshark to look at the packets to confirm my theory, it did, but I also found another interesting thing - there is an empty line between the first Set-Cookie header and the second. I came to me that this is probably what tripped up the HttpURLConnection code, and I was right. I changed the Cookie code in the standard library to use \n instead of \r\n(so it interpreted as an empty line) as the separator and it solved the problem.

But I don't want to just patch a standard python library like that, it's not deployable. Don't know whether I should fix this in selenium-rc or not, in which case someone suggested I use the apache httpclient in place of HttpURLConnection.

Update: It turns out you can use the same trick as I displayed here to patch Python libraries, which is what I did:
    import Cookie
    def make_myoutput():
        org = Cookie.BaseCookie.output
        def myoutput(self, attrs=None, header='Set-Cookie: ', sep='\n'):
            return org(self, attrs, header, sep=sep)
        return myoutput

    Cookie.BaseCookie.output = make_myoutput()
Posted by Toby about 1 year ago about java, maven, programming, python and selenium (0 comments)

Web Dev with Google App Engine

I wrote my first Google App Engine app! It's located at justtodolist.appspot.com. It's yet another todo list - I have been a tadalist user for a while and thought I could make it slightly better, and so I did. Here is to jot down some thoughts on GAE.

First up, the things I like.
  1. The number one benefit of GAE for me is definitely one step deployment. No that you couldn't step up one step deployment for rails apps, but it just takes a lot of work to set this stuff up the first time. With GAE, it's one step deploy the first time. I would say it's easier even than php. A large part because of benefit number two
  2. there is no database to set up. As most people know by now, you use GQL/Big Table on GAE, and it is very different from relational databases. Setting up is really minimal. You specify your model in a DSL similar to what you'd write with SQLObject or Elixir, or Data Mapper for Ruby folks. And then boom! you are running.
  3. There's no user authentication to setup either, it's basically Gmail authentication, if you are willing to go along, that is. The User API is very simple, and you can start using it right away
  4. Development server is nice, it picks up changes immediately when you save any project file
  5. The number of projects files you have is very minimal, it's very non-cluttery.
So as you can see GAE is great for rapid prototyping. Now for things I am not that crazy about. Actually, most of the benefits I spoke of has some caveats:
  1. Although deployment is easy. Sometimes issues arise from the fact that things work slightly differently in production vs development. Such as, you need indices to build fully in production for the app to be ready to run, or for some reason, transaction rules work differently in production vs development(I haven't dug down to this fully, but it might be a bug)
  2. Big Table is cool, it's supposed to be super scalable, but there are a couple of things that are annoying about it. I can get over the fact that it's fundamentally different from relational databases: things like you can't do aggregate queries, joins and so forth. For performance too, some things you will just have to do differently than you would normally with relation databases. I am okay with that. What pains me is that there is no proper data migration path. When you change your models(add or remove fields, and so forth), the old stuff just stick around. To migrate the old data, you basically have to manually write a script that loops over the existing data structures and modify them, but the script has to be triggered from an http request just like everything else because that's the only way to run your code on GAE on the production server... getto! Also, I am extremely annoyed that while in development you can just clear your datastore. There is no analog in production. I realize though that this is a work in progress and that things will get better in the future.
  3. Well, the caveat for using Gmail authentication is that... your users must have a Gmail account, duh... I am sure you can use other authentication schemes if you want, I don't see anything preventing that
  4. Yes the development server is nice, but for some reason it was progressively running slower on my company Dell D820 laptop. This is so especially if you perform extensive modification of the data models.
Here are some other thoughs:
  1. The development console is not good, man! It's no where near the usability of the python interactive shell. I've heard of an alternative but haven't seen it yet.
  2. I haven't dug into how to do TDD with it yet, but I've read it's possible.
  3. Again, Big Table is VERY different from relational databases. I originally ported my todo list app from a SQLObject -> MySQL backend, with only 2 data models: TodoList and TodoItem. In Big Table we still have the 2, but they look kinda different. I had to change it because of performance reasons. I'll put the discussion of that on a separate post.
  4. I haven't got a great handle of how transaction/entity groups work. I thought I did, until my transaction code didn't work, will have to look closer into it. Documentation on this is kinda sparse. Right now my code is non-transactional.
Open Source GAE Apps?

Here's another thought. Will it be possible to popularize open source GAE apps? I think the most important reason for the popularity of open source php apps is the ease of deployment. With the ease of deployment of GAE, I think conditions might be ripe for there to emerge a movement of good open source GAE apps. Then again though, people might resist the vendor specific/non-open source nature of GAE. We will see.

Posted by Toby about 1 year ago about appengine, google, programming and python (0 comments)

Impressions of Turbogears 4 months in

I've been a user of Turbogears for 4 months now, working on a client facing app. The app has not gone production yet, so I don't have much insight on deployment, but I have a lot of experience on the development side - I started from scratch - and here is what I learned so far.

Freedom of Choice
Turbogears as a framework is pretty agnostic of different components such as ORM, or template engine. Although there is a default choice, I found it wasn't hard to stray away from it.

Mako
I ended up choosing Mako as the template engine because, coming from rails, I felt kid and genshi were too heavyweight for my taste since they are based on XSLT and requires your markup to be valid XML before it can do anything, which obviously means there's an XML parsing step it has to do. Mako is more like erb in that it's "text-based", e.i. it's perfectly fine to render non-valid XML code. But Mako turned out to be much more than another erb. With Mako you can easily write helper template functions and reuse them everywhere. You can also write inversion-of-control template style functions which take in a partial template and calls it inside its body. It always puzzled me why you couldn't do that with erb or haml or most of the ruby template engines easily. With erb, you have to write a partial view as a separate file, but calling a partial with local parameters is inconvient, you have to write something like:

render :partial => 'my_control', :locals => {:control_id => 'con', 
    :height=> '50px'}

and since this is so inconvient, i usually end up wrapping it with a helper method like:

def my_control(control_id, height)
    render :partial => 'my_control', :locals => {
        :control_id => control_id, :height=> height}
end

Of course this is partially due to the culture in rails that you normally write helpers in ruby rather than in a template language. In Mako you don't have to do this extra step, which makes me happy. Now if you want to do the inversion of control thing, in erb it's even worse! In mako you would just do this. So, in general, I am able to refactor my views a lot easier with mako and therefore I find myself doing it a lot more often.

SQLAlchemy
SQLAlchemy is a main stream ORM in the python community. It's direction is different from that of ActiveRecord and is a lot more similar to Hibernate of Java but also has similarities to Ambition of Ruby and Linq of .NET. It is similar to Hibernate in that it is very fully featured, has sessional transaction management, and can coupe with a large variety of schemas. It is similar to ambition and linq in that you can contruct queries in your host language in a very succint and elegant way(I know you can build queries in Hibernate's criteria API too, but it's not quite elegant). I like SQLAlchemy a lot! Here's a couple of sqlalchemy tricks I like. First one:

    fields = [
        User.user_name,
        User.display_name,
        User.email_address
    ]
    results = User.query.filter(or_(*[f.ilike('%' + q + '%') 
        for f in fields]))

The above code does a wildcard partial string match of the string q against any of the three fields listed in the fields list. Second example:

    page = User.query[10:20]
           
This looks like array slicing, but no, it's slicing against the query results! It's smart enough to build the query using OFFSET and LIMIT or equivalent. You can easily do pagination with this technique.

Python's Named Parameters
Another thing I like when working with turbogears is python's named parameters. Whereas rubists use the hash as the poor man's name parameters, python has real named parameters, which is not only safer, but more elegant.

Python's Polluted Name Space
I run into this problem once in a while, but I hit on it 2 or 3 times in the last week! In python, list, dict, str, int, etc. are the names of fundamental types in the language, therefore you can't(actually you can, but don't want to) use them as variable names. More than once I've tried to use list as a variable name, which python doesn't complain about immediately but causes a cryptic error down the road.
I've also tried use from as a variable name, which turns out to be a keyword in the language, this causes a syntax error, which you don't see until you realize it's a keyword. Now, of course, more languages suffer from this problem, ruby isn't any different, but I think ruby has better error messages for these syntax errors: it will tell you the symbol that is unexpected and what symbols it was expecting.

Little Verbosity
Turbogears is more verbose than rails most prominently in 2 areas: import statements and method decorators. Rails files, usually have at most 2 requires(counterpart of import in python), most of the time none at all. My TG controller files and the model.py(the file with all the model objects) usually have about 10 to 15 lines of includes, my Mako template files usually 2 to 4 lines of includes. This has a lot to do with the design of the language. The python interpreter requires each .py file to act like a module, and as a module, it must identify all of its dependences, the ruby interpreter does not require this and so your controller code, for example, doesn't need to explicitly require anything before it has what it needs to do its work.
I think method decorators are cool, but they can also be overused and become cluttery. Some of my controller methods have more than 4 or 5 lines of decorators, consisting of the mandatory expose(), access restriction spec, and form parameter validators. That's a bit much. I also don't like the fact that you need an expose() decorator for every single controller method. Rails has no such thing and usually specifies such things at the top of the class, which has pros and cons vs TG's approach, but is at the end less cluttery.

Posted by Toby about 1 year ago about programming, python, ruby and turbogears (0 comments)

Parameter List Chaining in Python

Python has great support for variable length parameter lists, you have optional parameters:

>>> def o(opt=1):
...   print opt
...
>>> o()
1
>>> o(2)
2
>>> o(opt=3)
3


then you have the one star for any number of parameters:

>>> def f(*params):
...   for p in params: # params is a list
...     print p
...
>>> f(1)
1
>>> f(1,2,3,4)
1
2
3
4

and you got two stars for keyword parameters:

>>> def g(**kws):
...   for item in kws.items(): # kws is a dict
...     print item
...
>>> g(one=1)
('one', 1)
>>> g(one=1, two=2, three=3)
('three', 3)
('two', 2)
('one', 1)

You can also use them together along with regular parameters and optional parameters as long as they follow the ordering: regular > optional > single star(varargs) > double star(keyword args):

>>> def h(req, opt=None, *params, **kws):
...   print 'req=', req
...   print 'opt=', opt
...   print 'params=', params
...   print 'kws=', kws
...
>>> h(1,2,3,4)
req= 1
opt= 2
params= (3, 4)
kws= {}
>>> h(1,two=2,three=3,four=4)
req= 1
opt= None
params= ()
kws= {'four': 4, 'two': 2, 'three': 3}
>>> h(1,two=2,three=3,opt=4)
req= 1
opt= 4
params= ()
kws= {'two': 2, 'three': 3}

If you have a handle on a list or a dict object, you can use it/them directly as the parameter list when you call a function:

>>> lst = [1,2,3]
>>> f(*lst)
1
2
3
>>> dct = dict(one=1,two=2,three=3)
>>> g(**dct)
('one', 1)
('three', 3)
('two', 2)

A techique I developed is to chain function calls lazily, sort of a poor man's partial function application. Say I have a function that takes a lot of optional parameters, let's say it generates a text field of some sort:

def textfield(label, name, size=10, classname='', id=None, 
    onclick=None, onkeypress=None):
    # blah

Now for my specific screen I want to specify a number of these parameters by default, and only vary the label and name parameters the rest of the way, so I do something like:

def tf(*params, **kws):
    textfield(size=20, classname='cool-field', *params, **kws)

now I can call it like this:
tf('Name', 'name')
tf('Birthdate', 'birthdate', onkeypress="checkBirthdate(this)")

The cool part about this is that I can add new optional parameters to textfield without having to change tf and yet still be able to use the new parameters from tf. I could have just as well use the partial function from the functional library, but this is pretty easy too.
Posted by Toby about 1 year ago about programming and python (0 comments)