I've been a user of Turbogears for 4 months now, working on a client facing app. The app has not gone production yet, so I don't have much insight on deployment, but I have a lot of experience on the development side - I started from scratch - and here is what I learned so far.

Freedom of Choice
Turbogears as a framework is pretty agnostic of different components such as ORM, or template engine. Although there is a default choice, I found it wasn't hard to stray away from it.

Mako
I ended up choosing Mako as the template engine because, coming from rails, I felt kid and genshi were too heavyweight for my taste since they are based on XSLT and requires your markup to be valid XML before it can do anything, which obviously means there's an XML parsing step it has to do. Mako is more like erb in that it's "text-based", e.i. it's perfectly fine to render non-valid XML code. But Mako turned out to be much more than another erb. With Mako you can easily write helper template functions and reuse them everywhere. You can also write inversion-of-control template style functions which take in a partial template and calls it inside its body. It always puzzled me why you couldn't do that with erb or haml or most of the ruby template engines easily. With erb, you have to write a partial view as a separate file, but calling a partial with local parameters is inconvient, you have to write something like:

render :partial => 'my_control', :locals => {:control_id => 'con', :height=> '50px'}

and since this is so inconvient, i usually end up wrapping it with a helper method like:

def my_control(control_id, height)
    render :partial => 'my_control', :locals => {:control_id => control_id, :height=> height}
end

Of course this is partially due to the culture in rails that you normally write helpers in ruby rather than in a template language. In Mako you don't have to do this extra step, which makes me happy. Now if you want to do the inversion of control thing, in erb it's even worse! In mako you would just do this. So, in general, I am able to refactor my views a lot easier with mako and therefore I find myself doing it a lot more often.

SQLAlchemy
SQLAlchemy is a main stream ORM in the python community. It's direction is different from that of ActiveRecord and is a lot more similar to Hibernate of Java but also has similarities to Ambition of Ruby and Linq of .NET. It is similar to Hibernate in that it is very fully featured, has sessional transaction management, and can coupe with a large variety of schemas. It is similar to ambition and linq in that you can contruct queries in your host language in a very succint and elegant way(I know you can build queries in Hibernate's criteria API too, but it's not quite elegant). I like SQLAlchemy a lot! Here's a couple of sqlalchemy tricks I like. First one:

    fields = [
        User.user_name,
        User.display_name,
        User.email_address
    ]
    results = User.query.filter(or_(*[f.ilike('%' + q + '%') for f in fields]))

The above code does a wildcard partial string match of the string q against any of the three fields listed in the fields list. Second example:

    page = User.query[10:20]
           
This looks like array slicing, but no, it's slicing against the query results! It's smart enough to build the query using OFFSET and LIMIT or equivalent. You can easily do pagination with this technique.

Python's Named Parameters
Another thing I like when working with turbogears is python's named parameters. Whereas rubists use the hash as the poor man's name parameters, python has real named parameters, which is not only safer, but more elegant.

Python's Polluted Name Space
I run into this problem once in a while, but I hit on it 2 or 3 times in the last week! In python, list, dict, str, int, etc. are the names of fundamental types in the language, therefore you can't(actually you can, but don't want to) use them as variable names. More than once I've tried to use list as a variable name, which python doesn't complain about immediately but causes a cryptic error down the road.
I've also tried use from as a variable name, which turns out to be a keyword in the language, this causes a syntax error, which you don't see until you realize it's a keyword. Now, of course, more languages suffer from this problem, ruby isn't any different, but I think ruby has better error messages for these syntax errors: it will tell you the symbol that is unexpected and what symbols it was expecting.

Little Verbosity
Turbogears is more verbose than rails most prominently in 2 areas: import statements and method decorators. Rails files, usually have at most 2 requires(counterpart of import in python), most of the time none at all. My TG controller files and the model.py(the file with all the model objects) usually have about 10 to 15 lines of includes, my Mako template files usually 2 to 4 lines of includes. This has a lot to do with the design of the language. The python interpreter requires each .py file to act like a module, and as a module, it must identify all of its dependences, the ruby interpreter does not require this and so your controller code, for example, doesn't need to explicitly require anything before it has what it needs to do its work.
I think method decorators are cool, but they can also be overused and become cluttery. Some of my controller methods have more than 4 or 5 lines of decorators, consisting of the mandatory expose(), access restriction spec, and form parameter validators. That's a bit much. I also don't like the fact that you need an expose() decorator for every single controller method. Rails has no such thing and usually specifies such things at the top of the class, which has pros and cons vs TG's approach, but is at the end less cluttery.

In Javascript:
myfunction.toString()
This gives you the entire definition of the function as a string, with which you can parse to get the argument names, or in prototype.js you can just do:
myfunction.argumentNames()

In Python:
myfunction.func_code.co_varnames

In Ruby, there is no easy way to do this, although there is A way, which involves setting a trace function on the function and then executing it to get at the local variables at execution time, see here.

I have been making the same mistake a few times recently: instead of writing
things.each do |thing| ... end
Where things is a method, not a variable, I would write:
things do |thing| ... end
which has the end result of doing nothing. And I would have a hard time tracking down the bug - even when I have tests written which are failing because of it. I think it's because you just don't see a mistake like this, it's just one of those things where you can stare at the line all day and still not have a clue. Admittedly, this is a stupid mistake, but I also blame ruby for being extremely lose in its method invocation behavior when dealing with blocks. A block on a method is like an optional argument on ALL methods, not just the ones where you mean to use blocks. So this mean you can pass a block to any method, even if it's as simple as:
def hello
  'hello'
end
I have enumerated a handful of other languages including Smalltalk, Python, Haskell, Lisp, Java, and none of them behave this way. This is just one example of why Ruby is on the extremely unsafe end of the language spectrum. And this is also why there's a bigger need for automated testing in Ruby software, because the intepreter is so forgiving that even little mistakes like these - which are normally easily caught at runtime, if not compile time - can go through undetected.

Another unsafe thing about Ruby software is the frequent use of hashes as the poor man's named parameters. If you misspell the name of the key, there would be no complaint. And when you find out something's wrong, you wouldn't know where to look first.

Yet another is the fact that arrays simply return nil when you give it an out-of-bounds index:
>> [][7]
=> nil

I guess being unsafe is just part of the Ruby culture. In related news: Javascript is extremely unsafe too, arguably even more so than Ruby, since, for example it's not strict about enforcing the number of parameters you can pass into a function..

After learning of growl from Micah's post, I decided to port it to the windows equivalent: snarl. It was pretty easy:

1. Follow Micah's instructions except for the growl bits
   
2. Install Snarl from their website, this is a setup .exe
3. gem install ruby-snarl
4. get the icon images from Micah's post and put them some where. I put them in c:/devtools/autotest redgreen icons/
5. Put the code below in your .autotest, in place of the stuff for growl:

require 'autotest/redgreen'
require 'snarl'
module Autotest::Growl
  def self.notify title, msg, img
    Snarl.show_message title, msg, img
  end
 
  Autotest.add_hook :ran_command do |autotest|
    icon_path = 'c:/devtools/autotest redgreen icons/'
    output = autotest.results.grep(/\d+\s.*examples?/).last.slice(/(\d+)\s.*examples?,\s(\d+)\s.*failures?(?:,\s(\d+)\s.*pending)?/)
    if output =~ /[1-9]\sfailures?/
      notify "Test Results", "#{output}", "#{icon_path}rails_fail.png"
    elsif output =~ /pending/
      notify "Test Results", "#{output}", "#{icon_path}rails_pending.png"
    else
      notify "Test Results", "#{output}", "#{icon_path}rails_ok.png"
    end
  end
     
end


Screenshot:

This is a follow up to Advantures in Testing with Mocks. I have been growing dissatisfied with the mockist style of testing recently. The reason is, I was trying to learn it, but it just required me to stub out everything that was called. Not only was it tedious, it also cause the test code to be way too dependent on the implementation code. It got to the point where to write a test, I first have to look at the implementation to see what are the methods that are called. I was all but ready to call it quits on the mockist style.

But, yesterday I found stub_everything, which allowed me to find a way to embrace mocks once again. Basically, stub_everything('my object') is used in place of mock('my object'), which gives you a mock object that will return nil for any message sent to it that it doesn't understand without bombing out. This means that I can escape specifying every method call that has to go on during the tested method invocation.

Also, I realized that mocks are not to be used in all situations, but should rather be used strategically in certain spots. Basically, you want to avoid having your tests be dependent on your code. To give an example, let's say you want to test a controller method that looks up a user from a user ID. Naturally, you would stub out User.find, right? But what if the implementation used User.find_by_id instead? The problem is that there's more than one way to do this, and therefore the implementation is too maliable. Therefore, if it were me, I would not use a mock for a situation like this. Mocks and stubs are to be used only for cases where the method call - or message - is stable, there's only one method available to accomplish the given task, and when theh message is meaningful. For all messages that are not meaningful to be tested for the particular behavior, use stub_everything to ignore them. Here's an example:
describe "audit" do
  controller_name :project
  before :each do
    setup_projects # load data into db
    @user = stub_everything('user')
  end
   
  it "should audit when user is logged in" do
    controller.stub!(:current_user).and_return(@user)
    @user.should_receive(:audit).once
    get :show, {:id => @project.id}, {:user => 1}
  end
end
Here, I justify stubing out the controller's current_user method because that's the only way to get at the current_user that the authentication system provides. As for the expectation for audit, it's also a stable and non-aliased method. But even more importantly, that's exactly what I want to test! This is a case when I want to test the fact that the :audit method was called more so than I want to test its outcome as encoded in the database, because as I change the implementation of audit, its database representation could very well change, heck, audit info might not even be store in the database. As for the rest of the data - how project is looked up - that's retrieved from the database like normal. Real mockist testing people would say that db accesses are slow. There's a point to be made that project look up can also use a stub_everything, but one drawback of that is if the implementation does a multi-level object traversal like: project.name.length, then returning nil for name won't work, and you have to write some extra mocking code. Hmm, perhaps what is needed is a really_stub_everything which instead of returning nils for everything, returns itself instead, then you can really ignore all interactions with this object. Hmm..., that's a thought.

And so, I think that from now on, my mock testing approach - which is not really the mockist way - is to still use fixtures or other db setup templates by default, but only use mocks at select places where it makes sense.

Design Patterns have come under mild attact lately, as seen here, and here(well, this one's not exactly recent). As a result, I decided to revisit some of the design patterns to tell truth from myth for myself.

I am going to start with the State Pattern, just because. Here we go. Straight from the Gof:

Intent
Allow an object to alter its behavior when its internal state changes. The object will appear to change its class.

Back up a minute! The object will appear to change its class? Sounds like voodoo. A TCPConnection implementation is used here as an example. A TCPConnection behaves differently depending on which state it's in. So how do you encapsulate the state-specific behavior? Basically you abstract it out into a class hierarchy, which has TCPState as the abstract parent, and the specific states as the children: listening, connection established, and closed, as the the concrete state implementations. A TCPConnection object will dispatch to one of these three State strategies depending on which state it's actually in whenever it's gets an event that needs attention. In that sense, I guess TCPConnection can "appear to change its class", okay, doesn't sound that impressive after you explain it... You could always implement this by having switch statements all over the place, but this doesn't isolate all the behaviors of a state into one place - Gof mentions this in the consequences section. BTW, the consequences section is one of the best features of this book, the problem is, until you can gasp the pattern(whichever one), you probably won't understand this section. The second consequence Gof lists is that this pattern makes state transitions explicit, okay, valid, there are other ways to make this explicit(like having a state enum variable), but this is certainly one way. The third consequence is that the state object, if it has no variables, can be shared amongst any number of context objects(the holder of the states, in the example this would be TCPConnection).

Does this pattern stand on it's own? I give it a resounding yes! Does "modern" dynamic language features make this pattern unnecessary? No.

Now, with that out of the way, let's explore something a bit more exciting...

Here's an example, which was brought up by a former co-worker of mine. He had such a problem: a User could be a RemoteUser or a LocalUser, but a RemoteUser could become a LocalUser and vice versa. He wanted to know the best way to model this in the database. From his design, User would be the parent class and the other two the subclasses. I was quick to tell him to put everything in one table and use one field to specify the state of remote or local. I also thought that requiring an object to change it's type(notice that this is essentially what the State Pattern is trying to do, or to workaround not being able to do) was a ridiculous notion because the OO system just doesn't allow for that. So, clearly, changing the type of an object is something that people want to be able to do. The problem is that "remote" and "local" are really adjectives. In OO, there are only nouns and verbs. Conventionally, adjectives are represented by instance variables inside objects, in this case the variable would probably be named "state" or be a boolean named either "remote" or "local". But, the behavior of the object has to change depending on the state of this variable. The State Pattern is a way of simulating this, or to put it another way - a workaround. Workaround denotes clugginess and accidental complexity, is there any other way to do this?

Enter Mixology. This is a small ruby library from Something Nimble. It gives you the ability to unmix your mixins. Why would you want to do that? Why, it allows for a complete different way to implement the State Pattern! Just read the article in the link. I'll wait...

Basically, the way I see it, Mixology is turning the assumption of favoring composition to inheritence up side down. Well, it's not just Mixology, it's also multiple inheritence/mixins. Why? Well, why is composition more flexible than inheritence? Because 1) you can once use inheritence once(one parent) - but this problem goes away with multiple inheritence; 2) you can not change child parent relationships during runtime - Mixology fixes that. Wow! So does that mean we can start using inheritence everywhere we've been using composition? What would the world come to? Okay... I left out one reason, perhaps the most important one... 3) There's a tight coupling between the base class and the subclass, because of the implicit context in which the subclass code I plug in will be called - Erich Gamma. I can't argue with that one. But, still, inheritence has become much more powerful, with the addition of multiple inheritence and runtime switching of inheritence relationships.

Stepping back now to address the issue of design patterns being attacked. So, are design patterns bogus? I think that I have shown it's not. The example of Mixology is not meant to degrade the State Pattern but rather it serves to show that the Gof book is not the last word on design patterns and that there is room to add to and/or improve them(or worsen, if for example, switching inheritence relationships at runtime is not your cup of tea). It is much like how the Constitution was not the last word on how the government should behave.

This book chapter is a must-read for rubyists. A picture is worth a thousand words... a few pictures with words that explain them is even better... yeah. I also dig the conversational style of this writing, has a Java Puzzlers kind of vibe(apparently was actually adopted from the book The Little Schemer, the url to the book in progress is here).

Yum! I love Ruby vs Smalltalk flame wars. This one is about whether TDD renders debuggers useless. Giles Bowkett of Ruby vs James Robertson of Smalltalk. I would have to talk James' side here. Although great TDD/BDD can remove the need for the debugger, it's not easy to do great TDD/BDD. It's an art, and takes a good amount of commitment and effort to do, as well as training(I am sure I still don't know the best ways to test in some situations). Basically, TDD is something you have to go out of your way in order to do well, and to say that everyone should do great TDD is a bit like forcing a religion down someone's throat. I think I do OK TDD/BDD. I don't use the debugger on Ruby projects, but would I use it if there were a good one? Yes.

Stumbled into Neal Ford's post today which spawn off several other blog posts else where too, including Avi Bryant's new blog. How timely(this was actually one month ago)! I was just musing about this very thing and that's how I found the this blog post. Smalltalk has an adventage over Ruby(an pretty much any other language) because the runtime is always running. Runtime = development time = compile time. This allows for a lot of introspection to occur during development. Things like code completion, for example, are done by inspecting the objects in the runtime, rather than by inspecting the source code like in modern Java IDEs. Of course it can do more than that: things like excuting arbitrary code when you are at a breakpoint, to give another example. You could also query the system for the object class hierarchies to do anything you want with them, including creating new classes and methods on the fly.

Now, one thing about the Ruby metaprogramming techniques some complain about is the fact that they are hard to debug, because you can't see the code that you are executing. In contrast, you can get the source code to any method during runtime, which means you can easily debug through it(yes, but aside from that, debugging in Ruby still sucks anyway). It's the explicit vs implicit thing again. But the advantage of the Ruby approach is that it's more dry - you specify the declaration once, and that's it, the code that's generated is never seen anywhere, it removes the issue that all code generating tools have: that there's a versioning problem between the generated code and the spec that feeds into the generator.

Which way is better? Clearly(how can you argue with "clearly"? you can't), neither one is. Is there a best of both worlds? Hard to say.

It's worth pointing out that Ruby metaprogramming techniques are a very new development. Although it has it's roots in LISP, I think it is a new twist in the ways it is used in the context of OOP. Can these techniques be applicable to other languages? Yes, they are being adopted in many other languages, including Groovy, Python, and Javascript. Also, these techniques are usually used in programming frameworks, in which the ultimate end-user interface is a programming API. And, thus, the goal of all(most?) of these techniques is to automatically generate code that a developer writes code against. From a single programmer's perspective, these are just refactoring techniques, or design patterns, because they don't really give you anything you didn't have before in terms of what you can do(power?), but rather lets you structure your code in a way that's even more dry than what was possible before. As a result you can create very consice APIs with very little code.

Going back to the ActiveRecord has_many example, some have mentioned that this can be done without using code generation. In Smalltalk, apparently this is done by using an annotation. The class code can then presumably query the annotation information when it needs to get an association, I assume using something like method_missing(I'll have to check this), which you could do the exact same thing in Ruby, only in place of the annotation you'd still have the class initializer. In deed, that is how Hibernate does it for Java as well. So why do the Ruby guys like to use code generation in that case? It's just their style! If someone develops a new kung fu style that kicks your ass, are you going to say he's doing it wrong? Also, in Ruby method_missing is slow, so there's an incentive to avoid it.

I want to catalogue the ruby metaprogramming patterns that are used in the ruby community. Here are some blog links:

1. Inheriting class methods from modules
2. Ola bini's list of 10 techniques