Posts about ruby

HTTP Server in 5 Lines With Webrick

Usually when I am prototyping a web UI - either in Javascript or Flex, I would just write a static html, because that's the simplest thing that works. But, once in a while, it doesn't work because of the security restrictions that the browser imposes on local files. Maybe you want to use ajax calls(which is sometimes problematic on IE), trying to use the google maps api, or the FABridge, whatever the reason may be. Well, you can get around this problem easily using this ruby script:
require 'webrick'
server = WEBrick::HTTPServer.new :Port => 1234
server.mount "/", WEBrick::HTTPServlet::FileHandler, './'
trap('INT') { server.stop }
server.start
This runs a web server at http://localhost:1234/ which mounts the top level directory to your current directory.

Update: Oops, it's not exactly that easy after all. In order to prevent caching - which you will want to do if you are doing development - you will want to write an extra class. The modified script:
require 'webrick'
class NonCachingFileHandler < WEBrick::HTTPServlet::FileHandler
  def prevent_caching(res)
    res['ETag']          = nil
    res['Last-Modified'] = Time.now + 100**4
    res['Cache-Control'] = 'no-store, no-cache, must-revalidate, post-check=0, pre-check=0'
    res['Pragma']        = 'no-cache'
    res['Expires']       = Time.now - 100**4
  end
  
  def do_GET(req, res)
    super
    prevent_caching(res)
  end

end

server = WEBrick::HTTPServer.new :Port => 1234
server.mount "/", NonCachingFileHandler , './'
trap('INT') { server.stop }
server.start
Not 5 lines anymore, bummer! The code for NonCachingFileHandler was stolen  from unittest_js.
Posted by Toby 5 months ago about html, javascript, programming and ruby (0 comments)

The New-Found-Love for Dynamic Languages

Posted by Toby 7 months ago about haskell, programming and ruby (0 comments)

Null in 10 Languages

  • Java, Javascript, C#: null
  • Ruby, Smalltalk, Lisp, OCaml: nil
  • Lisp: ()
  • Python: None
  • Haskell: Nothing
  • C: 0
Posted by Toby 10 months ago about c, c#, haskell, java, javascript, lisp, ocaml, programming, python, ruby and smalltalk (0 comments)

ActiveRecord Gotcha

I came across this interesting gotcha while debugging through someone else's rails code. When you set a model's one-to-many attribute(an perhaps other association types as well), let's say:
user.friends = new_friend_list
It doesn't always set it. As far as I can figure from my blackbox testing, it does an equal check first between the old value and the new value, and then sets only if they are different.

Now, in Ruby's == is easily overridable, you can easily make it do anything you want. It happens that Array#== returns true iff both arrays have the same number of elements and each element in one array is == to the element of the other array in the same position. It also happens that ActiveRecord's base class' == returns true as long as the objects are of the same class and the primary keys equal.

Which brings us to the gotcha of the day. The code in question built an array, say new_friend_list independently, starting with an id list passed in from a from, finding each friend in the database and adding them to the array, but, in the meantime, modifying a field in each of the friends, say...friend.charma++. It then goes on to set the friends attribute in the manner you'd already seen above, and saves user. Well, the charma field that was modified in each of the friends retrieved did not get saved, because the set did not happen: the new_friend_list pointed to the same list of friends user was already associated to.

Hmm..., yeah. Watch out for this one. Debugging this one was not fun. It definitely didn't fit with the principle of least surprise.
Posted by Toby 10 months ago about gotcha, programming, rails and ruby (0 comments)

Auto-mixin in Python

Python has multiple inheritence - a feature that's really handy because it allows the programing idiom called mixins - where you write a a bunch of methods on a "mixin" class and then later on attaching all of those methods onto another class simply by mixin-it-in. In Python, the process of mixin-it-in is simply to add it to the parent list of the class:
class MyObject(MyParent, MyMixin):
   ...
In Ruby, you can automatically mix in a mixin programmatically because the mix-it-in is done within the class definition, and because of its open classes. This is how the rails helpers are automatically found in a directory and mixed into the controllers for example.
How would you do this in Python? Let's say we simplify the problem to this: For a given Controller subclass named ProductController, for example, if there exists a helper class by naming convention: ProductControllerHelper, then we mix it in. Here's my solution:
class Meta(type):
    def __new__(cls, name, bases, dct):
        helper_mixin = globals().get('%sHelper' % name)
        if helper_mixin:
            bases = list(bases)
            bases.append(helper_mixin)
            return type.__new__(cls, name, tuple(bases), dct)
        else:
            return type.__new__(cls, name, bases, dct)
        
class Controller(object):
    __metaclass__ = Meta
        
class ProductControllerHelper(object):
    def method1(self):
        print "method1 invoked"
        
class ProductController(Controller):
    def index(self):
        self.method1()
        
        
if __name__ == '__main__':
    ProductController().index()
Output:
method1 invoked
It worked!

But! We are not done. What if what you want to mix in depends on how you configure the class inside the class definition? In my case, I wanted to mix in a bunch of classes when I write a DSL-like declaration like this:
class MyTestCase(TestCase):
    all = fixture(SeleniumRCServer, BrowserSession, DBData, Login)
    each = fixture(AddDelivery)
    ...
This is for a testing framework I am working on. I have a test case, and I want to specify its test context(fixtures) in the manner shown. Each object in the fixture(..) definition is a mixin. So I want to mix them all into MyTestCase.
This case is harder than the previous case because when __new__ is invoked, the class definition hasn't been processed yet, and therefore you won't have access to the class attributes all and each. The hack I came up with is this:
class TestCaseMetaClass(type):
    def __init__(cls, name, bases, dct):
        if len(bases) == 1 and bases[0] != unittest.TestCase:
            bases = list(bases)
            bases.extend(cls.all)
            bases.extend(cls.each)
            cls.__real__ = type(name, tuple(bases), dct)
        else:
            type.__init__(name, bases, dct)
        
    def __call__(cls, *params, **kws):
        if hasattr(cls, '__real__'):
            return cls.__real__(*params, **kws)
        else:
            return type.__call__(cls, *params, **kws)
            
class TestCase(unittest.TestCase):
    __metaclass__ = TestCaseMetaClass
Basically, I first override __init__ of the metaclass(rather than __new__) in order to record what was declared in all and each, and create a new class on the fly mixing in all the mixins defined in the fixture(..) definitions. We store that class in an attribute __real__ attached to the class. Then we override __call__ for the metaclass to return an instance of the class held in the __real__ attribute instead of the actual class. Oh yes, this is really ugly, but it worked, at least for what I was doing. If you can write to a classes' __bases__ attribute(which defines the parents of the class), then this would be much easier, but I haven't figured out how to do it if it can be done at all. But because of this limitation, it looks unlikely you can do something like Ruby's mixology in Python.

Update: the better solution to the second part is found:
class Meta2(type):
    def __new__(cls, name, bases, dct):
        mixins = []
        all = dct.get('all')
        if all:
            mixins.extend(all)
        each = dct.get('each')
        if each:
            mixins.extend(each)
        bases = list(bases)
        bases.extend(mixins)
        return type.__new__(cls, name, tuple(bases), dct)
        
class TestCase(object):
    __metaclass__ = Meta2
    
class Fixture1(object):
    def method2(self):
        print "method2 invoked"
    
class Fixture2(object):
    def method3(self):
        print "method3 invoked"
        
class BlahBlahTest(TestCase):
    all = [Fixture1]
    each = [Fixture2]
    def blah_test(self):
        self.method2()
        self.method3()
Posted by Toby about 1 year ago about metaprogramming, mixin, programming, python and ruby (0 comments)

Ruby Gotcha of the Day

Okay, so in ruby:
>> nil or 'value'
=> "value"
because nil acts as false when used with boolean operators and any other object acts as true. Now let's assign the result to a temp variable a:
>> a = nil or 'value'
=> "value"
>> a
=> nil
Whoa?? Can you guess? In ruby the assignment operator is not special. It acts just like any other operator and does not have a lower precedence than the or operator. Therefore the expression
a = nil
executes first, which evaluates to nil, and assigns the value nil to a, and then the result nil is used in the expression nil or 'value'. Therefore to do what we intended we need to do:
>> a = (nil or 'value')
=> "value"
Wow! That is messed up.
Posted by Toby about 1 year ago about gotcha, programming and ruby (0 comments)

Impressions of Turbogears 4 months in

I've been a user of Turbogears for 4 months now, working on a client facing app. The app has not gone production yet, so I don't have much insight on deployment, but I have a lot of experience on the development side - I started from scratch - and here is what I learned so far.

Freedom of Choice
Turbogears as a framework is pretty agnostic of different components such as ORM, or template engine. Although there is a default choice, I found it wasn't hard to stray away from it.

Mako
I ended up choosing Mako as the template engine because, coming from rails, I felt kid and genshi were too heavyweight for my taste since they are based on XSLT and requires your markup to be valid XML before it can do anything, which obviously means there's an XML parsing step it has to do. Mako is more like erb in that it's "text-based", e.i. it's perfectly fine to render non-valid XML code. But Mako turned out to be much more than another erb. With Mako you can easily write helper template functions and reuse them everywhere. You can also write inversion-of-control template style functions which take in a partial template and calls it inside its body. It always puzzled me why you couldn't do that with erb or haml or most of the ruby template engines easily. With erb, you have to write a partial view as a separate file, but calling a partial with local parameters is inconvient, you have to write something like:

render :partial => 'my_control', :locals => {:control_id => 'con', 
:height=> '50px'}

and since this is so inconvient, i usually end up wrapping it with a helper method like:

def my_control(control_id, height)
    render :partial => 'my_control', :locals => {
:control_id => control_id, :height=> height}
end

Of course this is partially due to the culture in rails that you normally write helpers in ruby rather than in a template language. In Mako you don't have to do this extra step, which makes me happy. Now if you want to do the inversion of control thing, in erb it's even worse! In mako you would just do this. So, in general, I am able to refactor my views a lot easier with mako and therefore I find myself doing it a lot more often.

SQLAlchemy
SQLAlchemy is a main stream ORM in the python community. It's direction is different from that of ActiveRecord and is a lot more similar to Hibernate of Java but also has similarities to Ambition of Ruby and Linq of .NET. It is similar to Hibernate in that it is very fully featured, has sessional transaction management, and can coupe with a large variety of schemas. It is similar to ambition and linq in that you can contruct queries in your host language in a very succint and elegant way(I know you can build queries in Hibernate's criteria API too, but it's not quite elegant). I like SQLAlchemy a lot! Here's a couple of sqlalchemy tricks I like. First one:

    fields = [
        User.user_name,
        User.display_name,
        User.email_address
    ]
    results = User.query.filter(or_(*[f.ilike('%' + q + '%') 
for f in fields]))

The above code does a wildcard partial string match of the string q against any of the three fields listed in the fields list. Second example:

    page = User.query[10:20]
           
This looks like array slicing, but no, it's slicing against the query results! It's smart enough to build the query using OFFSET and LIMIT or equivalent. You can easily do pagination with this technique.

Python's Named Parameters
Another thing I like when working with turbogears is python's named parameters. Whereas rubists use the hash as the poor man's name parameters, python has real named parameters, which is not only safer, but more elegant.

Python's Polluted Name Space
I run into this problem once in a while, but I hit on it 2 or 3 times in the last week! In python, list, dict, str, int, etc. are the names of fundamental types in the language, therefore you can't(actually you can, but don't want to) use them as variable names. More than once I've tried to use list as a variable name, which python doesn't complain about immediately but causes a cryptic error down the road.
I've also tried use from as a variable name, which turns out to be a keyword in the language, this causes a syntax error, which you don't see until you realize it's a keyword. Now, of course, more languages suffer from this problem, ruby isn't any different, but I think ruby has better error messages for these syntax errors: it will tell you the symbol that is unexpected and what symbols it was expecting.

Little Verbosity
Turbogears is more verbose than rails most prominently in 2 areas: import statements and method decorators. Rails files, usually have at most 2 requires(counterpart of import in python), most of the time none at all. My TG controller files and the model.py(the file with all the model objects) usually have about 10 to 15 lines of includes, my Mako template files usually 2 to 4 lines of includes. This has a lot to do with the design of the language. The python interpreter requires each .py file to act like a module, and as a module, it must identify all of its dependences, the ruby interpreter does not require this and so your controller code, for example, doesn't need to explicitly require anything before it has what it needs to do its work.
I think method decorators are cool, but they can also be overused and become cluttery. Some of my controller methods have more than 4 or 5 lines of decorators, consisting of the mandatory expose(), access restriction spec, and form parameter validators. That's a bit much. I also don't like the fact that you need an expose() decorator for every single controller method. Rails has no such thing and usually specifies such things at the top of the class, which has pros and cons vs TG's approach, but is at the end less cluttery.

Posted by Toby about 1 year ago about programming, python, ruby and turbogears (0 comments)

Getting the Argument Names of Your Function

In Javascript:
myfunction.toString()
This gives you the entire definition of the function as a string, with which you can parse to get the argument names, or in prototype.js you can just do:
myfunction.argumentNames()

In Python:
myfunction.func_code.co_varnames

In Ruby, there is no easy way to do this, although there is A way, which involves setting a trace function on the function and then executing it to get at the local variables at execution time, see here.




Posted by 宝宝妈, Toby about 1 year ago about javascript, programming, python and ruby (0 comments)

How Safe is Your Programming Language

First, let me define what I mean by safe: the earlier a programming language catches a programming error for you, the safer it is. Haskell is extremely safe, whereas php is extremely unsafe. Some examples:
  • Errors can be caught at compile time, such as mispelling of function names
  • Errors can be caught at runtime, such as NullPointerExceptions, but it may be caught early or later in runtime, there's a continum
  • Errors can be caught a type time if you have a method completion capable IDE, for exmple
  • Errors can be caught a unit test time, but that out of the scope for this article, because I want to talk about languages and not testing practices
  • Errors can be caught at QA testing time vs in production, these are also a bit out of scope
How do you measure the safety of a programming language? One can take specfic classes of errors and see how and when a language catches them. I will use the notation >> to mean safer than. A score is given for each language for each type of error corresponding to how safe they are.

Null Pointers
  • Java, Ruby, Python, C#, Smalltalk, Javascript, LISP - catch at runtime when you try to derefernce the pointer
  • C, C++ - never catches it, they merely core dump, which does not tell you what the specific error was
  • Haskell, OCaml - catches it at compile time, because you cannot have Null values unless you specifically define it in the type, and the compiler requires you to handle the null case
    So in this case: Haskell, OCaml (2) >> Java, Ruby, Python, C#, Smalltalk, Javascript, LISP (1) >> C, C++ (0)

Array Index Out Of Bounds
  • Java, Python, C#, Smalltalk - catch at runtime when you try to index the array
  • Ruby, Javascript - never catches it, merely returns null if you try to index an array with an out of bounds index
  • C, C++ - never catches it, may or may not core dump.
  • Haskell, LISP, OCaml - although you can use Arrays if you really want/need to, in which case the error is caught at runtime like Java, etc. The prefered practice of these languages is to use linked-lists instead, which will never have index out of bounds errors
    So: Haskell, LISP, OCaml (2) >> Java, Python, C#, Smalltalk (1) >> Ruby, Javascript, C, C++ (0)

Type Errors
  • Java, C# - these are caught at compile time if you never use casting in your program. But if you do use casting, these will be caught at runtime as ClassCastExceptions at time of cast.
  • Ruby, Python, Javascript, Smalltalk, LISP - caught at runtime when a method on the object is accessed
  • C, C++ - Compile time if you don't use casting. Never caught and possible core dumps if you do.
  • Haskell, OCaml - Compile time always, casting is not allowed.
    Haskell, OCaml (3) >> Java, C# (2) >> Ruby, Python, Javascript, Smalltalk, LISP (1) >> C, C++ (0)

Mispelled Named Parameters
  • Java, C#, C, C++, Haskell, Javascript - these languages do not have this feature
  • Python, Smalltalk - caught at time of method invocation
  • Ruby - rubists employ a poor-man's named parameters by just passing in a Hash to methods, the languages supports syntactic sugar that make this style of method calling look like named parameters, but mispelled keys are never caught
  • OCaml - caught at compile time
    OCaml (2) >> Python, Smalltalk (1) >> Ruby (0)

Wrong Number of Parameters
  • Python, Smalltalk, LISP - caught at method invocation
  • Javascript - never caught, unsupplied parameters an simple set to null, and it never hurts to use too many parameters
  • Ruby - caught at method invocation time for the normal cases, but blocks are a special type of parameter which is always optional. So if you supply a block to a method that does not require it, ruby does not complain
  • Haskell, Java, C#, C, C++ - compile time
    Haskell, OCaml, Java, C#, C, C++ (3) >> Python, Smalltalk, LISP (2) >> Ruby (1) >> Javascript (0)

So a language can be safer or less safe under different contexts. There obviously are other types of errors that I have not listed, but this will do for a simple comparison. I've already given scores for each language for each error type. Since some languages do not have the named parameters feature, I will average the score for each language over the error types that are relevant to it. So the Tally is:
  1. Haskell - scores: [2, 2, 3, 3], average: 2.5
  2. OCaml - scores: [2,2,3,2,3], average: 2.4
  3. Java, C# - scores: [1,1,2,3], average: 1.75
  4. LISP - scores: [1,2,1,2], average: 1.5
  5. Python, Smalltalk - scores: [1,1,1,1,2], average: 1.2
  6. C, C++ - scores: [0,0,0,3], average: 0.75
  7. Ruby - scores: [1,0,1,0,1], average: 0.6
  8. Javascript - [1,0,1,0], average: 0.5
It's not really fair that OCaml is scoring lower than Haskell, they should be the same, the difference is because Haskell doesn't have named parameters so the average is skewed - my scoring system is not perfect. Haskell is safer than OCaml in one aspect though. Which is that in Haskell, purely functional code is guaranteed to have no side effects and will never throw exceptions.
Posted by Toby about 1 year ago about c, c#, haskell, java, javascript, lisp, ocaml, programming, python, ruby and smalltalk (0 comments)

Ruby: Extremely Unsafe

I have been making the same mistake a few times recently: instead of writing
things.each do |thing| ... end
Where things is a method, not a variable, I would write:
things do |thing| ... end
which has the end result of doing nothing. And I would have a hard time tracking down the bug - even when I have tests written which are failing because of it. I think it's because you just don't see a mistake like this, it's just one of those things where you can stare at the line all day and still not have a clue. Admittedly, this is a stupid mistake, but I also blame ruby for being extremely lose in its method invocation behavior when dealing with blocks. A block on a method is like an optional argument on ALL methods, not just the ones where you mean to use blocks. So this mean you can pass a block to any method, even if it's as simple as:
def hello
  'hello'
end

I have enumerated a handful of other languages including Smalltalk, Python, Haskell, Lisp, Java, and none of them behave this way. This is just one example of why Ruby is on the extremely unsafe end of the language spectrum. And this is also why there's a bigger need for automated testing in Ruby software, because the intepreter is so forgiving that even little mistakes like these - which are normally easily caught at runtime, if not compile time - can go through undetected.

Another unsafe thing about Ruby software is the frequent use of hashes as the poor man's named parameters. If you misspell the name of the key, there would be no complaint. And when you find out something's wrong, you wouldn't know where to look first.

Yet another is the fact that arrays simply return nil when you give it an out-of-bounds index:
>> [][7]
=> nil


I guess being unsafe is just part of the Ruby culture. In related news: Javascript is extremely unsafe too, arguably even more so than Ruby, since, for example it's not strict about enforcing the number of parameters you can pass into a function..

Posted by Toby about 1 year ago about programming and ruby (0 comments)

Autotest popup notifications with Snarl

After learning of growl from Micah's post, I decided to port it to the windows equivalent: snarl. It was pretty easy:
  1. Follow Micah's instructions except for the growl bits
  2. Install Snarl from their website, this is a setup .exe
  3. gem install ruby-snarl
  4. get the icon images from Micah's post and put them some where. I put them in c:/devtools/autotest redgreen icons/
  5. Put the code below in your .autotest, in place of the stuff for growl:

require 'autotest/redgreen'
require 'snarl'
module Autotest::Growl
  def self.notify title, msg, img
    Snarl.show_message title, msg, img
  end
 
  Autotest.add_hook :ran_command do |autotest|
    icon_path = 'c:/devtools/autotest redgreen icons/'
    output = autotest.results.grep(/\d+\s.*examples?/).last.slice(/(\d+)\s.*examples?,\s(\d+)\s.*failures?(?:,\s(\d+)\s.*pending)?/)
    if output =~ /[1-9]\sfailures?/
      notify "Test Results", "#{output}", "#{icon_path}rails_fail.png"
    elsif output =~ /pending/
     
notify "Test Results", "#{output}", "#{icon_path}rails_pending.png"
    else
     
notify "Test Results", "#{output}", "#{icon_path}rails_ok.png"
    end
  end
     
end


Screenshot:

Posted by Toby about 1 year ago about programming, ruby and testing (0 comments)

Mock Testing and stub everything

This is a follow up to Advantures in Testing with Mocks. I have been growing dissatisfied with the mockist style of testing recently. The reason is, I was trying to learn it, but it just required me to stub out everything that was called. Not only was it tedious, it also cause the test code to be way too dependent on the implementation code. It got to the point where to write a test, I first have to look at the implementation to see what are the methods that are called. I was all but ready to call it quits on the mockist style.

But, yesterday I found stub_everything, which allowed me to find a way to embrace mocks once again. Basically, stub_everything('my object') is used in place of mock('my object'), which gives you a mock object that will return nil for any message sent to it that it doesn't understand without bombing out. This means that I can escape specifying every method call that has to go on during the tested method invocation.

Also, I realized that mocks are not to be used in all situations, but should rather be used strategically in certain spots. Basically, you want to avoid having your tests be dependent on your code. To give an example, let's say you want to test a controller method that looks up a user from a user ID. Naturally, you would stub out User.find, right? But what if the implementation used User.find_by_id instead? The problem is that there's more than one way to do this, and therefore the implementation is too maliable. Therefore, if it were me, I would not use a mock for a situation like this. Mocks and stubs are to be used only for cases where the method call - or message - is stable, there's only one method available to accomplish the given task, and when theh message is meaningful. For all messages that are not meaningful to be tested for the particular behavior, use stub_everything to ignore them. Here's an example:
describe "audit" do
  controller_name :project
  before :each do
    setup_projects # load data into db
    @user = stub_everything('user')
  end
   
  it "should audit when user is logged in" do
    controller.stub!(:current_user).and_return(@user)
    @user.should_receive(:audit).once
    get :show, {:id => @project.id}, {:user => 1}
  end
end
Here, I justify stubing out the controller's current_user method because that's the only way to get at the current_user that the authentication system provides. As for the expectation for audit, it's also a stable and non-aliased method. But even more importantly, that's exactly what I want to test! This is a case when I want to test the fact that the :audit method was called more so than I want to test its outcome as encoded in the database, because as I change the implementation of audit, its database representation could very well change, heck, audit info might not even be store in the database. As for the rest of the data - how project is looked up - that's retrieved from the database like normal. Real mockist testing people would say that db accesses are slow. There's a point to be made that project look up can also use a stub_everything, but one drawback of that is if the implementation does a multi-level object traversal like: project.name.length, then returning nil for name won't work, and you have to write some extra mocking code. Hmm, perhaps what is needed is a really_stub_everything which instead of returning nils for everything, returns itself instead, then you can really ignore all interactions with this object. Hmm..., that's a thought.

And so, I think that from now on, my mock testing approach - which is not really the mockist way - is to still use fixtures or other db setup templates by default, but only use mocks at select places where it makes sense.
Posted by Toby over 2 years ago about mocks, programming, ruby and testing (0 comments)