Tag > programming

tadalist is unsecured

by airportyh posted at 10-06-2008 10:14AM - Comments (0)   programming tech web
I always look to 37signals when I need design ideas or looking for precedence for how to solve certain problems. My problem this time was: how to gracefully handle your Ajax calls failing due to your login session expiring? I looked at tadalist. To my surprise, even after I deleted the session cookie, modifying the list still worked. I looked at the request headers and it seemed like they simply weren't securing the app at all. I wrote a quick socket program to verify this, and it was true: any one who has the ID of your list and/or items can make modifications to them. I guess they just felt this app is not mission critical enough to worry about this kind of stuff. What about the todolist in basecamp? Hmm... let me check... result: they don't really handle this gracefully in Basecamp. If you try to check off a list which auth, it just hangs there foreever. I guess I'll have to creative and come up with my own solution here.

Ditching e texteditor and komodo edit for SciTE and Launchy

by airportyh posted at 10-01-2008 11:46PM - Comments (0)   editors launchy programming scite
I have had it with e-texteditor and Komodo Edit! I wrote a comparison of the two and have been switching back and forth between the two since then. While both editors have really great features, the basics are JUST NOT THERE. E-texteditor: each new version just crashes more and more, now everytime I use tortoise SVN from the project pane it crashes, it's not as snappy as I want either in general. Komodo Edit: the file browser filter is just too painfully slow! The textbox on top of the file browser is a live search, except if you want a live search to work, it better load FAST, otherwise every key I type is going to cause me to wait. Plus, the filter is not smart, it doesn't allow typos, I have type the name right or I am going to miss the file I want(E-texteditor doesn't have this problem, they have a pretty good quick launch).

SciTE is a pretty popular open source code editor. It's lightweight and it's fast. It's not as featureful as the other two, but it opens REALLY fast. When I first decided to switch to it, I thought, oh great, now I have to use a file explorer as my project pane and drag files onto my editor to open it? That's kind of a hassel too. So I changed my file associations so that most of my project files .py, .html, .mak etc open with SciTE by default. Now I can just click on the file and it opens in the editor. But then it hit me, I can open them with Launchy! Launchy is a great launcher app for Windows and I have been using it for a long time and have been very happy with it. I configured it to scan my project directory, and vola! Now I can open any file in the project at thought speed! Under a second, definitely. Way, way faster than I could do in e-texteditor or Komodo Edit. All I do is hit Alt-SPACE start typing the name of the file I want to open until it shows up as the selected choice, and then hit Enter. Usually it doesn't take but 3-5 key strokes to get the file you want to show up, and typos are ok, I can type shgrp for example and it'll give me show_group. I am really loving this way of working, I can get to stuff so much quicker. The reason it's so quick, is because of the speed of both Launchy and SciTE. Launchy runs in the background, and is really fast and responsive to your keystrokes. SciTE launches instantly.

The only problem now though, is that since launching a file using SciTE opens a new SciTE window instead of in a tab in the existing one, I now end up with a lot of SciTE windows open and they can be hard to manage. If those windows are grouped in the task bar(which XP does for you when there are many similar windows), it will help, but having it open as a tab would be even better. I might just dig into their code one day and see if I can make this happen.

BTW: To give you an idea how fast SciTE is compared to the other two. E-texteditor took 8 seconds to start up, after which there was still a popup dialog I had to click, which frozen it up for another 5-10 seconds, but my first click didn't work so I had to click again to get rid of the dialog to make it go away before I can start working. Komodo Edit took 25 seconds just to come up but about 32 seconds for it to be interactive(stopped loading). SciTE opens in 1 second or less everytime.

Selenium-rc proxy server war story

by airportyh posted at 10-01-2008 11:04PM - Comments (0)   java maven programming python selenium
I am trying out selenium yet again to help our non-existent QA team. I have to admit that I kinda flaked out on unit testing too(I kinda got sick of TDD in rails bogging me down to be honest), which I mean to catch up on, but, for now, I wanted to focus on selenium, since, with an ajax app like ours, too many things could go wrong, it could be either client side or server side or a combination, so intergration testing I think can really buy us a lot. Besides, I really wanted to know if selenium works well or not.

Anyway, this is not a post about selenium, exactly, I'll probably dedicate another to it later, when I have had sufficient experience with it. The problem I faced in this episode is that when I run the app through the selenium-rc proxy server, my flash messages aren't showing up! Yes, the proxy server is muffling my flash messages, what could it be? This first thought was it had to be a cookie issue since the flash message is implemented as a one-time use cookie, but the other cookies worked fine, or I wouldn't have been able to login to the app.

From looking at the cherrypy code, it looks like it's setting 2 cookies, one for auth, the other for the flash message, and it generates 2 Set-Cookie headers. I suspected that this was screwing up the selenium proxy server. I decided to dig into the selenium server code and put in some debug statements.

Aww! Getting the selenium-rc source and then building it with maven 2 is back to the painful slow Java days. Man! Maven, do you really have to download the entire internet just to build the project? Maven just symbolizes all that I hate about Java. It's bloated; it's framework heavy; it forces things on you that you don't need; running a build with it is glacial; build plugins is a big hassel, did I leave out anything? I think Maven may be the worst thing that has happened to the Java community...but I digress.

Basically I tracked it down to the part where the proxy server gets the header fields from the HttpURLConnection object(part of the standard Java API), but it drop the flash message cookie somehow. I suspected it was because that API just doesn't cope with duplicate header names in the response, but that seems strange that this has never come up. Googling found that people have been able to get dupliately named headers using the getHeaderFieldKey(int n) and getHeaderField(int n) methods, which take in a positional parameter. This is what the proxy server was doing, but yet it didn't work.

I used wireshark to look at the packets to confirm my theory, it did, but I also found another interesting thing - there is an empty line between the first Set-Cookie header and the second. I came to me that this is probably what tripped up the HttpURLConnection code, and I was right. I changed the Cookie code in the standard library to use \n instead of \r\n(so it interpreted as an empty line) as the separator and it solved the problem.

But I don't want to just patch a standard python library like that, it's not deployable. Don't know whether I should fix this in selenium-rc or not, in which case someone suggested I use the apache httpclient in place of HttpURLConnection.

button's value attribute

by airportyh posted at 09-29-2008 04:50PM - Comments (0)   crossbrowser javascript programming
If you use a button element as a submit button in a form, like:
<button id="mybutton">My Button</button>
$('mybutton').value in FF will get you an empty string whereas in IE you would get "My Button"

justtodolist updates and appengine stuff

by airportyh posted at 09-06-2008 02:40AM - Comments (0)   appengine migration programming
I made some enhancements again to justtodolist. There's a couple of interesting points I came across so I thought I would jot it down.

The stateful interactive shell sample app.
This is a must for any app engine developer! It's leaps and bounds closer to the python shell than the default interactive shell. Get it from here. It needs some tweaking to get it working along side your app. Hint, aside from modifying your app.yaml, you will likely have to modify the app config in shell.py as well.

Again, simplicity works!
I made a simplification in my model in the in workings of the ordering of the lists, and the result was much less/simpler and less buggy code.

Data model migration.
So as I pointed out before, migration in app engine is not really well supported, essentially, you are on your own. If you want to change the type of a property - for example, I wanted to change a ListProperty(int) to a ListProperty(db.Key) - you will likely completely break your app, because the data will no validate. There is no way to migrate the data, because there's no way to access the existing data. The only workaround that I could come up with was to rename the property(well, actually, adding a new one and leaving the old one, at least temporarily). After that I could write a migration script. Now that I had the interactive shell on production as well(the default shell was only available for development), I could run it easily through the shell instead of writing a handler with the sole purpose of running the migration script.

My co-worker pointed out to me that an automatic migration scheme might be the future for the appengine data store and similar object oriented datastores. The idea was very interesting. Basically, we could come up with a scheme for migrating data models using renaming mechanisms like what I did above. But, we could do it all behind the scenes inside a framework. I can see how it would work and how it could be done:
  1. you would have a global schema version number. Everytime you modify the schema the version number would increment, ideally, the framework would somehow detect the change you made and up the version automatically.
  2. you would have a sort of version control for every property of every object, and have a numbered naming scheme for them, so: name_1, name_2, for example. Every change to the property would up the property version number.
  3. Each object would have a version property. This means that the system can operate with objects that have different schema versions at the same time without breaking.
  4. Removing a property is easy, nothing more really needs to be done. Unless you want to provide a way derive a value to the old missing property, in which case you can provide a property descriptor of the existing object to do it.
  5. Adding a property is easy too, you can provide a function of the existing object to provide an initial value if you want, which will be invoked the first time that the property is used.
  6. Renaming a property can be done by specifying the property with an old_name directive like: new_name = db.StringProperty(old_name="old_name") then the code just has to create the new property and initialize it to the value of the old property the first time it is used.
  7. Change the type of a property can be done by using an old_type directive much like the old_name directive for renaming, and then you may provide a conversion function of the existing object that returns the new initial value. Behind the scenes, the property version number will be incremented, so you are really adding a new property, which is aliased somehow back to the same name later.
Sounds good...There's one problem though, as you edit the schema in your .py file with directives like old_name and initial value functions which only apply to the most recent migration. But, since as we've seen, migration happens on the fly(as the data is accessed), there needs to be a historical archive of all migration directives, i.e. real version control. One way to do it would be to write explicit migration definitions as in rails migration. So something like:
add_property(Person, 'age', db.IntegerProperty())
instead of editing the models directly. I think this way is sufficient. I was kinda hoping I could just edit the models directly willy-nilly though. Maybe you can write tool that inspects your model definition and diffs it with the last version and generate a migration definition?

Anyway, better get some sleep. This is definitely an interesting idea and a very concrete one for a project.


Making justtodolist work for chrome and safari

Since Chrome is so great, I wanted to use it for all my browser needs. But the apps I've been working on myself have not been targeted for WebKit - the rendering engine used by chrome, and also used my Safari. This is mostly due to laziness on my part - having to handle two browsers at a time is chaos enough. But Safari compatibility is on my todo list for more than one of my projects, and chrome is a good push for this cause. And so I went in today and made just todo list. compatible with Chrome and Safari. The differences I encountered were:
  1. This is really prototype specific - prototype adds an empty underscore parameter for all Ajax request for WebKit browsers. Because apparently, Safari errors out when you try to post with an empty body on XHR calls. And, who knows, this might already be fixed in WebKit. I happened to have code that was not handling this fact on the server side. This was no biggy.
  2. This is an error on my part really, but I had a form with a text input inside. But also registered for the key events on the text input so that when enter is pressed I called form.submit(). The key event code is really unnecessary because an enter on the text input triggers the form submit on both FF and IE. So, although on the other 2 browsers this was no problem, in WebKit this triggers 2 posts. So... just DON'T do this.

Google Chrome

by airportyh posted at 09-02-2008 10:33PM - Comments (0)   browser google javascript programming tech
Google just released this awesome browser. They have a great comic strip walking you through what the big deal is about, which is a must read for web developers. I've been using the browser for half a day and it's been pretty rock solid for all the sites I normally use. Javascript is clearly MUCH faster thanks to the v8 engine, which I think Steve Yegge alluded to a while ago. I like many of it's different features, and think it's a great boost to the browser world as well as javascript, i.e. the NBL(Next Big Language).

Play Date with jQuery

by airportyh posted at 08-31-2008 11:47PM - Comments (0)   javascript jquery programming prototype
I had a new toy project and started out with my trusty sidekick: prototype.js but a bit of the way in i decide to try my hand at jQuery instead, since I've already read a lot about jQuery and really like the monad-like paradigm which allows you to write elegant "flow" code without having to worry about null values anywhere in the chain.

It was quite easy translating my existing prototype.js code to jQuery(probably just around 30 lines).
  • The end result was probably 20% more succient.
  • I love the css 3 capable selectors, it's very refreshing to be able to use attribute and partial string match selectors which normally you don't get to use for your work. 
  • I also like the way attributes are set as chained methods(again, in the monad flavor), you do something like $('#myid').attr('value') to get the value and $('#myid').attr('value', 1) to set the value to 1. This again, allows you to chain stuff together. What's the point of this? ...I guess I just like the monad thing.
  • I think the jQuery syntax for behavioral javascript is prettier than prototype.js + lowpro.js
I really have nothing bad to say about jQuery. Even the UI components seem pretty on par with prototype.js, there's jqueryui, from which I used the autocomplete plugin. It's a little bit unfortunate that I had already invested about 1800 lines of javascript code on writing javascript components based on prototype.js and scriptaculous at work, otherwise had I started over I would very probably use jQuery. I will definitely go jQuery for my other projects.

Are there anything from prototype.js that I miss? Well, yes, I miss the array extensions from prototype, which lets you use ruby/smalltalk/lisp-like internal iterator style list operations. jQuery has a $.each, but it's not quite as pretty... but it gets the job done. And I haven't looked, but I am not sure there's an equivalent to prototype's Event.KEY_RETURN, Event.KEY_BACKSPACE and such in jQuery. Also, I had gotten comfortable to the OO model from prototype, and doing without it will be different, but then again I am adaptable. Plus, there's nothing at all stopping you from using jQuery and prototype.js together.

The z-index IE bug

by airportyh posted at 08-23-2008 02:38PM - Comments (0)   css javascript programming
Yeah, the z-index IE bug, have you heard of it? I was bitten by it a couple of days ago. In my case, I had a list of text fields in a form layed out from top to bottom. The problem is, I have these interactive drop downs on that can activate just under the text fields when you focus on the field and there is a validation error, which I use relative/absolute positioning to get in the right place, and so the position: relative declaration triggers the IE bug which causes the text field beneath the one that has the focus to occlude its drop down. My solution? I now have a javascript snippet that runs as the dom is ready that sets the z-index of all the containers(the one that has position: relative) of the text fields in descending order. So for example, if you have 10 text fields, the first one would have its z-index: 10 and the last one would have z-index: 1.

In a related note, if you use the relative/absolute positioning technique, the child element that has position: absolute is considered the child of the parent element that has position: relative in terms of z-index, which if you then have overflow: hidden on the parent element or its ancestors, the child element would get cut out if it's position is out of bounds, vs if you didn't have position: relative set on the parent element then the child would not get cut out because it's considered to be the child of the root element in terms of z-index.

The Hidden Singleton with Javascript

by airportyh posted at 08-23-2008 01:04PM - Comments (0)   javascript programming
This pattern is well documented else where, I think I first saw it in the Definitive Javascript book. Let's say you want to provide a function that does stuff with a static variable, but don't want that static variable to be global(visible to the outside)? You would do this:

myfunction = function(){
    var mysingleton = new Singleton();
    return function(x, y){
        return mysingleton.doit(x,y);
    };
}();


so now you can do myfunction(1,2) which in reality calls the doit method of the same instance of Singleton everytime. As a more concrete example, you can create a counter function like this:

counter = function(){
    var i = 0;
    return function(){
        return i++;
    };
}();


Each time you call counter() the result increases by 1.

A variation of this pattern is to extend an existing function/method by replacement. This is similar to the ruby practice of aliasing a method to a different name and then sticking a new method in it's place that at some point calls the original version. Anyway, for example, let's say you have an existing function from an existing library... say scriptaculous effects' Effect.Shake. You want to change the default options without having to rename the function(so that all your existing code that depends on it doesn't not need to change), you would do:

    Effect.Shake = function(){
        var original = Effect.Shake;
        return function(element, options){
           
original(element, Object.extend({
                distance: 5,
                duration: 0.5
              }, options || {}));
        };
    }();


So I've changed the default distance to 5 and duration to 0.5 throughout my app for any calls to shake() by adding this in one place only.

Relativize that thing: todays CSS war story

by airportyh posted at 08-22-2008 12:43AM - Comments (0)   css html javascript programming
Today, I had a seemingly simple problem: position a floating label element inside of a text input field, inside a elastic layout. To achieve an effect like:

Moreover, I wanted to implement this as a generic javascript widget that I can reuse throughout the application. This was hard, real hard.

At first, I just used position: absolute and set the position of the label relative the the root document, getting the coordinates required from the position and dimension of the companion text field. Well, this didn't work so well when the window is resized. Too bad we got an elastic layout, if we had a fixed width one I would have been done. My quick and dirty solution was to register the window.onresize callback and reset the positions of the labels whenever the window is resized. This actually worked well in IE, but not so in FF. The problem with FF is that the callback is invoked too sparsely, it's just not responsive enough when you are just dragging the corner of the window around, but you see these labels in all the wrong places while you are dragging around the corner of the browser until you leave your mouse still for a second, at which point the labels receive the event and jump to where they should be. This just plain looks bad, and although your users probably won't resize the window too much, I think it will definitely give the impression of a cheesy app.

So, my first attempt at a work around is a brute force one. I wrote my onresize callback functionality for FF that is more responsive. Here was the code.
 /* this is a funny hack in FF to get window resize events more often */
var ffonresize = function(){
    var listeners = [];
    var dim = document.viewport.getDimensions();
    var pe = null;
    pe = new PeriodicalExecuter(function(pe) {
        var nd = document.viewport.getDimensions();
        if (!(nd.width == dim.width && nd.height == dim.height)){
            dim = nd;
            listeners.each(function(l){l(nd)});
        }
    }, 0.2);
    return function(listener){
        listeners.push(listener);
    }
}();

/*
...
*/
if (Prototype.Browser.Gecko)
    ffonresize(this.__onResize);
else
    Event.observe(window, 'resize', this.__onResize);


This improved the situation, but... not good enough. Then I thought, there must be a way with position: relative to do it. After all, isn't that what it's for?

I tried making the label element position: relative, but this didn't work: you could shift the position of the element relative to where it would have been, but the space for where it would have been is still taken up, and now it's an empty space...
I did some research. One of the best resources is quirksmode again, this is also a helpful article. It turns out the common practice to use is to have a container that is relative and then a child under it that is absolute, which will cause the child to use the parent's position as the reference point. But trying the things out didn't work at first. After some trial and error, I figured out that, not only does the parent have to be position: relative, it also had to be display: block. My parent container happened to be table cells, so I either had to set their display to block or add a div under them. Setting their display to block didn't work for me because it completely messed up the layout of the table, so I had to go with the second method.

Adding the div was annoying because not only did I have to go to a few different places in the app to add the markup, it also made my widget depend on more. Of course, I guess I could inject the div dynamically, but I haven't tried that yet. So, this worked, I no longer had to use the onresize callback and reset the position of the labels, I just set:

position: absolute;
top: 2px;
right: 5%;


Top of 2px to give the top some space, right of 5% to give enough space for the label. Both are relative to the divs I newly added. Well, this kinda worked... except for my short textfields. See, I had long text fields that take up almost 100% of the width of the parent, but also shorter ones that take up about 60%, so while it worked well for the long textfield, sitting right inside it on the right edge, it was well outside of the short textfields. I wanted to use javascript to figure out what's the % of the width of the text field, which I could use to calculate where the label should be, but that number is in my css, and using javascript you can only get dimensions in terms of pixels, i.e., all the percentage info is lost.

What to do? Well, I calculated the percentage by doing a division between the text field and its parent's width in pixels, this is my code(using prototype.js):
right: (1 + 100 * (1 - this.field.getWidth() / this.field.up().getWidth())) + "%"

Yeah, it's pretty crazy, but it worked, beautifully. It works in IE7, FF3, and Safari, didn't work in Opera, but that could have been because other javascript bugs I had wrt Opera.

Maintaining Order in GAE

by airportyh posted at 08-15-2008 12:01AM - Comments (0)   appengine googleappengine programming
In a previous post I alluded to problems I encountered with using GAE's object model. This is the follow up to that.

In my todo list app, a main feature was to make reordering your todo list a breeze. When I wrote it with mysql, I used the standard relational database pattern to store an ordered list: create a field SEQ in the items table which holds the a sequence number(integer) that represents the order of the item in the list. But this means for every reordering I do, I have to update all active items in the list. I figured since I am only going to have small lists, it shouldn't matter, and it worked fine with mysql. When I did a port of this to GAE, reordering became terribly slow. A friend of mind told me that this is just a characteristic of object databases. My solution was to create a list field in the parent object(list), which remaps the sequence numbers into the correct order for the items. This way, updating the order only took saving the list object. Of course, this approach is more complicated. There are a couple of other possible approaches:
  1. As my friend mentioned: do a link list in the database. Instead of a SEQ field you would have a NEXT field which points to the item that's next to this one. This would make small reorderings(of the type: you take one object and move it to a different place in the list), a constant time operation, in terms of number of updates(3 updates total).
  2. Make the SEQ field a float, which would allow you to insert a item in between 2 other items with only one update. But because of numerical precision issues, sooner or later you would have to relabel the whole list for the SEQ numbers to be not too close together, which would be triggered like a garbage collection operation would be.
My current approach I think is about on par with these 2 in terms of complexity. I like the linked list solution because it's more scalable, you can really reorder/maintain arbitrarily large lists without missing a beat. For my app though, I only need to handle small lists, so I have no reason to switch for now