Tim's Experiences with Google AppEngine

At first it seemed promising.

Free hosting, sounded good for someone just learning web programming.  Infinitely scalable and fault tolerant.   Google makes the claim that "we wear the pager for you".   

I liked the way they'd thrown the SQL/RDBMS model out the window - I always thought that SQL/RDBMS's were old-fashioned.  I also appreciated their new approach to transactions:  basically they make very little attempt to implement transactions.  Transactions don't work well in a cloud environment, and are overrated anyway - perhaps with the exception of bank software.  AppEngine was available for Java and was integrated with IntelliJ (and Eclipse).   I didn't want the hassle of having to install & configure RDBMS software.  It all sounded right for me.


Then I tried it

I have a long list of things that I don't like about AppEngine.  

1. Chief of all is performance:  you can make at most 1..10 writes per object per second.   My application relates to high school timetables ( www.edval.com.au ).  I found that I was unable to upload a timetable within the 30 second limit allotted to a single page-request - regardless of whether I modeled it as one big object or lots of little objects.  I inquired at the AppEngine mailing list, and was told by a Google AppEngine engineer:  "If your application needs to make 120 writes in a single page-request, then you need to think about a fundamental redesign of your application".  I responded that this was only 200K of data - why can't the BigTable-based AppEngine datastore handle 200K of data in a page request?  There was an explanation of the difficulties of synchronising the writes across the 3 copies of each object...an explanation that left me unconvinced at a high level...this being the era where (a)  SmartsGroup ( http://www.smartsgroup.com ) can handle 2 billion transactions per day on a single server node, and (b) Oracle boast that their database can handle 22,000 transactions per second, and (c) the web software I subsequently wrote can upload the timetable in the blink of an eye.   
    It seemed to me that AppEngine was designed for implementing social networking sites.  My application is not a social networking site.  I deal with a couple of different access patterns, but it's all on a very small amount of data.   Maybe that was my mistake...writing an application which doesn't use much data, but then if Google AppEngine can't handle 100 writes per second then does this mean my app is too big or too small?

2. Despite the fact that Google AppEngine had been going for over a year, the software they provide you with to browse your data-store is extremely basic.  It looks like it was written by a 1st-year computer science undergraduate.  This was quite a struggle for me...learning the data-store programming model without being able to see the data except through a peep-hole.   I couldn't nuke my whole data-store, I couldn't clear a single table...one time I spent 20 minutes manually deleting rows from a table 10 at a time.

3. To use the AppEngine datastore programming API, it was a very steep learning curve.  It's quite unfamiliar and there are a lot of gotcha's.  But maybe I'm just a bit slow.  

4. For a newbie programmer, it didn't matter much that I couldn't install 3rd party libraries and tools.  However, I'm sure an experienced Java programmer would come to regret not being able to install their own tools.  I could see that this was going to ultimately bite me.   I can understand the rationale for locking down the AppEngine environment, so this is a concern about the basic concept of a locked-down application framework.

5. With some effort, I was able to point my own domain at AppEngine, however only with HTTP, not with HTTPS.  From the sound of things, this is a fundamental limitation, which won't be addressed any time soon.

6. I got a little bit worried that (a) it took me 48 hours for my app-id to start working, and (b) there was a significant outage of AppEngine during the time I was developing.

7. I was concerned about the legal implications of where the servers are located.  My customers have legal requirements that the servers be located in Australia.  This is not because they lack an understanding of the international nature of IT and hosting services, but because they need the servers to be under the jurisdiction of the Australian Federal Police.  I didn't actually investigate this issue, whether it's possible to limit my application to only Australian nodes or not.


Will someone else create a Cloud Application Framework?

I see the concept as promising.  It's attractive to have someone else "wear the pager".   A cloud database is probably very difficult to install, configure and maintain, and if this is tied to the application framework then I think it's probably a reasonable tradeoff (the lack of flexibility versus the benefits).  Maybe AppEngine is merely the first in a line of cloud application frameworks.


Dr Tim Cooper
tim at edval.com.au
Comments