{"id":35,"date":"2008-04-14T03:42:03","date_gmt":"2008-04-14T03:42:03","guid":{"rendered":"http:\/\/robubu.com\/?p=35"},"modified":"2008-04-14T03:42:03","modified_gmt":"2008-04-14T03:42:03","slug":"scaling-databases-and-googles-app-engine","status":"publish","type":"post","link":"https:\/\/robubu.com\/?p=35","title":{"rendered":"Scaling Databases and Google’s App Engine"},"content":{"rendered":"

Assaf Arkin<\/a>: if anything I wrote sounds vaguely familiar because you somehow managed to dumb your RDBMS into storing structured data in BLOBs, added versions and timestamps on all records, grappled with minimizing transactions and locks, denormalized data like there\u2019s no tomorrow, or relied too much on a message queue, then time to rethink. Are you using a hammer to polish your china? (Tip: not a good idea, invest in soft cloth)<\/p>\n

<\/i>I know I am late (particularly as an IBMer) to the whole couchDB<\/a> \/ bigtable<\/a> thing<\/a>, but I have to admit that until I read Assaf’s article I didn’t really understand what all the fuss was about. <\/p>\n

I also didn’t understand the fuss about google’s app engine<\/a> and shared the view that it looked like “The World’s Worst Webhost<\/a>“. My current webhost<\/a> lets me run ruby, perl, php and<\/u> python with either mysql or postgres, so why would I want to limit myself to python and some proprietary, lock-in database. The answer is, of course, scale.<\/p>\n

And so here is the dilemma. If you are developing a new web application from scratch what do you do if you think \/ hope that one day you will be wildly successful? There is the flickr \/ facebook \/ myspace path i.e. use a single relational database till it breaks, then use master \/ slave and then shard (gory details for myspace<\/a>) and now there is the google app engine path i.e. build the application from the start in a way that is guaranteed to scale out with no herculean efforts. <\/p>\n

This is not an easy choice. The problem with the google path is that their datastore has no joins, limited transaction support and some random query limitations<\/a>. Now, I have never built an application with those limitations, but I would guess that it takes longer to build and requires a whole new way of thinking<\/a>.<\/p>\n

The google app engine path seems difficult to justify for a new app. It’s built on the premise that the app will be wildly successful and most web apps aren’t. Justifying the additional upfront expense to investors \/ management won’t be easy and given that many big names have already successfully traversed the other path, it may be better to start by choosing one of the world’s better webhosters and only invest in scaling the db when the money starts rolling in.<\/p>\n

<\/p>\n","protected":false},"excerpt":{"rendered":"

Assaf Arkin: if anything I wrote sounds vaguely familiar because you somehow managed to dumb your RDBMS into storing structured data in BLOBs, added versions and timestamps on all records, grappled with minimizing transactions and locks, denormalized data like there\u2019s no tomorrow, or relied too much on a message queue, then time to rethink. Are … Continue reading “Scaling Databases and Google’s App Engine”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false},"version":2}},"categories":[1],"tags":[],"class_list":["post-35","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/robubu.com\/index.php?rest_route=\/wp\/v2\/posts\/35","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/robubu.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/robubu.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/robubu.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/robubu.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=35"}],"version-history":[{"count":0,"href":"https:\/\/robubu.com\/index.php?rest_route=\/wp\/v2\/posts\/35\/revisions"}],"wp:attachment":[{"href":"https:\/\/robubu.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=35"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/robubu.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=35"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/robubu.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=35"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}