Comments on: Building a high volume app without a RDMS or Domain objects http://thebull.macsimumweb.com/building-a-high-volume-app-without-a-rdms-or-domain-objects/ Thoughts on poker, programming and other stuff. Sun, 07 Sep 2008 14:36:07 +0000 http://wordpress.org/?v=2.6 By: Joe http://thebull.macsimumweb.com/building-a-high-volume-app-without-a-rdms-or-domain-objects/#comment-6915 Joe Mon, 05 May 2008 15:45:08 +0000 http://thebull.macsimumweb.com/2007/04/04/building-a-high-volume-app-without-a-rdms-or-domain-objects/#comment-6915 Hello, Your RSS button seems kinda tiny. Can you make it a little bigger? Thanks, Joe Hello,

Your RSS button seems kinda tiny. Can you make it a little bigger?

Thanks,
Joe

]]> By: DAR http://thebull.macsimumweb.com/building-a-high-volume-app-without-a-rdms-or-domain-objects/#comment-2276 DAR Thu, 23 Aug 2007 18:48:16 +0000 http://thebull.macsimumweb.com/2007/04/04/building-a-high-volume-app-without-a-rdms-or-domain-objects/#comment-2276 One other issue with this setup from a more practical perspective - the choice of S3 itself carries some potential negative consequences with it: * It *does* have outages from time to time. If you're using it as the backing store of a high-volume web app, then when it goes down your app goes down. Probably best not to tie your site's reliability to AWS's. Most web companies that use S3 (e.g., SmugMug) use it as backup or "cold" storage for that reason. * There's propagation latency inherent to S3. So if one of your EC2 instances stores something to S3, other instances might not see it for a while until it propagates through. One other issue with this setup from a more practical perspective - the choice of S3 itself carries some potential negative consequences with it:

* It *does* have outages from time to time. If you’re using it as the backing store of a high-volume web app, then when it goes down your app goes down. Probably best not to tie your site’s reliability to AWS’s. Most web companies that use S3 (e.g., SmugMug) use it as backup or “cold” storage for that reason.

* There’s propagation latency inherent to S3. So if one of your EC2 instances stores something to S3, other instances might not see it for a while until it propagates through.

]]> By: Thoughts on Computing » Blog Archive » Scaling Digg … Shards and the DB http://thebull.macsimumweb.com/building-a-high-volume-app-without-a-rdms-or-domain-objects/#comment-1244 Thoughts on Computing » Blog Archive » Scaling Digg … Shards and the DB Wed, 09 May 2007 13:19:49 +0000 http://thebull.macsimumweb.com/2007/04/04/building-a-high-volume-app-without-a-rdms-or-domain-objects/#comment-1244 [...] As Frank Sommers discusses in this post and Robert McIntosh contends in here, much can be done for apps like Digg without even thinking of the database. While their comments start along the right path (even though I don’t agree with all of their observations), they necessarily pull up short of what’s possible. You may be wondering why I say “necessarily pull up short” … and that’s definitely a fair question. [...] [...] As Frank Sommers discusses in this post and Robert McIntosh contends in here, much can be done for apps like Digg without even thinking of the database. While their comments start along the right path (even though I don’t agree with all of their observations), they necessarily pull up short of what’s possible. You may be wondering why I say “necessarily pull up short” … and that’s definitely a fair question. [...]

]]> By: Bob Lozano http://thebull.macsimumweb.com/building-a-high-volume-app-without-a-rdms-or-domain-objects/#comment-1157 Bob Lozano Mon, 30 Apr 2007 16:03:51 +0000 http://thebull.macsimumweb.com/2007/04/04/building-a-high-volume-app-without-a-rdms-or-domain-objects/#comment-1157 Many good thoughts in this thread, interesting to consider in light of how Digg and other similar sites are scaling etc. I think that assuming no db is a great formalism (see where it takes you and all that), but not the whole picture. In <a href="http://www.appistry.com/blogs/bob/architecture/scaling-digg-shards-and-the-db/" rel="nofollow">this post</a> I start to think down the "this is good, what else might help" parth a bit. Thx for exploring this point - it's definitely time to figure out how to scale the both the data layer and the apps that depend on them cheaper & easier. Many good thoughts in this thread, interesting to consider in light of how Digg and other similar sites are scaling etc. I think that assuming no db is a great formalism (see where it takes you and all that), but not the whole picture. In this post I start to think down the “this is good, what else might help” parth a bit.

Thx for exploring this point - it’s definitely time to figure out how to scale the both the data layer and the apps that depend on them cheaper & easier.

]]> By: Putting my non-RDMBS idea to the test » Thinking Outloud http://thebull.macsimumweb.com/building-a-high-volume-app-without-a-rdms-or-domain-objects/#comment-1143 Putting my non-RDMBS idea to the test » Thinking Outloud Fri, 27 Apr 2007 21:36:14 +0000 http://thebull.macsimumweb.com/2007/04/04/building-a-high-volume-app-without-a-rdms-or-domain-objects/#comment-1143 [...] I’ve been investigating the Amazon S3 service for while now and even opined about building a scalable application that used that service for data storage instead of a relational database. In my article, Building a high volume app without a RDBMS or domain objects, I talked about using a relatively simple flat file format such as XML to store the data in S3 and combined with the right kind of caching, have a reliable, performant and scalable application. [...] [...] I’ve been investigating the Amazon S3 service for while now and even opined about building a scalable application that used that service for data storage instead of a relational database. In my article, Building a high volume app without a RDBMS or domain objects, I talked about using a relatively simple flat file format such as XML to store the data in S3 and combined with the right kind of caching, have a reliable, performant and scalable application. [...]

]]> By: The Bull http://thebull.macsimumweb.com/building-a-high-volume-app-without-a-rdms-or-domain-objects/#comment-1125 The Bull Thu, 26 Apr 2007 00:05:46 +0000 http://thebull.macsimumweb.com/2007/04/04/building-a-high-volume-app-without-a-rdms-or-domain-objects/#comment-1125 Thanks for the comment Steve. I completely agree that RDBM's have lots to offer and you mention only a fraction of them. My thoughts were along the lines of those apps that don't need the things that a RDBMS is really good at, like transactions. Databases can be good for caches (such as MySQL's in memory tables and such), and other things that can be done with other systems. relational databases are just so common these days that I was curious about other alternatives. Thanks for the comment Steve. I completely agree that RDBM’s have lots to offer and you mention only a fraction of them. My thoughts were along the lines of those apps that don’t need the things that a RDBMS is really good at, like transactions. Databases can be good for caches (such as MySQL’s in memory tables and such), and other things that can be done with other systems.

relational databases are just so common these days that I was curious about other alternatives.

]]> By: Steven Wilmot http://thebull.macsimumweb.com/building-a-high-volume-app-without-a-rdms-or-domain-objects/#comment-1124 Steven Wilmot Wed, 25 Apr 2007 23:24:34 +0000 http://thebull.macsimumweb.com/2007/04/04/building-a-high-volume-app-without-a-rdms-or-domain-objects/#comment-1124 Although I'm a database consultant (and make a living from advising clients on appropriate use of an RDBMS, so this article was an interesting read. I agree with Jon Frisby's earlier comments that discarding the database entirely isn’t necessarily a good thing. For large systems like the ones mentioned, each technology has its own place. - File-system storage and storing multiple copies of data (stored initially by a natural key) are a good way to start. - Large-scale caching systems will read-only replicated copies of the database (or appropriate sections of it) also have their place. - However, even in these situations, RDBMS systems still offer advantages such as proper transacional locking on concurren access (perhaps of configuration-settings or financial management) Many articles that I've read today seem to start by suggesting that their particular design paradigm is (or could be made) applicable as the solution to too many problems. The successful systems seem to be the ones that don't try a "one solution fits all" approach, but instead only use RDBMS systems in appropriate places, making the bast use of caching, queued-updates, and replication, using each technology to its own advantage. Although I’m a database consultant (and make a living from advising clients on appropriate use of an RDBMS, so this article was an interesting read.

I agree with Jon Frisby’s earlier comments that discarding the database entirely isn’t necessarily a good thing.

For large systems like the ones mentioned, each technology has its own place.
- File-system storage and storing multiple copies of data (stored initially by a natural key) are a good way to start.
- Large-scale caching systems will read-only replicated copies of the database (or appropriate sections of it) also have their place.
- However, even in these situations, RDBMS systems still offer advantages such as proper transacional locking on concurren access (perhaps of configuration-settings or financial management)

Many articles that I’ve read today seem to start by suggesting that their particular design paradigm is (or could be made) applicable as the solution to too many problems.

The successful systems seem to be the ones that don’t try a “one solution fits all” approach, but instead only use RDBMS systems in appropriate places, making the bast use of caching, queued-updates, and replication, using each technology to its own advantage.

]]> By: Scaling Without A Database « François Schiettecatte’s Blog http://thebull.macsimumweb.com/building-a-high-volume-app-without-a-rdms-or-domain-objects/#comment-1122 Scaling Without A Database « François Schiettecatte’s Blog Wed, 25 Apr 2007 22:03:11 +0000 http://thebull.macsimumweb.com/2007/04/04/building-a-high-volume-app-without-a-rdms-or-domain-objects/#comment-1122 [...] Scaling Without A Database Before I wrote up an article entitled “Scaling and Uptime”, I came across another article by Frank Sommers entitled “Scaling Without A Database” which is well worth reading. It is a take on another article by Robert McIntosh entitled “Building a high volume app without a RDMS or Domain objects.” [...] [...] Scaling Without A Database Before I wrote up an article entitled “Scaling and Uptime”, I came across another article by Frank Sommers entitled “Scaling Without A Database” which is well worth reading. It is a take on another article by Robert McIntosh entitled “Building a high volume app without a RDMS or Domain objects.” [...]

]]> By: Pragmatic Dictator » Blog Archive » When Esoteric Becomes Mainstream http://thebull.macsimumweb.com/building-a-high-volume-app-without-a-rdms-or-domain-objects/#comment-1069 Pragmatic Dictator » Blog Archive » When Esoteric Becomes Mainstream Sun, 15 Apr 2007 17:17:08 +0000 http://thebull.macsimumweb.com/2007/04/04/building-a-high-volume-app-without-a-rdms-or-domain-objects/#comment-1069 [...] There’s been a lot of chatter recently in the blogosphere about the technical direction of the database. This discussion has been ongoing for some time and dates back to at least Bosworth’s utterings. [...] [...] There’s been a lot of chatter recently in the blogosphere about the technical direction of the database. This discussion has been ongoing for some time and dates back to at least Bosworth’s utterings. [...]

]]> By: The Bull http://thebull.macsimumweb.com/building-a-high-volume-app-without-a-rdms-or-domain-objects/#comment-1060 The Bull Fri, 13 Apr 2007 21:11:37 +0000 http://thebull.macsimumweb.com/2007/04/04/building-a-high-volume-app-without-a-rdms-or-domain-objects/#comment-1060 The replicate to a read-only db is rather common, and one could argue that you could accomplish this withing MySQL without having to actually replicate since MySQL's default engine is optimized for reads anyway. Replicating would distribute the load however. I've seen how eBay partitions their system and it is pretty slick and complicated. As for keys, this is one of the reasons I don't like DB generated keys. The replicate to a read-only db is rather common, and one could argue that you could accomplish this withing MySQL without having to actually replicate since MySQL’s default engine is optimized for reads anyway. Replicating would distribute the load however.

I’ve seen how eBay partitions their system and it is pretty slick and complicated. As for keys, this is one of the reasons I don’t like DB generated keys.

]]>