About Shaping Clouds
This is the personal blog by Michiel Sikkes, partner and lead developer in Firmhouse. It's about building scalable web apps and services in the cloud.
Introducing Cloud Ideas
This blog is about inspiring people of what’s available in the cloud and how they can make use of it.
Today I’m going to start experimenting with the concept of “cloud ideas”.
I have a ton of ideas on new web services, online social games or mobile apps that make use of the web and things in the cloud. Some of these ideas evolve into a personal prototype, some as input for Firmhousr projects but there’s still a large gap to fill because I don’t have infinite time, and some ideas just aren’t interesting for Firmhouse.
I encourage you to take the ideas, build apps or businesses on it, discuss it with personal friends or business partners. Hell, you can start the next Google if you want to. I will license the text of the ideas under a Creative Commons license (note: not the idea itself!) This way I’d like to bring people together, and make cool stuff, socially, businesswise or technical.
After reading a cloud idea and you want to bring it further, feel free to contact me and I’d be happy to discuss it personally, over e-mail, skype or other means. Perhaps I can even connect you to other people.
Moving to an open world where we can inspire each other to create such great stuff :)
Pole in the sea
Pole in the sea, originally uploaded by michiels.
I love this picture. I took it on a trip to the Dutch coast in Scheveningen. I really like the variations in the cloud’s colors and the movement in the sea.
Kurhaus in Scheveningen
Mind your company growth
I really like this paragraph from the following blogpost
http://productblog.37signals.com/products/2008/12/getting-real-is-all-about-being-lean-and-agile-and-keeping-things-simple.html
“From my own experience I know it’s really easy to overcomplicate things, as your company grows you start adding various control mechanisms, and before you know it your implementing controls mechanisms to control your control mechanisms. Getting Real reminds small businesses why they should stay lean and agile, and why they shouldn’t try to overcomplicate things.”
Migrating very large databases
Currently, tweetburner.com has one table which is really big. We’re busy optimizing our database to make it more scalable and run faster on big calculations. For this, we need to set up multiple replication servers.
Normally you would just dump all tables into a sql or text file to copy them over to another database server. However, with a table of over 21 million (yes!) rows the SELECT query will lock the table for updates and therefore other reads so our service becomes unreachable.
By migrating your database into chunks you can make this process less painful.
We noticed that selecting 100,000 rows from a table it takes up to 0.8 second to run the query. If you select a million rows it can take up to 5 seconds. Apperantly the number grows exponentially. Keep in mind what selecting 21 million rows will take.
With a little scripting we can copy this table in about 22 times. 22 * 0.8 with 22 times a table lock of up to 1s this is less painful for your service and thus your users.
Conclusion: When migrating or backing up large tables, split them into acceptable chunks depending on row and table size so you won’t deadlock your service. Do this my selecting each 100,000 rows, put them in a dump file and copy them to your backup location. You can merge them after that.
The Sequel Pro client is really good
I really love the Seqel Pro MySQL client for Mac OS X. It’s the “replacement” of Cocoa MySQL.
You dan download it here: http://sequelpro.com
I’m going to try out Posterous in the next few days for posting little stuff that comes on my mind. They are harder to explain in Tweets (for example, code samples are attachted) but they are not as structured as a blog article with an introduction, core and closure.
Let me know what you think!
To bad the url shortener doesn’t support Tweetburner though :(
Best practice for account settings
I have a question to ask you. There are a lot of ways to implement account settings. I’ll name three here. What’s your favorite option?
On User model
In this settings type, all the settings are made available on the User model of your web application. So, for example, each setting has it’s own column on the users table. The settings can be fetched by using User.setting directly from your MVC framework.
The advantage of this is that it is easy to set up. The disadvantage is that it is non-scalable as hell. You would be ending up with 20+ columns in a users table.
Key-value pairs
This method is used by – for example – WordPress. They have a separate wp_usermeta table with the following relevant columns: user_id, meta_key, meta_value. This way, each user can have more settings than the default user stuff like password and username.
Disadvantage is that it will take more work to nicely bind all the keys as attributes to the User model. Advantage is that it is scalable.
Hash on the Model
A third option implements both. You would put one column named “settings” on the User model and fill it with a serialized text-based hash (ie. YAML) and deserialize it every time you read or save settings. The Rails plugin acts_as_configurable does this.
What’s your take on this?
Last Saturday, we at Firmhouse started our latest weekend project. The idea is simple: one of our ideas get’s developed in two days to a very “dirty” version to test a concept, get users and just have fun. This time it is the Twumult project. Twumult is a service that let’s you easily follow your favorite Twitter users on one or multiple dashboards. You can for example add a dashboard for your friends or for your co-workers. Check out the screenshot below.
If you want to participate in the development, you can do various things. We love to hear your ideas and thoughts:
- Go to twumult.com and just use the app!
- Follow us on our @firmhouse Twitter account for updates.
- Participate on the Firmhouse product blog.
- Leave your thoughts on our Get Satisfaction page.
- Or, just leave us a personal message :-)
The dashboard view of the first "dirty" version of Twumult
