Just for fun !!!

So today, just for fun, I had a wild idea !!!

Windows Web Server 2008 hosting IIS 7 running Python 2.7 + Tornado + Django 1.3 doing the same thing I was able to achieve with Ubuntu + Python 2.7 + Django + wsgi + Tornado + nginx !!!

I doubt the Windows performance will match that of Ubuntu however this has been all kinds of fun !!!

See also:  Running Django on Windows (with performance tests) !!!

I was originally interested in doing Django 1.3 with Windows Web Server 2008 and IIS 7 so what I found was pretty darned cool.

Point and click installations for all this stuff was nothing less than amazing !!!  Especially for Windows !!!  WTG Microsoft !!!

What I really want is WebDav so I can share my huge pile of files with myself and only myself via the Internet and WebDav seems to do the trick…  problem is Windows Web Server 2008 doesn’t seem to know how to do WebDav so I have to improvise a bit by making this do the trick as follows:

Tornado does wsgi !!!

Now IIS 7 does Tornado !!!

Easy as 1,2,3 !!!

Oh, and this puts me one step closer to having my own Private Cloud in my home !!!

 

 

 

Advertisements

Why does “crappy” code not perform worse than it seems to ?

I like good code like the next person however I am wondering why people spend so much time focusing on “crappy” code patterns that don’t seem to perform worse than expected ?!?

Here’s a thought:  If you want to complain about “crappy” code then why not fix all the crappy code you become aware of including all that stuff you cannot seem to understand !!!

Or maybe just realize the fact that your idea of “crappy” code may be another man’s treasure !!!   Oh wait, someone already thought of this when the following was coined, “…one man’s trash is another man’s treasure…”. Circa. 1570’s !!!

Or to put this another way…   those who feel some coding patterns may be “crappy” might just be limiting their own ability to perceive really useful code they would surely have overlooked !!!

I am NOT in favor of truly crappy code !!!

But, I am also NOT in favor of being so close-minded that I cannot see how useful alternative coding styles may be.

Single instances of “crappy” code cannot be crappy !!!

If you have to produce 1000’s or 1,000,000’s of iterations to see a performance problem from a single use chunk of code then there is very little reason to think about any such performance problem !!!

I might be impressed with those who rail against “crappy” code who also make darned sure all the code they can see in their field of vision is not also “crappy” code !!!

Scenario #1

For instance, consider the following pattern that was recently classified as being “crappy” by an intrepid band of hearty Python coders in an company we dare not name here…

This is “crappy” !

toks = ‘one two three four five six seven eight nine ten’.split()

This is not !

toks = [‘one’, ‘two’, ‘three’, ‘four’, ‘five’, ‘six’, ‘seven’, ‘eight’, ‘nine’, ‘ten’]

The crappy version was said to be “crappy” because the split was seen as being unnecessary because the non-crappy version was said to be the desired effect.

Those who said the crappy version was crappy have probably never had to load a list from a comma delimited jungle of data they may have had to spend several days entering manually by hand otherwise they may have gotten themselves some useful experience as to why the crappy version might be somewhat less crappy after-all.

Scenario #1 Benchmarks

I like benchmarks.  I like to know when code may perform badly at run-time at scale.

The crappy version for Scenario #1 runs 5x slower than the non-crappy version for 1,000 iterations.

The crappy version for Scenario #1 runs 7.6x slower than the non-crappy version for 10,000 iterations.

The crappy version for Scenario #1 runs 7.65x slower than the non-crappy version for 100,000 iterations.

The crappy version for Scenario #1 runs 7.66x slower than the non-crappy version for 1,000,000 iterations.

Um, if you turn-off the Python GC the performance issues seem to disappear for a while !!!  Just a thought…

Scenario #1 Analysis

The real question is this:  “Was the crappy code being used enough for the comments this code pattern elicited ?”  Probably not !!!

The justification for call the crappy version truly crappy was the performance concerns and there are some rather glaring performance concerns to be sure but ONLY when the crappy version was being used 1000 times more than it actually was being used.

Those who claimed the crappy version was “crappy” had to magnify the usage pattern by a minimum of 1000 times before the crappy version’s performance data might be measurable.

I agree the crappy version would be truly crappy if it were the actual source of some kind of measurable performance issue related to the loss of revenues or some other demonstrable effect that was actually causing some kind of problem.

The problem, as I saw it, had nothing to do with how crappy the code pattern may have been because let’s face it this is crappy code if it were to be used often enough for a problem to exist.

The problem, as I saw it, was a group of people all agreeing there was a problem where no problem really existed at-all simply because in their minds they were magnifying the problem by 1000 times just to be able to catch a glimpse of some kind of problem when there was no real problem at-all.

This piece of crappy code may have a real-world non-crappy use case that could have saved someone a lot of time, given the right set of circumstances related to having to maintain a huge data set by hand that had to be loaded into a list at run-time.  The desire to make this crappy-looking code non-crappy in a use case that could NEVER be actually measured as being crappy is the problem !!!  Far more time could have been spent entering all those commas and quote marks just to make the code less crappy than the effort would have been worth.

Why any person or group of people who are supposed to be intelligent talented software engineers would claim a harmless chunk of code was harmful given the actual use-case that existed in the actual source of the context for the original question which was related to a single use of the crappy version is well beyond my ability to comprehend in real terms.

The person who raised the issue was supposed to have more than 20+ yrs programming experience !!!  He found a single reference to the crappy version in a source file that was probably being used exactly once per iteration of some larger program.  WOW !!!  Talk about yelling “FIRE” in a crowded room !!!

The people who agreed with him were even more of a mystery because these people are supposed to be among the best and the brightest at this particular unnamed company and they went along with the idea that was raised by the one guy who should have known better than to yell “FIRE” in a crowded room.

It is interesting to note, these same people who were able to inflate a potentially crappy use-case beyond the original scope are the same people who are seemingly okay with all the following truly crappy coding patterns they seem to wish to do nothing about:

  • An Eager-loading ORM that maintains and uses exactly 1 Database Cursor per Session !!!
    • Why was this NEVER changed ???

I will stop here with the analysis because I think this one point bears further analysis.

How in the world do these crappy detecting software engineers allow an Eager-loading ORM to exist in the first place ???   And the company wants this sort of thing corrected !!!

I have to wonder about the skills these crappy detecting software engineers actually possess when they cannot find a way to remove the Eager-loading ORM in the first place !!!!

Removal of the Eager-loading ORM would be easy enough, for me to accomplish, but then I can tell the difference between crappy code that is really crappy versus crappy code that only seems to be crappy.

Well You See Timmy…

Now for the moral of this tale…

People who live in glass houses always seem overly eager to throw stones when their own houses have cracks in the walls so wide everyone knows there are problems.

I have no issues with people who can see imagined problems that don’t exist so long as they have their own houses in order but this was not the case in this instance.

These very same people, who seemed more than willing to detect crappy code patterns where there was no crappy code use case are the very same people who seem unwilling or unable to resolve glaring performance issues in a huge pile of code.

The rest of the issues these people could focus on are as follows:

  • mod_python rather than wsgi
  • Django not being used-all but then there is no discernible web framework being used at-all.
  • Eager-loading ORM – easy to resolve with Django.
  • Non-scalable Web App – because mod_python is being used rather than wsgi, for instance.
  • Development environment issues – all developers share a single instance of the run-time – each developer gets a different virtual host but all development being done in a single Linux instance.
    • Okay, this one truly baffles me !!!
    • How difficult can it be to get each developer their own Linux instance at a moment in time when everyone has a Cloud-based solution for doing this ?!?

Look, all these issues can be easily handled but none of them are being handled at-all.  Why ???

The reason(s) all these glaring issues are not being handled is easy… lack of experience and lack of skill in the developer community.

Nobody wants to make any real changes or nobody is able to make any real changes.

Dragging along ancient code from the deep past and then being either afraid to update it or unwilling to update it is more than ridiculous !!!

Solutions !!!

The solution for all this is also easy but very difficult to implement !!!

Rewrite your code every 18 months !!!

Better tools are being churned-out all the time.

Django is a proven Web Framework !!!

Wsgi is a proven technology stack !!!

Python+Django+tornado+wsgi+nginx equals a scalable Web App that scales as easy as you can build an automated process for spinning-up one more Linux Virtual Machine in the Cloud !!!

Or let’s put this another way…

Python+Django+tornado+wsgi+nginx was easy enough for me to handle all by my self – not that I might not have wanted to do this with a team of others – there just weren’t that many others I might have done this with.

The moment I achieved my first stable installation of Python+Django+tornado+wsgi+nginx I knew it was the way to go !!!

Python+Django runs as a separate wsgi web server with performance comparable to what you get from the Google App Engine, oddly enough, and “yes” I have run benchmarks that tell me this based on the data.

Tornado is a stand-alone Python-based Web Server with very good performance characteristics.

Tornado talks to an instance of a Python+Django web app via wsgi.

Nginx talks to an instance of Tornado that talks to an instance of a Python+Django web app via wsgi.

Why so many web servers in this stack ???

Why use Tornado at-all ???

I happen to know a little something I call Latency Decoupling that tends to make web pages serve much faster the more layers of web servers you use.

Nginx connected to many Tornado servers each connected to one or more wsgi Web Apps is far more efficient serving web content than Nginx connected directly to that very same wsgi web app.

Latency Decoupling kicks-in and your end-users have happy faces.

Ability to Scale the Web App also increases !!!

Many instance of the Web App within each Tornado instance !!!

Many Tornado instances within each Nginx instance !!!

Deployment gets easier !!!

Now with a single Python or Ant script you can spin-up yet another Amazon EC2 instance – connect-up the Nginx instances using a Load Balancer of some kind (nginx also does Load Balancing) and before you know it you have architected a really cool Django Cloud Solution that nobody else seems to have just yet.

Build a slick Control Panel for your Django Cloud Users and bingo you have the ability to grow a Web App from a single instance to any number just by clicking a button on a web page !!!

The only other detail would be how you monetize all this into something you can use to generate revenue.

All of this was easy enough to build when you have all the parts.

All of this should be easy enough for any company to use, if only they had I.T. staffers who had played around with these kinds of solutions but alas that seems to be lacking in most companies except for a few.

Too bad most would-be skilled programmers would tend to scoff at most of what’s written in this article as being some form of “crazy”… but then once upon a time the notion of generating electricity was also seen as being “crazy” along with the notion of gravity and quantum physics.  I can live with what others wish to say… so long as I get to build something really cool along the way.

Automatic SEO Optimization –> Test Drive it today !!!!

When Any and All URLs based on the CargoChief.Com domain leads the end-user to the CargoChief site you are free to explore SEO (Search Engine Optimization) simply by using whatever URLs you wish to use such as the following:

http://www.cargochief.com/book/your/next/shipment/through/us/

or

http://www.cargochief.com/helps/you/connect/with/shippers/

or

http://www.cargochief.com/gets/your/pallets/shipped/today/

or

Whatever your marketing people might wish to use… you can feel free to be as creative about your message and how your URLs are positioned in whatever search engines you may wish to use…

Yet another useful innovation you may wish to begin using today.

Go ahead and give it a try…  the URL you bookmark will be the SEO-friendly URL rather than the actual end-point even when the actual end-point takes you to the CargoChief Cloud.

Only works with http://www.CargoChief.Com or CargoChief.Com domains.

Never see another 404 page again !!!

Brought to you by Vyper Logix Corp, innovation for the 21st Century ad beyond.

Yet another Ruby on Rails Programming Test *yawn*

So here we are again with yet another Ruby on Rails Programming Test !!!

Maybe someday people will just take as proof everything else I have publishing here but that day has apparently not yet arrived…

The Problem – Not exactly any kind of real business problem…

Rails Exercise:

We’d like you to create a simple Rails 3 application.
Create models that allow a user to follow another user.
The user model’s only attribute is “name”. You may scaffold the create action.
Create an interface that accepts post requests that allows users to
“follow” other users.
The user’s “index” action should list all of the users.
The user’s “show” action should show:
1) The user’s name
2) The users the user is currently following (with a button to remove
that following)
3) The users the user is not following (with a button to add that following)
4)  The users currently following this user
Do not create a login system for this exercise.

The Solution

Download it from this link; only 600 KB  !!!

Ruby on Rails with Cherokee & Passenger for Ubuntu 11.04

Sometimes people get the feeling I just don’t like Ruby on Rails because the name “python” appears in my URL for this Blog… well I do like Ruby however since Ruby is such a performance HOG I have to temper my enthusiasm with a certain sense of reality by knowing how to do things Ruby really sucks at by using Python and other languages.  Every single thing that might cause Ruby on Rails to crash and die or simply fail to meet the performance criteria are the exact same things that NEVER cause problems for Python/Django.  You can say what you want about whatever you want but… Ruby is NOT the fastest gal on the dance floor although she has a ton of enthusiasm and wants to dance all night, Ruby just cannot dance as much or as fast as some of the other gals in the room.  Ruby will make you feel like she is a fun gal to be with but… at the end of the day, Ruby is just too much of a big fat performance HOG.  The Ruby Community needs to do a whole lot more work to make Ruby more nimble on the dance floor however to be fair Ruby has been getting more nimble due to the contributions of the Phusion folks who have given us Ruby Enterprise Edition and Passenger even though, and I recall this clearly because I was doing the Ruby thing when this was taking place in 2008, the Phusion folks held-back Passenger when it was new so they could get some cash for their efforts – so much for the Open Source Movement.  All things being equal, and they rarely are, I do like Ruby on Rails as much as any guy could enjoy dancing with a over-weight gal who has two left feet because after-all sometimes it is those over-weight gals who can be more fun at the party even though we guys all want fashion models it’s just not all that realistic to hold-out for your typical fashion model who would not give us mere mortal geeks the time of day let alone date us or whatever one does after the party these days.

The Challenge…

Recently someone told me there were “issues” while installing and using Ruby on Rails with Phusion Passenger for Ubuntu 11.04 however… as is generally the case with the work I do, I did not notice any problems but then I rarely notice “issues” since I take them in-stride and press-on to get the work done; actually I enjoy the little bumps along the way.

Step #1

Install Ubuntu 11.04 in Virtual Box or VmWare.

Make sure you choose to install “ssh” or you may have to issue the following command once the installation completes so you can use SSH to “talk” to your Ubuntu Appliance.

apt-get install openssh-server openssh-client

Step #2 – Install Cherokee Web Server

sudo apt-get update

sudo apt-get upgrade –V

sudo apt-get install python-software-properties

sudo add-apt-repository ppa:cherokee-webserver/ppa

sudo apt-get update

sudo apt-get install cherokee –V

You can test it using the following:

sudo cherokee-admin -bxxx.xxx.xxx.xxx -pxxxx -T32

nohup cherokee-admin -bxxx.xxx.xxx.xxx -pxxxx -T32 &

Step #3 – Install Ruby Enterprise Edition

wget http://rubyenterpriseedition.googlecode.com/files/ruby-enterprise_1.8.7-2011.03_i386_ubuntu10.04.deb

sudo dpkg -i ruby-enterprise_1.8.7-2011.03_i386_ubuntu10.04.deb

ruby -v

Step #4 – Install MySQL and Subversion

Okay this is where there was a minor bump but nowhere near what I would call an “issue”.

You will not encounter any bumps or issues if you follow this guide – the bumps and issues are all gone.

apt-get install build-essential curl bison openssl libreadline6 libreadline6-dev zlib1g zlib1g-dev libssl-dev libyaml-dev libsqlite3-0 libsqlite3-dev sqlite3 libxml2-dev libxslt-dev autoconf libmysqlclient-dev

What does all this do ?  Well it’s a bit of Ubuntu magic but it works.  I had to spend a bit of time doing some research however since I have been down this sort of path many times I was happy to learn something new and not only because someone told me this might be difficult and I just cannot resist doing something someone said was “difficult” because it never is, for me…

apt-get install mysql-server

apt-get install subversion

Step #5 – Check-out your Source code from SVN

Hey, this one is up to you but here’s a hint that will probably work for you.

Make a directory in your non-root user account, call this whatever you wish and put it where you wish.

Check to make sure the “bundler” gem is installed by using the following command:

gem list –local

If you don’t see that “bundler” gem you will have to issue the following command:

sudo gem install bundler

Then check-out your source code using:

svn checkout https://place-the-url-for-your-source-code-here folder-name-goes-here –username username-goes-here

Then issue the following commands:

cd folder-name-goes-here

bundler update

Right here is where you might have encountered some kind of “issue” with the mysql2 gem due to some kind of problem with building the native gem however that won’t happen if you used the directions found in this article. (If your Rails code doesn’t use MySQL then you are on your own, my source code does and it was fine…)

Step #6 – Start-up your Passenger Standalone Instances

cd folder-name-goes-here

passenger start

The cool thing about using Passenger Standalone is that it uses nginx and this means you can reverse-proxy your way to health and happiness by using Cherokee as the reverse-proxy.  Cherokee is a lightweight Web Server that is faster than Apache 2.x while using less RAM (it’s the uses less RAM thing that is more interesting since you will want to use your RAM to pack-in as many Passenger Standalone Instances as possible since Ruby on Rails is such a performance hog…).

Step #7 – Create some slick scripts for your Application Server

Write some slick scripts in the /etc/init.d directory to automate the start-up and shutdown of your Application Server.

Step #8 – Install Monit and Munin

See the details here. You will have to work-out the nitty-gritty details as to how to make this work for your specific needs.

Monit can be used to detect when your Rails Instances need to be killed and they will need to be killed whenever they run amok by allocating too much RAM or running away with too much CPU.

The one problem Monit may not be able to detect is whenever your Rails Instances cease to function as they sometimes might, from time to time.

Step #9 – Install Webmin, just for fun !

See the details here.  Once again, you will have to do some of the work to make this work for your needs.

Conclusion

At the end of the day, it just does not make any real sense to deploy your 20th Century Web Server any more because Cloud Computing is here and here to stay.

The Google App Engine makes Ruby on Rails make sense and not only because jRuby strikes my funny bone in just the right way, but then so does PHP for Java.

On the other hand… not everyone in Corporate America knows about or is willing to use the Google App Engine – they would rather use Amazon EC2 and pay for that resource than possibly get it for FREE from Google or maybe I am the only one who likes the challenge of getting it for FREE from Google rather than paying Amazon some real money.  Who is to say… I do enjoy the Challenge of seeing just how far I can take Google along for the ride before they mug me for money… LOL

 

%d bloggers like this: