Riddle me this… ?!?

How can a language with half the speed give 200% better performance ?!?

Facts:

#1 – Java is supposed to run way faster than Python.

#2 – Java servlets are single threaded due to complexity of threading model.

#3 – Python can more easily deal with real OS threads. (No link required because this is Axiomatic.)

Fact #2

Whenever Java programmers talk about Java, in hushed low voices, in relation to performance problems they are having with Java – not that anyone wants to talk about any perceived performance issues where Java is being used… they have to acknowledge the undeniable fact that Java is simply single-threaded.

Java is only single-threaded because the code required to make Java multi-threaded is so god-awful complex that nobody would want to have to deal with it AND the requirement to write thread-safe code.  Actually thread-safe code has gone the way of the dodo bird in Corporate America.  Why ?  Because your typical programmer (I hate to use the term “Software Engineer” here because most programmers are no more an “engineer” than milk is carbonated, which as we all know milk is not at-all carbonated unless you can find fizzy milk and as we all know when you walk through your local grocery store you ain’t gonna see much carbonated milk and hence there are very few real software engineers in Corporate America contrary to popular belief and all self-delusions aside…).

Thread-safe code is about as fun to write as Drano is to drink, for most Java programmers (aka. code slingers).  Let’s face it, most Java coders hate to solve real problems when they can have so much more fun writing Java code.  Why spend all that time working-out how to make your code thread-safe when there is very little reason to do so since each Servlet instance runs in its own thread anyway ?!?

Fact #3

Anyone who has spent enough time with Python to learn how it works knows Python handles thread as easy as 1..2..3.. which means Python wants to be multi-threaded.

Python has that really cool Global Interpreter Lock which to the casual uninformed coder seems like a really bad idea until you figure-out it really does help you write thread-safe code without so much as having to even think about whether or not your code is in-fact truly thread-safe.  Each thread gets to execute a certain number of instructions before the next thread gets control and so on until all threads have had their turn.  The GIL ensures there is never more than exactly 1 thread being executed at the same time.  In essence the GIL makes multi-threaded code pretty much automatically thread-safe while allowing software engineers to engineer their software using multiple threads.

Stackless Python !

Stackless Python makes micro-threading easy and is in-fact easier to code than Python OS threads.

Stackless Python is at the heart and core of one very popular and powerful MMORPG known as Eve Online.

Why has nobody ever coded an MMORPG using Java?

Java-based MMORPG…  Okay maybe there are some Java-based MMORPG Engines out there; supported by a ton of C++ code rather than pure Java code.  But… Why are there no Java-based MMORPG games I want to play ?  None are as popular as Eve Online !  Eve Online is the ONLY MMORPG game that features a single unsharded Game Universe !  (Eve Online appears at the top of that search, doesn’t it ?!?)

Why do skilled Java coders not push their Java servers to the hilt ?

So who even knows what the “hilt” is where Java is concerned anyway ?

Typical Java coders who spend their time in J2EE-ville are far less concerned about pushing the performance envelope than in just writing some pretty easy to understand Java code… so because very few Enterprise-level Java coders know what the “hilt” is none of them would know how to push it to the “hilt”.  Also pushing it to the “hilt” would require some pretty insane coding and who wants to have to do that anyway ?!?

Python makes it easy to push it to the “hilt” !

Let’s face it, when you have as rich a set of data types to use when writing your code you can very easily find yourself pushing your code to the “hilt” and beyond.

Python makes the job of writing very efficient code as easy as possible without losing your ability to handle complex coding problems.

Java, as a language, is not about coding complex problems easily.  Java is all about readability and making code easy to understand.  Java coders don’t even think in terms of writing efficient complex code – they are all about just making the code work about as simplistically as possible.  Java does not make data manipulations fast, efficient and powerful the way Python does, IMHO.

Obviously I would rather code Python than Java !

But like everything, there is a reason for this… Nuff said !

Advertisements

Charting Stress Test for Flash and the Google Cloud

This is a story about LitePoint.Com and how they need to hire someone who can resolve their terribly messed-up Flash Builder 4 Code that cannot perform real-time charting of only 5000 data points.

Once upon a time a skilled software engineer went on an interview with the aforementioned company who said they wanted to hire someone who has worked with Charting Apps that chart a bunch of data points, in this case some 5000 every 250 ms.  The skilled programmer told them he could do the work.  They asked what past coding projects could they view as proof of the skill our rugged Hero said he had.  All went well for the first part of the interview until… The two guys who had created their problems had their chance to interview our rugged Hero.  Neither of those guys had any experience to speak of with Flash or Flex (Flash Builder 4 is really Flex).

One of these guys was a C++ programmer and neither knew anything about Object Finalization, as if knowing how to cause objects to be garbage collection is all that big of a deal – which it is not, of course.  Let’s forget the fact that one need not be the least bit concerned about either object finalization nor garbage collection when performing real-time charting of 5000 data points every 250 ms – because none of that is the least bit required.

Let’s also forget the fact that one need not create a bunch of objects to get 5000 data points charted every 250 ms; unless someone tried to build their own charting display objects and they had no clue what they were doing.

Let’s also forget the fact that not only can Flash Builder 4 produce Apps that can chart 5000 data points every 250 ms one can also easily optimize this system to handle far more than just 5000 data points every 250 ms.

Let’s also forget the fact that our two llama C++ guys who caused their own problems were trying to convince their bosses that Flash Player cannot handle charting 5000 data points every 250 ms, as-if the only way to get this done would be to code it in C++.

Let’s also forget the fact that if one really wanted to one could shove C++ code into a Flash App to get the performance up, if one had to do so.  It’s called Alchemy, not that those two Llama’s had any clue about this aspect of Flash anyway.

So now that we have chosen to ignore all the facts we can focus on the solution which by the by only required 12 hours time to build but then this is how a skilled programmer handles such trivial tasks as charting 5000 data points every 250 ms.  This is worth 12 hours because this is a non-issue for Flash Builder 4.

See also: Charting Demo – go ahead and download it.  The published version handles 5000 data points every 250 ms with ease.  The simplest of optimizations could easily boost performance to 20,000 data points every 250 ms with ease.  Additional optimizations would result in 50,000 or more data points charted every 250 ms, also with ease.  Just in case Flash Player might prove too slow one could always achieve this goal using any number of techniques capable of displaying 30 charting updates per second by using some optimization techniques any suitably skilled programmer should be able to manage with ease.

Anyway, Charting Demo took 12 hours to build – time-permitting some additional features may be added such as a slick Dashboard with gauges and dials to show the performance.  It would also be nice to boost the performance as high as possible just to prove this can be done with Flash.

Moral of the story is… if you need to hire someone to fix your problems you might as well hire the guy who says he can get the work done… who knows he might even get it done right away and save you some money OR you can just live with your problems and keep on interviewing until you find Mr. Perfect who may or may not be able to get the job done.  Either way, this is nothing but a trivial problem for those who are suitably skilled – the rest cause the problems without knowing how to fix them.

BTW – The skilled programmer who wanted to fix these problems for that wayward company has also NEVER created a problem he could not fix – hey, I am talking about myself anyway in the 3rd person – this is a story so deal with the literary stuff however you wish… those who create problems they cannot fix should also not piss those off who can fix whatever problems anyone may care to create.

It should be noted Charting Demo was created with a couple Frameworks (Flex and the Google App Engine) that don’t exist anywhere else; Flash Builder 4 for the client and Python/Django 1.1 for the server.  Reusable code make Application Development a breeze – 12 hours is pretty quick for any development effort and this one has some pretty cool features – all those features are 100% reusable.  No reason to spend time reinventing the wheel – unless you are in Corporate America and cannot figure-out how to make your code reusable.

AmazingFacts.Org sells 50,000 units of the Product produced by Ray C Horn !!!

Their development investment was less than $50,000.00 !!!

Not bad for a product that was produced for Windows and Mac in English and Spanish based on a prototype that was produced during the first 30 hours of development !!!

Try as hard as you might you cannot beat this value in any single individual contributor !!!

Way to go Ray !!!

PDFXporter Project Diary :: Rapid Project Development :: Day #12

Read the Disclaimers and Warranties and download the Pre-Alpha now

Day #12 – Billable Hours #1 -4

Reworking the Security Model a bit at this time

Making sure the Security Model is a bit more secure… And now the Security Model is more secure.

The Goal now… Export PDF Bank Statement as .XLS or .CSV

Now I can focus on exporting the information (ASCII Text) from a PDF Bank Statement as Data in the form of .XLS or .CSV.

Day #12 – Billable Hours #5 -8

Beginning to get data from Bank Statement PDFs

The process has begun… getting data… building the GUI… fleshing-out the API… building the Cloud Support.

Google App Engine requires API Requests to complete within 10 secs.

Much processing is required to handle the conversion of ASCII Text to Data for each page of the Bank Statement PDF. The process must be carved-up into a pipelined process for Personal Version Users who are running the App from the Cloud. It may be necessary to process the ASCII Text into Data via the Client rather than the server however for now the process is being run in the Cloud since code can be reused from a previous effort for this purpose; and let’s face it, Python is a much better language for processing text into data than AS3.

Project Recap so far…

During the previous 12 days of development the following has been accomplished by one single individual contributor:

  • Reusable Cloud-based versioned API has been created, tested, validated and placed online with live code.
    • The Cloud-based API can easily be reused for ANY project or product that requires License Management.
      • If one Product can be produced then ANY number of Products can also be produced with minimal effort.
    • The License Management System can be easily extended to require Pay-Per-Use or Pay-Per-User or any other viable model.
      • The Payment processing System could be any usable provider such as PayPal or Google Checkout.
      • License Management System also supports Freeware.
      • License Management System does not use OpenID nor does it require Users to have Google Accounts.
        • OpenID or Google Account Authentication could be used going forward as options.
        • FaceBook Authentication is also an option going forward.
    • Terms and Conditions have been installed in the product.
    • Auto-Updater has been coded but needs to be tested using a real-world scenario.
    • Badge Installer for the Personal Version has been completed and is online.
    • Version 0.1.1.0 has been deployed and is online.
  • Reusable Adobe AIR 2.5 Framework has been created, tested, validated and placed online with live code.
    • Adobe AIR 2.5 also means there could easily be an Android App using much of the same code as is used by the Desktop App.
    • Native Installer means Native Process Support is automatic – this allows the Desktop versions to be augmented by some rather powerful client-side code, should this become a requirement.
    • Adobe AIR 2.5 also means the Desktop version could run in Windows, Mac and Linux however for now the pre-Alpha version is Windows only.

PDFXporter + Polymorphical Project Diary :: Rapid Project Development Day 13


PDFXporter Project Diary :: Rapid Project Development :: Day #11

Read the Disclaimers and Warranties and download the Pre-Alpha now:

Day #11 – Billable Hours #1 -4

The Goal now… Recode that lame Auto-Updater Code from Yesterday

Not going to go back over why this is being done but rest assured this code module will be cool once I am done with it. Honestly, I think the original author would have done what I am about to do had he thought about it but alas he probably doesn’t have the required experience level to allow him to go where I will take this code module.

Auto-Updater is no longer lame !

As it turns out the Auto-Updater code just needed a bit of server-side magic to complete that circuit. Still have to test this part of the product however for being in Day #11 there is no need to get all axel-wrapped over whether or not all the code is perfect since the ability to perform Auto-Updates will come out in the wash sooner or later and surely before the product has been releases as a Beta version.

Needless to say, I do not allow code into my own products unless I know it will work and has been thoroughly tested – again I do not even bother to call this “Unit Testing” because this is just a fact of life – code must be tested and so it shall be.

Day #11 – Billable Hours #5 -8

Badge Installer / Enterprise Edition / Trialware-Freeware-Personal Versions

Then Badge Installer for those who just don’t want to download the installer – Badge Installers know how to install the product from the web – click on an icon or something then watch the installation happen, this sort of thing. Obviously those who install using the Native Installer will have a more powerful Application due to the Native Process support that goes along with the Native Installer; those who install from the Badge Installer will be getting into the Trialware Version where they can try-it-before-buying-it sort of thing. The Native Installer Version will be the Enterprise Edition and the Badge Installer Version will be the Trialware/Freeware/Personal Version.

Downloads are too big !

Yup, I have produced so much code during the first 11 days… well Google App Engine has limits and one of them is the file size so now I have to relocate the downloads to another server farm – also Google App Engine has bandwidth limits but never fear all this takes is a bit of time. Google App Engine remains a free resource for me and not only because I know how to use that TCP/IP thing… but I do !

Project Recap so far…

During the previous 11 days of development the following has been accomplished by one single individual contributor:

  • Reusable Cloud-based versioned API has been created, tested, validated and placed online with live code.
    • The Cloud-based API can easily be reused for ANY project or product that requires License Management.
      • If one Product can be produced then ANY number of Products can also be produced with minimal effort.
    • The License Management System can be easily extended to require Pay-Per-Use or Pay-Per-User or any other viable model.
      • The Payment processing System could be any usable provider such as PayPal or Google Checkout.
      • License Management System also supports Freeware.
      • License Management System does not use OpenID nor does it require Users to have Google Accounts.
        • OpenID or Google Account Authentication could be used going forward as options.
        • FaceBook Authentication is also an option going forward.
    • Terms and Conditions have been installed in the product.
    • Auto-Updater has been coded but needs to be tested using a real-world scenario.
    • Badge Installer for the Personal Version has been completed and is online.
    • Version 0.1.1.0 has been deployed and is online.
  • Reusable Adobe AIR 2.5 Framework has been created, tested, validated and placed online with live code.
    • Adobe AIR 2.5 also means there could easily be an Android App using much of the same code as is used by the Desktop App.
    • Native Installer means Native Process Support is automatic – this allows the Desktop versions to be augmented by some rather powerful client-side code, should this become a requirement.
    • Adobe AIR 2.5 also means the Desktop version could run in Windows, Mac and Linux however for now the pre-Alpha version is Windows only.

PDFXporter Project Diary :: Rapid Project Development :: Day #12


PDFXporter Project Diary :: Rapid Project Development :: Day #10

Read the Disclaimers and Warranties and download the Pre-Alpha now:

Day #10 – Billable Hours #1 -5

The Goal now… Terms and Conditions of Service and Automatic Updates

Now the goal is to craft some Terms and Conditions for use and all that legal stuff nobody likes to think about except Lawyers and those who like to cover their buns.

Users must now read and respond to the Terms and Conditions before they can upload Bank Statement PDF Documents for processing.

Day #10 – Billable Hours #6 -8

Automatic Updates

Then Automatic Updates… this is why people love using Adobe AIR Apps – they keep themselves updated.

The Auto-Updater code I found doesn’t work very well so I am recoding it using my own specifications which as you may know from reading my blog posts, it has to work and when it don’t work I make it work. I am not a fan of XML at-all because when I am paying for the bandwidth JSON works better, when you pay for the bandwidth you can use XML, no worries. Also the code the author of the Auto-Updater module I found used a rather lame technique for fetching XML from a URL and let’s face it I have much better code that just works better – duh ! But seriously folks, what’s the point of using code that only works sometimes when you can deploy code that works all the time. By the time I am done with this Auto-Updater Code rewrite I will have a much more robust system for handling Auto-Updates for my Native Installer Version which should also work for the Bade Installer Version as well but I will have to see how this all works.

Badge Installer / Enterprise Edition / Trialware-Freeware Versions

Then Badge Installer for those who just don’t want to download the installer – Badge Installers know how to install the product from the web – click on an icon or something then watch the installation happen, this sort of thing. Obviously those who install using the Native Installer will have a more powerful Application due to the Native Process support that goes along with the Native Installer; those who install from the Badge Installer will be getting into the Trialware Version where they can try-it-before-buying-it sort of thing. The Native Installer Version will be the Enterprise Edition and the Badge Installer Version will be the Trialware/Freeware Version.

Project Recap so far…

During the previous 10 days of development the following has been accomplished by one single individual contributor:

  • Reusable Cloud-based versioned API has been created, tested, validated and placed online with live code.
    • The Cloud-based API can easily be reused for ANY project or product that requires License Management.
      • If one Product can be produced then ANY number of Products can also be produced with minimal effort.
    • The License Management System can be easily extended to require Pay-Per-Use or Pay-Per-User or any other viable model.
      • The Payment processing System could be any usable provider such as PayPal or Google Checkout.
      • License Management System also supports Freeware.
      • License Management System does not use OpenID nor does it require Users to have Google Accounts.
        • OpenID or Google Account Authentication could be used going forward as options.
        • FaceBook Authentication is also an option going forward.
    • Terms and Conditions have been installed in the product.
    • Initial version of the Auto-Updater has been installed but must be recoded due to lameness.
  • Reusable Adobe AIR 2.5 Framework has been created, tested, validated and placed online with live code.
    • Adobe AIR 2.5 also means there could easily be an Android App using much of the same code as is used by the Desktop App.
    • Native Installer means Native Process Support is automatic – this allows the Desktop versions to be augmented by some rather powerful client-side code, should this become a requirement.
    • Adobe AIR 2.5 also means the Desktop version could run in Windows, Mac and Linux however for now the pre-Alpha version is Windows only.

PDFXporter Project Diary :: Rapid Project Development :: Day #11


PDFXporter Project Diary :: Rapid Project Development :: Day #9

Read the Disclaimers and Warranties and download the Pre-Alpha now:

Day #9 – Billable Hour #1

The Goal now… code the PDF Export to Data Function

Now it is time to write the code that exports PDF content as Data. I wrote this code some weeks ago as a precursor for this project and to prepare for this project; so all I have to do is reuse some code from a past project.

Obviously for Version 1.0.0.0 all of this can be fairly simplistic – just export the contents of the PDF as Text and call it a day. The next version can refine this process further. The next version can export the PDF as an .XLS or .CSV. The version after that can export the PDF as a small Database (glob of Objects aka. JSON) – the client which is an Adobe AIR 2.5 App can provide allow the user to run reports on the data as well as process the data into Income and Expenses.

For now, all I need is the ability to upload the PDF – export the data into a Blob then allow the user to download the Blob as a JSON object and call it a day for Version 1.0.0.0

Version 1.0.0.0 Can be Freeware

The first several versions can be Freeware with no cost for Users at-all. Once I get enough Users I can establish a pay-per-use model – License Management changes not at all by then other than to limit the functions a non-paying user gets to use along with an extra dialog that asks for money via PayPal, etc.

Day #9 – Billable Hour #2

PDF Export function is online now !

PDF into Text is noting to talk about. This can be done with pretty much any PDF Reader program.

Next Up is the PDF to Data

Making Data out of Text you might export from your Bank Statement PDF is a bit more of a trick than simply getting ASCII Text. This is where PDFXporter will shine of course. In the meantime, I think I will spend a short time playing with the PDF Exporter function that runs in the Google Cloud just to make sure it will run fast enough. Again, you can call this Unit Testing if you must however I hardly doubt this is any more than just doing my job as a Software Engineer; write it and test it is about as profound as breathing air and pumping blood – has to be done either way.

Day #9 – Billable Hour #3-8

Now it is time to build and deploy the Distribution Site

Version 0.1.0.0 is online and ready for deployment so you and others reading this Blog Series can play with a real live Application so you might know what I am writing here is real and substantial. Surely your first response will be to disbelieve that anyone can produce a working Adobe AIR 2.5 Application in 9 short days of development – actually I could have done this faster if I were actually being paid to do the work – I tend to work faster when I am being paid although I am not sure why – call it a sense of urgency, if you must.

Next Step is to Deploy the Site people can use to test-drive PDFXporter

The Pre-Alpha Installer come first… The goal is to make a Native Installer for PDFXporter running in the Google Cloud.

You can sample this product now during the pre-Alpha stage here. Kindly report all issues to the proper Support Department.

Project Recap so far…

During the previous 9 days of development the following has been accomplished by one single individual contributor:

  • Reusable Cloud-based versioned API has been created, tested, validated and placed online with live code.
    • The Cloud-based API can easily be reused for ANY project or product that requires License Management.
      • If one Product can be produced then ANY number of Products can also be produced with minimal effort.
    • The License Management System can be easily extended to require Pay-Per-Use or Pay-Per-User or any other viable model.
      • The Payment processing System could be any usable provider such as PayPal or Google Checkout.
      • License Management System also supports Freeware.
      • License Management System does not use OpenID nor does it require Users to have Google Accounts.
        • OpenID or Google Account Authentication could be used going forward as options.
        • FaceBook Authentication is also an option going forward.
  • Reusable Adobe AIR 2.5 Framework has been created, tested, validated and placed online with live code.
    • Adobe AIR 2.5 also means there could easily be an Android App using much of the same code as is used by the Desktop App.
    • Native Installer means Native Process Support is automatic – this allows the Desktop versions to be augmented by some rather powerful client-side code, should this become a requirement.
    • Adobe AIR 2.5 also means the Desktop version could run in Windows, Mac and Linux however for now the pre-Alpha version is Windows only.

PDFXporter Project Diary :: Rapid Project Development :: Day #10


%d bloggers like this: