Bash 101 Redux

Bash 101 Redux 

Vyper Logix Corp Makes ITC (Inter-Thread Communications) Easy as 1,2,3 !!!

Take a look at the code sample found in this article !!!

Very Lean !!!

Very Cool !!!

Threaded !!!

Best of all the main process terminates itself once all the work has been done !!!

 

 

Code sample was printed using Wing IDE and Snag-It 10.

 

Python Type-Cast the Easy Way !!!

Python objects hold key and value pairs – this is what every object holds, key and value pairs.

Python objects that have a __dict__ make it easy to convert the object from one class to another simply by replacing the old object instance with another as follows:

See the attached file with a sample of how this can be done most simply. (type-cast-lessons.zip)

There are other ways to accomplish the same goals using more elaborate means however this works using less code.

What to do when your objects do not implements the magic methods required for the aforementioned code sample ?

Easy, you can use method injection as described here or here or here or here.

It also helps to use a really good Python Debugger in the form of the Wing IDE.  Stare at the debugger while debugging Python code just long enough and sooner or later you may begin to see how to accomplish goals you may find useful.

I am not recommending you try using the aforementioned code as part of any professional solutions unless  you know who will be doing a Peer Review of your code but I am saying these techniques may be useful in learning something about the object types being used as to whether or not the object types are meaningful.

The point being, you can either grouse and lament how things cannot be done or you can learn the Agile Method for getting things done…

String splits versus lists for static data ?!?

Every so often someone will ask a really interesting Python question, this is not one of those however the data seems to disagree with the opinions.

In some python code, I see this:

switch_fields = “id switch_model ip”.split()

which is surely the same as this:

switch_fields = [‘id’, ‘switch_model’, ‘ip’]

Since the intent is clearly to create a list of three items, why would someone create a string instead and invoke a string method to create a list?

My preference is for the “say what you want” side; I don’t consider myself to be a “Python Programmer”, though.

Some responses from various Python programmers…

I agree with saying what you want. I’d probably ask the author if they would ever be inclined to do the reverse, i.e. switch_field_str = ‘ ‘.join([‘id’, ‘switch_model’, ‘ip’]) to prove a point about how confusing/silly that can be.

 

     I agree, there is no logical or practical reason to have a static string that has a split performed on it every time the code is run. It is a waste of computational time.

Now for my response…

I was curious…

I had to run over 100,000 iterations to see a measurable difference…

Apart from the run-time differences one might find it easier to maintain code with static arrays loaded from string splits especially if one had to sit and type a lot of quote marks and commas… or maybe I have had far too many late night coding sessions in my past.

Thought I would share the attachment.

Seems the data does not really seem to support the opinions if the concern is computational time and performance… unless you wanted to do the string splits to list thing something close to 1,000,000 times; even 100,000 iterations is nothing you can measure let alone notice.

Eh, such is life.

Built-In Unit Test !!!

This code has a built-in Unit Test

You know what…  I think this means I know how to write Unit Tests !!!

Well I wrote this one anyway – this is my story and I am sticking to it… LOL

Yet another Ruby on Rails Programming Test *yawn*

So here we are again with yet another Ruby on Rails Programming Test !!!

Maybe someday people will just take as proof everything else I have publishing here but that day has apparently not yet arrived…

The Problem – Not exactly any kind of real business problem…

Rails Exercise:

We’d like you to create a simple Rails 3 application.
Create models that allow a user to follow another user.
The user model’s only attribute is “name”. You may scaffold the create action.
Create an interface that accepts post requests that allows users to
“follow” other users.
The user’s “index” action should list all of the users.
The user’s “show” action should show:
1) The user’s name
2) The users the user is currently following (with a button to remove
that following)
3) The users the user is not following (with a button to add that following)
4)  The users currently following this user
Do not create a login system for this exercise.

The Solution

Download it from this link; only 600 KB  !!!

Programming Test sponsored by NASA and RackSpace *yawn*

Programming Tests that have no bearing on the reality of software engineering are a real hoot – this one comes to us from NASA and RackSpace – I especially liked the one from my early childhood – the one that counts coins… LOL

Enjoy !!!

dev project 3 – Voccio

dev project 3 – Voccio (response on 11-17-2011)

What is the Twisted Framework ?!?

Every so often I am asked to describe certain frameworks like the Twisted Framework and this is my response for that one.

The Twisted Framework (Disclaimer: Some of the code in this document was written by Ray C Horn but the rest was not since it may be from some Open Source sources… nothing like stating the obvious…)

SleepyMongoose meets the Python after dark

 

For a while now I have been looking for a reasonable

REST-based NoSQL Database I can use from the Google App Engine just to avoid
having to pay Google for their database fees.

Earlier today… I was able to make the SleepyMongoose
snuggle-up with the Python for some after-hours fun in the dark.

SleepyMongoose

You can read-up about SleepyMongoose here.

It’s a REST-based interface for MongoDB that runs as a lightweight Python HTTP
Server – I will eventually get around to making this SleepyMongoose wake-up by
shoving some Stackless Python into it just for fun and because Stackless always
makes Python work better.

So what does the SleepyMongoose need once I get around to
using it from the Google App Engine ?

SleepyMongoose needs a slick Python wrapper so I don’t gotta
deal with all that REST stuff which after-all the REST interface is only needed
because Google doesn’t seem to care much about bandwidth consumption for
anything other than their own database and it just makes sense to make Google
give me a FREE No-Cost MongoDb Proxy which is what this is all leading to
anyway.

SleepyMongoose meets the Python

 So, since I am a pretty good Python coder I chose to whip-up

a Magic little object that allows me to get away from REST and right into
Python objects.

I implemented the SleepyMongoose API as documented in their Wiki.

Keep in mind the code presented here requires the Vyper
Logix Library as found here (shameless
plug, of course
).

Here’s a sample of what the usage looks like:

from vyperlogix import
mongodb

 

if (__name__ == ‘__main__’):

    import os,sys

   

    sm = mongodb.SleepyMongoose(‘127.0.0.1:27080’).db(‘gps_1M’).collection(‘num_connections’)

    results = sm.connect()

    print ‘sm.connect() #1
–> ‘
,results

 

    results = sm.server(‘mongodb://127.0.0.1:65535’).connect()

    print ‘sm.connect() #2
–> ‘
,results

 

    results = sm.server(‘mongodb://127.0.0.1:65535’).name(‘backup’).connect()

    print ‘sm.connect() #3
–> ‘
,results

 

    results = sm.docs({“x”:2}).insert()

    print ‘sm.insert() #1
–> ‘
,results

 

    results = sm.find({“x”:2})

    print ‘sm.find() #1 –>
,results

 

    results = sm.criteria({“x”:2}).remove()

    print ‘sm.remove() #1
–> ‘
,results

 

    results = sm.docs().insert({“x”:3})

    print ‘sm.insert() #1
–> ‘
,results

 

    results = sm.find({“x”:3})

    print ‘sm.find() #2 –>
,results

 

    results = sm.remove({“x”:3})

    print ‘sm.remove() #2
–> ‘
,results

 

    results = sm.docs([{“x”:2},{“x”:3}]).insert()

    print ‘sm.insert() #3
–> ‘
,results

 

    results = sm.criteria({“x”:2}).remove()

    print ‘sm.remove() #3
–> ‘
,results

 

    results = sm.criteria([{“x”:3}]).remove()

    print ‘sm.remove() #4
–> ‘
,results

 

    results = sm.insert([{“x”:2},{“x”:3}])

    print ‘sm.insert() #4
–> ‘
,results

 

    results = sm.remove({“x”:2})

    print ‘sm.remove() #5
–> ‘
,results

 

    results = sm.remove([{“x”:3}])

    print ‘sm.remove() #6
–> ‘
,results

    pass

Here’s the rest of the code:  (Simple and to the point, no nonsense code,
in a few hours…)

__copyright__ = “””\

(c). Copyright 2008-2011,
Vyper Logix Corp., All Rights Reserved.

 

Published under Creative
Commons License

(http://creativecommons.org/licenses/by-nc/3.0/)

restricted to
non-commercial educational use only.,

 

http://www.VyperLogix.com
for details

 

THE AUTHOR VYPER LOGIX
CORP DISCLAIMS ALL WARRANTIES WITH REGARD TO

THIS SOFTWARE, INCLUDING
ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND

FITNESS, IN NO EVENT SHALL
THE AUTHOR BE LIABLE FOR ANY SPECIAL,

INDIRECT OR CONSEQUENTIAL
DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING

FROM LOSS OF USE, DATA OR
PROFITS, WHETHER IN AN ACTION OF CONTRACT,

NEGLIGENCE OR OTHER
TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION

WITH THE USE OR
PERFORMANCE OF THIS SOFTWARE !

 

USE AT YOUR OWN RISK.

“””

import logging

 

import urllib2

 

import simplejson

 

from vyperlogix import
misc

from vyperlogix.misc import _utils

from vyperlogix.classes import MagicObject

 

arg0 = lambda args:args[0] if (misc.isIterable(args)) and (len(args) > 0) and (misc.isString(args[0])) else None

list0 = lambda args:args[0] if (misc.isIterable(args)) and (len(args) > 0) and ( (misc.isList(args[0])) or (misc.isDict(args[0])) ) else []

int0 = lambda args:args[0] if (misc.isIterable(args)) and (len(args) > 0) and (misc.isInteger(args[0])) else None

bool0 = lambda args:args[0] if (misc.isIterable(args)) and (len(args) > 0) and (misc.isBooleanString(args[0])) else None

 

__only__ = lambda value,target:value if (str(value).lower().capitalize() == str(target).lower().capitalize()) else None

 

class SleepyMongoose(MagicObject.MagicObject2):

    def __init__(self,sleepy_mongoose):

        ”’

        See
also: https://github.com/kchodorow/sleepy.mongoose/wiki/

        ”’

        toks = sleepy_mongoose.split(‘:’)

        try:

            self.__sleepy_mongoose__ =
sleepy_mongoose if (misc.isString(toks[0]) and misc.isInteger(int(toks[-1]))) else None

        except:

            self.__sleepy_mongoose__ =
None

        self.__database__ = None

        self.__collection__ = None

        self.__server__ = None

        self.__server_name__ = None

        self.__last_exception__ =
None

        self.__n__ = None

        self.__docs__ = None

        self.__criteria__ = None

        self.__newobj__ = None

        self.__fields__ = None

        self.__sort__ = None

        self.__skip__ = None

        self.__limit__ = None

        self.__explain__ = None

        self.__batch_size__ = None

        self.__id__ = None

       

    def __http_get__(self,url):

        data = None

        try:

            response = urllib2.urlopen(url)

            json = response.read()

            data = simplejson.loads(json)

        except Exception, ex:

            self.__last_exception__ =
_utils.formattedException(details=ex)

            data = None

        return data

 

    def __http_post__(self,url,parms=()):

        from vyperlogix.url._urllib2 import
http_post

        data = None

        try:

            json = http_post(url,parms)

            data = simplejson.loads(json)

        except Exception, ex:

            self.__last_exception__ =
_utils.formattedException(details=ex)

            data = None

        return data

 

    def __handle_exceptions__(self):

        if (not self.__sleepy_mongoose__):

            logging.error(‘ERROR: Cannot understand “SleepyMongoose(%s)”‘
% (self.__sleepy_mongoose__))

        elif (not self.__database__):

            logging.error(‘ERROR: Cannot understand “db(%s)”‘ % (self.__database__))

        elif (not self.__collection__):

            logging.error(‘ERROR: Cannot understand “collection(%s)”‘
% (self.__collection__))

   

    def __getattr__(self,name):

        self.__n__ = name

        return super(SleepyMongoose, self).__getattr__(name)

 

    def __call__(self,*args,**kwargs):

        if (self.__n__ == ‘db’):

            self.__database__ = arg0(args)

            self.__reset_magic__()

        elif (self.__n__ == ‘collection’):

            self.__collection__ =
arg0(args)

            self.__reset_magic__()

        elif (self.__n__ == ‘server’):

            self.__server__ = arg0(args)

            self.__reset_magic__()

        elif (self.__n__ == ‘name’):

            self.__server_name__ =
arg0(args)

            self.__reset_magic__()

        elif (self.__n__ == ‘docs’):

            self.__docs__ = list0(args)

            self.__reset_magic__()

        elif (self.__n__ == ‘criteria’):

            self.__criteria__ = list0(args)

            self.__reset_magic__()

        elif (self.__n__ == ‘fields’):

            self.__fields__ = list0(args)

            self.__reset_magic__()

        elif (self.__n__ == ‘sort’):

            self.__sort__ = list0(args)

            self.__reset_magic__()

        elif (self.__n__ == ‘skip’):

            self.__skip__ = int0(args)

            self.__reset_magic__()

        elif (self.__n__ == ‘limit’):

            self.__limit__ = int0(args)

            self.__reset_magic__()

        elif (self.__n__ == ‘batch_size’):

            self.__batch_size__ =
int0(args)

            self.__reset_magic__()

        elif (self.__id__ == ‘id’):

            self.__id__ = int0(args)

            self.__reset_magic__()

        elif (self.__n__ == ‘explain’):

            self.__explain__ =
__only__(bool0(args),‘True’)

            self.__reset_magic__()

        elif (self.__n__ == ‘newobj’):

            self.__newobj__ = list0(args)

            self.__reset_magic__()

        elif (self.__n__ == ‘find’):

            if (self.__sleepy_mongoose__)
and (self.__database__) and (self.__collection__):

                p = []

                if (self.__criteria__):

                    p.append(tuple([“criteria”,simplejson.dumps(self.__criteria__)]))

                    self.__criteria__ = None

                if (self.__fields__):

                    p.append(tuple([“fields”,simplejson.dumps(self.__fields__)]))

                    self.__fields__ = None

                if (self.__sort__):

                    p.append(tuple([“sort”,simplejson.dumps(self.__sort__)]))

                    self.__sort__ = None

                if (self.__skip__):

                    p.append(tuple([“skip”,‘%d’%(self.__skip__)]))

                    self.__skip__ = None

                if (self.__limit__):

                    p.append(tuple([“limit”,‘%d’%(self.__limit__)]))

                    self.__limit__ = None

                if (self.__explain__):

                    p.append(tuple([“explain”,‘%s’%(self.__explain__)]))

                    self.__explain__ = None

                if (self.__batch_size__):

                    p.append(tuple([“batch_size”,‘%d’%(self.__batch_size__)]))

                    self.__batch_size__ = None

                if (self.__server_name__):

                    p.append(tuple([“name”,self.__server_name__]))

                url = http://%s/%s/%s/_%s’
% (self.__sleepy_mongoose__,self.__database__,self.__collection__,self.__n__)

                return self.__http_post__(url,tuple(p))

            else:

                self.__handle_exceptions__()

            self.__reset_magic__()

        elif (self.__n__ == ‘insert’):

            if (self.__sleepy_mongoose__)
and (self.__database__) and (self.__collection__):

                p = []

                if (self.__docs__):

                    p.append(tuple([“docs”,simplejson.dumps(self.__docs__)]))

                    self.__docs__ = None

                else:

                    p.append(tuple([“docs”,simplejson.dumps(list0(args))]))

                    self.__docs__ = None

                if (self.__server_name__):

                    p.append(tuple([“name”,self.__server_name__]))

                url = http://%s/%s/%s/_%s’
% (self.__sleepy_mongoose__,self.__database__,self.__collection__,self.__n__)

                return self.__http_post__(url,tuple(p))

            else:

                self.__handle_exceptions__()

            self.__reset_magic__()

        elif (self.__n__ == ‘update’):

            if (self.__sleepy_mongoose__)
and (self.__database__) and (self.__collection__):

                p = []

                if (self.__criteria__):

                    p.append(tuple([“criteria”,simplejson.dumps(self.__criteria__)]))

                    self.__criteria__ = None

                if (self.__newobj__):

                    p.append(tuple([“newobj”,simplejson.dumps(self.__newobj__)]))

                    self.__newobj__ = None

                if (self.__server_name__):

                    p.append(tuple([“name”,self.__server_name__]))

                url = http://%s/%s/%s/_%s’
% (self.__sleepy_mongoose__,self.__database__,self.__collection__,self.__n__)

                return self.__http_post__(url,tuple(p))

            else:

                self.__handle_exceptions__()

            self.__reset_magic__()

        elif (self.__n__ == ‘more’):

            if (self.__sleepy_mongoose__)
and (self.__database__) and (self.__collection__):

                p = []

                if (self.__id__):

                    p.append(tuple([“id”,‘%d’%(self.__id__)]))

                    self.__id__ = None

                if (self.__batch_size__):

                    p.append(tuple([“batch_size”,‘%d’%(self.__batch_size__)]))

                    self.__batch_size__ = None

                if (self.__server_name__):

                    p.append(tuple([“name”,self.__server_name__]))

                q = ‘?’+‘&’.join([‘=’.join(list(t)) for t in p]) if (len(p) > 0) else

                url = http://%s/%s/%s/_%s%s’
% (self.__sleepy_mongoose__,self.__database__,self.__collection__,self.__n__,q)

                return self.__http_get__(url)

            else:

                self.__handle_exceptions__()

            self.__reset_magic__()

        elif (self.__n__ == ‘cmd’):

            if (self.__sleepy_mongoose__)
and (self.__database__):

                p = []

                _c_ = list0(args)

                if (_c_):

                    p.append(tuple([“cmd”,simplejson.dumps(_c_)]))

                url = http://%s/%s/_%s’ % (self.__sleepy_mongoose__,self.__database__,self.__n__)

                return self.__http_post__(url,tuple(p))

            else:

                self.__handle_exceptions__()

            self.__reset_magic__()

        elif (self.__n__ == ‘remove’):

            if (self.__sleepy_mongoose__)
and (self.__database__) and (self.__collection__):

                p = []

                if (self.__criteria____):

                    p.append(tuple([“criteria”,simplejson.dumps(self.__criteria____)]))

                    self.__criteria____ = None

                else:

                    p.append(tuple([“criteria”,simplejson.dumps(list0(args))]))

                    self.__criteria____ = None

                if (self.__server_name__):

                    p.append(tuple([“name”,self.__server_name__]))

                url = http://%s/%s/%s/_%s’
% (self.__sleepy_mongoose__,self.__database__,self.__collection__,self.__n__)

                return self.__http_post__(url,tuple(p))

            else:

                self.__handle_exceptions__()

            self.__reset_magic__()

        elif (self.__n__ == ‘connect’):

            _token = self.__n__

            if (self.__sleepy_mongoose__):

                p = []

                if (self.__server__):

                    p.append(tuple([“server”,self.__server__]))

                if (self.__server_name__):

                    p.append(tuple([“name”,self.__server_name__]))

                url = http://%s/_%s’ % (self.__sleepy_mongoose__,_token)

                return self.__http_post__(url,tuple(p))

            else:

                self.__handle_exceptions__()

            self.__reset_magic__()

        else:

            logging.debug(‘DEBUG: Cannot understand “%s(%s,%s)”‘ % (self.n,args,kwargs))

        return self

 

 

Java is half the performance of Python and Ruby Fails to run this benchmark !

Benchmarks are about as valuable as… lies, damned lies and…

However given the limited value in doing benchmarks this particular use-case can highlight the relative weakness of Java while highlighting the relative strength of Python while highlighting the fact that even though Ruby was built for recursion it can FAIL completely or at-best give sloppy performance.

Be warned, if you love Java and/or Ruby you might not like this benchmark and the results presented here…

First the problem:

Factorial… plain and simple.

I wanted to see how well Stackless Python fares against standard Python where Stackless used recursion or something closer to recursion and regular Python 2.5.5.x used generators.  Bottom line is, generators smoke but you already knew this.

Then I wanted to see how well Ruby 1.8.7 versus Ruby 1.9.2 might fare since Ruby is known to be heavily optimized for recursion.  Bottom line is Ruby 1.8.7 falls dead-even with Stackless Python (DUH!) and well behind the Python Generator however Ruby 1.8.7 fares very well against Java and Java came in dead last across the board.  Ruby 1.9.2 fails completely for the most extreme use-case, as shown below.

The Results:

Factorial program using Stackless Python.
1000! / 998! = 999000
         8 function calls in 0.010 CPU seconds
10000! / 9998! = 99990000
         8 function calls in 0.234 CPU seconds

Factorial program using Python Generators.
1000! / 998! = 999000
         2004 function calls in 0.002 CPU seconds
10000! / 9998! = 99990000
         20004 function calls in 0.135 CPU seconds

Factorial program using Ruby 1.8.7.
1000! / 998!1000! / 998! = 999000
  0.016000   0.000000   0.016000 (  0.010002)
             user     system      total        real
10000! / 9998!10000! / 9998! = 99990000
  0.421000   0.062000   0.483000 (  0.555111)

Factorial program using Ruby 1.9.2.
1000! / 998!1000! / 998! = 999000
  0.000000   0.000000   0.000000 (  0.003000)
             user     system      total        real
10000! / 9998! FAILS due to Recursion Limit

Factorial program using Java.
1000!/998! = 999000
Runtime is :0.017

10000!/9998! = 99990000
Runtime is :0.389

Results Analysis: (From Fastest to Slowest, Top-down)

All runtimes are given in seconds unless otherwise indicated.
Factorial program using Ruby 1.9.2.
10000! / 9998!                                                          FAILS due to Recursion Limit

Factorial program using Python Generators.
1000! / 998! = 999000                                                    0.002 CPU seconds

Factorial program using Ruby 1.9.2.
1000! / 998! = 999000                   0.000000   0.000000   0.000000 (  0.003000)

Factorial program using Stackless Python.
1000! / 998! = 999000                                                    0.010 CPU seconds

Factorial program using Ruby 1.8.7.
1000! / 998!1000! / 998! = 999000         0.016000   0.000000   0.016000 (  0.010002)

Factorial program using Python Generators.
10000! / 9998! = 99990000                                                0.135 CPU seconds

Factorial program using Java.
1000!/998! = 999000                                                      0.017

Factorial program using Stackless Python.
10000! / 9998! = 99990000                                                0.234 CPU seconds

Factorial program using Java.
10000!/9998! = 99990000                                                    0.389

Factorial program using Ruby 1.8.7.
10000! / 9998!10000! / 9998! = 99990000   0.421000   0.062000   0.483000 (  0.555111)

Ruby 1.9.2 came in first or last, depending on how you look at such things, but only because it failed to run for the most extreme case.
Python 2.5.5.x Generators smoke, as expected !
Ruby 1.9.2 comes in just behind Python Generators. So much for Ruby 1.9.2 being the Python killer.
Stackless Python comes in just ahead-of or dead-even with Ruby 1.8.7 depending on how one takes the Ruby profiler results.
Java comes in ahead of Ruby 1.8.7 cross the board but only because one has to use BigDecimal to get Java to handle the large numbers. Python handles the transition from regular integers to large numbers as they are being processed - Java might fare better if this were the case for Java but sadly it is not.
Ruby 1.8.7 comes in dead last.
Download the code.

Final Thoughts

Obviously Ruby Generators would have fared better than recursion however given that Ruby recursion is supposed to smoke I felt it was only fair to test recursion in Ruby with Python Generators because Python Generators are supposed to smoke.  Test what is supposed to smoke against what is supposed to smoke.
Java sucks for any problem not covered by the typical design of the JVM however since Python has been tweaked over the years it tend to perform very well and more than adequately given it is dynamic where Java is more static at runtime.  If Python had been more heavily optimized it surely would have done much better however the goal here was to use off-the-shelf solutions rather than the hand-optimized kind.
Needless to say, Python could be made to run a whole lot faster than Java especially when the typical boundaries are being pushed well beyond the typical limits.  This is the ugly secret about Java but then look at how Java is being used in the real-world and you too will see where and how Java fails to please.

Python is the other white meat !

Java is the Enterprise favorite now but... Python is the language that gets the most lovin' when it comes to squeezing the most out of any programming problem.  I have personally pushed Python to the mat in terms of performance and it comes back asking for more.  I do not see anyone in the Enterprise doing anything like this with Java however I do see more multi-tier solutions with Java than would be required to make the systems work.
%d bloggers like this: