What does Google think about Python ?!?

Earlier today I had the opportunity to speak with someone from Google who shared these thoughts with me in regard to Python…

  • Python tuple objects are immutable.
    • Tuple objects lack the methods that allow elements of a tuple to be modified however… tuple objects can be recast into lists, list objects are not so immutable, after the change has been made the lists can be recast as tuples, as follows:
      • x = (1,2) # x is a tuple
      • try:
      •     x[0] = 11 # this throws an error
      • exception:
      •     print ‘Error Will Robinson…’
      • x = list(x) # now x is a list
      • x[0] = 11 # this will not throw an error
      • x = tuple(x) # x is now a tuple once again
    • Obviously the guy from Google who thinks he knows Python failed to understand this one simple thing about tuple objects – “tuples were made to be cast as lists”.
      • Python tuple objects are as immutable as your typical window pane is not breakable… so long as you “think” your typical window pane cannot be broken and nobody ever picks-up a rock you are right… however… rocks were made to be thrown by people who have hands.
      • The fact is… there are no objects in Python that are truly immutable however tuple objects lack the methods that allow tuple members to be changed unless one recasts a tuple as a list and then back again after one makes the desired changes.
    • Additionally, one could create a tuple subclass that allows tuple members to be modified.
  • When a global variable is assigned a tuple it is not possible to make any changes to that global variable.
    • This is just False !!!
      • There is NOTHING special about variables that hold onto tuple object instances.
    • It is possible to make Constants using Python however this does not come with Python’s Standard Library.
      • Tuples are NOT Constants !!!
        • Variables that hold onto tuple pointers are NOT Constants !!!
      • Not even a Python Constant will be truly immutable even when a Global Variable is used to point to the Instance of a Python Constant.
        • The only way to make a Global or Static Variable in Python capable of being truly immutable is to place that Constant into a Module that lacks the ability to allow the Constant to be changed at runtime.
        • Your typical Python Global Variable can be changed, which is to say your typical Global Variable is Read-Write and can be assigned to a different value that can be a different Instance of a Python object.
  • The special thing about inheritance is multiple inheritance.
    • One man’s special is another man’s who-cares ?!?
    • Apparently even Google’s Search Engine does not even know “what is special about python inheritance“… LOL
      • You might think something “special” about Python might be clearly special to everybody however this is just NOTthe case in this instance…
        • Is it any wonder I might not know this specific aspect when, it seems, not enough other people do know about it to make it to the first page of a Google search… ROFL
          • On the other hand, not even this Google guy was aware of how special this aspect of Python was to himself but then we can all hold things close to our own hearts without getting approval from the rest of the world…

I was disappointed in how little this one guy from Google seems to know about Python OR maybe I just dug deeper into Python than Google has…

OR

Maybe this Google guy think only hackers play-around with Python to the point of knowing the language inside and out and maybe this Google guy is only able to use Python in the most superficial way possible. ?!?

The great thing about computers is Computer Science… we can all test our assumptions to see whether or not our assumptions are correct or as-correct as we with them to be.

Also it’s refreshing to me to realize how human this one Google Guy is because he is not immune from wishful thinking where Python is concerned.  He “wishes” Python was a much better language than it is.  He “wishes” tuples are actually as immutable as he “wishes” them to be even though the language does not support his wishes, he continues to harbor his fanciful wishes.  This speaks well of his inner-child…

I like to stick with the Science… the way Python actually works… devoid of my own wishful thinking… Python tuples are no more immutable than my nose is unable to be cut off… I prefer to leave my nose intact just as one would have to prefer to leave a tuple instance unchanged but not because a tuple CANNOT be changed, just that one prefers a tuple to be immutable.

Try it for yourself and you be the judge !!!

Advertisements

SleepyMongoose meets the Python after dark

 

For a while now I have been looking for a reasonable

REST-based NoSQL Database I can use from the Google App Engine just to avoid
having to pay Google for their database fees.

Earlier today… I was able to make the SleepyMongoose
snuggle-up with the Python for some after-hours fun in the dark.

SleepyMongoose

You can read-up about SleepyMongoose here.

It’s a REST-based interface for MongoDB that runs as a lightweight Python HTTP
Server – I will eventually get around to making this SleepyMongoose wake-up by
shoving some Stackless Python into it just for fun and because Stackless always
makes Python work better.

So what does the SleepyMongoose need once I get around to
using it from the Google App Engine ?

SleepyMongoose needs a slick Python wrapper so I don’t gotta
deal with all that REST stuff which after-all the REST interface is only needed
because Google doesn’t seem to care much about bandwidth consumption for
anything other than their own database and it just makes sense to make Google
give me a FREE No-Cost MongoDb Proxy which is what this is all leading to
anyway.

SleepyMongoose meets the Python

 So, since I am a pretty good Python coder I chose to whip-up

a Magic little object that allows me to get away from REST and right into
Python objects.

I implemented the SleepyMongoose API as documented in their Wiki.

Keep in mind the code presented here requires the Vyper
Logix Library as found here (shameless
plug, of course
).

Here’s a sample of what the usage looks like:

from vyperlogix import
mongodb

 

if (__name__ == ‘__main__’):

    import os,sys

   

    sm = mongodb.SleepyMongoose(‘127.0.0.1:27080’).db(‘gps_1M’).collection(‘num_connections’)

    results = sm.connect()

    print ‘sm.connect() #1
–> ‘
,results

 

    results = sm.server(‘mongodb://127.0.0.1:65535’).connect()

    print ‘sm.connect() #2
–> ‘
,results

 

    results = sm.server(‘mongodb://127.0.0.1:65535’).name(‘backup’).connect()

    print ‘sm.connect() #3
–> ‘
,results

 

    results = sm.docs({“x”:2}).insert()

    print ‘sm.insert() #1
–> ‘
,results

 

    results = sm.find({“x”:2})

    print ‘sm.find() #1 –>
,results

 

    results = sm.criteria({“x”:2}).remove()

    print ‘sm.remove() #1
–> ‘
,results

 

    results = sm.docs().insert({“x”:3})

    print ‘sm.insert() #1
–> ‘
,results

 

    results = sm.find({“x”:3})

    print ‘sm.find() #2 –>
,results

 

    results = sm.remove({“x”:3})

    print ‘sm.remove() #2
–> ‘
,results

 

    results = sm.docs([{“x”:2},{“x”:3}]).insert()

    print ‘sm.insert() #3
–> ‘
,results

 

    results = sm.criteria({“x”:2}).remove()

    print ‘sm.remove() #3
–> ‘
,results

 

    results = sm.criteria([{“x”:3}]).remove()

    print ‘sm.remove() #4
–> ‘
,results

 

    results = sm.insert([{“x”:2},{“x”:3}])

    print ‘sm.insert() #4
–> ‘
,results

 

    results = sm.remove({“x”:2})

    print ‘sm.remove() #5
–> ‘
,results

 

    results = sm.remove([{“x”:3}])

    print ‘sm.remove() #6
–> ‘
,results

    pass

Here’s the rest of the code:  (Simple and to the point, no nonsense code,
in a few hours…)

__copyright__ = “””\

(c). Copyright 2008-2011,
Vyper Logix Corp., All Rights Reserved.

 

Published under Creative
Commons License

(http://creativecommons.org/licenses/by-nc/3.0/)

restricted to
non-commercial educational use only.,

 

http://www.VyperLogix.com
for details

 

THE AUTHOR VYPER LOGIX
CORP DISCLAIMS ALL WARRANTIES WITH REGARD TO

THIS SOFTWARE, INCLUDING
ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND

FITNESS, IN NO EVENT SHALL
THE AUTHOR BE LIABLE FOR ANY SPECIAL,

INDIRECT OR CONSEQUENTIAL
DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING

FROM LOSS OF USE, DATA OR
PROFITS, WHETHER IN AN ACTION OF CONTRACT,

NEGLIGENCE OR OTHER
TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION

WITH THE USE OR
PERFORMANCE OF THIS SOFTWARE !

 

USE AT YOUR OWN RISK.

“””

import logging

 

import urllib2

 

import simplejson

 

from vyperlogix import
misc

from vyperlogix.misc import _utils

from vyperlogix.classes import MagicObject

 

arg0 = lambda args:args[0] if (misc.isIterable(args)) and (len(args) > 0) and (misc.isString(args[0])) else None

list0 = lambda args:args[0] if (misc.isIterable(args)) and (len(args) > 0) and ( (misc.isList(args[0])) or (misc.isDict(args[0])) ) else []

int0 = lambda args:args[0] if (misc.isIterable(args)) and (len(args) > 0) and (misc.isInteger(args[0])) else None

bool0 = lambda args:args[0] if (misc.isIterable(args)) and (len(args) > 0) and (misc.isBooleanString(args[0])) else None

 

__only__ = lambda value,target:value if (str(value).lower().capitalize() == str(target).lower().capitalize()) else None

 

class SleepyMongoose(MagicObject.MagicObject2):

    def __init__(self,sleepy_mongoose):

        ”’

        See
also: https://github.com/kchodorow/sleepy.mongoose/wiki/

        ”’

        toks = sleepy_mongoose.split(‘:’)

        try:

            self.__sleepy_mongoose__ =
sleepy_mongoose if (misc.isString(toks[0]) and misc.isInteger(int(toks[-1]))) else None

        except:

            self.__sleepy_mongoose__ =
None

        self.__database__ = None

        self.__collection__ = None

        self.__server__ = None

        self.__server_name__ = None

        self.__last_exception__ =
None

        self.__n__ = None

        self.__docs__ = None

        self.__criteria__ = None

        self.__newobj__ = None

        self.__fields__ = None

        self.__sort__ = None

        self.__skip__ = None

        self.__limit__ = None

        self.__explain__ = None

        self.__batch_size__ = None

        self.__id__ = None

       

    def __http_get__(self,url):

        data = None

        try:

            response = urllib2.urlopen(url)

            json = response.read()

            data = simplejson.loads(json)

        except Exception, ex:

            self.__last_exception__ =
_utils.formattedException(details=ex)

            data = None

        return data

 

    def __http_post__(self,url,parms=()):

        from vyperlogix.url._urllib2 import
http_post

        data = None

        try:

            json = http_post(url,parms)

            data = simplejson.loads(json)

        except Exception, ex:

            self.__last_exception__ =
_utils.formattedException(details=ex)

            data = None

        return data

 

    def __handle_exceptions__(self):

        if (not self.__sleepy_mongoose__):

            logging.error(‘ERROR: Cannot understand “SleepyMongoose(%s)”‘
% (self.__sleepy_mongoose__))

        elif (not self.__database__):

            logging.error(‘ERROR: Cannot understand “db(%s)”‘ % (self.__database__))

        elif (not self.__collection__):

            logging.error(‘ERROR: Cannot understand “collection(%s)”‘
% (self.__collection__))

   

    def __getattr__(self,name):

        self.__n__ = name

        return super(SleepyMongoose, self).__getattr__(name)

 

    def __call__(self,*args,**kwargs):

        if (self.__n__ == ‘db’):

            self.__database__ = arg0(args)

            self.__reset_magic__()

        elif (self.__n__ == ‘collection’):

            self.__collection__ =
arg0(args)

            self.__reset_magic__()

        elif (self.__n__ == ‘server’):

            self.__server__ = arg0(args)

            self.__reset_magic__()

        elif (self.__n__ == ‘name’):

            self.__server_name__ =
arg0(args)

            self.__reset_magic__()

        elif (self.__n__ == ‘docs’):

            self.__docs__ = list0(args)

            self.__reset_magic__()

        elif (self.__n__ == ‘criteria’):

            self.__criteria__ = list0(args)

            self.__reset_magic__()

        elif (self.__n__ == ‘fields’):

            self.__fields__ = list0(args)

            self.__reset_magic__()

        elif (self.__n__ == ‘sort’):

            self.__sort__ = list0(args)

            self.__reset_magic__()

        elif (self.__n__ == ‘skip’):

            self.__skip__ = int0(args)

            self.__reset_magic__()

        elif (self.__n__ == ‘limit’):

            self.__limit__ = int0(args)

            self.__reset_magic__()

        elif (self.__n__ == ‘batch_size’):

            self.__batch_size__ =
int0(args)

            self.__reset_magic__()

        elif (self.__id__ == ‘id’):

            self.__id__ = int0(args)

            self.__reset_magic__()

        elif (self.__n__ == ‘explain’):

            self.__explain__ =
__only__(bool0(args),‘True’)

            self.__reset_magic__()

        elif (self.__n__ == ‘newobj’):

            self.__newobj__ = list0(args)

            self.__reset_magic__()

        elif (self.__n__ == ‘find’):

            if (self.__sleepy_mongoose__)
and (self.__database__) and (self.__collection__):

                p = []

                if (self.__criteria__):

                    p.append(tuple([“criteria”,simplejson.dumps(self.__criteria__)]))

                    self.__criteria__ = None

                if (self.__fields__):

                    p.append(tuple([“fields”,simplejson.dumps(self.__fields__)]))

                    self.__fields__ = None

                if (self.__sort__):

                    p.append(tuple([“sort”,simplejson.dumps(self.__sort__)]))

                    self.__sort__ = None

                if (self.__skip__):

                    p.append(tuple([“skip”,‘%d’%(self.__skip__)]))

                    self.__skip__ = None

                if (self.__limit__):

                    p.append(tuple([“limit”,‘%d’%(self.__limit__)]))

                    self.__limit__ = None

                if (self.__explain__):

                    p.append(tuple([“explain”,‘%s’%(self.__explain__)]))

                    self.__explain__ = None

                if (self.__batch_size__):

                    p.append(tuple([“batch_size”,‘%d’%(self.__batch_size__)]))

                    self.__batch_size__ = None

                if (self.__server_name__):

                    p.append(tuple([“name”,self.__server_name__]))

                url = http://%s/%s/%s/_%s’
% (self.__sleepy_mongoose__,self.__database__,self.__collection__,self.__n__)

                return self.__http_post__(url,tuple(p))

            else:

                self.__handle_exceptions__()

            self.__reset_magic__()

        elif (self.__n__ == ‘insert’):

            if (self.__sleepy_mongoose__)
and (self.__database__) and (self.__collection__):

                p = []

                if (self.__docs__):

                    p.append(tuple([“docs”,simplejson.dumps(self.__docs__)]))

                    self.__docs__ = None

                else:

                    p.append(tuple([“docs”,simplejson.dumps(list0(args))]))

                    self.__docs__ = None

                if (self.__server_name__):

                    p.append(tuple([“name”,self.__server_name__]))

                url = http://%s/%s/%s/_%s’
% (self.__sleepy_mongoose__,self.__database__,self.__collection__,self.__n__)

                return self.__http_post__(url,tuple(p))

            else:

                self.__handle_exceptions__()

            self.__reset_magic__()

        elif (self.__n__ == ‘update’):

            if (self.__sleepy_mongoose__)
and (self.__database__) and (self.__collection__):

                p = []

                if (self.__criteria__):

                    p.append(tuple([“criteria”,simplejson.dumps(self.__criteria__)]))

                    self.__criteria__ = None

                if (self.__newobj__):

                    p.append(tuple([“newobj”,simplejson.dumps(self.__newobj__)]))

                    self.__newobj__ = None

                if (self.__server_name__):

                    p.append(tuple([“name”,self.__server_name__]))

                url = http://%s/%s/%s/_%s’
% (self.__sleepy_mongoose__,self.__database__,self.__collection__,self.__n__)

                return self.__http_post__(url,tuple(p))

            else:

                self.__handle_exceptions__()

            self.__reset_magic__()

        elif (self.__n__ == ‘more’):

            if (self.__sleepy_mongoose__)
and (self.__database__) and (self.__collection__):

                p = []

                if (self.__id__):

                    p.append(tuple([“id”,‘%d’%(self.__id__)]))

                    self.__id__ = None

                if (self.__batch_size__):

                    p.append(tuple([“batch_size”,‘%d’%(self.__batch_size__)]))

                    self.__batch_size__ = None

                if (self.__server_name__):

                    p.append(tuple([“name”,self.__server_name__]))

                q = ‘?’+‘&’.join([‘=’.join(list(t)) for t in p]) if (len(p) > 0) else

                url = http://%s/%s/%s/_%s%s’
% (self.__sleepy_mongoose__,self.__database__,self.__collection__,self.__n__,q)

                return self.__http_get__(url)

            else:

                self.__handle_exceptions__()

            self.__reset_magic__()

        elif (self.__n__ == ‘cmd’):

            if (self.__sleepy_mongoose__)
and (self.__database__):

                p = []

                _c_ = list0(args)

                if (_c_):

                    p.append(tuple([“cmd”,simplejson.dumps(_c_)]))

                url = http://%s/%s/_%s’ % (self.__sleepy_mongoose__,self.__database__,self.__n__)

                return self.__http_post__(url,tuple(p))

            else:

                self.__handle_exceptions__()

            self.__reset_magic__()

        elif (self.__n__ == ‘remove’):

            if (self.__sleepy_mongoose__)
and (self.__database__) and (self.__collection__):

                p = []

                if (self.__criteria____):

                    p.append(tuple([“criteria”,simplejson.dumps(self.__criteria____)]))

                    self.__criteria____ = None

                else:

                    p.append(tuple([“criteria”,simplejson.dumps(list0(args))]))

                    self.__criteria____ = None

                if (self.__server_name__):

                    p.append(tuple([“name”,self.__server_name__]))

                url = http://%s/%s/%s/_%s’
% (self.__sleepy_mongoose__,self.__database__,self.__collection__,self.__n__)

                return self.__http_post__(url,tuple(p))

            else:

                self.__handle_exceptions__()

            self.__reset_magic__()

        elif (self.__n__ == ‘connect’):

            _token = self.__n__

            if (self.__sleepy_mongoose__):

                p = []

                if (self.__server__):

                    p.append(tuple([“server”,self.__server__]))

                if (self.__server_name__):

                    p.append(tuple([“name”,self.__server_name__]))

                url = http://%s/_%s’ % (self.__sleepy_mongoose__,_token)

                return self.__http_post__(url,tuple(p))

            else:

                self.__handle_exceptions__()

            self.__reset_magic__()

        else:

            logging.debug(‘DEBUG: Cannot understand “%s(%s,%s)”‘ % (self.n,args,kwargs))

        return self

 

 

PHP Adventures with Google App Engine (Part 4)

Then I got the bright idea to just slap the WordPress code into a GAE PHP+pQg Project to see what happens…

This is how I approach this sort of task…  Just Do It !

I have been in job interviews where I said something like, “…why not just slap the code in there to see what happens…”. And I was met with snickers and laughs…  if only they knew how completely effective doing that would be… the joke is on them.

By the time I am done, cleaning-up all the obvious errors I will know exactly and precisely what it took to make WordPress 3.1.3 work in a GAE+pQg project.  Then all I need to do is a DIFF via SVN and bingo, I know exactly how to patch WordPress 3.1.3 for others to use via an automated script.  All I need to do is be present for the process…

After a bit more than 2 years of waiting for others to get this done… it seems, I do know the process after-all.

Easy, peasy !

Just Do It !

Stay tuned, folks, WordPress 3.1.3 is heading for GAE sometime soon – well sooner or later, depending on how much time I can pump into this task.  Either way, the GAE SDK will tell me how to get this done, one step at a time.

Proof of Life, right from the GAE SDK for Java !

http://localhost:8888/wordpress/ <– this is where I begin…

Got this right away after just slapping the stock WordPress source for 3.1.3 into the GAE+pQg Project from earlier in this series.

All I need to do now is watch the errors and pick-up the pieces – the SDK will tell me what I need to do, indirectly…

Get rid of references to wp-config.php

So far there will not be a need to look for this “wp-config.php” file when running in the GAE…  gotta make a change there.

And then…

Followed by…

Hmmm… now what should I do this all this ?

Doing this will cause some errors… but what the hay, no harm and no foul… right ?

Wondering what happens next ?!?  So am I !!!

DUH !!!

Now, who didn’t see this one coming ?!?

Time to get some work done on this little problem…

And then I found that one PHP file where all the Db action lives…  Wonder what I can do with this ?  Hmmm ?!?

Keep in mind, I am as causal a PHP coder as anyone might ever care to deal with… But PHP via Java and this does appeal to my sense of irony. Doubly so when it lives in GAE !

Some Analysis

This whole problem of getting WordPress in the Google Cloud would not even exist if the silly WordPress coders had thought to build a nice Abstract Interface through which Database operations are being handled.  On the other hand, maybe this was done and it is simply not all that obvious to me – either way, this is not an overly complex problem.

The nice Abstract Interface I am talking about could have been as easy as having used an nice ORM.  Had WordPress been coded using Python and Django all this could have been over and done already because there would not be an issue of getting WordPress into GAE.  Oops, now you all will think I am biased – darn !

In any case, there is no such thing as a dead db when dealing with GAE !

Making Progress… Onto the next Error of many !

Hmmm…. This might just be a nice time to initialize the Tables using pQg but I think I will get past this one before working that one out.

Just to Recap…

By now I have a method for knowing when, in a crude way, WordPress 3.1.3 is running in the GAE and I have by-passed the first two issues…

(1). It don’t matter whether or not the Db can provide a Connection – GAE’s Database is always online, or so the assumption goes for now.

(2). It don’t matter whether or not the select method does anything or not – much of this sort of thing will be moot when running in GAE.

All this will be refined going forward but for now I just need a crude method for getting there.

Hmmm… PHP Debugging Support Sucks for GAE !

Good thing I have Zend Studio 7.x with a lifetime license.

Guess I need to see where the code goes after those two issues get by-passed…  this is where we separate the men from the boys… I am not one of the boys… LOL

Got Past all this !

Looks nasty, doesn’t it ?  I thought so too for about 10 secs… then I got over it while watching the Star Trek Marathon on SyFy Channel.

Soon as I put in just a bit of code from the previous steps I am now getting something much happier from WordPress 3.1.3 !

It is almost as-if I might just know what I am doing, huh ?!?

Adding my own flair to the process…

Imagine a guy, like me, with more than 3 decades of software development experience having more than enough PHP experience to get the job done with PHP but… I don’t even like PHP and here I am doing what the rest of the PHP coders on the planet have not been able to get done with the latest WordPress 3.x in the GAE !!!   Of course, I have spent more than a casual amount of time doinking around with PHP and WordPress – I even toyed with the idea of recoding WordPress using Django but that was way too involved to matter.

Once I get this done and working I will be LOL all over the place all the while I am building a slick little online web based system for replicating this process for the rest of the world for a small nominal fee, of course.

So who in the room didn’t see this one coming ?

To be expected… LOL

This is a most curious warning…

Fatal Error: ‘wp_die’ is an unknown function.

This one popped-out along the way… this feels like a sleeping bug in the mainline WordPress 3.x codeline.  I am fixing it in my patch for GAE.

 

PHP Adventures with Google App Engine (Part 3)

Sample Project for Google App Engine 1.5.0

You can download this sample project. Working Quercus with working pQd installed and ready to roll.

As always, some assembly may be required to fit your specific needs.  It’s pretty easy to get to this point for those who know where to find the information.

WordPress 3.1.3 for Google App Engine

At first blush, it seems to me, it should be pretty easy to do pretty much whatever anyone might want to do with WordPress simply by making some changes to one single PHP source file which in this case would be the wp-db.php file.  This one file handles SQL interactions for WordPress, generally for MySQL however any Db could be used including but not limited to some kind of REST interface in place of MySQL.

Why there are no ORM’s for PHP such that those who maintain the WordPress Source would not have wanted to use one is a mystery to me but then I would want to use an ORM just to make it easier to connect to a different Db just in case that might become an option.  A bit of research indicates there are some ORM’s for PHP however none of those made their way into the WordPress PHP source code.

Level of Effort for Maintenance of Google AppEngine Patch for WordPress

Assuming all the changes get made to the wp-db.php file and none are required in any other files… the level of effort to maintain this level of functionality for Quercus and the Google App Engine would be minimal and should pose no problems going forward.  My interest lies in creating an automated tool I could monetize such that anyone could upload and install the latest version of WordPress in the Google App Engine… well this speaks for itself.  I would personally love to see millions of people using the Google App Engine for this one purpose alone and if I could make a buck for each installation that would be gravy.  I am just amazed Google has not done this already.

This may become more than just a weekend project just to get WordPress 3.1.3 into the Google Cloud but it is at-least more than possible.

Enjoy.

Letter to Google…

My greatest interests are with Python (5+ yrs spread over the last 10+ yrs) and Android (2+ yrs) and whenever I can do both at the same time (and believe me I have tracked down the tools for doing this) I am right there doing what I can.

…Universal World-Wide Geotagging is my current pet project with a keen eye on making it easy for poor slow Hadoop to retrieve data elements for any region or area of interest driven by Google Maps, of course.  I would want to replace Hadoop with pyTables Pro but then I think Bid Data is all about performance rather than not and for some odd reason Python seems to blow right past whatever Java thinks it can do, in my experience.

I think I can Geotag anything using nothing more than 16 bytes (more or less) of metadata all the while never needing to do any complex computations (Haversine) when retrieving data; driven by whatever area or region a user may be viewing via Google Maps.  This plus a couple well-placed indexes and users can view heat maps in real-time, updated whenever the Map changes in any way.

I am, of course, a huge fan of the Google App Engine and I enjoy using it to the extent possible and then some – these days I am lifting the limits one at a time just to see if I can.  The most recent GAE Limit I lifted was the outgoing GMail limit – now I can easily send more than 2000 emails a day without having to enable billing.

I think I would enjoy the opportunity to build a Cloud based on Stackless Python along with some other nifty Python Tools I have been collecting over the past 5+ years.  Stackless Python + pyTables + Psyco + Cython + Parallel Python  + Some_Work = Cloud with some cool capabilities others have not yet touched on.  I would love to see what happens once people are able to turn persistent Tasklets loose in an Elastic Cloud with distributed data and distributed processing.  In the meantime, I am enjoying what I can do with GAE.

Cheers…

More Android Adventures

Getting some good downloads just from word of mouth and no advertising while working on the back-end to make sure I won’t have to pay any money to Google for the Google App Engine.

BTW – I now have a slick method for ensuring only 2000 emails per day are sent by aggregating over a 1 week period; all excess emails are shunted to a PHP Script running on a low-cost resource I have secured in 5 year intervals (unlimited bandwidth and unlimited disk space, yeah – one of those…).

See the status:

Google App Engine Adventures (May 2011)

I have been quite busy with the Google App Engine lately what with launching my Android App (Free Starbucks), shameless plug and all.

What I need now, more than anything else, before trying to get more serious about giving away loaded Starbucks Cards to those unsuspecting people who told me they wanted to get one or not, is to make sure I don’t have to pay any money to Google just to send some silly emails.

Google allows 2000 emails a day for Free (non-paid) Google App Engine Apps.  The strange part about this is that Google is a Premier EMail Service Provider who surely must have far more capacity to send emails than just 2000 per day per App.

No matter the reason for this desire to make money from a service that is FREE through other means… all one need do is code a Task Queue to handle those 2000 emails per day with the over-flow being sent to a REST Web Service running on another system for which I do pay something like $5.00 per month for a 5 year period of time.  All EMails over 2000 per day get shunted to GMail via a REST Web Service at the rate of 500 requests per second, this is the maximum Task Queue Rate.

What I might do is code this Gmail Handler to only shunt emails to the REST Web Service whenever more than 14,000 emails per week have piled up since one should expect some peak periods during a business day with fewer emails being queued-up on the weekends – might as well let the Google App Engine deal with those 2000 emails per day.

The other method that works would be to track how fast Emails are being queued and then shunt those that would overflow 14,000 per week hit the REST Web Service.

As you can see there are several ways to deal with the overage all without ever causing the Google App Engine to ask for any money even when the App is in a PAID status.

The curious thing for me as a software engineer is this… Google developed a slick API for Gmail before the Google App Engine ever existed so why would the Google Engineers not think all we developers might have to do is code a nice REST Web Service running on some low-cost PHP Server (and there are tons of them out there including those that are far less costly than what one might ever want to pay to Google just to send emails)… and yet here we are with Google wanting to realize some revenue from the Google App Engine being thwarted by their own Free EMail Service through another provider.  What I say is this… “Why not just lift the silly emails per day limit and be done with it… ?!?”  The level of effort is not so much that one might be tempted to skip the issue and just let Google collect money for what is otherwise a FREE Service anyway.  On the other hand, if all I had was some kind of XMPP Server running at home on a laptop I could get this email thingy done for the cost of my in-home High-Speed Broadband; GAE allows 46 million XMPP sends with 1046 GB of transfer for outgoing XMPP activitiy – if all I had was a laptop-based XMPP Server running I could shunt 46 million emails per day with each email being allowed a heafty 24 KB of text – well within the limits of what would be needed to service 46 million notifications via email for my Free Starbucks Android App.  Darn, now I have to go spend some time on this to see what doing this might require… seems too enticing a thing to do than not since I already have a couple laptops running all day every day in my home – one of which is a file replication work-horse and the other I use for other things like development and the like.

What’s next for me is making sure I can handle tons of Data via the Google App Engine without having to pay any money to Google since I pretty much have that Email Limit lifted.

BTW – In case you wondered about the PHP REST Web Service thingy – one can find a suitable script almost anywhere, I did, but then I hacked that script to deal with POST Requests because POST and REST just makes more sense.

For some reason, Amazon EC2 is just not any fun to hack this way because Amazon won’t even let me play with their Cloud unless I have given them my credit card number but Google seems more than willing to let me play for Free.

%d bloggers like this: