SSH Tunnel Magic

SSH Tunnels can be quite useful for those of us who use them to craft our own very secure yet very simple VPNs.

Consider the following diagram:

2-24-2013 10-23-46 AM


Laptop sitting behind a Corporate Firewall connects with an EC2 Instance via SSH forming a secure Tunnel.

EC2 Instance #1 forms an SSH Tunnel to EC2 Instance #2.

Laptop is then able to Tunnel directly to EC2 #2 thus allowing access to an RDS Cluster and a Memcache Cluster directly as though they were connected directly to the laptop.

Why do all this ?

Very simply one can more readily develop a Web App meant to run in an EC2 Instance that use RDS and Memcache hosted at Amazon when the development environment is able to use both RDS and Memcache during development.  Deployment becomes quite trivial.

The Real Deal

The real point for all this should be rather obvious… Port 22 was the only port allowed through the Firewall however Port 2222 is the port EC2 #2 uses for SSH… obviously it was not possible to use Port 2222 directly from the Laptop.  SSH Tunnel resolves this and many other problems without having to make any changes to the Firewall.


Just for fun !!!

So today, just for fun, I had a wild idea !!!

Windows Web Server 2008 hosting IIS 7 running Python 2.7 + Tornado + Django 1.3 doing the same thing I was able to achieve with Ubuntu + Python 2.7 + Django + wsgi + Tornado + nginx !!!

I doubt the Windows performance will match that of Ubuntu however this has been all kinds of fun !!!

See also:  Running Django on Windows (with performance tests) !!!

I was originally interested in doing Django 1.3 with Windows Web Server 2008 and IIS 7 so what I found was pretty darned cool.

Point and click installations for all this stuff was nothing less than amazing !!!  Especially for Windows !!!  WTG Microsoft !!!

What I really want is WebDav so I can share my huge pile of files with myself and only myself via the Internet and WebDav seems to do the trick…  problem is Windows Web Server 2008 doesn’t seem to know how to do WebDav so I have to improvise a bit by making this do the trick as follows:

Tornado does wsgi !!!

Now IIS 7 does Tornado !!!

Easy as 1,2,3 !!!

Oh, and this puts me one step closer to having my own Private Cloud in my home !!!




Ubuntu Enterprise Linux 11.04

Everyone knows RHEL (Red Hat Enterprise Linux) is all “Enterprise” just because “Enterprise” is in the name – Doh !

Now I give you Ubuntu Enterprise Linux 11.04

Method #1

This is really super-simple because as my readers know, the simpler the better even when simpler is largely overlooked by the masses just because it may be perceived to be just-too-simple.

  1. Install RHEL (any version works) in the computer of your choice, the computer cannot be older than 2009 to work. (DO NOT USE A VM)
  2. Install the latest VirtualBox version 4 or later.
  3. Create a VM using VirtuBox 4.x in RHEL.
  4. Install Ubuntu Server or Desktop 11.04 in the VM you created in Step #3.
  5. Done !
Now all I/O will flow through the Magic Unicorn OS known as Red Hat – and now Ubuntu is using Red Hat for everything one might want to use Red Hat for.
Just in case some of you are reaching for your phones to call the nearest Asylum to have little ole me admitted on a 72 hour administrative hold… LOL  Keep in mind this is being done by your nearest Citrix Xen Server 5.6 because it too uses RHEL as the host OS in which you will be expected to run your Guest OS in a VM – hehe.
I actually proposed this to a Manager I have been working with but the idea was not embraced, probably because it just makes too damned much sense especially if the product you are managing runs in Debian but your I.T. support people are telling you they only want to support RHEL because it is the blessed OS for the Enterprise.
This same technique works great for any guest OS – Windows in Red Hat and other variations.

Method #2

Install apt-get in Red Hat – don’t laugh this has been done – google it.
At the end of the day, Linux code is Linux code and works in any Linux – all roads lead to the same China where Linux is concerned.  Even those who say they are adding special powers to their favorite Linux (RHEL) are really doing very little other than branding the same Linux everyone else is using for their own use.  If any Linux were gonna have Magic Unicorn Powers it would surely be RHEL because it says it is “Enterprise” and it should surely be measurable better than the rest but this just ain’t the case, in real terms.
Slap apt-get into RHEL and you get the best of Ubuntu in RHEL without Ubuntu.


Crazy ideas only seem “crazy” until they catch on with the masses.
Not all that long ago we all might have scoffed at the idea of using Virtual Machines rather than real computers until we learned just what the Cost of Operation is for a real server – deploy real servers by the thousands and then get ready to buy your own small power plant because that’s exactly what you will need every time you see the electric bill.
The idea of running a VM inside a host OS is nothing more than yet another way to achieve the same goal as VMWare for ESX and other products you have to spend real money to get.

PogoPlug is FASTER than DropBox for my needs !!!

Been using PogoPlug for a while in parallel with DropBox with PogoPlug Agent running on one of my Mac Mini boxes (seems Apple is good for something after-all albeit a bit pricey as compared with the Atom alternatives since Mac Mini’s run pretty cheap).

PogoPlug is FASTER !!

Well FASTER than DropBox anyway…

FASTER than that damned silly NAS I wasted money on last year – it has something like 8 TB but is as slow as molasses even though it is connected to my LAN via 1 Gbps.  Go figure.

PogoPlug Agent is fast enough to be useful when it says I have 14 TB spread between the slow NAS and the Mac Mini that has 6 TB in the form of 3 TB USB 2.0 drives.  Damn Apple for not supporting USB 3.0 !   LOL   But at-least USB 2.0 is FASTER than SATA II NAS !!!  If you can believe that !!!  Or maybe the Mac Mini is just FASTER than whatever OS is running in the SATA II NAS ?!?   Who knows why it is FASTER… I just like FASTER.

PogoPlug Agent is FAST-enough to allow me to go searching for files from Windows 7 while at work with reasonably quick results even when no file indexing was done by me other than whatever PogoPlug is doing for me.

Is PogoPlug Secure ?

Who knows… all I know is if I didn’t build it then it is likely not secure enough for my tastes.

PogoPlug is likely not doing SSH when shipping files to and fro – seems too fast for any kind of encryption but for all I know they use SSL or nothing.

Back to learning more about the PogoPlug while I trust their software is not allowing every one on the planet into my files like DropBox has been known to do.

Needless to say, I am getting closer to ditching DropBox for my most important files… just getting some other things done before I get around to it while giving PogoPlug a spin for a while.


MySQL for BigData

If all you have to work with is MySQL but you have PetaBytes to store… you could be in trouble unless… you happen to be me…

Assumption #1

Relational databases love executing really small SQL Statements.

Assumption #2

Relational databases do NOT have to use any relational features.

Assumption #3

Networked Object-Oriented data models are very efficient when all you have to work with is a Relational Db as the data management platform.

Assumption #4

BigData solutions tend to use really big heaps of key/value storage systems because the data can be spread-out over a large number of modes easily.

Assumption #5

Many instances of MySQL can execute the same query faster than a single instance because the distributed query can be executed in parallel.

Assumption #6

Forget everything you ever thought you knew about how to cluster MySQL because all that crap won’t help you when you have PetaBytes to store and manage efficiently.

Solution #1

Store your BigData in many instances of MySQL (think 10’s or 100’s) using a Networked Object-Oriented Data Model where key/value pairs are linked to form objects using nothing but Metadata in the form of key/value pairs while spreading the data out to all available MySQL nodes and then execute the SQL required to retrieve Collections of Objects in parallel and MySQL can be nice and fast for BigData.

Caveat #1

Do you know what is meant by “Networked Object-Oriented Data Model” ?!?  Probably not but this gives you something to figure-out while looking for all those cheap computers you will use to form your MySQL Network.

Caveat #2

Do you know what is meant by “executing the same SQL Statement in Parallel” ?!?  Probably not but this gives you something to figure-out while you think about the prior Caveats.

Caveat #3

Do you know the process of fetching data from all those MySQL Instances can be done using a single SQL Statement ?!?  Probably not, but then you probably forgot to read-over and understand Assumption #6 from above.  Think about Collections of Objects more than Rows of Data.

Caveat #4

Keep it super-simple.  Super-Simple runs faster than the other thing.

Computers are really stupid but can be fast.

Stupid requires simple.

Simple is FAST.

BigData is FAST when the solution is parallel but stupid simple.

Caveat #5

Try to optimize each MySQL Instance by increasing the available RAM to a minimum of 4 GB per instance using 32-bit MySQL running in a 32-bit Linux OS but use VmWare Workstation to run each instance using a separate CPU Core with a minimum of 1 VmWare Workstation Instance per CPU Core.  Unless you can find a MySQL Implementation that automatically uses multiple cores and then you have to give some serious thought to how to make all them MySQL Instances execute the same SQL Statements in parallel – better think about this one for a while… I already know how to do this but you might not.


HADOOP Optimization Technique #1

HADOOP is slow !

BigData should be FAST !

Single Server installations for HADOOP tend to want to use the entire multi-core CPU for one single HADOOP instance.

Assumption #1

The Java JVM has NOT been optimized for multiple cores for anything other than garbage collection when one uses an out of the box JRE.

Assumption #2

The HADOOP has NOT been optimized for multiple cores for anything other than garbage collection based on Assumption #1.

Assumption #3

Most servers HADOOP might run on probably have multiple cores especially when Intel or AMD chips are being used due to the need to keep Moore’s Law alive in a Universe where the upper bound for CPU performance is the RAM bus speed.

Assumption #4

VmWare Workstation Appliances can be run each using a separate core when the host OS is Windows Server 2008 R2.

Assumption #5

VmWare Workstation Appliance Instances will be run at the HIGH Priority setting (one level below Real-time for Windows Server 2008 R2).

Assumption #6

VmWare Workstation Appliance Instances will be given 4 GB RAM using 32-bit HADOOP in a 32-bit Linux OS; all software being used is 32-bit.  No 64-bit code will be used.

Possible Solution #1

If the server has 4 cores when run 4 instances of HADOOP each in a separate VmWare Appliance where each VmWare Workstation instance is dedicated to one of the available cores.

Scale for the number of cores.

Continue packing-in separate VmWare Instances using VmWare Workstation until the aggregate performance begins to degrade and then use empirical performance data to determine the optimal configuration.

Caveat #1

Solution #1 has not yet been tried however based on the available information it should produce better performance for HADOOP and/or Java in general.


DropBox.Com is out, PogoPlug.Com is in !!!

Read the fine print from your latest DropBox Policy Statement and you will notice your files belongs to them !

DropBox has been known to leave the door open whenever they wish which means your files can and have been viewed by others.

DropBox likes to play games with your files and mine to get the most from their storage back-end systems.

DropBox is about to get the boot from me simply because there is a better solution with a much lower cost.

The better solution, this month, is PogoPlug.Com

What is PogoPlug ?

If you want to buy the optional hardward you can do so however it ain’t required.  This may be why there are so many PogoPlug units up for sales at eBay lately.

All you need to use PogoPlug is their FREE software.

Download the FREE software, install it on every computer you want to access remotely and then you got your own DropBox without letting others get into your files, potentially.

PogoPlug works GREAT for small files, obviously

A test on a 2.5 GB file resulted in a very slow process of getting that file into my LAN at home but after several hours it was done.

Small files are a breeze, obviously.

Keep in mind I am not using the PogoPlug device when copying files to my own PogoPlug Storage connected to my Mac Mini where the PogoPlug software is installed, so who knows how much faster the PogoPlug device can copy files when the action is performed using the PogoPlug device.  I have concerns as to how easily the PogoPlug device can share files on a LAN – that use-case is more interesting to me than being able to use my own DropBox… time will tell.  Who knows, I may be selling my PogoPlug device on eBay once I have gotten my fun out of it.  Had I known how the PogoPlug software works I would not have bothered with the PogoPlug device, but that’s life !

Now all I need is to get rid of Catch.Com !

I am not all that comfy with allowing others to handle my valuable data with TLC.  Heck, I am not all that into letting Google do it and I surely don’t want DropBox messing around with my goodies any more.

We all should be skeptical of those who claim to provide FREE services because they all got to make some $$ somehow and most of the time they will try selling whatever they can to raise money – they could be selling your information.

Beware… and be safe.  Always use safe computing, use encryption !!!

%d bloggers like this: