PyCon 2010

I just made my submission to give a 30 minute talk at PyCon (US) 2010 in late February of the upcoming year. I was contacted by PyCon staff with the suggestion that I present a talk sharing my various Python based experiences at various places of employ. I made sure to get the submission in earlier this evening before the ability to do so was shutdown as the window for submisions closed for the 2010 event. Now I must wait until sometime in November to hear as to whether or not I'll be presenting. I will keep everyone posted either way.

Update to Post Regarding Hacking & Ruby

This will be a very short entry as it is rather late and I'm looking forward to sleep. I would do but I do feel that I have to get some observations off of my chest after the past 6 hours of exploring ruby (for the third time).

#1. Ruby isn't as intuitive as one might suspect. Maybe python and others of similar influence (groovy) have raised the bar too high in terms of dynamic language syntax and expectations. The standard ruby idioms are inconsistent and ill-named in several cases, mostly involving native data sets.

#2. Namespaces in Ruby are an even bigger mess than perl. To some degree, perl's system seemed to make sense yet from what I've read, seen and with which I experimented, I find the namespace setup for Ruby to be subpar and dare I saw far from fluid in implementation details.

#3. Ruby is indeed very slow, especially when working with the Array types in combination with large datasets and continual pre-requisite 'include?' method calls for each datum in said set. I did find that I was able to achieve the same results wanted via Hash population followed by a dump of keys to an Array with a noticable speedup, removing the need for the very slow 'include?' method. Membership tests are a joy of high level languages, but a drain on some resources, ruby more than others though without a doubt.

#4. The novelty of mutable and immutable version of method calls (collect! vs. collect, slice! vs. slice) is just that. A novelty. This is an ambiguity which I believe does not help to further ease of readability and usability. It further necessitates that non-standard library code implement similar idioms and 'practices' for uniformity's sake with the downside being a snowball effect in this area.

#5. Ruby isn't sure if it wants to be perl, c, smalltalk or itself as can be determined by the mix and match of terms, keywords and standard method names. It doesn't feel like a concrete language that was purpose built, but more like an object system with various sources for tacking on the remaining pieces of the language so as to round out the feature range.

These experiences with Ruby (for the third time) may have been different had I not been spoiled by Python (most notably), or were I not coding in the field for the past 15 years. This is not the case nonetheless. I couldn't see myself coding in this language for anything mission critical or heavy duty and after looking at the problems many of the ruby back-ended software systems and/or websites vs. the other high-level dynamic languages have suffered, it becomes quite clear when industry giants such as Google and IBM throw their weight behind Python.

This isn't meant to be an argument starting post about Ruby vs. Python as they can be found elsewhere, though if the shoe fits...

Ruby & Project Realisations

This isn't going to be a long post as it is rather late in the evening (morning) and not only am I trying to rest my leg (hyperextended my knee playing football (soccer for the Americans out there) on Monday evening), but I'm also in need of greater amounts of sleep having a four month old daughter for whom I am the primary care giver starting tomorrow given that my wife works in the academic world.

Short and to the point (for me at least) is that I'm delving back into Ruby (for the third time chronologically), but for the second time on a 'serious' level (i.e. with the intent to actually produce usable code and not simply proof-of-concept understanding code). I'm realising that while I love python which has been part of my daily work for the past five plus years, moreso Django/python in the past two, that it is becoming my 'Java/C#' if you will. By that I mean that it is my work language. It is a clean and elegant language which allows me to focus on getting what I wish completed, completed with minimal fuss and easy maintainability due to its explicit albeit brief and neatly aligned syntax. I feel though that something is missing.

If I can go back a little (and long time readers from previous versions of this blog circa 2002-2006 would remember me discussing this before) and bring up what eventually became my professional lingua of frustration: perl. Larry Wall's masterpiece which I utilised professionally from as far back as 1995 albeit I was working with rexx and pascal(!) more so then. I used perl and was attracted to it because of its expressive hacker roots, but was eventually disgusted by the lack of a decent enforceable object model for doing any kind of OOP work, not to mention maintainability was not its strong suit regardless of how meticulous one might be as a software engineer/coder, etc. This is what ultimately lead me to look at ruby but only briefly as it had residual taste of perl all over it. I found python shortly thereafter and have been happy ever since, until recently.

Sure I've looked and learned other languages in the meantime (as well as used them for personal and professional purposes), but just for the past three weeks or so I've realised that some of python's strong suit do indeed take some of the more guttural joy out of hacking out code. In my line of work I find formality and structure do wonders at getting solid code and meeting my clients' needs, which is the whole point. I'm at the point professionally where I don't get calls or emails saying that "something broke". It is much akin to Apple computers. Things just work without fail, as should be expected.

This ties into my other piece of the recent puzzle. I'm doing web framework design and implementation (amongst other custom software components) for primarily lifestyle, art and fashion magazines. It does pay the bills and it is at least involved with a creative branch of what can be a boring industry (publishing), though I find myself pining for more intellectually/scientific/theoretical research based projects/content. This isn't going to be happening anytime soon where I'm currently spending my efforts (professionally as it were). I have no design to stop doing what I'm doing and for whom I'm doing said work. I enjoy the relationship I have with my clients and there isn't anything wrong there. I'm being kept busy with new work so that's nothing about which to complain.

What I am looking to do is start working on some more experimental/theoretical designs and codebases/classes/packages in Ruby so that I can further explore the language and enjoy the more 'hack' mindedness which I find comes with such an expressive language. I will most definitely share my results with all the CodeDEVL readership (as well as podcast subscribers). I may even post a screen-cast soon as my copy of Snow Leopard for my Octo-Mac Pro (8-Core) should be here on Friday and includes new screen-cast capturing built-in to Quicktime X.

If anyone is in the Doylestown region of Pennsylvania and would like to meet up to talk code, please drop me a line. My email is simply 'eric' at this domain (assuming you're not reading this from the source blogger domain but the domain for which the header image at the top of the page states clearly.

I'll keep everyone informed. Until next time..

-Eric

New Projects & Version Control Systems

It has been over a month and a half since my last entry and I apologise for the delay. I have been working tirelessly to finish up my primary design, development, implementation and maintenance functions on what has been my primary project for the past thirteen months. I've been doing this so that I may jump head first into my next project, which while not being a 'huge' undertaking is big enough. By big enough I mean that doing well at the project will land me the role of redesigning a major site from the ground up into the Django framework.

For the past 13 months I've been working heavily in the world of Django and find myself absolutely enamoured with it. I have designed a considerable amount of applications for Django, though none of them are released to the world because they are the product of a contract. My field of expertise was never in that of online applications and/or frameworks so it is stil a new world for me (if we exclude the nine years running a twelve line bulletin board system on a heavily modified Amiga).

I've always been happiest working in middleware and engineering backend systems, though I have to say that I'm finding a certain level of pleasure from the results of my front-end work. When i wrote backend systems, the only others who could appreciate said system(s) other than myself, were developers privy to the project. On frontend systems, there are actual users who I would like to think 'benefit' from my designs and implementations. It is nice to know from actual users that your work is appreciated. Call it ego stroking, or call it what you will but I find that it does inspire one to keep pressing on regardless. I happy to be doing these projects and as soon as it is launched, I'll be happy to post about it here.

This brings me onto the second purpose of this entry, my new favourite version control system. In the beginning of my experience using version control systems back in the 1990's, I simply used CVS like the rest of the world (sans those lucky enough to use Perforce). As time and technologies advanced to newer and better systems, I followed along and moved up to Subversion. Subversion was a well done upgrade for CVS users and easy enough for the uninitiated to learn. I used this most recently when I did work for Curlington Boat Factory (name changed to project myself) writing their point-of-sale returns authorisation/queue system in Python for four hundred some odd cash registers. This worked our quite well despite the fact that the software house for whom my partner (at the time) and I were contracting weren't able to provide the subversion server on their machines until we were ⁷/₈^ths of the way through said project. We ended up (very early on mind you) taking matters into our own hands and setting up on my partner's server at his apartment. This wasn't looked upon highly by said software house, but any CVS on any server regardless of not being in control of the software house was far better than no source control at all.

Next we come to my brief stint working for a OCD kid who started his business at the right time with mom and dad's money, but due to poor decisions on his behalf watched it dwindle in popularity, further plagued by poor management and attempts to treat a small business as if it were a conglomerate thereby sealing its fate with a tender kiss on its cheek. I thankfully left the company before the owner went psycho and let go of all the talent in a horribly vicious manner. Before all of that ugliness transpired, I went forward with trying the newest and brightest trend in distributed version control. I am speaking of the one and only 'Git', championed by a certain arrogant (albeit brilliant) Finnish kernel programmer. There was an immediate joy in using Git and I have to say that most striking feature is the blazing speed when dealing with a large quantity of files. I established all of the existing software base for said company (over ten years worth) into its first repository, utilising Git. Yes, for over ten years, there was no VCS in place. Git really did shine in this role, though I am pretty sure that due to myself and the system administrator not longer being there, cob webs must be forming in the places where Git once speedily did its work.

Let us fast forward to my newest discovery (if you can actually call it one as such). On this newest project which I just started today full time, I have been doing administrative setup work for the past few weeks on and off in small gaps of time as they availed themselves to me. The normal deal of establishing a dedicated server on a FreeBSD host (as Linux just will not cut it for me even though I've been using it since kernel v0.99d), along with all the necessary bits. Once the development languages, utilities, database and web servers were in place the issue of VCS came into play. For this time around I decided that Git, while great for many projects doesn't seem as pythonic a package as I'd like. Using Python long enough really makes one desire beauty, power, simplicity and consistency in all of their tools. This brings us to the most pythonic vcs of them all, Mercurial (known simply by its periodic table element, Hg). Mercurial, like Git is also a distributed version (revision) control system. Easy to setup, easy to use and highly recommended. I'm not going to wax poetic about the differences between these systems as they all serve a purpose. What I will do is state very clearly that Mercurial feels the most natural, works rather well and is quick at what it does. Your mileage may vary.

Until next time...

Django Projects Galore and Vile Ethics of other Coding Firms..

It has happened yet again. I've been brought on for another Django project. This time it is about taking an existing site for yet another magazine publisher and converting their Wordpress driven site into a real full-blown site complete with blogs, forums, user profiles, dynamic main page content, complete customisation from within the framework and included applications and ultimately a site in which a developer is not needed for day to day changes.

As it exists, this publication is an off shoot of their primary magazine. A magazine whose website was written and managed by an outside from that I believe coded the entire site so as to require additional invoicing and servicing for all but the most minute changes. This is a disgusting business model and one with which I've had the misfortune of experiencing whilst working as a sub-contractor to a sub-contractor for Burlington Coat Factory.

The H-1B Visa Project Manager who wasn't too fond of me because I don't believe that coding in dress attire and/or a tie makes someone a better worker (to the contrary, i will NOT wear dress attire for day to day work as it is a pointless expression of old brick-and-mortar mindsets). He also was the first time that I was reprimanded for having an eloquent solution that adapted automatically to the growth needs of the end-clients database/system. I wrote the software to handle dynamically gathering and sequencing additional 'like' fields as they were added to Burlington's transaction schema. The way I designed and wrote the software, the MOMENT the schema changed, my software contributions would immediately include relevant changes, without a restart of any of the daemons I engineered. I was told that the reason why i shouldn't have done this is because the sub-contractor for which i was writing this code could then go back and charge an exorbitant amount of money each time minor changes were made. This disgusts me, and I find it ethically wrong.

I am an engineer and work independently by choice as I can first and foremost hold myself and solutions I produce, to higher standards; delivering what my clients want and need, not solely based upon what they say they want and most definitely not building them into a corner for profit over common decency and professional standards.

Once the project has been completed, I will be quite happy to share the url(s) with CodeDEVL readers.

SimulaE Cartography Program - simulae-kaart.py

As a quick follow up to the previous entry regarding rooms being a void and not an object, I can happily announce that the first working alpha of simulae-kaart has been committed to code this evening.

The function of this bit of code is simple:  Produce the necessary wall/barrier objects needed to create the voidspaces we call rooms.

The current working codebase does the following already:

* Allow graphical design in a 2d environment utilising unicode representative tokens.

* Returns Multidimensional Lists designating literal start & end points of wall objects

* Allows for walls in traditional cardinal orientation (north-south, east-west, nw-se & ne-sw)

* Also allows for arbitrary wall/barrier positioning at any angle (true 360 degrees).

* Default scale based on cubic decimetres, variable scaling coming soon.

Upcoming functionality to be added:

- Auto population of a given map grid

- Auto insertion of portal objects inside any giveen barrier/wall object.

- Automatic map scaling down to the smallest micro and largest macro levels

Stay tuned for further information.  If anyone else is interested in signing up for future beta

testing, I can be reached at this domain, under the email account of eric. 

-e

SimulaE - Model Update

I've made mention of my virtual world simulation project on multiple blog entries, most recently related to Ruby and making a rewrite of the existing engine in said language to test out its applicability (the language not the simulation), which by the way I found to be the lesser language for this kind of application, but I'm not saying anything about the language as a whole.  Either way, onto SimulaE, which after all is what this post is about today.

I have been rattling my brains (and occasionally those of my friends) regarding the basic SimulaE Object model, which up to this point has served its purpose.  Though the time has come for it to evolve.  It can implemented in a simpler manner and I have known this in my mind all along though it hasn't been an issue in the process of designing the parser which has for the most part been satisfactorily completed. Now that my focus has returned to the object model, I feel it a fine time to share that update thus far.  Comments are always welcome and most are appreciated.  

The fault before was that I broke the objects into the wrong sub classes.  Originally I had the parent Object class,  and subclasses for Room objects, Exist Objects and Person Objects.  This is a horrible idea and leads to unnecessary complexity.  

The newest model:

Object (super class)

MovableObject (isa Object)

PortalObject (isa Object, isnota MovableObject)

PortalObjects, hereafter POB are a much more dynamic version of the previous "room" objects.   A POB behaves in the following manner(s):

1. Object(s) enter into the object containing said portal.

2. Exits an object when contained therein.

3. Can exit (when specified) to a specific object, though by default exists to the parent container object.

If a MovableObject, hereafter MOB, is not contained within the same container as the POB, POB leads into the container.  

If the MOB is contained within the same container as the POB, POB leads out of the container.

By working in this manner, we ensure that the simulation object model more closely mimics the real world, whilst still allowing for exceptions to transpire for non-real world based applications of this group of models as well as anything they may be working on at CERN in Switzerland which breaks our current understanding of physics. 

I haven't had time to implement this new set of models yet, but as I write that code, I will be posting the revised Python source.

Quick Django Tip: Dynamic Application Object Retrieval

In my recent django adventures I needed to introduce site-wide search functionality and in the process of doing so, encountered a small roadblock towards doing so.  Apparently due to the nature of Django's API for db interaction (as of the last stable release version), there is a limitation as to the use of python variables in API calls.  I found this to be a hinderance, but only for so long.  

The follow code snippet was something I whipped together which by utilising Python's 'eval' built-in, overcame the aforementioned limitation regarding the API's ability to interpolate native Python (e.g. non-django explicit) varaibles.

Things to know to understand the following example:  

 search_input is a list of cleaned and pre-processed user-driven terms, split into separate expressions, (e.g. ["dynamic langauge", "agile", "programming", "paradigm"]). 

 search_schema is a dictionary in which the key is the django model/class through whose objects we are attempting to search, and the value is a list of specific model attributes to attempt said search.  (e.g.   User_Profile : ['firstname' , 'lastname', 'address_1', 'bio_info', 'favourite_books'])

container_xref is a simple alias mapping for the actual django application names to our internal references inside this search code base.  Obviously this whole bit could be written without said setup, but for readability given the scope of the actual application involved, and the fact that I was not searching simply a few static fields in one django application, but several dozen fields through about two dozen separate applications, this container_xref dict was appropriate.   It is through this mapping dict which we place any matched object results (so as to not waste any additional space via unnecessary list initialisations.) for eventual results generation.

Note: the key "total_results" in the container_xref was a simple means of keeping track of overall search matches, rather than relying upon the Django templating engine (view) from doing work responsible from the processing (controller) perspective.  In retrospect, there are better ways this could have been handled, and in future point revisions, this will be addressed.

------

for search_string in search_input:

  for application in search_schema.keys():

    for attribute in search_schema[application]:

code_to_eval = "%s.objects.filter( %s__icontains='%s' ).order_by('-id')" %       

                     (str(application), str(attribute), str(search_string))

        try:

          eval_results = eval(code_to_eval)

          for eval_result in eval_results:

            if eval_result not in container_xref[application]:

              container_xref[application].append(eval_result)

              container_xref['total_results'] += 1

        except Exception:

    ### Case specific exception handler types, assignments and resultant actions 

          ### specific to each application in which the above is implemented, go here.

-----

As can be seen from the above, simple inline substitution proceeded by evaluation of said string results in post-compilation dynamic search functionalities within django, addressing simple problems one might run into with the existing API which will most likely be addressed in future versions.  

Your results may vary.

Eric

Ruby: Somehow I overlooked this Gem of a Language

About 5 years ago I started looking into a language that prior to it's "Rails" fame, was lesser known and even lesser utilised.  I tried it a bit and found it leaving me wanting more.  I've kept tabs on it over the years, reading the tutorials and writing several quasi-AI experimental applications for my SimulaE research, but I ended up being enticed by Python, a language which I stand by, including the wonderful (but until recently unused by me) framework Django (Python's Rail's equivalent, focused on Publishing).   

I've programmed professionally using Python for several contracts/years now and find it quite enjoyable.  In fact, I'm currently coding specifically in Python for Inkedmagonline.com, but that doesn't mean that I don't continue my personal exploration and education for both personal and professional reasons.  I decided to re-experience Ruby by picking up the hallowed PickAxe book and giving it another honest chance.  I'm glad I did.  

I believe that Python, and the values espoused in Tim Peter's "De Zen van Python" (The Zen of Python) (my copy just happens to be in Dutch, otherwise I'd post it for others) have helped me to look at Ruby in a different light.  There are some key differences in the two languages, but I can see now the inherent power in Ruby that I was overlooking before.  In fact, some of those key pieces, syntactically as they were which make Ruby so enticing this time around are the very same 'features' I feel are missing in Python.  It only took me working in an environment with situations where said language features would prove the best solution to the problem(s) on hand for me to realise it.

I am not going to spend time detailing all of the specifics, though I may mention one or two nonetheless.  I'm more so bringing this point up so that others might be reminded that giving something new a single chance might be to your own disadvantage.  After all, I didn't like Python the first time I tried it either.  I think it is partially a matter of how we grow as developers that allow us to know what we're missing, that same spark of realisation that gives us the "a ha" of relief when we find it hiding in a new language, programming methodology, etc.  

What brought me back to looking into Ruby a second time is of all things, Smalltalk.  The whole "everything is an object" concept is nothing new to me, or to programming languages.  However in dynamic strongly typed languages, it is.  More importantly is manner of how even rudimentary objects such as integers, floats and strings are treated in Ruby.  They have methods which can be both called using the standard instance.methodname call format, and have their standard methods overridden.  The second being something far more wonky and kludgy in Python (and a non-option in perl).  

The fact that key methods are instance based such as "len" or "length" for example makes a world of difference for consistency.  It speaks to the overall design that "Matz" (Yukihiro Matsumoto creator of Ruby) had in mind during the planning phase.  In Python, a language in which everything is truly an object as well, this starts to get rather confusing.  While Python does treat every integer and string as an object, it mixes the traditional functional paradigm for calling items such as 'len' so that to find the value of 'a', one would type len(a), as opposed to the more object based a.len ..  This seems counter-intuitive and quite frankly a real surprise when you look at the overall design of Python.

I'm not ripping on Python as I do wholeheartedly enjoy the language, I'm just starting to feel aches and pains over decisions which are ingrained into the language, as well as not being seen as an issue or being addressed in py3k (or Python 3000/Python v3.0) as it were.  I just think that my eyes have been opened to Ruby again and I like what I'm seeing.  I am actively looking to find a future professionally as it were utilising it as nothing beats having fun while accomplishing what one would hope accounts to 'great' things.  We'll see what the future holds.

Next Step:  Migrate my SimulaE virtual world/real model object simulation from Python into Ruby as a test run.  Lather, rinse, repeat and then see what the side-by-side comparison's look like.

Until next time...

Arc: An evolution of Lisp/Scheme, or a outdated implementation at launch.

So the day has finally come in which prolific geek & insightful essayist Paul Graham along with Robert Morris released Arc, their evolutionary love child of Lisp & Scheme.  I think it is safe to say that many of us have (and will continue to) read Paul's wonderful essays on a multitude of geek and coder centric topics, and generally with much joy and agreement.   Many of us have been following the work that Graham and Morris have been undertaking with the new baby "Arc". 

Now that this day has arrived, we can see that it wasn't as deserving of all the pomp and circumstance to which we were planning for it to be attributed.  Seems that there are a considerable amount of deficiencies and intentional short comings to the language.  Normally this wouldn't be seen as anything out of the ordinary for a 'new' language, and would be generally a non-issue.  

The problem is that this is the year 2008 and people have come to expect more from their languages.  Ignoring established standards along with ignoring the need for designing to meet the needs of developers globally all the while using the cop-out of it being purely for exploratory programming is just bad form.   

The world doesn't need another Lisp, the original is wonderful just as it is, that's part of its beauty.  I don't see anything in Arc that couldn't be done as functions and/or macros in Lisp that required the entire 'creation' of a new language.  The way I see it is this; for prototyping as well as production usage we already have several languages that excel in those domains, specifically Lisp and Python.  

Generally the idea of rewriting existing languages with little difference from their predecessor(s) is a waste of time and effort which could've been better spent elsewhere.   There are exceptions to this scenario such as that of Ruby.  It is a language that in the past I used to dislike because of certain key flexibilities much akin to the reasons for spaghetti perl, but it successfully fixes many of perl's wrongs, and corrects some of Pythons short comings as well.  It serves serves enough of a purpose as that of supplanting perl with a better thought out design, and thanks to Rails, its future looks solid.  

Arc on the other hand is like a still born fetus.  Much was expected and there was potential to be just like its parent(s), but death was announced during a delayed delivery and upon further examination, it was discovered that it wasn't a proper offspring, but a clone in fetal form.  My advice to Paul and Robert is the following:  recall the release, and make it truly something worthy of release, with proper compatibility and compliance with modern norms.  Make it usable to others as opposed to just a few tinkerers interested more in lisp and scheme basics. 

An on a more pleasant note to Paul;  Please keep on writing your essays and providing your viewpoints as they are appreciated, I just wish that your judgement call in the case of 'Arc' was as well thought out as your writings have proven to be over time.

The New Django Powered Inked Magazine Website is Up!

This will be rather brief as it is more of an announcement than one of my more traditional journal entries. 

After a  short two or so months from start to finish, I have successfully setup my first Django powered website.  Inked Magazine has been relaunched effective January 2nd, 2008.  This replaces the Joomla powered site which existed prior to both myself and the current ownership of the magazine were involved in the project.  

Phase one has been completed, with extras.  The customer photo galleries, the cover photo application, the user profile system as well as the main magazine feature areas have been established and are active.  As of today, I will have available for all registered users the ability to host a blog on our site (using our software which I wrote in a few days), mind you it is still early in the feature process, though it without doubt serves most blog authors needs.  

I will be adding features as time progresses, but until mid-January 2008, I'm on a tight schedule to build the entire forum application so that the beginning of the "New and Improved" Inked Network can go live.  

I don't think one needs Ruby on Rails when you have Django.  Much of the same great functionality and without having to use Ruby all the while using Python.  [Update: After having revisited Ruby as a language on its own, sans Rails notoriety, I've found that my previous assertions regarding Rails specifically was unwarranted.  It simply took my experiences with Django and Python to make Ruby and Rails far clearer to me.]

I will return to my normal posting after the Forum application is up and running.  

Till then, keep on coding.

Django and Gantt Charts

It has now been almost a full week since I started the complete inkedmag.com site and infrastructure redesign in Python using the Django web publishing framework on FreeBSD, and I am happy to report that it is awesome. Mind you we're talking version .96 of the product, yet it truly is a dream with which to work.

I only recently started working with (and writing about) Cheetah, a wonderful python template engine, and had to very quickly learn yet another (Django's own template system). I must admit that Cheetah is easier to ready and learn quickly, but Django's system is considerably more agile in terms of conditionals and modifiers inside the template itself. There even happens to be a simple mechanism for cycling through a list continually changing on each iteration of the loop within which the cycle conditional resides. Simply put, it is wonderful for automatically changing the background colour of a row in a list.

For those unfamiliar with Django, it is simply one of the better web frameworks for content publishing on the web these days. While learning curve can be a little steep for some pieces of the framework, as a whole the speed at which once can produce working pages and applications is staggering. The ease and elegance of the system truly makes one enjoying creating new applications within the framework.

The first application I chose to migrate from native python into the framework was a simple store locator. The new version not only is considerably less lines of code, the database management was done for me at application creation/initialisation. I then simply exported the data from my existing application and imported it into the new table(s) Django created.

I could go on waxing poetic about every little bell and whistle, but I'd just be paraphrasing what many others have already pointed out online and otherwise. Don't think of it as Ruby on Rails because it isn't, though that isn't to be taken as an insult to Ruby. It is much more focused, cleaner and far simpler to setup and get running, including all of its own admin interfaces for the applications you create, as well as its own standalone development web server. Check it out, you won't be disappointed. This is going to save me a considerable amount of time.

Which brings me to my second point; Gantt charts. They are simply not something I find myself utilising on any regular basis, though I think that is going to change. I'm my own boss and have found that gantt charts produce the easiest visual way to show people the various pieces necessary for a project, when each portion can be expected to start and finish, all in parallel with the other projects for which I'm responsible (and/or coordinating).

I feel that the use of this tool more than others really gives a great method by which to see which projects will take the bulk of the time, and what projects overlap, etc. We have a system rewrite to produce and a whole server to replace, not to mention migrating certain custom software into the framework all before the new year. This is doable, but only because we've clearly set realistic (though tight nonetheless) goals and time frames. Consider using a gantt chart if you have more than one project or component of a project which needs to be done in a given time frame. Use one if you need to share with one or more people your schedule and need them to understand as quickly and clearly as possible that with which you are juggling or dealing. You find yourself quickly addicted to its usability.

Where Javascript Helps the User Experience.

     As is well known by a good deal of the regular readers of this blog, I have moved back into the world of being an independent Software Engineer, in an open ended contract with Pinchazo Publishing Group, Inc.  Their best known publications are Nylon (featured recently in the newest iPhone commercials from Apple), and the recently re-launched Inked, a tattoo-culture centric magazine, both of which are distributed globally.  

I bring up all of these specifics because it marks a decidedly big shift in my own coding career.  I have traditionally worked on back-end and middle-ware systems, making incompatible systems play nicely together, hardly have I ever had to deal with front ends and end user interactivity.  Sure, I did the web page for Thelesis, a non-profit group, including the framework and almost all graphics, and continue to maintain that site to this very day.  As a whole though, I never felt a desire to deal with the front end, I like the logic behind the interface point of view.  Well, now I'm in a situation where I'm needed to make tools with which end users will interact primarily.  Odd change eh?

It was at my previous employer, Blue Gravity Communications, a wonderful FreeBSD centric (with some Linux) hosting company that I found myself needing to really start to learn Javascript in order to convenience the end users in the selection processes.  It was here that I started to learn more about a language with which I never thought I would have a need.   I couple this to mention from a good friend about a post from a wonderful developer (my aforementioned friend's previous co-worker), regarding how wonderful javascript can be in one's toolbox.

With all of this in mind, I needed to jump into the world of user friendly interfaces.  I know from my own experiences perusing the web that I know what a non-intrusive interface is like, but it really isn't best to ask developers what a good interface is all about.  By nature, we are far simpler in our needs and all too willing to overlook certain practises that we don't see as a problem.  Keep in mind, many developers, myself included, still prefer command line interfaces because of how much quicker they generally are.  

I've already called upon javascript for certain pre-submit form checking, which is ultimately a convenience to the end user because it saves them having to reload the page, or worse off, play hit and miss with multiple loops of the process of submitting and seeing what was wrong with their form submission.  This is a very unfriendly approach in 2007 which is sadly still utilised by many large web based corporations.  

This time was different as I was coding the first version (what I would normally tag as a beta) of Inked magazine's online tattoo gallery.  The concept is simple, allow users (and store owners) to upload tattoo photos for general viewing on the website, and if a specific photo is from a tattoo shop/parlour in our tattoo shop database (covering 4 out of every 5 shops in the United States), make a link to that shop so that browsers of the gallery can associate certain quality work with a given producer of body art.  Very simple, great use for your standard LAMP (in this case BAMP [BSD, Apache, MySQL and Python]) configuration. 

However, end users could care less about the underlying technology, they care about ease of use.   It was with this mentality in mind that I approached the gallery's first incarnation.   Limit the amount of non-photo graphics (for speed), limit the amount of time a page actually needs to be refreshed and/or requested, and make the controls relevant and simple to understand.

This was (I believe successfully) achieved by use of a strong reliable template engine for the purpose of controlling what user control elements were presented for navigation on any given screen.  Ultimately, if a person is browsing paginated libraries of content, we only wish to have he navigation controls relevant to where said individual is in their browsing activities, visible.  Meaning that if a person is on the first page of a five page gallery, don't render the button that links to the first page, and don't render the button which links to the previous page (as it is non-existant).  Likewise, we don't want have buttons for the "next" page, or the "final page" when we're actually on it.  This may seem logical, which I'd like to think it is, yet so many seem to overlook these kinds of details.  These are details which can cause frustration from users who unintentionally click on a button which goes to the same page they are already browsing, or in the case of a "ghosted" button, make them wonder why it isn't working at all.  Only present that which is needed, and nothing else if at all possible.

More importantly, and even less obstructive, javascript for auto-zooming the photo gallery images themselves without having to pop-up a window, or even worse, replace the current viewing page with a simple image link or dedicated page with headers and footers in addition to the image.  These elements are time killers, and javascript is one wonderful way in which to resolve the problem.  Not only does this kind of visual add an interactive feel to the page(s), far more similar to the way a user would experience their own operating system (especially these days with candy like OSes), but it means they aren't hindered by unnecessary delays and can focus clearly on that for which they came to the page in the first place.  To view photos of tattoos that interest them, or share theirs with the world.

Cheetah, Python's Powerful Template Engine

About six months ago I wrote an entry about using the Template Toolkit for perl, and how I found that it was almost as if giving perl a little taste of Python. Now, fast forward to present time and I find myself as my own boss once again and in a dedicated open-ended contract with Pinchazo Publishing Group for Nylon Magazine and more recently, Inked Magazine. This opportunity has also proved to be beneficial for me in that I get to choose the technologies with which to arm these businesses moving forward for their presence on the internet.

This brings me to some realisations to which i came today. Python's template engine "Cheetah" is considerably better than aforementioned Template Toolkit for perl. I'm currently writing a new online gallery application using Python, MySQL, Javascript, CSS and of course HTML on a BSD server running Apache 2.2. Today was the first actual coding day for implementing my design, and while there were certain changes of some underlying routines, I have to say that it is moving along much quicker and smoother than alloted/anticipated. I attribute this heavily to the ease of use found within the Cheetah library.

Some template engines add a quasi familiar set of language constructs which make using such a system doable but with that kludgey feeling. That is not that case with Cheetah and in true Python fashion, it integrates using constructs that closely parallel the standard Python syntax, as well as offering several additional alternatives to help adapt in various situations and code bases.

The beauty of using template system (as has been said before) is that you add an additional layer of separation of code from display to the point that in team/diverse environments, the coders and artists don't interfere with one another. A simple protocol of self-discipline for each individual to stick to their roles ensures that both content and display functionalities can be developed, and changed simultaneously without concern over coordinating the end result. The busier the schedule, the crazier the deadline, the quicker (and with a much higher level of confidence and lower level of stress) that a project can be implemented/modified/redesigned.

Once the gallery is fully operational, I will be updating this post to add a working link to the site. It should only be a few days from the committal of this blog entry, so keep up to date by subscribing via the codedevl rss feed (courtesy of atom).

Building a Better Box for a Client

As was mentioned in a previous entry, I stated that I was willing to try being an independent contractor again sometime.  That time is now.  As such, my new corporate overlords are a media publishing group and I’ve been called in to do a ground up architecture and engineering job, along with continued long term maintenance.  Unlike before this situation appeals to me because it lacks on of the most common issues in the realm of development in general, legacy upkeep.  Sure, there is the little issue pertaining to a php application which needs to be put on a website short term, but after that we’ll be trying to limit php to specific applications on a limited (only as-needed) basis.

Ultimately we’re looking at starting with a fresh new remote server and that being said, my own experience brings me down to a quasi LAMP setup.  Traditionally I’ve found that when I want a rock solid remote host, one which I know can go years on end in a reliable manner,  I chose FreeBSD.  Nothing against Linux other than I find it fine for a Desktop or a Server, but more so the desktop than the server.  I find that there still is no substitue for Apache when it comes to matters related in pushing out pages to the web.  

Next is a point of contention, the database.  I’ve been using MySQL since version 3.23.24a (or something around that revision number), and have found that it met my needs about half of the time.  Much of it (at the time) revolved around the issues pertaining to MySQL’s myisam faults and weaknesses regarding concurrence in high insert/update environments.   I know some people out there (many actually) will start arguing this point right away, and I still say unto you that this is a known weakness.  The myisam database storage engine is designed for speed, not high-concurrecy, nor transaction safety.  When paird with the InnoDB engine, and the removal of the auto-commit flag (as it negates the whole point of using a transaction safe engine), most of those issues disappear.  The other issues pertain to foreign keys, store procedures, etc., which have been slowly addressed in versions since the 3.xx base.  Now we’re at the 5.xx family and much has improved.  

However, all of these points still cause MySQL to pale in comparison compared to PostgreSQL.  True, MySQL has proven to be very capable and very popular, especially among the Linux crowd and cheap hosting crowd.  I will be installing MySQL on the new machine to handle support of third party web applications, though when it comes to hosting any important data, there can be only one choice, and it isn’t MySQL.  PostgreSQL is the clear winner here, the closest db engine we have to Oracle without being Oracle.  

Finally, we approach the last letter in our acronym.  The ‘P’, which can stand for a multitude of langauges scripting, web and otherwise.  We have PHP which is wonderful for quick and simple (an a handful of not so quick and simple) web based applications.  It is an easy language for the novice to learn, and in the hands of an expert, even more so capable, though it has its faults, and among those security being the top.  Much effort has been made (especially post 4.2.3 and 5.x versions/trees, and I hope to see this evolution continue, though I still don’t see myself using it much as I don’t feel compelled by the language as a whole.  

Next we move to another ‘P’, which in actuality is a ‘p’, perl.  The oldest of the languages we’re discussing here, but not by that much of a time frame.  Perl grew out of the personal needs of a C programmer, Larry Wall as a combination replacement of both sed and awk (amongst other Unix utilities).  I’ve been paid to code in Perl for the better part of the past 11 or so years, and I can say after all of that time several things.  On the good side, perl is found everywhere, has a large code base, and is fast.   On the bad side, I’ll have to limit my dislikes and faults found within perl so that this entry doesn’t go on for thousands of words.  Limiting my issues with perl we will see that it allows, almost seduces people into writing ugly, cryptic code.  Yes, yes, yes, the code some perl monks/mongers write may be very crafty.  Crafty does not equate with great, let alone good quality.  

All too often we see people referring to the TIMTOWDI (There Is More Than One Way to Do It) mindset of perl as being a benefit, though I see it (time and time against, countless of codebases later, even the CPAN library) as being a flaw and weakness.  If you don’t enforce a certain level of clean design into the language itself, you end up with a mess, or as many others have stated, a write-only language, one which even the author(s) of programs cannot read/decipher down the line.  My suggestion is for perl coders to follow Java coding guidelines.  I mean, we’re talking about a language that doesn’t has several decent levels of rules and coding enforcement (such as the ‘use strict’ pragma), but is so foolish as to allow people to code in a manner contrary to that pragma when it already exists in the core language.  How about a proper exception handling system?  Eval blocks or non-core/second-class libraries do not make a proper first class handling system.  This is asinine in a language that has been around for over 20 years as of this writing.  I could go on, but I’d rather not.

This brings us to a non-p ‘P’ in LAMP, Ruby.  Ruby to me is an evolution of perl in many regards, especially its object based design and proper exception handling system, however it still fails miserably in the sense of massive overuse of tokens and pascal-esque verbatim block terminators.  Rails has made Ruby a mainstream language, and I do feel that it has considerable potential ever more so than Rails alone, but it still has a ways to go when it comes to speed and cleanliness.  Matz has be working hard on it, and I’d like to think there are great things ahead for the language from the land of the rising sun, but at the current moment, I still find it lacking as non-web specific development platform.

Finally we come to where I’m heading, and I’m sure others have already figured that one out.  Python rounds out the last ‘P’ in the equation.  Python is almost as old as Perl, and is rooted in development languages as opposed to the shell and various utilities.  In this language we see a very capable, 100% object-based development language which is capable of handling coding projects of any size which espouses clean design, human readability, code re-use, distributable byte-code compiled classes/applications and proper exception handling as a first class citizen.  

So as we can see where, the solution i find most reliable and long-term maintainable with minimal development time, maximum return for design/coding efforts, security and platform flexibility is simple.  So it isn’t technically a “LAMP” solutions, more as it is a BAMPP solution encompassing BSD for the OS, Apache for the web serving, MySQL and PostgreSQL for the database(s), and Python for application development.  

I came to the above choices after years of experimenting and experiencing and I do suggest others experiment on their own if they have that luxury/time frame available to them, but I do offer the above as a recommendation as I would (and have, and will) bet my own future livelihood on the flexibility and reliability of the aforementioned combination of technologies.  

Write Source Code for Other Developers, Not the Computer.

I’m not sure as to whom to attribute the following statistic, but i believe it was something along the lines of this;  Code is read vs. written on at a 10:1 ratio, meaning that the is far more reviewing of any specific codebase than there is writing to said code.  Furthermore, the majority of software positions involve maintaining and modifying existing code as opposed to creation of new code from the ground up.

To what does all of this allude?  The importance of writing clean code.  Knowing full well that other developers are going to have to read, understand and most likely modify your code in question at some point(s) in the future.  This is where our responsibility as software professionals (even in the case of hobbyists) comes into play.  

Several languages have tried to address this problem by intrinsic design decisions.  Most notably among those in recent times are Java and Python.  Java does so by its explicitness by design, and Python by its forced formatted a la the whitespace requirement.  Both are effective in what they do, however there are still a multitude of ways in which both can be written in a harder to read format.  Obviously choice of variable, function, class and object reference names is a very large point of readability (or not) which really cannot be enforced by a language specification.  Let us take a look at this very issue and while we’re at it, i’ll be clear that this is not a Python vs. Java issue discussion.

All too easily so many coders (I know this from having had to look at, understanding and refactor their code) overlook one of the best sources for building readable code, and that is their naming convention.  There have been several best practices and coding style specifications documents produced that one might think me as flogging a dead horse, but I assure you this is not the case.  

In the following examples we see a variation of languages and how we might commonly see the same variable name referenced (and initialised as it were):

Smalltalk:

num_of_doors = 4 ;

Python:

numberOfDoors = 4    OR    numDoors = 4    OR    number_of_doors = 4

Ruby:

numberOfDoors = 4;    OR    numDoors = 4;    OR    number_of_doors = 4;

Java, C#:

int numberOfDoors = 4;    OR    int numDoors = 4;    OR    int number_of_doors = 4;

Lisp:

number-of-doors := 4;

C, C++:

int intNumDrs = 4;    OR    int num_drs = 4;    OR    int int_drs = 4;

Perl:

my $vzoiuwriozufsd = 0x04;

The point here is that there are many varied ways in which the same variable can be referenced.  I am of the opinion that much along the lines of Guido van Rossum of Python (and to a lesser extent ABC) fame, that there really should be one and only one obvious way to do it.  This isn’t to say that I think everyone should code in the same language, and speak the same tongue, etc.  What it does mean though, is that to be understood by others (and sometimes by ourselves), we need consistency, and unless we have a set of strict guidelines set out for us as software engineers, developers, etc., we might as well code in our own made up dialects.  

I am of the opinion that a proper interpreter, compiler, virtual machine, etc., should be more than capable of quickly turning long variable, class, function and method names into concise tokens with small internal footprints.  So much to the point that there is no excuse for not being verbose.  At one point in time, every single byte of allocated memory for names of the aforementioned items was a crucial issue which required extreme concise naming conventions to be followed.  Those times are gone in this day and age, allowing us to be clearer and more expressive.  

I can see using single letter counter variable names, but never could I imagine naming a class, method or function in such a sparse manner.  I like to think that clean code reads somewhat like a choose your own adventure book, were it to have a greater variety of options available.  Functional or Object Oriented is immaterial here, as cleanly written code isn’t tied to a specific construct or paradigm.  I think most of the following rules are applicable to pretty much every language out there.  Emphasis below pertains to items that I feel are not language specific guidelines.

As can be seen, most of the above are applicable to languages other than Python.  I find myself at my current place of employment having to deal with the problems for which this list addresses.  Much of what I’m doing is updating a legacy code base that is literally plagued with dozens of individual programs and modules that are blatant attacks on decent code.  They (collectively) single-handedly break most of the above guidelines.  

First off it is almost entirely written in perl, which instantly shoots down the Readability counts factor (and no, it wasn’t done with the strict pragma, and yes it uses a bunch of requires and plenty of global variables).  

Secondly, errors don’t pass silently because there is no built-in exception handling in perl.  Evals of code blocks does not equate to a proper exception system, nor does an add-in module.  Exceptions are something which need to be a core part of the design of the language, and perl falls far short of the bottom of the heap on this issue alone.  

Thirdly, when one is expected to maintain code in an environment wherein the expectation is to follow the existing coding schema as it were, with global variables, no exception handling, etc., it truly becomes a daunting task because one must force his/herself to think ‘wrong’.  The logical and/or proper solution that is naturally though of as a solution would only lead to reprimand, simply because trying to think in such a manner will produce mistakes, primarily because trained seasoned professionals don’t think in the same manner as the less experienced coder(s) responsible for the legacy code int eh first place.

Finally, (I’ll leave it to three to be nice to those few perl hackers who’ve read this far), after ten plus years of coding in perl, I’ve come to learn that the TIMTOWDI (There Is More Than One Way to Do It) mantra of perl is one of the biggest problems that arise from the language.  It is this careless and dare I say reckless mindset which has led to so many atrocities in the professional coding world.  

My point is simple enough to follow.  Write readable code, as it is a defining factor as to how far you’ve matured in the field of software development.  It doesn’t necessarily mean you are even that good at what you do, but what it does do is show how you understand a rudimentary problem that so many others have failed to realise.  Readability Counts, and without it, we are truly lost.  

It’s been a while... I’ve been busy coding away. Here’s an update.

    I’ve been rather busy recently now that our beta version of software where I am employed has made its way to production.  Since that has transpired, all of our Trac entries can be attacked in a more systematic manner.  Here’s a little rundown of what I’ve been doing.

    Handling my son’s integration into his newest school year endeavour, as well as my wife’s into hers.  She just completed her masters degree and is starting her second year (first full year) as a teacher of Biology..  to kids born the year that she and I graduated (together) from high school.

    Creating the backlog of CodeDevl.com podcasts, and editing.  I never realised exactly how much time it takes to edit a podcast recording.  For every five minutes spoken, there are ten minutes spent editing and cleaning up.

    Learning and implementing GIT version control/repository software at our place of employ, as well as my local network as a replacement for Subversion (SVN).  

    Wrote a python (base classes pure) application which handles all migration of beta software to both the GIT repository paths as well as handling moves to production (including automated changes to certain header includes).  I’m rather happy with this application as it has saved many issue from transpiring.  Due to the haphazard manner in which some of the code base is arranged (particularly the beta vs. live paths), problems can and have occurred, hence my reasons for taking the initiative to create said program.  

    Additional work with re-learning Java, and keeping current with other technologies (Python 3K/3000/3.0), Javascript, Ruby, Smalltalk concepts and to a lesser degree Lisp (not including additional emacs functionalities).

    I do promise that I will be continuing to update both this written journal as well as the podcast site, and just wanted to let those reading that I have not dropped off the face of the earth, just immersed myself back into the changing flow at my workplace. 

Python's 'Pickle' Module

Recent changes to the SimulaeObject class have proven to be a big leap forwards in regards to flexible object manipulation for both testing and live environments.  The issue at hand was in regards to how to load and/or save object 'types' as it were.  Initial thoughts were to go with a simple configuration file which contained each and every object a la something akin to the httpd.conf file from Apache 1.3 (& 2.x?) for handling virtual hosts.  Then it dawned on me that this was a mistake which I personally made once before, and was almost about to fall victim to once more.  

The solution was sitting right in front of me all along, the standard library's 'pickle' module (or alternatively the 'cpickle' variation for speed's sake).  Due to the issues presented in a virtual world simulation the topic of object blueprints and simulation population ease come to mind rather quickly.  The best environment for manipulating these new objects will ultimately be via the methods provided by both the the SimulaE package (SimulaE.loadObject(filename_to_load, [optional_load_path]) as well as at the code level in the parent class in SimulaE.SimulaeObject.saveObject(filename_to_save_as, [optional_save_path]).

By going this route it has been realised that a simple loop and/or script-like routine could be used to create all the generic object templates needed for design platforms, and for customisation all one need do is use the loadObject routine, make the necessary alterations and re-save said item as a more concrete, concise and specialised object, named appropriately of course.  

Let us take an example of a room type object (more simply put, a larger container object).  We'll use simple constructor information here for the sake of staying on focus.  We're going to make a 4 metre x 5.5 metre dining room, with a 2.75 metre ceiling, simple called "Dining Room".  It will be empty sans an already pickled to storage butler object we've created for the sake of this example (whose filename is simply "butler_jeeves"), though future discussions on SimulaE's container and stack loading methods are forthcoming.

import SimulaE

generic_dr = SimulaE.SimulaeObject(name='Dining Room', width=4.0, length=5.5, height=2.75)

generic_butler = SimulaE.loadObject('butler_jeeves')

generic_dr.addToContents(generic_butler)

generic_dr.saveObject('diningroom4x5.5x2.75wButler')

We now have a pickled version of this generic dining room of the aforementioned dimensions, sporting its own copy of Jeeves the butler, saved to a physical file on whatever storage device we're set to utilise.  We can overwrite said object by simply saving the item with the filename as it exists on whatever storage device is in use.  There is also the flexibility of specifying alternative file save and load paths with allow for multiple parallel simulations and/or individuals to work in safe separate but equal spaces, very much along the lines of a Unix mentality.  Another advantage by working in this manner is being able to create a path full of simulation objects and copy said path en masse for alternative and/or backup purposes.

Either way you slice the pickle (module), it proves to be quite (ful)filling, making one feel quite (programmatically) satisfied.