New Projects & Version Control Systems

It has been over a month and a half since my last entry and I apologise for the delay. I have been working tirelessly to finish up my primary design, development, implementation and maintenance functions on what has been my primary project for the past thirteen months. I've been doing this so that I may jump head first into my next project, which while not being a 'huge' undertaking is big enough. By big enough I mean that doing well at the project will land me the role of redesigning a major site from the ground up into the Django framework.

For the past 13 months I've been working heavily in the world of Django and find myself absolutely enamoured with it. I have designed a considerable amount of applications for Django, though none of them are released to the world because they are the product of a contract. My field of expertise was never in that of online applications and/or frameworks so it is stil a new world for me (if we exclude the nine years running a twelve line bulletin board system on a heavily modified Amiga).

I've always been happiest working in middleware and engineering backend systems, though I have to say that I'm finding a certain level of pleasure from the results of my front-end work. When i wrote backend systems, the only others who could appreciate said system(s) other than myself, were developers privy to the project. On frontend systems, there are actual users who I would like to think 'benefit' from my designs and implementations. It is nice to know from actual users that your work is appreciated. Call it ego stroking, or call it what you will but I find that it does inspire one to keep pressing on regardless. I happy to be doing these projects and as soon as it is launched, I'll be happy to post about it here.

This brings me onto the second purpose of this entry, my new favourite version control system. In the beginning of my experience using version control systems back in the 1990's, I simply used CVS like the rest of the world (sans those lucky enough to use Perforce). As time and technologies advanced to newer and better systems, I followed along and moved up to Subversion. Subversion was a well done upgrade for CVS users and easy enough for the uninitiated to learn. I used this most recently when I did work for Curlington Boat Factory (name changed to project myself) writing their point-of-sale returns authorisation/queue system in Python for four hundred some odd cash registers. This worked our quite well despite the fact that the software house for whom my partner (at the time) and I were contracting weren't able to provide the subversion server on their machines until we were ⁷/₈^ths of the way through said project. We ended up (very early on mind you) taking matters into our own hands and setting up on my partner's server at his apartment. This wasn't looked upon highly by said software house, but any CVS on any server regardless of not being in control of the software house was far better than no source control at all.

Next we come to my brief stint working for a OCD kid who started his business at the right time with mom and dad's money, but due to poor decisions on his behalf watched it dwindle in popularity, further plagued by poor management and attempts to treat a small business as if it were a conglomerate thereby sealing its fate with a tender kiss on its cheek. I thankfully left the company before the owner went psycho and let go of all the talent in a horribly vicious manner. Before all of that ugliness transpired, I went forward with trying the newest and brightest trend in distributed version control. I am speaking of the one and only 'Git', championed by a certain arrogant (albeit brilliant) Finnish kernel programmer. There was an immediate joy in using Git and I have to say that most striking feature is the blazing speed when dealing with a large quantity of files. I established all of the existing software base for said company (over ten years worth) into its first repository, utilising Git. Yes, for over ten years, there was no VCS in place. Git really did shine in this role, though I am pretty sure that due to myself and the system administrator not longer being there, cob webs must be forming in the places where Git once speedily did its work.

Let us fast forward to my newest discovery (if you can actually call it one as such). On this newest project which I just started today full time, I have been doing administrative setup work for the past few weeks on and off in small gaps of time as they availed themselves to me. The normal deal of establishing a dedicated server on a FreeBSD host (as Linux just will not cut it for me even though I've been using it since kernel v0.99d), along with all the necessary bits. Once the development languages, utilities, database and web servers were in place the issue of VCS came into play. For this time around I decided that Git, while great for many projects doesn't seem as pythonic a package as I'd like. Using Python long enough really makes one desire beauty, power, simplicity and consistency in all of their tools. This brings us to the most pythonic vcs of them all, Mercurial (known simply by its periodic table element, Hg). Mercurial, like Git is also a distributed version (revision) control system. Easy to setup, easy to use and highly recommended. I'm not going to wax poetic about the differences between these systems as they all serve a purpose. What I will do is state very clearly that Mercurial feels the most natural, works rather well and is quick at what it does. Your mileage may vary.

Until next time...

Django Projects Galore and Vile Ethics of other Coding Firms..

It has happened yet again. I've been brought on for another Django project. This time it is about taking an existing site for yet another magazine publisher and converting their Wordpress driven site into a real full-blown site complete with blogs, forums, user profiles, dynamic main page content, complete customisation from within the framework and included applications and ultimately a site in which a developer is not needed for day to day changes.

As it exists, this publication is an off shoot of their primary magazine. A magazine whose website was written and managed by an outside from that I believe coded the entire site so as to require additional invoicing and servicing for all but the most minute changes. This is a disgusting business model and one with which I've had the misfortune of experiencing whilst working as a sub-contractor to a sub-contractor for Burlington Coat Factory.

The H-1B Visa Project Manager who wasn't too fond of me because I don't believe that coding in dress attire and/or a tie makes someone a better worker (to the contrary, i will NOT wear dress attire for day to day work as it is a pointless expression of old brick-and-mortar mindsets). He also was the first time that I was reprimanded for having an eloquent solution that adapted automatically to the growth needs of the end-clients database/system. I wrote the software to handle dynamically gathering and sequencing additional 'like' fields as they were added to Burlington's transaction schema. The way I designed and wrote the software, the MOMENT the schema changed, my software contributions would immediately include relevant changes, without a restart of any of the daemons I engineered. I was told that the reason why i shouldn't have done this is because the sub-contractor for which i was writing this code could then go back and charge an exorbitant amount of money each time minor changes were made. This disgusts me, and I find it ethically wrong.

I am an engineer and work independently by choice as I can first and foremost hold myself and solutions I produce, to higher standards; delivering what my clients want and need, not solely based upon what they say they want and most definitely not building them into a corner for profit over common decency and professional standards.

Once the project has been completed, I will be quite happy to share the url(s) with CodeDEVL readers.

Django 1.0 has been Released, Film at 23:00.

I know that it has been a while since I have last posted, but I'm not one to post meaningless empty ramblings on a regular basis. I just want people to know that I'm still alive and will still be posting when I have something worthy of your time.

I will mention in brief that Django v1.0 has been released and we are all thankful to BDFL's Adrian Holovaty and Jacob Kaplan-Moss for their amazing efforts in making this release come to fruition. I would also like to add that the biggest 'gotcha' between .96.x and 1.0 is in the standardisation of certain db api arguments. maxlength has become (more aptly) max_length. There is also now a DecimalField type which can be used rather than the less accurate (for monentary purposes) FloatField. Read all about it at the Django Website.

Finally, for those interested, I have also started another non-computer related blog over at Bucks Bicycling, a site related to vehicular, commuter, recreational and sport cycling. I'm currently working on a fixed gear conversion of an old cheapie AMF Roadmaster Scorcher as well as a 1985 Schwinn Sprint road bike.

I will be posting shortly pending upon the official status of the newest Django 1.0 based project on which I've been secretly working.

Till then..

Quick Django Tip: Dynamic Application Object Retrieval

In my recent django adventures I needed to introduce site-wide search functionality and in the process of doing so, encountered a small roadblock towards doing so.  Apparently due to the nature of Django's API for db interaction (as of the last stable release version), there is a limitation as to the use of python variables in API calls.  I found this to be a hinderance, but only for so long.  

The follow code snippet was something I whipped together which by utilising Python's 'eval' built-in, overcame the aforementioned limitation regarding the API's ability to interpolate native Python (e.g. non-django explicit) varaibles.

Things to know to understand the following example:  

 search_input is a list of cleaned and pre-processed user-driven terms, split into separate expressions, (e.g. ["dynamic langauge", "agile", "programming", "paradigm"]). 

 search_schema is a dictionary in which the key is the django model/class through whose objects we are attempting to search, and the value is a list of specific model attributes to attempt said search.  (e.g.   User_Profile : ['firstname' , 'lastname', 'address_1', 'bio_info', 'favourite_books'])

container_xref is a simple alias mapping for the actual django application names to our internal references inside this search code base.  Obviously this whole bit could be written without said setup, but for readability given the scope of the actual application involved, and the fact that I was not searching simply a few static fields in one django application, but several dozen fields through about two dozen separate applications, this container_xref dict was appropriate.   It is through this mapping dict which we place any matched object results (so as to not waste any additional space via unnecessary list initialisations.) for eventual results generation.

Note: the key "total_results" in the container_xref was a simple means of keeping track of overall search matches, rather than relying upon the Django templating engine (view) from doing work responsible from the processing (controller) perspective.  In retrospect, there are better ways this could have been handled, and in future point revisions, this will be addressed.

------

for search_string in search_input:

  for application in search_schema.keys():

    for attribute in search_schema[application]:

code_to_eval = "%s.objects.filter( %s__icontains='%s' ).order_by('-id')" %       

                     (str(application), str(attribute), str(search_string))

        try:

          eval_results = eval(code_to_eval)

          for eval_result in eval_results:

            if eval_result not in container_xref[application]:

              container_xref[application].append(eval_result)

              container_xref['total_results'] += 1

        except Exception:

    ### Case specific exception handler types, assignments and resultant actions 

          ### specific to each application in which the above is implemented, go here.

-----

As can be seen from the above, simple inline substitution proceeded by evaluation of said string results in post-compilation dynamic search functionalities within django, addressing simple problems one might run into with the existing API which will most likely be addressed in future versions.  

Your results may vary.

Eric

The New Django Powered Inked Magazine Website is Up!

This will be rather brief as it is more of an announcement than one of my more traditional journal entries. 

After a  short two or so months from start to finish, I have successfully setup my first Django powered website.  Inked Magazine has been relaunched effective January 2nd, 2008.  This replaces the Joomla powered site which existed prior to both myself and the current ownership of the magazine were involved in the project.  

Phase one has been completed, with extras.  The customer photo galleries, the cover photo application, the user profile system as well as the main magazine feature areas have been established and are active.  As of today, I will have available for all registered users the ability to host a blog on our site (using our software which I wrote in a few days), mind you it is still early in the feature process, though it without doubt serves most blog authors needs.  

I will be adding features as time progresses, but until mid-January 2008, I'm on a tight schedule to build the entire forum application so that the beginning of the "New and Improved" Inked Network can go live.  

I don't think one needs Ruby on Rails when you have Django.  Much of the same great functionality and without having to use Ruby all the while using Python.  [Update: After having revisited Ruby as a language on its own, sans Rails notoriety, I've found that my previous assertions regarding Rails specifically was unwarranted.  It simply took my experiences with Django and Python to make Ruby and Rails far clearer to me.]

I will return to my normal posting after the Forum application is up and running.  

Till then, keep on coding.

Django and Gantt Charts

It has now been almost a full week since I started the complete inkedmag.com site and infrastructure redesign in Python using the Django web publishing framework on FreeBSD, and I am happy to report that it is awesome. Mind you we're talking version .96 of the product, yet it truly is a dream with which to work.

I only recently started working with (and writing about) Cheetah, a wonderful python template engine, and had to very quickly learn yet another (Django's own template system). I must admit that Cheetah is easier to ready and learn quickly, but Django's system is considerably more agile in terms of conditionals and modifiers inside the template itself. There even happens to be a simple mechanism for cycling through a list continually changing on each iteration of the loop within which the cycle conditional resides. Simply put, it is wonderful for automatically changing the background colour of a row in a list.

For those unfamiliar with Django, it is simply one of the better web frameworks for content publishing on the web these days. While learning curve can be a little steep for some pieces of the framework, as a whole the speed at which once can produce working pages and applications is staggering. The ease and elegance of the system truly makes one enjoying creating new applications within the framework.

The first application I chose to migrate from native python into the framework was a simple store locator. The new version not only is considerably less lines of code, the database management was done for me at application creation/initialisation. I then simply exported the data from my existing application and imported it into the new table(s) Django created.

I could go on waxing poetic about every little bell and whistle, but I'd just be paraphrasing what many others have already pointed out online and otherwise. Don't think of it as Ruby on Rails because it isn't, though that isn't to be taken as an insult to Ruby. It is much more focused, cleaner and far simpler to setup and get running, including all of its own admin interfaces for the applications you create, as well as its own standalone development web server. Check it out, you won't be disappointed. This is going to save me a considerable amount of time.

Which brings me to my second point; Gantt charts. They are simply not something I find myself utilising on any regular basis, though I think that is going to change. I'm my own boss and have found that gantt charts produce the easiest visual way to show people the various pieces necessary for a project, when each portion can be expected to start and finish, all in parallel with the other projects for which I'm responsible (and/or coordinating).

I feel that the use of this tool more than others really gives a great method by which to see which projects will take the bulk of the time, and what projects overlap, etc. We have a system rewrite to produce and a whole server to replace, not to mention migrating certain custom software into the framework all before the new year. This is doable, but only because we've clearly set realistic (though tight nonetheless) goals and time frames. Consider using a gantt chart if you have more than one project or component of a project which needs to be done in a given time frame. Use one if you need to share with one or more people your schedule and need them to understand as quickly and clearly as possible that with which you are juggling or dealing. You find yourself quickly addicted to its usability.