Archive for the ‘Programming’ Category

The [System Administrator’s] Oath

December 6, 2015

Bob Martin (author of Clean Code, The Clean Coder, and evangelist of software craftsmanship) recently posted The Programmer’s Oath. Let’s say we have jobs as System Engineer’s or System Administrator’s. We may write code to automate our work, to manage, maintain, and operate our systems and environments, but we don’t necessarily work as Programmer’s do writing applications for business functions. Would the Programmer’s Oath apply? Are its promises relevant to systems engineering/administration work? Perhaps it does. Here’s a version with some of the words changed as The System Administrator’s Oath.

In order to defend and preserve the honor of the profession of systems engineers and administrators,
I Promise that, to the best of my ability and judgment:

1. I will not create harmful infrastructure or systems.

2. The infrastructure and systems that I create will always be my best work. I will not knowingly make changes to infrastructure or systems that are defective either in behavior or structure.

3. I will produce, with each change, a quick, sure, and repeatable proof that every element of the infrastructure and systems work as it should.

4. I will make frequent, small, changes so that I do not impede the progress of others.

5. I will fearlessly and relentlessly improve the infrastructure and systems at every opportunity. I will never make the infrastructure worse.

6. I will do all that I can to keep the productivity of myself, and others, as high as possible. I will do nothing that decreases that productivity.

7. I will continuously ensure that others can cover for me, and that I can cover for them.

8. I will produce estimates that are honest both in magnitude and precision. I will not make promises without certainty.

9. I will never stop learning and improving my craft.

Regardless of how you might feel about taking oaths, Uncle Bob’s promises are something to contemplate as we constantly work to become better IT Professionals.


Reimagining the Implementation of TeX (and the Luxury of Modern Programming Methods)

January 24, 2015

Glenn Vanderburg, Engineering Director for Architecture at LivingSocial (@glv, has a personal project underway to implement TeX’s algorithms in the Clojure programming language. His work was recently described in an article published on InfoQ. Glenn also presented his work at the Clojure/conj conference last year in 2014, his excellent talk is posted on YouTube.

The InfoQ article and YouTube presentation provide a high-level summary of the state of technology and software development capabilities available in the early 1980’s which influenced (really restricted and confined) the creativity and results of a programmer at the time. By implementing TeX in Clojure, a functional programming language with capabilities for multi-threaded processing, Glenn applies modern methods, and makes several now-and-then observations along the way, to the implementation of a large-scale programming masterwork created by Professor Knuth more than 30 years ago.

“The ways we can program today are luxuries made possible by decades of small advances.”

Glenn also describes the potential benefits for reuse that would be available with a modern implementation giving several examples of how Web publishing and eBook readers might take advantage of using TeX’s excellent typesetting algorithms creating beautiful page layouts and more natural reading experiences.


Agile is not fast, it’s frequent.

July 23, 2010

The principles (and the intention) of Agile Development are often confused with the word agile which people interpret to mean fast (in the sense of being quick and doing things other than in a thorough way). Of course that is an incorrect interpretation. Agile methods are not intended to be fast, if anything Agile teaches us to be iterative, repetitive, and to break down larger problems into smaller pieces that can be more easily understood, completed with higher quality, and delivered on an expected schedule. From that perspective, Agile teaches us to be Frequent in the way we approach our work.

So Frequent may in fact be another principle of Agile Development. As a principle it doesn’t only have to apply to software development work. We can apply Frequent methods to other disciplines too.

Playing iTunes .m4a Files on a Squeezebox2

May 30, 2010

I encountered a problem playing iTunes .m4a files on Squeezebox2 devices. It was interesting that from the same Squeezebox Server the files played fine on a Squeezebox3, but when attempting to play on a Squeezebox2 an error appeared on the Squeezebox Server’s Web interface, “Problem: Can’t open file for: <file>.m4a”. Other files, such as .mp3 filetypes, played fine.

I validated that the .m4a file existed in the music directory and there were no permission issues. To troubleshoot the problem I activated debug logging for os.files, os.paths, player.source, player.streaming, and scan.scanner and then restarted the Squeezebox Server; the logging messages were recorded in the server.log file. The problem was straightforward:

Slim::Player::TranscodingHelper::checkBin (242)   Found command: [faad] -q -w -f 1 $FILE$ | [lame] –silent -q $QUALITY$ $BITRATE$ – –
Slim::Utils::Misc::findbin (100) Looking for executable: [lame]
Slim::Utils::Misc::findbin (111) Checking for lame in /usr/share/squeezeboxserver/Bin/i386-linux/lame
Slim::Utils::Misc::findbin (111) Checking for lame in /usr/share/squeezeboxserver/Bin/lame
Slim::Utils::Misc::findbin (111) Checking for lame in /usr/kerberos/bin/lame
Slim::Utils::Misc::findbin (111) Checking for lame in /usr/local/bin/lame
Slim::Utils::Misc::findbin (111) Checking for lame in /bin/lame
Slim::Utils::Misc::findbin (111) Checking for lame in /usr/bin/lame
Slim::Utils::Misc::findbin (111) Checking for lame in /usr/bin/lame
Slim::Utils::Misc::findbin (111) Checking for lame in /usr/local/bin/lame
Slim::Utils::Misc::findbin (111) Checking for lame in /usr/libexec/lame
Slim::Utils::Misc::findbin (111) Checking for lame in /usr/sbin/lame
Slim::Utils::Misc::findbin (130) Didn’t find binary for lame
Slim::Player::TranscodingHelper::checkBin (265)    couldn’t find binary for: lame
Slim::Player::TranscodingHelper::getConvertCommand2 (421) Error: Didn’t find any command matches for type: mp4

So it seems that lame is needed to stream music data from an m4a file to a Squeezebox2 device. (But lame is not used when streaming music data to a Squeezebox3. That’s interesting too.)

I needed lame for Redhat Linux, it’s an open source project maintained on SourceForge here. After downloading the source, run a ./configure, make, and make install. And that fixed the problem.

Move Messages to Folders with Outlook VBA

March 27, 2010

Let’s say your email management approach is to save important messages to a folder other than your Outlook Inbox. You might save messages from outside vendors to a folder named “Vendor Documents” or messages related to corporate policy to a “Policies” folder. Here’s an Outlook VBA macro that helps to file those messages from your Inbox in a single click:

Sub MoveToFolder(folderName)

 mailboxNameString = "Mailbox - Firstname Lastname"

 Dim olApp As New Outlook.Application
 Dim olNameSpace As Outlook.NameSpace
 Dim olCurrExplorer As Outlook.Explorer
 Dim olCurrSelection As Outlook.Selection

 Dim olDestFolder As Outlook.MAPIFolder
 Dim olCurrMailItem As MailItem
 Dim m As Integer

 Set olNameSpace = olApp.GetNamespace("MAPI")
 Set olCurrExplorer = olApp.ActiveExplorer
 Set olCurrSelection = olCurrExplorer.Selection

 Set olDestFolder = olNameSpace.Folders(mailboxNameString).Folders(folderName)

 For m = 1 To  olCurrSelection.Count
    Set olCurrMailItem = olCurrSelection.Item(m)
    Debug.Print "[" & Date & " " & Time & "] moving #" & m & _
                ": folder = " & folderName & _
                "; subject = " & olCurrMailItem.Subject & "..."
    olCurrMailItem.Move olDestFolder
 Next m

End Sub

Some customizations are necessary to make this all work for your email system. First, copy-and-paste the program to the Visual Basic Editor in Outlook (which you can open with Alt-F11). You’ll need to create a new VB Module (use the Insert->Module menu choice) to hold the macro if you don’t already have custom Outlook macros defined.

Second, the text highlighted in the blue bold font needs to be edited for the name of your Exchange mailbox; you can usually find the name of your mailbox on the left side of the Outlook window at the top, it starts with “Mailbox” like in the example above.

Third, you’ll need to add other VBA macros which call the MoveToFolder() macro with the correct folder name passed as  a parameter. For example, we might create two additional macros one each for the Vendor Documents and the Policies folders; the macro for Vendor Documents might be called MTV() and the Policies folder macro MTP():

Sub MTV()
 MoveToFolder ("Vendor Documents")
End Sub

Sub MTP()
 MoveToFolder ("Policies")
End Sub

(Note, the destination folders should already exist in your Outlook folder list.)

The last step is to assign the macros to Outlook toolbar buttons so you can move the selected messages in your Inbox to the appropriate folders with a click. If you’re managing multiple destination folders, and you created multiple macros (one for each folder), you can also create a new toolbar to organize all your MoveToFolder() macros. Here’s a Microsoft Knowledge Base article that describes how to assign Outlook VBA macros to toolbar buttons: How to Assign a Macro to a Toolbar Button.

The MoveToFolder() macro will work on a single selected message, or a group of messages selected in a sequential range (with a Shift-Click), or a non-sequential range (with Ctrl-Clicks). Also, an activity message will be written to the VBA Editor’s Immediate Window (which is opened with Ctrl-G) so you can go-back and view MoveToFolder() activity. A total win for email convenience, give it a try.

A Creative Look at Pair Programming

March 15, 2009

I follow John Mayer the singer and songwriter on Twitter. Besides being a fan of his music, on Twitter Mayer (@johncmayer) is creative, funny, and he provides some transparency into his musical projects which are interesting. Mayer also writes the Battle Studies Mid-Action Report blog where he uploads videos and photos that are referenced in his Twitter postings.

A few days ago Mayer made a statement on Twitter, in @reply to a question from a follower, which I found intriguing. He replied:

I still write in front of a mirror.

You can view the full post, which also includes the original question here.

I was struck by the statement since, for me, it is completely counterintuitive to what you would expect the creative writing process to be. Writing is a very intellectually internal process. You typically hear of writers having a quiet workplace in the woods, or in an attic, separated from people and activity in order to create an environment to be alone with yourself and your thoughts. The process of creating words from thoughts, emotions, or feelings (whatever they may be) is usually considered to be a very individual process requiring deep introspection and contemplation achieved by developing an ability to see yourself from within.

So, I would think that writing in front of a mirror is opposite to being introspective. To me it seems as close as you can get to introducing a second person into the process without having someone else in the room. The mirror then is a tool to externalize the creative process, to forcefully move creativity outside of yourself, or possibly an attempt to see yourself as someone else in order to develop a collaborative process when there is only one person involved. That scenario is so contradictory to the way that I think about creative writing, it is so out-of-the-box and unexpected, that I frankly find it brilliant.

There is also a creative process that occurs in software development and programming. I’ve always related to the description that Fred Brooks gives of the software development process in his book The Mythical Man-Month. Brooks, referring to the book The Mind of the Maker by Dorothy Sayers who identifies creativity with three stages, the idea, the implementation, and the interaction, says:

A book, then, or a computer, or a program comes into existence first as an ideal construct, built outside time and space, but complete in the mind of the author. It is realized in time and space, by pen and ink, and paper, or by wire, silicon, and ferrite. The creation is complete when someone reads the book, uses the computer, or runs the program, thereby interacting with the mind of the maker.

Brooks then goes on to describe computer programming using the short phrase:

The programmer builds from pure thought-stuff.

The statement describes a computer program being built (or developed, created, written) from the thoughts (or ideas, concepts) of a single individual.

Several years ago, and together with a close colleague, we implemented the eXtreme Programming (XP) software development methodology at The methodology was popularized by its author, Kent Beck, in the book eXtreme Programming Explained: Embrace Change. The premise behind XP is to take all the things that people generally consider to be good for creating quality software and to do those things to the extreme. For example, everyone generally considers testing to be directly related to quality; the more you can test a program the fewer errors it is likely to have. So in XP testing is done to the extreme by writing the tests for a program before the program is actually written. The “test first” approach, or sometimes called “test driven development,” can be viewed as being radical by someone who is used to working in a more traditional software development methodology where tests are written after the program is created.

Another principle of XP usually considered to be radical is “pair programming.” When pair programming two people sit together to write a program. In XP Explained Beck describes pair programming as follows:

Pair programming is a dialog between two people trying to simultaneously program (and analyze and design and test) and understand together how to program better. It is a conversation at many levels, assisted by and focused on a computer.

When I was first learning about XP I clearly remember having a deep and surprising reaction to the concept of pair programming: How can something as personal as writing software, transcribing my thoughts, my concepts, and my ideas, my pure thought-stuff, be shared with someone else simultaneously? How is it possible that writing a program, which can be an intimately creative process for an individual, can be accomplished by working with someone else? For programmers who haven’t done it before, the concept of pair programming can be radical, out-of-the-box, and counterintuitive, which was exactly my initial reaction to Mayer’s statement about writing in front of a mirror.

Pair programming introduces a second party into the creative process. It’s an activity that forces you to communicate, to share and verbalize your thoughts and ideas, and to collaborate in a completely different way. When pair programming, someone else, a second person, is there with you while you’re going through the activity of creative expression. Both people experience intellectual and personal benefits as a result of that collaboration and there are significant benefits to the overall quality of the product, the computer program or the software application, they produce together.

If writing in front of a mirror, as Mayer does, has a similar effect to introducing a second person into the creative process, then the mirror becomes the tool that enables Mayer to “pair program” with himself. Mayer is generally considered to be a gifted artist and it would seem that he is leveraging the pairing concept to make himself, and his art (his music), better.

Knowing that Mayer uses a mirror in the writing process makes the pair programming concept appear less radical. Perhaps when we work alone we limit the expression of our creativity to our own individual experiences. Perhaps collaboration with another person, the second person effect, is required to really broaden the possibilities of our own individual creativity. And perhaps when we pair program to create computer software the second person is really the mirror.

Lower Your OO Expectations for the Windows PowerShell

December 28, 2008

For programmers who are used to leveraging object oriented techniques, even within modern scripting languages, don’t expect too much from the Windows PowerShell. Here’s a quote from the PowerShell Online Help file:


Definition: Declares the name of a class, as well as a definition of the variables, properties, and methods that comprise the class.

Although it is possible to create a class in Windows PowerShell, its not a very straightforward process and definitely goes beyond the scope of this introductory manual. So, for the time being, forget we even mentioned it.

Yep, that’s a verbatim quote. PowerShell is still a verision 1.0 product so hopefully the situation will improve in future versions of the software.

If you’re looking for more documentation for the Windows PowerShell, here are some references.

The Windows PowerShell 1.0 Documenation Pack includes a getting started guide, a user guide/primer, a quad-fold reference card, and release notes for version 1.0.

Microsoft has a .chm Help file for PowerShell which is a convenient (and searchable) reference to supplement the PowerShell online Get-Help CommandLet. They refer to the .chm Help file as the “Graphical Help File” which you can download by clicking here.

O’Reilly publishes a pocket reference which is a very good source of information for the Windows PowerShell if you are an experienced programmer. They also publish the Windows PowerShell Cookbook for the language which is in the standard O’Reilly Cookbook format that is available for other programming languages.

The Windows PowerShell Technology Center has many other helpful resources if you are getting involved in PowerShell Programming.

The next version of PowerShell, version 2.0, is available in a Community Technology Preview (something like a beta version) by clicking here. And hopefully future versions of PowerShell will provide better support for object oriented freatures of the language.

A Text Report for Outlook Calendars

November 15, 2008

My prior firm created a great little application for Outlook and Exchange calendars, here’s how it worked. People in the Systems Group entered their out-of-office time (vacation time, sick days, work from home days, etc) into an Exchange Public Folder calendar called “Vacations.” The application, which ran overnight sort of in a batch mode, would read the calendar and format a text report suitable for reading in Outlook or on a Blackberry device. The report, as I remember it, looked something like this:

This is formatCalendarReport.wsf running on 11/9/2008 11:06:30 AM...

     Calendar: Personal Folders/Calendar...
Report period: 11/10/2008 through 11/13/2008 (4 days)...

Monday, 11/10/2008:
         Subject: Sal - Vacation
            From: 11/10/2008   To: 11/12/2008

Tuesday, 11/11/2008:
         Subject: Sal - Vacation
            From: 11/10/2008   To: 11/12/2008
         Subject: Jim - Working from home
            Body: Please contact me at 212.555.1212 as needed.
            From: 11/11/2008   To: 11/11/2008

Wednesday, 11/12/2008:
         Subject: Sal - Vacation
            From: 11/10/2008   To: 11/12/2008
         Subject: Rob - Cisco seminar
            Body: I will be out of the office in the afternoon.
            From: 11/12/2008 12:00:00 PM   To: 11/12/2008 5:00:00 PM

Thursday, 11/13/2008:
         No calendar events for today.

This is formatCalendarReport.wsf ending on 11/9/2008 11:06:31 AM...

Now keep in mind that this calendar report is not something that Outlook or Exchange provides as a standard feature, so a developer in the Systems Programming Group wrote a custom Windows application to format calendar entries to be delivered by email. Shared Public Folder Calendars were widely used by different groups and departments in the firm and people generally liked the daily email format (rather than having to manually view several calendars every day in Outlook).

When I started a job at a new company I wasn’t really surprised to see Exchange Public Folder calendars used for similar shared event scheduling to coordinate vacation time, software releases, and on-call support. I quickly missed the convenience of receiving a daily email so I set out to recreate the report in a script.

A Script to Format Outlook Calendars

formatCalendarReport.wsf is a Windows Scripting Host (WSH) application that uses the VBScript programming language to access Exchange Public Folders and print calendar events in a format similar to the listing above. The script can be run from the Windows CMD prompt using the WSH cscript command. If you’re using a current Service Pack version of Windows XP you should find that WSH and the cscript command are already installed on your system, so download the script to your computer and try to produce usage instructions by running it with the /? parameter as follows:

C:\>cscript /nologo formatCalendarReport.wsf /?

Prints a report of events in an Outlook calendar for a period of time.

Usage: formatCalendarReport.wsf /calendar:value [/numdays:value]
          [/emptyevents[+|-]] [/emptybody[+|-]]

calendar    : pathname of the Outlook Calendar
numdays     : number of days to process in the report
emptyevents : include dates without events, default emptyevents+
emptybody   : include empty body fields, default emptybody+


   formatCalendarReport.wsf /calendar:"Personal Folders/Calendar"
      /numdays:14 /emptyevents+ /emptybody+

The WSH script and its output include a few enhancements compared to the version that I remember. First, when the script runs it produces greeting and closing lines that show the start and end times of the command:

This is formatCalendarReport.wsf running on 11/9/2008 11:06:30 AM...
This is formatCalendarReport.wsf ending on 11/9/2008 11:06:31 AM...

And the runtime parameters are printed at the top of the formatted report:

      Calendar: Personal Folders/Calendar...
 Report period: 11/11/2008 through 11/13/2008 (4 days)...

If there are no scheduled events for a given date, the report will include a message like this:

Thursday, 11/13/2008:
         No calendar events for today.

And if multiple events are scheduled, the subject, body, the start, and the end times of each event are printed as follows:

Tuesday, 11/11/2008:
         Subject: Sal - Vacation
            From: 11/10/2008   To: 11/12/2008
         Subject: Jim - Working from home
            Body: Please contact me at 212.555.1212 as needed.
            From: 11/11/2008   To: 11/11/2008

Notice that calendar appointments which are entered as “All day events” show their start and end date ranges without times (as in the two events immediately above). However, calendar appointments which have specific start and end times (for example, events that are not defined as “All day events”) show the time of day details as in this example:

 Subject: Rob - Cisco seminar
    Body: I will be out of the office in the afternoon.
    From: 11/12/2008 12:00:00 PM   To: 11/12/2008 5:00:00 PM

An Example Command

Here’s how to run it on your system. The script provides command line options to customize the format of the report. Below is a runtime version of the command that uses all the currently supported options:

cscript /nologo formatCalendarReport.wsf
        /calendar:"Personal Folders\Calendar" /numdays:7
        /emptyevents- /emptybody-

cscript is a Windows command that invokes WSH to run a .wsf script, and the /nologo option tells WSH to omit a banner message (which reduces some clutter from the output). formatCalendarReport.wsf is the name of the script.

The Microsoft Exchange Calendar to read is specified by the /calendar option. The calendar is a double-quoted pathname within the Outlook folder list. For example:

/calendar:"Personal Folders\Calendar"

selects a calendar under Outlook’s Personal Folders list. Alternatively, a calendar called Vacations in the Exchange Public Folders list might be accessed like this:

/calendar:"Public Folders\All Public Folders\Vacations"

The formatCalendarReport.wsf script starts from the current date and (by default) creates a report for the next 14 days. In the example above, the /numdays option is used to create a shorter report for seven days (starting from today’s date).

If a day in the calendar doesn’t have any scheduled events, a line item is added to the report that reads “No calendar events for today.” To exclude dates without events, use the /emptyevents- parameter. (By default /emptyevents+ is implied.)

Similarly, a “Body:” line item is included by default, so add /emptybody- if you would like to exclude body fields from the report that don’t contain any data.

System Requirements for the Script

The script requires Windows Scripting Host. If you are running a modern version of Windows (Windows XP, Vista, or Windows Server 2003/2008) you’ll find WSH and the cscript command included in your system. If you are running a different or older version of Windows, or if your system doesn’t have the cscript command, check the Microsoft Scripting FAQ to determine how to install WSH on your system.

The script also requires Outlook Redemption. Outlook Redemption facilitates access to Outlook and Exchange data via scripting languages without the Windows security layer continually confirming access through a pop-up window. Outlook Redemption can be downloaded from the following Web site: Follow the instructions in the readme.txt file to install Redemption on your system to run the formatCalendarReport.wsf script.

A connection to Outlook (for a mailbox in Personal Folders) or an Exchange Server (for a mailbox stored in Exchange) is required. The script connects using the privileges of current user. If you setup the script to run as a Windows Scheduled Task remember to configure logon information (both username and password) for the proper connection to your mailbox.

Additional Comments and Follow-up

A .txt filetype has been added to the name of the script to avoid problems with virus scanners or firewalls when you download the file. Be sure to remove the .txt extension after the download completes so you won’t have a problem running it on your computer.

formatCalendarReport.wsf will format a calendar into a text listing but it doesn’t email the report. A follow-up post will demonstrate how a separate Windows tool can be used to email the completed report to a list of addresses.

Please note that the script was only tested on a Windows XP system running Service Pack 3.

In the meantime, if you download the script and try it on your system, and please post a comment so we can hear something about your experiences running it.