There are, however, many classes of applications for which text files are the preferred means of storing data.  One of the main reasons is that when data is stored in a relational database, editing it is not a trivial task.  A well-normalized database is not easily updated via an SQL command line.  More often than not, a dedicated, graphical editor is needed to model the complex relationships.

Several years ago, when I wrote TaskForest, one of the initial design requirements was that it be easily configurable with just a shell prompt and one’s favorite text editor. Many of the servers I cared for for schools and non-profits were old boxes which I administered by logging into them via ssh. So when it came to designing job definitions and dependencies, I chose a text file representation. The benefits of text files over a graphical user interface for this include:

Easy Remote Access

All you need is the ability to get to a command line and a text editor on the machine that holds the data files. With the such low client access requirements, virtually any old machine that has internet access and an ssh client can be used to administer the system. I have often worked on my own taskforestd server from a local internet cafe using a Putty.exe downloaded minutes earlier.

Mobile Access

Text files also make work relatively easy using a mobile ssh client like Idokorro Mobile SSH. A dedicated mobile client would be ideal, but short of that, the text file approach assures low bandwidth usage and easy-to-make changes.

Flexibility

The simple, easily parseable format of text files allows us to build richer graphical clients later that would use a graphical interface to specify relationships between jobs.

Source Control

The text based format makes it easy to place the data files under source control. You can also easily diff different versions of the same data file.

Grep

When you have dozens of job group files and hundreds of jobs, you may need to answer questions like: “Are we still running Job J?” This can easily be answered by grepping the files for job J.

Low footprint

When you’re designing an open-source application you may want to minimize the complexity of the system by not forcing dependencies on major subsystems like GUI libraries and Relational Databases.  Of course there is a point at which a such dependencies are inevitable – you have to periodically re-evaluate your decisions and determine whether decisions that were correct, say, a year ago are still correct today.

Choosing A Text Format

In the case of the TaskForest project, the most difficult task by far was choosing which text format to use.  I went through several iterations trying to find one that was simple to read and write, and yet rich enough to model the domain space completely.  What worked for me (and might work for you) was to ask myself how I would represent the data I’m trying to save given just a pencil and paper.  Drawing in a notebook gave me the flexibility to sketch and edit easily, and once I had a good representation, converting that to a text file was a simple task.

The emacs editor

In their book “The Pragmatic Programmer,” Andrew Hunt and David Thomas stress to “Use a single editor well. The editor should be an extension of your hand; make sure your editor is configurable, extensible, and programmable.”  I couldn’t agree with this more.  A good editor is a very flexible tool.  Once you become an expert at it, you can use it to accomplish tasks almost as quickly as you can think of them.  You need to process a file as a one-off?  You could write a perl or python script, or an awk/sed one-liner, but often you could also do it in an editor – with the benefit of ‘Undo.’

A good editor is configurable, so that you can use it the way that works best for you.  Most editors have more than one way to do things, and there is no right way, just the way that works for you and helps you reach your goal quickly and efficiently. A good editor is extensible, so that it can grow along with your needs.  You need syntax highlighting for the new programming language that was invented after your editor?  You should be able to go online and download an extension module – or write one yourself.  A good editor is programmable.  Your job requires you to document functions and change histories a certain way – your editor should allow you to change its behavior, so that you don’t have to repeat the same kinds of tasks over and over again.

One of the best ways to learn what you can do with editors is to watch others.  Do you have a coworker whose fingers fly across the keyboard while she bends text files to her will? It may not be just that she’s a fast typist – she’s probably really comfortable using her editor.  Search for these people, ask for advice and take notes.  Try different options out, pick an editor, and invest the time to learn it well. You’ll be doing your career a favor.

There is no real correct answer to the question “Which editor is best?”  But there is one absolute: If you work on a Linux or Unix system, you must know how to use the vi editor.  This is because every Linux or Unix system has at least that one editor installed.  Depending on your line of work, it may not be unusual to find yourself on a machine that has only that editor installed.  I started off using vi in college and took the time to learn it well. Years later I found myself in a short lull while switching from the IS41 to the VLR products at Motorola. I took advantage of the downtime by teaching myself emacs and have been primarily an emacs user for since then (about 11 years now). While I prefer emacs, I still use vi every so often, and consider myself fortunate because I know both.

There are many other editors out there and a lot of them would probably meet Hunt’s and Thomas’s criteria, but I would like to add one more to their list: A good editor must be usable without a windowing system. I have also played the role of system administrator, and none of my hosted machines ever had a windowing system installed. This means I could not run eclipse or even jedit. The only editors I could install and use were ones that would work in a text window (an xterm). So it was either vi or emacs.

Pick a good editor, one that meets the criteria listed above, and learn it well. Spend the time to become an expert. The time you invest will be paid back in increased productivity, power and flexibility.

Real Programmers - from http://xkcd.com/378