My do-files start with a handful of header commands that I found useful at various times. They might look something like this:

clear
set more off
set type double
set mem 100m
pause on

This stuff varies a bit occasionally (I may set matsize or specify the version) but it's a bit like a cover sheet, in that it's almost always the same thing. I could have equally well put all of this in profile.do.

Next come the main sections of the do-file:

   Overview
   Globals
   Program definitions
   Program execution

The Overview section is all comments. It's got a structure of its own. It always consists of three parts. The first is just a list of the names of the programs defined in this do-file. The second describes what each program does, in one paragraph per program. The third describes which programs are called explicitly (because some programs on the list can be components of others) and in what order. The first part, the quick list, is useful for when you're fishing for code you might want to recycle. If you give your programs descriptive names, a quick look at the top of any do-file is usually enough to tell you if you're going to find useful stuff there.

The Globals section defines the macros such as file paths that will be used by more than one program. Since local macros are local to programs, anything that you mean to be shared across multiple programs must be a global. Having them all bundled here has another advantage. If you send your do-file to be run on another computer, all your file path changes are made in one place, once.

The Program definitions section does just what you might expect. Programs here can be stand-alone things that you call explicitly in the last section, or can be components, called implicitly. Defining such components is useful when you need to use the same code more than once. If that code is broken, you only need to fix it in one place.

You might want two types of comments with your program definitions. One is a header before the program define line (or, if you're cautious, before the capture program drop line) that tells you at least whether your program takes any arguments, lists them if yes, and tells you a little about each of them. You'd want to know, at a minimum, which ones are string and which are numeric. The other is a set of in-line comments, throughout the program definition, as needed. I find it useful to explain any local macros I declare with a couple of words at least.

The Program execution section does the actual work. It has consequences on disk and on screen.

I settled on this way of writing do-files after I took an online class on C++ at NC State. I treat Stata programs inside a do-file the way one would treat functions inside a .cpp file. My Program execution section is the equivalent of main(). In C++, function declarations (also known as prototypes) are mandatory and go at the top of the source file. My do-file equivalent for those is the Overview section. Except, of course, Overview is not mandatory at all. It consists solely of comments.

Having to submit to this sort of discipline might strike you as negating the benefits of Stata's easygoing nature. After all, if you pined for structure, you'd be programming in SAS, where it's mandatory. 

Well, there are two good very good reasons for structure: one is that your code must be portable across your team; another is that it must be readable two weeks later, when you will have forgotten all about it. But I still wouldn't want it imposed upon me by the design of the programming environment. Neither of those very good reasons overrides the importance of on-the-job fun. When you program, you want flow. You need to be free to write up things as they come to you. The programming environment should accommodate that. Only in the tidying-up stage, when your thinking's done and your problem's solved, should you need to worry about structure.

Programming environments that impose structure on you, as opposed to letting you volunteer it, result in beautiful code that takes a long time to write and robs you of most of the pleasure of solving the original problem. The latter might well cause you to do a mediocre job of it. When help like this also costs you more in licensing fees and specialized labor, that's just insult upon injury.

Some context first: one of my ongoing projects is a suite of Stata do-files for helping certain daily newspapers optimize their subscriber retention, using survival analysis. I have several clients. Some of their needs are common, so they can be met with one do-file, others are not so common, so they need custom code. I use global macros as bridges between custom and common code. You need this introduction to see what I'm doing in my example below.

One client is sending me a weekly snapshot of its subscriber database in the MS Access format, in a file called Submaster_YYYYMMDD.mdb. Suppose that that file resides in c:/data/my access file comes from here/. Then I must read it into Stata and save it as subsYYYYMMDD.dta in c:/data/my stata file goes here/. Now suppose that I spread this project across several do-files, so I use some internal names for these file paths in the form of global macros defined once:


global datafrom "c:/data/my access file comes from here/"
global datato "c:/data/my stata file goes here/"


Also suppose that I save the date of arrival part of the file name, YYYYMMDD, as a global macro:


global t "YYYYMMDD"


Finally, I don't want my lines to be too long, so I use local macros to compress things a bit whenever that looks practical. Now let's have a look at the actual example:


capture confirm file "${datato}subs${t}.dta"
if _rc!=0 {

   local mdbpath "${datafrom}"
   local mdbname "Submaster"
   local source "MS Access Database;DBQ=`mdbpath'`mdbname'_${t}.mdb"
   capture confirm file "`mdbpath'`mdbname'_${t}.mdb"

   if _rc==0 {
      #delimit ;
      local vars "
      ${${paper}index},
      prod_code,
      start_orig,
      last_start_date,
      last_stop_date,
      expire_date,
      rate_table,
      paper_code,
      rate,
      credit_card_usage";

      drop _all;
      odbc load,
      exec("SELECT `vars' FROM `table' WHERE (prod_code='1D' OR prod_code='3D')")
      dsn("`source'");
      save "${datato}subs${t}", replace;
      #delimit cr
   }
}


First, I check if the Stata file subsYYYYMMDD.dta exists in the path ${datato} with "capture confirm file". If it does not, then I check if the Access file Submaster_YYYYMMDD.mdb exists in the path ${datafrom}. If yes, then I read it and save subsYYYYMMDD.dta. The exec() part of odbc load is for running SQL syntax. Just to make sure that you appreciate the coolness of this: any SQL line that you could execute in Access, no matter how complicated, can be put between the exec() parentheses. Mine is simple -- I wrote the SQL commands in all caps for clarity -- but yours needn't be. You can also combine Stata locals with SQL syntax like I did.

I switch to the semicolon delimiter when it helps readability. It allows the line with SQL syntax to wrap around, and the listing of my `vars' local as one word per line. This, of course, is optional. The checks for the existence of either file are also optional. They keep my do-files from exiting with an error if I happen to run them on a day when either file does not exist -- either because the source file Submaster has not arrived yet, or because it hasn't been translated yet into Stata.