Stata, from version 8 onward, has done a wonderful job of taking the pain out of the process as much as practical. The Stata 10 graphics menu is a sight to behold and the ability to click-and-edit graphs interactively, if you haven't tried it yet, will blow your mind. Better yet, anything you do interactively leaves a command trail that you can save, so next time you need the same graph you can just run the command counterpart.

Still, the topic is heavy enough to warrant its own book and you probably will want to buy it. What I'm proposing here is just one option for mass-producing graphs and changing them in bulk, as needed.

If you ever looked at a graph command, you may have noticed that the commands for three or four graphs that show different facets of the same analysis, maybe feeding off the same data set, have pretty large common chunks. I did, and I thought it'd be nice if these common chunks showed up in only one place where if I need to edit them I can do so once and for all.

Local macros can accomplish that. You store parts of a full command in separate strings, some common, some specific to each graph, and save these strings as locals. Calling the graph command is then simply a matter of chaining all these locals back together.

Here's some code that does that for three different graphs, each with some quirk that needs to be accommodated:

// all graphs to draw
local graphs "retention cos revenue"

// graphs for only the test group
local lilgraphs "cos revenue"

// graph titles
local retention_title "Retention rates by target cohort, oldest first"
local cos_title       "Changes of service by snapshot date"
local cos_subtitle    "Accounts on a different FOD from when targeted"
local revenue_title   "Current revenue per targeted subscriber"

// var labels
label var count_then     "Accounts targeted"
label var cos            "Change of service"
label var retention_rate "Retention rate"
label var age            "Days since targeted"
label var control        "Group"
label var date_target    "Snapshot date"
label var revenue_pc_now "Average revenue per targeted subscriber"
label var count_now      "Targeted subscribers now active"
foreach time in then now {
  format count_`time' %6.0fc
}
label define control_label 0 "Control Accounts" 1 "Targeted Accounts"
label values control control_label
levelsof date_target, local(date_ticks) // note 1 

// generic graph settings
local xtick_settings_1 "xtick(none) xlabel(none) xmtick(`date_ticks')" // note 2
local xtick_settings_2 "xmlabel(`date_ticks', angle(vertical) labsize(small))"
local xtick_settings   "`xtick_settings_1' `xtick_settings_2'"
local ytick_settings_1 "ylabel(,angle(horizontal) labsize(small) axis(1))"
local ytick_settings_2 "ylabel(,angle(horizontal) labsize(small) axis(2))"
local legend_settings  "legend(cols(1))"

// specific graph settings
foreach gph in `graphs' {
  local title        "title("``gph'_title'")"
  local subtitle     "subtitle("``gph'_subtitle'", size(small))" // note 3
  local `gph'_title  "`title' `subtitle' "
  local `gph'_saving "saving("`report_to'`gph'_`date'.gph", replace)"
}
local style "twoway line"
local retention_line "`style' retention_rate age, by(control, `retention_title') "
local cos_line       "`style' cos date_target, `cos_title' "
local revenue_line_1 "`style' revenue_pc_now date_target || "
local revenue_line_2 "line count_now date_target, yaxis(2) `revenue_title' "
local revenue_line   "`revenue_line_1' `revenue_line_2'"

local retention_ticks "`ytick_settings_1'"
local cos_ticks       "`xtick_settings' `ytick_settings_1'"
local revenue_ticks   "`cos_ticks' `ytick_settings_2' `legend_settings'"

// assemble complete commands
foreach gph in `graphs' {
  local `gph'_graph "``gph'_line' ``gph'_ticks' ``gph'_saving'"
}

// draw retention graph
`retention_graph'

preserve
keep if control==1
sort date_target
levelsof date_target, local(date_ticks)

// draw test group graphs
foreach gph in `lilgraphs' {
  ``gph'_graph'
}
restore

The code above is part of a larger effort to keep daily newspapers in business, to which I am a contributor. I had to draw up some reports for a client -- a media company that runs a bunch of local papers -- to show the status report for an experiment in subscriber retention started a while back. Certain subsets of monthly cohorts of subscribers are selected into a target and a control group, we do some magic to the target group, and then compare its retention, revenue, and change-of-service (cos) behavior to that of the control group. 

The cos and revenue graphs above need only use a subset of data in this case. The revenue graph is the kind with two y-axes, with average revenue per subscriber tracked along the first y axis and count of active subscribers tracked along the second. The common x axis is time.

Look at the block of code labeled "generic graph settings". Never mind the two separate xtick_settings locals. I am only defining them in order to trade width from length. For the purpose of this blog, I want my code to fit properly inside the column. I am stringing them together immediately afterwards as `xtick_settings'. The two `ytick_settings_' locals are a different matter, however. I need them both for the revenue graph (since it has two y-axes) and I need only the first for the retention and cos graphs. To see how that works later on, notice the difference in the definitions of the `cos_ticks' and `revenue_ticks' locals.

Now look at the in-line comments "note 1" and "note 2". Stata, by default, will keep you from cluttering your axes with too many tick marks and a jumble of overlapping labels. That's nice, but not what I want in this case. These are small graphs and I want every point on the x-axis ticked and labeled. I collect the labels on the line marked "note 1" as the list `date_ticks' and then use them as arguments for the xmtick() and xmlabel() options on the line marked "note 2" and the one below it.

If you look at the block of code labeled "graph titles" you will notice that only the cos graph has a subtitle. That's OK. Stata won't kick up a fuss if you invoke a missing subtitle. That is what I do in the line marked with the in-line comment "note 3".

Now all I need to do is assemble these pieces into full graph commands. Notice the line below the comment "draw retention graph": all of your graph command, with options and everything, fits into that little `retention_graph' local. Of course the point is not to hide your command away. It's to make the chunks available for editing in blocks that make sense. If you need to change the title of a graph, it's easier to look for it in a block of local definitions clearly labeled // graph titles than it would be to go fishing for it inside big messy graph commands. But the bigger gain is that you get to re-use common chunks: everything in the block labeled "generic graph settings" needs to be changed only in one place and applies everywhere, and even some of the "specific graph settings" can be thought of that way.

Of course, you don't want to go through this exercise in abstraction for every occasional graph. This stuff is strictly for mass production. You've got to be able to justify to the client a fixed cost of this size.

I don't usually use macros that way (OK, I wasn't even aware of the possibility until today) but today that need arose, so here's what I learned.

One example of the difference between a thing's looks and substance would be this:

di c(current_date)

What you see on the screen -- today that would be the string "12 Dec 2008" -- is different from what's behind it -- the value 17878, i.e., the number of days since January 1, 1960.

So you could simply do

local mydate c(current_date)
di "`mydate'"

This looks useful, and maybe it is. The local macro `mydate' above has stored the look of c(current_date), but it knows nothing about its content. You could change that if you defined that macro like this:

local mydate=date(c(current_date),"DMY")
di `mydate'

So `mydate' is now a number, not a string. Specifically, it's an actual date -- from which you can derive things like week(`mydate') or dow(`mydate') or what have you.

But a look of this content, as opposed to the content itself, might be exactly what you want. For example, you may want the YYYYMMDD equivalent of today's date. Easy:

local mydate: di %tdCYND date(c(current_date),"DMY")
di `mydate'

So much about that. It's Friday, can't work too hard.

Some context first: one of my ongoing projects is a suite of Stata do-files for helping certain daily newspapers optimize their subscriber retention, using survival analysis. I have several clients. Some of their needs are common, so they can be met with one do-file, others are not so common, so they need custom code. I use global macros as bridges between custom and common code. You need this introduction to see what I'm doing in my example below.

One client is sending me a weekly snapshot of its subscriber database in the MS Access format, in a file called Submaster_YYYYMMDD.mdb. Suppose that that file resides in c:/data/my access file comes from here/. Then I must read it into Stata and save it as subsYYYYMMDD.dta in c:/data/my stata file goes here/. Now suppose that I spread this project across several do-files, so I use some internal names for these file paths in the form of global macros defined once:


global datafrom "c:/data/my access file comes from here/"
global datato "c:/data/my stata file goes here/"


Also suppose that I save the date of arrival part of the file name, YYYYMMDD, as a global macro:


global t "YYYYMMDD"


Finally, I don't want my lines to be too long, so I use local macros to compress things a bit whenever that looks practical. Now let's have a look at the actual example:


capture confirm file "${datato}subs${t}.dta"
if _rc!=0 {

   local mdbpath "${datafrom}"
   local mdbname "Submaster"
   local source "MS Access Database;DBQ=`mdbpath'`mdbname'_${t}.mdb"
   capture confirm file "`mdbpath'`mdbname'_${t}.mdb"

   if _rc==0 {
      #delimit ;
      local vars "
      ${${paper}index},
      prod_code,
      start_orig,
      last_start_date,
      last_stop_date,
      expire_date,
      rate_table,
      paper_code,
      rate,
      credit_card_usage";

      drop _all;
      odbc load,
      exec("SELECT `vars' FROM `table' WHERE (prod_code='1D' OR prod_code='3D')")
      dsn("`source'");
      save "${datato}subs${t}", replace;
      #delimit cr
   }
}


First, I check if the Stata file subsYYYYMMDD.dta exists in the path ${datato} with "capture confirm file". If it does not, then I check if the Access file Submaster_YYYYMMDD.mdb exists in the path ${datafrom}. If yes, then I read it and save subsYYYYMMDD.dta. The exec() part of odbc load is for running SQL syntax. Just to make sure that you appreciate the coolness of this: any SQL line that you could execute in Access, no matter how complicated, can be put between the exec() parentheses. Mine is simple -- I wrote the SQL commands in all caps for clarity -- but yours needn't be. You can also combine Stata locals with SQL syntax like I did.

I switch to the semicolon delimiter when it helps readability. It allows the line with SQL syntax to wrap around, and the listing of my `vars' local as one word per line. This, of course, is optional. The checks for the existence of either file are also optional. They keep my do-files from exiting with an error if I happen to run them on a day when either file does not exist -- either because the source file Submaster has not arrived yet, or because it hasn't been translated yet into Stata.

Stata can do something similar. What you aim to abstract from when writing do-files is not necessarily any given operating system, though you can do that too. Rather, abstraction should ensure that your code is not a literal translation of how you think about the job. Your most specific ideas at the beginning of a project are the most likely to change before the job is done. Writing flexible code that can accommodate such changes is a lot more convenient than cutting out and retyping big swaths of your masterpiece as you go along. Macros, local and global, can help.

One obvious use for macros is file path abstraction. You may well start a project with a simple structure -- such as "c:/my project/" with subfolders "data" and "code". But what if you move it to a new letter drive? What if along the way your data and code need to be re-organized in different sub-folders?

Declaring your file paths once, at the top of the do-file, as macros such as

local sourcefiles "c:/my project/data/source/"

will enable you to call your sourcefile in the same way throughout the rest of the do-file:

use "`sourcefiles'mysource", clear

The obvious advantage to this is that all the file path declarations are in one place, and each appears just once. Should you have to change any of them, you can do so very quickly and 100% reliably, and that is not a metaphor. You only need one change, in one place. That's 100%.

If you move from Windows to UNIX, you will simply change your macro's definition to something like this:

local sourcefiles "/usr/home/yourusername/my project/data/source/"

Everything else in your do-file can stay the same. That is true platform independence. File names, variable lists for sorting or merging, constants, all can be equally easily abstracted away with macros. You can even use macros to substitute inside commands, like so:

local thisvar_is_ready "thisvar>0 & thisvar!=."
replace newvar=thatvar/thisvar if `thisvar_is_ready'

Should your macros be global or local? The difference is one of scope. If you declare a global macro it stays live for the entire lifetime of your Stata instance, so global macros can be inadvertently carried over to do-files that they have nothing to do with. Local macros are local to the do-files where they are defined. They die when the execution of their home do-file stops, either at the end of the file or as the result of an error. For this reason the general advice is that you should not use a global macro when a local one will suffice.

But the upshot of that advice is not that the cautious Stata user will never use global macros. If you organize your work in hierarchical do-files that call each other, then it makes sense to declare all your macros as globals in one place -- the master do-file that calls all others.

I use a combination of global and local macros, as I judge the circumstances to warrant either type. I do not have a default preference. To ensure against carrying globals over to do-files where they do not belong, you could start your do-files with capture macro drop _all. That would be drastic, but all the more effective.