Boundaries of Change and Variations

The blue represents differences from the "original".

One of the problems with using OO subclassing to handle variations is that the "boundaries of change or variations" often do not fall cleanly around a method's boundary. This results in awkward handling of the way subclasses are overridden, split up, moved, and named. I find that procedural approaches to be more "boundary friendly" in this regard.

One common OOP cliche is delta methods. This is where a class contains only the differences between the new class and the parent(s) by overriding methods that are different. This implies two things. The first, is that one did not have to, in any way, alter the parent class to get the differences; and second, that all the differences are in one spot.

However, as appealing as this cliche sounds, reality bites it on the knee. For one, if other variations (classes) start to share the "same differences", then one has to move code around if we want to factor out duplication of code. Second, grouping by difference may not always be the ideal grouping. Related operations are not next to their peers (see Shapes write-up). But, the main failure of delta mantra that we are looking at here is about how the granularity of differences is often smaller or different than method boundaries.

Dealing with this issue is important. Unlike OO textbook examples that practically manufacture problems and issues for OO to solve, this "problem pattern" is quite common in the business programming that I see. Changes come in all sorts of shapes and sizes.

The following issue is in some ways closely related to case statement issues. However, one of the big differences is that case statements are usually used when each variation implements each named operation mostly different than it's sibling variations. The issue here is dealing with implementations that are mostly the same for each variation, but with somewhat random areas of difference. Dealing with these shotgun-like spots of difference is the problem addressed here.

Let's look at an abbreviated example:

  method A
     [line 1]
     [line 2]
     [line 3]
     [line 4]
     [line 5]
     [line 6]
     [line 7]
  endMethod

1. Put an IF statement in method A around line 4.
2. Subclass the method and duplicate all lines except for line 4.
3. Split method A into smaller methods.

Solution #3 has multiple problems. First, it creates more complexity and clutter. Second, we risk making a mistake and harming code that already references method A. ("If" statements carry some change risk also, but far less.)

Third, assume that we put lines 1 through 3 into one method, 4 into another, and 5 through 7 into a third method. Calling these new methods may then resemble:

  method Do_client_X
     line_1_thru_3()
     line_4_alt()
     line_5_thru_7()
  endMethod

Okay, so now we split it into 7 different methods (one per line) in preparation for future flexibility:

  method A_1
     [line 1]
  endMethod
  method A_2
     [line 2]
  endMethod
  method A_3
     [line 3]
  endMethod
  ...[etc.]...

Although it may reduce implementation factoring, small methods increase "header proliferation". This is where the ratio of code devoted to method and class headers to the amount of implementation code gets higher and higher. Header proliferation can be seen as a form of poor factoring.

Many OO fans really do suggest that splitting algorithms into tiny methods (named blocks) is the way to go. However, this creates a lot of jumping around to read and work with related units. It approaches "Goto" spaghetti code that has rightfully fallen out of favor. See Tiny Methods Up Close below for more on this.

Many OO fans also suggest "refactoring" programs upon some changes. This is another way of saying, "rearrange the code". (I suspect "refactoring" is a euphemism to make this chore sound more dignified.) Reorganization can be risky. Many managers do not want programs altered except to add or fix features. If an significant error occurs due to reorganization, then heads roll. I have been told many times, "If it ain't broke, please don't fix it."

Some OO fans say that most boundaries are predictable in their experience, and therefore that boundary problems are not that common for them. The boundaries seem more predictable when making generic or semi-generic tools, but not for highly customized business applications. The closer you are to the client(s), the more unpredictable the boundaries of change are going to be in my experience. Many requested changes seem quite fickle and capricious to me, especially if the sales and marketing departments are involved. Their "creativity" is astounding at times. Perhaps some have better crystal balls than me, but a paradigm can't assume that everybody has such a gift. It seems roughly half of OO fans say splitting is relatively rare, and the other half say that their tools make the process simple. (see below).

OO fans sometimes respond by saying that one can get special code processing tools that help automate the splitting process. However, code tools can be also applied to solve possible organizational deficiencies of OO's competitors. Thus, code splitting tools sound more like a kludge plug than a solution.

I have envisioned an IDE that collapsed IF statement blocks. To expand the IF statement, you double-click on it, or have an "Expand" button in the middle of the block. It's feel would resemble file folder management tools bundled with some operating systems. It could also perhaps have queries applied to determine which blocks are collapsed and/or highlighted. I do not think such a tool would be much use for me, but OO fans seem bothered by nested IF statements for some reason; so perhaps IDE builders may want to think of some approaches to help them with their IF-conceptualizing deficiencies. See also Fancy IDE's.

I find that the best solution is often a simple IF statement:

  method A   // or Routine A
     [line 1]
     [line 2]
     [line 3]
     if client_id = X
        [line 4X]
     else
        [line 4]
     endIf
     [line 5]
     [line 6]
     [line 7]
  endMethod

  ...
  if current_client.feature_4 = FOO
     [line 4_foo]
  else
     [line 4]
  endIf
  ...

This allows a specific exception to grow into a standard feature via only a single line change of the code and a new table column. Unless you stick all the implementation in a top-most parent class, such "graduation to sharing" in OO requires physically moving code from the specific client subclass up to a parent class. (For this reason, some OO fans suggest putting all the implementation in a parent to begin with. The subclasses only have calls and references to the implementation methods, similar to the "Do_client_X" example above.)

Variation On A Sharing Theme

  sub taskX(...)
    ....
    if a.location = 5 
       doZ
    end if
    ....
  end sub

  sub taskX(...)
    ....
    if a.hasFeatureZ
       doZ
    end if
    ....
  end sub

If instead we put it in to a SatelliteOffice sub-class, it would have to be moved from that subclass to either the parent class, or some other class. ("doZ" may be larger than shown here.)

This is perhaps one reason why "refactoring" is so important to OO projects: OO does not bend easily when noun association changes.

There may be yet more dynamic aspects that fit this pattern. Per-client and per-location are just a few. Just about any "entity" is open to this: products, vendors, etc. "Task" is more invariant than noun instances or sub-classes in a good many cases.

IF Distaste?

1. Makes for messy code

2. Scattering of information related to a variation (client)

3. If statements are more fragile than methods

As far as "messy code", I find the "micro-method" OO alternative to be much much more messy and a lot more work (without splitting automation tools). Conditional indentation is less evil than scattering the sequence around in little chunks.

Note that subroutines can also be split if they get too long. However, factoring is the only decent reason I find to split a routine during maintenance. (i.e. Called from more than one location.) Splitting just to make the pieces smaller is not a good enough reason to change working code in my opinion. Adding IF exceptions usually does not create factoring problems such that a subroutine split is needed. (However, some proceduralists may still like splitting not related to factoring.)

The second complaint is that scattering information related to the variation (a Client) is "bad". These variations should be collected all in one place in OO thinking.

I agree that ideally they should all be collected in one place. However, this gets into the same Grand Proximity Tradeoff as found in case statement proximity battles. Pulling things together by their variation (subclass) forces a scattering of behaviorally related information. Behavioral grouping (putting things together based on their behavior) is important to the way I work and think. I do not want to give that up and the alleged benefits of the by-variation grouping are far too small to justify the trade.

The final complaint is that IF statements are somehow more fragile and/or uglier than method headers (the alternative). I guess the reasoning is that a conditional statement is code, while method headers are more formal or static. (Similar complaints pop up about case statements sometimes.)

Although this is a more or less pedantic complaint in my opinion, let's consider the idea of given the conditional statements names instead of leaving them in code.

   define "Client.clientID = 5" as variation 5
   ...
   variation 5 
      [custom code for client 5]     
   Regular 
      [regular, default code]
   endVariation
   ...

Note that the variation syntax can often be simplified using variable references if there are many variation checks in a routine:

  sub calc_taxes( clientTableHandle )
     var cid = clientTableHandle.clientID   // define a shortcut
     blah...
     if cid = 7
        [custom code for client 7]
     else
        [regular code]
     endIF
     blah...
     blah...
     if cid = 4
        [custom code for client 4]
     endIF     // not all exceptions need elses
     blah...
  end sub

A Closer Look

Let's use a smaller routine size for this example to approach the size that many OO fans prefer. A routine/method ("unit" here) may look something like this:

  Unit ShowColumns(rs)  // rs = record set
    rewind(rs)
    while GetNextRecord(rs)
       display(rs.prodName)
       display(rs.prodID)
       display(rs.prodPrice)
       display(rs.foobar)
    endWhile
  EndUnit

  Unit ShowColumns_B ( rs) overrides ShowColumns
    rewind(rs)
    while GetNextRecord(rs)
       display(rs.prodAbbrev)   // note change
       display(rs.prodID)
       display(rs.prodPrice)
       display(rs.foobar)
    endWhile
  EndUnit

(I suppose the loop control can be put outside, but then you have lots of puny little methods. I usually dislike those because you have to jump around to follow them. More on small methods later.)

I still prefer:

  Unit ShowColumns(rs)
    rewind(rs)
    while GetNextRecord(rs)
       if rs.clientID = X 
          display(rs.prodAbbrev)
       else
          display(rs.prodName)
       end if
       display(rs.prodID)
       display(rs.prodPrice)
       display(rs.foobar)
    endWhile
  EndUnit

  Unit ShowColumns(rs)
    rewind(rs)
    while GetNextRecord(rs)
       var fld = iif(rs.clientID = X, "prodAbbrev","prodName") 
       display(rs[fld])      // syntax varies per language
       display(rs.prodID)
       display(rs.prodPrice)
       display(rs.foobar)
    endWhile
  EndUnit

I still have a single routine/method while subclassing results in two: ShowColumns and ShowColumns_B. Most of the code in these two methods is identical. (Some OO fans may recommend even more splitting at this point.)

If there were four such exceptions in this code unit, then the OO method would have 4 times as much code, while Iffing would result in only about twice as much code.

Further, if we needed to change the standard way items are displayed, then OO would have to change all 4 copies of the unit, while I would only have to change one.

For example, suppose the product ID was too confusing because people did not know whether it was the reseller's product ID or the manufacturer's product ID. Thus, it is agreed that there will now be two ID's instead of one on all listings: "ProdID" and "MfrProdID".

OO would then have to find and change 4 methods while I would only have to change one.

Thus, my approach is at least numerically better in terms of code size, and "change hopping" (factoring). (Even if this example exaggerates Iffing's benefits, nobody has demonstrated that Iffing is objectively worse. Thus, it is fair to call it a wash. Except, iffing can do it with a simpler language/paradigm.)

Some might complain that too many IF's harm readability. Perhaps it is a subjective issue. To me the replication clutter and extra code of subclassing is more of a drawback than lots of IF's. Iffing may be indentation-happy, but method slicing is scatter-happy and header-happy (defining and naming many units). I would rather have a localized mess than a scattered mess.

Tiny Methods Up Close

I liken my development approach to creating a "micro- language" for a given task, with a few main routines that use the created sub-language. Thus, the high-level routines are sort of like pseudo-code, and the supporting routines, which are usually the smaller ones, define the nitty-gritty of the new sub-language. Note that the high-level routines are usually task- specific and not necessarily application-specific. Thus, I avoid the pitfalls of pure top-down design.

Common reasons to factor code into NU's is:

1. Reduce the size of the code by consolidating the same or similar code into a single spot. (Data compression algorithms use a very similar technique: they take repeated sections, give them a shorter name or ID, and then replace the duplicates with the new shorter name.)

2. Make it easier and safer to make changes because the change is in one place instead of multiple places

Note that sometimes things that are the same today may not be the same tomorrow. Thus, I am sometimes reluctant to factor two or more things into one NU if their similarity is fairly likely to be fleeting. Philosophies that dictate consolidating at the first sign of similarity sometimes have problems in this area.

  //---- Pre-factored
  x = a * b - foo(c) * c
  ...
  y = a * b - foo(c) * c

  //---- Post-factored version A
  x = aNU(a,b,c)
  ...
  y = aNU(a,b,c)
  ...
  sub aNU(a,b,c) {
     return  a * b - foo(c) * c
  }

  //---- Post-factored version B
  a,b,c: regional
  ...
  x = aNU()
  ...
  y = aNU()
  ...
  sub aNU() {
     return  a * b - foo(c) * c
  }

As you can see, we did not save a whole lot of code in the above example. The only real advantage is that if the formula changes, we only have to change it in one place (assuming they both use the same formula upon change, which is not always the case. Sometimes a change is to give one section a different implementation from another.)

One drawback of the factoring here is that if we want to see the implementation we have to allow our eyeballs and/or screen to jump down and back up perhaps twice while reading the code. (For simpler things we may have memorized it on the first pass, avoiding a second, assuming we are reading the whole thing.)

Some will argue that a well-named NU will not require one to take a look unless they need to know about the internals. This is yet another sound-good-on-paper idea that does not always play out as clean on a small level. Showing the implementation is often much more informative than a NU name in many or most cases. English is not very effective as describing many technical tasks. Further, some programmers or rather poor at naming NU's. It is a variation on "a picture is worth a thousand words" (code being the picture).

However, the benefits of NU with a reuse factor of 2 is still either positive or neutral for smaller NU in my opinion. A factor of 2 is close to the border-line though. Higher factors have significantly more benefit. Further, larger NU's are also more beneficial even at a reuse factor of 2. Thus, we can say roughly that:

  benefits = rf * s

However, to be able to handle the granularity needed for solving the OOP method boundary problem, the splitting may have to often be at a reuse factor of one! The splitting of code into small NU is no longer to factor out code pattern repetition, but simply to give names to sections of code so that the variations (subclasses) can reference them. The division is simply to prepare for the possibility of a future split.

I see no rational in having small NU's with a reuse factor of one other than to overcome OOP's boundary weakness. (I see some benefit for larger NU's if the sequence is fairly likely to need changing in the future. However, cut-n-paste technology has reduced that need a bit.)

In fact it makes things worse because one has to now jump around to read the code. (This is one reason why GOTO's were disfavored.) It is easier to keep the line of code in-line. Further, it can clutter-up the namespace. If you want to give such a thing a name or description, use a comment.

To justify micro-methods, OO fans (or small NU fans) need to either demonstrate that a vast majority of them have a reuse-factor of 2 or more in real code, or that there is some benefit to small methods with a re-use factor of 1 beyond fixing the potential boundary problem of the OOP paradigm. Any takers?

Testing?

Easier-to-test does not necessarily mean easier-to-read. It might also be more effort to manage the "packaging" of each unit, such as parameters and method/routine naming. More code and effort dedicated to packaging may itself introduce errors.

More study is needed to identify the ideal tradeoff point between readability and testability. It might also be up to the organization to choose quality (fitting to specifications) over development time (RAD) or reduced maintenance resources.

The best size may depend on the domain. A medical application may need strict testing, and small units might work better there. It does not matter that the code may take longer to read and understand, the extra resources may be worth it. However, a marketing-driven company may need (internal) applications that can adapt quickly. Being nimble and budget-minded may be more important than contract-like precision.

Isolation testing also often is not a good test of integration. Integration is often the trickiest part. Many of the stickiest bugs come from unanticipated integration side-effects. Well-tested trees does not guarantee a well-tested forest (or group of trees). The smaller the units, the further one is away from integration.

Some say that RAD results in quality because it allows actual users to test and explore earlier, which results in a design that better fits their needs. In this line of thinking, being usable is more important than fitting a formal specification. Formal specifications are often ill-conceived in many cases anyhow, in my experience.

It may also be claimed that small named units results in RAD and overall less maintenance effort, but I have yet to see how. Remember, we are talking about units with a very low reuse factor.

I also find that it is not necessary to test every single combination of factors, only that each decision path gets executed at least once. It would be nice to test every combination, but it may not be practical and has diminishing returns.

Procedural Delta Grouping

  call RoutineA(....)
  call RoutineB(....)
  call RoutineC(....)
  ....

Footnotes

For example, a bill layout might be the same for all clients initially, and different for only a few later on. A framework won't help much in such a case because the complexity of the framework is not worth a few spot differences. Report frameworks tend to be helpful if there are many instances of variation (such as a different layout per client) and the framework builders anticipated or gave custom control over the spots that will actually differ.

Even if you find/make a report framework, a good many non-report-related variations are rather random and spotty. It is not rational to build a large framework on top of every possible area of variation unless there are dozens or hundreds of variations to manage at that spot. One-off or few-off spots of variations are common in my experience and to extrapolate what a report framework can do to all or most of these spots is generally futile. It would result in a massively bloated and over-engineered application. Perhaps a reporting/list example is a bad example choice on my part because of its association with frameworks.

In the OO approach described above, there seems to be a tradeoff between the size of the methods and implementation factoring. The smaller the methods, the less duplication of implementation. However, it does result in more methods. Thus, it can be said that factoring implementation better results in less header factoring, or header proliferation.