Method Change Boundaries

Boundaries of Change and Variations

Change or variation granularity often does not match method granularity

Updated 7/1/2002



The blue represents differences from the "original".

One of the problems with using OO subclassing to handle variations is that the "boundaries of change or variations" often do not fall cleanly around a method's boundary. This results in awkward handling of the way subclasses are overridden, split up, moved, and named. I find that procedural approaches to be more "boundary friendly" in this regard.

One common OOP cliche is delta methods. This is where a class contains only the differences between the new class and the parent(s) by overriding methods that are different. This implies two things. The first, is that one did not have to, in any way, alter the parent class to get the differences; and second, that all the differences are in one spot.

However, as appealing as this cliche sounds, reality bites it on the knee. For one, if other variations (classes) start to share the "same differences", then one has to move code around if we want to factor out duplication of code. Second, grouping by difference may not always be the ideal grouping. Related operations are not next to their peers (see Shapes write-up). But, the main failure of delta mantra that we are looking at here is about how the granularity of differences is often smaller or different than method boundaries.

Dealing with this issue is important. Unlike OO textbook examples that practically manufacture problems and issues for OO to solve, this "problem pattern" is quite common in the business programming that I see. Changes come in all sorts of shapes and sizes.

The following issue is in some ways closely related to case statement issues. However, one of the big differences is that case statements are usually used when each variation implements each named operation mostly different than it's sibling variations. The issue here is dealing with implementations that are mostly the same for each variation, but with somewhat random areas of difference. Dealing with these shotgun-like spots of difference is the problem addressed here.
Predicting the future whims and change requests of management and marketing decisions is quite hard. My accuracy has improved greatly over the years, but is far from perfect. Human "creativity" from the change "requesters" is simply too great. Thus, to presume that one can usually pick the right boundaries at design time is absurd.

Let's look at an abbreviated example:

  method A
     [line 1]
     [line 2]
     [line 3]
     [line 4]
     [line 5]
     [line 6]
     [line 7]
  endMethod
Suppose a new client X (customer) needs method A, but is different in only line 4. Our choices seem to be:
  1. Put an IF statement in method A around line 4.
  2. Subclass the method and duplicate all lines except for line 4.
  3. Split method A into smaller methods.
The problem with #2 is that now we have poor factoring. We are duplicating most of the code (6 of the 7 lines).

Solution #3 has multiple problems. First, it creates more complexity and clutter. Second, we risk making a mistake and harming code that already references method A. ("If" statements carry some change risk also, but far less.)

Third, assume that we put lines 1 through 3 into one method, 4 into another, and 5 through 7 into a third method. Calling these new methods may then resemble:

  method Do_client_X
     line_1_thru_3()
     line_4_alt()
     line_5_thru_7()
  endMethod
However, what if a new client Y comes along and is the same as our original method A, except for line 3? Do we play the splitting game again? Plus, we now have to rework our Do_client_X method (at least).

Okay, so now we split it into 7 different methods (one per line) in preparation for future flexibility:

  method A_1
     [line 1]
  endMethod
  method A_2
     [line 2]
  endMethod
  method A_3
     [line 3]
  endMethod
  ...[etc.]...
It is messy (in my opinion), but it reduces factoring.
Although it may reduce implementation factoring, small methods increase "header proliferation". This is where the ratio of code devoted to method and class headers to the amount of implementation code gets higher and higher. Header proliferation can be seen as a form of poor factoring.
However, client Z comes along and is different in only part of line 6. Do we now split up the method for line 6 into methods also? It is now growing into a Cat-in-the-Hat type of problem. (Re: a famous Dr. Seuss book, The Cat in the Hat Comes Back, in which every cat has a smaller cat in his hat. Sort of "fractal cats" if you will.)

Many OO fans really do suggest that splitting algorithms into tiny methods (named blocks) is the way to go. However, this creates a lot of jumping around to read and work with related units. It approaches "Goto" spaghetti code that has rightfully fallen out of favor. See Tiny Methods Up Close below for more on this.

Many OO fans also suggest "refactoring" programs upon some changes. This is another way of saying, "rearrange the code". (I suspect "refactoring" is a euphemism to make this chore sound more dignified.) Reorganization can be risky. Many managers do not want programs altered except to add or fix features. If an significant error occurs due to reorganization, then heads roll. I have been told many times, "If it ain't broke, please don't fix it."

Some OO fans say that most boundaries are predictable in their experience, and therefore that boundary problems are not that common for them. The boundaries seem more predictable when making generic or semi-generic tools, but not for highly customized business applications. The closer you are to the client(s), the more unpredictable the boundaries of change are going to be in my experience. Many requested changes seem quite fickle and capricious to me, especially if the sales and marketing departments are involved. Their "creativity" is astounding at times. Perhaps some have better crystal balls than me, but a paradigm can't assume that everybody has such a gift. It seems roughly half of OO fans say splitting is relatively rare, and the other half say that their tools make the process simple. (see below).
If you split a method, you may have to also split all the "polymorphic siblings." In other words, you have to make all the method name signatures match for each variation (subclass). Thus, a single exception (oddity) can trigger the splitting of dozens or more methods. It is a snow ball effect. (It reminds me of manually performing the node-splitting effort of B-trees, which are often used for data indexing.) I have dubbed this process "polymorphic split cascade". One OO fan admitted to having about 40 edit windows open on occasion to perform such "polymorphic cascade surgery". (He seemed a fast and accurate typist/mouser, so the quantity did not bother him.)

OO fans sometimes respond by saying that one can get special code processing tools that help automate the splitting process. However, code tools can be also applied to solve possible organizational deficiencies of OO's competitors. Thus, code splitting tools sound more like a kludge plug than a solution.

I have envisioned an IDE that collapsed IF statement blocks. To expand the IF statement, you double-click on it, or have an "Expand" button in the middle of the block. It's feel would resemble file folder management tools bundled with some operating systems. It could also perhaps have queries applied to determine which blocks are collapsed and/or highlighted. I do not think such a tool would be much use for me, but OO fans seem bothered by nested IF statements for some reason; so perhaps IDE builders may want to think of some approaches to help them with their IF-conceptualizing deficiencies. See also Fancy IDE's.
I find it strange that OO fans accept this splitting process as a standard chore. This need to split and reorganize the code, especially repeating it in order to make polymorphic siblings match, is rather silly.

I find that the best solution is often a simple IF statement:

  method A   // or Routine A
     [line 1]
     [line 2]
     [line 3]
     if client_id = X
        [line 4X]
     else
        [line 4]
     endIf
     [line 5]
     [line 6]
     [line 7]
  endMethod
If the same feature starts to apply to other clients, then I often put a "feature switch" or "feature selector" field in the variation (Client) table:
  ...
  if current_client.feature_4 = FOO
     [line 4_foo]
  else
     [line 4]
  endIf
  ...
("current_client" is a record "pointer" in this case. Also "feature_4" would normally have a more meaningful name such as "is_nonprofit".)

This allows a specific exception to grow into a standard feature via only a single line change of the code and a new table column. Unless you stick all the implementation in a top-most parent class, such "graduation to sharing" in OO requires physically moving code from the specific client subclass up to a parent class. (For this reason, some OO fans suggest putting all the implementation in a parent to begin with. The subclasses only have calls and references to the implementation methods, similar to the "Do_client_X" example above.)

Variation On A Sharing Theme

A similar theme to the above per-client variations growing into shared units can be also found in locational themes. For example, suppose satellite office #5 originally had a variation known as "Z":
  sub taskX(...)
    ....
    if a.location = 5 
       doZ
    end if
    ....
  end sub
If later it turns into a shared feature, it can become:
  sub taskX(...)
    ....
    if a.hasFeatureZ
       doZ
    end if
    ....
  end sub
When it changes from location-specific feature to a shared feature, only the IF statement has to be changed in most cases.

If instead we put it in to a SatelliteOffice sub-class, it would have to be moved from that subclass to either the parent class, or some other class. ("doZ" may be larger than shown here.)

This is perhaps one reason why "refactoring" is so important to OO projects: OO does not bend easily when noun association changes.

There may be yet more dynamic aspects that fit this pattern. Per-client and per-location are just a few. Just about any "entity" is open to this: products, vendors, etc. "Task" is more invariant than noun instances or sub-classes in a good many cases.

IF Distaste?

Common complaints of "iffing" from OO fans include:

  1. Makes for messy code

  2. Scattering of information related to a variation (client)

  3. If statements are more fragile than methods

Let's look at each of these.

As far as "messy code", I find the "micro-method" OO alternative to be much much more messy and a lot more work (without splitting automation tools). Conditional indentation is less evil than scattering the sequence around in little chunks.

Note that subroutines can also be split if they get too long. However, factoring is the only decent reason I find to split a routine during maintenance. (i.e. Called from more than one location.) Splitting just to make the pieces smaller is not a good enough reason to change working code in my opinion. Adding IF exceptions usually does not create factoring problems such that a subroutine split is needed. (However, some proceduralists may still like splitting not related to factoring.)

The second complaint is that scattering information related to the variation (a Client) is "bad". These variations should be collected all in one place in OO thinking.

I agree that ideally they should all be collected in one place. However, this gets into the same Grand Proximity Tradeoff as found in case statement proximity battles. Pulling things together by their variation (subclass) forces a scattering of behaviorally related information. Behavioral grouping (putting things together based on their behavior) is important to the way I work and think. I do not want to give that up and the alleged benefits of the by-variation grouping are far too small to justify the trade.

The final complaint is that IF statements are somehow more fragile and/or uglier than method headers (the alternative). I guess the reasoning is that a conditional statement is code, while method headers are more formal or static. (Similar complaints pop up about case statements sometimes.)

Although this is a more or less pedantic complaint in my opinion, let's consider the idea of given the conditional statements names instead of leaving them in code.

   define "Client.clientID = 5" as variation 5
   ...
   variation 5 
      [custom code for client 5]     
   Regular 
      [regular, default code]
   endVariation
   ...
This makes it a bit more formal than an IF statement. However, I don't really see much benefit in it. It would be a poor expenditure of language complexity to add something like that.

Note that the variation syntax can often be simplified using variable references if there are many variation checks in a routine:

  sub calc_taxes( clientTableHandle )
     var cid = clientTableHandle.clientID   // define a shortcut
     blah...
     if cid = 7
        [custom code for client 7]
     else
        [regular code]
     endIF
     blah...
     blah...
     if cid = 4
        [custom code for client 4]
     endIF     // not all exceptions need elses
     blah...
  end sub
It does not get much shorter and simpler than that.

A Closer Look

Let's take a closer look at an example where one client wants the product listing report or screen to show the product abbreviation instead of the full title. (See footnote about using report frameworks instead.) Suppose their employees are used to reading the abbreviated name, and thus they consider it faster to read.

A regular product listing
ProductProd.ID PriceFoo
Chicken Plucker123-5489.99zerg
Lint Finder6635212.99Yig
1 Gal. Lard333135.89Mulder
Pillow Lock553-1285.95Rox

Let's use a smaller routine size for this example to approach the size that many OO fans prefer. A routine/method ("unit" here) may look something like this:

  Unit ShowColumns(rs)  // rs = record set
    rewind(rs)
    while GetNextRecord(rs)
       display(rs.prodName)
       display(rs.prodID)
       display(rs.prodPrice)
       display(rs.foobar)
    endWhile
  EndUnit
Now, if the one client wants the abbreviated product description instead, an OO approach may resemble:
  Unit ShowColumns_B ( rs) overrides ShowColumns
    rewind(rs)
    while GetNextRecord(rs)
       display(rs.prodAbbrev)   // note change
       display(rs.prodID)
       display(rs.prodPrice)
       display(rs.foobar)
    endWhile
  EndUnit
Here, you have poor factoring. You are replicating everything except for a minor change.

(I suppose the loop control can be put outside, but then you have lots of puny little methods. I usually dislike those because you have to jump around to follow them. More on small methods later.)

I still prefer:

  Unit ShowColumns(rs)
    rewind(rs)
    while GetNextRecord(rs)
       if rs.clientID = X 
          display(rs.prodAbbrev)
       else
          display(rs.prodName)
       end if
       display(rs.prodID)
       display(rs.prodPrice)
       display(rs.foobar)
    endWhile
  EndUnit
An alternative way to write this may be:
  Unit ShowColumns(rs)
    rewind(rs)
    while GetNextRecord(rs)
       var fld = iif(rs.clientID = X, "prodAbbrev","prodName") 
       display(rs[fld])      // syntax varies per language
       display(rs.prodID)
       display(rs.prodPrice)
       display(rs.foobar)
    endWhile
  EndUnit
(The IIF function evaluates the first parameter, and returns the second parameter if true, or the third parameter if false. Variations of IIF are found in several languages, including spreadsheets.)

I still have a single routine/method while subclassing results in two: ShowColumns and ShowColumns_B. Most of the code in these two methods is identical. (Some OO fans may recommend even more splitting at this point.)

If there were four such exceptions in this code unit, then the OO method would have 4 times as much code, while Iffing would result in only about twice as much code.

Further, if we needed to change the standard way items are displayed, then OO would have to change all 4 copies of the unit, while I would only have to change one.

For example, suppose the product ID was too confusing because people did not know whether it was the reseller's product ID or the manufacturer's product ID. Thus, it is agreed that there will now be two ID's instead of one on all listings: "ProdID" and "MfrProdID".

OO would then have to find and change 4 methods while I would only have to change one.

Thus, my approach is at least numerically better in terms of code size, and "change hopping" (factoring). (Even if this example exaggerates Iffing's benefits, nobody has demonstrated that Iffing is objectively worse. Thus, it is fair to call it a wash. Except, iffing can do it with a simpler language/paradigm.)

Some might complain that too many IF's harm readability. Perhaps it is a subjective issue. To me the replication clutter and extra code of subclassing is more of a drawback than lots of IF's. Iffing may be indentation-happy, but method slicing is scatter-happy and header-happy (defining and naming many units). I would rather have a localized mess than a scattered mess.


Tiny Methods Up Close

Some OO fans say that really small methods "improve the design" of the system beyond just "fixing" the "boundaries of change" issue (above). However, they are very imprecise in describing exactly why. Some procedural programmers also like mostly tiny subroutines. However, I prefer a mix of small and medium. Thus, it may be a subjective decision. I won't dictate to people how to think.
I liken my development approach to creating a "micro- language" for a given task, with a few main routines that use the created sub-language. Thus, the high-level routines are sort of like pseudo-code, and the supporting routines, which are usually the smaller ones, define the nitty-gritty of the new sub-language. Note that the high-level routines are usually task- specific and not necessarily application-specific. Thus, I avoid the pitfalls of pure top-down design.
However, there are some logical problems to very small methods and routines. I will refer to methods and routines as "named units" or NU for short. The primary problem relates to the "reuse factor". This is the number of different places that the NU is called or referenced. For example, if a NU is called from 3 different places, then it has a reuse factor of 3.

Common reasons to factor code into NU's is:

  1. Reduce the size of the code by consolidating the same or similar code into a single spot. (Data compression algorithms use a very similar technique: they take repeated sections, give them a shorter name or ID, and then replace the duplicates with the new shorter name.)

  2. Make it easier and safer to make changes because the change is in one place instead of multiple places
Note that sometimes things that are the same today may not be the same tomorrow. Thus, I am sometimes reluctant to factor two or more things into one NU if their similarity is fairly likely to be fleeting. Philosophies that dictate consolidating at the first sign of similarity sometimes have problems in this area.
A factoring example:
  //---- Pre-factored
  x = a * b - foo(c) * c
  ...
  y = a * b - foo(c) * c

  //---- Post-factored version A
  x = aNU(a,b,c)
  ...
  y = aNU(a,b,c)
  ...
  sub aNU(a,b,c) {
     return  a * b - foo(c) * c
  }

  //---- Post-factored version B
  a,b,c: regional
  ...
  x = aNU()
  ...
  y = aNU()
  ...
  sub aNU() {
     return  a * b - foo(c) * c
  }
Here we did some factoring of 2 lines of similar code. Version A uses parameters, whereas version B uses regional variables, which reduces the need to pass around and maintain parameters. (Regional variables would usually be class-level variables in OOP or module-level variables in procedural code. Pascal allows nested routines where the scope of the parent routine is "inherited".)

As you can see, we did not save a whole lot of code in the above example. The only real advantage is that if the formula changes, we only have to change it in one place (assuming they both use the same formula upon change, which is not always the case. Sometimes a change is to give one section a different implementation from another.)

One drawback of the factoring here is that if we want to see the implementation we have to allow our eyeballs and/or screen to jump down and back up perhaps twice while reading the code. (For simpler things we may have memorized it on the first pass, avoiding a second, assuming we are reading the whole thing.)

Some will argue that a well-named NU will not require one to take a look unless they need to know about the internals. This is yet another sound-good-on-paper idea that does not always play out as clean on a small level. Showing the implementation is often much more informative than a NU name in many or most cases. English is not very effective as describing many technical tasks. Further, some programmers or rather poor at naming NU's. It is a variation on "a picture is worth a thousand words" (code being the picture).

However, the benefits of NU with a reuse factor of 2 is still either positive or neutral for smaller NU in my opinion. A factor of 2 is close to the border-line though. Higher factors have significantly more benefit. Further, larger NU's are also more beneficial even at a reuse factor of 2. Thus, we can say roughly that:

  benefits = rf * s
Where "rf" is the reuse factor and "s" is the size of the NU. (There should probably also be a "fixed cost overhead" factor of NU's, but I wish to keep it simple for now.)

However, to be able to handle the granularity needed for solving the OOP method boundary problem, the splitting may have to often be at a reuse factor of one! The splitting of code into small NU is no longer to factor out code pattern repetition, but simply to give names to sections of code so that the variations (subclasses) can reference them. The division is simply to prepare for the possibility of a future split.

I see no rational in having small NU's with a reuse factor of one other than to overcome OOP's boundary weakness. (I see some benefit for larger NU's if the sequence is fairly likely to need changing in the future. However, cut-n-paste technology has reduced that need a bit.)

In fact it makes things worse because one has to now jump around to read the code. (This is one reason why GOTO's were disfavored.) It is easier to keep the line of code in-line. Further, it can clutter-up the namespace. If you want to give such a thing a name or description, use a comment.

To justify micro-methods, OO fans (or small NU fans) need to either demonstrate that a vast majority of them have a reuse-factor of 2 or more in real code, or that there is some benefit to small methods with a re-use factor of 1 beyond fixing the potential boundary problem of the OOP paradigm. Any takers?


Testing?

Some have suggested that tiny named units (above) makes testing easier because one has less factors or variations to test per unit. This is the best justification for small units that I have heard so far. However, readability and integration-level testing may be sacrificed.

Easier-to-test does not necessarily mean easier-to-read. It might also be more effort to manage the "packaging" of each unit, such as parameters and method/routine naming. More code and effort dedicated to packaging may itself introduce errors.

More study is needed to identify the ideal tradeoff point between readability and testability. It might also be up to the organization to choose quality (fitting to specifications) over development time (RAD) or reduced maintenance resources.

The best size may depend on the domain. A medical application may need strict testing, and small units might work better there. It does not matter that the code may take longer to read and understand, the extra resources may be worth it. However, a marketing-driven company may need (internal) applications that can adapt quickly. Being nimble and budget-minded may be more important than contract-like precision.
I doubt the ideal is one and two-line units, however. Around ten sounds much more reasonable to me.

Isolation testing also often is not a good test of integration. Integration is often the trickiest part. Many of the stickiest bugs come from unanticipated integration side-effects. Well-tested trees does not guarantee a well-tested forest (or group of trees). The smaller the units, the further one is away from integration.

Some say that RAD results in quality because it allows actual users to test and explore earlier, which results in a design that better fits their needs. In this line of thinking, being usable is more important than fitting a formal specification. Formal specifications are often ill-conceived in many cases anyhow, in my experience.

It may also be claimed that small named units results in RAD and overall less maintenance effort, but I have yet to see how. Remember, we are talking about units with a very low reuse factor.

I also find that it is not necessary to test every single combination of factors, only that each decision path gets executed at least once. It would be nice to test every combination, but it may not be practical and has diminishing returns.


Procedural Delta Grouping

If one does want to go the "delta" route, it is possible to do delta-grouping using procedural programming also. However, your approach may heavily depend on the features and limitations of particular languages because function scoping rules vary widely across languages. They also suffer from many of the problems of their OOP cousins. But there may be cases where it is still appropriate.

The simplest approach is to have a list of subroutines, something like:

  call RoutineA(....)
  call RoutineB(....)
  call RoutineC(....)
  ....
Every "instance" (variation) of the module could contain this list, but with explicit implementations of those routines that are different for a given instant. For example, we may want to implement our own version of routine-B's logic above, and thus not call routine-B. We could perhaps paste in a copy of routine-B's contents in place of where routine-B's call would normally have been, and then alter it.

A drawback of this approach is that one has to repeat the list of routines for each instance. If we have many instances, then such repetition may be considered poor factoring. (Note that if the parameters vary, then it still may be the best option.) To get around problems like this, some languages allow "local" routines to take precedence over other modules which have routines of the same name. Thus, we could "override" any of those in the library modules. The details on doing this vary greatly per language, however. Some experimentation may be in order. Generally languages like Lisp with its lambda abilities will make this kind of thing easier.


Footnotes

In the real world, a component/framework/utility would probably be used for listings like the one in the above example. Reporting is used as an example here primarily because of it's familiarity to most programmers. (Frameworks/utilities are available for both paradigms.) However, frameworks are often not available for many tasks, or are not geared to handle all the variations that may come along.

For example, a bill layout might be the same for all clients initially, and different for only a few later on. A framework won't help much in such a case because the complexity of the framework is not worth a few spot differences. Report frameworks tend to be helpful if there are many instances of variation (such as a different layout per client) and the framework builders anticipated or gave custom control over the spots that will actually differ.

Even if you find/make a report framework, a good many non-report-related variations are rather random and spotty. It is not rational to build a large framework on top of every possible area of variation unless there are dozens or hundreds of variations to manage at that spot. One-off or few-off spots of variations are common in my experience and to extrapolate what a report framework can do to all or most of these spots is generally futile. It would result in a massively bloated and over-engineered application. Perhaps a reporting/list example is a bad example choice on my part because of its association with frameworks.

In the OO approach described above, there seems to be a tradeoff between the size of the methods and implementation factoring. The smaller the methods, the less duplication of implementation. However, it does result in more methods. Thus, it can be said that factoring implementation better results in less header factoring, or header proliferation.



Main | OOSC2
© Copyright 2000 by Findy Services and B. Jacobs