Current (typical) solution
The typical and most widely used solution of constructing the "InputDocument" is to develop units which know how to 1) access given data sources and 2) turn the *raw* data into XML structured string. Then, there is the final unit which consolidates all the elementary strings and compose them into the resulting "InputDocument".
Figure 3.5: Data Flow Diagram: existing solution of construction of the "InputDocument" consisting of elements residing in various data sources
Once the "InputDocument" exists in a string form, the processing continues as described earlier, see figure 3.5.
The disadvantages
The obvious disadvantages of such a solution are:
High memory consumption: caused by the following factors:
Data duplication: the data is duplicated because it is being transformed (copied) from its original location (string on the heap, database resultset cache, object heap, file stream buffer) into new one containing the required XML format (in the figure, it is the set of "sub-element"s exiting the "Convert Unit"s).
Next increase of the memory consumption is caused by the "Consolidation Unit" which consolidates the individual "sub-element"s into the resulting "root element", which creates another copy of all the elements in memory.
Post-processing: there could be another data duplicity based on a technique used for processing the string input into "XSLT engine" (stream cache, etc).
Data volume growth by the XML tags: the resulting XML string contains the content data as well as the metadata --- the XML tags. The metadata information significantly increases the length of the resulting string (could be 5-50%).
All in memory at once: since the "root element" need to be processed (parsed) by other external component (parser) than the string must be completely constructed before any further processing starts. This does not have effect on higher memory consumption but it requires frequent allocation and deallocation of large chunks of memory (the memory size could be large especially when accessing large data volume from a database).
Large CPU load: is cased by these factors:
All in memory at once: this item is shared with the previous ("High memory consumption") section. Since this approach requires large chunks of memory needed to be frequently allocated and deallocated, then this has higher demands on the memory management system (memory swapping, etc) which in turn causes higher CPU load.
Necessity to parse the XML tags: is an obvious CPU load increase. All parsers, even the very optimized ones, are performance bottle necks.
Cumbersome design: because all the "Conversion Unit"s are custom built to transform the source data into XML format, it is more difficult to build a standard components intended for such a transformation.
The implementation also tends to produce "bad code" because all they do is constructing a string containing the XML tags mixed with the content. Therefore there is a mix of the data retrieval "logic" and the requested XML structure formating "logic".