{"id":12260,"date":"2024-07-12T13:54:45","date_gmt":"2024-07-12T19:54:45","guid":{"rendered":"https:\/\/www.mrc-productivity.com\/techblog\/?post_type=ht_kb&p=12260"},"modified":"2024-07-12T14:33:18","modified_gmt":"2024-07-12T20:33:18","slug":"working-with-duplicate-data-in-reports","status":"publish","type":"ht_kb","link":"https:\/\/www.mrc-productivity.com\/techblog\/?ht_kb=working-with-duplicate-data-in-reports","title":{"rendered":"Working with Duplicate Data in Reports"},"content":{"rendered":"\n

Overview<\/h2>\n\n\n\n

As a consultant, one of the most common scenarios I see is customers who have datasets with one-to-many relationships. <\/p>\n\n\n\n

To understand what a one-to-many-relationship is, imagine a m-Power Regular Report template with an order header table that is joined to an order line items table. At runtime, the report prints a row for every line item in the order, meaning the fields from the header record will be repeated (duplicated).<\/p>\n\n\n\n

Often times developers will opt to show or hide repeating data, and this is sufficient for their reporting needs. But what happens when a repeated value is numeric <\/em>and needs to be counted only once for further subtotaling in the report? <\/p>\n\n\n\n

This techblog post will address that question and teach you how to address this situation via Freemarker.<\/p>\n\n\n\n

If you are not familiar with Apache Freemarker, it is strongly <\/strong>encouraged to first read this documentation here<\/a> to understand basic syntax and usage. <\/p>\n\n\n\n

Example Report<\/h2>\n\n\n\n

Observe the following tables, which have a one-to-many relationship.<\/p>\n\n\n\n

One header record in the ORDERHEAD table…<\/p>\n\n\n

\n
\"\"\/\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button>
Figure 1. The first 10 rows of the Order Header table<\/figcaption><\/figure>\n<\/div>\n\n\n

Is linked to multiple records in the ORDERDET table…<\/p>\n\n\n

\n
\"\"\/\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button>
Figure 2. The first 10 rows of the Order Details table<\/figcaption><\/figure>\n<\/div>\n\n\n

A report built over these two tables would net a result similar to what is shown in Figure 3. The report has the detail level, two subtotal levels, and a grand total (not shown).<\/p>\n\n\n\n

\"\"\/\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button><\/figure>\n\n\n\n

The report is structured as such:

Detail Level
Subtotal Level 2 – Order Number (ONUM)
Subtotal Level 1 – Order Type (OTYPE)
Grand Total<\/p>\n\n\n\n

Observe the Order Amount column, which is a value from the header table. The amount is naturally repeated for any order with more than one line item and subsequently summed on the subtotal lines of the report.<\/p>\n\n\n\n

Using Order #300158 as an example, the order amount to print on the subtotal by order number line is $4,471. But because there are two line items for this order, the order amount sums to $9,482. <\/p>\n\n\n\n

To appropriate show $4,471 on the order subtotal, first we’ll go the Field Settings of the data model and specify the Order Amount’s aggregate option as “No Subtotal”. Once done, rebuild the report. <\/p>\n\n\n\n

\"\"\/<\/figure>\n\n\n\n

While this appropriately gives the right amount wanted on the Total for #300158 and #301506 lines (Subtotal Level 2), it does not give the desired total on the “Total for EDI Orders” line (Subtotal Level 1). <\/p>\n\n\n\n

This is because as a “No Subtotal” value, the report just prints the order amount from the very last line item’s order amount which is $25,843.<\/p>\n\n\n\n

\"\"\/\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button><\/figure>\n\n\n\n

The total amount for EDI orders ($30,584) will need to be summed behind the scenes and printed in that cell. <\/p>\n\n\n\n

$4,471 + $ 25,843 = $30,584<\/strong>. <\/p>\n\n\n\n

Using Freemarker, we will be able to do the summation in a custom variable. <\/p>\n\n\n\n

The idea here will be that every time we reach a subtotal by Order Number (subtotal level 2), we will add that order’s amount to the previous total. Once we reach the subtotal by Order Type (subtotal level 1) the summed value can be printed. The report will repeat this process until all the different orders subtotals lines and the order type subtotal lines have printed.<\/p>\n\n\n\n

Steps<\/h2>\n\n\n\n

1. In the HTML source of the report, find the <!-- <#list report as section> --><\/code> comment. This is where the data table execution begins.<\/p>\n\n\n\n

2. Create a custom Freemarker variable above this comment, that will store the summed amount for subtotal level 1. In this example, that variable is called s1_amount<\/code><\/strong> and is initialized to a value of 0.<\/p>\n\n\n\n

<!-- END_SORT_TABLE_HEADER -->\n<!-- END_REPORT_TABLE_HEADER -->\n<!--<#assign s1_amount = 0 >-->\n<\/strong><!-- <#list report as section> -->\n<!-- <#assign lastRow = section.lastDetail> -->\n<!-- <#assign noSubtotal = section.lastDetail> --><\/code><\/pre>\n\n\n\n
3. Further down in the HTML source, find the subtotal level 2 section. In this block add another assign statement that will add the current order amount (lastRow.OAMOUNT<\/code>) to the current value of s1_amount<\/code>.<\/p>\n\n\n\n
<!-- <#assign subTotal2 = section.subtotals[2]> -->\n<!-- <#if subTotal2.dsp != 0> -->\n<!--<#assign s1_amount = s1_amount + lastRow.OAMOUNT_o >--><\/strong>\n  <tr id=\"level_[ONUM]\" class=\"level-row\">\n<td class=\"rsd\" align=\"center\" colspan=\"5\">Subtotal Level 2 by [ONUM]<\/td>\n  <\/tr>\n  <tr class=\"sub2Background\" data-level=\"2\">\n    <td class=\"s2 td_left\"><\/td>\n    <td class=\"s2 td_right\">Total for #${lastRow.ONUM?html}<\/td>\n    <td nowrap=\"nowrap\" class=\"\"><\/td>\n    <td class=\"s2\">&nbsp;<\/td>\n    <td class=\"s2 text-right\">${lastRow.OAMOUNT?html}<\/td>\n  <\/tr>\n<!--<\/#if> --><\/code><\/pre>\n\n\n\n
Each time the report reaches a Subtotal Level 2 line it will loop through the above block of code and add the current subtotal line’s order amount (lastRow.OAMOUNT_o<\/code>) to the previously stored total. <\/p>\n\n\n\n
Once each Subtotal Level 2 line has been looped through, the final total of ${s1_amount} <\/code>will known and can be printed on Subtotal Level 1.<\/p>\n\n\n\n
4. To reference a custom Freemarker variable, simply use the ${variablename}<\/code>in the HTML source, where ‘variablename’ is whatever name given to your custom variable. <\/p>\n\n\n\n
<td class=\"s1 text-right\">${s1_amount}<\/strong><\/td><\/code><\/pre>\n\n\n\n
From the m-Painter GUI, I can easily insert that value as well:<\/p>\n\n\n\n
\"\"\/\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button><\/figure>\n\n\n\n
5. While back in the HTML source, edit the Subtotal Level 1 section code. After <\/strong>the s1_amount <\/code>has printed in the report, the value of s1_amount <\/code>will need to be initialized back to 0. <\/p>\n\n\n\n
<!-- <#assign subTotal1 = section.subtotals[1]> -->\n<!-- <#if subTotal1.dsp != 0> -->\n<tr id=\"level_[OTYPE]\" class=\"level-row\">\n  <td class=\"rsd\" align=\"center\" colspan=\"5\">Subtotal Level 1 by [OTYPE]<\/td>\n<\/tr>\n<tr class=\"sub1Background\" data-level=\"1\">\n  <td class=\"s1 td_left\">Total for&nbsp;${lastRow.OTYPE?html}&nbsp;orders<\/td>\n  <td class=\"s1 td_right\">&nbsp;<\/td>\n  <td nowrap=\"nowrap\"><\/td>\n\n  <td class=\"s1\">&nbsp;<\/td>\n  <td class=\"s1 text-right\">${s1_amount}<\/strong><\/td>\n<\/tr>\n<!--<#assign s1_amount = 0 >-->\n<\/strong><!--<\/#if> --><\/code><\/pre>\n\n\n\n
5. Save m-Painter. <\/p>\n\n\n\n
6. The grand total value will also need to be computed in a similar fashion. This time, a new variable named grand_amount<\/code> <\/strong>will be created, right after the s1_amount<\/code> variable. <\/p>\n\n\n\n
<!-- END_SORT_TABLE_HEADER -->\n<!-- END_REPORT_TABLE_HEADER -->\n<!--<#assign s1_amount = 0 >-->\n<!--<#assign grand_amount = 0 >-->\n<\/strong><!-- <#list report as section> -->\n<!-- <#assign lastRow = section.lastDetail> -->\n<!-- <#assign noSubtotal = section.lastDetail> --><\/code><\/pre>\n\n\n\n
7. Further down in the HTML source, find the subtotal level 1 section again. In this block add another assign statement that will add the subtotal’s amount (s1_amount)<\/code> to the current total (grand_amount<\/code>).<\/p>\n\n\n\n
<!-- <#assign subTotal1 = section.subtotals[1]> -->\n<!-- <#if subTotal1.dsp != 0> -->\n<!--<#assign grand_amount = grand_amount + s1_amount >--><\/strong>\n<tr id=\"level_[OTYPE]\" class=\"level-row\">\n  <td class=\"rsd\" align=\"center\" colspan=\"5\">Subtotal Level 1 by [OTYPE]<\/td>\n<\/tr>\n<tr class=\"sub1Background\" data-level=\"1\">\n  <td class=\"s1 td_left\">Total for&nbsp;${lastRow.OTYPE?html}&nbsp;orders<\/td>\n  <td class=\"s1 td_right\">&nbsp;<\/td>\n  <td nowrap=\"nowrap\"><\/td>\n  <td class=\"s1\">&nbsp;<\/td>\n  <td class=\"s1 text-right\">&nbsp;${s1_amount}<\/td>\n<\/tr>\n<!--<#assign s1_amount = 0 >-->\n<!--<\/#if> --><\/code><\/pre>\n\n\n\n
Each time the report reaches a Subtotal Level 1 line it will loop through the above block of code and add the current subtotal line’s order amount to the previously stored total. <\/p>\n\n\n\n
Once each Subtotal Level 1 line has been looped through, the final total of ${grand_amount}<\/code> will known and can be printed on the Grand Total level.<\/p>\n\n\n\n
8. Reference ${grand_amount}<\/code> on the Grand Total level of the report.<\/p>\n\n\n\n
\"\"\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button><\/figure>\n\n\n\n
<td class=\"grand text-right\">${grand_amount}<\/strong><\/td><\/code><\/pre>\n\n\n\n
Save m-Painter again. <\/p>\n\n\n\n
Results<\/h2>\n\n\n\n
Here I will show the report output. The first picture shows the report ran with details:<\/p>\n\n\n\n
\"\"\/\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button><\/figure>\n\n\n\n
The two EDI type orders total $30,584<\/strong> ($4,471+$25,843 = $30,584)<\/p>\n\n\n\n
The two FAX type orders total $27,958<\/strong>. ($27,003 + 954 = $27,958).<\/p>\n\n\n\n
Now, here is the same report without the Detail Level included:<\/p>\n\n\n\n
\"\"\/\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button><\/figure>\n\n\n\n
The Grand Total across all order types is $124,083<\/strong> ($30,584 + $27,958 + $36,883 + $28,656).<\/p>\n\n\n\n
<\/p>\n","protected":false},"excerpt":{"rendered":"
Learn how to use Freemarker to handle subtotaling in your report output. This example will walkthrough a report with a one-to-many join, and show how to specifically count duplicate detail records only one time. <\/p>\n","protected":false},"author":12,"comment_status":"closed","ping_status":"closed","template":"","format":"standard","meta":{"footnotes":""},"ht-kb-category":[158],"ht-kb-tag":[],"class_list":["post-12260","ht_kb","type-ht_kb","status-publish","format-standard","hentry","ht_kb_category-customcode"],"_links":{"self":[{"href":"https:\/\/www.mrc-productivity.com\/techblog\/index.php?rest_route=\/wp\/v2\/ht-kb\/12260","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.mrc-productivity.com\/techblog\/index.php?rest_route=\/wp\/v2\/ht-kb"}],"about":[{"href":"https:\/\/www.mrc-productivity.com\/techblog\/index.php?rest_route=\/wp\/v2\/types\/ht_kb"}],"author":[{"embeddable":true,"href":"https:\/\/www.mrc-productivity.com\/techblog\/index.php?rest_route=\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/www.mrc-productivity.com\/techblog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=12260"}],"version-history":[{"count":39,"href":"https:\/\/www.mrc-productivity.com\/techblog\/index.php?rest_route=\/wp\/v2\/ht-kb\/12260\/revisions"}],"predecessor-version":[{"id":12306,"href":"https:\/\/www.mrc-productivity.com\/techblog\/index.php?rest_route=\/wp\/v2\/ht-kb\/12260\/revisions\/12306"}],"wp:attachment":[{"href":"https:\/\/www.mrc-productivity.com\/techblog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=12260"}],"wp:term":[{"taxonomy":"ht_kb_category","embeddable":true,"href":"https:\/\/www.mrc-productivity.com\/techblog\/index.php?rest_route=%2Fwp%2Fv2%2Fht-kb-category&post=12260"},{"taxonomy":"ht_kb_tag","embeddable":true,"href":"https:\/\/www.mrc-productivity.com\/techblog\/index.php?rest_route=%2Fwp%2Fv2%2Fht-kb-tag&post=12260"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}