MODULE 8 · SEQUENTIAL FILES · 7/8

The match-merge master file update

20 min30 XPExercise

This is the most important algorithm in batch COBOL. For decades, banks, insurers and utilities have updated their master files every night the same way: read yesterday's old master and today's transactions, both sorted by key, and write a complete new master.

Why not just update in place?

The pattern was born on magnetic tape, where you cannot rewrite a record in the middle. But it survived onto disk because it is so robust:

The old master is never touched. If the job fails, you fix the problem and rerun it from the same inputs. (On z/OS, masters are usually generation data groups: BANK.MASTER(0) is today's, (+1) the new one.)
Records can be added and deleted, not just changed.
It reads each file once, front to back: the fastest possible access, even for millions of records.

The balance-line algorithm

Both files must be sorted by the same key. You keep one current record from each file, compare their keys, and take one of three paths:

Comparison	Meaning	Action
master key < transaction key	no (more) transactions for this master	write the master to the new file, read the next master
master key = transaction key	a transaction for this master	apply it to the master in memory, read the next transaction
master key > transaction key	a transaction with no master	report it as an error (or add a new record), read the next transaction

Notice that a matched transaction does not write the master. Reading the next transaction first means several transactions for the same account are all applied before the master finally moves on to the new file via the first path.

HIGH-VALUES at end of file

When one file runs out, the other still has records to process. The standard trick: at end of file, set that file's key to HIGH-VALUES (the highest possible value of every byte). Now every real key compares lower, so the remaining records of the other file flow through naturally, and the loop ends when both keys are HIGH-VALUES:

           PERFORM 8000-READ-MASTER
           PERFORM 8100-READ-TXN
           PERFORM UNTIL WS-MAST-KEY = HIGH-VALUES
                     AND WS-TXN-KEY  = HIGH-VALUES
               EVALUATE TRUE
                   WHEN WS-MAST-KEY < WS-TXN-KEY
                       PERFORM 1000-WRITE-MASTER
                       PERFORM 8000-READ-MASTER
                   WHEN WS-MAST-KEY = WS-TXN-KEY
                       PERFORM 2000-APPLY-TXN
                       PERFORM 8100-READ-TXN
                   WHEN OTHER
                       PERFORM 3000-UNMATCHED-TXN
                       PERFORM 8100-READ-TXN
               END-EVALUATE
           END-PERFORM

and the read paragraphs copy the key into working storage:

       8000-READ-MASTER.
           READ OLD-MASTER
               AT END MOVE HIGH-VALUES TO WS-MAST-KEY
               NOT AT END MOVE MAST-ACCT TO WS-MAST-KEY
           END-READ.

Use separate WS-...-KEY fields rather than the keys in the record areas: after AT END the record area is undefined, and it has to hold HIGH-VALUES reliably.

Tracing it

Masters A, C, D; transactions C, C, E:

Master	Txn	Path
A	C	A < C: write A
C	C	equal: apply first C
C	C	equal: apply second C
C	E	C < E: write updated C
D	E	D < E: write D
HIGH	E	HIGH > E: E has no master
HIGH	HIGH	done

Sorted input is a precondition

If either file is out of sequence, the algorithm silently produces a wrong master: transactions get rejected as "no master" and records can be duplicated. Production programs often check that each key is greater than the previous one and abend if not. The files are normally sorted by a SORT step in the JCL before this program runs.

On the job

You will see this structure in almost every shop, often decades old and extended many times over. Once you recognise the three-way key comparison, even a 5,000-line update program becomes readable: find the main loop, and everything else hangs off those three paths.

Your task

Apply today's price changes to the product master using a match-merge. All three files are line sequential and sorted by item number.

Old master, DD name OLDMAST:

Field	Picture	Positions
`OLD-ITEM`	`X(6)`	1–6
`OLD-DESC`	`X(20)`	7–26
`OLD-PRICE`	`9(5)V99`	27–33

Price changes, DD name PRICETXN:

Field	Picture	Positions
`TXN-ITEM`	`X(6)`	1–6
`TXN-PRICE`	`9(5)V99`	7–13

New master, DD name NEWMAST: same layout as the old master.

Every old master record goes to the new master, with its price replaced by the matching price change if there is one. An item can have more than one change; the last one wins. For a change with no matching master, display NO MASTER: and the item number, and carry on.

At the end display the control totals:

MASTERS:  005
TRANS:    004
APPLIED:  003
REJECTED: 001
WRITTEN:  005

The starter has the files, the counters and the two read paragraphs (which set the keys to HIGH-VALUES at end of file).

← OPEN EXTEND and REWRITE Challenge: nightly account update →

Run your program to see its output here. The first visible test's input and datasets are used.

Submit to grade your program against every test.