Preface
The segment edit/compression exit facility of DL/1 enables the user to specify INFOPAK as the compression routine for segments during their movement between the application program I/O area and the Data Base buffer pool.
Segment compression/expansion routines are supported by DL/1 for variable length segments in HDAM and HIDAM data bases only. If the segments that you want to compress are fixed length, they can be easily changed to variable length by the addition of a 2 byte length field at the front of the data portion of the segment.
Only variable-length segments, which contain their data lengths in the first two bytes of the segments themselves, can be processed by the compression routine. The same compression routine can be used for different segment types in the same, or different, databases.
The routine to be used for segment length modification is named by the operand COMPRTN in the SEGM statement in the DBDGEN operation for the data base. This routine is brought into storage from a DOS/VS core image library by a DOS/VS LOAD macro of the DL/1 initialization routine. Multiple specifications of the same compression routine result in its beings loaded only once. INFOPAK is therefore reusable and reentry for online use.
Any physically defined variable-length segment of a HIDAM or HDAM database may be specified by the DBDGEN process as being compressible. It is invalid for INDEX, HISAM, and HSAM databases, and must not be specified for a logical child segment or a secondary index source segment.
The operation of the compression routine is transparent to the application programs. After INFOPAK is installed and the database is compressed, no further maintenance is necessary. INFOPAK does not use compression tables, but rather is a reentry module that performs data compression through an algorithm independent of the type of data contained in each segment.
Because INFOPAK uses the DL/1 compression exit facility, all DL/1 utilities will function with no problems. During a normal UNLOAD cycle, the segments will be decompressed and then compressed during RELOAD.
Disk copy utilities not using DL/1 services can copy the database in a compressed mode with a savings in time proportional to the actual compression percentage.
Keys are not compressed in the DOS version of INFOPAK.
Program Installation
To install the INFOPAK programs requires only one program step to load the object modules to your core image library.
The installation tape is a 1600 BPI labeled tape and is produced using CSERV.
The tape contains six object load modules; TESTPAK, TESTPAK2 INFOPAK (INFCMP00 and INFCMPPE) and two utility programs, UNLDPAK and RELDPAK for special handling of fixed length segments.
Prototype JCL to install the six modules is furnished below. You will have to modify according to your installation standards. The six phases will take about 100 blocks of storage in your core image library.
Sample JCL to load programs to your library :
// JOB INSTALL
// MTC REW,180
// ASSGN SYSIPT,X\"180\",VOL=INFxxx
// TLBL SYSIPT,\"INFO.DOS.TESTPAK\"
// LIBDEF CL,TO=yyyyy (yyyyy= your CIL name)
// OPTION CATAL
INCLUDE
INCLUDE
// EXEC LNKEDT
/*
/&
USING TESTPAK
The two trial programs TESTPAK and TESTPAK2 will quickly show the savings that INFOPAK offers. The TESTPAK and TESTPAK2 Programs are loaded to your library when you install INFOPAK.
TESTPAK uses the simple compression routine (INFCMP00) and TESTPAK2 uses the maximum compression routine (INFCMPPE). The use of TESTPAK2 is similar to that of TESTPAK, so only TESTPAK is described in this chapter.
TESTPAK is a DL/1 batch program that permits simulation of the effects of the compression routine (INFCMP00) on your database. TESTPAK actually uses the compression routine contained in INFOPAK, and in the course of sequentially reading the database, calls the routine for each segment and produces a compression report by segment type and total database.
To further insure data integrity, TESTPAK also decompresses each segment in a work area after it has been compressed, and verifies that the segment is exactly as it was before compression.
In addition to showing the savings that INFOPAK will provide, TESTPAK will also provide an idea of how much disk space to allocate for the new compressed files when they are reloaded.
note
TESTPAK is a test program and the restrictions mentioned below are just for TESTPAK and not INFOPAK.
-
Segment length over 4096 is not supported in this version of TESTPAK.
-
Physical segments are fine. Secondary indexes and relational or logical segments cause TESTPAK a problem in figuring out the relationships in the DL/1 control block and usually result in a message like "Internal error compression table". This is not a disaster, INFOPAK will process these OK, when you unload and reload the database, but TESTPAK can't handle them.
-
PL1 PSB's are not supported in this version. Change your PL1 PSB to look like a COBOL or assembler PSB and then run.
Sample JCL to execute TESTPAK
// EXEC PROC=INFO
// UPSI 00000010
// DLBL IJSYSUC,"SIPOE.USER.CATALOG",,VSAM
// ASSGN SYS040,X"351"
// DLBL DBDHCE,"INFO.DBDHCE",,VSAM,CAT=SIPOEWC
// EXTENT SYS040,SYSWK1
// EXEC DLZRRC00,SIZE=1536K
DLI,TESTPAK,PSBHCD
/*
Executing TESTPAK
The standard DL/1 batch procedure (example: DL1BATCH) is used while furnishing two parameters:
{wrapper="1" role="DL"}
-
MBR=TESTPAK :
Names the program to load under the control of DL/1.
-
PSB=XXXXXXX :
Names the PSB whose first PCB must be a reading PCB (PROCOPT = G or A) of the database that you are going to use with TESTPAK.
TESTPAK reads the database corresponding to the above PCB while working directly with the information in the DL/1 control blocks and writes a statistical output record to SYSLST.
***************************************************************************************************************************
* STATISTICS ON DATABASE COMPRESSION PAGE : 1 *
***************************************************************************************************************************
* INFOTEL (1) DATABASE NAME : FM010P *
***************************************************************************************************************************
(2) (3) (4) (5) (6) (7)
SEGMENT SEGMENT -- MIN/MAX NUMBER OF NUMBER OF BYTES NUMBER OF BYTES COMPRESSION
NAME OCCURRENCES IN OUT GAINS
P010A 60 3 180 51 71.67 %
P010B 40 / 100 6 360 180 50.00 %
P010C 30 6 180 108 40.00 %
P010D 100 0 0 0 0.00 %
TOTAL/BASE 15 720 339 52.92 %
The TESTPAK Report
TESTPAK will produce a report showing the following items: (reading left to right on the report)
-
Heading, Name of DBD of the database used. (DBDNAME of DBDGEN)
-
Name of each segment read. (SEGM of DBD)
-
Length of the segment: Fixed length segments will have one number for segment length. Variable segments will have two numbers, minimum and maximum lengths found.
-
Number of occurrences of each segment type read by TESTPAK and the total number of segments in the database.
-
Number of bytes before compression of segment type. This is the actual number of bytes that each segment type uses. Variable segments were added up as TESTPAK read them.
-
Number of bytes after compression of each segment type by INFOPAK.
-
Compression percentage for each type of segment processed and a grand total percentage for the database.
This last column is the actual percentage gain procured by INFOPAK on each type of segment and on the database itself. This allows the judging of the relative contribution of INFOPAK by the kind of segment and of the total database.
USING INFOPAK
In order to use segment compression with DL/1, the segments must be defined as variable length and DL/1 requires that the compressed database is HDAM or HIDAM. DOS INFOPAK does not compress keys.
To use INFOPAK you have to perform the following steps:
-
Modify the DBD to include INFOPAK.
-
Modify segment length.
-
Unload and reload the database.
Two compression routines are unloaded during the installation of INFOPAK:
-
The module INFCMPPE is a high compression routine that can be used to maximize compression and performance.
-
The module INFCMP00 achieves a good compression rate with a lower compression CPU consumption than INFCMPPE.
Later in this manual, all the examples will use INFCMPPE, but you can use it with INFCMP00 as well.
Modifying the DBD to include INFOPAK
Any physically defined variable-length segment of a HIDAM or HDAM database may be specified by the DBDGEN process as being compressible. It is invalid for INDEX, HISAM, or HSAM databases, and must not be specified for a logical child segment or a secondary index source segment.
The INFOPAK routine to be used for segment compression is named by the operand COMPRTN in the SEGM statement of the DBDGEN operation for the database.
{wrapper="1" role="DL"}
-
COMPRTN=INFCMPPE :
indicates that you want to compress this segment.
Example: SEGM NAME=....,BYTES=(xx,yy),COMPRTN=INFCMPPE
Modifying segment length
- Variable length segments
{wrapper="1" role="DL"}
-
BYTES=(xx,yy) :
indicates respectively the maximum and minimum length of the variable segment. If your segments are already defined as variable length, BYTES=(xx,yy) you must add 8 bytes to the maximum length size. INFOPAK uses the 8 bytes for an I/O work area. So if you already have variable length segments, just add 8 bytes to the maximum length.
Maximum length, xx = old maximum segment length + 8 bytes
(work area)
Example:
Variable length segment before modifying:
SEGM NAME=SEGVAR,PARENT=....,BYTES=(90,9),...
┌──────┬─────────┬───────────────────────────────┐
│ Size │ KEY │ data, data, data, data, data │
│ fld │ 7 bytes │ │
└──────┴─────────┴───────────────────────────────┘
Variable length segment after modifying for INFOPAK format:
SEGM NAME=SEGVAR,PARENT=....,BYTES=(98,9),COMPRTN=INFCMPPE,...
┌──────┬─────────┬───────────────────────────────┬───────────┐
│ Size │ KEY │ data, data, data, data, data │ work area │
│ fld │ 7 bytes │ │ 8 bytes │
└──────┴─────────┴───────────────────────────────┴───────────┘
- Fixed length segments -- special modifications needed
In order for INFOPAK to be used with fixed length segments (BYTES=xx), it is necessary to redefine the fixed length segments to variable length (BYTES=xx,yy) so that they can be compressed and then modify the DBD and PCB so the existing application programs continue to view the segment as fixed length for normal processing.
To modify the fixed length segment, you must shift the fixed length segment to the right 2 bytes to allow for the variable length size field which is placed at the front of the variable length segment. The size field is loaded with each segment to inform DL/1 of the length of data in the segment.
In order for the existing application programs to view the segment as it was before, we must define a new field, (to hold the old fixed length segment) which is placed immediately after the variable length counter START=3 and is equal in length to the old fixed segment.
The maximum length specified for the new variable segment will be 2 bytes for the variable length size field, plus the old fixed length, plus 8 bytes for the INFOPAK work area (specified with the parameter BYTES=(xx,yy)).
The minimum length size would be Key Length plus 2 bytes for the variable length counter.
Bytes=(xx,yy)
Maximum length, xx = 2 bytes + old segment length + 8 bytes
(size field) (work area)
Minimum length, yy = 2 bytes + Key Length
(size field)
Example:
Fixed length segment before modifying: Length = 60 Bytes
SEGM NAME=SEGFIX,PARENT=....,BYTES=60
┌─────────┬───────────────────────────────┐
│ KEY │ data, data, data, data, data │
│ 7 bytes │ │
└─────────┴───────────────────────────────┘
Fixed length segment after modifying to variable format:
SEGM NAME=SEGFIX,PARENT=\...,BYTES=(70,9),COMPRTN=INFCMPPE,\...
FIELD NAME+SEGFIXT,START=3,BYTES=60
┌──────┬─────────┬───────────────────────────────┬───────────┐
│ Size │ KEY │ data, data, data, data, data │ work area │
│ fld │ 7 bytes │ │ 8 bytes │
└──────┴─────────┴───────────────────────────────┴───────────┘
Through the concept of program SENSITIVITY, DL/1 allows you to define to the existing programs only the fields that you are concerned with or wish to access with the macro instruction SENFLD, using the parameter START=xx, where xx is the relative starting position of the field in the I/O area.
For each PCB that accesses the modified segment the macro SENSEG will have to be followed by the macro SENFLD citing the new field, using the START=3 parameter to allow for the shifting to the right of the old fixed length segment. Once these modifications have been made, the existing application programs will continue to view the segment as fixed length, and DL/1 will manage the variable portion.
Because of the change in format of the fixed segments, it is necessary to use the special unload-reload utilities that are on your installation tape to accommodate the addition of the variable length size field. The utilities are discussed on Page 10.
To summarize, review the following:
DBD before modification
DBD NAME=DBD1,ACCESS=.....
DATASET DEVICE=......,DD1=......,SIZE=......
SEGM NAME =SEGFIX1,PARENT=......,BYTES=50 --**(Fixed Length)**
SEGM NAME =SEGFIX2,PARENT=......,BYTES=60
FIELD NAME=(SEGFIX2K,SEQ,U),START=1,BYTES=5
SEGM NAME=SEGVARB,PARENT=.....,BYTES=(72,7)
PCB DBDNAME=.....,PROCOPT=.....,KEYLEN=.....
SENSEG NAME=SEGFIX1,PARENT=.....
SENSEG NAME=SEGFIX2,PARENT=.....
SENSEG NAME=SEGVARB,PARENT=.....
To compress the database, the DBD is coded:
DBD NAME=DBD1,ACCESS=......
DATASET DEVICE=.....,DD1=......,SIZE=.....
SEGM NAME=SEGFIX1,PARENT=..,BYTES=(60,9)
,COMPRTN=INFCMPPE -New Variable-
FIELD NAME=SEGFIX1T,START=3,BYTES=50 -Old Fixed-
SEGM NAME=SEGFIX2,PARENT=....,BYTES=(70,9),COMPRTN=INFCMPPE
FIELD NAME=(SEGFIX2K,SEQ,U),START=3,BYTES=5
FIELD NAME=SEGFIX2T,START=8,BYTES=55
SEGM NAME=SEGVARB,PARENT=...,BYTES=(80,7),COMPRTN=INFCMPPE
The PCB permitting access to the modified segment is coded:
PCB DBDNAME=DBD1,PROCOPT=...,KEYLEN=...,...
SENSEG NAME=SEGFIX1,PARENT=...
SENFLD NAME=SEGFIX1T
SENSEG NAME=SEGFIX2,PARENT=....
SENFLD NAME=SEGFIX2K
SENFLD NAME=SEGFIX2T
SENSEG NAME=SEGVARB,PARENT=...
UNLOADING - LOADING THE DATA BASE
This procedure consists of three steps:
-
Unloading of the data base with the old DBD
-
Utilizing AMS: IDCAMS DELETE/DEFINE CLUSTER
-
Reloading the data base with new DBD
For unloading-reloading, two cases are possible.
FIRST CASE
Your original database segments were all variable length and no segments are subject to a change in format, ie. (fixed to variable). This is the easiest case. Do the following :
-
Run the Unload program with the old DBD. Use a standard utility(DLZURGUO for example) or an application program.
-
Run AMS Utility: IDCAMS DELETE/DEFINE CLUSTER.
-
Reload using a standard utility (DLZURGLO for example) with the new DBD and a PSB containing a PCB authorizing access in the initial load (PROCOPT=L or LS) to the base if one employs an application program.
If your segments were variable length before, no more modifications are necessary. Your DL/1 application programs will continue to view the segments as they did before.
SECOND CASE : INFOTEL UTILITIES
In this case, at least one segment requires a format change, ie., is changed from fixed length to variable length.
In the event of a change of format of segments because of the appearance of a length counter, it is necessary to employ two utilities (UNLDPAK and RELDPAK), that were also loaded to your program library when the INFOPAK tape was installed. These utilities will allow for the change from fixed length segments to variable length segments on the database and allow DL/1 (using your new DBD) to take over the management of the variable length size field on the reload. On the reload, INFOPAK will place the compressed length in the size field and DL/1 will then take control and manage the variable portion. These utility programs are just for the first time load. After your compressed database is loaded, you will treat it just like any other HDAM or HIDAM database. You can then use the standard DL/1 utilities for database maintenance.
The following page shows the program flow and prototype JCL job stream to do the complete unload-reload for this second case. Both utilities will issue a "normal end" message at EOJ.
Program Flow - Unload,reload - Fixed to Variable length segments
Í─────────────────À *\-\-\-\-- UNLOADING THE DATA BASE
║ Data Base ║ // UPSI 00000010
║ (uncompressed) ║ //
Ë────────┬────────¢ // Database assigns, extents
│ //
│ //
V //
┌─────────────────┐ // ASSGN SYS001,TAPE,VOL=XXXXXX
│ Utility │ // TLBL FICUNLD
│ Program UNLDPAK │ // EXEC DLZRRCOO,SIZE=1536K
└────────┬────────┘ DLI,UNLDPAK,PSBxxx
│ /*
│ *\-\-\-\-\-\-\-- REWIND TAPE
│ *
V // UPSI 1
DTFMT = FICUNLD // ASSGN SYSnnn,SYS001
// EXEC DITTO
SYS001 \$\$DITTO REW SYSOUT=SYSnnn
│ \$\$DITTO EOJ
│ /*
V *\-\-\-\-\-- AMS UTITILITY
┌─────────────────┐ //
│ AMS Utility │ // Assigns
│ IDCAMS │ //
│ DELETE, DEFINE │ // EXEC IDCAMS,SIZE=AUTO
└────────┬────────┘ parameters IDCAMS (DELETE,DEFINE)
│ /*
│ *
V *\-\-\-\-\--RELOADING THE DATABASE
DTFMT = FICRELD // UPSI 00000010
//
SYS003 // Data base assigns, extents
│ //
│ // ASSGN SYS003,SYSnnn
V // TLBL FICRELD
┌─────────────────┐ // EXEC DLZRRCOO,SIZE=1536K
│ Utility │ DLI,RELDPAK,PSByyy
│ Program RELDPAK │ /*
└────────┬────────┘ /&
│
V
Í─────────────────À
║ Compressed ║ HIDAM or HDAM only
║ Data Base ║ **See following page for program notes**
Ë─────────────────¢
NOTES TO UNLOAD-RELOAD PROCEDURE
Unload Program UNLDPAK
The user is cautioned to insure that the PSB for UNLOADING contains a PCB sensitive to all segments and all fields of the database.
Ending message PROGRAM OF UNLOADING \-- NORMAL END
note
Before running the RELDPAK Program insure that:
IBM - PTF\'s \# UP 46554
\# UP 46555
have been applied to DL/1\...\...\...\...\...\....
RELDPAK will bomb if the PTF\'s have not been applied.
Load Program RELDPAK
Function authorized: loading initial (PROCOPT=L or LS) the PCB of reloading must allow, with the program RELDPAK, the SENFLD macros to permit DL/1 to take control of the length counter. Insure the PCB contains recognition of all the segments of the base, to all the fields of the segments not being subjected to change of format and to all the fields except the length counter for the segments passing from fixed length to variable length. During the reload, INFOPAK will place the compressed length in the size field and DL/1 will then take control and manage the variable portion.
Ending Message PROGRAM OF LOADING \-- NORMAL END