Preface
The INFOPAK IDMS product enables compression of IDMS data bases while remaining totally transparent to application programs. INFOPAK IDMS is easily installed and does not require any changes to internal IDMS programs.
-
a compression module, delivered as a load module on a magnetic tape must be linked with two compression and decompression procedures supplied, and the result stored in a load module library.
-
the RECORD DESCRIPTION of the diagram must call the compression modules prior to storing or modifying (CALL BEFORE STORE or BEFORE MODIFY) and after reading (CALL AFTER GET).
In all cases, the initial load is executed with the standard unload/reload procedures.
The operation of the compression routine is transparent to the application programs. Employing artificial intelligence techniques, INFOPAK IDMS uses high speed scanning to perform a three-dimensional analysis of data and then makes compression decisions based on the type of data. The scanning program may decide to use several different encoding methods within the data for optimum performance/compression. The primary compression technique used by INFOPAK IDMS is derived from the HUFFMAN algorithm, and enables data to be represented in the form of bit strings of variable length. Additional techniques allow up to 31 bytes of blanks or packed fields of zero to be compressed to a single byte. Naturally, when these types of fields are eventually updated, they will occupy more space. INFOPAK IDMS compresses user data only.
INFOPAK IDMS also supports the use of hardware compression feature. This option supports compression/decompression on MVS systems even if the hardware instruction is not available(simulation). The hardware compression option of INFOPAK IDMS can be activated at any time. The INFOPAK IDMS compression module is able to differentiate software compressed records from hardware compressed records allowing a database to gradually change from hardware(including simulated hardware) to software compression or vice versa. INFOPAK IDMS hardware compression is transparent to application programs.
TESTPAK
TESTPAK is a utility program that runs in pure batch mode and executes a total or partial sequential read of an IDMS data base and compresses each of the records read. TESTPAK prints reports showing the compression percentage INFOPAK will provide for each IDMS data base tested.
Using TESTPAK
The installation procedure copies the COBOL source code modules TSTPAKPA, TSTPAKE1 and TSTPAKE2 into the library referenced by the IDMSSRC ddname (ref installation procedure).
The following steps are necessary to put TESTPAK into operation:
-
Run IDMS pre-compile, compile, and link TSTPAKPA.
-
Run TSTPAKPA with the IDMS dictionary as input and a parameter card specifying the SCHEMA to analyze, producing source TSTPAKDB.
-
Create loadmod TSTPAKEV by using the standard IDMS compile procedure, pointing to the source(TSTPAKDB).
-
Run TSTPAKEV with the IDMS data base to be evaluated as input producing a compression gains report.
Running TSTPAKPA
This program accesses the IDMS dictionary and requires the following datasets:
{wrapper="1" role="DL"}
-
PAKWKCOP Output file
This file will be the complete source program that will contain the code necessary to perform the compression testing. This source will be used to create load module TSTPAKEV.
-
PAKWKSRA Output file
This file will contain information from the IDMS dictionary that TSTPAKEV needs to calculate the compression ratio according to the lengths of records and prefixes.
-
TSTPAKE1
-
TSTPAKE2
Those two files are used in input. they correspond to the members TSTPAKE1 and TSTPAKE2 unloaded under the ddname IDMSSRC during the installation process. They are used to generate the source program TSTPAKDB.
-
PAKWKSCH Input file
One parameter card coded as follows:
\-\-\--5\-\--10\-\--15\-\--20\-\-\-\-\--28
SSSSSSSSvvvvssssssssdddddddd
where:
{wrapper="1" role="DL"}
-
SSSSSSSS
is the SCHEMA name,
-
vvvv
is the version number,
-
ssssssss
is the SUB-SCHEMA name,
-
dddddddd
is the Secondary dictionary name(optional).
example
Example of TSTPAKPA :
//jobname JOB
//*
//ENTPARAM EXEC PGM=TSTPAKPA
//STEPLIB DD DSN=ERI.LOADLIB,DISP=SHR
// DD DSN=ERI.IDMS.LOADLIB,DISP=SHR
// DD DSN=SYS4.IDMS100.LOADLIB,DISP=SHR
//SYSCTL DD DSN=SYS4.IDMS.SYSCTL,DISP=SHR
//TSTPAKE1 DD DSN=????????.????????(TSTPAKE1),DISP=SHR
//TSTPAKE2 DD DSN=????????.????????(TSTPAKE2),DISP=SHR
//SYSLST DD SYSOUT=*
//SYSJRNL DD DUMMY
//SYSUDUMP DD SYSOUT=*
//SYSOUT DD SYSOUT=Y
//IDMSEDIT DD SYSOUT=*
//DBJORNAL DD DUMMY
//*
//* PLACE REQUIRED IDMS FILES HERE
//*
//PAKWKCOP DD DSN=TSTPAK.PDS.SOURCES(TSTPAKDB),
// DISP=SHR
//PAKWKSRA DD DSN=WKSRA,DISP=(NEW,KEEP,DELETE),
// DCB=(RECFM=FB,LRECL=48,BLKSIZE=4800),
// UNIT=D,SPACE=(TRK,(1,1))
//PAKWKSCH DD *
SCHEMAS40001SUB4002dictn
/*
note
If your system requires registration of programs by sub-schema, then the source (TSTPAKDB) should be registered. This may be accomplished by changing the program name to one that is authorized.
Create TSTPAKEV
Using the standard IDMS compile procedure, compile, with the NODYNAM option, and link, with the NORENT option, the source TSTPAKDB producing loadmod TSTPAKEV. The link must include INFCMRE0 to resolve entry point INFCMR00. This module is in the INFOPAK load library.
Linkedit of TSTPAKEV:
//* EXAMPLE JOB for IDMS COMPILE AND LINK of TSTPAKEV
//PRE EXEC PGM=IDMSDMLC
//SYSLST DD SYSOUT=*
//SYSCTL DD DSN= ,DISP=SHR
//SYSPCH DD DSN=&&DMLC,DISP=(,PASS),UNIT=????,
// DCB=BLKSIZE=3120,SPACE=(CYL,(1,1))
//SYSIPT DD DSN=TSTPAK.PDS.SOURCES(TSTPAKDB),DISP=SHR
//* STANDARD COMPILE AND LINK
//COB EXEC PGM=IKFCBL00,COND=(4,LT,PRE),
// PARM=\'APOST,NODYNAM,LANGLVL(1)\'
//SYSPRINT DD ????????
//SYSPUNCH DD ????????
//SYSUT1 DD UNIT=????,SPACE=(1024,(120,120))
//SYSUT2 DD UNIT=????,SPACE=(1024,(120,120))
//SYSUT3 DD UNIT=????,SPACE=(1024,(120,120))
//SYSUT4 DD UNIT=????,SPACE=(1024,(120,120))
//SYSUT5 DD UNIT=????,SPACE=(1024,(120,120))
//SYSLIB DD ???????? cob copy srclib
//SYSLIN DD DSN=&&LOADSET,DISP=(MOD,PASS),UNIT=????,
// SPACE=(3040,(40,40)),
// DCB=(BLKSIZE=3040,LRECL=80,RECFM=FB)
//SYSIN DD DSN=&&DMLC,DISP=OLD
//*
//LKED EXEC PGM=IEWL,PARM=\'LIST,XREF,MAP\'
//SYSPRINT DD SYSOUT=*
//SYSTERM DD SYSOUT=*
//SYSUT1 DD UNIT=SYSDA,SPACE=(3200,(100,100))
//SYSLIB DD DISP=SHR,DSN=INFOTEL.MVSLOAD
// DD DISP=SHR,DSN=??.VSClIB autolink lib
//SYSLIN DD DSN=&&LOADSET,DISP=OLD
// DD *
INCLUDE SYSLIB(INFCMRE0)
ENTRY TSTPAKEV
NAME TSTPAKEV(R)
/*
//SYSLMOD DD DISP=SHR,DSN=INFOTEL.TESTPAK.LOAD
Run TSTPAKEV
TSTPAKEV must be executed in the IDMS environment to access the SCHEMA analyzed.
It uses the following data sets:
{wrapper="1" role="DL"}
-
PRINT Output file
report.
-
PAKWKSRA Input file
This file contains information from the IDMS dictionary collected by TSTPAKPA.
-
PAKWKOCC Input file
The parameter cards are coded as follows:
\-\-\--+\-\-\--1-
nnnnnnnnnnn
AREA=
where:
{wrapper="1" role="DL"}
-
nnnnnnnnnnnn
is the number of records to be analyzed from each AREA. This card is not required. If omitted the whole data base will be read.
-
AREA=
optional, if absent all AREAS and records accessed by the sub-scheme will be processed, however one or more AREA cards may be presented to limit the compression test to those indicated.
Example of TSTPAKEV:
//jobname JOB
//*
//TESTPAK EXEC PGM=TSTPAKEV
//STEPLIB DD DSN=ERI.LOADLIB,DISP=SHR
// DD DSN=ERI.IDMS.LOADLIB,DISP=SHR
// DD DSN=SYS4.IDMS100.LOADLIB,DISP=SHR
//SYSCTL DD DSN=SYS4.IDMS.SYSCTL,DISP=SHR
//SYSLST DD SYSOUT=*
//SYSJRNL DD DUMMY
//SYSUDUMP DD SYSOUT=*
//SYSOUT DD SYSOUT=Y
//IDMSEDIT DD SYSOUT=*
//DBJORNAL DD DUMMY
//*
//* FILES REQUIRED OR OPTIONAL
//*
//PRINT DD SYSOUT=*
//PAKWKSRA DD DSN=WKSRA,DISP=SHR
//PAKWKOCC DD *
000000000020
AREA=TEST-AREA-001
AREA=TEST-AREA-003
/*
Twenty records will be analyzed, from each area.
TSTPAK results
TESTPAK prints a single report showing the compression gains for the SCHEMA selected.
A sample of the TESTPAK report is printed below:
**********************************************************************************************************************
* --- INFOPAK IDMS --- COMPRESSION STATISTICS PAGE : 1 *
**********************************************************************************************************************
* INFOTEL DATA BASE NAME : SCHEMAS40001 *
**********************************************************************************************************************
RECORD NAME RECORD LENGTH NUMBER OF OCCURRENCES NUMBER OF BYTES IN NUMBER OF BYTES OUT COMPRESSION GAINS
FQ4A 44 21 924 567 38.64 %
FQ4B 36 9 324 183 43.82 %
FQ4C 52 0 0 0 0.00 %
FQ4D 124 34 4,216 2,480 64.66 %
FQ4E 140 13 1,820 824 54.73 %
FQ4F 68 0 0 0 0.00 %
FQ4G 64 39 2,496 903 63.62 %
FQ40 224 35 7,840 4,975 36.54 %
FQ41 292 87 25,404 7,012 72.40 %
FQ42 36 3 108 60 44.44 %
FQ43 36 1 36 19 47.22 %
FQ44 60 1 60 16 73.33 %
FQ45 44 0 0 0 0.00 %
FQ46 264 19 5,016 586 88.32 %
FQ47 176 54 9,504 2,496 74.37 %
TOTAL BASE 316 57,748 19,071 66.98 %
|1| |2| |3| |4| |5| |6|
-
Record name (alphabetic sequence)
-
Record length (includes prefix keys!!)
-
Number of occurrences of each record
-
Number of bytes before compression, keys and data portion of record
-
Number of bytes after compression, keys and data portion of record
-
Compression gain : this is the percentage of compression savings for each record and for the entire base. Only the data to the right of the key can be compressed(IDMS rules).
\[6\] = (\[4\] - \[5\]) / \[4\]
This percent is the maximum value and does not include control information for IDMS records, free space etc...
The exact level of compression can be precisely measured by compressing an actual data base using the compression routine.
Using INFOPAK
The following steps are to be performed:
-
Create the JCL using the installation sheet as a model. Please verify that the CPU serial numbers on the installation sheet are correct.
-
Insert a JOB card in front of the deck.
-
Complete the installation sheet with the actual library names. If you are using temporary libraries, please insure that all members are copied to the final libraries when making the permanent installation.
-
Change the name SYSDA (temporary files), if necessary.
-
Mount the tape as required on a 1600 bpi tape reader.
-
Submit the job.
-
Check the return code (return code = 0).
-
If the version of IDMS is before IDMS VERSION 12 skip this step. If IDMS is at VERSION 12 or HIGHER then :
-
delete source members, INFOCOMP and INFODCOM.
-
rename member INFOCO12 to INFOCOMP.
-
rename member INFODC12 to INFODCOM
-
Assemble and Linkedit the source code modules, INFOCOMP and INFODCOM using the IDMS macro library and link with the load module INFCMRE0. The resulting two load modules are the compression(INFOCOMP) and decompression(INFODCOM) modules.
-
Assemble and Linkedit the source code modules, INFOCOMP and INFODCOM using the IDMS macro library and link with the load modules INFCMRE0 and INFFPAL0. The resulting two load modules are the compression(INFFCOMP) and decompression(INFFDCOM) modules supporting compress in place for variable length records.
-
Compile TSTPAKPA. This module is to be compiled with your standard IDMS procedure. It provides access to your IDMS dictionary and is located in the dataset referenced by ddname IDMSSRC.
-
Copy the product tape datasets to permanent disk as described in steps 1 and 2 of "Reinstallation without the product tape". These disk datasets should be excluded from migration products as they are required for CPU upgrades and Disaster Recovery procedures.
-
If the job has abended there will usually be a self explanatory message in the SYSOUT. If this message indicates that an abnormal return code was issued by a utility program, the explanation will usually be found at the end of the listing (SYSPRINT).
The most common errors are:
-
an erroneous confidential code,
-
a misspelled DDNAME or DSNAME,
-
an insufficient region size (see EXEC card),
It is normal to receive a message stating that the DD statement for SYSLIB is missing.
Activating compression
Four steps are necessary:
-
Unload the data base.
-
Modify the SCHEMA and Sub-Schemas for compression.
-
Resize the Area using TSTPAK results(%).
-
Reload the data base.
Activating compress in place
Two steps are necessary:
-
Verify that the records are variable length or use the restructure function of IDMS to change them to variable length.
-
Modify the SCHEMA for INFOPAK compression.
Linkedit
The two assembler source code modules, INFOCOMP and INFODCOM, are to be compiled with the IDMS macros library and linked with the load module INFCMRE0. The two resulting load modules are the corresponding compression and decompression modules. This step should have been completed at installation time.
Unload the data base
The standard IDMS data base unload procedure used by your installation is used to accomplish the unload of the data base.
Modify the SCHEMA
The procedure, INFOCOMP and INFODCOM, must be specified in the SCHEMA for each type of record to be compressed. Compression must be called before each DML write function (STORE, MODIFY) and decompression must be called after the read function (GET).
note
It is necessary to indicate at the SCHEMA level for fixed length records, MINIMUM ROOT LENGTH IS CONTROL LENGTH.
example ::Example: record SCHEMA description statements
STANDARD SCHEME DEFINITION USING INFOPAK COMPRESSION
ADD AREA NAME IS TEST-AREA
CALL INFOCOMP BEFORE FINISH
CALL INFODCOM BEFORE FINISH.
ADD RECORD NAME IS TEST-REC
SHARE STRU OF REC TEST-REC VERSION 1
RECORD ID IS 0001
LOCATION MODE IS
CALC USING TEST-KEY DUPLICATES ARE NOT ALLOWED
WITHIN AREA TEST-AREA
MINIMUM ROOT LENGTH IS CONTROL LENGTH
MINIMUM FRAGMENT LENGTH IS RECOR LENGTH
CALL INFOCOMP BEFORE STORE
CALL INFOCOMP BEFORE MODIFY
CALL INFODCOM AFTER GET
.
.
.
compress in place for variable length records standard definition
using INFOPAK compression
ADD AREA NAME IS TEST-AREA
CALL INFFCOMP BEFORE FINISH
CALL INFFDCOM BEFORE FINISH.
ADD RECORD NAME IS TEST-REC
SHARE STRU OF REC TEST-REC VERSION 1
RECORD ID IS 0001
LOCATION MODE IS
CALC USING TEST-KEY DUPLICATES ARE NOT ALLOWED
WITHIN AREA TEST-AREA
MINIMUM ROOT LENGTH IS CONTROL LENGTH
MINIMUM FRAGMENT LENGTH IS RECOR LENGTH
CALL INFFCOMP BEFORE STORE
CALL INFFCOMP BEFORE MODIFY
CALL INFFDCOM AFTER GET
.
.
.
CONVERT FROM IDMS COMPRESSION TO INFOPAK COMPRESSION USING RESTRUCTURE
#1: Original SCHEMA definition using IDMSCOMP
ADD AREA NAME IS TEST-AREA
CALL IDMSCOMP BEFORE FINISH
CALL IDMSDCOM BEFORE FINISH.
ADD RECORD NAME IS TEST-REC
SHARE STRU OF REC TEST-REC VERSION 1
RECORD ID IS 0001
LOCATION MODE IS
CALC USING TEST-KEY DUPLICATES ARE NOT ALLOWED
WITHIN AREA TEST-AREA
MINIMUM ROOT LENGTH IS CONTROL LENGTH
MINIMUM FRAGMENT LENGTH IS RECOR LENGTH
CALL IDMSCOMP BEFORE STORE
CALL IDMSCOMP BEFORE MODIFY
CALL IDMSDCOM AFTER GET
.
.
.
#2: INTERIM SCHEMA definition for RESTRUCTURE ONLY
TO CONVERT FROM idms COMPRESSION to INFOPAK COMPRESSION
ADD AREA NAME IS TEST-AREA
CALL INFOCOMP BEFORE FINISH
CALL IDMSDCOM BEFORE FINISH.
ADD RECORD NAME IS TEST-REC
SHARE STRU OF REC TEST-REC VERSION 1
RECORD ID IS 0001
LOCATION MODE IS
CALC USING TEST-KEY DUPLICATES ARE NOT ALLOWED
WITHIN AREA TEST-AREA
MINIMUM ROOT LENGTH IS CONTROL LENGTH
MINIMUM FRAGMENT LENGTH IS RECOR LENGTH
CALL INFOCOMP BEFORE STORE
CALL INFOCOMP BEFORE MODIFY
CALL IDMSDCOM AFTER GET
.
.
.
Reload the data base
The standard IDMS data base reload procedure used by your installation will accomplish the reload of the data base.
Removing compression
Three steps are necessary to:
-
Unload the data base.
-
Modify the SCHEMA.
-
Reload the data base.
Unload the data base
The standard IDMS data base unload procedure for your installation is used to accomplish the unload of the data base.
Modify the SCHEMA
The procedure, INFOCOMP and INFODCOM, must be removed from the SCHEMA for each type of compressed record.
example ::Example: record SCHEMA description statements
Remove the following standard compression statements
CALL INFOCOMP BEFORE STORE
CALL INFOCOMP BEFORE MODIFY
CALL INFODCOM AFTER GET
or
compress in place for variable length records
CALL INFFCOMP BEFORE STORE
CALL INFFCOMP BEFORE MODIFY
CALL INFFDCOM AFTER GET
The standard SCHEMA modification procedure used by your installation will accomplish this step.
Reload the data base
The standard IDMS data base reload procedure used by your installation will accomplish the reload of the data base.