audit2query3k utility
---------------------

Revision: 2008-11-03

audit2query3k is a utility that reads audit files and outputs
changes in a syntax suitable for use with the query3k utility.

The audit2query3k was initially created to assist in recovering
data from forward log files and allow re-applying information
using the query3k utility.

It is derived from the read_audit_demo.c example source code
that is available on request and demonstrates how to read
Eloquence audit files.

When processing the audit file, audit2query3k reorders the 
changes. Newly added records are temporarily kept in memory
and subsequent update and delete operations on these records
are accounted for internally. The newly added records are
added at the end of the geerated output file.
For records not newley added, query3k find and replace or
delete commands are generated.

When updating or deleting records, audit2query3k uses the
record number to locate previous records.  To provide a
limited protection against re-use of a deleted record it 
attemtps to verify the search items match the expected 
values.


Installation:

The audit2query3k.tar.gz archive is a compressed tar archive
that holds the binaries for HP-UX PA-RISC and IA64.  Please
unpack it in a temprary directory:

  gzcat audit2query3k.tar.gz | tar xvf -
  
This will unpack the files audit2query3k.pa and audit2query3k.ia64
in the current directory.


Usage:

./audit2query3k -help
usage: audit2query3k [options] auditfile
options:
 -help   - show usage (this list)
 -v      - increase verbosity
 -t f:t  - filter records from:to time (specified in seconds)


How to use:

- Use the fwaudit utility to create an audit file. 

- All (relevant) forward log files should be specified to the
  fwaudit utility at once so they are merged in a common audit file.

- A separate audit file must be created for each database. It may 
  be advisable to consider creating a separate audit file for each 
  data set.

  The following commands may be used with the fwaudit utility
  to obtain a quick overview on the operations and the affected
  databases and data sets recorded in the specified forward log files.

  fwaudit -r fw*log | awk '/^DB/ {print $1 FS $2}' | sort | uniq -c

      4 DBDELETE DB1.SET1
     30 DBDELETE DB1.SET3
      3 DBPUT DB1.SET1
   2459 DBPUT DB2.SET3
    117 DBUPDATE DB1.SET3

- When the -v option is present, additional comments are output
  to the generated query3k batch file.

The example below creates an audit file DB1.SET1.audit holding all 
changes for DB1 and SET1. Then the audit2query3k utility is used to
create a query3k batch script from these changes.

  fwaudit -o DB1.SET1.audit -e "DB1.SET1" fw*log
  audit2query3k DB1.SET1.audit > DB1.SET1.q3k
  query3k < DB1.SET1.q3k


query3k syntax notes:

- The generated query3k script does not include the command to
  open the database.
- The delete command does not provide a confirmation ("yes")
  and only works in batch mode (XEQ or input redirected).
- All strings are enclosed in quotes
- New records are added with the ADD command one at a time
- Existing records are updated/deleted by using the record
  number and verifying the search items still match.
- The generated query3k scripts do not open the database.


Limitations:

- audit2query3k currently only supports the item types: I, K, X, U
- audit2query3k currently does not support any arrays (items with
  item count > 1).
- When a non-supported item type is found, audit2query3k aborts
  with an internal error message.
- The audit2query3k can only process audit files created on a
  compatible platform (same character set and byte order).
- The generated query3k script should only be used as a guideline.
  It depends on the application logic if and to what extent is
  applicable.
- The audit2query3k utility is currently only available for the 
  HP-UX PA-RISC and HP-UX IA64 platform.
- The changes that can be processed is limited to the memory that
  can be allocated (~1 GB for each invocation).