DOC4GL - 4gl programs documenter

Users Manual

Table of contents

Doc4GL documentation index page

About this document

This users manual is part of "DOC4GL" documentation set. Please see Doc4GL documentation index page for other related documents.

This User Manual is intended for person that will use already installed and configured DOC4GL tools.

Introduction

Background

During the software development life cycle (SDLC) of a project , the documentation that associates the implementation in 4gl code to the functionality becomes rapidly unsynchronized. This fact occurs due to the form and place where this normally is stored.

The designer usually have to construct a document to define the functions, it's return values and parameters. It is common to use CASE tools such as the "Power Designer", putting inside of the processes definition the documentation descriptions of the program to be built.

In the initial phase of project development (if the manager was careful), it is probable that the documentation reflects the written code. When SDLC enters a phase of the most advanced processes, normally immediately after the first usable version, the chaos starts. The programmer has to make decisions, and the designer or manager does not have time to control everything. Thus the source code gets modified, but as the programmers by definition don't like to work on documentation, documentation is immediately unsynchronized and obsolete. The more time passes, more unsynchronized it is. Until it becomes useless.

It would be easy to blame the programmers, however, the documentation is usually hidden somewhere on the server, in a obscure directory inside a document (usually in .doc format ) normally with many pages. The context change is such, that it's not easy to demand such an effort. The approaches initiated with some work, as for example literate programming are in this respect recently more used.

The Java programming language is paradigmatic: The reference documentation of all API's is created by automatic processes, with information contained in source code. With this integration, we can easily note this improvements:

Back to Table of Contents

About DOC4GL

This tool gives 4gl programmers functionality already present in Java (with Javadoc) or in the C/C++ (with projects as DoxyGen):

Obviously, the programmers and project managers have to enforce the practice of amending the embedded documentation during the modifications to the programs. Also, they must manually define some patterns to the comments that will be inserted in the programs source code.

DOC4GL set of tools, sometimes also referred to as "fgldocumenter", or "fgldoc", where originally developed by Sergio Alexandre Ferreira for MoreData, and donated to Aubit 4gl project under GNU GPL license. It is now developed, maintained and supported as part of Aubit 4gl project.

Back to Table of Contents

For the impatient

For users not inclined to reading complete manual, some typical sequences for installation and execution of the tool are presented in this document. However, it is recommended for users that will perform installation and configuration tasks, to read other documents in DOC4GL documentation set, in particular the Technical Manual.

Back to Table of Contents

Compilation

Please see Installation and Compilation instructions for details.

Back to Table of Contents

Installation

Please see Installation and Compilation instructions for details.

Back to Table of Contents

Execution

Here are few typical examples of execution with non-graphical User Interface, called Command line User Interface, or Character User Interface (CUI).

To import information from 4GL source code to repository (normalized):

$ fgldoc --ui=CUI --action=importFgl --import_directory=/tmp --load_recursive

To import information from CSV (Comma Separated Values) file to repository (normalized or not normalized):

$ fgldoc --ui=CUI --action=importCSV --sheet_file=/tmp/comments.csv --load_recursive --normalize

To export from repository to 4gl source fiels:

$ fgldoc --ui=CUI --action=export2Fgl --sheet_file=<fileName> --export_directory=<directoryName>

To export from repository to CSV (Comma Separated Values) file:

$ fgldoc --ui=CUI --action=export2Csv --export_directory = <directoryName> --export_source=<fglSourceName.4gl>

To export form repository to HTML formatted file:

$ fgldoc --ui=CUI --action=export2Html --export_html_directory=<directory_name>

 

Back to Table of Contents

Components

"fgldoc" - Interface for execution of documentation commands

This component (program) is used to execute actions provided by 4gl documenter. The majority of the documentation functions are executed via this program.

It is written in Perl programming language, and accepts following execution options:

Back to Table of Contents

CUI  user interface

CUI stands for Command line User Interface. It's primarily used for the batch execution.

It is activated with option --ui=CUI.

Since there will be no interaction with the users, all necessary options must be supplied by command line parameters, or be available in the configuration file (normally ".fgldocrc").

Back to Table of Contents

GUI Interface

GUI interface was created using "Tk" Perl package. Therefore, Perl "Tk" module has to be installed.

Doc4GL-GUI.jpg (46933 bytes)

Click on the image to see screenshot of Doc4GL GUI interface in action, with main,  options and Import configuration windows open.

A common mistake is to attempt execution in GUI mode, from terminal emulator window, without defining environment variable "$DISPLAY" to associated X GUI display. Please set your DISPLAY variable to appropriate value.

Following are GUI interface menu options:

File -> Open

Opens and reads the file that contains execution options for the tasks.

By default, this file is "~/.fgldocrc".

File -> Save

Saves the current execution options in the configuration file (normally "~/.fgldocrc").

File -> Exit

Saves the configuration and exits the program.

Import -> CSV

Imports comments from CSV file (Character Separated) to the database repository in relational format.

Import is performed in two phases:

  • Load of the file into database table with the same structure.
  • Normalize the information for the defined database relational model.
You can specify:
  • File  to load:   Full name of the file to load, using File Chooser. Default extension is "txt".
  • Clean the repository: Not used
  • Normalization: When de-activated, second phase is not executed.
  • Logging: When activated, each loaded string is sent to log file.

If the user selects Cancel button, or close the window, the operation is not executed, and focus is returned to the main menu.

The file must have as separator the <TAB> character between columns, and <Enter> between lines.

It must contain 7 columns whose information must be in this order:

  • moduleName: Name of the 4gl module.
  • idProcess: Process code same as in p4gl_process table
  • functionName: Name of the function
  • parametro: Parameter
  • return: Description of the return values.
  • call  : (FIXME: ????)
  • comments: Text comment for the function

Import -> 4gl Sources

Imports the available information and comments from 4gl source code.

The information to provide in the window is:

  • Name of package: Name of package that 4gl modules will be associated with. For now, used only to differentiate modules.
  • Location of the directory: Full name of the directory under which 4gl sources to parse are available.
  • Stack of modules: Full stack of the 4gl modules that it is to be parsed.
  • Load comments: Load the fgldoc comments in the repository.
  • Parse Only: Test if parsing is done successfully, do not load any information in the database.
  • Repository: Information necessary to establish the connection with the database.
  • Logging of database table use: If activated, information on table usage is loaded in the repository.
  • Load parameters: If activated, indicates that parser must load the parameters of the functions in the repository.
  • Load Strings: If activated, specify strings that will be loaded in the repository. Not used.
  • Load local variables: If activated, indicates that the local variables will be loaded in the repository.
  • Recursive: If activated, existing files with 4gl extension will be added to the stack of all 4gl files, in all sub-directories below the one specified.

If the user press  the Cancel button, or close the window, the operation is not executed and focus is returned to the main menu.

Export -> CSV

Exports all the information available in the repository, to a text file with character separator (CSV).

It takes the normalized data and inserts it in the de-normalized table p4gl_excel, before writing to the file.

The interface available in this version presents only one file to choose, the file and location where it is to be created..

If the user press  the Cancel  button, or close the window, the operation is not executed and focus is return to the main menu.

Export -> 4gl Sources

Inserts the information available in the repository, in the 4gl source files as fgldoc comments.

The module comment is inserted at the beginning of the file.

The comments for each function are inserted just before its implementation in the source code.

It allows to configure the following options:

  • Directory : Directory name where the 4gl source to insert the comments are located.
  • 4gl module  : Presents all the 4gl files that are in the chosen directory so the user can chose the one where to insert the comments.
  • fgldoc comment replacement : If activated, it substitutes fgldoc comments that exist in the file, otherwise  leaves them and creates a new one.

If the user press  the <Cancel>  button, or close the window, the operation is not executed, and focus is return to the main menu.

Export -> HTML

Although web browser interface written in JSP is also available, and can be used to browse on-line the information stored in the repository,  some times is useful to be able to access that information directly from pure HTML files, and not to be dependent of the complex infrastructure with RDBMS, web server and  JSP engine.

Doc4GL-HTML.jpg (66049 bytes) 

This option allows to generate a fully linked HTML document structure, and allows user to browse it off-line, using only web browse, directly form the file system.

This way we can write the information to CD-ROM, or store it in one compressed archive file (for example, ".tgz") that can be sent by e-mail. It exports all data available in repository.

If the user press  the <Cancel>  button, or close the window, the operation is not executed, and control is returned to the main menu.

Options -> Show Log

It shows of hides log window during the program execution.

Options - > Configure

Allows to configure the program execution options, namely:
  • 4gl doc repository: Information needed for establishing the database connection.
  • "syspgm" repository: Not used at the moment.
  • CSV import file: Name of the default character separated file (CSV) used to load repository.
  • XML file: Not used at the moment.
  • CSV Export file: Name of the default file name to receive data for export actions.
  • XML export file: Not used at the moment.

Back to Table of Contents

Web browser interface

Build in JSP (Java Server Pages) server-side scripting language, this component allows on-line access to the  information in repository, from an html browser.

Doc4GL-JSP.jpg (66747 bytes) 

Screen (web page)  is divided in tree windows (frames):

The business processes frame presents the list of processes that exists in a table, with 4gl functions associated. Each one of these processes is a link that opens the window (page in the lower left HTML page frame) that will show the list of modules. There is an option to select all the processes.

On the basis of the selected (or all) process, list is displayed of 4gl modules that contains functions associated to a process (with tag @process). Each  module is a link that opens in frame comments of a module. It will have an option to show to all, in accordance with the selected process.

The comments of a module (or all of them) presents a summarized list of the functions of that module, followed later with more detailed information existing in the repository, namely:

Back to Table of Contents

Working with DOC4GL

Configuration file

The configuration file is used to configure execution actions, options and needed data.

This file can be manually created or recorded using the available options when fgldoc  is executed (in GUI or CUI user interface). By default, it's name is "fgldocrc" and is kept in the user home directory ("~" or $HOME). In this version, the file contains Perl variables assignment  (kind of object serialization).

The variables available to configure tools actions are:

Following is an example of automatically generated configuration file:

 

$csvImportFile='/usr/home/sergio/MOB.txt';
$xmlImportFile='4gl_mapping.xml';
$UI='GUI';
$action='export2Fgl';
$importInformixDir='/usr/informix';
$importInformixServer='unstable';
$importHost='debian';
$importDatabase='p4gl_repository';
$importUser='sergio';
$exportInformixDir=undef;
$exportInformixServer=undef;
$exportHost='debian';
$exportDatabase='p4gl_repository';
$exportUser='sergio';
$csvExportFile='4gl_mapping.txt';
$xmlExportFile='4gl_mapping.xml';
$clearRepositoryOnImportCSV=undef;
$normalize='1';
$clearRepositoryOnImportXML='false';
$loadComments=undef;
$parseOnly=0;
$loadTableUsage=undef;
$loadParameters=1;
$loadStrings=0;
$loadLocalVariables=0;
$sourceDir='/usr/home/sergio/tmp/src/AMARELA/4gl';
$replaceFgldoc=0;
$fglSource='BdAmrCliente.4gl';
$logShowed='1';
$importFglLocation='/usr/home/sergio/AMARELA/src/4gl';
$loadFilesRecursive='1';= '1';

Back to Table of Contents

Parser and Loader

This executable program does the file parsing (with lexical and syntactic analysis) of the 4gl source file and all the DOC4GL special embedded comments. The name of the executable is "p4gl". It allows the following execution options:
This executable is the core component of  DOC4GL tool. When a project is well organized and structured, the functional documentation will be made by the programmers in each source program, and loaded to the repository using this tool.

The parser is implemented with the reserved word limitation, that some 4gl compilers like Informix and 4js do not have. Because of that, the source code loaded should not contain reserved words as name of identifiers (example: you cant using display as variable name).

If this happens, and you intend to load the information from these programs, you can use one special comment that when inserted, makes parser ignore the 4gl code inserted between it. This  special comment is :

{IGNORE}

...and ignores the code until it finds a new equal comment. For example:

...
{IGNORE}
DEFINE display SMALLINT
(IGNORE}
...

If you try to load the information with reserved words without using special  IGNORE comment, the parser will report syntax error.

Click here to see example of HTML module documentation generated using p4gl -c switch.

Back to Table of Contents

Repository

The repository is build in a fully normalized database, and is supported only by an Informix RDBMS, at the moment. We are working to have in a new version the ability to use other RDBMS such as PostgreSQL or MySQL.

In the technical manual we present more detailed information about the data model.

Back to Table of Contents

Documenting existing project

The goal of development for this tool was the need to know more about 4gl projects, to enable   modifications to that source code. In the process, we gain experience with this kind of work. Given the characteristics of the DOC4GL tool, we can say that this allows us to document a 4gl project in an iterative and evolutionary form.

This happens because it is possible to do imports and exports in both directions, in several information formats, allowing as to modify it in this formats, and loaded it again (CSV, RDBMS, 4gl) and still be able to convert it in another format (with the exception of the generated HTML, that is unidirectional).

To document a 4gl project, we could follow this steps:

  1. Load a first version of 4gl information that originates in 4gl modules, using the import from 4gl sources to the database repository. After this action, all the modules, functions and parameters should be kept in the repository.
  2. Generate a CSV file with this loaded information.
  3. Edit the information in Excel, or other spreadsheet application.
  4. Update the repository information, with the one edited in the spreadsheet.
  5. Export the information to 4gl source modules.
  6. Modify the comments in the 4gl source modules.
  7. Load the information from 4gl sources as needed.

Back to Table of Contents

Documenting new project

If we are starting with a new 4gl project, the development team should be strictly required to insert the DOC4GL comments during the programming work. This would prevented unnecessary modifications, and simplify the maintenance.

If used, the list of TO-DO tasks to be performed, can be an important aid in the development process. Programmer can enter this TO-DO tasks in the source code that he is working on, without then necessity to change to another environment. He should describe tasks witch remain for later implementation, since often during the development, we still remember details that we can not implement at that time, for various reasons.

Back to Table of Contents

Future versions

Current goals of Aubit 4gl development team for DOC4GL package are:

Manual:

Known bugs:

Back to Table of Contents

Support

Support for this package is provided under OpenSource development model, on the Aubit 4gl project mailing list. Please see Aubit 4gl project home and project pages for details.

Compatibility

Compatibility level, regarding parsing of 4GL language code, is dependency of the used parser. At the moment, this is fully compatible with Informix-4gl syntax up to version 7.2, version 7.3 was not tested. Aubit, 4Js and Plexus extensions to 4gl language will have unpredictable results. We intend to replace parser used in DOC4GL with Aubit 4gl parser in the future.

License

This software is released under GNU General Public License (GPL)

Developers

On-line API documentation for all Doc4GL components.

Doc4GL documentation index page  Aubit 4gl project  Aubit 4gl compiler documentation menu