Doc4GL documentation index page
This manual presents the DOC4GL tools from the implementation point of view. This document is part of DOC4GL documentation set. Please see Doc4GL documentation index page for other related documents.This Technical Manual is intended for person that will install, configure and maintain DOC4GL installation, and for developers that will enhance it's functionality.
This diagram details architecture of the DOC4GL tools:
In conceptual terms, it can be divided in the following components:
The repository is the place (database) where the meta-data information about 4GL programs is stored.The tool interacts with this information, importing it from 4GL source code files. It also allow exporting data in other formats. It is possible to access data on-line, through available interfaces (in Web, GUI or CUI environments) or with SQL.
The documenter is composed from modules programmed with 4GL programming languages, and DOC4GL tools. It was designed in attempt to guarantee the integrity of both, but this goal is still not fully reached.Program written in Perl serves as user interface, for the execution of the different documenter actions. It allows execution in graphical environments (GUI) or command line (CUI) without any interaction with the user (except when generating messages) facilitating batch processing.
Also implemented is set of utilities, that executes each one of the components, for example, management of errors and writing to log files. This is perhaps the ratio of the dependences, given that still did not reach one level of null dependence.
Parser and Extractor of documentation from 4gl source code
Receiving the name of 4gl source file as parameter, it performs syntactic analysis, and loads the information into memory and later storing (if corresponding option was chosen) in a relational repository inside of a database.It was build using the following tools:
It can be executed with the following options:
- Bison / Yacc
- Flex / Lex
- C language compiler
- Informix ESQL/C
p4gl [ - h ] [ - d ] [ - i ] [ - v ] [ - w level ] - f file.4gl
- -h: Display this help message
- -d: Debug mode
- -i: Insert in the repository
- -v: Verbose mode
- -w: Warning level (1-10)
- -c: Generate documentation
- -s: Standard comments will be documentation
- -p package: Name of package
Access to the repository
The access to the repository (for reading and writing) is built in Informix ESQL/C, and as such it can only access and create the repositories in Informix databases, for the moment. Access to other databases will be implemented using Aubit 4gl ODBC connection methods.Name of the database where the repository will be installed is specified using the DATABASE environment variable.
Exporter for 4gl sources
Developed in Perl under the object/class model, it reads the information from the repository, for one or more intended modules, and generates the comments in fgldoc format and inserts them inside the 4gl source files.CSV Importer
Developed using Perl with Object/Class model, attempts to open the file specified and load information to a database with exactly the same format. Optionally, later it can perform normalization of that information into normalized tables in the repository.Note that loaded information in the CSV file format is of little or no use to the user.
CSV Exporter
Reads the information from fully normalized tables in relational database, un-normalize it in the table p4gl_excel and unloads it in character separated format (CSV).HTML Generation
To be able to use this information in off-line environments, it is possible to generate a set HTML formatted files with all the repository information. This information cannot be modified or loaded, and loses the SQL access ability.This type of functionality is often used by similar documentation systems, as for example the DoxyGen or Javadoc.
Developed in JSP (Java Server Pages), it allows on-line access to the information stored about the programs, that reside in the repository. Database access is performed through JDBC.Because it is the nature of JSP that they are compiled dynamically by the respective container, there will have to be compiled and installed in the correct place, for the Web/JSP server, like TomCat.
The repository of the DOC4GL has a relational structure, with referential integrity. It constitutes the central storage of the fgldoc tool. It is, for now, implemented and tested only using Informix DBMS.
Click on the image to see picture of Doc4GL database repository model as designed in ERwin.
Click here to open Adobe Acrobat (.pdf) file containing the image of Doc4GL database repository model showing all tables and there relationships as modeled in ERwin. You can download this model as ERwin (.er1) file here.
Given the normalization of the structure (and the fact that it don't use serial nor rowid), the conversion to other RDBMS (such as PostgreSQL) should not be to complicated.
Following is the description of each table of the relational repository. We currently don't have descriptions for each column, but this is on the TO-DO list.
Summary of database tables:
p4gl_excel Image of the CSV file (Character Separated Values) used for loading with information coming from a spreadsheet. p4gl_module Name of the 4gl Source file. p4gl_package Package. Abstraction with the goal of having one entity that provide unique source file name. In most cases it corresponds to the directory (normally not fully qualified) where the source file reside. p4gl_process Processes abstraction that define the business processes implemented by the code. It could be organized in a hierarchy p4gl_function Functions implemented in the modules. p4gl_fun_parameter Function parameters p4gl_fun_process Business processes implemented by a function p4gl_fun_return Return values of a function p4gl_fun_todo Stack of actions to be done by a function p4gl_module_prog 4gl source modules that are linked in a program p4gl_program 4gl executable program p4gl_table_usage Database table usage in functions
Identifies Package, that typically corresponds to a directory. This way, it provides uniqueness of source files.
|
Column
|
Description
|
| id_package | |
| comments |
Executable program to be compiled. It does not support programs with non-unique name. It is not used at the moment. When the syspgm or makefile importer is implemented, it will be the place where the executables are defined.
|
Column
|
Description
|
| program_name | |
| comments |
Identifies 4GL module (4gl source file).
|
Column
|
Description
|
| id_package | |
| module_name | |
| comments |
Definition of dependencies of 4gl source files, for the construction of one programs. Not used at this time. FIXME: Does not track usage of form, menu or help files.
|
Column
|
Description
|
| program_name | |
| id_package | |
| module_name |
Identifies each 4gl function.
|
Column
|
Description
|
| id_package | |
| module_name | |
| function_name | |
| function_type | |
| comments |
Identifies an analyzed and described process that the software implements.
|
Column
|
Description
|
| id_package | |
| den_process | |
| comments |
Associates functions the processes.
|
Column
|
Description
|
| id_package | |
| module_name | |
| function_name | |
| id_process |
Associates parameters to functions
|
Column
|
Description
|
| id_package | |
| module_name | |
| function_name | |
| item_num | |
| var_name | |
| data_type | |
| comments |
Returns values of a function.
|
Column
|
Description
|
| id_package | |
| module_name | |
| function_name | |
| item_num | |
| var_name | |
| data_type | |
| comments |
Modifications to be made in function code, in the future.
|
Column
|
Description
|
| id_package | |
| module_name | |
| function_name | |
| item_num | |
| comments |
Describes database tables used by functions and modules.
|
Column
|
Description
|
| id_package | |
| module_name | |
| function_name | |
| table_name | |
| operation |
'I','U','D','S' = Insert, Update, Delete, Search (Query) |
Not normalized data loaded directly from a spreadsheet.
|
Column
|
Description
|
| module_name | |
| id_process | |
| function_name | |
| parameters | |
| returns | |
| called | |
| comments |
To be compiled and executed, the DOC4GL documenter have to fulfill a certain number of requirements, that are describe here.Run-time
You need to install the following software in order to run the compiled tools. This also applies if you download pre-compiled binary distribution of DOC4GL.All software listed here is free software, either under GPL license, or free for use, expect Informix database server, which is a commercial product. This dependency will be removed in next version.
- Perl language interpreter, version >= 5.6
- Perl DBI module
- Perl DBD::Informix module
- Perl::Tk module
- Perl::Data module
- Perl::Getopt module
- Informix CLI (Part of Informix CSDK (Client SDK) - also contains ESQL/C)
- Informix JDBC drivers
- Informix database server (commercial)
- JSP enabled web server, for example Jakarta Tomcat
- Java Run-time environment - JRE, also referred to as JVM (for example, Sun J2SE or Kaffe implementation)
Compilation
To compile the DOC4GL tools from source code, you will need:
Obtaining the DOC4GL software package
Currently, DOC4GL is available only in source code form, and only form Aubit 4gl CVS repository. Pre-compiled binary packages and source code distributions, both tar-balls (.tgz) and RPM and Debian (.deb) will be available soon. Please see instructions for accessing CVS on Aubit 4gl project CVS page.
Execute './configure' script before proceeding.
Location and prerequisites
DOC4GL application package consists of 3 core components:
1) Web browser interface component, used to allow viewing and searching generated documentation served by web server, using web browser.
Location: doc4gl/browser
2) Perl CUI and GUI Manager front-end, used to allow viewing and searching generated documentation from graphical interface (GUI) or command line interface (CUI) including batch processing.
Location: doc4gl/manager
3) Parser / loader , core component that performs all core documentation creation tasks.
Location: doc4gl/parser
The components that need compilation are: the parser / loader (p4gl) and additional classes of the web browser manager.
Compilation
You have to compile DOC4GL programs from source code and install it, in order to use it.
The compile configuration is kept in a makefile include file called "FgldocDefs.mk". This is the only place to make changes in order to compile and install the tool. The steps to compile are:
- Execute './configure' script
- (optional) Edit FgldocDefs.mk to verify that it reflects your configuration
- (optional) Type make clean
- Type make
- Type make install
Compiling, installing and using Web browser and GUI interface components is optional, but recommended. If you want to compile and install just core Parser and loader component, including CUI command line interface, cd into doc4gl/parser directory before invoking make command.
Installation
Execute "make install" from DOC4GL root directory or from component subdirectory. By default, if settings in compile-time configuration file "FgldocDefs.mk" where not changed, binaries will be installed in /usr/local/bin. Please make sure that directory that will be used to install executable files is in your PATH environment variable.
Directory doc4gl/sql contains SQL DDL (data definition language) scripts, that user must execute against designated database (currently, since Informix is only database option, execute them using dbaccess or isql tools) to create database structures needed by doc4gl applications. This should be done automatically by 'configure' script:
- inslib_rep.sql - Creates repository structure for scan_fgl / p4gl documenter
- p4gl_rep.sql - FIXME: Creates some tables, probably tables needed for reports generation.
- p4gl_rep_drop.sql - drops all tables created by p4gl_rep.sql script.
- p4gl_del.sql - deletes ALL data from doc4gl related tables
- load_process.sql - is used to manually load data from ".unl" files into p4gl_process table, typically from "p4gl_process.unl" file
- p4gl_process.unl - unload file is used by load_process.sql script. It contains ... FIXME?
The configuration file has the export format of Perl data, created using the "Data::Dumper" module.It can be manually modified, however, care should be taken that each string can be correctly evaluated by the Perl "eval" instruction.
Doc4GL documentation index page Aubit 4gl project Aubit 4gl compiler documentation menu
