Modern COBOL: Event Logging Tutorial

You will learn and create a centralized analytics for COBOL programs. You will implement Syslog event-handler and deploy the latest ELK (Elasticsearch, Logstash, Kibana) service which aggregates application-level messages. This article is a successor of Modern COBOL: Microservice Tutorial.

Kibana dashboard on Elastic

Preconditions

You have learned basic principles, methods and standards of COBOL. In this tutorial we’ll use GnuCOBOL — a free COBOL compiler which implements a substantial part of the COBOL 85, COBOL 2002 and COBOL 2014 standards and X/Open COBOL, as well as many extensions included in other COBOL compilers.

You have Docker, a command-line virtualization tool, installed.

You have Docker Compose, a command-line tool for defining and running multi-container Docker applications, installed.

You have Git, an open source distributed version control client, installed.

You may use any text editor you like, but I recommend Visual Studio Code (or its open-source version VSCodium) with COBOL-syntax extension bitlang.cobol installed.

Components

Microservice is a minimalistic, independently deployable, self-contained network service. We may run many instances of the same microservice simultaneously, and balance the traffic among them. Here we’ll extend a high-precision currency exchange microservice from Modern COBOL: Microservice Tutorial by implementing of the event-handler which sends the messages to the ELK service for further analysis.

ELK is an acronym for three open-source projects: Elasticsearch, Logstash, and Kibana. Elasticsearch is a full-text search engine that stores the messages; Logstash is a preprocessor that receives and transforms the messages for Elasticsearch; Kibana is a visualization dashboard in Elasticsearch that presents impressive charts and graphs.

We’ll use Syslog format that is a de facto standard logging solution since 1980s, documented by the Internet Engineering Task Force under RFC 3164. A wide variety of devices and applications across many platforms use Syslog. Our ELK instance exposes Syslog as an input, and Kibana as an output, listening on the ports such as:

• 5140 — Syslog UDP input
• 5601 — Kibana dashboard

Please find both components on GitHub and copy to your machine. Directory elk contains the latest (at the time of writing) open-source version of ELK, virtualized by Docker Compose. Directory microservice contains the COBOL program, virtualized by Docker.

Events

During its life-time the microservice deals with different types of events triggered by applications, hardware and users, feeding the system journals and printing to the console. In this tutorial we’ll focus on three application-level events: informational, debugging and errors. Let’s implement simple event-handler in COBOL, instead of ubiquitous stdout and stderr, invisible from the outside of the container:

identification division.
program-id. event-handler.
...
procedure division using l-facility, l-severity, l-message returning omitted.
  move send-udp(
    "localhost",
    5140,
    trim(syslog(function MODULE-CALLER-ID "COBOL" l-facility l-severity l-message))
  ) to ws-syslog-sent.
end program event-handler.

Function send-udp sends UDP Syslog messages, created by syslog function, to the port 5140 of the ELK service. Each Syslog message is labeled with a facility code, indicating an originator of the message, and assigned a severity level. Since our microservice handles HTTP requests from users, we point the Syslog user facility to all events in the microservice (you may choose any other convention you like). On the top of microservice.cbl you’ll find the definitions:

>>DEFINE CONSTANT SYSLOG-SEVERITY-ERRROR "3"
>>DEFINE CONSTANT SYSLOG-SEVERITY-DEBUG "7"
>>DEFINE CONSTANT SYSLOG-SEVERITY-INFORMATIONAL "6"
>>DEFINE CONSTANT SYSLOG-FACILITY-USER "8"

Let’s notify our ELK service when

• the filesystem error occurred, line #32;
• the microservice started, line #45;
• the microservice is unable to handle the request, line #97;
• the computing error occurred, line #106;
• the computing finished, line #109.

Only the first event is fatal for the microservice. Other events we’ll collect for further demonstration and analysis. As you can see, lines #45 and #109 use >>D debug line control that instructs GnuCOBOL to compile those lines only if the -fdebugging-line command-line option is set. You may turn them on by adding this option to the ENTRYPOINT declaration in the Dockerfile:

ENTRYPOINT ["/bin/ash", "-c", "/usr/local/bin/cobc -j -x -fdebugging-line src/microservice.cbl"]

Dependencies

Our microservice depends on HTTP server, ECB CSV parser and Syslog client. All these packages are available on COBOL Package Registry — cobolget.com. Docker automatically installs required dependencies during the build of the image, by using an open-source COBOL package management tool — cobolget. Here’s a an one-liner in the Dockerfile for that:

RUN cobolget -t bca12d6c4efed0627c87f2e576b72bdb5ab88e34 install

Test

Let’s launch the ELK in the separate console:

$ cd elk 
$ docker-compose up

Please wait a few minutes and open http://localhost:5601 in your browser. You will see the main menu of Kibana. Please open new console and start the microservice in daemon mode:

$ cd microservice
$ docker build --tag microservice . 
$ docker run -d -i --name microservice -p 8000:8000 microservice

The microservice awaits HTTP request GET /<currency>/<amount> on port 8000 and respond JSON {"amount": <amount>}, where

• <currency> is a tree-letter ISO currency code, i.e. USD
• <amount> is a numeric value separated by dot, i.e. 999.999

Any mismatching requests, unsupported currencies, as well as calculation errors will result in 404 Not Found responses.

You can test the microservice by the links below:

• http://localhost:8000/USD/99.99 returns JSON;
• http://localhost:8000/ABC/99.99 returns HTTP 404
• http://localhost:8000/USD/9999999999999999 returns HTTP 404.

Finally, please go back to the Kibana tab. We’ll configure the dashboard telling Kibana right Elasticsearch index pattern for our messages — events.

1. Click Discover in Visualize and Explore Data section.
2. Click Check for new data.
3. Input events in Index pattern of Step 1 of 2: Define index pattern section.
4. Click Next step.
5. Select @timestamp in Time Filter field name Refresh of Step 2 of 2: Configure settings section.
6. Click Create index pattern.
7. Click Discover or http://localhost:5601/app/kibana#/discover.

On the dashboard you will see the Syslog messages sent by our microservice. If not, please select Today Time window at the top-right corner. Time zones in your browser and in the Docker containers may differ.

Conclusion

You have created and successfully deployed the centralized analytics for your COBOL microservice, based on open-source Elasticsearch, Logstash and Kibana. You have implemented the event-handler in Syslog format by using Git, Docker and COBOLget package manager.

Building advanced visualizations and performing aggregation-based analysis is out of scope of this tutorial. Official Kibana guide might be a reasonable starting point.

Please contact me if you have any corrections or feedback.