Insights Docker Installation

Owned by Jeyanthi (Unlicensed)
Last updated: Jun 20, 2022 by Vishwajit Mankar (Unlicensed)


Insights docker image does the following:

  • Initial setup - create directories and set env variables like INSIGHTS_HOME
  • OS Version: Alpine 3.10.0-1160.21.1.el7.x86_64  Architecture: amd64  For RHEL: CentOS7
  • Java - 1.8.0_212-b04 (JVM Vendor: IcedTea)
  • Python 2.7.16 (pika 0.11.2) (not in 8.0-alpine image)
  • Update server-config.json : Download latest server-config.json file from Nexus docroot repo. Get the required database parameters from config.env to update server-config.json dynamically. config.env conatins Noe4j IP Postgres IP, GrafanaDBUser, GrafanaDBPassword, RabbitMQ IP. These IP's to be added as trustedHosts in server-config.json
  • Update uiConfig.json
  • Update rabbitmqIP in config.json for AgentDaemon
  • AgentDaemon (latest) (not in 8.0-alpine image)
  • Grafana - v7.1.0 (8101355285)
  • Apache Tomcat - 9.0.36
  • Apache httpd server - Apache/2.4.46 (CentOS)
  • Insights latest artifacts being used to deploy inside container. PlatformService, PlatformUI, PlatformEngine, PlatformInsightsWebHook, PlatformWorkflow(PlatfformReports).

Note

neo4j, postgres and rabbitmq are running externally and these ip details are given in config.env file as an input to docker image. 

Prerequisite:

  • Docker
  • Java
  • Neo4j
  • Postgresql
  • RabbitMQ

Install Docker:

On RHEL/Centos7:

Latest method:

Reference: https://docs.docker.com/engine/install/centos/

Alternate method:

  • yum update

  • yum install yum-utils

  • yum-config-manager --enable rhel-7-server-rhui-extras-rpms 
  • yum install docker
  • systemctl start docker
  • systemctl status docker
  • systemctl enable docker (start docker at system boot)

For RHUI 3 repo ID(rhel-7-server-rhui-extras-rpms ) to configure yum-config-manager, please visit - https://access.redhat.com/articles/4599971

On Ubuntu:

  • apt-get update
  • apt-get upgrade
  • apt-get install docker.io
  • systemctl start docker
  • systemctl enable docker

For docker commands please visit -  https://docs.docker.com/edge/engine/reference/commandline/docker/

Install Java, Neo4j, Postgresql and ElasticSearch:

Use the below scripts to install Java, Neo4j, postgresql and ElasticSearch(optional) for Insights.

Java: https://github.com/CognizantOneDevOps/Insights/blob/master/PlatformDeployment/RHEL7/reference-DocRoot-Scripts/insights_java.sh

Neo4j: https://github.com/CognizantOneDevOps/Insights/blob/master/PlatformDeployment/RHEL7/reference-DocRoot-Scripts/insights_neo4j.sh

Postgresql: https://github.com/CognizantOneDevOps/Insights/blob/master/PlatformDeployment/RHEL7/reference-DocRoot-Scripts/insights_postgres12.sh

RabbitMQ: https://github.com/CognizantOneDevOps/Insights/blob/master/PlatformDeployment/RHEL7/reference-DocRoot-Scripts/insights_rabbitmq.sh

ElasticSearch: https://infra.cogdevops.com/repository/docroot/insights_install/installationScripts/latest/RHEL/scripts/insights_es.sh

Allow Grafana to access Postgresql:

Follow the below steps to allow Grafana to connect to Postgresql. 

  1. Update listen address to 0.0.0.0 in /var/lib/pgsql/9.5/data/postgresql.conf file. 

               listen_addresses = '0.0.0.0'

          This can be postgres public ip as well to restrict over network and allow this port in network for grafana ip. Find the file "postgresql.conf" if it is not in the specified location and update accordingly.

      2. Allow the Grafana IP under IPV4 section in /var/lib/pgsql/9.5/data/pg_hba.conf file. Add your ip address in the below format. xx.xxx.xxx.xx is the public ip address of intended grafana. In case, postgresql is hosted on localhost then no need to do this step. 

          # IPv4 local connections:

          host    all             all             127.0.0.1/32            trust

          host    all             all             xx.xxx.xxx.xx/32    trust

           Note that specifying IPv4 as 0.0.0.0/0 in pg_hba.conf file will allow all incoming connections which is not secure.

       3. Restart postgresql.

            systemctl restart  postgresql-9.5.service

Network

Make sure that proper connection/port opening is available to connect to Data layer components which are hosted externally in another instance (Postgresql, Neo4j , RabbitMQ)

Step 1: Pull Insights Docker Image

Docker Hub Repo:

docker pull devopsbasservice/onedevopsinsights:8.0-alpine-v2

(OR)

OneDevOps Repo:

Insights docker repo is using the self signed certificate. Allow the insights docker repo cert in docker daemon by following the below steps.

  1. Add the below content in the file /etc/docker/daemon.json

             {
                 "insecure-registries" : ["infra.cogdevops.com:8090"]
             }

      2. Execute the following commands to reload and restart docker

          systemctl daemon-reload
          systemctl restart docker   (or) service docker restart

     3. Pull the insights docker image to local.

         docker pull infra.cogdevops.com:8090/onedevopsinsights:latest

         (OR)

         docker pull infra.cogdevops.com:8090/onedevopsinsights:8.0-alpine-v2

        We can also run Insights application and Insights agents in  2 different containers. 

        a. Application container image

           docker pull devopsbasservice/onedevopsinsights:app

        b. Agent container image

           docker pull devopsbasservice/onedevopsinsights:agent       

Reference: https://docs.docker.com/engine/reference/commandline/dockerd/#daemon-configuration-file

Step 2: Run Insights Docker Image

  • config.env - All the required inputs such as hostIP - host machine IP where you are running docker container which will have apache httpd,grafana and InsightsUI, neo4jIP - Neo4j ,postgresIP - postgresql , grafanaDBUser and grafanaDBPass- password for the postgres db superuser "grafana" created during postgres setup, rabbitMQIP - RabbitMQ, rabbitMqUser and  rabbitMqPassword - rabbitMQ username(iSight) and password. These values are set in container environment.
  • Run docker image with volume mount to /opt directory. volume /opt will help to recover data in a worst case scenario like container crash/Exited. Also, you can add any other required directory from container as volume mount by extending additional -v parameter in docker run command to recover data. 

"docker images" to lists all the local images. You may run container either Repository:Tag or ImageID

config.env

hostIP=xx.xxx.xx.xxx
neo4jIP=xx.xxx.xx.xxx
postgresIP=xx.xxx.xx.xxx
grafanaDBUser=grafana
grafanaDBPass=<grafanaDBpassword>
rabbitmqIP=xx.xxx.xx.xxx
rabbitMqUser=iSight
rabbitMqPassword=<mqPassword>

 Note: Replace xx.xx.xxx.xx with respective IP address or Hostname. Do not use any quotes when you provide values.

  • Alpine config.env - All the required inputs such as hostInstance - host machine IP or hostname where you are running docker container which will have apache tomcat,grafana and InsightsUI, servicePort - Tomcat exposed port for host machine, garfanaPort - grafana exposed port for host machine, neo4jIP - Neo4j , neo4jToken - neo4j authToken, postgresIP - postgresql , grafanaDBUser and grafanaDBPass- password for the postgres db superuser "grafana" created during postgres setup, rabbitMQIP - RabbitMQ, rabbitMqUser and  rabbitMqPassword - rabbitMQ username(iSight) and password. These values are set in container environment.

Alpine config.env

hostInstance=<valid host or IP>
servicePort=<docker-exposed-hostTomcatPort>
grafanaPort=<docker-exposed-hostGrafanaPort>
neo4jIP=<valid host or IP>
neo4jHttpPort=7474
neo4jBoltPort=7687
neo4jUser=neo4j
neo4jPassword=<neo4jPassword>
postgresIP=<valid host or IP>
postgresPort=5432
grafanaDBUser=grafana
grafanaDBPass=<grafanaDBPassword>
rabbitmqIP=<valid host or IP>
rabbitMqUser=iSight
rabbitMqPassword=<mqPassword>
rabbitMqPort=<mqPort>
   
     
       Alpine config.env for Insights agents container- 

config.env
rabbitmqIP=<mq IP>
rabbitmqPort_Agent=<mq port>
rabbitMqUser=<mq user>
rabbitMqPassword=<mq password>

 Note: Replace xx.xx.xxx.xx with respective IP address or Hostname. Do not use any quotes when you provide values.

Docker Hub Repo:

docker run -dit --name onedevopsinsights -v insights-opt-vol:/opt -p 80:80 -p 3000:3000 -p 8080:8080 --env-file config.env devopsbasservice/onedevopsinsights:latest

OneDevOps Repo:

docker run -dit --name onedevopsinsights -v insights-opt-vol:/opt -p 80:80 -p 3000:3000 -p 8080:8080 --env-file config.env infra.cogdevops.com:8090/onedevopsinsights:latest

(OR)

docker run -dit --name onedevopsinsights -v insights-opt-vol:/opt -p 3000:3000 -p 8080:8080 --env-file config.env infra.cogdevops.com:8090/onedevopsinsights:8.0-alpine-v2

(OR)

docker run -it --name onedevopsinsights -v insights-opt-vol:/opt -p 80:80 -p 3000:3000 -p 8080:8080 --env-file config.env 8db313c60400

docker run

-dit runs the container in detached mode. For interactive mode use  -it. 

 -p helps to expose specified ports. If port is already in use, try with other port of host to connect. 
   Make sure that Grafana runs on 3000 by default.


Wait until the container is fully up(~7-10 mins. Just to confirm, check PlatformEngine process "ps -ef | grep Engine"). Then, below Insights URL's are accessible.

http://hostIP:3000 - grafana
http://hostIP:15672/ - rabbitmq
http://hostIP:8080/app - Insights UI


Container Data Recovery - /opt

  1. If above container (onedevopsinsights) is crashed/Exited, then run the new container(onedevopsinsights1) as given below with different volume then do the docker copy with previous container(onedevopsinsights) volume. 

    docker run -dit --name onedevopsinsights1-v insights-opt-vol1:/opt -p 80:80 -p 3000:3000 -p 8080:8080 --env-file config.env devopsbasservice/onedevopsinsights:8.0-alpine-v2       

  2. docker copy agents directory from host's onedevopsinsights volume(insights-opt-vol) path to latest/new container (onedevopsinsights1) /opt path. Below "docker cp" command will copy /opt/insightsagents/PlatformAgents. You may copy required directory and files of /opt in a similar way w.r.to your scenario.

    docker cp /var/lib/docker/volumes/insights-opt-vol/_data/insightsagents/PlatformAgents 6d600898f52a:/opt/insightsagents/                                                                                                                 

Useful docker commands:

List the active container: docker ps

List the active/exited container: docker ps -a

Check container logs: docker logs <container-id>

Connect to Container: docker exec -it <container-id> bash

Top Section

©2021 Cognizant, all rights reserved. US Patent 10,410,152