Page Comparison

To create a Data Archival record, the following steps needs to be followed: 

1. Go to the Data Archival page and click on the add button on the top right corner. An ElasticTransfer agent needs to be configured in the system as it is a pre-requisite for Data Archival record creation. 

2. 

   Enter the following details in the Add screen. 

   • Archival Name: Name of the archival record. 

   • Records from: Starting date from which the records need to be fetched in the container. 

   • Records to: End date till which the records need to be fetched in the container. 

   • Days to Retain: Number of days till which the container needs to be active. 

   Click on the save button on the top right corner to save the record. 

   Image RemovedImage Added

3. A Data Archival record is created with the state ‘INPROGRESS’. Container creation is in progress now. 

   

4. After the container is created, the Data Source URL column will contain the URL of the container and the boltport column will contain the boltport for the Neo4j container.

   

   The URL is clickable and it will open the Neo4j browser for the container. You will need to provide the boltport while login in to the Neo4j browser. 

   

5. After the Data Archival record reaches expiration date, the container will be terminated and the record in Data Archival page will show ‘TERMINATED’ status. 

   

• The message that will be sent to the ElasticTransfer agent for container creation will be of the following format: 

  Code Block
  { "archivalName": "Archive_1", "startDate": "2018-03-01T00:00:00Z", "endDate": "2018-10-31T00:00:00Z", "daysToRetain": 10 }

• ElasticTransfer agent will send the following message to RabbitMQ’s SYSTEM_ELASTICTRANSFER_DATA queue so that it can be consumed by the engine and the URL of the container updated in the record. 

  Code Block
  language json
  { "data":[ { "status":"Success", "execId":"fd0da051-8bcc-11eb-b09b-005056955e85", "sourceUrl":"http://<ip>:<port>", "containerID":"c68f74bef5aaa825663a327c78dacd5934fc4ec1385f4c7c91d8e3676f62355e", "archivalName":"Archive_1", "toolName":"ELASTICTRANSFER", "message":"Node Count = 7390", "categoryName":"SYSTEM" , "boltPort":7687 } ], "metadata":{ "dataUpdateSupported":false } }

• The following message will be sent to ElasticTransfer agent to terminate the container. 

  Code Block
  { "task":"remove_container", "containerID":"c68f74bef5aaa825663a327c78dacd5934fc4ec1385f4c7c91d8e3676f62355e" }

• The following json is a sample daemon.json that should be present in /etc/docker/ path in Docker machine.

  Code Block
  { "debug": true, "insecure-registries" : ["<hostname/ip>:<port>"], "hosts": ["unix:///var/run/docker.sock","tcp://<ip>:<port>"] }