InSights Purging(Neo4j)

Looking for Neo4j purging?

This setup includes the steps to perform Neo4j data purging and back up in Elastic search.

The logic is about deleting data in Neo4j every 6 months using delete query and maintaining the data in ES (Note: it will happen in batch processing - In process of automation)
This is done to show old single stat dashboards using ES.
Old Correlational dashboard like traceability might not work which are dependent on Neo4j data

Steps For Setup

Step 1:

Step 2:

Open neo4j.conf, append _mm-yyyy (mm-yyyy is current month and year) to existing ES index name
This helps to pertain the data in previous index of ES
Time factor based naming convention helps to purge data in ES

Step 3:

Step 4:

We need to delete / purge nodes and there are multiple ways to do it
We can use bolt protocol (rest call) to delete
Example Query: (MATCH (n:CI) with n as n,toInt(n.inSightsTimeDate) as time where time<(timestamp()/1000)-340*86400 delete n)
Note: It will happen in batch processing - In process of automation

Step 5:

Top Section