Offline comparison of death acts with Persons_EN

Purpose

This process fetches death acts data from mimir service and uses it for comparison with persons data.

Key points

1. This process uses cron parameter to configure its start time.

2. This process is performed in mpi scheduler pod.

Configuration

Service logic

Step 1. Death acts list for comparison

1. Get list of death acts from that are ready to be compared to persons - perform RPC call to mimir service, dracs_death_acts table, with following parameters:

   1. act_record_operation_name = ‘1' or '4’

   2. persons_compare_status = ‘READY’

2. Sort obtained list in ascending order by dracs_death_acts.updated_at field

3. Limit obtained list with DRACS_DEATH_ACTS_SELECT_BATCH_SIZE value

4. In case no death acts were fetched - end process.

Step 2. Data preparation and potential candidates select

1. Get death act from obtained list, update its status in dracs_death_acts table:

   1. set persons_compare_status = ‘IN_PROCESS’

2. Prepare death act data for comparison process

   1. perform regexp for each of the fields:

      1. for name, surname and patronymic change:

         1. [ --'] to ''

         2. 'є' to 'е'

         3. 'и' to 'і'

      2. for sex change:

         1. 1 to MALE

         2. 2 to FEMALE

         3. everything else to null

      3. for doc_seizes.series_numb change:

         1. [ /%#№ _-]  to ''

   2. for date_birth change mask from dd.mm.yyyy to yyyy-mm-dd

   3. for numident normalize value - check that it is 10 symbols, if not true - set null

3. Get active persons (is_active=true, status=active) as candidates for comparison with death act from MPI db using following predicate blocks:

   1. tax_id

   2. documents.number

   3. birth_date + last_name

   4. settlement_id + last_name

4. In case no candidates were found, update death acts persons compare status:

   1. set persons_compare_status = ‘PROCESSED’

5. Go to next death act in obtained list (return to p.1 of Step 2)

Step 3. Death act comparison process

1. Get person from obtained candidates list

2. Prepare persons data for comparison process accoding to Deduplication process NEW | DeduplicationprocessNEW Datacleaningandpreparation

3. Compare death act data with person data using logistic regression method, as implemented in Deduplication process:

   1. For each variable field use separate calculation process based on the table below.

   2. Calculate final comparison score between death act and person.

Step 4. Verification candidates

Before performing this step, check that death act record is still in persons_compare_status = 'IN_PROCESS'.
In case status is changed - skip this step, go to next death act in obtained list (return to p.1 of Step 2).

Based on death act and person comparison score, there are three flows that can be performed:

• white zone (comparison score is greater than PERSON_DRACS_DEATH_ACT_FULL_MATCH_SCORE value)

• grey zone (comparison score is between PERSON_DRACS_DEATH_ACT_PARTIAL_MATCH_SCORE and PERSON_DRACS_DEATH_ACT_FULL_MATCH_SCORE values)

• black zone (comparison score is less than PERSON_DRACS_DEATH_ACT_PARTIAL_MATCH_SCORE value)

Step 4.1. White zone

White zone indicates that death act relates to person, therefore it can be stated that person highly likely is deceased and must be deactivated.

1. Create verification candidate between death act and person in MPI db, person_verification_candidates table, set values:

   1. id = autogenerate uuid

   2. person_id = id of a person from mpi.persons

   3. entity_id = id of death act from mimir.dracs_death_acts

   4. entity_type = ‘dracs_death_act’

   5. status = ‘CONFIRMED’

   6. config = variables that were used in comparison process (p.3 of Step 3)

   7. details = additional details of comparison process

   8. score = logistic regression comparison score

   9. inserted_at = now()

   10. updated_at = now()

2. Deactivate person

   1. set persons.status = inactive, persons.updated_at = now()

   2. terminate active declarations for person

   3. deactivate active persons authentication methods

   4. revert active verification candidates for person in person_verification_candidates table by person_id

      1. set status = ‘NOT_CONFIRMED’

      2. set updated_at = now()

3. Update persons verification status in person_verifications table:

   1. set dracs_death_act_id = id of death act from mimir.dracs_death_acts

   2. set dracs_death_verification_status = ‘VERIFIED’

   3. set dracs_death_verification_reason = ‘AUTO_OFFLINE’

   4. set dracs_death_online_status = ‘COMPLETED’

   5. set updated_at = now()

   6. set updated_by = system_user()

Step 4.2. Grey zone

Grey zone indicates that death act possibly relates to person, therefore it can be stated that person may be deceased and must be inspected by doctor or NHS data steward (or both).

1. Create verification candidate between death act and person in MPI db, person_verification_candidates table, set values:

   1. id = autogenerate uuid

   2. person_id = id of a person from mpi.persons

   3. entity_id = id of death act from mimir.dracs_death_acts

   4. entity_type = ‘dracs_death_act’

   5. status = ‘NEW’

   6. config = variables that were used in comparison process (p.3 of Step 3)

   7. details = additional details of comparison process

   8. score = logistic regression comparison score

   9. inserted_at = now()

   10. updated_at = now()

2. Update persons verification status in person_verifications table based on current dracs death verification status of person:

   1. in case dracs_death_verification_status = ‘NOT_VERIFIED’ - do not update persons dracs death verification status

   2. in case dracs_death_verification_status = ‘VERIFICATION_NEEDED’ and dracs_death_verification_reason = ‘MANUAL_NOT_CONFIRMED’ - do not update persons dracs death verification status

   3. in case dracs_death_verification_status = ‘VERIFICATION_NEEDED’ and dracs_death_verification_reason = ‘MANUAL_CONFIRMED’ - do not update persons dracs death verification status

   4. in case dracs_death_verification_status = ‘VERIFICATION_NEEDED’ and dracs_death_verification_reason = ‘ONLINE_TRIGGERED’ - update persons dracs death verification status:

      1. set dracs_death_verification_status = ‘NOT_VERIFIED’

      2. set dracs_death_verification_reason = ‘AUTO_OFFLINE’

      3. set updated_at = now()

      4. set updated_by = system_user()

   5. in case dracs_death_verification_status = ‘VERIFICATION_NEEDED’ and dracs_death_verification_reason = ‘INITIAL’ - update persons dracs death verification status:

      1. set dracs_death_verification_status = ‘NOT_VERIFIED’

      2. set dracs_death_verification_reason = ‘AUTO_OFFLINE’

      3. set updated_at = now()

      4. set updated_by = system_user()

   6. in case dracs_death_verification_status = ‘IN_REVIEW’ and dracs_death_verification_reason = ‘MANUAL’ - do not update persons dracs death verification status

   7. in case dracs_death_verification_status = ‘VERIFIED’ - update persons dracs death verification status:

      1. set dracs_death_verification_status = ‘NOT_VERIFIED’

      2. set dracs_death_verification_reason = ‘AUTO_OFFLINE’

      3. set updated_at = now()

      4. set updated_by = system_user()

Step 4.3. Black zone

Black zone indicates that death act does not relate to person, therefore it can be stated that person highly likely is not deceased.
No further actions should be taken with person or verification candidates.

Step 5. Processed death act

1. When death act is fully compares with all candidates from list, change its status in mimir.dracs_death_acts table

   1. set persons_compare_status = ‘PROCESSED’

2. Go to next death act in obtained list (return to Step 2)