ЕСОЗ - публічна документація

Offline comparison of death acts with Parties_EN

Purpose

This process fetches death acts data from mimir service and uses it for comparison with parties data.

Key points

  1. This process uses cron parameter to configure its start time.

  2. This process is performed in ehealth scheduler pod.

Configuration

Value

Description

Example

Value

Description

Example

PARTIES_DRACS_DEATH_ACTS_COMPARE_SCHEDULE

Cron parameter, represents start time of offline comparison of death acts with parties process

*/3 * * * *

PARTIES_DRACS_DEATH_ACTS_COMPARE_BATCH_SIZE

Number of death acts that will be processed in one cycle of the process

100

PARTY_DRACS_DEATH_ACT_FULL_MATCH_SCORE

Upper limit of comparison score for grey zone, lower limit of comparison score for white zone

0.9

PARTY_DRACS_DEATH_ACT_PARTIAL_MATCH_SCORE

Upper limit of comparison score for black zone, lower limit of comparison score for grey zone

0.7

Service logic

Step 1. Death acts list for comparison

  1. Get list of death acts from that are ready to be compared to parties - perform RPC call to mimir service, dracs_death_acts table, with following parameters:

    1. act_record_operation_name = ‘1' or '4’

    2. parties_compare_status = ‘READY’

  2. Sort obtained list in ascending order by dracs_death_acts.updated_at field

  3. Limit obtained list with DRACS_DEATH_ACTS_SELECT_BATCH_SIZE value

  4. In case no death acts were fetched - end process.

Step 2. Data preparation and potential candidates select

  1. Get death act from obtained list, update its status in dracs_death_acts table:

    1. set parties_compare_status = ‘IN_PROCESS’

  2. Prepare death act data for comparison process

    1. perform regexp for each of the fields:

      1. for name, surname and patronymic change:

        1. [ --'] to ''

        2. 'є' to 'е'

        3. 'и' to 'і'

      2. for sex change:

        1. 1 to MALE

        2. 2 to FEMALE

        3. everything else to null

      3. for doc_seizes.series_numb change:

        1. [ /%#№ _-]  to ''

    2. for date_birth change mask from dd.mm.yyyy to yyyy-mm-dd

    3. for numident normalize value - check that it is 10 symbols, if not true - set null

  3. Get parties with at least one active employee as candidates for comparison with death act from PRM db using following predicate blocks:

    1. tax_id

    2. documents.number

    3. birth_date + last_name

  4. In case no candidates were found, update death acts parties compare status, go to next death act in obtained list:

    1. set parties_compare_status = ‘PROCESSED’

Step 3. Death act comparison process

  1. Get party from obtained candidates list

  2. Prepare parties data for comparison process accoding to Deduplication process NEW | DeduplicationprocessNEW Datacleaningandpreparation

  3. Compare death act data with party data using logistic regression method, as implemented in Deduplication process:

    1. For each variable field use separate calculation process based on the table below.

    2. Calculate final comparison score between death act and party.

Variable

Description

parties

dracs death act

Variable

Description

parties

dracs death act

d_first_name

levenshtein distance(first_name1, first_name2)

first_name

name

d_last_name

levenshtein distance(last_name1, last_name2)

last_name

surname

d_second_name

levenshtein distance(second_name1, second_name2)

second_name

patronymic

d_documents_bin

min(levenshtein distance(document1, document2)) for any types of documents

documents.number

doc_seizes.series_numb

docs_same_number_bin

min(same/not) number

documents.number

doc_seizes.series_numb

birth_settlement_substr

min(position(birth_settlement_1 in birth_settlementt_2) and position(birth_settlement_2 in birth_settlementt_1)

-

birth_locality (if not null or birth_locality_type <> ‘Район’), else birth_district (if not null), else birth_region

authentication_methods

same/not authentification OTP number flag

-

-

residence_settlement_flag

same/not residence settlement flag

-

locality (if not null or locality_type <> ‘Район’), else district (if not null), else region

d_tax_id

levenshtein distance(tax_id1, tax_id2)

tax_id

numident

gender_flag

same/not gender

gender

sex

twins_flag

distance last_name <=2, same birth_date, distance in document numbers between 1 and 2

last_name,

first_name,

birth_date,

documents.number

surname,

name,

date_birth,

doc_seizes.series_numb

Step 4. Verification candidates

Before performing this step, check that death act is still in parties_compare_status = 'IN_PROCESS'.
In case status is changed (death act was updated by other processes) - skip this step, go to next death act in obtained list (return to step 2).

Based on death act and party comparison score, there are two flows that can be performed:

  • white zone (comparison score is greater than PARTY_DRACS_DEATH_ACT_FULL_MATCH_SCORE value) or grey zone (comparison score is between PARTY_DRACS_DEATH_ACT_PARTIAL_MATCH_SCORE and PARTY_DRACS_DEATH_ACT_FULL_MATCH_SCORE values)

  • black zone (comparison score is less than PARTY_DRACS_DEATH_ACT_PARTIAL_MATCH_SCORE value)

Step 4.1. White or grey zone

White zone indicates that death act relates to party, therefore it can be stated that party highly likely is deceased and must be inspected by legal entity owners.

Grey zone indicates that death act possibly relates to party, therefore it can be stated that party may be deceased and must be inspected by legal entity owners.

  1. Create verification candidate between death act and party in PRM db, party_verification_candidates table, set values:

    1. id = autogenerate uuid

    2. party_id = id of a party from prm.parties

    3. entity_id = id of death act from mimir.dracs_death_acts

    4. entity_type = ‘dracs_death_act’

    5. status = ‘NEW’

    6. config = variables that were used in comparison process (p.3 of Step 3)

    7. details = additional details of comparison process

    8. score = logistic regression comparison score

    9. inserted_at = now()

    10. updated_at = now()

  2. Update parties verification status in party_verifications table based on current dracs death verification status of party:

    1. in case dracs_death_verification_status = ‘NOT_VERIFIED’ - do not update parties dracs death verification status

    2. in case dracs_death_verification_status = ‘VERIFICATION_NEEDED’ and dracs_death_verification_reason = ‘ONLINE_TRIGGERED’ - update parties dracs death verification status:

      1. set dracs_death_verification_status = ‘NOT_VERIFIED’

      2. set dracs_death_verification_reason = ‘AUTO_OFFLINE’

      3. set updated_at = now()

      4. set updated_by = system_user()

    3. in case dracs_death_verification_status = ‘VERIFICATION_NEEDED’ and dracs_death_verification_reason = ‘INITIAL’ - update parties dracs death verification status:

      1. set dracs_death_verification_status = ‘NOT_VERIFIED’

      2. set dracs_death_verification_reason = ‘AUTO_OFFLINE’

      3. set updated_at = now()

      4. set updated_by = system_user()

    4. in case dracs_death_verification_status = ‘VERIFIED’ - update parties dracs death verification status:

      1. set dracs_death_verification_status = ‘NOT_VERIFIED’

      2. set dracs_death_verification_reason = ‘AUTO_OFFLINE’

      3. set updated_at = now()

      4. set updated_by = system_user()

Step 4.2. Black zone

Black zone indicates that death act does not relate to party, therefore it can be stated that party highly likely is not deceased.
No further actions should be taken with party or verification candidates.

Step 5. Processed death act

  1. When death act is fully compares with all candidates from list, change its status in mimir.dracs_death_acts table

    1. set parties_compare_status = ‘PROCESSED’

  2. Go to next death act in obtained list (return to step 2)

ЕСОЗ - публічна документація