• Purpose

• Key points

• Configuration

• Data flow

  • UML

  • Step description

    • Step 1. Get list of Persons

    • Step 2. Call InfoRNOKPPDRFO

    • Step 3. Create or update drfo_data

    • Step 4. Update Person

    • Step 4.1. Update another Persons

    • Step 4.2. Update Parties

    • Step 2.1. Update Person

    • Step 5. Call FindRegistrationDRFOQuery

    • Step 6. Create Job

    • Step 7. Call FindRegistrationDRFOAnswer

    • Step 8. Create or update drfo_data

    • Step 9. Update Person

    • Step 9.1. Update another Persons

    • Step 9.2. Update Parties

    • Step 10. Update Person

    • Step 11. Update Person

    • Step 12. Update Person

    • Step 13. Technical error returned

• Log info

Purpose

This job designed for initial and further regular synchronization of Persons with the DRFO registry. In addition, it can work on a schedule to make synchronization of updated Persons via Person Request whose DRFO data hasn’t been updated online due to technical reasons.

Key points

1. Job can modify Person data in MPI: tax_id and no_tax_id fields. But, these fields does not update if person is female with age > 16, so FindRegistrationDRFO service does not call for it.

2. Job gets DRFO records using strict Person data, such as last name, first name etc. (w/o any assumptions and data modifications)

3. It is assumed that DRFO record is unique within RNOKPP. So if such record does not exists in drfo_data table (MIMIR db) it will be created, if exists - updated.

4. The result of the job changes verification status and reason according to received result from DRFO registry. Also, if success the DRFO data is saved and linked to the Person.

5. Verification status of another Persons/Parties associated with updated DRFO data may be changed too.

Configuration

Data flow

Scheduller starts according to DRFO_PERSON_SYNCHRONIZATION_SCHEDULE cron parameter and gets number of entries pointed at PERSONS_SYNCHRONIZATION_BATCH_SIZE chart parameter. After, it creates an oban job that selects Persons which match to some filter parameters. For each the job receives Person data on input and returns the result of reconciliation with the DRFO registry to output (details in “Step description” section below):

• if RNOKPP is found and matches to the person’s tax_id or passport/national_id number - set verification status to VERIFIED along with verification reason AUTO. Also, update no_tax_id to “false” (if it has not set to “false” yet) for tax ID returned in RNOKPP; update no_tax_id to “true” (if it has not set to “true” yet) for passport/national_id number returned in RNOKPP. The DRFO data is saved and linked to the Person.

• if RNOKPP is found and doesn’t match to the person’s tax_id or passport/national_id number.
  For passport/national_id number - set verification status to NOT_VERIFIED along with verification reason AUTO if any document with such number does not exist in person documents; if any document with such number exists - set verification status to VERIFIED along with verification reason AUTO. Update no_tax_id to true (if it has not set to true yet) and tax_id to null (if it is not null).
  For tax_id returned in RNOKPP - just update person’s tax_id to RNOKPP value, set verification status to VERIFIED along with verification reason AUTO and update no_tax_id to false (if it has not set yet). DRFO data is saved and linked to the Person;

• if person identified as deceased - the verification status sets to VERIFICATION_NEEDED and verification reason to POTENTIALLY_DECEASED. Update no_tax_id and tax_id to null because DRFO does not return RNOKPP in this case and we don’t know what it really was.
  DRFO data is not saved and not linked to the Person;

• if person identified as having closed RNOKPP - the verification status sets to NOT_VERIFIED along with verification reason AUTO. Update no_tax_id and tax_id to null because DRFO does not return RNOKPP in this case and we don’t know what it really was.
  DRFO data is not saved and not linked to the Person;

• if RNOKPP is not found for the person - set verification status to NOT_VERIFIED for adults and VERIFIED for underaged person (configured in no_self_auth_age global parameter, equal 14 at the moment); verification reason set to AUTO. Update no_tax_id, tax_id and drfo_data_id to null.
  DRFO data is not saved and not linked to the Person;

• if technical error received - the verification status and reason doesn’t change. DRFO data is not saved, Person is not updated;

UML

...

Step description

Step 1. Get list of Persons

Get list of Persons with status=active and is_active=true who meets the following criteria (using logical AND):

• mpi.person_verifications.drfo_synced_at <= current_date - DRFO_PERSON_VALIDATION_PERIOD_DAYS OR drfo_synced_at is null

• has tax_id or (hasn’t tax_id and age > no_self_auth_age)

• exclude persons who has drfo_verification_status = IN_REVIEW and drfo_verification_reason = AUTO (such status means they has already being processed by the job at the moment)

Order selected records in descending way by:

• drfo_verification_status = VERIFICATION_NEEDED and drfo_verification_reason = ONLINE_TRIGGERED(should be at the top of the list)

Step 2. Call InfoRNOKPPDRFO

Check person’s RNOKPP - invoke RPC call with third-party service DRFO Trembita methods_ph2_EN | InfoРNОКPPDRFO using following request params:

• RNOKPP = mpi.persons.tax_id if exists, else - mpi.person_documents.number. For document number: get document with the latest issued_at, but exclude those with an expiration_date that is less than the current date, and type should be in (“PASSPORT”, “NATIONAL_ID”).

  • if RNOKPP not defined OR RNOKPP format doesn't match pattern (^(?![ЫЪЭЁ])[А-ЯҐЇІЄ]{2}[0-9]{6}$|^[0-9]{9}$|^[0-9]{10}$) - InfoRNOKPPDRFO shouldn't be called:

    • If person is FEMALE with age > 16, then set person_verification for it:

      • drfo_data_id = null

      • drfo_data_result = null

      • drfo_synced_at = current datetime

      • drfo_verification_status = NOT_VERIFIED

      • drfo_verification_reason depends on conditions:

        • if RNOKPP not defined - TAX_ID_AND_PASSPORT_DOCUMENT_ABSENT

        • if RNOKPP format doesn't match pattern - INVALID_RNOKPP_FORMAT

      • drfo_unverified_at = current datetime

    • else - DRFO data synchronization for Persons (CSI-3043) | Step 5. Call FindRegistrationDRFOQuery should be initialized

• last_name = mpi.persons.last_name

• first_name = mpi.persons.first_name

• middle_name = mpi.persons.second_name (skip if empty)

• date_birth = mpi.persons.birth_date

Update person_verifications record by person_id:

• drfo_verification_status = IN_REVIEW

• drfo_verification_reason = AUTO

• drfo_unverified_at = null (if not null)

Step 3. Create or update drfo_data

If RNOKPP check in Step 2 was successful and RESULT=0 returned. Get record from mimir.drfo_data table by rnokpp:

• if not found - create a record.

• if found - updated the record.

Step 4. Update Person

Invoke the Step If person_verifications.drfo_verification_status still IN_REVIEW (because if changed to VERIFICATION_NEEDED+ONLINE_TRIGGERED, then its mean that person record has been changed while current sync with DRFO performed)

After DRFO record was stored in Step 3 - update persons record in MPI db:

• no_tax_id:

  • set false (if it has not set to false yet) for tax_id returned in RNOKPP;

  • set true (if it has not set to true yet) for passport/national_id number returned in RNOKPP

Update person_verifications record by person_id:

• drfo_data_id = mimir.drfo_data.id

• drfo_data_result = 10 according to DRFO_RESULT dictionary.

• drfo_synced_at = date and time when synchronization has been performed (current datetime)

• drfo_verification_status = VERIFIED

• drfo_verification_reason = AUTO

• drfo_unverified_at = null (if not null)

Step 4.1. Update another Persons

Invoke the Step If person_verifications.drfo_data_verification_status is not IN_REVIEW

If drfo data has been changed in Step 3 and another active Persons linked to it (have the same drfo_data_id in person_verifications table), then compare person’s data with drfo data: last_name, first_name, middle_name and date_birth. If these data are different, then update person_verification data for these persons:

• drfo_data_id = null

• drfo_data_result = null

• drfo_synced_at = null

• drfo_verification_status = VERIFICATION_NEEDED

• drfo_verification_reason = ONLINE_TRIGGERED

• drfo_unverified_at = null (if not null)

Step 4.2. Update Parties

Invoke the Step if party_verifications.drfo_verification_status is not IN_REVIEW

If drfo data has been changed in Step 3 and Parties are linked to it (have the same drfo_data_id party_verifications table), then compare party’s data with drfo data: last_name, first_name, middle_name and date_birth. If these data are different,update party_verification data:

• drfo_data_id = null

• drfo_data_result = null

• drfo_synced_at = null

• drfo_verification_status = VERIFICATION_NEEDED

• drfo_verification_reason = ONLINE_TRIGGERED

• drfo_unverified_at = null (if not null)

Step 2.1. Update Person

Invoke the Step If person_verifications.drfo_data_verification_status still IN_REVIEW

If person is FEMALE with age > 16, then break job and set person_verification for it:

• If RNOKPP check in Step 2 returned RESULT that is not 0:

  • drfo_data_id = null

  • drfo_data_result = value according to DRFO_RESULT dictionary (negative for InfoRNOKPPDRFO service)

  • drfo_synced_at = current datetime

  • drfo_verification_status = NOT_VERIFIED

  • drfo_verification_reason = AUTO

  • drfo_unverified_at = current datetime

• If RNOKPP check in Step 2 was not successful (error):

  • rollback to previous drfo_verification_status and drfo_verification_reason (that was before IN_REVIEW set in Step 2)

Step 5. Call FindRegistrationDRFOQuery

Invoke the Step If person_verifications.drfo_data_verification_status still IN_REVIEW

If RNOKPP check in Step 2 was not successful (error) or RESULT returned is not 0. Try to find valid RNOKPP invoking RPC call with third-party service DRFO Trembita methods_ph2_EN | FindRegistrationDRFOQuery using following request params:

• document.series_number = mpi.person_documents.number. Get number of document with the latest issued_at with types: “PASSPORT”, “NATIONAL_ID”, “BIRTH_CERTIFICATE”, “BIRTH_CERTIFICATE_FOREIGN”, “PERMANENT_RESIDENCE_PERMIT”, “TEMPORARY_CERTIFICATE”. If one of these documents was used in Step 2, then chose another one at this Step. Documents with type “PASSPORT” or “NATIONAL_ID” are priority.

• person.last_name = mpi.persons.last_name

• person.first_name = mpi.persons.first_name

• person.middle_name = mpi.persons.second_name (skip if empty)

• person.date_birth = mpi.persons.birth_date

Step 6. Create Job

At this step an oban job creates. It awaits a response on application returned in Step 5 and includes steps described below.

Step 7. Call FindRegistrationDRFOAnswer

Invoke the Step If person_verifications.drfo_data_verification_status still IN_REVIEW

Invoke RPC call with third-party service DRFO Trembita methods_ph2_EN | FindRegistrationDRFOAnswer in a loop while getting error = 1002 (application in process). The FindRegistrationDRFOAnswer uses applicationNumber and applicationDate as parameters (were returned in Step 5.)

Step 8. Create or update drfo_data

If valid RNOKPP was found: application with Info.RNOKPP and Info.result=0 returned in Step 7. Get record from mimit.drfo_data table by rnokpp:

• if not found - create a record.

• if found - updated the record.

Step 9. Update Person

Invoke the Step If person_verifications.drfo_data_verification_status still IN_REVIEW

After valid DRFO was stored in Step 8 update Person in MPI db and set relation with DRFO data via person_verifications table.

For RNOKPP as passport number returned (match corresponding regex), set to persons record:

• no_tax_id = true (if it hasn’t been set to true before)

• tax_id = null (if it is not null)

For RNOKPP as tax ID returned, set to persons record:

• tax_id = RNOKPP (if it is different from tax_id)

• no_tax_id = false (if it hasn’t been set to false before).

Update person_verifications record by person_id:

• drfo_data_id = mimir.drfo_data.id

• drfo_data_result = 20. The value according to DRFO_RESULT dictionary.

• drfo_synced_at = date and time when synchronization has been performed (current datetime)

• drfo_verification_reason = AUTO

• drfo_verification_status:

  • set VERIFIED for rnokpp as tax ID or passport number (if non-expired document with such number exists in person documents) returned

  • set NOT_VERIFIED for rnokpp as passport number returned if non-expired document with such number not exists in person documents

• drfo_unverified_at:

  • if drfo_verification_status = VERIFIED then set to null (if not null)

  • if drfo_verification_status = NOT_VERIFIED then set to current datetime

Step 9.1. Update another Persons

The same logic as described in Step 4.1.

Step 9.2. Update Parties

The same logic as described in Step 4.2.

Step 10. Update Person

Invoke the Step If person_verifications.drfo_data_verification_status still IN_REVIEW

If person was identified as potentially deceased in DRFO registry (RNOKPP not received): application with Info.result = 5 returned in Step 7 - update Person record in MPI db, without relation to drfo_data. Update following person’s data:

Update person_verifications record by person_id:

• drfo_data_id = null (if it is not null)

• drfo_data_result = 25. The value according to DRFO_RESULT dictionary.

• drfo_synced_at = date and time when synchronization has been performed (current datetime)

• drfo_verification_status = NOT_VERIFIED

• drfo_verification_reason = AUTO

• drfo_unverified_at = current datetime

• dracs_death_verification_status = VERIFICATION_NEEDED

• dracs_death_verification_reason = ONLINE_TRIGGERED

• dracs_death_online_status = READY

Step 11. Update Person

Invoke the Step If person_verifications.drfo_data_verification_status still IN_REVIEW

If person identified as having closed RNOKPP in DRFO registry (RNOKPP not received): application with Info.result = 6 returned in Step 7 - update Person record in MPI db, without relation to drfo_data. Update following person’s data:

Update person_verifications record by person_id:

• drfo_data_id = null (if it is not null)

• drfo_data_result = 26. The value according to DRFO_RESULT dictionary.

• drfo_synced_at = date and time when synchronization has been performed (current datetime)

• drfo_verification_status = NOT_VERIFIED

• drfo_verification_reason = AUTO

• drfo_unverified_at = current datetime

Step 12. Update Person

Invoke the Step If person_verifications.drfo_data_verification_status still IN_REVIEW

If RNOKPP not found or person was not uniquely identified: application with Info.result in (1, 2) returned in Step 7 - update Person record in MPI db, without relation to drfo_data. Update verification data only:

• no_tax_id = null (if it is not null)

• tax_id = null (if it is not null)

Update person_verifications record by person_id:

• drfo_data_id = null (if it is not null)

• drfo_data_result = 21 or 22. The value according to DRFO_RESULT dictionary depending on the returned result

• drfo_synced_at = date and time when synchronization has been performed (current datetime)

• drfo_verification_reason = AUTO

• drfo_verification_status:

  • set NOT_VERIFIED if person age >= no_self_auth_age

  • set VERIFIED if person age < no_self_auth_age

• drfo_unverified_at:

  • if drfo_verification_status = VERIFIED then set to null (if not null)

  • if drfo_verification_status = NOT_VERIFIED then set to current datetime

Step 13. Technical error returned

If technical error: application with Info.result in (4,3), error in (1000, 1004), timeout, etc. returned in Step 7 - the Person is considered unprocessed.

If person_verifications.drfo_data_verification_status still IN_REVIEW:

• rollback to previous drfo_verification_status and drfo_verification_reason (that was before IN_REVIEW set in Step 2)

Log info

To find details about synchronization of specific person in the Elastic you can use following filters:

• request_id: 5539b7df-644b-410b-9bda-ef6c754036b9 - get info about person with person_verifications.id = 5539b7df-644b-410b-9bda-ef6c754036b9 within all drfo jobs

• request_id: 5539b7df-644b-410b-9bda-ef6c754036b9 AND request_id: persons_synchronization-* - get info about person with person_verifications.id = 5539b7df-644b-410b-9bda-ef6c754036b9 within all PersonsSynchronization oban jobs (InfoRNOKPPDRFO)

• request_id: 5539b7df-644b-410b-9bda-ef6c754036b9 AND request_id: get_registration_drfo_answer-* - get info about person with person_verifications.id = 5539b7df-644b-410b-9bda-ef6c754036b9 within all GetRegistrationDrfoAnswer oban jobs (FindRegistrationDRFOAnswer)

• request_id: "9d1f3c8e-bec7-452a-8e9f-cab7e8dcae58" AND (request_id: persons_synchronization-1234 OR request_id: get_registration_drfo_answer-4411) - get info about person with person_verifications.id = 5539b7df-644b-410b-9bda-ef6c754036b9 in PersonsSynchronization job with oban_job_id=1234 and in GetRegistrationDrfoAnswer job with oban_job_id=4411