Add Support for Pre-Approved Datasets

[elmiomar]

Overview

This PR updates the record handling logic in our distribution service to differentiate between datasets that require SME approval and those that are pre-approved.

Changes

• Updated the createRecord() method to first create records in Salesforce then handle all post-processing (caching, emailing) based on the dataset type, only after successful creation.
• Added a new RECORD_PRE_APPROVED_STATUS, which is used for datasets that are pre-approved, allowing them to be cached and sent to the user without SME intervention.
• Leveraged the RPA configuration to identify pre-approved datasets; each dataset ID is mapped to a set of approvers in the configuration file. If no approvers are specified for a given dataset ID, it is considered pre-approved. This is just temporary until we add the new metadata fields.
• Added a new method in RecordResponseHandlerImpl to handle pre-approved datasets by sending both confirmation and download emails in a single step.

Updated workflow

• User submits request to download dataset.
• createRecord will be called with user info.
• Blacklist Check: rejects user if email/country is blacklisted.
• Dataset Type Check: sets RECORD_PRE_APPROVED_STATUS if pre-approved; otherwise, RECORD_PENDING_STATUS.
• Salesforce Record Creation: sends request to create a record in Salesforce.
• Post-Processing:
  • Rejected: ends process.
  • Pre-approved: caches dataset, sends confirmation and download emails.
  • SME-required: sends confirmation email to user and approval email request to SME.
• SME Approval (when required):
  • Approve: caches dataset, sends download link to user.
  • Decline: updates status to declined, notifies user.

Testing

• Tested end-to-end using oar-docker. I was able to download files, only individually.