Voice Activities Data Pipeline

Voice Activities Data Pipeline is responsible processing all the voice activities related data from CX. It extracts data from sources (MongoDB) applies necessary transformations to align with schema requirements, and loads the data into target database. The respective target database schema for this pipeline is given in the following document: Reporting Database Schema | Voice_activities . Key tasks include for this data pipeline include:

• Validating data fields for accuracy and ensuring required fields are present.

• Mapping data structure fields to the target schema.

• Handling data updates and upserts to maintain data integrity.

Changes required in the activities_data_pipeline_config.yaml file:

Query

A query in the yaml file is a dictionary containing query configurations for different pipelines. For voice activities data pipeline the query is as follows

• voice_activities: Configurations for the voice activities pipeline.

  • database: Name of the MongoDB database from where activities data is being extracted. Example: "conversation-manager_db".

  • collection_name: Name of the MongoDB collection. Example: "ConversationActivities".

  • filter: Query filter applied to fetch data. Example: {"activity.name": "VOICE_ACTIVITY"}

  • replication_key: Field used to track updates. Example: "recordCreationTime".

  • transformation: Transformation function name. Example: "transform_voice_activities_data".

  • num_batches: Number of data batches. Example: 50.

  • query_keys: Reserved for gold queries ( for loading data in gold table if needed ).