Professional Machine Learning Engineer Certification

Professional Cloud Data Engineer Exam Prep Workshop

DreamsPlus offers expert-led training in Singapore to prepare cloud engineers for Google’s Professional Cloud Data Engineer Exam. Guided by experienced trainers, you will explore the latest concepts and industry-leading best practices in cloud data engineering. This comprehensive program ensures you master the Professional Cloud Data Engineer curriculum and equips you to pass the exam with confidence. Take the first step to advancing your cloud engineering career in Singapore’s dynamic tech ecosystem.

Syllabus 

Section 1: Designing data processing systems (~22% of the exam)

1.1 Designing for security and compliance. Considerations include: 

• Identity and Access Management (such as organization policies and Cloud IAM)
• Data security, including key management and encryption
• Privacy (including Cloud Data Loss Prevention API and personally sensitive information)
• Regional factors (data sovereignty) pertaining to data storage and access
• Compliance with laws and regulations

 1.2 Designing for reliability and fidelity. Considerations include:

   ●  Preparing and cleaning data (e.g., Dataprep, Dataflow, and Cloud Data Fusion)

    ●  Monitoring and orchestration of data pipelines

    ●  Disaster recovery and fault tolerance

    ●  Making decisions related to ACID (atomicity, consistency, isolation, and durability) compliance and availability

    ●  Data validation

1.3 Designing for flexibility and portability. Considerations include:

• Data staging, cataloging, and discovery (data governance) 
•  Designing for data and application mobility (e.g., multi-cloud and data residency requirements) 
•  Mapping present and future business requirements to the architecture

1.4 Designing data migrations. Considerations include:

• Analyzing current stakeholder needs, users, processes, and technologies and creating a plan to get to desired state
• Planning migration to Google Cloud (e.g., BigQuery Data Transfer Service, Database Migration Service, Transfer Appliance, Google Cloud networking, Datastream)
• Designing the migration validation strategy
• Designing the project, dataset, and table architecture to ensure proper data governance 

Section 2: Ingesting and processing the data (~25% of the exam)

2.1 Planning the data pipelines. Considerations include:

• defining data transformation rationality 
• defining data sources and sources 
• Networking fundamentals
• Encrypting data

2.2 Building the pipelines. Considerations include:

• Data cleansing
• Identifying the services (e.g., Dataflow, Apache Beam, Dataproc, Cloud Data Fusion, BigQuery, Pub/Sub, Apache Spark, Hadoop ecosystem, and Apache Kafka)
• Transformations

        ○  Batch

        ○  Streaming (e.g., windowing, late arriving data)

        ○  Language

        ○  Ad hoc data ingestion (one-time or automated pipeline)

• Data acquisition and import
• Integrating with new data sources 

2.3 Deploying and operationalizing the pipelines. Considerations include:

• Automation and coordination of tasks (such as workflows and cloud composers)
• Continuous Integration and Deployment, or CI/CD

Section 3: Storing the data (~20% of the exam)

3.1 Selecting storage systems. Considerations include:

• Analyzing data access patterns
• Choosing managed services (e.g., Bigtable, Spanner, Cloud SQL, Cloud Storage, Firestore, Memorystore)
• Planning for storage costs and performance
• Lifecycle management of data

3.2 Planning for using a data warehouse. Considerations include:

• Creating the infrastructure to support data access patterns
•  Determining the level of data normalization 
• Charting business requirements

3.3 Using a data lake. Considerations include:

• processing the lake (setting up data discovery, access, and cost constraints)
• Processing data
• monitoring the data lake

3.4 Designing for a data mesh. Considerations include:

• Constructing a data mesh utilizing Google Cloud products (such as Dataplex, Data Catalog, BigQuery, and Cloud Storage) in accordance with criteria
•  Data segmentation for use by remote teams
• Constructing a Federated Governance Framework for Dispersed Information Systems

 Section 4: Preparing and using data for analysis (~15% of the exam)

4.1 Preparing data for visualization. Considerations include:

• Establishing connections to tools 
• Precalculating fields 
• BigQuery materialized views (view logic) 
• Determining the level of detail in temporal data 
• Resolving issues with poorly performing queries
• Identity and Access Management (IAM) and Cloud Data Loss Prevention (Cloud DLP)

4.2 Sharing data. Considerations include:

• Creating guidelines for data sharing 
• Disseminating datasets 
•  Disseminating reports and visualizations 
•  Analytics Hub 

4.3 Exploring and analyzing data. Considerations include:

• Data preparation for feature engineering (machine learning model training and serving) 
• Data discovery

Section 5: Maintaining and automating data workloads (~18% of the exam)

5.1 Optimizing resources. Considerations include:

• Minimizing costs per required business need for data
• Ensuring that enough resources are available for business-critical data processes
• Deciding between persistent or job-based data clusters (e.g., Dataproc)

5.2 Designing automation and repeatability. Considerations include:

• For Cloud Composer, creating directed acyclic graphs (DAGs) 
• Repeatedly scheduling jobs 

5.3 Organizing workloads based on business requirements. Considerations include:

• On-demand, flexible, and flat-rate slot pricing (based on fixed capacity or flexibility index)
• Interactive or bulk query assignments

5.4 Monitoring and troubleshooting processes. Considerations include:

• Monitoring scheduled consumption
• Troubleshooting error messages, billing difficulties, and quotas 
• Observability of data processes (e.g., cloud monitoring, cloud logging, BigQuery admin panel) 
• Managing workloads, such as jobs, queries, and computing capacity (reservations).

5.5 Maintaining awareness of failures and mitigating impact. Considerations include:

• Building a system with fault tolerance and restart management
•  Executing tasks across several zones or regions 
• Getting ready for data loss and corruption
• Replication and failover of data (such as Redis clusters and cloud SQL)