Kategorien durchsuchen
Entdecken
Fiverr Pro
Deutsch
$
USD
I will migrate your data pipeline to medallion architecture
Data Engineer, Databricks and Fabric Certified, 4 years Experience
Data Engineer with 4 years of experience building enterprise data solutions. Databricks Certified Data Engineer Associate and Microsoft Fabric Certified Data Engineer Associate. I specialize in produc...
Über diesen Service
Databricks Certified Data Engineer | Medallion Architecture Specialist
Struggling with messy data pipelines? I'll migrate your data to scalable Medallion Architecture (Bronze-Silver-Gold) on Databricks.
WHAT YOU GET:
- Bronze Layer: Raw data ingestion from databases, cloud storage, APIs
- Silver Layer: Cleaned, deduplicated data with quality checks
- Gold Layer: Business-ready analytics tables with aggregations
- Delta Lake for ACID transactions and time travel
- Orchestration setup (Airflow/Azure Data Factory)
- Complete documentation and diagrams
WHY MEDALLION?
- Separates raw, processed, and analytics-ready data
- Easy debugging and lineage tracking
- Incremental processing reduces costs
- Scalable for batch and real-time workloads
MY EXPERTISE:
- 4+ years data engineering
- Databricks Certified Associate Developer
- Built production pipelines for B2B sales and e-commerce
- Proficient in PySpark, Python, SQL, Azure, AWS
WHAT I NEED:
- Current data sources and formats
- Business metrics to track
- Access credentials (securely shared)
Transform your data chaos into organized lakehouse! Order now.
Tools und Plattformen:
Azure Data Factory
•
Andere
Mein Portfolio
Meine weiteren Dienstleistungen im Bereich Datentechnik
FAQ
What data sources can you connect to?
I work with databases (PostgreSQL, MySQL, SQL Server), cloud storage (S3, Azure Blob, GCS), data warehouses (Snowflake, Synapse), and APIs. If you have a custom source, message me first to confirm compatibility.
Do I need a Databricks account already?
Yes, you'll need an active Databricks workspace (AWS, Azure, or GCP). If you don't have one, I can guide you through the setup, but the subscription cost is separate from my service.
What's the difference between Bronze, Silver, and Gold layers?
Bronze = raw data as-is from sources. Silver = cleaned, validated, deduplicated data. Gold = business-ready analytics tables with aggregations and joins. This separation makes debugging easier and improves performance.
Will the pipeline run automatically after delivery?
Yes! I'll set up orchestration (Airflow or Azure Data Factory) so your pipeline runs on a schedule (daily, hourly, etc.). You'll also get monitoring alerts for failures.
What if my data volume is very large?
I optimize for performance using partitioning, caching, and incremental loads. For datasets over 1TB or complex transformations, message me before ordering so I can assess if Premium tier or custom pricing is needed.
