Databrick architect

Tech Mahindra · Hyderabad, Telangana · Refreshed 26 days ago

About this role

JD: Location : Pan India • 8+yrs. in Azure Databricks technologies. • Hands on experience in Pyspark. • Good understanding of ETL , BI and DW technologies. • Hands-on experience on Jason or XML files as sources. • Experience in autoloader for incremental loading and explode options for flattening data • Experience in loading Delta live tables • Experience in PySpark or Spark using data frames. • Design, implement, and maintain data pipelines for data ingestion, processing, and transformation in Azure. • Work together with data scientists and analysts to understand the needs for data and create effective data workflows. • Create and maintain data storage solutions including Azure SQL Database, Azure Data Lake, and Azure Blob Storage. • Utilizing Azure Data Bricks to create and maintain ETL (Extract, Transform, Load) operations. • Implementing data validation and cleansing procedures will ensure the quality, integrity, and dependability of the data. • Improve the scalability, efficiency, and cost-effectiveness of data pipelines. • Monitoring and resolving data pipeline problems to guarantee consistency and availability of the data. • Good communication skills – Oral & Written Responsibilities • Create plans and drawings which clearly outline architectural design of project • Utilize knowledge of engineering, math, science and materials in order to create ideal finished product • Conduct site evaluations to ensure quality of work and adherence to safety standards • Oversee and m

Listings may come from company or aggregator sources. Confirm current availability and requirements on the original application page.