Note: The job is a remote job and is open to candidates in USA. Collins Aerospace is seeking an experienced Python and PySpark Developer to design, build, and optimize our next-generation big data pipelines. In this role, you will handle large-scale datasets, optimize distributed computing clusters, and bridge the gap between raw data ingestion and production-ready analytics.

Responsibilities

• Design and deploy robust batch and streaming ETL/ELT pipelines using PySpark and Python
• Optimize Spark jobs by tuning configurations, managing partitioning, and resolving data skew or OOM (Out of Memory) errors
• Implement modern data lakehouse architectures using Delta Lake, Iceberg, or Hudi
• Build and maintain complex workflow DAGs using orchestration tools like Apache Airflow
• Develop backend Python services or REST APIs (e.g., Fast API, Flask) to expose processed data to downstream applications
• Write clean, modular, and unit-tested code while participating in rigorous code reviews

Skills

• Must be a U.S. Citizen
• Strong proficiency in Python (OOP, concurrency, data structures) and advanced SQL
• Deep production experience with Apache Spark / PySpark (Data Frames, Spark SQL, RDDs)
• Hands-on experience with cloud data platforms like AWS (EMR, Glue), Azure (Databricks), or GCP
• Experience working with Snowflake, Big Query, Redshift, or Synapse
• Proficient with Git, Docker, and automated deployment pipelines
• PySpark MLlib or deploying Machine Learning models to production
• Familiarity with streaming technologies like Apache Kafka or Spark Structured Streaming
• Databricks Certified Data Engineer or Apache Spark Developer certifications

Benefits

• Medical, dental, and vision insurance
• Three weeks of vacation for newly hired employees
• Generous 401(k) plan that includes employer matching funds
• Participation in the Employee Scholar Program (ESP)
• Life insurance and disability coverage
• Employee Assistance Plan, including up to 8 free counseling sessions

Company Overview