ETL Engineer

Компания "Балхаш Системс"

We have 30 years of expertise in designing and building custom software systems. We provide software development services focusing on complex high-load applications, AI and BI solutions, and mobile apps.

Project description:

You’ll be joining a DNA-Encoded Libraries (DEL) project for a major pharmaceutical company.

The team includes at least 4 developers and 3 scientists.

Tech Stack

The infrastructure includes:

• a large PostgreSQL database.
• ETL pipelines using:
  • Apache Airflow (Python-based orchestration)
  • AWS services (Lambda, Batch).
• a Streamlit-based web portal for users to enter or upload data.

All parts are closely connected, and you’ll contribute to each.

Responsibilities:

• Build new features in the Streamlit user portal.
• Work with scientists to gather and define requirements.
• Write and optimize complex SQL queries.
• Develop and maintain Airflow ETL pipelines.
• Build or updаtеAWS Lambda and Batch jobs.
• Ensure seamless integration across systems.

Required Skills

• Strong Python
• Familiarity with backend frameworks (Flask, FastAPI, etc.).
• Experience with PostgreSQL and writing complex SQL.
• Experience with Python ORMs (like SQLAlchemy, Peewee, Pony).
• Proficient with Apache Airflow (config, setup, custom DAGs).
• Some experience with Streamlit or similar tools.
• Experience with AWS Lambda and Batch.
• Familiarity with Jenkins, Docker, Kubernetes.

Nice to Have

• Experience in pharma.
• Experience specifically with DEL projects.

Our offer as your future employer:

• official registration
• flexible work schedule
• possibility to work remotely
• opportunities for professional growth

Открыть контакты работодателя

Зарегистрируйтесь или войдите, чтобы открыть контакты работодателя