Job Description

TITLE: Data Engineer

LOCATION: Scottsdale (AZ) or Chicago (IL), will not pay for parking in Chicago.--- F2F Interview
Duration :: Contract

Need Visa Independent Consultants (No Employer)

INTERVIEW MODE:

1. Video with Hiring Manager
2. Onsite in-person with team members
3. Video

MUST HAVES:

• Need to be familiar with SAS and able to convert in Spark and Python.
• Ability to migrate data analysis, data manipulation, and statistical modeling tasks from SAS to Python.
• (Critical to have experience) Hadoop Admin.
• (Critical to have experience) Spark Admin.
• (Critical to have experience) Linux.
• (Critical to have experience) Windows.
• (Nice to have experience) AWS.
• (Nice to have experience) Tableau.

QUESTIONS THAT NEED TO BE ANSWERED BY CANDIDATE: Submission summaries need to address the "Must Haves" and "Nice To Have"

JOB DESCRIPTION:

This role will be interfacing directly with Data Scientists, etc, and will need to be personable and have the ability to work issues and deal with people. It's a lot of tech, but it's a lot of internal coordinating and communicating.

The Data Science & Analytics team is undergoing a long-term tech transformation and migration effort. The team has many assets (data processing, reports, model development scripts) that have been developed using legacy software, particularly SAS. Very few people on the team are cross trained in both SAS and modern higher performance methods such as Spark or optimized Python.

The Data Science and Analytics teams as well as the ML Ops team are fully committed, and need to augment our resources with external support to

1. Help convert legacy code-based assets to modern high-performance tools (SAS to Python)
   1. Existing data processing scripts including data movement, cleaning, and aggregation
   2. Value Testing Process. This scores a potential customers data through our models to help determine the value of EWS solutions
   3. SQL/Hive query performance tuning and enhancement
2. Develop shared toolkit to automate certain data science processes
   1. Data profiling
   2. Feature importance and effectiveness evaluation
3. Automate documentation of model development processes
4. Assist in upskilling existing team

Project Specifics

1. Code Modernization for VT, MV&P, DS, DICA and CIR teams on existing programs/processes
   1. from SAS/Hive to Python/Scala/Pyspark/SQL or other modern highly efficient technology that fits the Early Warning's current on-prem environment and set up an easy conversion path for future state in ADP/Model Factory
   2. coordinate with MLOps team to onboard new data sources that exist in SAS environment but not in Newton
   3. For new VTs, work with the relevant parties to ensure Project plans account for MLOps engagement to build the capability (other processes potentially as well key capabilities in general can be requested to be built by MLOps from scratch)
   4. Training team to ensure proper adoption/transition to the team
2. Hive code efficiency evaluation and modernization
   1. Evaluate legacy repeated Hive queries commonly used by the analytics community
   2. Upgrade the legacy code to Scala/Pyspark or other modern highly efficient technology that fits the Early Warning's current on-prem environment and set up an easy conversion path for future state in ADP/Model Factory
   3. Training team to ensure proper adoption/transition to the team
3. Analytics ToolKit / Capability (shared among all teams)
   1. When existing open-source packages not available or not fitting our modeling need, Create standard, re-usable, highly efficient procedures for end-to-end model development, validation and evaluation, for example:
      1. Data profiling tool (evaluate data missing, value ranges, outlier, categorical features etc.)
      2. Feature effectiveness triaging toolset for XGBoost or other non-transparent models
      3. Provide standard generation of outputs of various model stages that aligns with model governance documentation requirements.
   2. Provide a template for efficient python-based project structure that enables efficient run, test, debug and deploy pipeline.
   3. Engage with MLOps for design, code review and approval this is within MLOps roles/resp but this SOW will help to bridge the short term resource gap
4. Report Automation
   1. Replace the current SAS/VisualBasic process with automate standard report automation using the modeling outputs. Collaborate with the tech writer and analytics team to standardize template and output. This include both validation report and initial model development report (auto-inserted with template), this may depend on when we have a DR replacement
   2. Engage with MLOps for design, code review and approval this is within MLOps roles/resp but this SOW will help to bridge the short term resource gap
5. Training / Upskilling Analytics Teams
   1. Create training/onboarding materials and provide hands-on practice training environment with target adoption outcome
   2. Work with Corp Learning & Development to develop programming training path using existing platforms and tools (LinkedIn Learning and Udemy)
   3. Provide office hour and troubleshooting support
   4. Conduct regular code guidance for the team in partnership with MLOps
6. Day 1 Monitoring Script

Create Day 1 model monitoring script when MLOps resource are not available

Please share your Updated Resume @ yashwant@kanakits.com

Job Tags

Similar Jobs