-->

⏱️ Estimated Reading Time: 12 minutes

Introduction

As the importance of AI/ML technology rapidly increases in modern enterprises, there is an explosive demand for engineers who can go beyond simply creating models to build ML systems that actually work in production environments.

The Made-With-ML project, which has received 40.6k stars on GitHub, provides a practical ML engineering guide that reflects this reality. This post is a hiring guide that outlines the essential competencies that ML engineers should possess in our company based on the core content of that project.

Why Do We Need This Guide?

Current Problems in the Hiring Market

# Problems with typical job postings
traditional_requirements = {
    "machine_learning": "Ability to use PyTorch, TensorFlow",
    "programming": "Python programming experience",
    "theory": "Knowledge of statistics and mathematical theory",
    "experience": "3 years of data analysis experience"
}

# What's actually needed in production
production_requirements = {
    "full_stack_ml": "End-to-end process from design to deployment to operations",
    "engineering": "Application of software engineering principles",
    "scalability": "Experience building large-scale systems",
    "reliability": "Ability to operate stable services"
}

New Paradigm Proposed by Made-With-ML

Made-With-ML presents how to build ML systems in actual production environments through a 4-stage process: Design · Develop · Deploy · Iterate.

Core Competency Checklist

1. Basic Programming & Development Environment (Foundation)

Essential Technology Stack

# Python ecosystem proficiency
python_skills = {
    "core": ["Python 3.8+", "Type hints", "Async/await"],
    "data_science": ["NumPy", "Pandas", "Scikit-learn"],
    "deep_learning": ["PyTorch", "Transformers", "Ray"],
    "web": ["FastAPI", "Pydantic", "Uvicorn"],
    "testing": ["pytest", "unittest", "coverage"],
    "packaging": ["pip", "conda", "poetry", "requirements.txt"]
}

Development Environment Management

  • Virtual environment management: conda, virtualenv, poetry
  • Dependency management: requirements.txt, pyproject.toml
  • Code quality: pre-commit hooks, linting, formatting
  • Jupyter environment: Efficient notebook utilization

2. Data Engineering

Data Pipeline Construction

# Data processing pipeline example
class DataProcessor:
    def __init__(self, config: DataConfig):
        self.config = config
    
    def extract_data(self) -> pd.DataFrame:
        """Extract data from various sources"""
        pass
    
    def transform_data(self, df: pd.DataFrame) -> pd.DataFrame:
        """Data preprocessing and transformation"""
        pass
    
    def validate_data(self, df: pd.DataFrame) -> bool:
        """Data quality validation"""
        pass
    
    def load_data(self, df: pd.DataFrame) -> None:
        """Save preprocessed data"""
        pass

Essential Competencies

  • Data quality management: Data validation, outlier detection
  • Streaming data processing: Real-time data pipelines
  • Data version control: Utilizing DVC, Git-LFS
  • Large-scale data processing: Distributed processing with Ray, Dask

3. Model Development & Training

Model Design Principles

# Model architecture design example
class ProductionModel:
    def __init__(self, config: ModelConfig):
        self.config = config
        self.model = self._build_model()
    
    def _build_model(self) -> torch.nn.Module:
        """Scalable model architecture"""
        pass
    
    def train(self, train_data: DataLoader) -> Dict[str, float]:
        """Reproducible training process"""
        pass
    
    def evaluate(self, test_data: DataLoader) -> Dict[str, float]:
        """Comprehensive model evaluation"""
        pass
    
    def save_model(self, path: str) -> None:
        """Save model checkpoint"""
        pass

Core Technologies

  • Distributed training: Ray Train, PyTorch Distributed
  • Hyperparameter tuning: Ray Tune, Optuna
  • Model optimization: Quantization, pruning, knowledge distillation
  • Experiment management: MLflow, Weights & Biases

4. Model Deployment & Serving

Production Serving Architecture

# FastAPI-based model serving
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
import ray
from ray import serve

@serve.deployment
@serve.ingress(app)
class MLService:
    def __init__(self, model_path: str):
        self.model = self.load_model(model_path)
    
    @app.post("/predict")
    async def predict(self, request: PredictionRequest) -> PredictionResponse:
        try:
            result = await self.model.predict(request.data)
            return PredictionResponse(prediction=result)
        except Exception as e:
            raise HTTPException(status_code=500, detail=str(e))

Deployment Strategies

  • Containerization: Docker, Kubernetes
  • API design: RESTful API, GraphQL
  • Load balancing: Ray Serve, Nginx
  • Monitoring: Prometheus, Grafana

5. MLOps & Infrastructure

CI/CD Pipeline

# GitHub Actions example
name: ML Pipeline
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Setup Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.9'
      - name: Install dependencies
        run: |
          pip install -r requirements.txt
      - name: Run tests
        run: |
          pytest tests/ --cov=madewithml --cov-report=xml
      - name: Data tests
        run: |
          pytest tests/data --dataset-loc=$DATASET_LOC
      - name: Model tests
        run: |
          pytest tests/model --run-id=$RUN_ID

Operations Management

  • Model version control: Model registry, A/B testing
  • Performance monitoring: Latency, throughput, accuracy tracking
  • Incident response: Rollback, canary deployment
  • Cost optimization: Auto-scaling, resource management

Practical Experience Evaluation Criteria

1. Junior Level (1-2 years experience)

Essential Competencies

junior_requirements = {
    "programming": {
        "python": "Basic syntax, object-oriented programming",
        "data_handling": "Pandas, NumPy utilization",
        "visualization": "Matplotlib, Seaborn usage"
    },
    "ml_basics": {
        "algorithms": "Understanding supervised/unsupervised learning",
        "frameworks": "Scikit-learn, PyTorch basics",
        "evaluation": "Cross-validation, performance metrics understanding"
    },
    "tools": {
        "version_control": "Basic Git commands",
        "environment": "Jupyter, VSCode utilization",
        "documentation": "README writing, code comments"
    }
}

Sample Evaluation Problems

# Data preprocessing and model training
def evaluate_junior_candidate():
    """
    Task to create a classification model with given dataset
    and evaluate performance
    """
    tasks = [
        "Data exploration and visualization",
        "Build preprocessing pipeline",
        "Model training and evaluation",
        "Result interpretation and improvement suggestions"
    ]
    return tasks

2. Mid-Level (3-5 years experience)

Essential Competencies

mid_level_requirements = {
    "engineering": {
        "architecture": "Modularization, class design",
        "testing": "Unit testing, integration testing",
        "optimization": "Code optimization, profiling"
    },
    "ml_engineering": {
        "pipelines": "End-to-end pipeline construction",
        "deployment": "API serving, container deployment",
        "monitoring": "Model performance tracking"
    },
    "collaboration": {
        "code_review": "Code review participation",
        "documentation": "Technical documentation writing",
        "mentoring": "Junior developer guidance"
    }
}

Sample Evaluation Problems

# Production system design
def evaluate_mid_level_candidate():
    """
    Task to design and implement ML system
    deployable to actual service
    """
    tasks = [
        "System architecture design",
        "API endpoint implementation",
        "Test code writing",
        "Docker container configuration",
        "Monitoring dashboard construction"
    ]
    return tasks

3. Senior Level (5+ years experience)

Essential Competencies

senior_requirements = {
    "leadership": {
        "architecture": "System architecture design",
        "decision_making": "Leading technical decision-making",
        "team_leading": "Team leadership experience"
    },
    "advanced_ml": {
        "research": "Understanding latest research trends",
        "optimization": "Large-scale system optimization",
        "innovation": "Introducing new technologies"
    },
    "business": {
        "strategy": "Connecting business goals with technology",
        "communication": "Communication with non-developers",
        "project_management": "Project management"
    }
}

Interview Question Guide

1. Technical Interview Questions

System Design Questions

# Large-scale recommendation system design
interview_questions = {
    "system_design": [
        "Design a recommendation system for 100 million users",
        "How would you build an online learning system that reflects real-time feedback?",
        "Explain a model deployment strategy for A/B testing"
    ],
    "optimization": [
        "How would you reduce model inference latency to under 10ms?",
        "How would you solve GPU memory shortage problems?",
        "How to minimize communication overhead in distributed training?"
    ],
    "troubleshooting": [
        "How would you respond when model performance suddenly drops in production?",
        "How to detect and respond to data drift?",
        "How would you evaluate and improve bias in model prediction results?"
    ]
}

2. Coding Test Problems

Practical Coding Challenges

# Model serving API implementation
class CodingChallenge:
    def __init__(self):
        self.tasks = [
            "Implement model serving API with FastAPI",
            "Input data validation and preprocessing",
            "Batch prediction optimization",
            "Error handling and logging",
            "Unit test writing"
        ]
    
    def evaluate_solution(self, solution: str) -> Dict[str, float]:
        """
        Evaluate solution based on:
        - Code quality (30%)
        - Performance optimization (25%)
        - Error handling (20%)
        - Test code (15%)
        - Documentation (10%)
        """
        pass

Team Role Definitions

1. ML Research Engineer

research_engineer_role = {
    "primary_focus": "Research and implementation of new model architectures",
    "required_skills": [
        "Paper implementation ability",
        "Experiment design and analysis",
        "Advanced PyTorch utilization",
        "Distributed training experience"
    ],
    "deliverables": [
        "SOTA model benchmarks",
        "Custom model architectures",
        "Experiment result analysis reports"
    ]
}

2. ML Platform Engineer

platform_engineer_role = {
    "primary_focus": "ML infrastructure and platform construction",
    "required_skills": [
        "Kubernetes, Docker",
        "Cloud services (AWS, GCP)",
        "CI/CD pipelines",
        "Monitoring systems"
    ],
    "deliverables": [
        "ML pipeline automation",
        "Model deployment systems",
        "Performance monitoring dashboards"
    ]
}

3. MLOps Engineer

mlops_engineer_role = {
    "primary_focus": "Model lifecycle management",
    "required_skills": [
        "MLflow, Kubeflow",
        "Model version control",
        "A/B test design",
        "Data drift monitoring"
    ],
    "deliverables": [
        "Model registry management",
        "Automatic retraining systems",
        "Performance degradation alert systems"
    ]
}

Learning Roadmap

1. Step-by-Step Learning Guide

Phase 1: Foundation (2-3 months)

foundation_learning_path = {
    "week_1_4": {
        "topic": "Python & Basic Tools",
        "resources": [
            "Advanced Python syntax",
            "Git & GitHub",
            "Jupyter Lab",
            "VS Code setup"
        ]
    },
    "week_5_8": {
        "topic": "Data Science Basics",
        "resources": [
            "NumPy, Pandas",
            "Matplotlib, Seaborn",
            "Statistics basics",
            "Data preprocessing"
        ]
    },
    "week_9_12": {
        "topic": "Machine Learning Basics",
        "resources": [
            "Scikit-learn",
            "Model evaluation",
            "Cross-validation",
            "Hyperparameter tuning"
        ]
    }
}

Phase 2: Intermediate (3-4 months)

intermediate_learning_path = {
    "month_1": {
        "topic": "Deep Learning & PyTorch",
        "projects": [
            "Image classification model",
            "Natural language processing basics",
            "Transfer learning practice"
        ]
    },
    "month_2": {
        "topic": "Web Development & API",
        "projects": [
            "FastAPI basics",
            "Model serving API",
            "Database integration"
        ]
    },
    "month_3": {
        "topic": "Containers & Deployment",
        "projects": [
            "Docker basics",
            "Model containerization",
            "Cloud deployment"
        ]
    }
}

Phase 3: Advanced (4-6 months)

advanced_learning_path = {
    "quarter_1": {
        "topic": "Distributed Systems & Scaling",
        "projects": [
            "Ray distributed training",
            "Kubernetes deployment",
            "Monitoring systems"
        ]
    },
    "quarter_2": {
        "topic": "Production Systems",
        "projects": [
            "Made-With-ML project implementation",
            "CI/CD pipelines",
            "A/B testing systems"
        ]
    }
}

Practical Project Portfolio

1. Essential Project List

Junior Portfolio

junior_portfolio = {
    "project_1": {
        "title": "End-to-end classification project",
        "description": "From data collection to model deployment",
        "tech_stack": ["Python", "Pandas", "Scikit-learn", "FastAPI"],
        "github_example": "https://github.com/your-repo/classification-project"
    },
    "project_2": {
        "title": "Deep learning image classifier",
        "description": "CNN model using PyTorch",
        "tech_stack": ["PyTorch", "torchvision", "Streamlit"],
        "github_example": "https://github.com/your-repo/image-classifier"
    }
}

Mid-Level Portfolio

mid_level_portfolio = {
    "project_1": {
        "title": "Distributed training system",
        "description": "Large-scale data distributed processing",
        "tech_stack": ["Ray", "PyTorch", "Docker"],
        "github_example": "https://github.com/your-repo/distributed-training"
    },
    "project_2": {
        "title": "Real-time recommendation system",
        "description": "Streaming data processing",
        "tech_stack": ["Kafka", "Redis", "FastAPI", "Kubernetes"],
        "github_example": "https://github.com/your-repo/recommendation-system"
    }
}

2. Project Evaluation Criteria

Code Quality Checklist

def evaluate_project_quality(project_repo: str) -> Dict[str, bool]:
    """Project quality evaluation checklist"""
    criteria = {
        "code_structure": "Modularized code structure",
        "documentation": "README, docstring completeness",
        "testing": "Unit test coverage > 80%",
        "ci_cd": "GitHub Actions setup",
        "containerization": "Docker container configuration",
        "monitoring": "Logging and monitoring implementation",
        "scalability": "Scalable architecture"
    }
    return criteria

Hiring Process Guide

1. Document Screening Checklist

Resume Evaluation Criteria

resume_evaluation_criteria = {
    "education": {
        "preferred": ["Computer Science", "Statistics", "Mathematics", "Physics"],
        "alternative": ["Bootcamp", "Online courses", "Certifications"],
        "weight": 20
    },
    "experience": {
        "ml_projects": "Number of ML project experiences",
        "production_experience": "Production deployment experience",
        "collaboration": "Team project experience",
        "weight": 40
    },
    "technical_skills": {
        "programming": "Python, SQL proficiency",
        "ml_frameworks": "PyTorch, TensorFlow experience",
        "cloud_platforms": "AWS, GCP, Azure experience",
        "weight": 30
    },
    "portfolio": {
        "github_activity": "GitHub activity frequency",
        "project_quality": "Project completeness",
        "documentation": "Documentation level",
        "weight": 10
    }
}

2. Technical Interview Process

3-Stage Interview Structure

interview_process = {
    "stage_1": {
        "type": "Technical screening",
        "duration": "1 hour",
        "format": "Video interview",
        "content": [
            "Basic concept questions",
            "Coding test (live coding)",
            "Project experience sharing"
        ]
    },
    "stage_2": {
        "type": "Advanced technical interview",
        "duration": "2 hours",
        "format": "In-person or video",
        "content": [
            "System design problems",
            "Practical situation problem solving",
            "Code review session"
        ]
    },
    "stage_3": {
        "type": "Cultural fit & final interview",
        "duration": "1 hour",
        "format": "Meeting with team members",
        "content": [
            "Teamwork and communication skills",
            "Learning willingness and growth mindset",
            "Company vision alignment"
        ]
    }
}

Conclusion and Next Steps

Key Points Summary

  1. Practical competencies: Problem-solving ability in actual production environments rather than theoretical knowledge
  2. End-to-end pipeline understanding: Understanding from data collection to model deployment and monitoring
  3. Software engineering: Fundamentals like code quality, testing, documentation, version control
  4. Continuous learning: Ability to adapt to rapidly changing technology trends

Action Items for Recruiters

action_items_for_recruiters = [
    "Update existing JD based on Made-With-ML standards",
    "Build technical interview question bank",
    "Establish coding test platform",
    "Include practical projects in onboarding program",
    "Plan regular technology trend update sessions"
]

Learning Roadmap for Candidates

learning_roadmap_for_candidates = [
    "Complete Made-With-ML project",
    "Complete 3+ personal portfolio projects",
    "Open source project contribution experience",
    "Technical blog operation or presentation experience",
    "Accumulate actual production deployment experience"
]

References

This guide is structured around the practical approach of the Made-With-ML project, focusing on the competencies needed in actual production environments. We will continue to update it and incorporate feedback from the field.

참고