{"id":71,"date":"2025-06-29T03:14:04","date_gmt":"2025-06-29T03:14:04","guid":{"rendered":"https:\/\/aiopsschool.com\/blog\/?p=71"},"modified":"2025-06-29T03:14:05","modified_gmt":"2025-06-29T03:14:05","slug":"what-is-seldon-core","status":"publish","type":"post","link":"https:\/\/aiopsschool.com\/blog\/what-is-seldon-core\/","title":{"rendered":"What is Seldon Core"},"content":{"rendered":"\n

Seldon Core<\/strong> is an open-source platform for deploying, scaling, and managing machine learning models on Kubernetes<\/strong>. It is designed to help organizations move ML models from experimentation to production<\/strong> efficiently, providing advanced tools for inference, monitoring, and governance.<\/p>\n\n\n\n

\n\n\n\n

Key Points: What is Seldon Core?<\/strong><\/h2>\n\n\n\n
    \n
  • Kubernetes-native ML Model Serving:<\/strong>
    Seldon Core uses Kubernetes to deploy and manage machine learning models as scalable, production-ready services.<\/li>\n\n\n\n
  • Framework Agnostic:<\/strong>
    You can deploy models trained in any popular framework<\/strong> (e.g., TensorFlow, PyTorch, scikit-learn, XGBoost, H2O, ONNX, custom code, etc.).<\/li>\n\n\n\n
  • Advanced Inference Graphs:<\/strong>
    Unlike basic model servers, Seldon Core allows you to create inference graphs<\/strong>\u2014complex pipelines of pre-processing, prediction, post-processing, A\/B testing, ensembles, etc.<\/li>\n\n\n\n
  • Automatic Scaling:<\/strong>
    Leverages Kubernetes features for auto-scaling<\/strong>, high availability<\/strong>, and resource management<\/strong>.<\/li>\n\n\n\n
  • Production Features:<\/strong>\n
      \n
    • Canary & A\/B Deployments:<\/strong> Safely test new model versions.<\/li>\n\n\n\n
    • Rolling Updates:<\/strong> Update models without downtime.<\/li>\n\n\n\n
    • Advanced Routing:<\/strong> Direct requests by rules\/weights.<\/li>\n\n\n\n
    • Monitoring & Metrics:<\/strong> Integrates with Prometheus, Grafana, and supports model explainability, outlier, drift detection, etc.<\/li>\n<\/ul>\n<\/li>\n\n\n\n
    • Standardized APIs:<\/strong>
      REST and gRPC endpoints for easy integration with your applications.<\/li>\n\n\n\n
    • Custom Components:<\/strong>
      You can add your own logic as pre\/post processors or combine multiple models\/algorithms into a pipeline.<\/li>\n<\/ul>\n\n\n\n
      \n\n\n\n

      How Does Seldon Core Work?<\/strong><\/h2>\n\n\n\n
        \n
      1. You define a deployment<\/strong> using a Kubernetes Custom Resource called a SeldonDeployment<\/code> (YAML file).<\/li>\n\n\n\n
      2. Seldon Core spins up the appropriate pods (containers) for each model and pipeline step.<\/li>\n\n\n\n
      3. Models are exposed via REST\/gRPC endpoints.<\/li>\n\n\n\n
      4. Traffic can be split, routed, monitored, and scaled\u2014all with Kubernetes primitives.<\/li>\n<\/ol>\n\n\n\n

        Simple Example: SeldonDeployment YAML<\/strong><\/h3>\n\n\n\n
        apiVersion: machinelearning.seldon.io\/v1\nkind: SeldonDeployment\nmetadata:\n  name: my-model\nspec:\n  predictors:\n  - name: default\n    replicas: 1\n    graph:\n      children: []\n      implementation: SKLEARN_SERVER\n      modelUri: gs:\/\/my-bucket\/sklearn-model\/\n      name: classifier\n<\/code><\/pre>\n\n\n\n
        \n\n\n\n
        Why Use Seldon Core?<\/strong><\/h2>\n\n\n\n
          \n
        • Deploy any ML model in any language or framework.<\/strong><\/li>\n\n\n\n
        • Manage model lifecycles<\/strong> with updates, versioning, and traffic control.<\/li>\n\n\n\n
        • Monitor model health and data drift<\/strong> in production.<\/li>\n\n\n\n
        • Scale to thousands of models,<\/strong> as needed for real-world enterprise use.<\/li>\n<\/ul>\n\n\n\n
          \n\n\n\n
          Comparison: Seldon Core vs. Others<\/strong><\/h2>\n\n\n\n
          Tool<\/th>Multi-Framework<\/th>Inference Graphs<\/th>Canary\/A\/B<\/th>K8s Native<\/th>Monitoring\/Explainability<\/th><\/tr><\/thead>
          Seldon Core<\/td>Yes<\/td>Yes<\/td>Yes<\/td>Yes<\/td>Yes<\/td><\/tr>
          KFServing<\/td>Yes<\/td>Basic<\/td>Yes<\/td>Yes<\/td>Yes<\/td><\/tr>
          TensorFlow Serving<\/td>No<\/td>No<\/td>No<\/td>No<\/td>No<\/td><\/tr>
          TorchServe<\/td>No<\/td>No<\/td>No<\/td>No<\/td>No<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
          \n\n\n\n
          When Should You Use Seldon Core?<\/strong><\/h2>\n\n\n\n
            \n
          • When you need to serve, scale, and manage ML models in Kubernetes production<\/strong>.<\/li>\n\n\n\n
          • For multi-framework model serving<\/strong>.<\/li>\n\n\n\n
          • If you want to build inference graphs<\/strong> (pipelines, ensembles, A\/B testing).<\/li>\n\n\n\n
          • When you need enterprise features<\/strong>: traffic splitting, monitoring, explainability, or drift detection.<\/li>\n<\/ul>\n\n\n\n
            \n\n\n\n
            <\/p>\n","protected":false},"excerpt":{"rendered":"
            Seldon Core is an open-source platform for deploying, scaling, and managing machine learning models on Kubernetes. It is designed to […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-71","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/aiopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/71","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/aiopsschool.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/aiopsschool.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/aiopsschool.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/aiopsschool.com\/blog\/wp-json\/wp\/v2\/comments?post=71"}],"version-history":[{"count":1,"href":"https:\/\/aiopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/71\/revisions"}],"predecessor-version":[{"id":72,"href":"https:\/\/aiopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/71\/revisions\/72"}],"wp:attachment":[{"href":"https:\/\/aiopsschool.com\/blog\/wp-json\/wp\/v2\/media?parent=71"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/aiopsschool.com\/blog\/wp-json\/wp\/v2\/categories?post=71"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/aiopsschool.com\/blog\/wp-json\/wp\/v2\/tags?post=71"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}