skip to Main Content

I need to change the flavors "sklearn_version" in mlflow from "0.22.1" to "1.0.0" on azure machine learning when I log my trained model, since this model will be incompatible with the sklearn version that I am using for deployment during inference. I could change the version of sklearn in conda.yml file by setting "conda_env" in

mlflow.sklearn.log_model(conda_env= 'my_env')

enter image description here

here is the screen shot of requirements.txt
enter image description here

however, sklearn version under flavors in MLmodel file remains unchanged and that is the file that causes problem:

enter image description here

and here is script that I use to create this mlflow experiment in azure machine learning notebooks.

import mlflow
from sklearn.tree import DecisionTreeRegressor

from azureml.core import Workspace
from azureml.core.model import Model
from azureml.mlflow import register_model


def run_model(ws, experiment_name, run_name, x_train, y_train):
    
    # set up MLflow to track the metrics
    mlflow.set_tracking_uri(ws.get_mlflow_tracking_uri())
    mlflow.set_experiment(experiment_name)  
    
    with mlflow.start_run(run_name=run_name) as run:
        
        # fit model
        regression_model = DecisionTreeRegressor()
        regression_model.fit(x_train, y_train)
    
        # log training score 
        training_score = regression_model.score(x_train, y_train)
        mlflow.log_metric("Training score", training_score)

        my_conda_env = {
                    "name": "mlflow-env",
                    "channels": ["conda-forge"],
                    "dependencies": [
                        "python=3.8.5",
                        {
                            "pip": [
                                "pip",
                                "scikit-learn~=1.0.0",
                                "uuid==1.30",
                                "lz4==4.0.0",
                                "psutil==5.9.0",
                                "cloudpickle==1.6.0",
                                "mlflow",
                            ],
                        },
                    ],
                }

        
        # register the model
        mlflow.sklearn.log_model(regression_model, "model", conda_env=my_conda_env)

    model_uri = f"runs:/{run.info.run_id}/model"
    model = mlflow.register_model(model_uri, "sklearn_regression_model")

if __name__ == '__main__':

    # connect to your workspace
    ws = Workspace.from_config()

    # create experiment and start logging to a new run in the experiment
    experiment_name = "exp_name"

    # mlflow run name
    run_name= '1234'

  
    # get train data
    x_train, y_train  = get_train_data()
    
    run_model(ws, experiment_name, run_name, x_train, y_train)

Any idea how can change the flavor sklearn version in MLmodel file from "0.22.1" to "1.0.0" in my script?

With many thanks in advance!

2

Answers


  1. Chosen as BEST ANSWER

    I was finally able to solve this issue. Apparently flavors within mlflow MLfile use the version of installed scikit-learn within the workspace. all I needed to to was to upgrade the scikit-learn from cli within workspace.

    1. open cli in azure machine learning notebooks
    2. upgarde your intended flavor package. My case was scikit-learn, in my case I used 1.0.2:
     pip install --upgrade scikit-learn==1.0.2
    
    1. log your model in mlflow by running your training script one more time. enter image description here

  2. The modifications of the versions must be from "requirements.txt". We need to manually override the versions we need and move the build to the pipeline.

    Manually edit the following code

    conda_env = {
        'channels': ['conda-forge'],
        'dependencies': [
            'python=3.8.8',
            'pip'],
        'pip': [
            'mlflow',
            'scikit-learn==0.23.2',
            'cloudpickle==1.6.0'
        ],
        'name': 'mlflow-env'
    }
    mlflow.sklearn.log_model(model, "my_model", conda_env=conda_env)
    

    The below code should be in conda.yml

    name: mlflow-env
    channels:
      - conda-forge
    dependencies:
    - python=3.8.8
    - pip
    - pip:
      - mlflow
      - scikit-learn==0.23.2
      - cloudpickle==1.6.0
    

    The following code block must be there in python_env.yaml

    python: 3.8.8
    build_dependencies:
      - pip==21.1.3
      - setuptools==57.4.0
      - wheel==0.37.0
    dependencies:
      - -r requirements.txt
    

    The following thing must be there in requirements.txt

    mlflow
    scikit-learn==0.23.2
    cloudpickle==1.6.0
    
    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search