MLFLOW 快速教學、Tracking Server建立

 

整體架構可參考上圖，mlflow可以針對你模型訓練的參數、loss與model進行追蹤與管理的工具，也提供deploy api來對續的部署進行串接。本文將會快速地介紹如何建立一個tracking server，若只是要本地端的試用，直接參考官方文件的quickstart即可。

其架構大致如上圖，client訓練模型可以log參數、loss、把訓練好的模型上傳，參數部分可存到本地端、或資料庫，model需要有文件儲存空間，像是S3、Azure blob、FTP server等等。

Server設定

Conda環境下安裝python套件

pip install mlflow

pip install azure-storage-blob

pip install mysqlclient

安裝MYSQL建立user

create user 'mlflow'@'%' identified by 'chase0024’;

Blob設定環境變數

Windows(cmd)

setx AZURE_STORAGE_CONNECTION_STRING "<yourconnectionstring>"

setx AZURE_STORAGE_ACCESS_KEY "<youraccesskey>"

Linux(bash)

export AZURE_STORAGE_CONNECTION_STRING="<yourconnectionstring>"

export AZURE_STORAGE_ACCESS_KEY "<youraccesskey>"

啟動Server

mlflow server --backend-store-uri mysql://dbname:password@127.0.0.1:3306/mlflow --default-artifact-root wasbs://containername@blobaccount.blob.core.windows.net/ --host=0.0.0.0

 

Client設定

Conda環境下安裝python套件

pip install mlflow

pip install azure-storage-blob

Blob設定環境變數

Windows(cmd)

setx AZURE_STORAGE_CONNECTION_STRING "<yourconnectionstring>"

setx AZURE_STORAGE_ACCESS_KEY "<youraccesskey>"

Linux(bash)

export AZURE_STORAGE_CONNECTION_STRING="<yourconnectionstring>"

export AZURE_STORAGE_ACCESS_KEY "<youraccesskey>"

 

範例語法

以sklearn為例

import numpy as np

from sklearn.linear_model import LogisticRegression

 

import mlflow

import mlflow.sklearn

 

if __name__ == "__main__":

    remote_server_uri = "http://192.168.105.55:5000/" # set to your server URI

    mlflow.set_tracking_uri(remote_server_uri)

    ARTIFACT_URI = “wasbs://mlflow-test@wasteresumedata.blob.core.windows.net/” #設定blob

    EXPERIMENT_NAME = “experiment_Tracking” #實驗名稱區分大小寫且不能重複

    mlflow.create_experiment(EXPERIMENT_NAME, artifact_location=ARTIFACT_URI)

    mlflow.set_experiment(EXPERIMENT_NAME)

    X = np.array([-2, -1, 0, 1, 2, 1]).reshape(-1, 1)

    y = np.array([0, 0, 1, 1, 1, 0])

    lr = LogisticRegression()

    lr.fit(X, y)

    score = lr.score(X, y)

    print("Score: %s" % score)

    mlflow.log_metric("score", score)

    mlflow.sklearn.log_model(lr, "test")

    #mlflow.sklearn.save_model(lr, "my_model")

    print("Model saved in run %s" % mlflow.active_run().info.run_uuid)

紀錄該筆訓練的參數、loss

可透過語法直接取用線上model座使用