MLFLOW 快速教學、Tracking Server建立
整體架構可參考上圖,mlflow可以針對你模型訓練的參數、loss與model進行追蹤與管理的工具,也提供deploy api來對續的部署進行串接。本文將會快速地介紹如何建立一個tracking server,若只是要本地端的試用,直接參考官方文件的quickstart即可。
其架構大致如上圖,client訓練模型可以log參數、loss、把訓練好的模型上傳,參數部分可存到本地端、或資料庫,model需要有文件儲存空間,像是S3、Azure blob、FTP server等等。
Server設定
Conda環境下安裝python套件
pip install mlflow
pip install azure-storage-blob
pip install mysqlclient
安裝MYSQL建立user
create user 'mlflow'@'%' identified by
'chase0024’;
Blob設定環境變數
Windows(cmd)
setx AZURE_STORAGE_CONNECTION_STRING
"<yourconnectionstring>"
setx AZURE_STORAGE_ACCESS_KEY
"<youraccesskey>"
Linux(bash)
export
AZURE_STORAGE_CONNECTION_STRING="<yourconnectionstring>"
export AZURE_STORAGE_ACCESS_KEY
"<youraccesskey>"
啟動Server
mlflow server --backend-store-uri
mysql://dbname:password@127.0.0.1:3306/mlflow --default-artifact-root
wasbs://containername@blobaccount.blob.core.windows.net/ --host=0.0.0.0
Client設定
Conda環境下安裝python套件
pip install mlflow
pip install azure-storage-blob
Blob設定環境變數
Windows(cmd)
setx AZURE_STORAGE_CONNECTION_STRING
"<yourconnectionstring>"
setx AZURE_STORAGE_ACCESS_KEY
"<youraccesskey>"
Linux(bash)
export
AZURE_STORAGE_CONNECTION_STRING="<yourconnectionstring>"
export AZURE_STORAGE_ACCESS_KEY
"<youraccesskey>"
範例語法
以sklearn為例
import numpy as np
from sklearn.linear_model import
LogisticRegression
import mlflow
import mlflow.sklearn
if __name__ == "__main__":
remote_server_uri = "http://192.168.105.55:5000/" # set to
your server URI
mlflow.set_tracking_uri(remote_server_uri)
ARTIFACT_URI =
“wasbs://mlflow-test@wasteresumedata.blob.core.windows.net/” #設定blob
EXPERIMENT_NAME = “experiment_Tracking” #實驗名稱區分大小寫且不能重複
mlflow.create_experiment(EXPERIMENT_NAME, artifact_location=ARTIFACT_URI)
mlflow.set_experiment(EXPERIMENT_NAME)
X = np.array([-2, -1, 0, 1, 2, 1]).reshape(-1, 1)
y = np.array([0, 0, 1, 1, 1, 0])
lr = LogisticRegression()
lr.fit(X, y)
score = lr.score(X, y)
print("Score: %s" % score)
mlflow.log_metric("score", score)
mlflow.sklearn.log_model(lr, "test")
#mlflow.sklearn.save_model(lr, "my_model")
print("Model saved in run %s" %
mlflow.active_run().info.run_uuid)
留言
張貼留言