predict-taxi-trip-duration-nb/develop_ml_application_tour.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "687334b0",
   "metadata": {},
   "source": [
    "# 基于机器学习数据库飞速上线AI应用\n",
    "大家平时可能都会打车，从出发的地点到目的地，行程耗时可能会存在多种因素，比如天气，是否周五，如果获取更准确的耗时预测，对人来说是一个复杂的问题，而对机器就变得很简单，今天的任务就是开发一个通过机器学习模型进行出租车行程耗时的实时智能应用，整个应用开发是基于[notebook](http://ipython.org/notebook.html)\n",
    "\n",
    "\n",
    "\n",
    "![出租车行程耗时预测](https://th.bing.com/th/id/Rcf52e9678006c3e99a98cf88a216e38d?rik=oQN4iVqyXXjYNg&riu=http%3a%2f%2fi1.hexun.com%2f2020-05-08%2f201272779.jpg&ehk=4eiCf7x4YL8wYL4D6wnqUybMtVEER6teKg1deOEcYys%3d&risl=&pid=ImgRaw)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e7559e1",
   "metadata": {},
   "source": [
    "## 初始化环境\n",
    "整个初始化过程包含安装fedb，以及相关运行环境，初始化脚步可以参考https://github.com/4paradigm/DemoApps/blob/main/predict-taxi-trip-duration-nb/demo/init.sh"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4be401ff",
   "metadata": {},
   "outputs": [],
   "source": [
    "!cd demo && sh init.sh"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f31d07f",
   "metadata": {},
   "source": [
    "## 导入行程历史数据到fedb\n",
    "\n",
    "使用fedb进行时序特征计算是需要历史数据的，所以我们将历史的行程数据导入到fedb，以便实时推理可以使用历史数据进行特征推理，导入代码可以参考https://github.com/4paradigm/DemoApps/blob/main/predict-taxi-trip-duration-nb/demo/import.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7a21f6c7",
   "metadata": {},
   "outputs": [],
   "source": [
    "!cd demo && python3 import.py"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f1217305",
   "metadata": {},
   "source": [
    "## 使用行程数据进行模型训练\n",
    "\n",
    "通过label数据进行模型训练，一下是这次任务使用的代码\n",
    "\n",
    "* 训练脚本代码 https://github.com/4paradigm/DemoApps/blob/main/predict-taxi-trip-duration-nb/demo/train_sql.py \n",
    "* 训练数据 https://github.com/4paradigm/DemoApps/blob/main/predict-taxi-trip-duration-nb/demo/data/taxi_tour_table_train_simple.snappy.parquet\n",
    "\n",
    "整个任务最终会生成一个model.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e063ee5e",
   "metadata": {},
   "outputs": [],
   "source": [
    "!cd demo && sh train.sh"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e843ece0",
   "metadata": {},
   "source": [
    "## 使用训练的模型搭建链接fedb的实时推理http服务\n",
    "\n",
    "基于上一步生成的模型和fedb历史数据，搭建一个实时推理服务，整个推理服务代码参考https://github.com/4paradigm/DemoApps/blob/main/predict-taxi-trip-duration-nb/demo/predict_server.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0cc7e074",
   "metadata": {},
   "outputs": [],
   "source": [
    "!cd demo && sh start_predict_server.sh"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4778edc1",
   "metadata": {},
   "source": [
    "## 通过http请求发送一个推理请求\n",
    "\n",
    "整个请求很简单，具体代码如下\n",
    "\n",
    "```python\n",
    "url = \"http://127.0.0.1:8887/predict\"\n",
    "req ={\"id\":\"id0376262\",\n",
    "\t\"vendor_id\":1,\n",
    "\t\"pickup_datetime\":1467302350000,\n",
    "\t\"dropoff_datetime\":1467304896000,\n",
    "\t\"passenger_count\":2,\n",
    "\t\"pickup_longitude\":-73.873093,\n",
    "\t\"pickup_latitude\":40.774097,\n",
    "\t\"dropoff_longitude\":-73.926704,\n",
    "\t\"dropoff_latitude\":40.856739,\n",
    "\t\"store_and_fwd_flag\":\"N\",\n",
    "\t\"trip_duration\":1}\n",
    "r = requests.post(url, json=req)\n",
    "print(r.text)\n",
    "print(\"Congraduation! You have finished the task.\")\n",
    "tmp = os.urandom(44)\n",
    "secret_key = base64.b64encode(tmp)\n",
    "print(\"Your Key:\" + str(secret_key))\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e1c2ced0",
   "metadata": {},
   "outputs": [],
   "source": [
    "!cd demo && python3 predict.py"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}