{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "\n\n# Project from a link layer\n\nIn this example, we show how to create an empty project and populate it with a\nnetwork coming from a link layer we load from a text file. It can easily be\nreplaced with a different form of loading the data (GeoPandas, for example).\n\nWe use Folium to visualize the resulting network.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Imports\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "from uuid import uuid4\nimport urllib.request\nfrom string import ascii_lowercase\nfrom tempfile import gettempdir\nfrom os.path import join\nfrom shapely.wkt import loads as load_wkt\nimport pandas as pd\nimport folium\n\nfrom aequilibrae import Project"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "We create an empty project on an arbitrary folder\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "fldr = join(gettempdir(), uuid4().hex)\nproject = Project()\nproject.new(fldr)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Now we obtain the link data for our example (in this case from a link layer\nwe will download from the AequilibraE website)\nWith data, we load it on Pandas\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "dest_path = join(fldr, \"queluz.csv\")\nurllib.request.urlretrieve(\"https://aequilibrae.com/data/queluz.csv\", dest_path)\n\ndf = pd.read_csv(dest_path)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Let's see if we have to add new link_types to the model before we add links\nThe links we have in the data are:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "link_types = df.link_type.unique()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "And the existing link types are\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "lt = project.network.link_types\nlt_dict = lt.all_types()\nexisting_types = [ltype.link_type for ltype in lt_dict.values()]\n\n# We could also get it directly from the project database\n# ``existing_types = [x[0] for x in project.conn.execute('Select link_type from link_types')]``"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "We add the link types that do not exist yet\nThe trickier part is to choose a unique link type ID for each link type\nYou might want to tailor the link type for your use, but here we get letters\nin alphabetical order\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "types_to_add = [ltype for ltype in link_types if ltype not in existing_types]\nfor i, ltype in enumerate(types_to_add):\n    new_type = lt.new(ascii_lowercase[i])\n    new_type.link_type = ltype\n    # new_type.description = 'Your custom description here if you have one'\n    new_type.save()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "We need to use a similar process for modes\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "md = project.network.modes\nmd_dict = md.all_modes()\nexisting_modes = {k: v.mode_name for k, v in md_dict.items()}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Now let's see the modes we have in the network that we DON'T have already in\nthe model\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "# We get all the unique mode combinations and merge them into a single string\nall_variations_string = \"\".join(df.modes.unique())\n\n# We then get all the unique modes in that string above\nall_modes = set(all_variations_string)\n\n# This would all fit nicely in a single line of code, btw. Try it!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Now let's add any new mode to the project\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "modes_to_add = [mode for mode in all_modes if mode not in existing_modes]\nfor i, mode_id in enumerate(modes_to_add):\n    new_mode = md.new(mode_id)\n    # You would need to figure out the right name for each one, but this will do\n    new_mode.mode_name = f\"Mode_from_original_data_{mode_id}\"\n    # new_type.description = 'Your custom description here if you have one'\n\n    # It is a little different because you need to add it to the project\n    project.network.modes.add(new_mode)\n    new_mode.save()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "We cannot use the existing link_id, so we create a new field to not loose\nthis information\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "links = project.network.links\nlink_data = links.fields\n# Create the field and add a good description for it\nlink_data.add(\"source_id\", \"link_id from the data source\")\n\n# We need to refresh the fields so the adding method can see it\nlinks.refresh_fields()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "We can now add all links to the project!\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "for idx, record in df.iterrows():\n    new_link = links.new()\n\n    # Now let's add all the fields we had\n    new_link.source_id = record.link_id\n    new_link.direction = record.direction\n    new_link.modes = record.modes\n    new_link.link_type = record.link_type\n    new_link.name = record.name\n    new_link.geometry = load_wkt(record.WKT)\n    new_link.save()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "We grab all the links data as a Pandas DataFrame so we can process it easier\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "links = project.network.links.data\n\n# We create a Folium layer\nnetwork_links = folium.FeatureGroup(\"links\")\n\n# We do some Python magic to transform this dataset into the format required by Folium\n# We are only getting link_id and link_type into the map, but we could get other pieces of info as well\nfor i, row in links.iterrows():\n    points = row.geometry.wkt.replace(\"LINESTRING \", \"\").replace(\"(\", \"\").replace(\")\", \"\").split(\", \")\n    points = \"[[\" + \"],[\".join([p.replace(\" \", \", \") for p in points]) + \"]]\"\n    # We need to take from x/y to lat/long\n    points = [[x[1], x[0]] for x in eval(points)]\n\n    line = folium.vector_layers.PolyLine(\n        points, popup=f\"<b>link_id: {row.link_id}</b>\", tooltip=f\"{row.link_type}\", color=\"blue\", weight=10\n    ).add_to(network_links)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "We get the center of the region we are working with some SQL magic\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "curr = project.conn.cursor()\ncurr.execute(\"select avg(xmin), avg(ymin) from idx_links_geometry\")\nlong, lat = curr.fetchone()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "map_osm = folium.Map(location=[lat, long], zoom_start=15)\nnetwork_links.add_to(map_osm)\nfolium.LayerControl().add_to(map_osm)\nmap_osm"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "project.close()"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.18"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}