thatsmydoing
/
blog


			
							123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227
							<!DOCTYPE html>
<html prefix="og: http://ogp.me/ns# article: http://ogp.me/ns/article# " vocab="http://ogp.me/ns" lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width">
<title>Pleasant Programmer (old posts, page 2) | Pleasant Programmer</title>
<link href="assets/css/rst.css" rel="stylesheet" type="text/css">
<link href="assets/css/code.css" rel="stylesheet" type="text/css">
<link href="assets/css/theme.css" rel="stylesheet" type="text/css">
<link href="assets/css/custom.css" rel="stylesheet" type="text/css">
<link rel="alternate" type="application/rss+xml" title="RSS" href="rss.xml">
<link rel="canonical" href="http://pleasantprogrammer.com/index-2.html">
<link rel="prev" href="index-3.html" type="text/html">
<link rel="next" href="index-1.html" type="text/html">
<script type="text/javascript" src="//use.typekit.net/iwm5axp.js"></script><script type="text/javascript">try{Typekit.load();}catch(e){}</script><!--[if lt IE 9]><script src="assets/js/html5.js"></script><![endif]-->
</head>
<body>
<a href="#content" class="sr-only sr-only-focusable">Skip to main content</a>
    
    <header id="header" role="banner"><div class="thomas">
            <img src="assets/img/thomas.gif" alt="DJ THOMAS IN DA HAUS"><img src="assets/img/thomas.png" alt="Pleasant Programmer">
</div>
        
    <h1 id="brand"><a href="http://pleasantprogrammer.com/" title="Pleasant Programmer" rel="home">

        <span id="blog-title">Pleasant Programmer</span>
    </a></h1>

        
    <nav id="menu" role="navigation"><ul>
<li class="twitter"><a href="http://twitter.com/pleasantprog">@pleasantprog</a></li>
                <li><a href="archive.html">Archives</a></li>
                <li><a href="categories/index.html">Tags</a></li>
                <li><a href="rss.xml">RSS</a></li>
    
    
    </ul></nav></header><div id="container">
         <main id="content" role="main"><div class="postindex">
    <article class="h-entry post-text"><header><h1 class="p-name entry-title">
            <a href="posts/highways-in-otp.html" class="u-url">Highways in OTP</a>
        </h1>
    </header><div class="e-content entry-content">
    <div>
<p>One of the weird things that happens with OTP is sometimes it gives absurdly roundabout routes. Here is OTP's suggested route for walking from UP to Ateneo:</p>
<p><img alt="Roundabout route from UP to Ateneo" src="galleries/transit/otproundabout.png"></p>
<p>This is just so hilariously wrong. It's much simpler to just walk along Katipunan Avenue.</p>
<p>OTP couldn't possibly be that dumb though, so there must be something we're doing wrong. If you notice, Katipunan Avenue is colored red compared to the other streets. OTP seems to be avoiding any path that goes along Katipunan Avenue. The problem might have something to do with the "road type" designated to Katipunan.</p>
<p>Apparently, by default OTP will consider roads of type <code>trunk</code> to be non-walkable and non-bikable. This is documented in the <a href="http://wiki.openstreetmap.org/wiki/OpenTripPlanner">OpenStreetMap wiki</a> and the <a href="https://github.com/openplans/OpenTripPlanner/wiki/GraphBuilder#permissions-and-bicycle-safety">OTP wiki</a> as well. There are actually multiple ways to go about this then. The first solution that came to mind was to just edit the original OSM XML file.</p>
<pre class="code literal-block">sed -i .bak s/trunk/primary/g manila.osm
</pre>


<p>And rebuild the graph. It doesn't really matter much because the OSM data isn't used to render the maps. It's just used to build the routing data. This is actually what I did for <a href="http://maps.pleasantprogrammer.com">maps.pleasantprogrammer.com</a>.</p>
<p>It's also possible to set the default way properties in OTP. Instead of disallowing walking and biking on <code>highway=trunk</code> we could allow that. This is not much better than the <code>sed</code> solution though. It's better since you keep the weighting done by OTP, but you're still saying that all trunks are walkable which might not be the case.</p>
<p>The most correct way to actually fix this is to go through each of the trunks and specifying <code>foot=yes</code> and <code>bicycle=yes</code> for those trunks that are actually walkable. You could either do this locally with the dumped data, or contribute it directly to OSM. I'm not sure on the particulars with updating OSM though.</p>
</div>
    </div>
    <small class="dateline">Posted: <time class="published dt-published" datetime="2013-07-24T23:15:57+08:00" title="2013-07-24 23:15">2013-07-24 23:15</time></small>
        | <small class="commentline">
        
    <a href="posts/highways-in-otp.html#disqus_thread" data-disqus-identifier="cache/posts/highways-in-otp.html">Comments</a>

</small>
    </article><article class="h-entry post-text"><header><h1 class="p-name entry-title">
            <a href="posts/elevation-data-in-otp.html" class="u-url">Elevation Data in OTP</a>
        </h1>
    </header><div class="e-content entry-content">
    <div>
<p><img alt="OpenTripPlanner showing elevation data" src="galleries/transit/otpelevation.png"></p>
<p>One thing I hadn't tested out last time was OTP's support for elevation data. It makes use of this by showing the elevation you have to traverse while walking along the suggested route. It can also take it into account when suggesting bike routes.</p>
<p>The <a href="https://github.com/openplans/OpenTripPlanner/wiki/FiveMinutes">5 minute tutorial</a> actually discusses the elevation data briefly, but a more in-depth thing you can look at is the <a href="https://github.com/openplans/OpenTripPlanner/wiki/GraphBuilder#elevation-data">GraphBuilder documentation</a>. It suggests using the ASTER dataset which is free but requires registration. I just opted to use the SRTM data available from the <a href="http://www.philgis.org/freegisdata.htm">PhilGIS website</a>.</p>
<p>I don't know about the ASTER dataset, but the PhilGIS data was in the ERDAS img format. OTP only supports GeoTIFF so there was a need to convert it beforehand. You can use <a href="http://www.gdal.org/">GDAL</a> for this. You'd just then run,</p>
<pre class="code literal-block"> gdal_translate srtm41_90m_phl.img phil.tiff
</pre>


<p>Afterwards, it's just a matter of following the OTP instructions on using a local elevation dataset. The process actually doubled the size of the generated Graph.obj so it might not be ideal if you're running on limited RAM.</p>
<p>I've actually hosted a <a href="http://maps.pleasantprogrammer.com">working example</a>. It's pretty much at the limits of the RAM so it might be slow and unreliable, but you can test it out just for fun. Please don't abuse it though.</p>
</div>
    </div>
    <small class="dateline">Posted: <time class="published dt-published" datetime="2013-07-23T18:23:00+08:00" title="2013-07-23 18:23">2013-07-23 18:23</time></small>
        | <small class="commentline">
        
    <a href="posts/elevation-data-in-otp.html#disqus_thread" data-disqus-identifier="cache/posts/elevation-data-in-otp.html">Comments</a>

</small>
    </article><article class="h-entry post-text"><header><h1 class="p-name entry-title">
            <a href="posts/graphserver.html" class="u-url">GraphServer</a>
        </h1>
    </header><div class="e-content entry-content">
    <div>
<p>Link: <a href="http://graphserver.github.io/graphserver/">http://graphserver.github.io/graphserver/</a></p>
<p>One other routing webapp I saw was GraphServer. It's actually more of a general purpose Graph library which supports GTFS and OSM data than an actual dedicated routing software like OpenTripPlanner. It's also based off python and C instead of Java, so it feels a lot less heavy.</p>
<p>The instructions on the website are already pretty good. There are just some minor errors with it. Where it says <code>gs_gtfsdb_build</code>, you should actually use <code>gs_gtfsdb_compile</code>. Also, when running <code>gs_osmdb_compile</code> you might need to use <code>-t</code> for tolerant in case you follow the instructions on chopping up the original OSM data.</p>
<p>A nice suggestion from the GraphServer instructions was to crop the OSM data to minimize the graph size. This is actually quite helpful if you downloaded the entire Philippine OSM dump. It reduced the original 900MB file to 135MB which was a lot more workable. I did hit a problem with their instructions though. The linked version of osmosis is an old one, which doesn't support 64-bit ids. The <a href="http://wiki.openstreetmap.org/wiki/Osmosis">latest version of Osmosis</a> easily did the job though.</p>
<p>The actual routing though, was not exactly good. I only tried one route which should normally take 1-2 transfers, it suggested a route which involved 4+ transfers. It also didn't provide any alternate routes aside from that one. I'm not sure if it's a limitation of the provided routeserver, but I didn't bother checking if it supported parameters which might provide better routes.</p>
<p>I think graphserver could be useful, but it seems more involved than say OpenTripPlanner. There do seem to be people who use graphserver for their routing apps, but for the bounds of the contest, or just as a side project, it might require too much effort.</p>
</div>
    </div>
    <small class="dateline">Posted: <time class="published dt-published" datetime="2013-07-23T14:48:29+08:00" title="2013-07-23 14:48">2013-07-23 14:48</time></small>
        | <small class="commentline">
        
    <a href="posts/graphserver.html#disqus_thread" data-disqus-identifier="cache/posts/graphserver.html">Comments</a>

</small>
    </article><article class="h-entry post-text"><header><h1 class="p-name entry-title">
            <a href="posts/transit-wand.html" class="u-url">Transit Wand</a>
        </h1>
    </header><div class="e-content entry-content">
    <div>
<p>Link: <a href="https://play.google.com/store/apps/details?id=com.conveyal.transitwand">http://transitwand.com</a></p>
<p>Overall, this was the simplest of the <a href="http://philippine-transit.hackathome.com/use-this-code/">open-source transit tools</a> to actually get up and running. There's already a deployed instance of the server, and you can easily download the phone app via the <a href="https://play.google.com/store/apps/details?id=com.conveyal.transitwand">Play Store</a>. Even running the server by yourself didn't have any of the hiccups I had with GTFS Editor.</p>
<p>The phone app is actually quite simple. It allows you to capture a trip, which will record your GPS coordinates as you ride public transit. It also allows you to mark points of the trip where you stop and also how long the stop took. Lastly, it allows you to record embarking and disembarking passengers which is potentially useful for ridership data.</p>
<p>After doing a capture session, you can review the data on the phone. It will plot out the route on a map, with markers for the stops. You then either delete the data if it looks wrong, or you can upload it to the Transit Wand server. Uploading involves registering an account, but it's free and you don't even actually need to put in a username or anything. It simply registers the phone's IMEI on the server and gives you a 6-digit identifier.</p>
<p>You can then use the 6-digit identifier to view the data on Transit Wand's server, which is good since uploading any data automatically deletes it from the phone. There really isn't much else you can do with it though. It just allows you to view the data, and export it as a <a href="https://en.wikipedia.org/wiki/Shapefile">Shapefile</a>.</p>
<p>As is, this is purely a data collection client-server app. Barring looking at the database, there is no way to get a list of phones which have collected data. Only the person who initiated the data collection knows the 6-digit code to view their data. There's also no way to extract the ridership information from the server yet. This isn't to say that the data won't eventually go public though.</p>
<p>An interesting thing you <em>can</em> do with the Transit Wand data is import it into GTFS Editor to make a new route. You don't even have to manually download and upload the data. Just type in your 6-digit identifier and it will give you a list of routes you've captured via Transit Wand. This is wonderful as you get all the stop data, as well as the shape of the route.</p>
<p>I imagine these two tools were how the DOTC came up with all the GTFS data we have now. What I don't understand is why the shape data isn't present. Importing from Transit Wand already gets you shape data. There are even facilities to edit the shape within the editor if clean up is necessary. The only problem I saw was the fact that you can't easily move stops, you have to input coordinates to change the position.</p>
<p>It <em>might</em> also be possible that when the DOTC was still collecting the data, the route collection or editing features weren't present yet. That would just be lame and depressing though.</p>
<p>Overall, Transit Wand does what it's supposed to do. You collect data, and then upload it to a server. There is a lot of room for improvement though. It would be nice to have a better API that allows access to more of the data. Building in analysis tools for the ridership data might also be a welcome thing. I imagine it would also be great if you could encourage people to use the app and upload their own trips.</p>
</div>
    </div>
    <small class="dateline">Posted: <time class="published dt-published" datetime="2013-07-15T22:45:20+08:00" title="2013-07-15 22:45">2013-07-15 22:45</time></small>
        | <small class="commentline">
        
    <a href="posts/transit-wand.html#disqus_thread" data-disqus-identifier="cache/posts/transit-wand.html">Comments</a>

</small>
    </article><article class="h-entry post-text"><header><h1 class="p-name entry-title">
            <a href="posts/fare-data.html" class="u-url">Fare Data</a>
        </h1>
    </header><div class="e-content entry-content">
    <div>
<p>As part of the data released by the DOTC, we also have the <a href="http://philippine-transit.hackathome.com/dataset-philippines-transit-information-service-gtfs/">fare matrix</a> for aircon buses, ordinary buses and jeeps. All as wonderful images. The data is also actually available from the <a href="http://ltfrb.gov.ph/main/farerates">LTFRB website</a>. Generally, the fare scheme is represented as "pay <em>X</em> pesos for the first <em>Y</em> kilometers, pay <em>Z</em> for every succeeding kilometer." Instead of a table, we can simply represent this as a formula instead,</p>
<pre class="code literal-block">base_fare + (distance - initial) * per_km
</pre>


<p>The relevant values for the three services are:</p>
<table>
<thead><tr>
<td>type</td>
    <td>base_fare</td>
    <td>initial</td>
    <td>per_km</td>
  </tr></thead>
<tbody>
<tr>
<td>bus aircon</td>
    <td>12.00</td>
    <td>5 km</td>
    <td>2.20</td>
  </tr>
<tr>
<td>bus ordinary</td>
    <td>10.00</td>
    <td>5 km</td>
    <td>1.85</td>
  </tr>
<tr>
<td>jeep aircon</td>
    <td>8.00</td>
    <td>4 km</td>
    <td>1.40</td>
  </tr>
</tbody>
</table>
<p>It isn't as simple as that though. Fares are also rounded to the nearest 25 centavos. So we'd need to round them off correctly. This can be achieved by doing,</p>
<pre class="code literal-block">round(calculated_fare * 4.0)/4.0
</pre>


<p>There's also the discounted fare for students, senior citizens and persons with disability. They get 20% off the fare (prior to rounding) and the resulting fare is rounded off as well.</p>
<p>Doing just this, we actually do get the same results as the fare matrices in the image for the most part. There are some discrepancies with the discounted jeep fares. I've tried to resolve it by tweaking around with the formulas, but it really doesn't make sense in any way. I presume these were manually adjusted for one reason or another.</p>
<p>Here's a <a href="uploads/farematrix.rb">script</a> that generates CSVs of all the three fare matrices. If you're too lazy to run it, here are links to the <a href="uploads/pub_aircon.csv">aircon bus</a>, <a href="uploads/pub_ordinary.csv">ordinary bus</a> and <a href="uploads/puj.csv">jeep</a> fare matrices.</p>
<h4>GTFS compatibility</h4>
<p>As is, the provided GTFS data does not have any fare data. I imagine this is because the existing spec doesn't have good support for distance-based fares like we have in the Philippines. Judging from the <a href="https://code.google.com/p/googletransitdatafeed/wiki/FareExamples">fare examples</a>, the only reasonable way we could implement distance-based fares is following example 6. This would involve setting a fare for each possible pair of stops based on the distance between them. This isn't exactly ideal. In fact, the people originally working on the DOTC project have voiced <a href="https://groups.google.com/forum/#!topic/gtfs-fare-wg/V63xRSnQJGw">issues</a> and made <a href="https://groups.google.com/forum/#!msg/gtfs-changes/uybrAokZ9Cg/rqlzXdMypUgJ">proposals</a> for having distance-based fares included into GTFS.</p>
<p>Apparently, public transit fares are a really complicated thing. You have fares based on distance, number of stops passed through, and transfers which may or may not cost extra. Not only that, you might have discounted fares, or first-class vs economy fares. The community will want to get it right before it's formally included in the spec. You can see the current state of the consolidated <a href="https://docs.google.com/document/d/1mK3--o5g4-3cCXaqmch92U63JTwChh0L2VCmcDViIlM/edit">GTFS fare proposal here</a>.</p>
<p>Even in it's proposal form though, we might have hope of being able to see these being used. There's currently a <a href="https://github.com/OneBusAway/onebusaway-gtfs-modules/pull/30">pull request</a> for supporting the distance-based fare scheme into the OneBusAway libraries. The libraries actually used by GTFS Editor and OpenTripPlanner for working with GTFS data.</p>
<h4>Remaining Problems</h4>
<p>Given all that, it would probably still be a long way before this allows us to make a really good routing app. We still don't have shape data, so the distance estimates would really be rough estimates at best. There's no support for rounding to the nearest centavo. I realize that's just nitpicking, but if we want something truly polished, even that has to be taken care of.</p>
<p>We also don't know if the jeeps or buses strictly follow the distance-based scheme. After all, if you can get on and off anywhere, you can't really measure distance that exactly. I assume they generally work off the notion of "zones" than actual distance travelled. In that sense, they work more similarly to the LRT which has fares based on how many stops you pass. For jeeps and buses, your fare is probably based more on how many "zones" you pass through.</p>
<h4>Conclusion</h4>
<p>Philip, a co-worker of mine at By Implication, had suggested that we might want to use a different model than what the GTFS proposes. I have to agree with him. At this point, the GTFS doesn't really fit with our system. But I do think that open data and standards are great. In fact, I applaud the developers who made proposals for the fare system, as those are great first steps towards making the GTFS a more universal standard.</p>
<p>Side note: I'd also actually really like to hear about the DOTC developers' experience with the project. It would be nice if they had a devblog.</p>
</div>
    </div>
    <small class="dateline">Posted: <time class="published dt-published" datetime="2013-07-13T21:15:09+08:00" title="2013-07-13 21:15">2013-07-13 21:15</time></small>
        | <small class="commentline">
        
    <a href="posts/fare-data.html#disqus_thread" data-disqus-identifier="cache/posts/fare-data.html">Comments</a>

</small>
    </article>
</div>

        <nav class="postindexpager"><ul class="pager clearfix">
<li class="previous">
                <a href="index-3.html" rel="prev">← Newer posts</a>
            </li>
            <li class="next">
                <a href="index-1.html" rel="next">Older posts →</a>
            </li>
        </ul></nav><script>var disqus_shortname="pleasantprog";(function(){var a=document.createElement("script");a.async=true;a.src="//"+disqus_shortname+".disqus.com/count.js";(document.getElementsByTagName("head")[0]||document.getElementsByTagName("body")[0]).appendChild(a)}());</script></main><footer id="footer" role="contentinfo"><p><a rel="license" href="http://creativecommons.org/licenses/by-sa/3.0/deed.en_US"><img alt="CC-BY-SA" style="border-width:0" src="http://i.creativecommons.org/l/by-sa/3.0/80x15.png"></a> © 2015 Thomas Dy - Powered by <a href="http://getnikola.com">Nikola</a></p>
            
        </footer>
</div>
    
    
    <script src="assets/js/konami.js"></script><script src="http://code.jquery.com/jquery-2.0.3.min.js"></script><script>
        var easter_egg = new Konami();
        easter_egg.code = function() {
            $(".thomas").toggleClass("whoa");
            $("body").scrollTop(0);
        }
        easter_egg.load();

        // love you, thomas!
        // yours, @_phi + @meggykawsek
    </script>
</body>
</html>