thatsmydoing
/
blog


			
							123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207
							<!DOCTYPE html>
<html lang="en-us">
<head>
	<meta charset="utf-8">
	<meta name="generator" content="Hugo 0.88.0" />
	<meta name="viewport" content="width=device-width, initial-scale=1">
	<link rel="stylesheet" href="/assets/css/theme.css">
	<link rel="alternate" href="/rss.xml" type="application/rss+xml" title="Pleasant Programmer">
	<script type="text/javascript" src="//use.typekit.net/iwm5axp.js"></script>
	<script type="text/javascript">try{Typekit.load();}catch(e){}</script>
	<title>Pleasant Programmer</title>
</head>

<body>
	<header id="header" role="banner">
		<div id="thomas">
			<img src="/assets/img/thomas.gif" alt="DJ THOMAS IN DA HAUS">
			<img src="/assets/img/thomas.png" alt="Pleasant Programmer">
		</div>
		<h1 class="site-title"><a href="/">Pleasant Programmer</a></h1>
		<nav id="menu" role="navigation">
			<ul>
				<li class="twitter">
					<a href="http://twitter.com/pleasantprog">@pleasantprog</a>
				</li>
				<li><a href="/pages/projects.html">projects</a></li>
				<li><a href="/posts.html">archives</a></li>
				<li><a href="/tags.html">tags</a></li>
				<li><a href="/rss.xml">rss</a></li>
			</ul>
		</nav>
	</header>
	<div id="container">


<main id="content" role="main">
<div class="postindex">
	
	<article class="h-entry post-text" itemscope itemtype="http://schema.org/Blog">
		<header>
			<h1 class="p-name entry-title" itemprop="headline">
				<a href="/posts/geocoding-services.html" class="u-url">Geocoding Services</a>
			</h1>
		</header>
		<div class="e-content entry-content">
			<p>A key component for any routing service is being able to do geocoding. Most people who are looking for routes most probably don&rsquo;t know exactly where their start and end points are on the map. Even then, manually looking for a location on a map is a time-consuming task.</p>
<p>The gold standard for doing geocoding right now is Google Maps. It&rsquo;s hard to find a better location search experience. If they actually provided routing for jeeps here in the Philippines, I imagine there wouldn&rsquo;t be <em>that</em> much you could do for the competition.</p>
<p>When the competition started though, I took it as a challenge to avoid Google Maps as much as possible. I wanted to see how much is currently possible with other options such as OpenStreetMap. In fact, OSM does have a geocoding service called <a href="http://nominatim.openstreetmap.org">Nominatim</a>.</p>
<p>Sadly, for a mapping app, what you want to do is not simply just geocoding. With geocoding, you take an address and turn it into coordinates. When you want to search for a place in a mapping app, you take part of an address, infer the rest of it, and give the user options to choose from.</p>
<p>Given a typical mapping app, you might type in &ldquo;ateneo&rdquo; and expect it to give you Ateneo de Manila University. With typical geocoding services like Nominatim or even Google&rsquo;s <a href="https://developers.google.com/maps/documentation/javascript/geocoding">geocoding API</a>, you probably won&rsquo;t get any result for this. What you want to use is the <a href="https://developers.google.com/maps/documentation/javascript/places">Places API</a> which provides an autocomplete search box. Using it, when you type in &ldquo;ateneo&rdquo;, it automatically suggests in the dropdown, &ldquo;Ateneo de Manila University&rdquo;.</p>
<p>A downside to using the Places API is that it&rsquo;s against the terms of service to use it with something that isn&rsquo;t Google Maps, which means no OpenStreetMap. If there were more time, writing your own autocompletion engine using OpenStreetMap&rsquo;s data will probably be a better long term solution.</p>
<p>For now, since the competition&rsquo;s deadline is just a few days away, I&rsquo;ll be using Google Maps.</p>

		</div>
		<small class="dateline">Posted: <time class="published dt-published" itemprop="datePublished" datetime="2013-09-25">2013-09-25</time></small>
		| <small class="commentline"><a href="/posts/geocoding-services.html#isso-thread">Comments</a></small>
	</article>
	</article>
	
	<article class="h-entry post-text" itemscope itemtype="http://schema.org/Blog">
		<header>
			<h1 class="p-name entry-title" itemprop="headline">
				<a href="/posts/jeep-and-bus-schedules.html" class="u-url">Jeep and Bus Schedules</a>
			</h1>
		</header>
		<div class="e-content entry-content">
			<p>Wouldn&rsquo;t it be wonderful if there were no buses or jeepneys in the Philippines over the weekends? It would truly be a cyclist&rsquo;s paradise. Imagine biking along EDSA, normally that would be a death sentence, but according to the GTFS data, you shouldn&rsquo;t worry. I can assure you, it&rsquo;s still a death sentence.</p>
<p>The GTFS spec defines 2 ways of statically specifying trip schedules. You can define the exact times that a service will arrive at a stop. You can also specify between what times the service is active and how often a new bus or jeep leaves the first stop. You also define which days those rules apply. You could say every MWF, the bus operates from 9:00AM to 9:00PM and every TTH, the bus services from 3:00AM to 11:00PM.</p>
<p>This should be sufficient in theory, but real world conditions like traffic or the weather could throw the schedules off. To solve this, there&rsquo;s another spec, GTFS-realtime. This allows transit agencies to push temporary schedule updates and service announcements.</p>
<p>Like much everything else about the Philippine transit system, there aren&rsquo;t really any &ldquo;schedules&rdquo; to speak of. It&rsquo;s generally whenever the buses or jeeps feel like it. So we have no static schedules. We don&rsquo;t have a central agency or the tracking technology to make it feasible to push updates via GTFS-RT.</p>
<p>Ideally, we shouldn&rsquo;t bother inputting the schedule information into GTFS. Only the route data is really important for jeeps and buses. However, the schedule information is required in the GTFS, and routing apps wouldn&rsquo;t work without it. So we have to add a reasonable trip schedule for jeeps and buses.</p>
<p>The current GTFS data does define these trip schedules. We assume that jeeps and buses operate between 6:00AM and 11:00PM and a new jeep passes by every 10 minutes. Also, jeeps and buses are defined to only operate on weekdays.</p>
<p>While there might be jeeps who change routes or don&rsquo;t operate on weekends, I&rsquo;m pretty sure that jeeps and buses run on weekends. We&rsquo;ll have to fix it ourselves temporarily since there&rsquo;s no central GTFS feed yet.</p>
<div class="highlight"><pre tabindex="0" style="color:#e5e5e5;background-color:#000;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-sh" data-lang="sh"><span style="color:#007f7f"># 724594 seems to be the service id used by jeeps and buses</span>
sed -i .bak <span style="color:#0ff;font-weight:bold">&#39;/^724594/ s/0,0/1,1/&#39;</span> calendar.txt
</code></pre></div><p>Another thing we could do is to adjust the time between buses, although the improvement is arguable. With the current 10 minutes between jeeps, it might provide some routes a significant advantage just because the timing is right. So you might get differing route suggestions depending on what time you planned the route. This makes sense when you&rsquo;re sure what the times are, so you can minimize the wait, but with jeeps, you never really know how long the wait will actually be.</p>
<p>If we set the frequency to one minute, it <em>might</em> give better routes by eliminating the timing issue. Or not, it&rsquo;s kind of hard to tell.</p>
<div class="highlight"><pre tabindex="0" style="color:#e5e5e5;background-color:#000;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-sh" data-lang="sh"><span style="color:#007f7f"># jeep and bus route ids tend to start with 72</span>
sed -i .bak <span style="color:#0ff;font-weight:bold">&#39;/^72/ s/,600/,60/&#39;</span> frequencies.txt
</code></pre></div><p>Overall, the problems we&rsquo;re having is a symptom of the mismatch between our transit system and the GTFS. It would be great if our transit system gets better and we don&rsquo;t need to do hackish things for it to fit the GTFS, but that&rsquo;s still a dream. For now, all we can really do is fit a triangle into a square hole.</p>

		</div>
		<small class="dateline">Posted: <time class="published dt-published" itemprop="datePublished" datetime="2013-07-28">2013-07-28</time></small>
		| <small class="commentline"><a href="/posts/jeep-and-bus-schedules.html#isso-thread">Comments</a></small>
	</article>
	</article>
	
	<article class="h-entry post-text" itemscope itemtype="http://schema.org/Blog">
		<header>
			<h1 class="p-name entry-title" itemprop="headline">
				<a href="/posts/highways-in-otp.html" class="u-url">Highways in OTP</a>
			</h1>
		</header>
		<div class="e-content entry-content">
			<p>One of the weird things that happens with OTP is sometimes it gives absurdly roundabout routes. Here is OTP&rsquo;s suggested route for walking from UP to Ateneo:</p>
<p><img src="../galleries/transit/otproundabout.png" alt="Roundabout route from UP to Ateneo"></p>
<p>This is just so hilariously wrong. It&rsquo;s much simpler to just walk along Katipunan Avenue.</p>
<p>OTP couldn&rsquo;t possibly be that dumb though, so there must be something we&rsquo;re doing wrong. If you notice, Katipunan Avenue is colored red compared to the other streets. OTP seems to be avoiding any path that goes along Katipunan Avenue. The problem might have something to do with the &ldquo;road type&rdquo; designated to Katipunan.</p>
<p>Apparently, by default OTP will consider roads of type <code>trunk</code> to be non-walkable and non-bikable. This is documented in the <a href="http://wiki.openstreetmap.org/wiki/OpenTripPlanner">OpenStreetMap wiki</a> and the <a href="https://github.com/openplans/OpenTripPlanner/wiki/GraphBuilder#permissions-and-bicycle-safety">OTP wiki</a> as well. There are actually multiple ways to go about this then. The first solution that came to mind was to just edit the original OSM XML file.</p>
<div class="highlight"><pre tabindex="0" style="color:#e5e5e5;background-color:#000;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-text" data-lang="text">sed -i .bak s/trunk/primary/g manila.osm
</code></pre></div><p>And rebuild the graph. It doesn&rsquo;t really matter much because the OSM data isn&rsquo;t used to render the maps. It&rsquo;s just used to build the routing data. This is actually what I did for <a href="http://maps.pleasantprogrammer.com">maps.pleasantprogrammer.com</a>.</p>
<p>It&rsquo;s also possible to set the default way properties in OTP. Instead of disallowing walking and biking on <code>highway=trunk</code> we could allow that. This is not much better than the <code>sed</code> solution though. It&rsquo;s better since you keep the weighting done by OTP, but you&rsquo;re still saying that all trunks are walkable which might not be the case.</p>
<p>The most correct way to actually fix this is to go through each of the trunks and specifying <code>foot=yes</code> and <code>bicycle=yes</code> for those trunks that are actually walkable. You could either do this locally with the dumped data, or contribute it directly to OSM. I&rsquo;m not sure on the particulars with updating OSM though.</p>

		</div>
		<small class="dateline">Posted: <time class="published dt-published" itemprop="datePublished" datetime="2013-07-24">2013-07-24</time></small>
		| <small class="commentline"><a href="/posts/highways-in-otp.html#isso-thread">Comments</a></small>
	</article>
	</article>
	
	<article class="h-entry post-text" itemscope itemtype="http://schema.org/Blog">
		<header>
			<h1 class="p-name entry-title" itemprop="headline">
				<a href="/posts/elevation-data-in-otp.html" class="u-url">Elevation Data in OTP</a>
			</h1>
		</header>
		<div class="e-content entry-content">
			<p><img src="../galleries/transit/otpelevation.png" alt="OpenTripPlanner showing elevation data"></p>
<p>One thing I hadn&rsquo;t tested out last time was OTP&rsquo;s support for elevation data. It makes use of this by showing the elevation you have to traverse while walking along the suggested route. It can also take it into account when suggesting bike routes.</p>
<p>The <a href="https://github.com/openplans/OpenTripPlanner/wiki/FiveMinutes">5 minute tutorial</a> actually discusses the elevation data briefly, but a more in-depth thing you can look at is the <a href="https://github.com/openplans/OpenTripPlanner/wiki/GraphBuilder#elevation-data">GraphBuilder documentation</a>. It suggests using the ASTER dataset which is free but requires registration. I just opted to use the SRTM data available from the <a href="http://www.philgis.org/freegisdata.htm">PhilGIS website</a>.</p>
<p>I don&rsquo;t know about the ASTER dataset, but the PhilGIS data was in the ERDAS img format. OTP only supports GeoTIFF so there was a need to convert it beforehand. You can use <a href="http://www.gdal.org/">GDAL</a> for this. You&rsquo;d just then run,</p>
<div class="highlight"><pre tabindex="0" style="color:#e5e5e5;background-color:#000;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-text" data-lang="text">gdal_translate srtm41_90m_phl.img phil.tiff
</code></pre></div><p>Afterwards, it&rsquo;s just a matter of following the OTP instructions on using a local elevation dataset. The process actually doubled the size of the generated Graph.obj so it might not be ideal if you&rsquo;re running on limited RAM.</p>
<p>I&rsquo;ve actually hosted a <a href="http://maps.pleasantprogrammer.com">working example</a>. It&rsquo;s pretty much at the limits of the RAM so it might be slow and unreliable, but you can test it out just for fun. Please don&rsquo;t abuse it though.</p>

		</div>
		<small class="dateline">Posted: <time class="published dt-published" itemprop="datePublished" datetime="2013-07-23">2013-07-23</time></small>
		| <small class="commentline"><a href="/posts/elevation-data-in-otp.html#isso-thread">Comments</a></small>
	</article>
	</article>
	
	<article class="h-entry post-text" itemscope itemtype="http://schema.org/Blog">
		<header>
			<h1 class="p-name entry-title" itemprop="headline">
				<a href="/posts/graphserver.html" class="u-url">GraphServer</a>
			</h1>
		</header>
		<div class="e-content entry-content">
			<p>Link: <a href="http://graphserver.github.io/graphserver/">http://graphserver.github.io/graphserver/</a></p>
<p>One other routing webapp I saw was GraphServer. It&rsquo;s actually more of a general purpose Graph library which supports GTFS and OSM data than an actual dedicated routing software like OpenTripPlanner. It&rsquo;s also based off python and C instead of Java, so it feels a lot less heavy.</p>
<p>The instructions on the website are already pretty good. There are just some minor errors with it. Where it says <code>gs_gtfsdb_build</code>, you should actually use <code>gs_gtfsdb_compile</code>. Also, when running <code>gs_osmdb_compile</code> you might need to use <code>-t</code> for tolerant in case you follow the instructions on chopping up the original OSM data.</p>
<p>A nice suggestion from the GraphServer instructions was to crop the OSM data to minimize the graph size. This is actually quite helpful if you downloaded the entire Philippine OSM dump. It reduced the original 900MB file to 135MB which was a lot more workable. I did hit a problem with their instructions though. The linked version of osmosis is an old one, which doesn&rsquo;t support 64-bit ids. The <a href="http://wiki.openstreetmap.org/wiki/Osmosis">latest version of Osmosis</a> easily did the job though.</p>
<p>The actual routing though, was not exactly good. I only tried one route which should normally take 1-2 transfers, it suggested a route which involved 4+ transfers. It also didn&rsquo;t provide any alternate routes aside from that one. I&rsquo;m not sure if it&rsquo;s a limitation of the provided routeserver, but I didn&rsquo;t bother checking if it supported parameters which might provide better routes.</p>
<p>I think graphserver could be useful, but it seems more involved than say OpenTripPlanner. There do seem to be people who use graphserver for their routing apps, but for the bounds of the contest, or just as a side project, it might require too much effort.</p>

		</div>
		<small class="dateline">Posted: <time class="published dt-published" itemprop="datePublished" datetime="2013-07-23">2013-07-23</time></small>
		| <small class="commentline"><a href="/posts/graphserver.html#isso-thread">Comments</a></small>
	</article>
	</article>
	
</div>
<nav class="postindexpager">
	<ul class="pager clearfix">
		
		<li class="previous">
			<a href="/page/2.html">&larr; Newer posts</a>
		</li>
		
		
		<li class="next">
			<a href="/page/4.html">Older posts &rarr;</a>
		</li>
		
	</ul>
</nav>

</main>


	<footer id="footer" role="contentinfo">
		<p>
		<a rel="license" href="http://creativecommons.org/licenses/by-sa/3.0/deed.en_US">
			<img alt="CC-BY-SA" style="border-width:0" src="https://licensebuttons.net/l/by-sa/3.0/80x15.png">
		</a> &copy; 2020 Thomas Dy - Powered by <a href="http://gohugo.io">Hugo</a></p>
	</footer>
</div>

<script src="/assets/js/konami.js"></script>
<script>
var easter_egg = new Konami();
easter_egg.code = function() {
	var el = document.getElementById('thomas');
	if(el.className == "whoa") {
		el.className = "";
	}
	else {
		el.className = "whoa";
	}
	document.body.scrollTop = document.documentElement.scrollTop = 0;
}
easter_egg.load();


</script>


<script
	data-isso="https://isso.pleasantprogrammer.com/"
	src="https://isso.pleasantprogrammer.com/js/count.min.js">
</script>


</body>
</html>