Home Explore Help
Sign In
thatsmydoing
/
blog
1
0
Fork 0
Files Issues 0 Pull Requests 0 Wiki
Tree: 15562479f1
Branches Tags
blog
/
output
/
index-1.html

index-1.html 38 KB
History Raw

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354 
  1. <!DOCTYPE html>
    2. 

  2. <html prefix="og: http://ogp.me/ns# article: http://ogp.me/ns/article# " vocab="http://ogp.me/ns" lang="en">
    3. 

  3. <head>
    4. 

  4. <meta charset="utf-8">
    5. 

  5. <meta name="viewport" content="width=device-width">
    6. 

  6. <title>Pleasant Programmer (old posts, page 1) | Pleasant Programmer</title>
    7. 

  7. <link href="assets/css/rst.css" rel="stylesheet" type="text/css">
    8. 

  8. <link href="assets/css/code.css" rel="stylesheet" type="text/css">
    9. 

  9. <link href="assets/css/theme.css" rel="stylesheet" type="text/css">
    10. 

  10. <link href="assets/css/custom.css" rel="stylesheet" type="text/css">
    11. 

  11. <link rel="alternate" type="application/rss+xml" title="RSS" href="rss.xml">
    12. 

  12. <link rel="canonical" href="http://pleasantprogrammer.com/index-1.html">
    13. 

  13. <link rel="prev" href="index-2.html" type="text/html">
    14. 

  14. <script type="text/javascript" src="//use.typekit.net/iwm5axp.js"></script><script type="text/javascript">try{Typekit.load();}catch(e){}</script><!--[if lt IE 9]><script src="assets/js/html5.js"></script><![endif]-->
    15. 

  15. </head>
    16. 

  16. <body>
    17. 

  17. <a href="#content" class="sr-only sr-only-focusable">Skip to main content</a>
    18. 

  18.     
    19. 

  19.     <header id="header" role="banner"><div class="thomas">
    20. 

  20.             <img src="assets/img/thomas.gif" alt="DJ THOMAS IN DA HAUS"><img src="assets/img/thomas.png" alt="Pleasant Programmer">
    21. 

  21. </div>
    22. 

  22.         
    23. 

  23.     <h1 id="brand"><a href="http://pleasantprogrammer.com/" title="Pleasant Programmer" rel="home">
    24. 

  24. 

  25.         <span id="blog-title">Pleasant Programmer</span>
    26. 

  26.     </a></h1>
    27. 

  27. 

  28.         
    29. 

  29. 

  30.         
    31. 

  31.     <nav id="menu" role="navigation"><ul>
    32. 

  32. <li class="twitter"><a href="http://twitter.com/pleasantprog">@pleasantprog</a></li>
    33. 

  33.                 <li><a href="archive.html">Archives</a></li>
    34. 

  34.                 <li><a href="categories/index.html">Tags</a></li>
    35. 

  35.                 <li><a href="rss.xml">RSS</a></li>
    36. 

  36.     
    37. 

  37.     
    38. 

  38.     </ul></nav></header><div id="container">
    39. 

  39.          <main id="content" role="main"><div class="postindex">
    40. 

  40.     <article class="h-entry post-text"><header><h1 class="p-name entry-title">
    41. 

  41.             <a href="posts/gtfs-editor.html" class="u-url">GTFS Editor</a>
    42. 

  42.         </h1>
    43. 

  43.     </header><div class="e-content entry-content">
    44. 

  44.     <div>
    45. 

  45. <p>Link: <a href="https://github.com/conveyal/gtfs-editor">https://github.com/conveyal/gtfs-editor</a></p>
    46. 

  46. <p><strong>TL;DR</strong> they really meant under development</p>
    47. 

  47. <p>When I first saw the source of GTFS Editor, I was ecstatic. They used <a href="http://playframework.com/">Play framework</a>!!! Not only that, they're targeting PostgreSQL as the main database. Those are our favorite tools for building webapps at By Implication. I was a bit sad though, when I saw it was on the 1.x release of Play though. I did have some experience with that release, but not as much compared to 2.x.</p>
    48. 

  48. <p>Getting it to actually run though, wasn't very pleasant. The initial setup was easy enough. Get <a href="http://www.playframework.com/download">Play 1.2.5</a>, install Postgres with PostGIS, clone the repo and create backing database in Postgres. Some minor additional steps you need are to create the PostGIS extension on the database. The schema is automatically generated and applied by Play so that should be all that's necessary. Wonderful. Then, run play, open a browser, go to <a href="http://localhost:9000">http://localhost:9000</a>, compilation error. Fantastic.</p>
    49. 

  49. <p>If you don't want to go through the technical details, you can just jump to the <a href="posts/gtfs-editor.html#conclusion">conclusion</a>.</p>
    50. 

  50. <h3>Let's Debug!</h3>
    51. 

  51. <p>I'll be splitting the next section up into 2 parts. In the first pass, I'll talk about what I did to just get the app to run but I won't try hard to fix any bugs. This generally is what I do when I try to get apps to run. I'll also be dropping enough information so that you can actually figure out what the real problem is. In the second pass, I'll explain what the problems were and how I fixed them.</p>
    52. 

  52. <h4>First Pass</h4>
    53. 

  53. <p>A thing to note about Play (and one of the reasons it's a lovely Java framework) is that you don't need to do manual compilation. Just edit some source files, refresh your browser and it will automatically do the compilation for you. One less argument for using PHP. It even shows you (in the browser!) the source and which line of code caused the compilation error. So that's what I saw, <code>Error: type Check already defined</code></p>
    54. 

  54. <pre class="code literal-block"><span class="nd">@Retention</span><span class="o">(</span><span class="n">RetentionPolicy</span><span class="o">.</span><span class="na">RUNTIME</span><span class="o">)</span>
    55. 

  55. <span class="nd">@Target</span><span class="o">({</span><span class="n">ElementType</span><span class="o">.</span><span class="na">METHOD</span><span class="o">,</span> <span class="n">ElementType</span><span class="o">.</span><span class="na">TYPE</span><span class="o">})</span>
    56. 

  56. <span class="kd">public</span> <span class="nd">@interface</span> <span class="n">Check</span> <span class="o">{</span> <span class="c1">// error here</span>
    57. 

  57. 

  58.     <span class="n">String</span><span class="o">[]</span> <span class="nf">value</span><span class="o">();</span>
    59. 

  59. <span class="o">}</span>
    60. 

  60. </pre>
    61. 

  61. 

  62. 

  63. <p>You also know that typical behavior among programmers where your program doesn't compile, but you keep trying to compile it anyway hoping that it will magically just work. That's what I did, and it actually ran. I couldn't really just let this pass, so I decided to try deleting <code>Check.java</code>. I got another compilation error, <code>Error: type Secure already defined</code></p>
    64. 

  64. <pre class="code literal-block"><span class="kd">public</span> <span class="kd">class</span> <span class="nc">Secure</span> <span class="kd">extends</span> <span class="n">Controller</span> <span class="o">{</span> <span class="c1">// error here</span>
    65. 

  65. 

  66.     <span class="nd">@Before</span><span class="o">(</span><span class="n">unless</span><span class="o">={</span><span class="s">"login"</span><span class="o">,</span> <span class="s">"authenticate"</span><span class="o">,</span> <span class="s">"logout"</span><span class="o">})</span>
    67. 

  67. 

  68.     <span class="kd">static</span> <span class="kt">void</span> <span class="nf">checkAccess</span><span class="o">()</span> <span class="kd">throws</span> <span class="n">Throwable</span> <span class="o">{</span>
    69. 

  69. </pre>
    70. 

  70. 

  71. 

  72. <p>At that point, I just decided to just debug it later. It works by just forcing it anyway. So I put <code>Check.java</code> back in and proceeded to just refresh until it compiled and ran.</p>
    73. 

  73. <p>The next problem is a sort of common thing most webapp developers have to solve one way or another. How do you set up the initial admin account? Phrased a different way, how do I login to this thing? The first thing I tried was just add a user into the <code>account</code> table directly. One problem though was how to set the password correctly. Plaintext obviously wouldn't work.</p>
    74. 

  74. <p>Another note regarding Play 1.x, it provides the <a href="http://www.playframework.com/documentation/1.2.5/secure">secure module</a> which handles logins and keeping state, you simply need to implement the method <code>boolean authenticate(String username, String password)</code>. It leaves the actual process of verifying the login to the programmer. This can be exploited by just making the method return <code>true</code> and then any login would work. No need to actually set the password. Excellent.</p>
    75. 

  75. <p>And we're logged in, just in time to encounter a runtime exception. This also works much like compilation errors in Play. It shows a page with the error and the relevant source lines. Now we get, <code>IndexOutOfBoundsException occured : Index: 0, Size: 0</code></p>
    76. 

  76. <pre class="code literal-block"><span class="k">if</span><span class="o">(</span><span class="n">session</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="s">"agencyId"</span><span class="o">)</span> <span class="o">==</span> <span class="kc">null</span><span class="o">)</span> <span class="o">{</span>
    77. 

  77. 

  78.     <span class="n">Agency</span> <span class="n">agency</span> <span class="o">=</span> <span class="n">agencies</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="mi">0</span><span class="o">);</span> <span class="c1">// error here</span>
    79. 

  79. 

  80.     <span class="n">session</span><span class="o">.</span><span class="na">put</span><span class="o">(</span><span class="s">"agencyId"</span><span class="o">,</span> <span class="n">agency</span><span class="o">.</span><span class="na">id</span><span class="o">);</span>
    81. 

  81.     <span class="n">session</span><span class="o">.</span><span class="na">put</span><span class="o">(</span><span class="s">"agencyName"</span><span class="o">,</span> <span class="n">agency</span><span class="o">.</span><span class="na">name</span><span class="o">);</span>
    82. 

  82. </pre>
    83. 

  83. 

  84. 

  85. <p>Apparently, we need to have an agency. That's generally simple enough. You just manually insert an agency into the <code>agency</code> table. After that's done, we finally have a view of the actual application. It's very Bootstrap-y, but that's just fine. The workflow though, is not perfectly intuitive, but I'll talk about that some other day.</p>
    86. 

  86. <p>That's not the end of it though, we still have to fix these bugs. The developer obviously didn't have to put up with this when they were working, so what happened? Also, the log is showing some weird things,</p>
    87. 

  87. <pre class="code literal-block">~        _            _
    88. 

  88. ~  _ __ | | __ _ _  _| |
    89. 

  89. ~ | '_ \| |/ _' | || |_|
    90. 

  90. ~ |  __/|_|\____|\__ (_)
    91. 

  91. ~ |_|            |__/
    92. 

  92. ~
    93. 

  93. ~ play! 1.2.5, http://www.playframework.org
    94. 

  94. ~
    95. 

  95. ~ Ctrl+C to stop
    96. 

  96. ~
    97. 

  97. CompilerOracle: exclude jregex/Pretokenizer.next
    98. 

  98. Listening for transport dt_socket at address: 8000
    99. 

  99. 23:32:14,943 INFO  ~ Starting /Users/thomas/Workspace/maps/gtfs-editor
    100. 

  100. 23:32:14,948 WARN  ~ Declaring modules in application.conf is deprecated. Use dependencies.yml instead (module.secure)
    101. 

  101. 23:32:14,948 INFO  ~ Module secure is available (/Users/thomas/.root/opt/play-1.2.5/modules/secure)
    102. 

  102. 23:32:15,830 WARN  ~ You're running Play! in DEV mode
    103. 

  103. 23:32:15,952 INFO  ~ Listening for HTTP on port 9000 (Waiting a first request to start) ...
    104. 

  104. 23:32:28,792 ERROR ~
    105. 

  105. 

  106. @6f02fa9dd
    107. 

  107. Internal Server Error (500) for request GET /
    108. 

  108. 

  109. Compilation error (In /app/controllers/Check.java around line 10)
    110. 

  110. The file /app/controllers/Check.java could not be compiled. Error raised is : The type Check is already defined
    111. 

  111. 

  112. play.exceptions.CompilationException: The type Check is already defined
    113. 

  113.     at play.classloading.ApplicationCompiler$2.acceptResult(ApplicationCompiler.java:246)
    114. 

  114.     at org.eclipse.jdt.internal.compiler.Compiler.handleInternalException(Compiler.java:672)
    115. 

  115.     at org.eclipse.jdt.internal.compiler.Compiler.compile(Compiler.java:516)
    116. 

  116.     at play.classloading.ApplicationCompiler.compile(ApplicationCompiler.java:282)
    117. 

  117.     at play.classloading.ApplicationClassloader.getAllClasses(ApplicationClassloader.java:426)
    118. 

  118.     at play.Play.start(Play.java:516)
    119. 

  119.     at play.Play.detectChanges(Play.java:630)
    120. 

  120.     at play.Invoker$Invocation.init(Invoker.java:198)
    121. 

  121.     at Invocation.HTTP Request(Play!)
    122. 

  122. 23:32:31,551 INFO  ~ Connected to jdbc:postgresql://127.0.0.1/gtfs_editor
    123. 

  123. SLF4J: Class path contains multiple SLF4J bindings.
    124. 

  124. SLF4J: Found binding in [jar:file:/Users/thomas/Workspace/maps/gtfs-editor/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    125. 

  125. SLF4J: Found binding in [jar:file:/Users/thomas/.root/opt/play-1.2.5/framework/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    126. 

  126. SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    127. 

  127. 23:32:32,490 INFO  ~ Initializing HBSpatialExtension
    128. 

  128. 23:32:32,492 INFO  ~ Attempting to load Hibernate Spatial Provider org.hibernatespatial.postgis.DialectProvider
    129. 

  129. 23:32:32,494 INFO  ~ Checking for default configuration file.
    130. 

  130. 23:32:32,496 INFO  ~ No configuration file hibernate-spatial.cfg.xml on the classpath.
    131. 

  131. 23:32:34,077 INFO  ~ Application 'gtfs-editor' is now started !
    132. 

  132. 23:32:34,151 INFO  ~ Bootstrapping Database...
    133. 

  133. 23:32:34,297 DEBUG ~ select count(*) as col_0_0_ from Agency agency0_ limit ?
    134. 

  134. play.exceptions.UnexpectedException: Unexpected Error
    135. 

  135.     at play.vfs.VirtualFile.contentAsString(VirtualFile.java:180)
    136. 

  136.     at play.templates.TemplateLoader.load(TemplateLoader.java:78)
    137. 

  137.     at play.test.Fixtures.loadModels(Fixtures.java:174)
    138. 

  138.     at jobs.BootstrapDatabase.doJob(BootstrapDatabase.java:57)
    139. 

  139.     at play.jobs.Job.doJobWithResult(Job.java:50)
    140. 

  140.     at play.jobs.Job.call(Job.java:146)
    141. 

  141.     at play.jobs.Job.run(Job.java:132)
    142. 

  142.     at play.jobs.JobsPlugin.afterApplicationStart(JobsPlugin.java:116)
    143. 

  143.     at play.plugins.PluginCollection.afterApplicationStart(PluginCollection.java:531)
    144. 

  144.     at play.Play.start(Play.java:547)
    145. 

  145.     at play.Play.detectChanges(Play.java:630)
    146. 

  146.     at play.Invoker$Invocation.init(Invoker.java:198)
    147. 

  147.     at play.server.PlayHandler$NettyInvocation.init(PlayHandler.java:189)
    148. 

  148.     at play.Invoker$Invocation.run(Invoker.java:276)
    149. 

  149.     at play.server.PlayHandler$NettyInvocation.run(PlayHandler.java:229)
    150. 

  150.     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
    151. 

  151.     at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    152. 

  152.     at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    153. 

  153.     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
    154. 

  154.     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
    155. 

  155.     at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
    156. 

  156.     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
    157. 

  157.     at java.lang.Thread.run(Thread.java:680)
    158. 

  158. Caused by: play.exceptions.UnexpectedException: Unexpected Error
    159. 

  159.     at play.vfs.VirtualFile.inputstream(VirtualFile.java:111)
    160. 

  160.     at play.vfs.VirtualFile.contentAsString(VirtualFile.java:178)
    161. 

  161.     ... 22 more
    162. 

  162. Caused by: java.io.FileNotFoundException: /Users/thomas/.root/opt/play-1.2.5/modules/docviewer/app/initial-agencies-data.yml (No such file or directory)
    163. 

  163.     at java.io.FileInputStream.open(Native Method)
    164. 

  164.     at java.io.FileInputStream.&lt;init&gt;(FileInputStream.java:120)
    165. 

  165.     at play.vfs.VirtualFile.inputstream(VirtualFile.java:109)
    166. 

  166.     ... 23 more
    167. 

  167. 23:32:34,316 ERROR ~ java.lang.RuntimeException: Cannot load fixture initial-agencies-data.yml: Unexpected Error
    168. 

  168. 23:32:40,989 DEBUG ~ select account0_.id as id15_, account0_.active as active15_, account0_.admin as admin15_, account0_.agency_id as agency9_15_, account0_.email as email15_, account0_.lastLogin as lastLogin15_, account0_.password as password15_, account0_.passwordChangeToken as password7_15_, account0_.username as username15_ from Account account0_ where account0_.username=? limit ?
    169. 

  169. 23:32:40,994 DEBUG ~ select count(*) as col_0_0_ from Account account0_ limit ?
    170. 

  170. 23:32:40,999 DEBUG ~ select nextval ('hibernate_sequence')
    171. 

  171. 23:32:41,051 DEBUG ~ insert into Account (active, admin, agency_id, email, lastLogin, password, passwordChangeToken, username, id) values (?, ?, ?, ?, ?, ?, ?, ?, ?)
    172. 

  172. 23:32:41,061 DEBUG ~ select agency0_.id as id24_, agency0_.color as color24_, agency0_.defaultLat as defaultLat24_, agency0_.defaultLon as defaultLon24_, agency0_.defaultRouteType_id as default12_24_, agency0_.gtfsAgencyId as gtfsAgen5_24_, agency0_.lang as lang24_, agency0_.name as name24_, agency0_.phone as phone24_, agency0_.systemMap as systemMap24_, agency0_.timezone as timezone24_, agency0_.url as url24_ from Agency agency0_ order by agency0_.name
    173. 

  173. 23:32:41,175 ERROR ~
    174. 

  174. 

  175. @6f02fa9dg
    176. 

  176. Internal Server Error (500) for request GET /
    177. 

  177. 

  178. Execution exception (In /app/controllers/Application.java around line 57)
    179. 

  179. IndexOutOfBoundsException occured : Index: 0, Size: 0
    180. 

  180. 

  181. play.exceptions.JavaExecutionException: Index: 0, Size: 0
    182. 

  182.     at play.mvc.ActionInvoker.invoke(ActionInvoker.java:237)
    183. 

  183.     at Invocation.HTTP Request(Play!)
    184. 

  184. Caused by: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
    185. 

  185.     at java.util.ArrayList.RangeCheck(ArrayList.java:547)
    186. 

  186.     at java.util.ArrayList.get(ArrayList.java:322)
    187. 

  187.     at controllers.Application.initSession(Application.java:57)
    188. 

  188.     at play.mvc.ActionInvoker.invoke(ActionInvoker.java:510)
    189. 

  189.     at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:484)
    190. 

  190.     at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:479)
    191. 

  191.     at play.mvc.ActionInvoker.handleBefores(ActionInvoker.java:328)
    192. 

  192.     at play.mvc.ActionInvoker.invoke(ActionInvoker.java:142)
    193. 

  193.     ... 1 more
    194. 

  194. </pre>
    195. 

  195. 

  196. 

  197. <p>After <code>23:32:34</code> is when I get the login page. <code>23:32:40</code> is after I've logged in.</p>
    198. 

  198. <h4>Second Pass</h4>
    199. 

  199. <p>So how did you do? First, the error that <code>type Check already defined</code> usually does mean that <code>Check</code> was already defined elsewhere. Looking in the app folder though, there was nothing of the sort. It's the only one there that was <code>Check.java</code>. But remember the secure module? Modules work by providing source files and Play just compiles them all together. Bingo, <code>Check.java</code>. Doing a diff shows nothing was changed. So the solution really was just simply delete <code>Check.java</code> and also <code>Secure.java</code>. No more compilation errors!</p>
    200. 

  200. <p>The next question is, how do you get the initial user? There actually is some code that looks like it creates the default admin user,</p>
    201. 

  201. <pre class="code literal-block"><span class="k">if</span><span class="o">(</span><span class="n">Security</span><span class="o">.</span><span class="na">isConnected</span><span class="o">())</span> <span class="o">{</span>
    202. 

  202.     <span class="o">...</span>
    203. 

  203.     <span class="n">Account</span> <span class="n">account</span> <span class="o">=</span> <span class="n">Account</span><span class="o">.</span><span class="na">find</span><span class="o">(</span><span class="s">"username = ?"</span><span class="o">,</span> <span class="n">Security</span><span class="o">.</span><span class="na">connected</span><span class="o">()).</span><span class="na">first</span><span class="o">();</span>
    204. 

  204.     <span class="o">...</span>
    205. 

  205.     <span class="k">if</span><span class="o">(</span><span class="n">account</span> <span class="o">==</span> <span class="kc">null</span> <span class="o">&amp;&amp;</span> <span class="n">Account</span><span class="o">.</span><span class="na">count</span><span class="o">()</span> <span class="o">==</span> <span class="mi">0</span><span class="o">)</span> <span class="o">{</span>
    206. 

  206.         <span class="n">account</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Account</span><span class="o">(</span><span class="s">"admin"</span><span class="o">,</span> <span class="s">"admin"</span><span class="o">,</span> <span class="s">"admin@test.com"</span><span class="o">,</span> <span class="kc">true</span><span class="o">,</span> <span class="kc">null</span><span class="o">);</span>
    207. 

  207.         <span class="n">account</span><span class="o">.</span><span class="na">save</span><span class="o">();</span>
    208. 

  208.     <span class="o">}</span>
    209. 

  209.     <span class="o">...</span>
    210. 

  210. <span class="o">}</span>
    211. 

  211. </pre>
    212. 

  212. 

  213. 

  214. <p>You can actually see this in action at <code>23:32:41,051</code> in the log. So what's wrong with all of this? The account creation happened after I've already logged in. In fact, <code>Security.isConnected()</code> checks whether the user is already logged in or not. How does this even make sense?</p>
    215. 

  215. <p>Lastly, we have the problem of the agencies. Just by looking at the log, you can safely say we're missing a file called <code>initial-agencies-data.yml</code>. Ok, apparently it's a <a href="http://www.playframework.com/documentation/1.2.5/test#fixtures">fixture</a> like you would use for testing. It's easy enough to infer what the file's contents should be. We just copy it over from the GTFS data.</p>
    216. 

  216. <p>But then where do you put the file? If you look at the log, it says <code>/Users/thomas/.root/opt/play-1.2.5/modules/docviewer/app/initial-agencies-data.yml</code> but that doesn't look right. That's in the Play distribution directory, probably not somewhere something app-specific should go into. Well, a fixture is used for testing, so maybe the <code>test/</code> directory? No, that doesn't work either since we're not running a test.</p>
    217. 

  217. <p>What I ended up doing was just looking at the sources for <code>Fixtures.load</code>. If you follow the stack trace, you end up finding <code>Play.javaPath</code> which sort of works like PATH for Fixtures and some other things. So where can we put the file? <code>app/</code> and <code>conf/</code>. And with that, we're done.</p>
    218. 

  218. <h4 id="conclusion">Conclusion</h4>
    219. 

  219. 

  220. <p>GTFS Editor is very much in development. Just getting it to run was problematic. There also seem to be a lot of missing issues judging from the Github Issues page. If you want to try it out for yourself, I suggest you clone <a href="https://github.com/thatsmydoing/gtfs-editor">my branch</a> as I've fixed the issues discussed earlier. The default login is <code>admin:admin</code>.</p>
    221. 

  221. <p>Even after getting it to run, it's still not quite usable. Not in the UX sense, but you really can't do much with it. There is no way to import the GTFS data into the webapp. There is something like import from TransitWand but even that is unclear to me. And even if we do get that running as well, we still don't have any data we can play around with. We would need database dumps from the already running tools for these to be of any use right now.</p>
    222. 

  222. </div>
    223. 

  223.     </div>
    224. 

  224.     <small class="dateline">Posted: <time class="published dt-published" datetime="2013-07-10T11:30:01+08:00" title="2013-07-10 11:30">2013-07-10 11:30</time></small>
    225. 

  225.         | <small class="commentline">
    226. 

  226.         
    227. 

  227.     <a href="posts/gtfs-editor.html#disqus_thread" data-disqus-identifier="cache/posts/gtfs-editor.html">Comments</a>
    228. 

  228. 

  229. </small>
    230. 

  230.     </article><article class="h-entry post-text"><header><h1 class="p-name entry-title">
    231. 

  231.             <a href="posts/open-trip-planner.html" class="u-url">Open Trip Planner</a>
    232. 

  232.         </h1>
    233. 

  233.     </header><div class="e-content entry-content">
    234. 

  234.     <div>
    235. 

  235. <p>Link: <a href="http://www.opentripplanner.org">http://www.opentripplanner.org</a></p>
    236. 

  236. <p><strong>TL;DR</strong> routes pretty well; data might cause weird issues</p>
    237. 

  237. <p>OpenTripPlanner, as the name implies, is a routing app. Given point A and point B, it can provide possible routes by taking transit, riding a bike, or a mix of both. You can also specify options on how much walking you're willing to do or if you prefer fewer transfers over trip time. It could be a good competitor to the transit directions of Google Maps.</p>
    238. 

  238. <p>It's actually in general use by the <a href="http://maps.trimet.org/">Trimet</a>, Portland's public transit system. I think a good reason why they deployed OpenTripPlanner is that Portland is a very bike friendly area. OpenTripPlanner's support for multi-modal (bike + transit) routing is one thing that even Google Maps doesn't have. This actually just screams <a href="http://philippine-transit.hackathome.com/prizes/">Inclusive Technology Award</a>.</p>
    239. 

  239. <h4>Setup</h4>
    240. 

  240. <p>Getting OpenTripPlanner up and running involves a bit more downloading than OneBusAway. I'd suggest going through the <a href="https://github.com/openplans/OpenTripPlanner/wiki/FiveMinutes">5-minute introduction</a> if you want to actually work with the Philippine data. You will also need to download the <a href="http://download.geofabrik.de/asia/philippines.html">Philippine data dump</a> from OSM. You will want the <code>osm.bz2</code> one (WARNING: 900MB unzipped).</p>
    241. 

  241. <p>Once you get the webapp running, you'll notice the map tiles won't load correctly. This is because the default tileset used is from Mapbox which doesn't provide publicly free tilesets anymore. To actually see things on the map, you should click the + on the upper-right of the map and change the base layer to something like Open Street Map or OSM Mapquest. This has been fixed in their latest sources.</p>
    242. 

  242. <h4>Issues</h4>
    243. 

  243. <p>The next thing you'll notice is that you can't search for a place. You can only just pick points on the map and route between those. I'm still not exactly sure if it's supposed to have it, since the Trimet one has it. But even then adding it by using the Google Maps or MapQuest APIs shouldn't be too difficult.</p>
    244. 

  244. <p>Another missing thing compared to the Trimet planner is being able to look at the routes akin to OneBusAway. And even then, Trimet's implementation isn't as good for exploring as OneBusAway is. Seeing the schedule or seeing which routes pass through a stop are left to an external site to do.</p>
    245. 

  245. <p>There have been some weird issues with the routing though. A common occurence is the steps are somewhat disconnected (see image below). The left half shows OpenTripPlanner, it shows that you get off at a "stop" in EDSA and you should magically teleport to Arguilla street and start walking. To be fair, Google Maps (right half) shows that there is a street from the "stop" to Arguilla.</p>
    246. 

  246. <p><img alt="OpenTripPlanner disconnected route" src="galleries/transit/otp1.png"></p>
    247. 

  247. <p>I can't say this isn't a bug. After all, OpenTripPlanner couldn't have magically known there really was a street there. If it did, it wouldn't just teleport you to the corner. I'm more inclined to think though that this is a result of not having any shape data. As a result, OpenTripPlanner might try to assume the stop could mean places within a certain radius of where it was defined.</p>
    248. 

  248. <p>Another weird issue that pops up sometimes is where it tells you to ride a jeep past the stop you want and then after a while, get off and ride a jeep back to your stop. A more general issue is that sometimes it won't give you the best route because it thinks you have to loop around to get to where you want to be. I don't really know how common this issue pops up though. It's highly dependent on where you put the marker. If you just move the marker down a little bit, it actually does give the correct route.</p>
    249. 

  249. <p><img alt="OpenTripPlanner loopy route" src="galleries/transit/otp2.png"></p>
    250. 

  250. <p>Much like the earlier problem, I can't tell if this is a bug or it's a result of the bad data. Once again though, I think the issue is more of bad data. If you look at the plotted UP-Katipunan route, the stops aren't even on the road. This probably makes it more difficult for OpenTripPlanner to actually tell if the stop and road are connected.</p>
    251. 

  251. <p><img alt="UP Katipunan Route" src="galleries/transit/upkatipunan.jpg"></p>
    252. 

  252. <p>From what they said during the launch, most of the route data was collected by getting a person to ride a jeep with a smartphone. That would explain why the coordinates aren't that exact. Even then, it would have been nice if they at least cleaned up the data by moving the stops to the road. They would have had to go over them to name the stops anyway.</p>
    253. 

  253. <h4>Conclusion</h4>
    254. 

  254. <p>Overall though, I really like OpenTripPlanner. It handles most of the hard parts of the challenge. It provides a REST API for doing routing with the GTFS + OSM data. There's also a lot of potential for additional open source work. A lot can be done to improve the default webapp. Adding a default location searcher would greatly improve usability. Adding in the route viewing features of OneBusAway would also be nice. Alternatively, you could even write your own client that just interfaces with the API.</p>
    255. 

  255. </div>
    256. 

  256.     </div>
    257. 

  257.     <small class="dateline">Posted: <time class="published dt-published" datetime="2013-07-09T23:16:12+08:00" title="2013-07-09 23:16">2013-07-09 23:16</time></small>
    258. 

  258.         | <small class="commentline">
    259. 

  259.         
    260. 

  260.     <a href="posts/open-trip-planner.html#disqus_thread" data-disqus-identifier="cache/posts/open-trip-planner.html">Comments</a>
    261. 

  261. 

  262. </small>
    263. 

  263.     </article><article class="h-entry post-text"><header><h1 class="p-name entry-title">
    264. 

  264.             <a href="posts/one-bus-or-maybe-jeep-away.html" class="u-url">One Bus (or maybe Jeep) Away</a>
    265. 

  265.         </h1>
    266. 

  266.     </header><div class="e-content entry-content">
    267. 

  267.     <div>
    268. 

  268. <p>Link: <a href="http://onebusaway.org/">http://onebusaway.org/</a></p>
    269. 

  269. <p><strong>TL;DR</strong> no routing; useless in Philippines</p>
    270. 

  270. <p>OneBusAway is a transit information app. It provides data on what bus stops are near you, which buses pass by. You can also get schedules and the route of a particular bus given the number. It can also provide realtime updates like how many minutes until the next bus arrives. It does not, however, provide routing. There is no support for providing directions to get from point A to point B.</p>
    271. 

  271. <p>It's comparable to what you get in some bus stops abroad. You'd get a vicinity map and a list of buses passing through the stop. You might also get the times when the next buses will pass. It's useful for locals who already know how to get around, and want to avoid waiting for the bus. But it's not particularly good for people who want to know how to get around the city.</p>
    272. 

  272. <p>OneBusAway is quite comprehensive in its platform support though. There is a webapp, apps for iOS, Android and Windows Phone, as well as SMS and Voice support. This would all be nice but we don't have the necessary infrastructure yet in the Philippines. We don't have bus or jeepney stops. We also wouldn't have realtime data to make the app particularly useful.</p>
    273. 

  273. <p>You can try it out for yourself by following their <a href="https://github.com/OneBusAway/onebusaway-application-modules/wiki/OneBusAway-Quickstart-Guide">Quickstart Guide</a>. One caveat is you will have to add <code>-P tripEntriesFactory.throwExceptionOnInvalidStopToShapeMappingException=false</code> when building the bundle. This has to do with the OneBusAway having difficulty matching the <a href="https://github.com/OneBusAway/onebusaway-application-modules/wiki/Stop-to-Shape-Matching">stops to the shape data</a>.</p>
    274. 

  274. <p>Here's some screenshots of the app with the Philippine data. Notice how you only see the stops but there isn't a line for the route. This is a problem with our GTFS data. Also, at some points it's hard to tell where the jeep is going to pass since there isn't any indication of order either. This is more of a OneBusAway problem. It usually expects there to be shape data available.</p>
    275. 

  275. <p><a href="galleries/transit/onebusaway1.png"><img alt="OneBusAway" src="galleries/transit/onebusaway1.png" title="All the stops along Katipunan Avenue are named Katipunan Avenue."></a></p>
    276. 

  276. <p>It doesn't really handle too many routes passing through a stop. The list just overflows past the bubble. You can still actually read it by panning the map. It's just a bit weird though.</p>
    277. 

  277. <p>If you also noticed, there are usually 2 of each route. This is how the jeepney data was modeled as jeep routes might be different going one way and going back. This isn't the case for all jeeps though, so it might also be an implementation issue with the GTFS editor.</p>
    278. 

  278. <p><a href="galleries/transit/onebusaway2.png"><img alt="OneBusAway" src="galleries/transit/onebusaway2.png"></a></p>
    279. 

  279. <p><a href="galleries/transit/onebusaway3.png"><img alt="OneBusAway" src="galleries/transit/onebusaway3.png"></a></p>
    280. 

  280. </div>
    281. 

  281.     </div>
    282. 

  282.     <small class="dateline">Posted: <time class="published dt-published" datetime="2013-07-09T01:53:59+08:00" title="2013-07-09 01:53">2013-07-09 01:53</time></small>
    283. 

  283.         | <small class="commentline">
    284. 

  284.         
    285. 

  285.     <a href="posts/one-bus-or-maybe-jeep-away.html#disqus_thread" data-disqus-identifier="cache/posts/one-bus-or-maybe-jeep-away.html">Comments</a>
    286. 

  286. 

  287. </small>
    288. 

  288.     </article><article class="h-entry post-text"><header><h1 class="p-name entry-title">
    289. 

  289.             <a href="posts/jeepney-and-bus-routes.html" class="u-url">Jeepney and Bus Routes</a>
    290. 

  290.         </h1>
    291. 

  291.     </header><div class="e-content entry-content">
    292. 

  292.     <div>
    293. 

  293. <p>In the <a href="posts/philippine-transit-app-challenge.html">last post</a>, I talked about how we now have data about jeepney and bus routes in the Philippines. The data is actually in the <a href="https://developers.google.com/transit/gtfs/">GTFS format</a>, which is the format the Google Maps consumes transit data. Apparently, the government will be submitting the GTFS data later this year. Transit directions for Metro Manila in Google Maps would be wonderful. That said, it definitely raises the bar for the app challenge people.</p>
    294. 

  294. <p>In the last post, I mentioned the quality of the data isn't quite good. Even before seeing the data, I was already a bit unsure of it. The key problem is how you model the routes. The GTFS format was inherently designed for more well developed and organized transit agencies which isn't exactly what we have in the Philippines now.</p>
    295. 

  295. <p>One potential problem is the nature of the jeeps and buses. GTFS routes are a collection of trips which are a sequence of stops. However, we don't have jeepney stops, and even if we did they still just stop anywhere. There are also times where jeeps will take a shortcut if no passengers need to get dropped off along their normal route.</p>
    296. 

  296. <p>From what I've seen of the data, they handled the first problem well enough. Stops are defined as where people typically get on the jeep or bus. This is good, but they didn't define a shape for the routes. There is no information as to which exact roads they pass through. All we have to go by are the stops to show the route on a map.</p>
    297. 

  297. <p><img alt="sample route" src="http://i.imgur.com/NSVlryE.jpg"></p>
    298. 

  298. <p>The problem isn't that bad though. The agencies could still add the shapes later on. Or maybe an app challenge participant could make an app around fixing the routes via crowd-sourcing or similar. The shape itself isn't that important for a rudimentary directions app, but if we want better apps, we will need better data.</p>
    299. 

  299. <p>There were also some minor issues with the data itself. Some of the files had extra columns. This normally isn't an issue, but it caused problems for <a href="https://github.com/harrisony/gtfs_SQL_importer">GTFS SQL importer</a>. There were also problems with matching the shape data with the stops when I tried it with <a href="http://onebusaway.org">OneBusAway</a>. They could probably be <a href="https://github.com/OneBusAway/onebusaway-application-modules/wiki/Stop-to-Shape-Matching">fixed</a> but that's for another day.</p>
    300. 

  300. </div>
    301. 

  301.     </div>
    302. 

  302.     <small class="dateline">Posted: <time class="published dt-published" datetime="2013-07-07T10:32:36+08:00" title="2013-07-07 10:32">2013-07-07 10:32</time></small>
    303. 

  303.         | <small class="commentline">
    304. 

  304.         
    305. 

  305.     <a href="posts/jeepney-and-bus-routes.html#disqus_thread" data-disqus-identifier="cache/posts/jeepney-and-bus-routes.html">Comments</a>
    306. 

  306. 

  307. </small>
    308. 

  308.     </article><article class="h-entry post-text"><header><h1 class="p-name entry-title">
    309. 

  309.             <a href="posts/philippine-transit-app-challenge.html" class="u-url">Philippine Transit App Challenge</a>
    310. 

  310.         </h1>
    311. 

  311.     </header><div class="e-content entry-content">
    312. 

  312.     <div>
    313. 

  313. <p>Last week, the DOTC launched the <a href="http://philippine-transit.hackathome.com">Philippine Transit App Challenge</a>. It's a competition to build something great using the newly available 1) jeepney/bus/rail routes and 2) traffic incident data in Metro Manila and Cebu.</p>
    314. 

  314. <p>I'm actually quite excited about this as it's not everyday our government does wonderful things. Many people have been waiting for this kind of data to be available. Before, the only way to figure out which jeeps to ride to get from A to B is by asking other people. The website of the LTFRB used to just have a list of jeepney routes, but nothing else, no maps or list of stops.</p>
    315. 

  315. <p>During the launch, they also presented how CITOM (the Cebu MMDA) is pilot-testing a <a href="http://cebutraffic.org/">traffic tracking system</a>. What they did was supply Android phones to taxi drivers. The phones send GPS data which can then be aggregated to see what the average speeds along streets are. This also benefits the taxi company as it provides easy tracking of their taxis compared to their old telephone/radio with pen-and-paper process.</p>
    316. 

  316. <p>The <a href="http://philippine-transit.hackathome.com/dataset-philippines-transit-information-service-gtfs/">route data</a> are already available, as well as the <a href="http://philippine-transit.hackathome.com/dataset-citom-traffic-alert-platform/">Cebu</a> and <a href="http://philippine-transit.hackathome.com/dataset-mmda-traffic-alert-platform/">Metro Manila</a> incident data. You do have to register to access them though. Right now, they're just one-off dumps of the data but the various agencies have promised to provide consistently updated data. This will be provided via ASTI later in the year.</p>
    317. 

  317. <p>I've just started looking at the route data and playing around with it. The quality could be better, but I'm glad we at least have something to work with.</p>
    318. 

  318. </div>
    319. 

  319.     </div>
    320. 

  320.     <small class="dateline">Posted: <time class="published dt-published" datetime="2013-07-07T00:17:37+08:00" title="2013-07-07 00:17">2013-07-07 00:17</time></small>
    321. 

  321.         | <small class="commentline">
    322. 

  322.         
    323. 

  323.     <a href="posts/philippine-transit-app-challenge.html#disqus_thread" data-disqus-identifier="cache/posts/philippine-transit-app-challenge.html">Comments</a>
    324. 

  324. 

  325. </small>
    326. 

  326.     </article>
    327. 

  327. </div>
    328. 

  328. 

  329.         <nav class="postindexpager"><ul class="pager clearfix">
    330. 

  330. <li class="previous">
    331. 

  331.                 <a href="index-2.html" rel="prev">← Newer posts</a>
    332. 

  332.             </li>
    333. 

  333.         </ul></nav><script>var disqus_shortname="pleasantprog";(function(){var a=document.createElement("script");a.async=true;a.src="//"+disqus_shortname+".disqus.com/count.js";(document.getElementsByTagName("head")[0]||document.getElementsByTagName("body")[0]).appendChild(a)}());</script></main><footer id="footer" role="contentinfo"><p><a rel="license" href="http://creativecommons.org/licenses/by-sa/3.0/deed.en_US"><img alt="CC-BY-SA" style="border-width:0" src="http://i.creativecommons.org/l/by-sa/3.0/80x15.png"></a> © 2015 Thomas Dy - Powered by <a href="http://getnikola.com">Nikola</a></p>
    334. 

  334.             
    335. 

  335.         </footer>
    336. 

  336. </div>
    337. 

  337.     
    338. 

  338.     
    339. 

  339.     
    340. 

  340.     
    341. 

  341. 

  342.     <script src="assets/js/konami.js"></script><script src="http://code.jquery.com/jquery-2.0.3.min.js"></script><script>
    343. 

  343.         var easter_egg = new Konami();
    344. 

  344.         easter_egg.code = function() {
    345. 

  345.             $(".thomas").toggleClass("whoa");
    346. 

  346.             $("body").scrollTop(0);
    347. 

  347.         }
    348. 

  348.         easter_egg.load();
    349. 

  349. 

  350.         // love you, thomas!
    351. 

  351.         // yours, @_phi + @meggykawsek
    352. 

  352.     </script>
    353. 

  353. </body>
    354. 

  354. </html>
    355.