index.html 35 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551
  1. <!DOCTYPE html>
  2. <html lang="en-us">
  3. <head>
  4. <meta charset="utf-8">
  5. <meta name="generator" content="Hugo 0.18.1" />
  6. <meta name="viewport" content="width=device-width, initial-scale=1">
  7. <link rel="stylesheet" href="/assets/css/theme.css">
  8. <link rel="alternate" href="/rss.xml" type="application/rss+xml" title="Pleasant Programmer">
  9. <script type="text/javascript" src="//use.typekit.net/iwm5axp.js"></script>
  10. <script type="text/javascript">try{Typekit.load();}catch(e){}</script>
  11. <title>Pleasant Programmer</title>
  12. </head>
  13. <body>
  14. <header id="header" role="banner">
  15. <div id="thomas">
  16. <img src="/assets/img/thomas.gif" alt="DJ THOMAS IN DA HAUS">
  17. <img src="/assets/img/thomas.png" alt="Pleasant Programmer">
  18. </div>
  19. <h1 class="site-title"><a href="/">Pleasant Programmer</a></h1>
  20. <nav id="menu" role="navigation">
  21. <ul>
  22. <li class="twitter">
  23. <a href="http://twitter.com/pleasantprog">@pleasantprog</a>
  24. </li>
  25. <li><a href="/posts.html">archives</a></li>
  26. <li><a href="/tags.html">tags</a></li>
  27. <li><a href="/rss.xml">rss</a></li>
  28. </ul>
  29. </nav>
  30. </header>
  31. <div id="container">
  32. <main id="content" role="main">
  33. <div class="postindex">
  34. <article class="h-entry post-text" itemscope itemtype="http://schema.org/Blog">
  35. <header>
  36. <h1 class="p-name entry-title" itemprop="headline">
  37. <a href="/posts/audventure.html" class="u-url">Audventure</a>
  38. </h1>
  39. </header>
  40. <div class="e-content entry-content">
  41. <p>Sometime around 2013 I wrote a clone of the GBA game <a href="https://www.nintendo.co.jp/n08/bit_g/">bit Generations
  42. SoundVoyager</a> called
  43. <a href="https://audventure.pleasantprogrammer.com">audventure</a>. SoundVoyager is
  44. actually a collection of mini-games where sound is the main focus. You can
  45. actually play the game blind, and at some point, that&rsquo;s pretty much what
  46. happens.</p>
  47. <h2 id="sound-catcher">sound catcher</h2>
  48. <p>The signature mini-game in SoundVoyager is sound catcher. In the mini-game, you
  49. can only move left and right at the bottom of the stage, while a &ldquo;sound&rdquo; falls
  50. from the top. Your goal is to catch the sound which is signified by a green dot.
  51. When you catch it, the sound or beat becomes part of the BGM and a new dot
  52. appears with a different sound.</p>
  53. <p>You can of course use your eyes and move accordingly, but if you put on
  54. earphones, you can actually hear where the dot is, either on your left or right,
  55. with it getting louder as it gets close to you. As you collect more sounds, the
  56. dot gets more and more transparent. Eventually (and this is where it gets fun),
  57. you won&rsquo;t be able to see the sounds anymore and will have to rely mostly on your
  58. ears.</p>
  59. <p>You can see what the original game looks like in <a href="https://www.youtube.com/watch?v=C12WRgfIOC8">this
  60. video</a> or you can play it under
  61. <em>sound safari</em> in <a href="https://audventure.pleasantprogrammer.com">audventure</a>.</p>
  62. <h2 id="webaudio-vs-flash">WebAudio vs Flash</h2>
  63. <p>At the time I wrote audventure, only Chrome supported WebAudio. Also, the API
  64. looked (and still looks) quite complicated. Flash on the other hand, was
  65. starting to die, but still well-supported so I went with that. For the most
  66. part, it worked okay though Chrome actually had timing issues when playing
  67. sounds. Now, it doesn&rsquo;t work in any browser. I tried to debug the issues but
  68. ultimately ended up just rewriting it to use WebAudio instead.</p>
  69. <p>For the game, I needed to simulate the source of the sound in 2D/3D space. Flash
  70. only really gives you stereo panning and volume control. With some maths, we can
  71. actually get an acceptable solution. Less importantly, I needed to be able to
  72. get frequency data of the currently playing &ldquo;sound&rdquo; to pulse the background. For
  73. this, I actually had to implement the feature in the Flash library I was using.</p>
  74. <p>With WebAudio, spatial audio is already built-in and you can simply give it the
  75. coordinates of the sounds and the listener. There are some other options to
  76. tweak, but for the most part, no complex math is needed. Getting frequency data
  77. for a sound is also actually built-in and didn&rsquo;t take too long to integrate.</p>
  78. <p>Overall, I was impressed by how much you can do with WebAudio out-of-the-box. I
  79. kind of understand why it&rsquo;s complicated, but there&rsquo;s some simple functionality
  80. that I wish was included. For example, there is no API to pause and then resume
  81. playing an audio buffer. You have to manually save the elapsed time and play
  82. from there.</p>
  83. <h2 id="other-mini-games">Other mini-games</h2>
  84. <p>So far I&rsquo;ve only actually implemented the sound catcher mini-game. There are
  85. around 4 different categories with slight variations in between.</p>
  86. <h3 id="sound-catcher-sound-slalom">sound catcher / sound slalom</h3>
  87. <p>I&rsquo;ve explained sound catcher a while ago; sound slalom is a minor variation on
  88. that. Instead of waiting for the &ldquo;sound&rdquo; to reach you, you now have to guide
  89. yourself in between 2 &ldquo;poles&rdquo; of sound, as in <a href="https://en.wikipedia.org/wiki/Slalom_skiing">slalom
  90. skiing</a>. But this time, you can
  91. also accelerate forward. The goal is to finish the course before the time runs
  92. out.</p>
  93. <h3 id="sound-drive-sound-chase">sound drive / sound chase</h3>
  94. <p>In sound drive, you&rsquo;re driving against the flow on a 5 lane road. You have to
  95. avoid oncoming cars, trucks and animals until you reach the end. You&rsquo;re allowed
  96. to change lanes and accelerate, and the game tracks your best time. Sound chase
  97. is pretty much the same, except you&rsquo;re trying to catch up to a &ldquo;sound&rdquo;.</p>
  98. <h3 id="sound-cannon">sound cannon</h3>
  99. <p>In sound cannon, you&rsquo;re immobile but can rotate within a 180 degree angle. Your
  100. goal is too shoot down &ldquo;sounds&rdquo; which are heading your way. If a sound reaches
  101. you, it&rsquo;s game over. You win when you kill all the sounds.</p>
  102. <h3 id="sound-picker-sound-cock">sound picker / sound cock</h3>
  103. <p>In sound picker, you can move in a giant square field where various sounds are
  104. scattered around. Your goal is to pick up all the sounds within the time limit.
  105. Sound cock is similar, except the sounds are chickens and you have to chase them
  106. around.</p>
  107. <h2 id="source-code">Source Code</h2>
  108. <p>If you want to see the source code, you can check it out
  109. <a href="https://git.pleasantprogrammer.com/games/audventure">here</a>. The sound files
  110. aren&rsquo;t in the repo though, since I&rsquo;m not quite sure about the licensing. If you
  111. want to contribute music or sound effects, I&rsquo;d gladly appreciate it.</p>
  112. </div>
  113. <small class="dateline">Posted: <time class="published dt-published" itemprop="datePublished" datetime="2017-11-19">2017-11-19</time></small>
  114. | <small class="commentline"><a href="/posts/audventure.html#disqus_thread" data-disqus-identifier="cache/posts/audventure.html">Comments</a></small>
  115. </article>
  116. </article>
  117. <article class="h-entry post-text" itemscope itemtype="http://schema.org/Blog">
  118. <header>
  119. <h1 class="p-name entry-title" itemprop="headline">
  120. <a href="/posts/openpreppad.html" class="u-url">OpenPrepPad</a>
  121. </h1>
  122. </header>
  123. <div class="e-content entry-content">
  124. <p>Smart electronics and IoT (Internet of Things) are all the rage these days. You
  125. have a lot of companies sprout up trying to make the next big thing, which also
  126. leads to a lot of failures big and small. Pebble, the maker of my smartwatch,
  127. got bought out by Fitbit recently. This left watch owners without any official
  128. support, but thankfully, community members <a href="http://rebble.io/">stepped up</a> to continue
  129. maintaining it.</p>
  130. <p>Another casualty of the IoT boom was the <a href="http://theorangechef.com/">Orange Chef</a> <a href="https://www.amazon.com/Orange-Chef-Smart-Scale-Silver/dp/B00KFW8L90">Prep Pad</a>. It&rsquo;s a
  131. bluetooth connected weighing scale to make it easy to track your calories and
  132. carb/fat/protein intake. My dad bought it last year only to find out that the
  133. app was incredibly buggy. The search function doesn&rsquo;t work which makes the whole
  134. thing practically useless. I also found out later that you can&rsquo;t even download
  135. the app to use the scale anymore.</p>
  136. <p><strong>Note</strong> I just found out as I was writing this post that it <em>may</em> get supported
  137. by <a href="http://www.prnewswire.com/news-releases/perfect-company-acquires-orange-chefs-prep-pad-related-ip-continues-momentum-in-the-connected-kitchen-300383178.html">another company</a>.</p>
  138. <p>So the app is useless, but at least you can use it as a scale, right?</p>
  139. <p><img src="/galleries/openpreppad/preppad.jpg" alt="Prep Pad" /></p>
  140. <p>Nope. The device has no display whatsoever. The only controls on it are the
  141. on/off button and a green LED that isn&rsquo;t even that useful at telling you whether
  142. it&rsquo;s on or not. At this point, it&rsquo;s just a giant paperweight.</p>
  143. <h2 id="reverse-engineering">Reverse Engineering</h2>
  144. <p>Since I essentially had nothing to lose, I tried poking at the thing to figure
  145. out how it works. I didn&rsquo;t really have experience with bluetooth besides trying
  146. to get my bluetooth mouse connected on Linux. The main thing I used then was
  147. <code>bluetoothctl</code> which is essentially a CLI for managing bluetooth devices so I
  148. started there.</p>
  149. <p>I started up <code>bluetoothctl</code> and turned on the Prep Pad. And it showed up!</p>
  150. <div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>[bluetooth]# power on
  151. [CHG] Controller ... Class: 0x00010c
  152. Changing power on succeeded
  153. [CHG] Controller ... Powered: yes
  154. [bluetooth]# scan on
  155. Discovery started
  156. [CHG] Device 1C:BA:8C:21:7C:BB RSSI: -51
  157. [CHG] Device 1C:BA:8C:21:7C:BB Name: CHSLEEV_00
  158. [CHG] Device 1C:BA:8C:21:7C:BB Alias: CHSLEEV_00
  159. </pre></div>
  160. <p>I then connected to it, which was surprisingly easy.</p>
  161. <div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>[bluetooth]# connect 1C:BA:8C:21:7C:BB
  162. Attempting to connect to 1C:BA:8C:21:7C:BB
  163. [CHG] Device 1C:BA:8C:21:7C:BB Connected: yes
  164. [CHG] Device 1C:BA:8C:21:7C:BB Name: CH BTScale_00
  165. [CHG] Device 1C:BA:8C:21:7C:BB Alias: CH BTScale_00
  166. </pre></div>
  167. <p>Now normally, when you turn the device on, the green light flashes occasionally.
  168. Once I connected to it, the green light stayed on permanently. Clearly, I was
  169. making progress. A lot of services were also discovered but I had no idea what
  170. those things were at that point.</p>
  171. <p>After a lot of poking around, I could check the general device information. You
  172. could get the hardware, software and firmware version. There&rsquo;s also the device
  173. serial number which was nowhere on the actual physical device.</p>
  174. <div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>[CHSLEEV_00]# select-attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0010/char0017
  175. [CH BTScale_00:/service0010/char0017]# attribute-info
  176. Characteristic - Firmware Revision String
  177. UUID: 00002a26-0000-1000-8000-00805f9b34fb
  178. Service: /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0010
  179. Value: 0x31
  180. Value: 0x2e
  181. Value: 0x31
  182. Value: 0x33
  183. Value: 0x41
  184. Value: 0x00
  185. Flags: read
  186. [CH BTScale_00:/service0010/char0017]# read
  187. Attempting to read /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0010/char0017
  188. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0010/char0017 Value: 0x31
  189. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0010/char0017 Value: 0x2e
  190. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0010/char0017 Value: 0x31
  191. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0010/char0017 Value: 0x33
  192. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0010/char0017 Value: 0x41
  193. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0010/char0017 Value: 0x00
  194. 31 2e 31 33 41 00 1.13A.
  195. [CH BTScale_00:/service0010/char0017]#
  196. </pre></div>
  197. <p>There was also a service which contained Accel Enable, Accel Range, Accel
  198. X-Coordinate, Accel Y-Coordinate, and Accel Z-Coordinate. I guess it stands for
  199. accelerometer, which is probably what it uses to weigh things.</p>
  200. <div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>[CHSLEEV_00]# select-attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char0024/desc0026
  201. [CH BTScale_00:/service0023/char0024/desc0026]# read
  202. Attempting to read /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char0024/desc0026
  203. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char0024/desc0026 Value: 0x41
  204. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char0024/desc0026 Value: 0x63
  205. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char0024/desc0026 Value: 0x63
  206. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char0024/desc0026 Value: 0x65
  207. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char0024/desc0026 Value: 0x6c
  208. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char0024/desc0026 Value: 0x20
  209. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char0024/desc0026 Value: 0x45
  210. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char0024/desc0026 Value: 0x6e
  211. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char0024/desc0026 Value: 0x61
  212. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char0024/desc0026 Value: 0x62
  213. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char0024/desc0026 Value: 0x6c
  214. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char0024/desc0026 Value: 0x65
  215. 41 63 63 65 6c 20 45 6e 61 62 6c 65 Accel Enable
  216. </pre></div>
  217. <p>I couldn&rsquo;t read from any of the Accel Coordinates. It kept saying permission
  218. denied. I could however, notify on them. But that didn&rsquo;t yield anything as well.
  219. What I <em>could</em> read was Accel Enable, which was set to 00. I guess that means it
  220. was off. After writing 01 to Accel Enable, I found I could get values out of
  221. Accel X-Coordinate! Also, the green LED which was permanently on turned off.</p>
  222. <div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>[CHSLEEV_00]# select-attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char0024
  223. [CH BTScale_00:/service0023/char0024]# write 01
  224. Attempting to write /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char0024
  225. [CH BTScale_00:/service0023/char0024]# select-attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char002a
  226. [CH BTScale_00:/service0023/char002a]# notify on
  227. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char002a Notifying: yes
  228. Notify started
  229. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char002a Value: 0x5b
  230. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char002a Value: 0xa3
  231. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char002a Value: 0x02
  232. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char002a Value: 0x00
  233. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char002a Value: 0x55
  234. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char002a Value: 0xa3
  235. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char002a Value: 0x02
  236. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char002a Value: 0x00
  237. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char002a Value: 0x59
  238. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char002a Value: 0xa3
  239. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char002a Value: 0x02
  240. [CHG] Attribute /org/bluez/hci0/dev_1C_BA_8C_21_7C_BB/service0023/char002a Value: 0x00
  241. </pre></div>
  242. <p>I tried pressing the scale down a few times, and the values changed accordingly.
  243. Now, I just had to figure out how to convert the values into grams. It looked
  244. like the values were 32-bit integers sent as 4 bytes. In the above example it
  245. would be <code>0x0002a35b</code>, <code>0x0002a355</code>, <code>0x0002a359</code> or 172891, 172855, 172899. The
  246. values also decrease as you exert more effort on the scale. So assuming you take
  247. the initial value as <em>tare</em>, you simply subtract any succeeding value from that
  248. <em>tare</em> and you get the &ldquo;weight&rdquo;.</p>
  249. <p>The values I got didn&rsquo;t seem to be in grams though. After weighing some things
  250. on an actual scale and comparing the values I got, I found I can just divide the
  251. values by 14 and get something in grams. That 14 is entirely a magic number
  252. though and I have no idea whether other Prep Pad&rsquo;s would have the same constant.</p>
  253. <h2 id="openpreppad">OpenPrepPad</h2>
  254. <p>With all that figured out, I went ahead and made a <a href="https://github.com/thatsmydoing/openpreppad">simple CLI application</a>
  255. to interface with the Prep Pad. Ironically, node was the simplest thing I found
  256. that had <a href="https://github.com/sandeepmistry/noble">nice bluetooth library support</a> so that&rsquo;s what I wrote it in. I
  257. also added most of the technical details in the README for that as well.</p>
  258. <p>While this is all well and cool, I doubt the intersection of Linux users and
  259. people who <s>got ripped off</s> bought the Prep Pad is anyone besides me. In
  260. light of that, I&rsquo;m in the process of making a React Native version of the app,
  261. but that&rsquo;s still a work in progress. Who knows, if the new owners of Prep Pad
  262. are good, I might not even need to finish it.</p>
  263. </div>
  264. <small class="dateline">Posted: <time class="published dt-published" itemprop="datePublished" datetime="2017-01-15">2017-01-15</time></small>
  265. | <small class="commentline"><a href="/posts/openpreppad.html#disqus_thread" data-disqus-identifier="cache/posts/.html">Comments</a></small>
  266. </article>
  267. </article>
  268. <article class="h-entry post-text" itemscope itemtype="http://schema.org/Blog">
  269. <header>
  270. <h1 class="p-name entry-title" itemprop="headline">
  271. <a href="/posts/haproxy-charset.html" class="u-url">Haproxy Charset</a>
  272. </h1>
  273. </header>
  274. <div class="e-content entry-content">
  275. <p>A common problem we encounter is for things like <em>ñ</em> not showing up correctly. This actually caused <a href="http://www.rappler.com/nation/politics/elections/2016/132894-human-error-hash-election-results-code-mismatch">some issues</a> in the recent Philippine elections, but this isn&rsquo;t about hash codes or anything like that.</p>
  276. <p>By default, we use UTF-8 for text storage and rendering. A problem is that browsers don&rsquo;t assume UTF-8 as the default and you need to have either a <code>&lt;meta charset=&quot;utf-8&quot; /&gt;</code> in the HTML or <code>Content-Type: text/html; charset=utf-8</code> in the headers. A few of our services don&rsquo;t set the <code>Content-Type</code> with the <code>charset=utf-8</code> part so you&rsquo;d get piñata instead of piñata.</p>
  277. <p>Being lazy, we usually just correct this at the reverse proxy side. It&rsquo;s trivial to do in nginx. You just need to add <code>charset utf-8;</code> to your configuration and you&rsquo;re good. For haproxy though, I couldn&rsquo;t readily find a solution for it and had to go through the docs to see what I could do.</p>
  278. <p>After a bit of experimenting, I had success with this:</p>
  279. <div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span># set content-type to utf-8 if not already
  280. acl has_charset hdr_sub(content-type) -i charset=
  281. rspirep (Content-Type.*) \1;\ charset=utf-8 unless has_charset
  282. </pre></div>
  283. <p>This is probably not the best way to do it. Arguably, we should just fix our services to have the correct <code>Content-Type</code> in the first place, but I can do that some other time.</p>
  284. </div>
  285. <small class="dateline">Posted: <time class="published dt-published" itemprop="datePublished" datetime="2016-06-24">2016-06-24</time></small>
  286. | <small class="commentline"><a href="/posts/haproxy-charset.html#disqus_thread" data-disqus-identifier="cache/posts/haproxy-charset.html">Comments</a></small>
  287. </article>
  288. </article>
  289. <article class="h-entry post-text" itemscope itemtype="http://schema.org/Blog">
  290. <header>
  291. <h1 class="p-name entry-title" itemprop="headline">
  292. <a href="/posts/cloudflare-shenanigans.html" class="u-url">Cloudflare Shenanigans</a>
  293. </h1>
  294. </header>
  295. <div class="e-content entry-content">
  296. <p>An old client of ours managed to convince a telco to zero-rate the data for their app. In order to whitelist it though, we needed to use plain HTTP for domain whitelisting. For HTTPS, they can only whitelist by IP address. Like any good developer, we were using HTTPS. Also, like any good developer, we put our server behind Cloudflare.</p>
  297. <p>Now the problem is that Cloudflare can put you behind <a href="https://www.cloudflare.com/ips/">any IP they own</a>, which is a huge range. There&rsquo;s no guarantee that the IP we have now is going to be the same later on. So we did the reasonable thing and asked them to whitelist all of the Cloudflare IPs. And the telco agreed! We were in total disbelief when that happened. But hey, if life gives you free internet, you take it.</p>
  298. <p>We never actually empirically tested whether other sites hosted on Cloudflare were also actually zero-rated. But I like to think that we saved a lot of people on their data costs from browsing Reddit and 4chan. But alas, good things must come to an end.</p>
  299. <p>A few months after we started beta testing the app, Cloudflare added more IPs to their range. Unfortunately, our server got moved to those new IPs which were not whitelisted yet. Apparently, the telco whitelisting process was incredibly convoluted and time consuming. Our client didn&rsquo;t want to bother asking them to whitelist more IPs. We also tried asking Cloudflare to move us back to the original IP range, but they could only do that if we were in their enterprise tier. We couldn&rsquo;t really afford that, so we looked for other options.</p>
  300. <p>Since Cloudflare was essentially just a giant reverse proxy, theoretically there should be no distinction between one IP address from another. The specific IP we get is probably just for load balancing. So we tried accessing the IPs in the range directly and just setting the Host header and it worked! But we get SSL errors because the IP itself doesn&rsquo;t have its own certificate.</p>
  301. <p>After more testing, we figured out that you could actually use any Cloudflare backed domain so long as we properly set the Host header. We just needed to find one still in the old range. Coincidentally, 4chan.org was. Which led to this wonderful commit</p>
  302. <div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>commit 123456789abcdef
  303. Author: ~~~~~~
  304. Date: ~~~~~~
  305. 4chan hack
  306. <span style="color: #000080; font-weight: bold">diff --git a/src/com/client/common/Util.java b/src/com/client/common/Util.java</span>
  307. <span style="color: #A00000">--- a/src/com/client/common/Util.java</span>
  308. <span style="color: #00A000">+++ b/src/com/client/common/Util.java</span>
  309. <span style="color: #800080; font-weight: bold">@@ -210,7 +210,8 @@ public class Util {</span>
  310. }
  311. public static String getServerAddress(Context context) {
  312. <span style="color: #A00000">- String address = &quot;https://backend.client.com&quot;;</span>
  313. <span style="color: #00A000">+ // String address = &quot;https://backend.client.com&quot;;</span>
  314. <span style="color: #00A000">+ String address = &quot;https://4chan.org&quot;;</span>
  315. if(!isDebug(context)) return address;
  316. try {
  317. <span style="color: #000080; font-weight: bold">diff --git a/src/com/client/common/logging/APIClient.java b/src/com/client/common/logging/APIClient.java</span>
  318. <span style="color: #A00000">--- a/src/com/client/common/logging/APIClient.java</span>
  319. <span style="color: #00A000">+++ b/src/com/client/common/logging/APIClient.java</span>
  320. <span style="color: #800080; font-weight: bold">@@ -101,6 +101,7 @@ public class APIClient {</span>
  321. private HttpResponse postInternal(String url, List&lt;NameValuePair&gt; data, boolean forRegistration) throws ClientProtocolException, IOException {
  322. HttpPost request = new HttpPost(Util.getServerAddress(mContext)+&quot;/api/&quot;+url);
  323. request.setHeader(&quot;X-API-VERSION&quot;, apiVersion);
  324. <span style="color: #00A000">+ request.setHeader(&quot;Host&quot;, &quot;backend.client.com&quot;);</span>
  325. if(data == null) {
  326. data = new ArrayList&lt;NameValuePair&gt;();
  327. </pre></div>
  328. <p>Eventually, we did decide to just abandon Cloudflare for the server. We probably weren&rsquo;t going to be the target of a DDOS or anything. This also allowed us to do more secure things like pinning the server certificate in the application itself. Clearly, this is what we should have just done in the first place, but at the time we just wanted a stopgap solution.</p>
  329. <p>I just still find it funny we were making people&rsquo;s phones go to 4chan.org everyday for more than a year.</p>
  330. </div>
  331. <small class="dateline">Posted: <time class="published dt-published" itemprop="datePublished" datetime="2015-12-25">2015-12-25</time></small>
  332. | <small class="commentline"><a href="/posts/cloudflare-shenanigans.html#disqus_thread" data-disqus-identifier="cache/posts/cloudflare-shenanigans.html">Comments</a></small>
  333. </article>
  334. </article>
  335. <article class="h-entry post-text" itemscope itemtype="http://schema.org/Blog">
  336. <header>
  337. <h1 class="p-name entry-title" itemprop="headline">
  338. <a href="/posts/tiddlywiki-in-the-sky-or-tiddlyweb-for-tw5.html" class="u-url">TiddlyWiki in the Sky (or TiddlyWeb for TW5)</a>
  339. </h1>
  340. </header>
  341. <div class="e-content entry-content">
  342. <p>I&rsquo;ve always liked <a href="http://tiddlywiki.com">TiddlyWiki</a>. Back when it first came out, it was really amazing. A wiki all in one file, that worked in the browser. It didn&rsquo;t need a backend, it would just save itself as an all new HTML file with all your posts inside. I&rsquo;ve used it a lot over the years, as a personal wiki/journal and a class notebook. I even had a blog with it at one point using one of the server-side forks.</p>
  343. <p>Now, there&rsquo;s TiddlyWiki5 which is a rewrite of the original TiddlyWiki that looks a whole lot snazzier, and I assume has better architecture overall. It also has experimental support for all the server-side platforms (particularly TiddlyWeb) that have cropped up.</p>
  344. <p>If you&rsquo;re just looking for a simple server setup for TiddlyWiki5, it has native support for that on its own. There&rsquo;s plenty of documentation on the site. But if you&rsquo;re looking for more advanced features (like storing your posts in git or a database), then you&rsquo;ll need to use it with TiddlyWeb. The problem is that most of the documentation for TiddlyWeb still refers to the old TiddlyWiki.</p>
  345. <p>To support TiddlyWiki5, we&rsquo;ll need a version of the wiki which has the TiddlyWeb plugin already installed and configured. After that, some tweaking is necessary to get TiddlyWeb to provide what the wiki requires.</p>
  346. <h2 id="setting-up-tiddlywiki">Setting Up TiddlyWiki</h2>
  347. <p>TiddlyWiki5 provides a command line tool via <code>npm</code> that allows building custom versions of the wiki. In fact, it comes with templates, called &ldquo;editions&rdquo;, that we can use for our setup. Assuming you already have it installed, create the wiki using</p>
  348. <div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>tiddlywiki mywiki --init tw5tank <span style="color: #408080; font-style: italic"># create wiki from template</span>
  349. </pre></div>
  350. <p>This creates a wiki intended for use with <a href="https://tank.peermore.com/">Tank</a>, which is built on top of TiddlyWeb. From here, you should look in <code>mywiki/tiddlers/system</code> which contain the entries for <code>SiteTitle</code>, <code>SiteSubtitle</code>, <code>DefaultTiddlers</code>, and <code>tiddlyweb-host</code>. The first 3 should be configured however you want. These are necessary because they&rsquo;re needed before the wiki can load them from the server. <code>tiddlyweb-host</code> contains the location of the TiddlyWeb server, this should be <code>http://localhost:8080/</code> if you&rsquo;re just testing locally. With everything configured, you can build the new wiki by running</p>
  351. <div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>tiddlywiki mywiki --build
  352. </pre></div>
  353. <p>This will output the wiki to <code>mywiki/output/tw5tank.html</code>. You can now serve it using your favorite local webserver, like <code>python -m http.server</code>.</p>
  354. <h2 id="setting-up-tiddlyweb">Setting Up TiddlyWeb</h2>
  355. <p>The TiddlyWeb tutorial recommends using <code>tiddlywebwiki</code> which has all the plugins setup for a nice wiki instance for the old TiddlyWiki. It has a lot of features that aren&rsquo;t really needed, so we won&rsquo;t go with that. So first, we&rsquo;ll need to install TiddlyWeb and any plugins we might want to use.</p>
  356. <div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>pip install tiddlyweb tiddlywebplugins.status tiddlywebplugins.cherrypy tiddlywebplugins.cors
  357. </pre></div>
  358. <p>Next, we&rsquo;ll need the tiddlyweb configuration in <code>tiddlywebconfig.py</code></p>
  359. <div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #408080; font-style: italic"># A basic configuration.</span>
  360. <span style="color: #408080; font-style: italic"># `pydoc tiddlyweb.config` for details on configuration items.</span>
  361. <span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">tiddlywebplugins.status</span>
  362. config <span style="color: #666666">=</span> {
  363. <span style="color: #BA2121">&#39;system_plugins&#39;</span>: [<span style="color: #BA2121">&#39;tiddlywebplugins.status&#39;</span>, <span style="color: #BA2121">&#39;tiddlywebplugins.cors&#39;</span>],
  364. <span style="color: #BA2121">&#39;secret&#39;</span>: <span style="color: #BA2121">&#39;36c98d6d14618c79f0ed2d49cd1b9e272d8d4bd0&#39;</span>,
  365. <span style="color: #BA2121">&#39;wsgi_server&#39;</span>: <span style="color: #BA2121">&#39;tiddlywebplugins.cherrypy&#39;</span>,
  366. <span style="color: #BA2121">&#39;cors.enable_non_simple&#39;</span>: <span style="color: #008000">True</span>
  367. }
  368. original_gather_data <span style="color: #666666">=</span> tiddlywebplugins<span style="color: #666666">.</span>status<span style="color: #666666">.</span>_gather_data
  369. <span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">_status_gather_data</span>(environ):
  370. data <span style="color: #666666">=</span> original_gather_data(environ)
  371. data[<span style="color: #BA2121">&#39;space&#39;</span>] <span style="color: #666666">=</span> {<span style="color: #BA2121">&#39;recipe&#39;</span>: <span style="color: #BA2121">&#39;default&#39;</span>}
  372. <span style="color: #008000; font-weight: bold">return</span> data
  373. tiddlywebplugins<span style="color: #666666">.</span>status<span style="color: #666666">.</span>_gather_data <span style="color: #666666">=</span> _status_gather_data
  374. </pre></div>
  375. <p>The tweaks involved are:</p>
  376. <ul>
  377. <li>using the status plugin which the wiki requires</li>
  378. <li>monkeypatching the status plugin for the wiki to use the correct &ldquo;recipe&rdquo;</li>
  379. <li>using cherrypy server instead of the buggy default one</li>
  380. <li>using cors since we&rsquo;re not hosting the wiki itself on the same server</li>
  381. </ul>
  382. <p>With that, we just need to create the store that will hold our data</p>
  383. <div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>twanager recipe default <span style="color: #BA2121">&lt;&lt;EOF</span>
  384. <span style="color: #BA2121">desc: standard TiddlyWebWiki environment</span>
  385. <span style="color: #BA2121">policy: {&quot;read&quot;: [], &quot;create&quot;: [], &quot;manage&quot;: [&quot;R:ADMIN&quot;], &quot;accept&quot;: [], &quot;write&quot;: [&quot;R:ADMIN&quot;], &quot;owner&quot;: &quot;administrator&quot;, &quot;delete&quot;: [&quot;R:ADMIN&quot;]}</span>
  386. <span style="color: #BA2121">/bags/default/tiddlers</span>
  387. <span style="color: #BA2121">EOF</span>
  388. twanager bag default <span style="color: #BA2121">&lt;&lt;EOF</span>
  389. <span style="color: #BA2121">{&quot;policy&quot;: {&quot;read&quot;: [], &quot;create&quot;: [], &quot;manage&quot;: [&quot;R:ADMIN&quot;], &quot;accept&quot;: [], &quot;write&quot;: [], &quot;owner&quot;: &quot;administrator&quot;, &quot;delete&quot;: []}}</span>
  390. <span style="color: #BA2121">EOF</span>
  391. </pre></div>
  392. <p>Finally, we can start the TiddlyWeb server</p>
  393. <div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>twanager server
  394. </pre></div>
  395. <h2 id="putting-it-all-together">Putting it all together</h2>
  396. <p>Once you have the TiddlyWeb server running, you can just go to wherever you&rsquo;re hosting the wiki html and it should work. You can try creating some posts, and the check mark on the sidebar should be red for a while and then turn black. Once that&rsquo;s done it&rsquo;s saved. You can refresh your browser and your posts should still be there.</p>
  397. <p>At this point, you can start customizing your TiddlyWeb instance, by changing your store to something like a database, or adding authorization. You can also tweak the server setup so you won&rsquo;t need CORS anymore.</p>
  398. <p>TiddlyWiki5 is still relatively new. I hope that eventually, support for server-side and the plugin ecosystem grows to be as great as the old TiddlyWiki.</p>
  399. </div>
  400. <small class="dateline">Posted: <time class="published dt-published" itemprop="datePublished" datetime="2015-12-24">2015-12-24</time></small>
  401. | <small class="commentline"><a href="/posts/tiddlywiki-in-the-sky-or-tiddlyweb-for-tw5.html#disqus_thread" data-disqus-identifier="cache/posts/tiddlywiki-in-the-sky-or-tiddlyweb-for-tw5.html">Comments</a></small>
  402. </article>
  403. </article>
  404. </div>
  405. <nav class="postindexpager">
  406. <ul class="pager clearfix">
  407. <li class="next">
  408. <a href="/page/2.html">Older posts &rarr;</a>
  409. </li>
  410. </ul>
  411. </nav>
  412. </main>
  413. <footer id="footer" role="contentinfo">
  414. <p>
  415. <a rel="license" href="http://creativecommons.org/licenses/by-sa/3.0/deed.en_US">
  416. <img alt="CC-BY-SA" style="border-width:0" src="https://licensebuttons.net/l/by-sa/3.0/80x15.png">
  417. </a> &copy; 2017 Thomas Dy - Powered by <a href="http://gohugo.io">Hugo</a></p>
  418. </footer>
  419. </div>
  420. <script src="/assets/js/konami.js"></script>
  421. <script>
  422. var easter_egg = new Konami();
  423. easter_egg.code = function() {
  424. var el = document.getElementById('thomas');
  425. if(el.className == "whoa") {
  426. el.className = "";
  427. }
  428. else {
  429. el.className = "whoa";
  430. }
  431. document.body.scrollTop = document.documentElement.scrollTop = 0;
  432. }
  433. easter_egg.load();
  434. </script>
  435. <script id="dsq-count-scr" type="text/javascript" src="//pleasantprog.disqus.com/count.js" async></script>
  436. </body>
  437. </html>