Help
RSS
API
Feed
Maltego
Contact
Domain > doublemap.github.io
×
Welcome!
Right click nodes and scroll the mouse to navigate the graph.
×
More information on this domain is in
AlienVault OTX
Is this malicious?
Yes
No
DNS Resolutions
Date
IP Address
2014-05-31
199.27.73.133
(
ClassC
)
2014-06-03
199.27.74.133
(
ClassC
)
2024-10-04
185.199.111.153
(
ClassC
)
Port 80
HTTP/1.1 200 OKConnection: keep-aliveContent-Length: 38810Server: GitHub.comContent-Type: text/html; charsetutf-8permissions-policy: interest-cohort()Last-Modified: Fri, 29 May 2015 16:58:32 GMTAccess-Control-Allow-Origin: *ETag: 55689ab8-979aexpires: Fri, 04 Oct 2024 05:50:00 GMTCache-Control: max-age600x-proxy-cache: MISSX-GitHub-Request-Id: 2CDA:356DC8:1B539C2:1C145E5:66FF7FB0Accept-Ranges: bytesAge: 0Date: Fri, 04 Oct 2024 05:40:00 GMTVia: 1.1 varnishX-Served-By: cache-bfi-krnt7300063-BFIX-Cache: MISSX-Cache-Hits: 0X-Timer: S1728020401.745729,VS0,VE95Vary: Accept-EncodingX-Fastly-Request-ID: 42db154e53488da48c52e35296d48ce7e860824f !DOCTYPE html>!--if IEMobile 7 >html classno-js iem7>!endif-->!--if lt IE 9>html classno-js lte-ie8>!endif-->!--if (gt IE 8)|(gt IEMobile 7)|!(IEMobile)|!(IE)>!-->html classno-js langen>!--!endif-->head> meta charsetutf-8> title>DoubleMap Engineering/title> meta nameauthor contentThe DoubleMap team> meta namedescription contentPython is slow. It’s still faster than a lot of other languages, butdescribing Python code as “fast” will probably get some weird …> !-- http://t.co/dKP3o1e --> meta nameHandheldFriendly contentTrue> meta nameMobileOptimized content320> meta nameviewport contentwidthdevice-width, initial-scale1> link relcanonical hrefhttp://doublemap.github.io> link href/favicon.png relicon> link href/stylesheets/screen.css mediascreen, projection relstylesheet typetext/css> link href/atom.xml relalternate titleDoubleMap Engineering typeapplication/atom+xml> script src/javascripts/modernizr-2.0.js>/script> script src//ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js>/script> script>!window.jQuery && document.write(unescape(%3Cscript src./javascripts/libs/jquery.min.js%3E%3C/script%3E))/script> script src/javascripts/octopress.js typetext/javascript>/script> !--Fonts from Googles Web font directory at http://google.com/webfonts -->link href//fonts.googleapis.com/css?familyPT+Serif:regular,italic,bold,bolditalic relstylesheet typetext/css>link href//fonts.googleapis.com/css?familyPT+Sans:regular,italic,bold,bolditalic relstylesheet typetext/css> script typetext/javascript> var _gaq _gaq || ; _gaq.push(_setAccount, UA-50542806-1); _gaq.push(_trackPageview); (function() { var ga document.createElement(script); ga.type text/javascript; ga.async true; ga.src (https: document.location.protocol ? https://ssl : http://www) + .google-analytics.com/ga.js; var s document.getElementsByTagName(script)0; s.parentNode.insertBefore(ga, s); })(); /script>/head>body > header rolebanner>hgroup> h1>a href/>DoubleMap Engineering/a>/h1> h2>The official tech blog of DoubleMap/h2> /hgroup>/header> nav rolenavigation>ul classsubscription data-subscriptionrss> li>a href/atom.xml relsubscribe-rss titlesubscribe via RSS>RSS/a>/li> /ul> form actionhttps://www.google.com/search methodget> fieldset rolesearch> input typehidden nameq valuesite:doublemap.github.io /> input classsearch typetext nameq results0 placeholderSearch/> /fieldset>/form> ul classmain-navigation> li>a href/>Blog/a>/li> li>a href/blog/archives>Archives/a>/li>/ul>/nav> div idmain> div idcontent> div classblog-index> article> header> h1 classentry-title>a href/blog/2015/05/29/optimizing-python/>Optimizing Python With Cython/a>/h1> p classmeta> time datetime2015-05-29T11:09:00-04:00 pubdate data-updatedtrue>May 29span>th/span>, 2015/time> /p> /header> div classentry-content>p>Python is slow. It’s still faster than a lot of other languages, butdescribing Python code as “fast” will probably get some weird looks frompeople who use compiled or JITted languages./p>p>Still, I love Python for its design and speed at which I can write good code.Most of the time, when writing Python for distributed or web applications, itdoesn’t matter. Your runtime is usually dominated by network traffic or dataaccesses than it is by raw computational speed. In those cases, the correctoptimization is often reducing network requests or indexing your data./p>p>Sometimes—just occasionally—we actually do need fast computation.Traditional Python wisdom has been to profile your code (perhaps usingcProfile), identify the hot spot, and rewrite it in C as a Python extension.I’ve had to do that before for a handwriting recognition algorithm with greatsuccess (it turned a 2–3 second computation into a practicallyreal-time one), but writing C code and then figuring out the FFI was a pain.I had to rewrite the algorithm in C, verify that my C version was correct anddidn’t have memory errors, and then figure out the right incantations to getit to cooperate with Python./p>p>Let’s try something different this time./p>h2>Problem/h2>p>At DoubleMap, geographic distance calculation is something that gets used alot, and we use the haversine function to calculate distance between twopoints on Earth. In some of our data analysis, we might run the haversinefunction millions of times in a tight loop, and it was causing some reports totake way too long./p>p>Profiling the code showed that the two heaviest operations were fetching datafrom the data store and the haversine function./p>p>strong>Play along at home:/strong> Clone the repo ata hrefhttps://github.com/doublemap/haversine-optimization>https://github.com/doublemap/haversine-optimization/a> and start from the firstcommit to see how the code changes step by step./p>p>The commands you should know are code>pip install cython/code> and code>python setup.pybuild_ext --inplace/code>./p>h2>Original code/h2>figure classcode>div classhighlight>table>tr>td classgutter>pre classline-numbers>span classline-number>1/span>span classline-number>2/span>span classline-number>3/span>span classline-number>4/span>span classline-number>5/span>span classline-number>6/span>span classline-number>7/span>span classline-number>8/span>span classline-number>9/span>span classline-number>10/span>span classline-number>11/span>span classline-number>12/span>span classline-number>13/span>span classline-number>14/span>span classline-number>15/span>span classline-number>16/span>span classline-number>17/span>span classline-number>18/span>span classline-number>19/span>span classline-number>20/span>span classline-number>21/span>span classline-number>22/span>/pre>/td>td classcode>pre>code classpython>span classline>span classkn>from/span> span classnn>math/span> span classkn>import/span> span classn>sin/span>span classp>,/span> span classn>cos/span>span classp>,/span> span classn>acos/span>/span>span classline>/span>span classline>/span>span classline>span classk>def/span> span classnf>haversine/span>span classp>(/span>span classn>coord1/span>span classp>,/span> span classn>coord2/span>span classp>):/span>/span>span classline> span classsd>"""Given two (lat, lng) tuples, returns the distance between them in/span>/span>span classline>span classsd> meters."""/span>/span>span classline> span classn>lat1/span>span classp>,/span> span classn>lng1/span> span classo>/span> span classn>coord1/span>/span>span classline> span classn>lat2/span>span classp>,/span> span classn>lng2/span> span classo>/span> span classn>coord2/span>/span>span classline>/span>span classline> span classk>if/span> span classn>lat1/span> span classo>>/span> span classmi>90/span> span classow>or/span> span classn>lat1/span> span classo></span> span classo>-/span>span classmi>90/span> span classow>or/span> span classn>lat2/span> span classo>>/span> span classmi>90/span> span classow>or/span> span classn>lat2/span> span classo></span> span classo>-/span>span classmi>90/span>span classp>:/span>/span>span classline> span classk>raise/span> span classne>ValueError/span>span classp>(/span>span classs>"Invalid latitude (should be between +/- 90)"/span>span classp>)/span>/span>span classline> span classk>if/span> span classn>lng1/span> span classo>>/span> span classmi>180/span> span classow>or/span> span classn>lng1/span> span classo></span> span classo>-/span>span classmi>180/span> span classow>or/span> span classn>lng2/span> span classo>>/span> span classmi>180/span> span classow>or/span> span classn>lng2/span> span classo></span> span classo>-/span>span classmi>180/span>span classp>:/span>/span>span classline> span classk>raise/span> span classne>ValueError/span>span classp>(/span>span classs>"Invalid longitude (should be between +/- 180)"/span>span classp>)/span>/span>span classline>/span>span classline> span classn>phi1/span> span classo>/span> span classp>(/span>span classmf>90.0/span> span classo>-/span> span classn>lat1/span>span classp>)/span> span classo>*/span> span classmf>0.0174532925/span>/span>span classline> span classn>phi2/span> span classo>/span> span classp>(/span>span classmf>90.0/span> span classo>-/span> span classn>lat2/span>span classp>)/span> span classo>*/span> span classmf>0.0174532925/span>/span>span classline> span classn>theta1/span> span classo>/span> span classn>lng1/span> span classo>*/span> span classmf>0.0174532925/span>/span>span classline> span classn>theta2/span> span classo>/span> span classn>lng2/span> span classo>*/span> span classmf>0.0174532925/span>/span>span classline>/span>span classline> span classn>c/span> span classo>/span> span classp>(/span>span classn>sin/span>span classp>(/span>span classn>phi1/span>span classp>)/span> span classo>*/span> span classn>sin/span>span classp>(/span>span classn>phi2/span>span classp>)/span> span classo>*/span> span classn>cos/span>span classp>(/span>span classn>theta1/span> span classo>-/span> span classn>theta2/span>span classp>)/span> span classo>+/span> span classn>cos/span>span classp>(/span>span classn>phi1/span>span classp>)/span> span classo>*/span> span classn>cos/span>span classp>(/span>span classn>phi2/span>span classp>))/span>/span>span classline> span classn>arc/span> span classo>/span> span classn>acos/span>span classp>(/span>span classn>c/span>span classp>)/span>/span>span classline> span classk>return/span> span classn>arc/span> span classo>*/span> span classmf>6367444.7/span>/span>/code>/pre>/td>/tr>/table>/div>/figure>p>Also, here’s the benchmark code that is used throughout:/p>figure classcode>div classhighlight>table>tr>td classgutter>pre classline-numbers>span classline-number>1/span>span classline-number>2/span>span classline-number>3/span>span classline-number>4/span>span classline-number>5/span>span classline-number>6/span>/pre>/td>td classcode>pre>code classpython>span classline>span classkn>from/span> span classnn>__future__/span> span classkn>import/span> span classn>print_function/span>/span>span classline>span classkn>from/span> span classnn>timeit/span> span classkn>import/span> span classn>timeit/span>/span>span classline>/span>span classline>span classk>print/span>span classp>(/span>span classn>timeit/span>span classp>(/span>span classs>"haversine((39.132213, -86.12439), (38.55213, -86.94910))"/span>span classp>,/span>/span>span classline> span classn>setup/span>span classo>/span>span classs>"from haversine import haversine"/span>span classp>,/span>/span>span classline> span classn>number/span>span classo>/span>span classmi>300000/span>span classp>))/span>/span>/code>/pre>/td>/tr>/table>/div>/figure>p>Pretty straightforward. Let’s try pre-compiling this code into a C extensionto see if that helps./p>p>Following thea hrefhttp://docs.cython.org/src/tutorial/cython_tutorial.html>Cython Basic Tutorial/a>,I renamed code>haversine.py/code> to code>haversine.pyx/code> and created a code>setup.py/code> file:/p>figure classcode>div classhighlight>table>tr>td classgutter>pre classline-numbers>span classline-number>1/span>span classline-number>2/span>span classline-number>3/span>span classline-number>4/span>span classline-number>5/span>span classline-number>6/span>/pre>/td>td classcode>pre>code classpython>span classline>span classkn>from/span> span classnn>distutils.core/span> span classkn>import/span> span classn>setup/span>/span>span classline>span classkn>from/span> span classnn>Cython.Build/span> span classkn>import/span> span classn>cythonize/span>/span>span classline>/span>span classline>span classn>setup/span>span classp>(/span>/span>span classline> span classn>ext_modules/span> span classo>/span> span classn>cythonize/span>span classp>(/span>span classs>"haversine.pyx"/span>span classp>)/span>/span>span classline>span classp>)/span>/span>/code>/pre>/td>/tr>/table>/div>/figure>p>Running code>python setup.py build_ext --inplace/code> built code>haversine.so/code> out of myunmodified Python code. Easy-peasy./p>p>Benchmark results (300,000 iterations):/p>p>strong>Original code:/strong> 2.85 secondsbr/>strong>Compiled:/strong> 2.01 seconds/p>p>So, 29% time savings without any modification to the original code./p>h2>C math functions/h2>p>Currently, we’re still using Python’s math functions. Let’s see if we can cutout some overhead by importing code>math.h/code> instead./p>p>Except, since this is Cython, all we need to do is:/p>figure classcode>div classhighlight>table>tr>td classgutter>pre classline-numbers>span classline-number>1/span>span classline-number>2/span>span classline-number>3/span>span classline-number>4/span>span classline-number>5/span>/pre>/td>td classcode>pre>code classdiff>span classline>span classgd>--- a/haversine.pyx/span>/span>span classline>span classgi>+++ b/haversine.pyx/span>/span>span classline>span classgu>@@ -1,4 +1,4 @@/span>/span>span classline>span classgd>-from math import sin, cos, acos/span>/span>span classline>span classgi>+from libc.math cimport sin, cos, acos/span>/span>/code>/pre>/td>/tr>/table>/div>/figure>p>Benchmark results (300,000 iterations):/p>p>strong>Original code:/strong> 2.85 secondsbr/>strong>Compiled:/strong> 2.01 secondsbr/>strong>libc.math:/strong> 1.33 seconds/p>p>So far we’ve saved 53% of the time from the original code./p>h2>C types/h2>p>Cython’s biggest extension of the Python language is its type annotations.Speed ups can be had by telling Cython the types of each variable in advance.We are dealing with geographic coordinates, so we’ll be using doubles forpretty much everything:/p>figure classcode>div classhighlight>table>tr>td classgutter>pre classline-numbers>span classline-number>1/span>span classline-number>2/span>span classline-number>3/span>span classline-number>4/span>span classline-number>5/span>span classline-number>6/span>span classline-number>7/span>span classline-number>8/span>span classline-number>9/span>span classline-number>10/span>span classline-number>11/span>span classline-number>12/span>span classline-number>13/span>span classline-number>14/span>span classline-number>15/span>span classline-number>16/span>span classline-number>17/span>span classline-number>18/span>span classline-number>19/span>span classline-number>20/span>span classline-number>21/span>span classline-number>22/span>span classline-number>23/span>span classline-number>24/span>span classline-number>25/span>span classline-number>26/span>span classline-number>27/span>span classline-number>28/span>span classline-number>29/span>span classline-number>30/span>/pre>/td>td classcode>pre>code classpython>span classline>span classk>def/span> span classnf>haversine/span>span classp>(/span>span classnb>tuple/span> span classn>coord1/span>span classp>,/span> span classnb>tuple/span> span classn>coord2/span>span classp>):/span>/span>span classline> span classsd>"""Given two (lat, lng) tuples, returns the distance between them in/span>/span>span classline>span classsd> meters."""/span>/span>span classline> span classn>cdef/span> span classn>double/span> span classn>lat1/span>/span>span classline> span classn>cdef/span> span classn>double/span> span classn>lng1/span>/span>span classline> span classn>cdef/span> span classn>double/span> span classn>lat2/span>/span>span classline> span classn>cdef/span> span classn>double/span> span classn>lng2/span>/span>span classline> span classn>lat1/span>span classp>,/span> span classn>lng1/span> span classo>/span> span classn>coord1/span>/span>span classline> span classn>lat2/span>span classp>,/span> span classn>lng2/span> span classo>/span> span classn>coord2/span>/span>span classline>/span>span classline> span classk>if/span> span classn>lat1/span> span classo>>/span> span classmi>90/span> span classow>or/span> span classn>lat1/span> span classo></span> span classo>-/span>span classmi>90/span> span classow>or/span> span classn>lat2/span> span classo>>/span> span classmi>90/span> span classow>or/span> span classn>lat2/span> span classo></span> span classo>-/span>span classmi>90/span>span classp>:/span>/span>span classline> span classk>raise/span> span classne>ValueError/span>span classp>(/span>span classs>"Invalid latitude (should be between +/- 90)"/span>span classp>)/span>/span>span classline> span classk>if/span> span classn>lng1/span> span classo>>/span> span classmi>180/span> span classow>or/span> span classn>lng1/span> span classo></span> span classo>-/span>span classmi>180/span> span classow>or/span> span classn>lng2/span> span classo>>/span> span classmi>180/span> span classow>or/span> span classn>lng2/span> span classo></span> span classo>-/span>span classmi>180/span>span classp>:/span>/span>span classline> span classk>raise/span> span classne>ValueError/span>span classp>(/span>span classs>"Invalid longitude (should be between +/- 180)"/span>span classp>)/span>/span>span classline>/span>span classline> span classn>cdef/span> span classn>double/span> span classn>ph1/span>/span>span classline> span classn>cdef/span> span classn>double/span> span classn>ph2/span>/span>span classline> span classn>cdef/span> span classn>double/span> span classn>theta1/span>/span>span classline> span classn>cdef/span> span classn>double/span> span classn>theta2/span>/span>span classline> span classn>cdef/span> span classn>double/span> span classn>c/span>/span>span classline> span classn>cdef/span> span classn>double/span> span classn>arc/span>/span>span classline>/span>span classline> span classn>phi1/span> span classo>/span> span classp>(/span>span classmf>90.0/span> span classo>-/span> span classn>lat1/span>span classp>)/span> span classo>*/span> span classmf>0.0174532925/span>/span>span classline> span classn>phi2/span> span classo>/span> span classp>(/span>span classmf>90.0/span> span classo>-/span> span classn>lat2/span>span classp>)/span> span classo>*/span> span classmf>0.0174532925/span>/span>span classline> span classn>theta1/span> span classo>/span> span classn>lng1/span> span classo>*/span> span classmf>0.0174532925/span>/span>span classline> span classn>theta2/span> span classo>/span> span classn>lng2/span> span classo>*/span> span classmf>0.0174532925/span>/span>span classline>/span>span classline> span classn>c/span> span classo>/span> span classp>(/span>span classn>sin/span>span classp>(/span>span classn>phi1/span>span classp>)/span> span classo>*/span> span classn>sin/span>span classp>(/span>span classn>phi2/span>span classp>)/span> span classo>*/span> span classn>cos/span>span classp>(/span>span classn>theta1/span> span classo>-/span> span classn>theta2/span>span classp>)/span> span classo>+/span> span classn>cos/span>span classp>(/span>span classn>phi1/span>span classp>)/span> span classo>*/span> span classn>cos/span>span classp>(/span>span classn>phi2/span>span classp>))/span>/span>span classline> span classn>arc/span> span classo>/span> span classn>acos/span>span classp>(/span>span classn>c/span>span classp>)/span>/span>span classline> span classk>return/span> span classn>arc/span> span classo>*/span> span classmf>6367444.7/span>/span>/code>/pre>/td>/tr>/table>/div>/figure>p>strong>Original code:/strong> 2.85 secondsbr/>strong>Compiled:/strong> 2.01 secondsbr/>strong>libc.math:/strong> 1.33 secondsbr/>strong>C types:/strong> 0.466 seconds/p>p>In three easy steps, we’ve reduced the run time of this function by 84%. Moreimportantly, we’re still writing what’s basically Python. We don’t have toremember to free our variables or check our array bounds. That additionalsafety is not free (and there are various flags to disable Cython safetychecks) but with Cython, optimizing a Python function doesn’t have to be arewrite-everything task./p>p>We probably could have gone further in optimizing this, but these speed-upsgot us into “good enough” territory, and that’s good enough./p>h2>Distribution/h2>p>Our optimized haversine function is available on PyPI under the name“cHaversine”. One key difference is that the distributed tarball includes theC code generated by Cython so that you, as the package user, don’t need Cythonto download and build the C extension./p>p>a hrefhttps://news.ycombinator.com/item?id9625737>Comments on Hacker News/a>/p>/div> /article> article> header> h1 classentry-title>a href/blog/2014/05/29/everyone-panic/>Everyone Panic! Almost Free Downtime Phone Alerts/a>/h1> p classmeta> time datetime2014-05-29T11:24:46-04:00 pubdate data-updatedtrue>May 29span>th/span>, 2014/time> /p> /header> div classentry-content>p>a hrefhttps://github.com/doublemap/everyonepanic>Uptime Robot (free) + App Engine (free) + Twilio (cheap) almost freedowntime alerts!/a>/p>ol>li>Uptime Robot notices that your website is not responding./li>li>Everyone Panic! checks Uptime Robot and sees that something is down./li>li>You get a call through Twilio telling you to panic./li>li>You get another call 15 minutes later because you still haven’t fixed theproblem./li>/ol>p>Sure, it’s easy to get emailed when your website goes down, but realhonest-to-goodness phone calls have an immediacy that’s hard to beat./p>p>Since our GPS bus-tracking system is used at all hours of the day, early on weintroduced automated monitoring. But when everyone is asleep, emails and textmessages go unnoticed until the morning and it sucks to not know where yourbus is at 1am./p>p>We quickly whipped an app together using Python to tieUptime Robot and Twilio together. If one of the items in Uptime Robot goesdown then the Python app will ask Twilio to call us. We made it, put it on AppEngine, and haven’t touched it since. Even though our automated monitoring hasgrown since then, this little app, with zero maintenance, still dutifullywatches Uptime Robot for any websites that are down./p>p>Literally, there have been zero commits to the original repo since the firstday. Now the code’s been cleaned up anda hrefhttps://github.com/doublemap/everyonepanic>put on GitHub/a>. You can downloadit, configure it, and throw it onto App Engine or Heroku in a few minutes./p>p>It’s not entirely free – you’ll need to pay Twilio for voice calls, but theprice is $0.02 per call and it’s hard to think of a situation where knowingyour website is down isn’t worth two cents. And, of course, you should keepyour Twilio account balance in the black./p>p>But for the hobbyist who doesn’t mind trying out a new app, this is a reallysimple way to have your phone ring when Hacker News tramples your latest petproject./p>p>a hrefhttps://github.com/doublemap/everyonepanic>Everyone Panic! on GitHub/a>/p>p>a hrefhttps://news.ycombinator.com/item?id7816241>Comments on Hacker News/a>/p>/div> /article> article> header> h1 classentry-title>a href/blog/2014/04/29/porting-600k-map-views-to-openstreetmap/>Porting 600k Map Views to OpenStreetMap/MapBox/a>/h1> p classmeta> time datetime2014-04-29T10:25:45-04:00 pubdate data-updatedtrue>Apr 29span>th/span>, 2014/time> /p> /header> div classentry-content>p>img classright src/images/doublemap-plus-mapbox.png>This past week, we here at DoubleMap made a big change that users probablyem>won’t/em> notice. Instead of Google Maps, all of our public maps are now poweredby MapBox with OpenStreetMap data./p>p>We have hundreds of thousands of hits per month to our online maps from peopleusing our real-time bus tracking to find out where their bus is. This numberdoesn’t even include transit riders using their agency’s custom app or publicTV displays./p>p>For a company whose main feature is showing buses move on a map, it was a bigdeal for everyone – we had to make sure that data at each of our clienttransit agencies was good, update some of our sales material that specificallymentioned Google Maps, decide on new providers, and figure out the overallcosts of switching and notswitching. We didn’t want to disrupt our transit agencies and the riders thatuse our service to catch the bus every day. The actual coding changes wererelatively straightforward./p>p>Thanks to MapBox, were able to switch everything over almost seamlesslywithout interrupting our users or creating a jarring change./p>h2>Evaluation/h2>p>It shouldn’t be a surprise that the main motivation was simply cost. Googledoesn’t publish its enterprise maps pricing, but it’s orders of magnitude moreexpensive than MapBox./p>p>We took a careful look at what we would have lost by switching away fromGoogle. The main feature that we can’t get anywhere else is Street View,something that nobody has come close to replicating. We also lost out onangled aerial imagery. Both of these are cool, but ultimately not worth theprice difference./p>p>video width592 stylewidth:592px loop autoplay> source src/images/iubdoublemap.mp4 typevideo/mp4> source src/images/iubdoublemap.webm typevideo/webm>/video>/p>h2>What’s in a map?/h2>p>Part of the thinking change that most people have to go through when switchingis that maps don’t always come in one giant bundle.“Google Maps” can refer to everything from the JavaScript library tothe pop-ups with user reviews to the navigation to Street View. WithOpenStreetMap, you get your choice of JavaScript mapping engine and differenttile sets./p>p>For example, you might choose to use a hrefhttp://leafletjs.com/>Leaflet/a> witha hrefhttp://www.opencyclemap.org/>OpenCycleMap/a> and a hrefhttps://www.mapbox.com/blog/mapbox-satellite/>MapBoxSatellite/a> as your layers, oryou might use a hrefhttp://openlayers.org/>OpenLayers/a> with your owna hrefhttp://tilestache.org/>tileserver/a>plus the a hrefhttp://www.thunderforest.com/landscape/>Thunderforest Landscape/a>map./p>p>em>Everything is swappable./em> When we were testing, it was a one-line change toswitch between MapQuest Open, CloudMade, and MapBox tiles./p>p>We ended up using Leaflet with a custom MapBox style plus MapBox Satellite,and it looks pretty awesome./p>h2>Map styling/h2>p>One of the complaints levied against OpenStreetMap is that their map stylelooks ugly. That’s not really fair, because OSM is there to provide the mapdata, and most people who see OSM data see it through a custom map style, likeon FourSquare or Pinterest (or DoubleMap)./p>p>The power and curse of OSM is that you have total control over how the maplooks. The power is that if you want a bubble-gum-pink map with 36pt Comic Sanslabels and no roads, that’s easily doable.The curse is that it can get complicated, such as when you decide thatyou need to label each US highway using its state-specific shield image.As TileMill matures, there will hopefully be more and more styles to build onrather than starting from scratch./p>p>One of the reasons why we initially went with CloudMade was because they hadan easy online map styler that let you point and click a feature, and thenchange that feature class’s colors. This let us mostly duplicate our customstyle we were using with Google Maps. CloudMade, however, decided toa hrefhttp://notes.ericjiang.com/posts/741>get out of non-enterprise services/a>not too long after I had gotten the map to look the way I wanted it to./p>p>em>Enter MapBox./em> MapBox has a really slick desktop program, TileMill, that letsyou generate map tiles using a CSS-ish language called CartoCSS. The downsideof TileMill is that you have to download raw map data, render the tiles youneed (which easily gets into the gigabytes), and then upload them somewhere.We have clients internationally and we would have had to download eachregion’s OSM data and render new tiles every time we got another customer./p>p>This is all fixed in a hrefhttps://github.com/mapbox/tm2>TM2/a>, which downloadsvector map tiles from MapBoxon the fly so you can upload just the compiled map style to MapBox andthey’ll handle all the data and rendering as needed./p>p>Here’s our result:/p>p>img src/images/osm-google-comparison.png>/p>p>Our custom style is designed to be muted with a limited color palette to letour agencies’ custom route colors pop when overlaid. I hope to write aboutTileMill 2 in the future, but right now it’s in “experimental” status./p>h2>Porting the source code/h2>p>Our original front-end code was written to use v2 of the Google Maps JS API.Once that was deprecated, we eventually rewrote it to use v3 of the GoogleMaps API. When we started investigating OSM as a replacement, we noticed howpolished the Leaflet library was. The Leaflet API is based on v2 of theGoogle Maps API so we were essentially doing the reverse of what we haddone in the past. All of the concepts mapped almost 1-to-1, and there wasnothing that Leaflet lacked in terms of functionality./p>figure classcode>div classhighlight>table>tr>td classgutter>pre classline-numbers>span classline-number>1/span>span classline-number>2/span>span classline-number>3/span>span classline-number>4/span>span classline-number>5/span>span classline-number>6/span>span classline-number>7/span>span classline-number>8/span>span classline-number>9/span>span classline-number>10/span>span classline-number>11/span>span classline-number>12/span>span classline-number>13/span>span classline-number>14/span>span classline-number>15/span>span classline-number>16/span>span classline-number>17/span>span classline-number>18/span>span classline-number>19/span>span classline-number>20/span>span classline-number>21/span>/pre>/td>td classcode>pre>code classdiff>span classline>span classgd>- r.polyline new google.maps.Polyline({/span>/span>span classline>span classgd>- path: points,/span>/span>span classline>span classgd>- map: r.visible ? map : null,/span>/span>span classline>span classgd>- strokeColor: "#"+r.color,/span>/span>span classline>span classgd>- strokeOpacity: 0.6,/span>/span>span classline>span classgd>- strokeWeight: 4,/span>/span>span classline>span classgd>- zIndex: 1/span>/span>span classline>span classgd>- })/span>/span>span classline>span classgd>- google.maps.event.addListener(r.polyline, 'mouseover',/span>/span>span classline>span classgd>- function(routeId) {/span>/span>span classline>span classgi>+ r.polyline new L.Polyline(points, {/span>/span>span classline>span classgi>+ color: "#"+r.color,/span>/span>span classline>span classgi>+ opacity: 0.6,/span>/span>span classline>span classgi>+ weight: 4/span>/span>span classline>span classgi>+ });/span>/span>span classline>span classgi>+ if(r.visible)/span>/span>span classline>span classgi>+ map.addLayer(r.polyline);/span>/span>span classline>span classgi>+/span>/span>span classline>span classgi>+ r.polyline.on('mouseover', function(routeId) {/span>/span>span classline> return function () { highlightRoute(routeId, true); };/span>span classline> }(r.id));/span>/code>/pre>/td>/tr>/table>/div>/figure>p>This above snippet shows how a bit of our polyline code changed./p>p>One happy side effect of this conversion was that the mobile web page (forusers who can’t or don’t use our iPhone and Android apps) runs better.Animating markers on Google Maps with a Chrome-on-Android user agent wouldcause unbearable flickering. Leaflet runs smoothly on mobile./p>h2>OpenStreetMap for the future/h2>p>We’re excited about joining the OpenStreetMap ecosystem. There’s an incredibleamount of work that’s been put into OSM by its contributors, and we hope tocontribute back what we can. One of our iPhone apps is already using MapBox ontop of Apple Maps, and OSM might be what bring sanity back to mobile maps.We’re also looking at how we can use OSM data for geocoding, routing, and anyother geo needs we might have – saving money while helping an open communitygrow is a win-win for the years to come./p>p>You can check out one of live bus-tracking maps ata hrefhttp://iub.doublemap.com/map/>iub.doublemap.com/a>./p>p>a hrefhttps://news.ycombinator.com/item?id7675799>Comments on Hacker News/a>/p>/div> /article> div classpagination> a href/blog/archives>Blog Archives/a> /div>/div>aside classsidebar> section>h1>a hrefhttp://www.doublemap.com/>img styleborder: 0; box-shadow: 0; border-radius: 0;-webkit-box-shadow:0 src/images/doublemap-logo.png />/a>/h1>p>DoubleMap provides real-time GPS systems for transit, including mobile apps,passenger counters, voice announcements, and more. We’re a small, polyglot shoplocated in Indianapolis, Indiana./p>p>a hrefhttp://www.doublemap.com/>www.doublemap.com/a>/p>/section>section> h1>Recent Posts/h1> ul idrecent_posts> li classpost> a href/blog/2015/05/29/optimizing-python/>Optimizing Python With Cython/a> /li> li classpost> a href/blog/2014/05/29/everyone-panic/>Everyone Panic! Almost Free Downtime Phone Alerts/a> /li> li classpost> a href/blog/2014/04/29/porting-600k-map-views-to-openstreetmap/>Porting 600k Map Views to OpenStreetMap/MapBox/a> /li> /ul>/section> /aside> /div> /div> footer rolecontentinfo>p> Copyright © 2015 - The DoubleMap team - span classcredit>Powered by a hrefhttp://octopress.org>Octopress/a>/span>/p>/footer> script typetext/javascript> (function(){ var twitterWidgets document.createElement(script); twitterWidgets.type text/javascript; twitterWidgets.async true; twitterWidgets.src //platform.twitter.com/widgets.js; document.getElementsByTagName(head)0.appendChild(twitterWidgets); })(); /script>/body>/html>
Port 443
HTTP/1.1 200 OKConnection: keep-aliveContent-Length: 38810Server: GitHub.comContent-Type: text/html; charsetutf-8permissions-policy: interest-cohort()Last-Modified: Fri, 29 May 2015 16:58:32 GMTAccess-Control-Allow-Origin: *ETag: 55689ab8-979aexpires: Fri, 04 Oct 2024 05:50:00 GMTCache-Control: max-age600x-proxy-cache: MISSX-GitHub-Request-Id: 4563:3FD743:F50414:FBFCFB:66FF7FB0Accept-Ranges: bytesAge: 0Date: Fri, 04 Oct 2024 05:40:00 GMTVia: 1.1 varnishX-Served-By: cache-bfi-krnt7300065-BFIX-Cache: MISSX-Cache-Hits: 0X-Timer: S1728020401.897783,VS0,VE98Vary: Accept-EncodingX-Fastly-Request-ID: 68a29c1ee9b34c064277cc8dbe9b96551abc9345 !DOCTYPE html>!--if IEMobile 7 >html classno-js iem7>!endif-->!--if lt IE 9>html classno-js lte-ie8>!endif-->!--if (gt IE 8)|(gt IEMobile 7)|!(IEMobile)|!(IE)>!-->html classno-js langen>!--!endif-->head> meta charsetutf-8> title>DoubleMap Engineering/title> meta nameauthor contentThe DoubleMap team> meta namedescription contentPython is slow. It’s still faster than a lot of other languages, butdescribing Python code as “fast” will probably get some weird …> !-- http://t.co/dKP3o1e --> meta nameHandheldFriendly contentTrue> meta nameMobileOptimized content320> meta nameviewport contentwidthdevice-width, initial-scale1> link relcanonical hrefhttp://doublemap.github.io> link href/favicon.png relicon> link href/stylesheets/screen.css mediascreen, projection relstylesheet typetext/css> link href/atom.xml relalternate titleDoubleMap Engineering typeapplication/atom+xml> script src/javascripts/modernizr-2.0.js>/script> script src//ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js>/script> script>!window.jQuery && document.write(unescape(%3Cscript src./javascripts/libs/jquery.min.js%3E%3C/script%3E))/script> script src/javascripts/octopress.js typetext/javascript>/script> !--Fonts from Googles Web font directory at http://google.com/webfonts -->link href//fonts.googleapis.com/css?familyPT+Serif:regular,italic,bold,bolditalic relstylesheet typetext/css>link href//fonts.googleapis.com/css?familyPT+Sans:regular,italic,bold,bolditalic relstylesheet typetext/css> script typetext/javascript> var _gaq _gaq || ; _gaq.push(_setAccount, UA-50542806-1); _gaq.push(_trackPageview); (function() { var ga document.createElement(script); ga.type text/javascript; ga.async true; ga.src (https: document.location.protocol ? https://ssl : http://www) + .google-analytics.com/ga.js; var s document.getElementsByTagName(script)0; s.parentNode.insertBefore(ga, s); })(); /script>/head>body > header rolebanner>hgroup> h1>a href/>DoubleMap Engineering/a>/h1> h2>The official tech blog of DoubleMap/h2> /hgroup>/header> nav rolenavigation>ul classsubscription data-subscriptionrss> li>a href/atom.xml relsubscribe-rss titlesubscribe via RSS>RSS/a>/li> /ul> form actionhttps://www.google.com/search methodget> fieldset rolesearch> input typehidden nameq valuesite:doublemap.github.io /> input classsearch typetext nameq results0 placeholderSearch/> /fieldset>/form> ul classmain-navigation> li>a href/>Blog/a>/li> li>a href/blog/archives>Archives/a>/li>/ul>/nav> div idmain> div idcontent> div classblog-index> article> header> h1 classentry-title>a href/blog/2015/05/29/optimizing-python/>Optimizing Python With Cython/a>/h1> p classmeta> time datetime2015-05-29T11:09:00-04:00 pubdate data-updatedtrue>May 29span>th/span>, 2015/time> /p> /header> div classentry-content>p>Python is slow. It’s still faster than a lot of other languages, butdescribing Python code as “fast” will probably get some weird looks frompeople who use compiled or JITted languages./p>p>Still, I love Python for its design and speed at which I can write good code.Most of the time, when writing Python for distributed or web applications, itdoesn’t matter. Your runtime is usually dominated by network traffic or dataaccesses than it is by raw computational speed. In those cases, the correctoptimization is often reducing network requests or indexing your data./p>p>Sometimes—just occasionally—we actually do need fast computation.Traditional Python wisdom has been to profile your code (perhaps usingcProfile), identify the hot spot, and rewrite it in C as a Python extension.I’ve had to do that before for a handwriting recognition algorithm with greatsuccess (it turned a 2–3 second computation into a practicallyreal-time one), but writing C code and then figuring out the FFI was a pain.I had to rewrite the algorithm in C, verify that my C version was correct anddidn’t have memory errors, and then figure out the right incantations to getit to cooperate with Python./p>p>Let’s try something different this time./p>h2>Problem/h2>p>At DoubleMap, geographic distance calculation is something that gets used alot, and we use the haversine function to calculate distance between twopoints on Earth. In some of our data analysis, we might run the haversinefunction millions of times in a tight loop, and it was causing some reports totake way too long./p>p>Profiling the code showed that the two heaviest operations were fetching datafrom the data store and the haversine function./p>p>strong>Play along at home:/strong> Clone the repo ata hrefhttps://github.com/doublemap/haversine-optimization>https://github.com/doublemap/haversine-optimization/a> and start from the firstcommit to see how the code changes step by step./p>p>The commands you should know are code>pip install cython/code> and code>python setup.pybuild_ext --inplace/code>./p>h2>Original code/h2>figure classcode>div classhighlight>table>tr>td classgutter>pre classline-numbers>span classline-number>1/span>span classline-number>2/span>span classline-number>3/span>span classline-number>4/span>span classline-number>5/span>span classline-number>6/span>span classline-number>7/span>span classline-number>8/span>span classline-number>9/span>span classline-number>10/span>span classline-number>11/span>span classline-number>12/span>span classline-number>13/span>span classline-number>14/span>span classline-number>15/span>span classline-number>16/span>span classline-number>17/span>span classline-number>18/span>span classline-number>19/span>span classline-number>20/span>span classline-number>21/span>span classline-number>22/span>/pre>/td>td classcode>pre>code classpython>span classline>span classkn>from/span> span classnn>math/span> span classkn>import/span> span classn>sin/span>span classp>,/span> span classn>cos/span>span classp>,/span> span classn>acos/span>/span>span classline>/span>span classline>/span>span classline>span classk>def/span> span classnf>haversine/span>span classp>(/span>span classn>coord1/span>span classp>,/span> span classn>coord2/span>span classp>):/span>/span>span classline> span classsd>"""Given two (lat, lng) tuples, returns the distance between them in/span>/span>span classline>span classsd> meters."""/span>/span>span classline> span classn>lat1/span>span classp>,/span> span classn>lng1/span> span classo>/span> span classn>coord1/span>/span>span classline> span classn>lat2/span>span classp>,/span> span classn>lng2/span> span classo>/span> span classn>coord2/span>/span>span classline>/span>span classline> span classk>if/span> span classn>lat1/span> span classo>>/span> span classmi>90/span> span classow>or/span> span classn>lat1/span> span classo></span> span classo>-/span>span classmi>90/span> span classow>or/span> span classn>lat2/span> span classo>>/span> span classmi>90/span> span classow>or/span> span classn>lat2/span> span classo></span> span classo>-/span>span classmi>90/span>span classp>:/span>/span>span classline> span classk>raise/span> span classne>ValueError/span>span classp>(/span>span classs>"Invalid latitude (should be between +/- 90)"/span>span classp>)/span>/span>span classline> span classk>if/span> span classn>lng1/span> span classo>>/span> span classmi>180/span> span classow>or/span> span classn>lng1/span> span classo></span> span classo>-/span>span classmi>180/span> span classow>or/span> span classn>lng2/span> span classo>>/span> span classmi>180/span> span classow>or/span> span classn>lng2/span> span classo></span> span classo>-/span>span classmi>180/span>span classp>:/span>/span>span classline> span classk>raise/span> span classne>ValueError/span>span classp>(/span>span classs>"Invalid longitude (should be between +/- 180)"/span>span classp>)/span>/span>span classline>/span>span classline> span classn>phi1/span> span classo>/span> span classp>(/span>span classmf>90.0/span> span classo>-/span> span classn>lat1/span>span classp>)/span> span classo>*/span> span classmf>0.0174532925/span>/span>span classline> span classn>phi2/span> span classo>/span> span classp>(/span>span classmf>90.0/span> span classo>-/span> span classn>lat2/span>span classp>)/span> span classo>*/span> span classmf>0.0174532925/span>/span>span classline> span classn>theta1/span> span classo>/span> span classn>lng1/span> span classo>*/span> span classmf>0.0174532925/span>/span>span classline> span classn>theta2/span> span classo>/span> span classn>lng2/span> span classo>*/span> span classmf>0.0174532925/span>/span>span classline>/span>span classline> span classn>c/span> span classo>/span> span classp>(/span>span classn>sin/span>span classp>(/span>span classn>phi1/span>span classp>)/span> span classo>*/span> span classn>sin/span>span classp>(/span>span classn>phi2/span>span classp>)/span> span classo>*/span> span classn>cos/span>span classp>(/span>span classn>theta1/span> span classo>-/span> span classn>theta2/span>span classp>)/span> span classo>+/span> span classn>cos/span>span classp>(/span>span classn>phi1/span>span classp>)/span> span classo>*/span> span classn>cos/span>span classp>(/span>span classn>phi2/span>span classp>))/span>/span>span classline> span classn>arc/span> span classo>/span> span classn>acos/span>span classp>(/span>span classn>c/span>span classp>)/span>/span>span classline> span classk>return/span> span classn>arc/span> span classo>*/span> span classmf>6367444.7/span>/span>/code>/pre>/td>/tr>/table>/div>/figure>p>Also, here’s the benchmark code that is used throughout:/p>figure classcode>div classhighlight>table>tr>td classgutter>pre classline-numbers>span classline-number>1/span>span classline-number>2/span>span classline-number>3/span>span classline-number>4/span>span classline-number>5/span>span classline-number>6/span>/pre>/td>td classcode>pre>code classpython>span classline>span classkn>from/span> span classnn>__future__/span> span classkn>import/span> span classn>print_function/span>/span>span classline>span classkn>from/span> span classnn>timeit/span> span classkn>import/span> span classn>timeit/span>/span>span classline>/span>span classline>span classk>print/span>span classp>(/span>span classn>timeit/span>span classp>(/span>span classs>"haversine((39.132213, -86.12439), (38.55213, -86.94910))"/span>span classp>,/span>/span>span classline> span classn>setup/span>span classo>/span>span classs>"from haversine import haversine"/span>span classp>,/span>/span>span classline> span classn>number/span>span classo>/span>span classmi>300000/span>span classp>))/span>/span>/code>/pre>/td>/tr>/table>/div>/figure>p>Pretty straightforward. Let’s try pre-compiling this code into a C extensionto see if that helps./p>p>Following thea hrefhttp://docs.cython.org/src/tutorial/cython_tutorial.html>Cython Basic Tutorial/a>,I renamed code>haversine.py/code> to code>haversine.pyx/code> and created a code>setup.py/code> file:/p>figure classcode>div classhighlight>table>tr>td classgutter>pre classline-numbers>span classline-number>1/span>span classline-number>2/span>span classline-number>3/span>span classline-number>4/span>span classline-number>5/span>span classline-number>6/span>/pre>/td>td classcode>pre>code classpython>span classline>span classkn>from/span> span classnn>distutils.core/span> span classkn>import/span> span classn>setup/span>/span>span classline>span classkn>from/span> span classnn>Cython.Build/span> span classkn>import/span> span classn>cythonize/span>/span>span classline>/span>span classline>span classn>setup/span>span classp>(/span>/span>span classline> span classn>ext_modules/span> span classo>/span> span classn>cythonize/span>span classp>(/span>span classs>"haversine.pyx"/span>span classp>)/span>/span>span classline>span classp>)/span>/span>/code>/pre>/td>/tr>/table>/div>/figure>p>Running code>python setup.py build_ext --inplace/code> built code>haversine.so/code> out of myunmodified Python code. Easy-peasy./p>p>Benchmark results (300,000 iterations):/p>p>strong>Original code:/strong> 2.85 secondsbr/>strong>Compiled:/strong> 2.01 seconds/p>p>So, 29% time savings without any modification to the original code./p>h2>C math functions/h2>p>Currently, we’re still using Python’s math functions. Let’s see if we can cutout some overhead by importing code>math.h/code> instead./p>p>Except, since this is Cython, all we need to do is:/p>figure classcode>div classhighlight>table>tr>td classgutter>pre classline-numbers>span classline-number>1/span>span classline-number>2/span>span classline-number>3/span>span classline-number>4/span>span classline-number>5/span>/pre>/td>td classcode>pre>code classdiff>span classline>span classgd>--- a/haversine.pyx/span>/span>span classline>span classgi>+++ b/haversine.pyx/span>/span>span classline>span classgu>@@ -1,4 +1,4 @@/span>/span>span classline>span classgd>-from math import sin, cos, acos/span>/span>span classline>span classgi>+from libc.math cimport sin, cos, acos/span>/span>/code>/pre>/td>/tr>/table>/div>/figure>p>Benchmark results (300,000 iterations):/p>p>strong>Original code:/strong> 2.85 secondsbr/>strong>Compiled:/strong> 2.01 secondsbr/>strong>libc.math:/strong> 1.33 seconds/p>p>So far we’ve saved 53% of the time from the original code./p>h2>C types/h2>p>Cython’s biggest extension of the Python language is its type annotations.Speed ups can be had by telling Cython the types of each variable in advance.We are dealing with geographic coordinates, so we’ll be using doubles forpretty much everything:/p>figure classcode>div classhighlight>table>tr>td classgutter>pre classline-numbers>span classline-number>1/span>span classline-number>2/span>span classline-number>3/span>span classline-number>4/span>span classline-number>5/span>span classline-number>6/span>span classline-number>7/span>span classline-number>8/span>span classline-number>9/span>span classline-number>10/span>span classline-number>11/span>span classline-number>12/span>span classline-number>13/span>span classline-number>14/span>span classline-number>15/span>span classline-number>16/span>span classline-number>17/span>span classline-number>18/span>span classline-number>19/span>span classline-number>20/span>span classline-number>21/span>span classline-number>22/span>span classline-number>23/span>span classline-number>24/span>span classline-number>25/span>span classline-number>26/span>span classline-number>27/span>span classline-number>28/span>span classline-number>29/span>span classline-number>30/span>/pre>/td>td classcode>pre>code classpython>span classline>span classk>def/span> span classnf>haversine/span>span classp>(/span>span classnb>tuple/span> span classn>coord1/span>span classp>,/span> span classnb>tuple/span> span classn>coord2/span>span classp>):/span>/span>span classline> span classsd>"""Given two (lat, lng) tuples, returns the distance between them in/span>/span>span classline>span classsd> meters."""/span>/span>span classline> span classn>cdef/span> span classn>double/span> span classn>lat1/span>/span>span classline> span classn>cdef/span> span classn>double/span> span classn>lng1/span>/span>span classline> span classn>cdef/span> span classn>double/span> span classn>lat2/span>/span>span classline> span classn>cdef/span> span classn>double/span> span classn>lng2/span>/span>span classline> span classn>lat1/span>span classp>,/span> span classn>lng1/span> span classo>/span> span classn>coord1/span>/span>span classline> span classn>lat2/span>span classp>,/span> span classn>lng2/span> span classo>/span> span classn>coord2/span>/span>span classline>/span>span classline> span classk>if/span> span classn>lat1/span> span classo>>/span> span classmi>90/span> span classow>or/span> span classn>lat1/span> span classo></span> span classo>-/span>span classmi>90/span> span classow>or/span> span classn>lat2/span> span classo>>/span> span classmi>90/span> span classow>or/span> span classn>lat2/span> span classo></span> span classo>-/span>span classmi>90/span>span classp>:/span>/span>span classline> span classk>raise/span> span classne>ValueError/span>span classp>(/span>span classs>"Invalid latitude (should be between +/- 90)"/span>span classp>)/span>/span>span classline> span classk>if/span> span classn>lng1/span> span classo>>/span> span classmi>180/span> span classow>or/span> span classn>lng1/span> span classo></span> span classo>-/span>span classmi>180/span> span classow>or/span> span classn>lng2/span> span classo>>/span> span classmi>180/span> span classow>or/span> span classn>lng2/span> span classo></span> span classo>-/span>span classmi>180/span>span classp>:/span>/span>span classline> span classk>raise/span> span classne>ValueError/span>span classp>(/span>span classs>"Invalid longitude (should be between +/- 180)"/span>span classp>)/span>/span>span classline>/span>span classline> span classn>cdef/span> span classn>double/span> span classn>ph1/span>/span>span classline> span classn>cdef/span> span classn>double/span> span classn>ph2/span>/span>span classline> span classn>cdef/span> span classn>double/span> span classn>theta1/span>/span>span classline> span classn>cdef/span> span classn>double/span> span classn>theta2/span>/span>span classline> span classn>cdef/span> span classn>double/span> span classn>c/span>/span>span classline> span classn>cdef/span> span classn>double/span> span classn>arc/span>/span>span classline>/span>span classline> span classn>phi1/span> span classo>/span> span classp>(/span>span classmf>90.0/span> span classo>-/span> span classn>lat1/span>span classp>)/span> span classo>*/span> span classmf>0.0174532925/span>/span>span classline> span classn>phi2/span> span classo>/span> span classp>(/span>span classmf>90.0/span> span classo>-/span> span classn>lat2/span>span classp>)/span> span classo>*/span> span classmf>0.0174532925/span>/span>span classline> span classn>theta1/span> span classo>/span> span classn>lng1/span> span classo>*/span> span classmf>0.0174532925/span>/span>span classline> span classn>theta2/span> span classo>/span> span classn>lng2/span> span classo>*/span> span classmf>0.0174532925/span>/span>span classline>/span>span classline> span classn>c/span> span classo>/span> span classp>(/span>span classn>sin/span>span classp>(/span>span classn>phi1/span>span classp>)/span> span classo>*/span> span classn>sin/span>span classp>(/span>span classn>phi2/span>span classp>)/span> span classo>*/span> span classn>cos/span>span classp>(/span>span classn>theta1/span> span classo>-/span> span classn>theta2/span>span classp>)/span> span classo>+/span> span classn>cos/span>span classp>(/span>span classn>phi1/span>span classp>)/span> span classo>*/span> span classn>cos/span>span classp>(/span>span classn>phi2/span>span classp>))/span>/span>span classline> span classn>arc/span> span classo>/span> span classn>acos/span>span classp>(/span>span classn>c/span>span classp>)/span>/span>span classline> span classk>return/span> span classn>arc/span> span classo>*/span> span classmf>6367444.7/span>/span>/code>/pre>/td>/tr>/table>/div>/figure>p>strong>Original code:/strong> 2.85 secondsbr/>strong>Compiled:/strong> 2.01 secondsbr/>strong>libc.math:/strong> 1.33 secondsbr/>strong>C types:/strong> 0.466 seconds/p>p>In three easy steps, we’ve reduced the run time of this function by 84%. Moreimportantly, we’re still writing what’s basically Python. We don’t have toremember to free our variables or check our array bounds. That additionalsafety is not free (and there are various flags to disable Cython safetychecks) but with Cython, optimizing a Python function doesn’t have to be arewrite-everything task./p>p>We probably could have gone further in optimizing this, but these speed-upsgot us into “good enough” territory, and that’s good enough./p>h2>Distribution/h2>p>Our optimized haversine function is available on PyPI under the name“cHaversine”. One key difference is that the distributed tarball includes theC code generated by Cython so that you, as the package user, don’t need Cythonto download and build the C extension./p>p>a hrefhttps://news.ycombinator.com/item?id9625737>Comments on Hacker News/a>/p>/div> /article> article> header> h1 classentry-title>a href/blog/2014/05/29/everyone-panic/>Everyone Panic! Almost Free Downtime Phone Alerts/a>/h1> p classmeta> time datetime2014-05-29T11:24:46-04:00 pubdate data-updatedtrue>May 29span>th/span>, 2014/time> /p> /header> div classentry-content>p>a hrefhttps://github.com/doublemap/everyonepanic>Uptime Robot (free) + App Engine (free) + Twilio (cheap) almost freedowntime alerts!/a>/p>ol>li>Uptime Robot notices that your website is not responding./li>li>Everyone Panic! checks Uptime Robot and sees that something is down./li>li>You get a call through Twilio telling you to panic./li>li>You get another call 15 minutes later because you still haven’t fixed theproblem./li>/ol>p>Sure, it’s easy to get emailed when your website goes down, but realhonest-to-goodness phone calls have an immediacy that’s hard to beat./p>p>Since our GPS bus-tracking system is used at all hours of the day, early on weintroduced automated monitoring. But when everyone is asleep, emails and textmessages go unnoticed until the morning and it sucks to not know where yourbus is at 1am./p>p>We quickly whipped an app together using Python to tieUptime Robot and Twilio together. If one of the items in Uptime Robot goesdown then the Python app will ask Twilio to call us. We made it, put it on AppEngine, and haven’t touched it since. Even though our automated monitoring hasgrown since then, this little app, with zero maintenance, still dutifullywatches Uptime Robot for any websites that are down./p>p>Literally, there have been zero commits to the original repo since the firstday. Now the code’s been cleaned up anda hrefhttps://github.com/doublemap/everyonepanic>put on GitHub/a>. You can downloadit, configure it, and throw it onto App Engine or Heroku in a few minutes./p>p>It’s not entirely free – you’ll need to pay Twilio for voice calls, but theprice is $0.02 per call and it’s hard to think of a situation where knowingyour website is down isn’t worth two cents. And, of course, you should keepyour Twilio account balance in the black./p>p>But for the hobbyist who doesn’t mind trying out a new app, this is a reallysimple way to have your phone ring when Hacker News tramples your latest petproject./p>p>a hrefhttps://github.com/doublemap/everyonepanic>Everyone Panic! on GitHub/a>/p>p>a hrefhttps://news.ycombinator.com/item?id7816241>Comments on Hacker News/a>/p>/div> /article> article> header> h1 classentry-title>a href/blog/2014/04/29/porting-600k-map-views-to-openstreetmap/>Porting 600k Map Views to OpenStreetMap/MapBox/a>/h1> p classmeta> time datetime2014-04-29T10:25:45-04:00 pubdate data-updatedtrue>Apr 29span>th/span>, 2014/time> /p> /header> div classentry-content>p>img classright src/images/doublemap-plus-mapbox.png>This past week, we here at DoubleMap made a big change that users probablyem>won’t/em> notice. Instead of Google Maps, all of our public maps are now poweredby MapBox with OpenStreetMap data./p>p>We have hundreds of thousands of hits per month to our online maps from peopleusing our real-time bus tracking to find out where their bus is. This numberdoesn’t even include transit riders using their agency’s custom app or publicTV displays./p>p>For a company whose main feature is showing buses move on a map, it was a bigdeal for everyone – we had to make sure that data at each of our clienttransit agencies was good, update some of our sales material that specificallymentioned Google Maps, decide on new providers, and figure out the overallcosts of switching and notswitching. We didn’t want to disrupt our transit agencies and the riders thatuse our service to catch the bus every day. The actual coding changes wererelatively straightforward./p>p>Thanks to MapBox, were able to switch everything over almost seamlesslywithout interrupting our users or creating a jarring change./p>h2>Evaluation/h2>p>It shouldn’t be a surprise that the main motivation was simply cost. Googledoesn’t publish its enterprise maps pricing, but it’s orders of magnitude moreexpensive than MapBox./p>p>We took a careful look at what we would have lost by switching away fromGoogle. The main feature that we can’t get anywhere else is Street View,something that nobody has come close to replicating. We also lost out onangled aerial imagery. Both of these are cool, but ultimately not worth theprice difference./p>p>video width592 stylewidth:592px loop autoplay> source src/images/iubdoublemap.mp4 typevideo/mp4> source src/images/iubdoublemap.webm typevideo/webm>/video>/p>h2>What’s in a map?/h2>p>Part of the thinking change that most people have to go through when switchingis that maps don’t always come in one giant bundle.“Google Maps” can refer to everything from the JavaScript library tothe pop-ups with user reviews to the navigation to Street View. WithOpenStreetMap, you get your choice of JavaScript mapping engine and differenttile sets./p>p>For example, you might choose to use a hrefhttp://leafletjs.com/>Leaflet/a> witha hrefhttp://www.opencyclemap.org/>OpenCycleMap/a> and a hrefhttps://www.mapbox.com/blog/mapbox-satellite/>MapBoxSatellite/a> as your layers, oryou might use a hrefhttp://openlayers.org/>OpenLayers/a> with your owna hrefhttp://tilestache.org/>tileserver/a>plus the a hrefhttp://www.thunderforest.com/landscape/>Thunderforest Landscape/a>map./p>p>em>Everything is swappable./em> When we were testing, it was a one-line change toswitch between MapQuest Open, CloudMade, and MapBox tiles./p>p>We ended up using Leaflet with a custom MapBox style plus MapBox Satellite,and it looks pretty awesome./p>h2>Map styling/h2>p>One of the complaints levied against OpenStreetMap is that their map stylelooks ugly. That’s not really fair, because OSM is there to provide the mapdata, and most people who see OSM data see it through a custom map style, likeon FourSquare or Pinterest (or DoubleMap)./p>p>The power and curse of OSM is that you have total control over how the maplooks. The power is that if you want a bubble-gum-pink map with 36pt Comic Sanslabels and no roads, that’s easily doable.The curse is that it can get complicated, such as when you decide thatyou need to label each US highway using its state-specific shield image.As TileMill matures, there will hopefully be more and more styles to build onrather than starting from scratch./p>p>One of the reasons why we initially went with CloudMade was because they hadan easy online map styler that let you point and click a feature, and thenchange that feature class’s colors. This let us mostly duplicate our customstyle we were using with Google Maps. CloudMade, however, decided toa hrefhttp://notes.ericjiang.com/posts/741>get out of non-enterprise services/a>not too long after I had gotten the map to look the way I wanted it to./p>p>em>Enter MapBox./em> MapBox has a really slick desktop program, TileMill, that letsyou generate map tiles using a CSS-ish language called CartoCSS. The downsideof TileMill is that you have to download raw map data, render the tiles youneed (which easily gets into the gigabytes), and then upload them somewhere.We have clients internationally and we would have had to download eachregion’s OSM data and render new tiles every time we got another customer./p>p>This is all fixed in a hrefhttps://github.com/mapbox/tm2>TM2/a>, which downloadsvector map tiles from MapBoxon the fly so you can upload just the compiled map style to MapBox andthey’ll handle all the data and rendering as needed./p>p>Here’s our result:/p>p>img src/images/osm-google-comparison.png>/p>p>Our custom style is designed to be muted with a limited color palette to letour agencies’ custom route colors pop when overlaid. I hope to write aboutTileMill 2 in the future, but right now it’s in “experimental” status./p>h2>Porting the source code/h2>p>Our original front-end code was written to use v2 of the Google Maps JS API.Once that was deprecated, we eventually rewrote it to use v3 of the GoogleMaps API. When we started investigating OSM as a replacement, we noticed howpolished the Leaflet library was. The Leaflet API is based on v2 of theGoogle Maps API so we were essentially doing the reverse of what we haddone in the past. All of the concepts mapped almost 1-to-1, and there wasnothing that Leaflet lacked in terms of functionality./p>figure classcode>div classhighlight>table>tr>td classgutter>pre classline-numbers>span classline-number>1/span>span classline-number>2/span>span classline-number>3/span>span classline-number>4/span>span classline-number>5/span>span classline-number>6/span>span classline-number>7/span>span classline-number>8/span>span classline-number>9/span>span classline-number>10/span>span classline-number>11/span>span classline-number>12/span>span classline-number>13/span>span classline-number>14/span>span classline-number>15/span>span classline-number>16/span>span classline-number>17/span>span classline-number>18/span>span classline-number>19/span>span classline-number>20/span>span classline-number>21/span>/pre>/td>td classcode>pre>code classdiff>span classline>span classgd>- r.polyline new google.maps.Polyline({/span>/span>span classline>span classgd>- path: points,/span>/span>span classline>span classgd>- map: r.visible ? map : null,/span>/span>span classline>span classgd>- strokeColor: "#"+r.color,/span>/span>span classline>span classgd>- strokeOpacity: 0.6,/span>/span>span classline>span classgd>- strokeWeight: 4,/span>/span>span classline>span classgd>- zIndex: 1/span>/span>span classline>span classgd>- })/span>/span>span classline>span classgd>- google.maps.event.addListener(r.polyline, 'mouseover',/span>/span>span classline>span classgd>- function(routeId) {/span>/span>span classline>span classgi>+ r.polyline new L.Polyline(points, {/span>/span>span classline>span classgi>+ color: "#"+r.color,/span>/span>span classline>span classgi>+ opacity: 0.6,/span>/span>span classline>span classgi>+ weight: 4/span>/span>span classline>span classgi>+ });/span>/span>span classline>span classgi>+ if(r.visible)/span>/span>span classline>span classgi>+ map.addLayer(r.polyline);/span>/span>span classline>span classgi>+/span>/span>span classline>span classgi>+ r.polyline.on('mouseover', function(routeId) {/span>/span>span classline> return function () { highlightRoute(routeId, true); };/span>span classline> }(r.id));/span>/code>/pre>/td>/tr>/table>/div>/figure>p>This above snippet shows how a bit of our polyline code changed./p>p>One happy side effect of this conversion was that the mobile web page (forusers who can’t or don’t use our iPhone and Android apps) runs better.Animating markers on Google Maps with a Chrome-on-Android user agent wouldcause unbearable flickering. Leaflet runs smoothly on mobile./p>h2>OpenStreetMap for the future/h2>p>We’re excited about joining the OpenStreetMap ecosystem. There’s an incredibleamount of work that’s been put into OSM by its contributors, and we hope tocontribute back what we can. One of our iPhone apps is already using MapBox ontop of Apple Maps, and OSM might be what bring sanity back to mobile maps.We’re also looking at how we can use OSM data for geocoding, routing, and anyother geo needs we might have – saving money while helping an open communitygrow is a win-win for the years to come./p>p>You can check out one of live bus-tracking maps ata hrefhttp://iub.doublemap.com/map/>iub.doublemap.com/a>./p>p>a hrefhttps://news.ycombinator.com/item?id7675799>Comments on Hacker News/a>/p>/div> /article> div classpagination> a href/blog/archives>Blog Archives/a> /div>/div>aside classsidebar> section>h1>a hrefhttp://www.doublemap.com/>img styleborder: 0; box-shadow: 0; border-radius: 0;-webkit-box-shadow:0 src/images/doublemap-logo.png />/a>/h1>p>DoubleMap provides real-time GPS systems for transit, including mobile apps,passenger counters, voice announcements, and more. We’re a small, polyglot shoplocated in Indianapolis, Indiana./p>p>a hrefhttp://www.doublemap.com/>www.doublemap.com/a>/p>/section>section> h1>Recent Posts/h1> ul idrecent_posts> li classpost> a href/blog/2015/05/29/optimizing-python/>Optimizing Python With Cython/a> /li> li classpost> a href/blog/2014/05/29/everyone-panic/>Everyone Panic! Almost Free Downtime Phone Alerts/a> /li> li classpost> a href/blog/2014/04/29/porting-600k-map-views-to-openstreetmap/>Porting 600k Map Views to OpenStreetMap/MapBox/a> /li> /ul>/section> /aside> /div> /div> footer rolecontentinfo>p> Copyright © 2015 - The DoubleMap team - span classcredit>Powered by a hrefhttp://octopress.org>Octopress/a>/span>/p>/footer> script typetext/javascript> (function(){ var twitterWidgets document.createElement(script); twitterWidgets.type text/javascript; twitterWidgets.async true; twitterWidgets.src //platform.twitter.com/widgets.js; document.getElementsByTagName(head)0.appendChild(twitterWidgets); })(); /script>/body>/html>
View on OTX
|
View on ThreatMiner
Please enable JavaScript to view the
comments powered by Disqus.
Data with thanks to
AlienVault OTX
,
VirusTotal
,
Malwr
and
others
. [
Sitemap
]