2 February 2021 
#407: quantum of sollazzo – The data newsletter by @puntofisso
Read it in your browser

Made with
 

I opened last week's newsletter with my blabber about the importance of data definitions, and I was reminded of my other little obsession: the pervasiveness of uncertainty in data. In an apt example of the Baader–Meinhof phenomenon, data journalism guru Donata Columbro tweeted about an article by Alice Corona on the DataNinja magazine, showing different strategies to deal with uncertainty in data visualization: it's originally in Italian, but the automatically translated English version is ok.
  The Institute for Government has published the 2021 edition of Whitehall Monitor, its data-driven analysis of the workings of the UK Civil Service. This year's monitor includes a look at the controversial topic of contract-awarding.
  I love this semi-serious campaign aiming to stop people just saying "hello" via chat or message. I mean, by all means say hello but ALSO say what you want! It's not rude (which is what most perpetrators think...)
  I loved this talk about making disagreement at work more productive, presented by Claire Knight at You Got This From Your Couch. Tim Harford's talk about slow-motion multitasking was an eye opener about my multitude of hobbies that I never master (TL;DR: I'm not going to turn into the next Einstein, sadly, but at least now I know this way of engaging with passions is a common variant). We have again – for the last week – some some sponsored geotastic content – Ed Freyfogle, who's the organiser of location-based service meetup Geomob, co-host of the Geomob podcast, and co-founder of the OpenCage Geocoder, has offered to introduce a series of points around the topic of geodata. His final entry, on geocoding at scale,  is below.

Till next week,

 

 

--- Sponsored content by Open Cage ---

Geocoding at scale

In our final installment in our series about using open data for geocoding we contemplate the challenges of geocoding at scale. What are the issues you face when you have many hundreds of thousands or even millions of coordinates or addresses to work on daily? At OpenCage we serve numerous customers in this category, and a common question that comes up is whether an API based solution can handle that type of scale.

An API-based solution, managed by experts, is almost always the most reliable and most affordable way to develop such an on-going system, as otherwise you will soon be spending a lot of valuable developer time making sure your geodata is staying current. As anyone who has worked with software can confirm: “Building is easy, maintaining is hard”.

Nevertheless, there are challenges that come with depending on any external service, one of course being network availability. At OpenCage we have multiple, fully-redundant data centers, and the availability of our service is independently and publicly monitored by a third party (current and past operational status can be seen at status.opencagedata.com).

Still, even with a highly-available service, some customers worry about the “cost” of crossing the internet to an external service. The fastest API query is the one you don’t even make; a smart caching strategy can go a long way to reducing usage. Because our geocoding API is built on open data you can cache the results as long as you like, and we’ve published a few tips and points to consider.

We hope you’ve enjoyed our series on the issues around geocoding with open data. While we’ve used our service as the example, we believe many of the concepts and considerations will apply regardless of the data processing tools and services you are building on. If you have questions regarding anything we discussed, please get in touch.

If you have any geocoding needs please give the OpenCage Geocoder a try.