August 16, 2018

Retrieving information about the contributors – An update of OSMPythonTools

OpenStreetMap python

The Python package OSMPythonTools has been updated for retrieving information about the contributors in OSM. OpenStreetMap (OSM) data is useful for many purposes, and different services to access the data exist. The Overpass API, Nominatim, and the OSM API are amongst them. The Python package OSMPythonTools provides easy access to these services, and allows, in addition, to send batch queries to these services, as well as an interface to mine and visualize the resulting data.

The OSMPythonTools have become advanced steadily, and they have gained wide visibility. After having been introduced about one and a half year ago, they have been downloaded almost 11,000 times. More than 50 people stargazed the OSMPythonTools on GitHub, which you are welcome to do as well if you like them.

What is new?

Since its introduction, the OSMPythonTools have been advanced in many areas. Here, I would like to demonstrate two recent additions, the ability to retrieve the geometry provided by Nominatim and the ability to deliver information about the contribution process. Let’s start with the installation. After having installed Python 3, you can install the OSMPythonTools as follows:

pip3 install OSMPythonTools

Assume you want to retrieve the geometry of Heidelberg as is being stored in OSM. If you provide the parameter wkt=True to the corresponding, you retrieve the geometry as well-known text:

heidelberg = nominatim.query('Heidelberg', wkt=True)
heidelberg.wkt()
# 'POINT(8.694724 49.4093582)'

Depending on whether a more complex geometry or just a point is associated to the place name, different kinds of geometries are returned.

Accessing the history and data about the creation

One might want to retrieve information about the history of OSM elements. As an example, one can query for the changes made by the user franz-benjamin in Heidelberg in March 2017:

from OSMPythonTools.nominatim import Nominatim
from OSMPythonTools.overpass import overpassQueryBuilder
from OSMPythonTools.overpass import Overpass

heidelberg = Nominatim().query('Heidelberg, Germany')
query = overpassQueryBuilder(area=heidelberg.areaId(), elementType='node', since='2017-01-01T00:00:00Z', to='2017-02-01T00:00:00Z', user='franz-benjamin', out='meta')
changedNodes = Overpass().query(query)

It is important to use out='meta' because this will fetch the meta data about the history and the creation of the corresponding elements:

changedNodes.elements()[0].version()
# 8
changedNodes.elements()[0].changeset()
# 44871772
changedNodes.elements()[0].user()
# 'franz-benjamin'
changedNodes.elements()[0].userid()
# 4913117
changedNodes.elements()[0].timestamp()
'2017-01-03T13:47:23Z'

Such information can also be queried in a more structured way. First, we define different dimensions that will be used to query the data:

from OSMPythonTools.nominatim import Nominatim
from OSMPythonTools.overpass import Overpass, overpassQueryBuilder
from OSMPythonTools.data import Data, dictRangeYears, ALL

from collections import OrderedDict

dimensions = OrderedDict([
  ('elementType', OrderedDict({
    'node': 'node',
    'way': 'way',
    'relation': 'relation',
  })),
  ('user', OrderedDict({
    'franz-benjamin': 'franz-benjamin',
    'tyr_asd': 'tyr_asd',
  })),
])

Then, we have to determine the ID of the area we would like to examine. As an example, we can determine this ID for Heidelberg:

nominatim = Nominatim()
areaId = nominatim.query('Heidelberg').areaId()

Finally, the data can be queried according to the dimensions as follows:

overpass = Overpass()

def fetch(elementType, user):
    query = overpassQueryBuilder(area=areaId, elementType=elementType, since='2017-01-01T00:00:00Z', to='2017-02-01T00:00:00Z', user=user, out='meta')
    return overpass.query(query, timeout=500).countElements()

data = Data(fetch, dimensions)

Note that we have restricted the query to a certain short period. On might easily use a temporal dimension to add further periods of time.

As before, the object data contains the results of the queries and can be imagined as being a table:

                            value
elementType user
node        franz-benjamin     43
            tyr_asd            24
way         franz-benjamin      7
            tyr_asd             7
relation    franz-benjamin      0
            tyr_asd             1

If you want more information, please have a look at the documentation on github: https://github.com/mocnik-science/osm-python-tools.