Documentation of Scoring Neighborhoods on the Earth Project ¶

Contents

Overview about this Project ¶

This project aims at using the Elo Rating System to quantitatively measure urban perceptions around the world. The measurement is based on digital surveys to humans, in which users are asked to compare two street view images in terms of some dimensions, for example, safety.

As a by-product of this project, I write a elorating module in Python, which can be generally used for other scoring projects that employ Elo Rating System. Particularly, as a computational economist, I note that even though Elo Rating System was originally designed to measure the relative skill levels of chess players based on competition records, it can be used as a Social Welfare Function that reveals collective preferences from individual preferences.

Currently, I use the survey data from Place Pulse 2.0, which covers 56 cities from 28 countries across 6 continents and keeps ongoing. I provide the reproducible Python scripts that calculate the Elo Rating scores in the repository of this project as a demonstration of the :mod:elorating module. I also visualize the outcome on interactive maps.

Furthermore, since Place Pulse 2.0 retrieves street view images from Google Maps, it does not cover mainland China (Google ends their consumer services in China since 2010, see more on Wikipedia). I plan to extend this project to cover mainland China in the future using street view images provided by Baidu Map. For now, I finished a program in Python that retrieves street view images. I have included this program in the repository.

elorating package ¶

This package contains the elorating.elorating module.

Installation ¶

To install the elorating package, clone this repository to your workplace, and run

>>> pip install /path/to/this/project

To uninstall, run

>>> pip uninstall elorating

Note

Importing this package will automatically import the members in the elorating module.

elorating module ¶

This module allows users to manage an Elo Rating project and calculate the scoring results.

Users can import data from files or manually operate elements within a project, calculate Elo Rating scores, and save data to a file. This class supports common operations and statistical methods. Scoring Neighborhoods on the Earth is a sample Elo Rating project using voting outcomes to measure urban perceptions.

class elorating.elorating.Element(element_id, info='')¶

A class used to represent an element to be added into the Elo Rating project.

An instance of this class is hashable. Instances of this class are the objects to work on in an Elo Rating project.

Parameters

element_id (object) – Any hashable object that can be used to uniquely identify the element.
info (str, optional) – Can store some information about this element such as name, alias, comments, etc. (default=’’)

element_id¶

a hashable object that represents an element.

Type: read-only

info¶

a string that stores descriptive information about an element.

Type: str

Example:

>>> import elorating
>>> el1 = elorating.Element('001774', info='The Uniersity of Chiacgo')
>>> el2 = elorating.Element('001739', info='Northwestern University')
>>> el1.element_id
'001774'
>>> el1 == el2
False
>>> hash(el1)
-7300844225016213262
>>> print(el1)
Element ID: 001774 ; Info: The Uniersity of Chiacgo.

property element_id

Return the id of this element. The id is read-only and cannot be changed over its lifetime.

Type: object

class elorating.elorating.Elo(base_score=1000, scale=100)¶

A class used to manage the Elo Rating project.

Parameters

base_score (int) – The default score of a newly added element. (default=1000)
scale (int) – The scoring scale used for normalizing the Elo scores to a user-defined scale.

Raises

InterruptedError – base_score is changed during the project.
TypeError – Some arguments do not satisfy their type requriements.

base_score¶

The default score of a newly added element.

Type: int

Example:

>>> import elorating
>>> scoring_universities = elorating.Elo()
>>> uchicago = elorating.Element('001774', info='The Uniersity of Chiacgo')
>>> northwestern = elorating.Element('001739', info='Northwestern University')
>>> scoring_universities.add(uchicago, northwestern)
[<elorating.Element object> Element ID: 001774 ; Info: The Uniersity of Chiacgo. , <elorating.Element object> Element ID: 001739 ; Info: Northwestern University. ]
>>> scoring_universities.update_rating(uchicago, northwestern, 1)
(1005.0, 995.0)
>>> scoring_universities.score(uchicago)
1005.0
>>> scoring_universities.win_prob(uchicago, northwestern)
0.5143871841659987
>>> print(scoring_universities.stats())

Highest: 1005.0
Lowest: 995.0
Mean: 1000.0
Stdev: 7.0710678118654755
>>> scoring_universities.remove(northwestern)
[<elorating.Element object> Element ID: 001739 ; Info: Northwestern University. ]
>>> scoring_universities.save_data(mute=1)
'./logfiles/elorating_log_Fri_Dec_6_15:45:54_2019.csv'

add(*elements)¶

Add an Element object to an Elo object.

Parameters: *elements (tuple(Element)) – one or more than one Element objects that need to be added into the Elo object.
Returns: A list of Element objects that have been added.
Raises: TypeError – at least one arguments is not Element object.

property base_score

Return the default rating score of a new element.

The default initial score cannot be change over the project to ensure the fairness of the rating.

import_data(filename)¶

Import a data file generated by this file to continue the project with the saved outcome.

The imported file must be a csv file that includes the three fields: ID, Info, RatingScore. The CSV should be consistent with the format requirements: - ID must be strings splitted by ‘,’ - Info must be a string - RatingScore must be a floating number

Raises

InterruptedError – Try to import data into an non-empty Elo object. Data from files can only be imported into an empty Elo object.
TyeError – The CSV file does not meet the format requirement.
KeyError – The CSV file is not found.

remove(*elements)¶

Remove an Element object to an Elo object.

Parameters: *elements (tuple(Element)) – one or more than one Element objects that need to be removed into the Elo object.
Returns: A list of Element objects that have been removed.
Raises: TypeError – at least one arguments is not Element object.

save_data(mute=0)¶

Save the scoring outcome to a csv file with three fields: ID, Info, RatingScore.

Parameters: mute (0 or 1) – if mute=1 the file path will not be printed; if mute=0 the file path will be printed when finishing saving data. (default=0)
Returns: The path of the saved data.

property scale¶: Return the scoring scale of the project, to which the exported data would be normalized to.

score(element)¶

Return the rating score of an element from current rating records.

Parameters: element (Element) – one Element object that need to be removed into the Elo object.
Returns: The score of this element.
Raises: KeyError – the element is not found in the current Elo object.

stats(option='all')¶

Calculate the statistics of current scoring outcome.

Parameters: option ('all', 'max', 'min, or 'stdev') – option=’max’: returns the highest score; option=’min’: returns the lowest score. option=’stdev’: returns the standard deviation.
Returns: Descriptive statistics (str).

update_rating(element1, element2, vote=0, K=10)¶

Update the rating score of [element1] and [element2] based on the racing or voting outcome.

Parameters

element1 (Element) – one Element object.
element2 (Element) – another Element object.
vote (0, 1, or 2, optional) – vote=0 tie; vote=1 element1 wins; vote=2 element2 wins. (default=0)
K (int, optional) – the parameter for the Elo Rating System. (default=10)

Returns

A tuple of two updated scores for the two elements (tuple(float, float)).

Raises

ValueError – the input value of value does not satisfy the requirement.

win_prob(element, opponent)¶

Return the predicted probability of one element beats another element.

Parameters

element (Element) – one Element object.
opponent (Element) – another Element object.

Returns

The predicted probability of one element wins another based on current Elo Rating scores.

Scoring Neighborhoods in 56 Cities (based on Place Pulse 2.0)¶

Overview ¶

Place Pulse is a crowdsourcing data collection project hosted by MIT. In this project, participants would be represented with two street view images for each time, and they would be asked to indicate their comparison between the two images in terms of one of the following questions (cited from Place Pulse):

Which Place Looks Safer?

Which Place Looks Wealthier?

Which Place Looks More Beautiful?

Which Place Looks More Boring?

Which Place Looks Livelier?

Which Place Looks More Depressing?

The second study Place Pulse 2.0 “extends the data collection to 56 cities from 28 countries across 6 continents”, making up a dataset containing 1,223,650 inidividual voting records.

As a demonstration of using the elorating module, I calculate the urban perceptions of all the 56 cities covered by the study based on the Place Pulse 2.0 data. The reproducible Python scripts are included in the repository of this project.

Interactive maps ¶

Interactive maps

Documentation of pp2_app ¶

Installation ¶

Clone the repository of this project to your workplace.

Install the elorating package.

>>> pip install /path/to/this/project

Install package dependencies

>>> pip install -r /path/to/this/project/pp2_app/package_requirement.txt

Tutorial ¶

To reproduce the map of some specific cities in terms of some dimensions:

>>> cd /path/to/this/project
>>> python3 pp2_app/pp2_visualization_app.py

To reproduce all the visualizations:

>>> cd /path/to/this/project
>>> python3 pp2_app/pp2_generate_all_maps.py

pp2_elorating_app reference ¶

This program uses http://pulse.media.mit.edu/data/ to demostrate an application of the elorating module.

To run from terminal:

>>> cd /path/to/this/project
>>> python3 pp2_app/pp2_elorating_app.py
Please input the path of Place Pulse 2.0 data <vote.csv>: .../pp2_20161010/votes.csv
Please choose a category: safety, lively, wealthy, beautiful, depressing, or boring: safety

This program uses the data from http://pulse.media.mit.edu/data/ to demostrate an application of the elorating module.

Dimension: safety

File length: 1223650
Reading Files |████████████████████████████████| 1223649/1223650

number of Elements: 111367
Loading Elements |████████████████████████████████| 111367/111367

number of valid records: 368926
Calculating Elo Scores |████████████████████████████████| 368926/368926


Data saved to ./logfiles/elorating_log_Fri_Dec_6_16:54:57_2019.csv



Highest: 1134.3642935436123
Lowest: 873.5102761124806
Mean: 1000.0
Stdev: 15.013932094550254

To import in Python (in the workplace ‘/path/to/this/project’):

import pp2_app.pp2_elorating_app

pp2_app.pp2_elorating_app.main(filename, category)¶

Calculate the Elo Rating scores of the locations in terms of certain dimension and export the data into a csv file.

Parameters

filename (str) –
the file path of the ‘vote.csv’ file downloaded from Place Pulse 2.0.
category (str) – can be one of the six dimensions: ‘safety’, ‘lively’, ‘wealthy’, ‘beautiful’, ‘depressing’, or ‘boring’.

Returns

The path of the exported data.

pp2_visualization_app reference ¶

This program calls the pp2_app.pp2_elorating_app module to calculate the Elo Rating scores of places in some cities in terms of some dimensions and export interactive maps to .html files.

To run from terminal:

>>> cd /path/to/this/project
>>> python3 pp2_app/pp2_visualization_app.py
Please input the path of Place Pulse 2.0 data <vote.csv>: /Users/tianluxin/Desktop/mpcs-python/project/autumn-2019-project-luxin-tian/data_and_files/pp2_20161010/votes.csv
cities: Chicago,Los Angeles
categories: safety,lively

This program uses the data from http://pulse.media.mit.edu/data/ to demostrate an application of the elorating module.

Dimension: safety
File length: 1223650
Reading Files |████████████████████████████████| 1223649/1223650

number of Elements: 111367
Loading Elements |████████████████████████████████| 111367/111367

number of valid records: 368926
Calculating Elo Scores |████████████████████████████████| 368926/368926


Data saved to ./logfiles/elorating_log_Fri_Dec_6_17:51:27_2019.csv


Highest: 1134.3642935436123
Lowest: 873.5102761124806
Mean: 1000.0
Stdev: 15.013932094550254
Interactive map saved to ./vis/pp2_Chicago_safety.html
Interactive map saved to ./vis/pp2_Los Angeles_safety.html

This program uses the data from http://pulse.media.mit.edu/data/ to demostrate an application of the elorating module.

Dimension: lively
File length: 1223650
Reading Files |████████████████████████████████| 1223649/1223650

number of Elements: 110890
Loading Elements |████████████████████████████████| 110890/110890

number of valid records: 267292
Calculating Elo Scores |████████████████████████████████| 267292/267292


Data saved to ./logfiles/elorating_log_Fri_Dec_6_17:52:25_2019.csv


Highest: 1129.7027310192975
Lowest: 880.1816699253212
Mean: 1000.0
Stdev: 12.060674162814488
Interactive map saved to ./vis/pp2_Chicago_lively.html
Interactive map saved to ./vis/pp2_Los Angeles_lively.html

To import in Python(in the workplace ‘/path/to/this/project’):

import pp2_app.pp2_elorating_app

pp2_app.pp2_visualization_app.to_map(filename, cityname_list, category_list)¶

Call the pp2_app.pp2_elorating_app to calculate the Elo Rating score of palces in some cities in terms of some dimensions.

Parameters

filename (str) –
the file path of the ‘vote.csv’ file downloaded from Place Pulse 2.0.
cityname_list (list(str)) – a list of city names, of which the Elo Rating scores will be calculated.
category_list (list(str)) – a list of measurement dimensions in ‘safety’, ‘lively’, ‘wealthy’, ‘beautiful’, ‘depressing’, or ‘boring’.

Export:: Interactive maps will be exported to ‘./vis/’ in .html files.

pp2_generate_all_maps reference ¶

Running this program will calculate the Elo Rating socres of the places in all the cities covered by Place Pulse 2.0 <http://pulse.media.mit.edu/data/> in terms of all the dimensions and export all the interactive maps.

To run from terminal:

>>> cd /path/to/this/project
>>> python3 pp2_app/pp2_generate_all_maps.py
(... the outputs are omitted)

Scoring Neighborhoods in Mainland China (in progress)¶

This part is still in progress. I plan to extend this project to cover mainland China in the future using street view images provided by Baidu Map. For now, I finished a program in Python that retrieves street view images from Baidu Map within the geographical areas that the user specifies. I have include this program in the repository and provide the documentation for it.

Documentation of baidu_app ¶

Installation ¶

Clone the repository of this project to your workplace.

Install the elorating package.

>>> pip install /path/to/this/project

Install package dependencies

>>> pip install -r /path/to/this/project/baidu_app/package_requirement.txt

Tutorial ¶

To retrive the static panorama street views within a specific area in mainland China, run from terminal:

>>> cd /path/to/this/project
>>> python3 baidu_app/retrieve_street_view.py
lower bound of latitude: 39.9465
upper bound of latitude: 39.9720
lower bound of longitude: 116.3281
upper bound of longitude: 116.3614
lat_resolution: 5
lon_resolution: 5
ak: ************************
>>> ls street_image
116.3281,39.9465.jpg            116.336425,39.9465.jpg          116.34475,39.9465.jpg           116.353075,39.9465.jpg          116.3614,39.9465.jpg
116.3281,39.952875.jpg          116.336425,39.952875.jpg        116.34475,39.952875.jpg         116.353075,39.952875.jpg        116.3614,39.952875.jpg
...

retrieve_street_view reference ¶

baidu_app.retrieve_street_view.main(a, b, c, d, lat_resolution, lon_resolution, ak, width=1024, height=512, fov=360)¶

Retrive the street view images within a specific area from Baiud Map.

Parameters

a (str, float) – lower bound of latitude
b (str, float) – upper bound of latitude
c (str, float) – lower bound of longitude
d (str, float) – upper bound of longitude
lat_resolution (int) – the difference between the latitudes of two coordinates.
lon_resolution (int) – the difference between the longitudes of two coordinates.
ak (str) – Baidu developer API ak token.
width (int) – the width of the street view image. (default=1024)
height (int) – the height of the street view image. (default=512)
fov (int) – the horizontol view range of the street view image. Can be integars in [10, 360]. (default=360)

Exports:: Image files will be saved to ./street_image/

baidu_api_request reference ¶

baidu_app.baidu_api_request.get_street_view(ak, location, width=1024, height=512, fov=360)¶

Request a static panorama street image from Baidu Map.

Parameters

ak (str) – Baidu developer API ak token.
location (str) – a string contains the geographical coordinates of a location ‘<longitude>,<latitude>’.
width (int) – the width of the street view image. (default=1024)
height (int) – the height of the street view image. (default=512)
fov (int) – the horizontol view range of the street view image. Can be integars in [10, 360]. (default=360)

Exports:: Image file will be saved to ./street_image/{location}.jpg

coordinate_generator reference ¶

baidu_app.coordinate_generator.coordinate_generator(a, b, c, d, lat_resolution, lon_resolution)¶

Yields the evenly distributed geographical coordinates within a square area.

For example, if lat_resolution=100 and long_resolution=150, this will generate 100*150 geographical coordinates. This generator yields discrete geographical coordinates within a square area.

Parameters

a (str, float) – lower bound of latitude
b (str, float) – upper bound of latitude
c (str, float) – lower bound of longitude
d (str, float) – upper bound of longitude
lat_resolution (int) – the difference between the latitudes of two coordinates.
lon_resolution (int) – the difference between the longitudes of two coordinates.

Returns

‘<longitude>, <latitude>’.

Return type

A geographical coordinates (str)

Scoring Neighborhoods on the Earth

An open source computational social science project based on crowdsourcing surveys to humans.

Documentation of Scoring Neighborhoods on the Earth Project ¶

Overview about this Project ¶

elorating package ¶

Installation ¶

elorating module ¶

Scoring Neighborhoods in 56 Cities (based on Place Pulse 2.0)¶

Overview ¶

Interactive maps ¶

Documentation of pp2_app ¶

Installation ¶

Tutorial ¶

pp2_elorating_app reference ¶

pp2_visualization_app reference ¶

pp2_generate_all_maps reference ¶

Scoring Neighborhoods in Mainland China (in progress)¶

Documentation of baidu_app ¶

Installation ¶

Tutorial ¶

retrieve_street_view reference ¶

baidu_api_request reference ¶

coordinate_generator reference ¶

Indices and tables ¶

Scoring Neighborhoods on the Earth

Navigation

Related Topics