2.1 User’s home residence

Studies of [13, 23] were amongst the early works of user home location prediction. After the deployment of Twitter in 2006, in 2010 [13] proposed an approach that uses language models on tweet text to predict a user’s home location. In the proposed approach, the authors identified local keywords within the tweet and used the identified keywords to predict the user’s home city. They obtained an accuracy of 51% within a 100-mile radius (161 km) of the actual user’s profiled home location. In 2011, [23] inferred user home location at a city level using location indicative words within the tweet content. A similar technique was applied in [24] where the authors used language models and machine learning algorithms to infer a user’s home country. By evaluating present tweet nouns and google word trends for a set of tweets from each user, the authors were able to geolocate 83% of the users within their home country. A more advanced approach was conducted by [12]. In their research, the authors used all publicly available tweet metadata fields to predict a users’ residence at varying levels of granularity. The authors obtained accuracy levels of 61%, 70%, and 80% for city, state and time zone, respectively.

In [25], user social graphs alongside the tweets content were used to infer the user home locations. Their highest F-measure reported from their study was 37.71%. Again using social graphs, [26], analysed user home locations and managed to infer about 80% users within the home city of Dublin. Recently, [11], used the time of opening of a twitter account together with the tweet language to infer the users home location. The authors we able to geolocate 71% of tweets within a country level and 51% within a city level. Although the precision values were measured on a large scale (state, country, time zone), these studies showed the possibilities of inferring a user’s residence location using various metadata fields and approaches.

2.2 Tweet message location

The message location refers to the location that a user makes reference to. The message location may not necessarily reflect the user’s in situ location. In instances such as traffic monitoring or sentiment analysis, researchers are more interested in topics related to specific locations and the user’s actual location is not essential. Earlier studies such as those of [27–29] focused on geolocating message locations so as to detect event specific tweets. In 2011, [27] used syntactical information to classify traffic information. Whilst using prepositions the authors were able to determine the start and end location of traffic incidents with an average accuracy of over 90%. In the same year [28] geotagged the points of interests using language and time models whilst [29] inferred message locations from disaster related tweets using a named entity recognition and geocoding. These studies already showed the high possibility of inferring location mentions within tweets.

More recently, [14] used tweet context to identify and classify tweets related to a flooding event. The authors used historically geotagged tweets along with inferred user locations to geolocate the radius of high priority flood victims. Their proposed approach returned a classification accuracy of 81% and a location prediction accuracy of 87%. By utilizing location information contained within a tweet message, [30] analysed user subjectivity and polarity during the 2016 USA elections. They found that extracting inferred location variables significantly increases the accuracy of sentiment and behavioural analysis. In [31], the authors created a hybrid approach that uses Twitter content and user network information to geocode a message location. Their designed method was able to pinpoint the location of event related tweets within a median error of 19 km, 50% of the time. To increase the knowledge of traffic flows, [15], developed a framework to extract the locations of traffic events from tweet text in Greater Mumbai. Their results showed that Twitter users in Greater Mumbai often share traffic information along with location mentions. Assuming that this is the case for other emergencies on a global scale, the need for inferring locations from Twitter data becomes evident.

2.3 Tweet’s point of origin

In addition to the above discussed location inference methods, the studies of [8, 9, 17, 32–34] predicted users’ locations at the time of sending a post (tweet’s point of origin). In 2011, [32] used language models to predict a tweet’s point of origin from a zip code to Country level. Similarly, [33] used rule generation and identified locations through noun and verb combinations. With this approach, the authors were able to geolocate 79% of the tweets within a country level and 66% within the users’ current location.

Using present associations between a location and its relevant keywords, [17] analysed the textual content of a tweet to predict the tweet’s point of origin. Using this approach, the authors managed to geolocate 45% of tweets within a radius of 10 km to the tweets’ Global Navigation Satellite System (GNSS) position. Additionally, [34] implemented a city level geolocation system to infer the uses’ location based on tweet text and user-declared metadata fields using a staking approach. Their approach was able to geolocate 66.5% within a 100 mile (161 km) radius.

Similarly [9] predicted the tweets’ point of origin by using a multi-elemental location inference method. They conducted their study on 2,409 flood related English posts generated between Sydney, Australia and surrounding areas. Their approach exploited three location-related elements in the form of the tweet’s textual content, profile location and the place labelling. They assigned the finest granular location of the three elements as the inferred tweet location and checked the accuracy by comparing with geotagged tweet coordinates. Their approach resulted in an accuracy of 60% and 83% of tweets within 10 km and 50 km radius, respectively.

There is no doubt that past studies have done a substantial amount of work in improving location inference models. In this paper, we adopt most of the already existing approaches and construct a methodological structure to enhance the precision of content based location inference models.

2.4 Incorporating non-geotagged datasets

The primary aim of location inference models is to identify locations in datasets which are not geotagged. A number of studies have incorporated non-geotagged datasets in order to evaluate the efficacy of these models. The research of [35] employed a multi indicator method to investigate the tweets’ point of origin. In their method the authors used 1.03 million worldwide geotagged posts and reported an inference accuracy of 22%, 37% and 54% on 1 km, 10km and 50km radius values, respectively. The authors acknowledged the possible bias in using only a geotagged dataset and evaluated their approach on a random selection of 10,000 tweets containing both geotagged and non-geotagged posts. Although no quality assessment could be performed, the authors found similarities in spatial indicators between the two datasets (geotagged and non-geotagged) which suggested that their proposed approach would also perform well on a non-geotagged dataset. Whilst their evaluation was done on a small dataset, their findings suggest possible relatedness between location mentions in a geotagged and a non-geotagged dataset.

While using the user’s previously geotagged posts, [17] developed a method that identifies a user’s location on a non-geotagged dataset. Their method identifies user locations from previously geotagged posts and learns the associations between each location and its relevant keywords to predict a tweet’s point of origin on a non-geotagged dataset. Using this approach, the authors managed to geolocate 45% of tweets within a radius 10 km to the tweets’ GNSS position. Their results show the possibility of inferring user locations from a non-geotagged dataset by associating locations with keywords. However, their approach was limited to inferring locations sent from location related services specifically Foursquare and Japans’ Loctouch.

Using a multi-source and multi-model based inference framework, [8] proposed a systematic approach to infer the tweets’ origin on a non-geocoded dataset by using Foursquare check in data for Manhattan, New York City, USA. From the Twitter dataset, the authors used 72,601 non-geotagged tweets generated from Facebook and Instagram sources. By geocoding only the extracted locations that coincided with Foursquare points of interests, they geocoded 34% and 44.28% of all tweets within a 250-meter and a 1 km radius, respectively, of a tweet’s attached GNSS coordinate. Although their approach returned high accuracy, their results are limited to Foursquare points of interest and do not represent other locations.

Whilst these methods have produced comparative performances, the methods have been limited to either a representation of a geotagged dataset or a selection of tweets from location related services such as Foursquare. Given that the ultimate goal of developing location inference methods is to geotag a non-geotagged dataset, we investigate how differences in source distribution would influence the performances of location inference methods.

3.1 spaCy

spaCy is a free, open-source library for advanced Natural Language Processing (NLP) in Python [36, 37]. spaCy supports over 64 language models trained on either newspaper articles, media, blogs, comments etc. spaCy’s pretrained English model has four pretrained models; small (sm), medium (md), large (lg) and transformer (trf). The models are distinguished by the size of the training dataset with training data size increasing from the sm to the trf model. For this study, we used spaCy version 3.0 trf pretrained high accuracy English model formally abbreviated as en_core_web_trf. This model was trained on 30K web pages, making it a decent fit when applied to a non-structured dataset such as tweets.

The Named Entity Recognition (NER) tool within spaCy extracts entities by entity prediction. This means that, instead of matching place names against a gazetteer, the model uses the text’s syntax to predict the probability of a word being a named entity. For example, in the sentence “I have to Google why Google has a lot of employees”, the model would correctly recognize the first Google as a verb and the latter as an organisation entity, despite being constructed in a similar manner. Because of the models’ reliance on sentence syntax for entity extraction [37], it is able to extract entities regardless of spelling errors or presence of noisy elements within the entity. In our previous study [16] we showed that the syntax reliance of spaCy gives it a much higher recall as compared to the gazetteer based DBpedia tool. However, factors such as incorrect punctuation or syntax errors affects the model’s performance.

3.2 Dataset

For ground truthing, we restricted our analysis to tweets with an attached GNSS position (geotagged tweets). Our dataset contained 16,347,823 geotagged tweets sent between August 2019 and April 2020. The tweets had a GNSS position falling within a bounding box that enclosed the United States of America excluding the states of Hawaii, Alaska and USA island states. The tweets were sent by 492,874 unique users and generated from 739 Twitter sources.

3.3 Sample selection

We selected eight tweet sources for the analysis. Our selection was based on the popularity of the tweet source field amongst users. That is, by counting the number of distinct users per Twitter source, we eliminated all twitter sources with less than 1,000 distinct users.

The ‘Total tweets’ column in Table 1 shows the number of tweets within our dataset per tweet source. Overall, tweets generated from third party applications (Instagram, Foursquare, Untapped) represented a larger percentage (>97%) of tweets in our dataset. This is in line with the findings of [10, 18, 19, 38], who found larger percentages of geotagged tweets to be generated from third-party applications (Instagram, Foursquare) than from native Twitter applications (Twitter for Android, Twitter for Windows). Dataset A consists of a 1% sample selection of each tweet source. Dataset B consists of a strategically selected dataset that models the tweet source distribution of a non-geotagged dataset following the findings of [10].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Number of geotagged tweets per tweet source for the full dataset (Total tweets) and the two selected datasets, Dataset A and Dataset B.

The table is ordered by descending order of the Total tweets column.

https://doi.org/10.1371/journal.pone.0282942.t001

In Fig 2 we show the geographical distribution of the two sample datasets. Overall, our two datasets were similarly distributed across the study area which suggests similar usage of tweet sources across the USA.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2.

Distribution of posts within Dataset A (left) and Dataset B (right). Visually, the posts have a comparatively similar distribution across the study area.

https://doi.org/10.1371/journal.pone.0282942.g002

3.4 Preprocessing

3.5 Location extraction

A distinctive feature of the adopted spaCy pre-trained model is its split location entities. spaCy has location entities split into four classes. (1) Geopolitical Entities (GPE), (2) Facilities (FAC), (3) Organisations (ORG) and, (4) Locations (LOC). GPE entities represent administrative units such as countries, states and cities. FAC contains buildings, airports, highways and bridges. ORG includes companies, agencies and institutions. LOC defines street names, mountains, lakes and water bodies [36]. We extracted these four location entities separately.

Since spaCy extracts entities based on entity predictions [37], it is able to distinguish geographical from non-geographical entities, for instance, disambiguating the word ‘Turkey’ as either, a country, animal, food or a person’s name based on sentence syntax. However, since most geocoding services do not take into account a tweet’s context, geographical location ambiguities remain a challenge [42–44]. For example, Georgia could refer to Georgia a country in Europe or Georgia a state in the USA.

To reduce ambiguity when geocoding location entities, additional location information has to be attached to each entity. An ORG entity for instance “McDonald’s” has to be combined with a GPE, FAC and or a LOC entity to the highest level of detail to achieve a “high probability” of being correctly geocoded. We use the term “high probability” to clarify the mere fact that more than one location entity, in this case “McDonald’s”, may as well be present in several cities, neighbourhoods or streets with similar geopolitical names.

To ensure high probability in the geocoded locations, we paired combinations of location entities within each post. The single location entities together with the combined location entities (Dual, Triple and Quad) resulted in fifteen location entity classes shown in Table 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Location entity classes with corresponding examples.

The examples are given as extracted within the tweet text hence the presence of spelling errors and incorrect punctuation in some of the examples. The table is sorted by alphabetical order in each category.

https://doi.org/10.1371/journal.pone.0282942.t002

3.6 Geocoding

Geocoding is a process of transforming human readable locations into latitude and longitude value pairs. We geocoded the extracted location entities using Nominatim and Google Maps geocoding API (Google Maps). Although several geocoders exist such as Esri Geocoding, GeoNames, Bing, HERE Maps, TomTom, etc. we restricted our analysis to Nominatim and Google Maps geocoders due to their worldwide popularity.

Nominatim uses crowdsourced data from OpenStreetMap (OSM) to match exact location names and return the associated coordinates for the input string [3]. Due to its crowdsourced nature, Nominatim contains place name aliases that allow for geocoding of often-used informal place names in tweets. However, a drawback of Nominatim is that it does not handle spelling errors well [15, 45]. For short and unstructured posts where spelling errors, noisy elements, and shorthand syntax are common, such as is the case with tweets, Nominatim fails to geocode a significant number of locations.

Contrasting to Nominatim, which returns locations based on exact name matches, Google Maps performs location cleaning and fuzzy matches on an input string and geocodes the resulting formatted address (see.Table 3). The advantage of this approach is that a higher number of locations are geocoded as spelling discrepancies and noisy elements are filtered out. However, the disadvantage is that some locations may be incorrectly formatted, which results in incorrect coordinates.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Examples of Google Maps geocoding API formatted and geocoded address from provided input string.

https://doi.org/10.1371/journal.pone.0282942.t003

In the example input strings in Table 3, “arizona, glendale, glendalearizona” is correctly formatted to “W Glendale Ave, Glendale, AZ 85301, USA” despite the errors present in the input string. On the contrary ‘chicagos’ was wrongly formatted and geocoded to an address in Georgia (GA), USA. Because each geocoder has its own pros and cons, we used both Nominatim and Google Maps API to geocode the fifteen categories of location entities defined in Table 2.

3.7 Displacement computations

As a validation, we computed the geodesic displacement between the inferred coordinates and the tweets’ attached GNSS coordinate. For comparison with measurements from related studies [8, 9, 17, 35], we computed displacement values within 1 km, 10 km and 50 km radius values.

3.8 Generating statistical results

The displacement computations returned individual precision values for each location entity and displacement class. To compare performance of each dataset, we grouped location entities per category. We generated four mutually exclusive groups by combing corresponding location entities in each category. Our groups are defined as follows,

1. Group 1 = Quad location entities
2. Group 2 = Triple location entities
3. Group 3 = Dual location entities
4. Group 4 = Single location entities

Based on our justification of combining location entities (s. 3.5), we expect to obtain an inverse relationship between the precision and recall values, with Group 1 having a high precision and low recall and Group 4 having a low precision and high recall. We define our precision and recall evaluation matrix as follows:

5.1 Result discussion

We found that the proportion of tweets from different sources impacts the performance of the developed location inference model. Specifically, we found that we obtain higher model performance values when we design our location inference model on a dataset with a higher percentage of tweets emanating from third party applications compared to the native Twitter application.

Our Dataset A, which contained a proportionate sample of a geocoded dataset, returned overall higher precision and recall values compared to Dataset B, which contained a higher percentage of posts generated in native Twitter applications. Assuming similarities in text content between the tweet sources of geotagged and non-geotagged, this result implies that location inference models produce higher performance values than what would be obtained in a non-geotagged dataset. However, this assumption needs to be checked before location inference models are developed and adopted to a non-geotagged dataset.

Apart from data selection, we were able to confirm the well-known fact that the choice of the geocoder has an influence on the model performance. The model performance was overall higher for Google Maps geocoded locations as compared to Nominatim. The main drawback that we observed for Nominatim is that it struggles to resolve place names that have some mismatches within the OSM database which was evidenced by the low recall for combined location entities (s. Table 5). This observation goes in line with the findings of [15, 16, 45] who found the OSM Nominatim API to be unable to geocode a substantial percentage of location names.

Owing to Nominatim’s struggle in resolving place names that did not have matches in the OSM database, the geocoder outperformed Google Maps precision values for combined location entities (s. Table 7). This finding can be attributed to Google Maps’ tendency to format location mentions before geocoding which leads to an overall higher recall though at the cost of precision. What this result implies, is that in cases where precision is of more important than recall, it would be best to use Nominatim combined location entities than the Google Maps geocoder.

Albeit the differences in geocoder performances, most importantly, we found that our enhanced location inference model outweighs similar studies which have used sample tweets extracted from a geotagged dataset [8, 9, 17, 35]. In Table 8 we compare precision values returned from related studies with the precision values that resulted from our model combination with the highest precision, and model with a trade-off between precision and recall from Dataset A and Dataset B. With an exception of [9] who outweighed our precision values for Dataset B at a 50 km radius (83% vs 81%), our location inference approach outweighed the precision values for existing literature sources for both Dataset A (higher percentage of tweets generated from third party Twitter applications) and Dataset B (higher percentage of tweets generated from native Twitter applications).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 8. Summary comparison of precision values between existing approaches which inferred a tweet’s point of origin and our developed approaches.

The table is arranged by ascending order of precision value within each error distance.

https://doi.org/10.1371/journal.pone.0282942.t008

5.2 Methods discussion

Considerably the most impactful decision for our analysis was the choice of the spaCy location extraction model. Using spaCy gave us high control over the extracted entities allowing us to test and combine precision values of various location combinations. The ability of spaCy to split location entities presents higher advantage over location extraction models with a single classification for location entities such as DBpedia, Geoparser.io, Stanford NLP, etc.

By combining geocoded tweets with a similar combination count of location entities (single location entities, dual location entities etc.), we produced four statistical groups from the 15 location entities. Whilst these four groups were sufficient for comparisons, a higher number of groups could have been formed by trying out various combinations of location entities. Whilst we were able to obtain high precision values from the four combinations, we acknowledge that prioritizing location entities based on precision values would have returned higher precision values.

Although our proposed method returned higher precision values compared to other studies, the lack of robustness in filtering out remote location mentions, impacted the performance of the location model. For instance, the post ‘Happy birthday to my brother in Sydney’ contained neither keywords nor temporal information that could be used to filter out the remote location mention. For future work, we suggest to use filtering approaches that take into account the grammatical structure of the sentence in order to improve precision values.

The use of only tweets generated within a USA bounding box limits the generalizability of our location inference model on world-wide dataset. Since the USA has a high density of mapped locations, it is possible that our high precision values were a result of bias towards the more popular USA cities. Since non-geocoded datasets are not constrained to specific countries or continents, there is a need to test the performance on our location inference on model on a world-wide dataset.

Another impacting factor that is worth mentioning is that users may also share imprecise or contradicting information about their location when sending the post. Since we compared the inferred locations with the tweets’ GNSS position, the contradicting location mentions may have negatively affected the precision of the results. Contradicting location mentions are quite common in location sharing services where the user has the choice to tag a location of their choice such as Instagram.