New release of the proDataMarket marketplace!

We are glad to announce second release of the proDataMarket marketplace!

What is proDataMarket Marketplace?

The proDataMarket marketplace is a virtual space that connects providers of open and proprietary real-estate and related contextual data with consumers of this data. On one hand, the marketplace aims at making it easier for data providers to publish, distribute and eventually reach out to potential consumers of their data. On the other hand, it helps data consumers discover and easily access data published at the Marketplace.

Access to the marketplace can be done through the marketplace landing page available at http://prodatamarket.eu.

proDataMarket marketplace
Landing page of the proDataMarket marketplace

 

Conceptually, there are two areas in the marketplace: Consumer site (area dedicated to data consumers in the marketplace) and Producer site (area dedicated to data producers in the marketplace).

Consumer site

Services available to data consumers have been deployed at the Consumer
Portal: https://store.prodatamarket.eu.

proDataMarket Consumer Marketplace Portal
Landing page of the proDataMarket Consumer Marketplace Portal

New look’n’feel

The data consumer services have seen further development since their initial release in the first period of the project. The Portal has been redesign following the feedback from the business case providers. New design includes landing page (see the screenshot above) with easy access to and search over the whole catalogue of data published in the marketplace.

Interactive geospatial data exploration

The Portal’s geospatial data analytics based on Amerigo Data Visualisation Service have been improved. Since its first release, map widgets have been transitioned from Leaflet to CartoDB, to support fast map rendering and provide a UI for maps configuration. New data exploration capabilities were added to the maps with data filtering widgets, that were developed on top of CartoDB. To accommodate different types of data of the business case providers, two types of filters have been implemented: discrete and continues.

data visualisation
proDataMarket marketplace data visualisation

Access to open and proprietary data

User profile management has been added to the Portal. Not-authorised users can browse through all open data available in the marketplace and samples of proprietary datasets, if their owners made them available for public. In order to get access to proprietary data, users have to sign up.

Purchase of proprietary data

Finally, authorised users can now buy proprietary data in the marketplace through the payment component, new feature of the Portal.

The purchase itself is implemented in the Purchase Management component that takes as input instructions about which data or data subsets are sold at which price. These instructions are passed to the component via a subscription configuration file. At the moment this file is prepared by the technical partners (SpazioDati, SINTEF and Ontotext) based on configuration options received from data producers (see data publisher instructions). In the future, data producers will be able to generate configuration file automatically using the  Data Pricing Setup component.

Please, note, this feature is available for proprietary paid datasets only such as “Social Network Thermometer by Municipality”, as demonstrated in the screenshot below. Open data (e.g., “State-owned buildings by municipality”) is public and free, hence, no subscription options are shown.

Subscription options
Proprietary dataset with subscription options

Producer site

Services available to data producers have been deployed at http://publish.prodatamarket.eu, the DataGraft portal. The latest release of the DataGraft portal has been announced in the recent blog post.

Current release of the marketplace includes a tutorial for data producers that describes the process of data publication from setting up a database to cleaning data and populating the database, to cataloging  data and configuring it visualisation at the at the Consumer portal. The tutorial is available at https://store.prodatamarket.eu/publisher_help/.

proDataMarket marketplace help for data producers
proDataMarket marketplace help for data producers

Marketplace Platform overview

Technical platform of the marketplace is composed of the tools, services and infrastructure developed to support two types of users: producers and consumers. Diagram below gives an overview of the marketplace and services it provides for data producers and data consumers.

Overview of the marketplace platform
Overview of the marketplace platform

ProDataMarket place as a toll for connecting real-estate data publishers and prospect data consumers

The main objective of the ProDataMarket project is to create a data marketplace for open and proprietary real-estate and related contextual data.

Marketplace is a place where data producers meet prospect data consumers. In addition to basic features for making data accessible and discoverable, marketplace can provide more tools to help data producers “advertise” their data and better engage with potential data consumers. Among such tools are those that help data producers explain the type of their data, its attributes and demonstrate its value. In this post we discuss how these tools are being realised in the ProDataMarket place.

Driving example

Let’s consider a national statistical office, for example, the Italian National Institute of Statistics (ISTAT). ISTAT wants to disseminate one of its datasets, a dataset with census cells that cover the Italian region of Piemonte. This dataset subdivides the region of Piemonte in census sections according to ISTAT’s 2011 National Census. A census section is the smallest geographic unit for which the statistical variables of a population census are taken.

ISTAT is interested in explaining to the prospect data consumers that the data can be useful when it is needed to:

  • determine inter-municipal boundaries
  • describe different areas of a city in terms of some geographically-bound characteristics

Marketplace: initial steps

Figure 1 illustrates initial steps that ISTAT performs at the marketplace to present her data.

Figure 1: The data producer prepares, describes and publishes her data at the marketplace, to make accessible and discoverable.

 

ISTAT prepares its data for publication, describes and catalogues it. Now, a prospect data consumer can discover and explore the dataset of census cells of the Piemonte region. While ISTAT made the data accessible and discoverable, data consumers still have to figure our themselves what type of data it is, what is inside and what is it useful for.

Marketplace: explaining the data types

To explain the type of the data, ISTAT creates and attaches visualisations to its data, as shown in Fig. 2.

Figure 2: The data producer creates visualisations, to explain the type of the data

 

In addition to preparing, describing and publishing Piemonte census sections dataset, ISTAT can create a map of all the census cells of the Piemonte region. This gives an illustrative example of the data to the prospect data consumers: when exploring the dataset, the data consumer can immediately see that the data contains polygons, each of which represents a geographic area of a census section.

Now that the type of the data is clearer, ISTAT can go further and explain various attributes of the data.

Marketplace: explaining attributes of the data 

Figure 3 illustrates steps that ISTAT performs at the marketplace, to give the data consumers a glimpse of the data attributes.

Figure 3: The data producer queries the data, to explain data attributes.

 

As mentioned above, the dataset of the driving example contains census cells’ geometries. Every cell is attach to a certain municipality. This information becomes useful if one wants to represent single municipalities on a map. For example, to represent the city of Turin, ISTAT can prepare a subset of the census cells by filtering on the municipality attribute of each cell. Similarly, other attributes of the data can be highlighted.

Marketplace: putting data into context to explain its value

With the help of the marketplace, ISTAT can prepare, describe and visualise as many subsets of the data, as she wants to. Finally, to showcase the value of the data and explain to the data consumer its value, ISTAT can put census cells into context, as illustrated in Fig. 4.

Figure 4: The data producer augments its data from other data sources, to show the “value in context”.

 

This last approach is realised through the Augmentation Service that supports querying a co-located data source using several functions to produce a new dataset. Currently, the Augmentation Service uses data from OpenStreetMap, to provide context. For example, ISTAT can use the service to extract the number of bus stops found nearby each census cell, or the distance to the closest train station, or the length of pedestrian paths in each census cell. Once the new augmented dataset is prepared, ISTAT can proceed with visualisations. For example, she can create a coloured map to show density of nearby bus stops in Turin.