New release of the proDataMarket marketplace!

We are glad to announce second release of the proDataMarket marketplace!

What is proDataMarket Marketplace?

The proDataMarket marketplace is a virtual space that connects providers of open and proprietary real-estate and related contextual data with consumers of this data. On one hand, the marketplace aims at making it easier for data providers to publish, distribute and eventually reach out to potential consumers of their data. On the other hand, it helps data consumers discover and easily access data published at the Marketplace.

Access to the marketplace can be done through the marketplace landing page available at http://prodatamarket.eu.

proDataMarket marketplace
Landing page of the proDataMarket marketplace

 

Conceptually, there are two areas in the marketplace: Consumer site (area dedicated to data consumers in the marketplace) and Producer site (area dedicated to data producers in the marketplace).

Consumer site

Services available to data consumers have been deployed at the Consumer
Portal: https://store.prodatamarket.eu.

proDataMarket Consumer Marketplace Portal
Landing page of the proDataMarket Consumer Marketplace Portal

New look’n’feel

The data consumer services have seen further development since their initial release in the first period of the project. The Portal has been redesign following the feedback from the business case providers. New design includes landing page (see the screenshot above) with easy access to and search over the whole catalogue of data published in the marketplace.

Interactive geospatial data exploration

The Portal’s geospatial data analytics based on Amerigo Data Visualisation Service have been improved. Since its first release, map widgets have been transitioned from Leaflet to CartoDB, to support fast map rendering and provide a UI for maps configuration. New data exploration capabilities were added to the maps with data filtering widgets, that were developed on top of CartoDB. To accommodate different types of data of the business case providers, two types of filters have been implemented: discrete and continues.

data visualisation
proDataMarket marketplace data visualisation

Access to open and proprietary data

User profile management has been added to the Portal. Not-authorised users can browse through all open data available in the marketplace and samples of proprietary datasets, if their owners made them available for public. In order to get access to proprietary data, users have to sign up.

Purchase of proprietary data

Finally, authorised users can now buy proprietary data in the marketplace through the payment component, new feature of the Portal.

The purchase itself is implemented in the Purchase Management component that takes as input instructions about which data or data subsets are sold at which price. These instructions are passed to the component via a subscription configuration file. At the moment this file is prepared by the technical partners (SpazioDati, SINTEF and Ontotext) based on configuration options received from data producers (see data publisher instructions). In the future, data producers will be able to generate configuration file automatically using the  Data Pricing Setup component.

Please, note, this feature is available for proprietary paid datasets only such as “Social Network Thermometer by Municipality”, as demonstrated in the screenshot below. Open data (e.g., “State-owned buildings by municipality”) is public and free, hence, no subscription options are shown.

Subscription options
Proprietary dataset with subscription options

Producer site

Services available to data producers have been deployed at http://publish.prodatamarket.eu, the DataGraft portal. The latest release of the DataGraft portal has been announced in the recent blog post.

Current release of the marketplace includes a tutorial for data producers that describes the process of data publication from setting up a database to cleaning data and populating the database, to cataloging  data and configuring it visualisation at the at the Consumer portal. The tutorial is available at https://store.prodatamarket.eu/publisher_help/.

proDataMarket marketplace help for data producers
proDataMarket marketplace help for data producers

Marketplace Platform overview

Technical platform of the marketplace is composed of the tools, services and infrastructure developed to support two types of users: producers and consumers. Diagram below gives an overview of the marketplace and services it provides for data producers and data consumers.

Overview of the marketplace platform
Overview of the marketplace platform

GraphDB Cloud: an on-demand enterprise ready RDF database

We, from Ontotext, are excited to announce GraphDB Cloud – the easy way to introduce you to a semantic database like our signature GraphDB product. The automated tasks in GraphDB Cloud save the organizations the time and effort to install and manage hardware and software as well as the cost to buy it. Compared to a do-it-yourself database, DBaaS offers developers the opportunity to cut down the time it took them to work with their databases and spend their valuable time on creating and innovating instead of administrating.

GraphDB Cloud is one part of the Cognitive Cloud solutions for low-cost and on-demand smart data management.

The users are with the following profile:

  • Small cognitive-technology oriented team in a big organization that needs low upfront and ongoing costs for a database.
  • Start-up companies without a database infrastructure, which requires a reliable technology that scales up along with their business.
  • Corporate solution architects working to solve the challenges their enterprise faces when handling huge amounts of data and information

As a next step, we want to invite you to watch our webinar “GraphDB Cloud – Enterprise Ready RDF Database on Demand” where we introduce you to the DraphDB Cloud console and advise you how you could create custom solutions to address your company’s specific data and information needs.

ProDataMarket place as a toll for connecting real-estate data publishers and prospect data consumers

The main objective of the ProDataMarket project is to create a data marketplace for open and proprietary real-estate and related contextual data.

Marketplace is a place where data producers meet prospect data consumers. In addition to basic features for making data accessible and discoverable, marketplace can provide more tools to help data producers “advertise” their data and better engage with potential data consumers. Among such tools are those that help data producers explain the type of their data, its attributes and demonstrate its value. In this post we discuss how these tools are being realised in the ProDataMarket place.

Driving example

Let’s consider a national statistical office, for example, the Italian National Institute of Statistics (ISTAT). ISTAT wants to disseminate one of its datasets, a dataset with census cells that cover the Italian region of Piemonte. This dataset subdivides the region of Piemonte in census sections according to ISTAT’s 2011 National Census. A census section is the smallest geographic unit for which the statistical variables of a population census are taken.

ISTAT is interested in explaining to the prospect data consumers that the data can be useful when it is needed to:

  • determine inter-municipal boundaries
  • describe different areas of a city in terms of some geographically-bound characteristics

Marketplace: initial steps

Figure 1 illustrates initial steps that ISTAT performs at the marketplace to present her data.

Figure 1: The data producer prepares, describes and publishes her data at the marketplace, to make accessible and discoverable.

 

ISTAT prepares its data for publication, describes and catalogues it. Now, a prospect data consumer can discover and explore the dataset of census cells of the Piemonte region. While ISTAT made the data accessible and discoverable, data consumers still have to figure our themselves what type of data it is, what is inside and what is it useful for.

Marketplace: explaining the data types

To explain the type of the data, ISTAT creates and attaches visualisations to its data, as shown in Fig. 2.

Figure 2: The data producer creates visualisations, to explain the type of the data

 

In addition to preparing, describing and publishing Piemonte census sections dataset, ISTAT can create a map of all the census cells of the Piemonte region. This gives an illustrative example of the data to the prospect data consumers: when exploring the dataset, the data consumer can immediately see that the data contains polygons, each of which represents a geographic area of a census section.

Now that the type of the data is clearer, ISTAT can go further and explain various attributes of the data.

Marketplace: explaining attributes of the data 

Figure 3 illustrates steps that ISTAT performs at the marketplace, to give the data consumers a glimpse of the data attributes.

Figure 3: The data producer queries the data, to explain data attributes.

 

As mentioned above, the dataset of the driving example contains census cells’ geometries. Every cell is attach to a certain municipality. This information becomes useful if one wants to represent single municipalities on a map. For example, to represent the city of Turin, ISTAT can prepare a subset of the census cells by filtering on the municipality attribute of each cell. Similarly, other attributes of the data can be highlighted.

Marketplace: putting data into context to explain its value

With the help of the marketplace, ISTAT can prepare, describe and visualise as many subsets of the data, as she wants to. Finally, to showcase the value of the data and explain to the data consumer its value, ISTAT can put census cells into context, as illustrated in Fig. 4.

Figure 4: The data producer augments its data from other data sources, to show the “value in context”.

 

This last approach is realised through the Augmentation Service that supports querying a co-located data source using several functions to produce a new dataset. Currently, the Augmentation Service uses data from OpenStreetMap, to provide context. For example, ISTAT can use the service to extract the number of bus stops found nearby each census cell, or the distance to the closest train station, or the length of pedestrian paths in each census cell. Once the new augmented dataset is prepared, ISTAT can proceed with visualisations. For example, she can create a coloured map to show density of nearby bus stops in Turin.

Istat and Statistics in proDataMarket project

Istat, the Italian National Institute of Statistics is a public research organization and is the main producer of Official Statistics in Italy.  It  was created  in 1926 under the name of “Central Institute of Statistics”.  The idea was to have a single and independent body to produce social and economic data useful for citizens, policy makers and in general for the whole Country. With more than 2000 employees, one main office in Rome and 18 regional  offices,  Istat produces constantly high quality data about social and economic phenomena. Istat is also the main organization inside the National Statistical System, a national network that connects together Public Administrations, local and regional bodies, Chambers of Commerce and other local organization’s statistical departments.   Istat is also one of the members of the European Statistical Network, an international  Community lead by Eurostat that works not only to provide comparable statistics at the European level but also promoting common projects for data harmonization, data sharing and statistical data transmission.

What we do

Statistics  means “science of the state”. By means of statistics, governments all over the world measure economic and social development as well as citizens wellbeing: living conditions, health, work, education, environment, social relationships. All official statistics are produced using administrative archives, sample surveys and censuses. Census is the most important and complex survey carried out by Istat. The last population and housing census  was carried out in October 2011 involving more than 24 millions households and more than 14 million buildings.

Role in proDataMarket project

Istat not only produces statistics to support the government or local administrations for decision making processes, but also to provide high quality data for universities, researchers, professionals, journalists, businesses. The awareness about the value of data and about the opportunities that data can offer to citizens and in general to the market is the main reason for Istat to join and to support the proDataMarket project. In this project, the Istat role is not only as a data provider, but also to support the project for data quality production as well as for methodological aspects when data are used in the expected business cases. More precisely, Istat will support business cases related to real estate market. The goal is to make data more accurate and suitable to generate new services or to improve existing services built on data,  making in the same time this market more transparent and useful for citizens and businesses.