NTTS is a bi-annual conference of official statistics – those institutions and people who provide governments with essential quantitative information about society and the economy. Key actors include Eurostat and National Statistical Offices of countries, sometimes statistical agencies of particular ministries or government departments, as well as international institutions that also produce data such as the OECD, and academics. Official statistics used to work primarily with surveys – asking citizens and firms to report on their habits, activities, situations. Examples are the Labour Force Survey, or the Survey on Income and Living Conditions, both conducted throughout the whole of Europe. Northern European countries initiated the tradition, now adopted by an increasing number of European countries, of supplementing surveys with administrative records: official registers of vital events such as births, deaths and marriages; tax returns filed with the fiscal administration; registers of attendance in public schools; admissions to, and discharges from, public hospitals; and so on. Administrative data have the advantage of being already available, with no need of bothering people with forms to fill; of being abundant  and sometimes even exhaustive; and of being very detailed and often much more accurate than any recollection by individuals. Traditionally, economists in particular have always being suspicious of surveys, and much more inclined to trust administrative data which suffer much less from any respondent-related cognitive or memory bias.

But Big Data are now shaking up this long-established world of surveys and administrative records. Very different, very promising, and typically produced by other actors (notably private companies rather than public-sector services), Big Data seem to question the existence and usefulness of more traditional data sources. Big Data are digital traces of our activities, collected by the electronic devices we now all use: our Internet usages, Facebook or Twitter accounts, credit card purchases, loyalty schemes with companies (like frequent flyer programmes with airlines, or customer rewards cards with supermarkets), use of public transport as recorded by our cards (such as Oyster in London or Navigo in Paris), CCTV films etc. The sheer amount of these data, the level of detail and the accuracy of the information collected are appealing: my credit card statement certainly provides a more precise account of my expenses of last month (when, where, how much, for what…), than what I may recollect even with my best effort. Companies see a lot of potential in use of Big Data (see the 2010 The Economist report on Data Deluge, or the 2011 McKinsey report, presenting Big Data as the next frontier for innovation) researchers are enthusiastic, and even public administrations now start realising this is a seachange.

How can official statistics take up this challenge? This was a key theme of the NTTS conference last week, debated in several occasions. I appreciated the effort to explore ways in which Big Data could be integrated into more traditional statistical data production techniques to improve the reliability of results and reduce respondent burden. For example, the Dutch presented their experience of using automated measures of traffic intensity and Twitter data (paper). But Big Data are not without problems, and I also sympathised with the effort of many to ouline the issues that arise with them (see for example this paper). Size does not eliminate the problem of quality: because of the very way they are collected, Big Data are unstructured and unsystematised, the sampling criteria are fuzzy, and the classical statistical analyses do not apply well. The more you zoom in (that is, the more you look at the detail), the more noise you find, so that you need to aggregate data (that is, to reduce a “big” micro-level dataset to a “smaller” macro one) to detect any meaningful tendency. This means that analysing Big Data as they are, without any caution, increases the likelihood of finding only spurious correlations. Thus, processing Big Data is problematic: although we do have sufficient computational capacity today, we still need to refine appropriate statistical analysis techniques and procedures to produce reliable results.

In a sense, the enthusiasm for Big Data is diametrically opposed to an opposed trend in social research: that of using randomized controlled trials (as in medicine) or at least quasi-experiments (like “natural experiments” in econometrics for example), which enable collection of data under controlled conditions and enable to detect causes and effects much more clearly and precisely than traditional, non-experimental, research. While these data have a lot more structure and scientific rigour than old-style surveys, Big Data are at the opposite and exhibit little to no structure at all, let alone scientific rigour.

It is good that these issues were highlighted from statisticians who have traditionally been at the very top of our societies’ data production efforts in terms of quality of data. Unfortunately, most enthusiastic accounts of Big Data leave aside these issues and simplistically assume that because the sample size is large, and because today’s computers are as powerful as ever before, data analysis will be straightforward. Big Data will certainly add to our society’s capability to retrieve and exploit information, but at the moment, they are not a miraculous solution and require further work to deliver what is now only a promise.

Involvement of statisticians is also good because of their experience in protecting confidentiality and privacy – an issue that many fans of Big Data often fail to seriously take into account. The idea of some, that because the gains in information from Big Data are so great, sacrificing our privacy is just a small price to pay, is just unacceptable – at least in the current state of things, where the potential of these data is not yet realized and people’s preferences about privacy protection are changing without fading away (see our work on the “End-of-Privacy” hypothesis). Official statisticians may have something to teach on how to manage that in a more controlled way, respectful of people’s wishes and the law.

So, what can official statistics do? The director of the Italian National Institute for Statistics ISTAT, Enrico Giovannini, has written about “Statistics 2.0” and how a higher state of play may be reached if official statistics takes up today’s challenges and thinks seriously about its role in this new, Big Data – dominated world. Two areas may be worth further work, notably production and dissemination of microdata (the best data quality possible – macro aggregates are less satisfactory for scientific and statistical analysis!) and further involvement with the open data movement.

So, a lot is changing… a process of gradual renewal, definitely worth following up.

(My presentation at NTTS 2013, on microdata access for research, is available here).

Meanwhile, the private sector also discovers a wealth of non-survey data, notably through the Internet and data mined from social networking and other online services – the so-called “big data”. Similarly to administrative data, these are traces of human activity that are recorded by some automated computer system, and do not depend on an individual’s memory, cognitive bias or psychological state. In this sense they can be more accurate than surveys, interviews and questionnaires. Even at the individual level, I can retrace my monthly expenditures more easily and precisely by looking at my credit card records online, than painstakingly trying to recall date, time, amount and place of each transaction. So, from the viewpoint of a researcher, mining large amounts of these data may give a very precise and accurate picture of a human activity, with a level of detail that surveys could never hope to achieve. Combined with the computational capacities of today, which allow handling larger amounts of data than was ever possible in the past, the social and economic sciences have a new tool in their hands that may bring a decisive improvement in their capacity to understand society and advise policy makers.

Particularly interesting is the vision of relational activities provided by big data from social networking services: relationships between people recorded as they access and use the service. When I’m on Twitter, it means that I’m talking to someone. When I’m on Facebook, it means that I have friends (or at least contacts if we don’t want to take the “friendship” metaphor too strictly). For a social networks scholar, this is great to know.

At the same time, we must be aware of what these tools actually record: it is what is allowed by the service itself, not necessarily what is perceived by users (or researchers, for that matter). For example, privacy is a notoriously fuzzy issue – few Internet users know exactly what personal information about themselves is stored by whom, for how long, and for which usages. I’m not going to mention again the scandals that have arisen repeatedly, notably around Facebook (see our project THEOP for that); but I’d like to stress here that as social scientists, we must (try to) know the conditions and limitations of data retrieval to be able to interpret it correctly, so as to enrich the analysis without distorting the interpretation and results. Big data without these “metadata” are of little use, I’m afraid.

This brings me to the point that I made in my SFR Player interview, that availability of big data does not necessarily mean that we can throw away our social science theories. In the past, the role of social and economic theory was often to compensate for lack of data. Now we no longer have this problem, but the need for theory doesn’ to go away: rather, the functions of theory change, as it now serves to interpret these data. We need good information about data quality, we need assumptions and hypotheses to test, we need ethical awareness through out the process, from data collection to data analysis and release of results and ensuing recommendations. We must be able to control the analysis of any big data within a theoretical framework that gives meaning to it.

So, big data are an opportunity, but also a challenge for the social sciences: will we be able to develop our theories to cope with these data? Will we be able to renounce it in favour of other approaches (my and Antonio Casilli‘s “small data” approach) when they help theoretical reflection more? These are new, and important, questions for the social sciences to answer in the near future.

]]> http://paolatubaro.wordpress.com/2013/03/02/again-on-data-and-big-data-in-social-research/feed/ 0 paolatubaro Data2 PaolaTubaro_videoITW http://paolatubaro.wordpress.com/2013/02/16/data-and-big-data-digital-traces-of-social-phenomena-to-nourish-research/ http://paolatubaro.wordpress.com/2013/02/16/data-and-big-data-digital-traces-of-social-phenomena-to-nourish-research/#comments Sat, 16 Feb 2013 11:13:42 +0000 paolatubaro http://paolatubaro.wordpress.com/?p=1099 ]]> I was interviewd by SFR PLAYER (an online magazine published by SFR, a major Telecom provider in France) on the changes induced by the use of big data in my work as a social science researcher. The video interview (in French) is available here. The same issue features an interview with danah boyd and various specialists of open data, data journalism, Internet data.

Interestingly, the video was shot at the École Normale Supérieure Campus Jourdan in Paris, a place that hosts part of Réseau Quetelet, the French national data service for the social sciences and humanities. The Jourdan unit ADISP handles primarily statistical data – data from surveys conducted by INSEE, the French national statistical agency, and administrative data such as those of the ministries of education and labour – for use in the social sciences.

In fact, research in social sciences has always used data as a basic ingredient. Data from official and public-sector statistics have long set the standard, and access to these data is ever more in demand today. European initiatives like the Data without Boundaries project in which I am taking part, aim precisely to bring improvements in this area.

We are now in the midst of a major upgrade today with the availability of big data, data from the Internet, the digital traces of our activities. They have the advantage that they can be retrieved, saved, coded and processed much faster, much more easily and in much larger amounts than more classical records such as registers of students in schools or of patients in hospitals. McKinsey has already pointed to potential economic benefits of big data for business, and research has taken notice too.

But big data do not automatically imply better quality of research. The sheer amount of data or even quality (completeness and richness for example) are not enough and may even raise problems. In an influential paper, danah boyd and Kate Crawford warn about potential risks, from privacy protection concerns to the narrowing down of the focus of social research and the misunderstandings that the mostly a-theoretical stance that comes with big data may bring about. I think the most interesting uses of big data are those that have a good empirical strategy: to really enhance social science research, you need an intelligent data collection design and organization of data, ideally to produce quasi-experimental conditions.

For example, I have in mind a couple of prominent studies conducted in the United States. Damon Centola conducted an experiment on a forum dedicated to health problems, looking at how changes in the structure of links on this forum, more or less apt to promote contact between participants, could favour the spread of healthier behaviors. And he could draw firm conclusions on this basis. Sinan Aral and Dylan Walker adopted a similarly strong experimental approach on a very large scale on Facebook to study peer influence – an object that has always fascinated social scientists, but that it is a challenge to identify. The possibility of extracting big data offers a plus here: it allows us to observe behaviours under controlled conditions on a large scale and in an everyday situation, unlike traditional psycho-social experiments in the lab, where people do not necessarily behave as, so to speak, in real life.

What matters most are theory and approach: data comes next. I worked on projects with Antonio Casilli in particular, where we chose to use “small data” rather than big data. The reason was that we were in situations where it was not possible to collect big data, especially when we wanted to react very quickly to British riots of August 2011 with a research paper.

When the facts took place, there was no data available online yet to do a study. This came afterwards, when Twitter made available its data to some colleagues, notably Farida Vis and her team who analysed it. But not in August. So how could we react to the events just-in-time? We did so through computer simulation and used the Internet in a different way, that is to say, to collect feedback, opinions, comments from users, activists, people who had seen the facts and could give us suggestions on how to improve our simulation. In this way, we could respond very quickly and participate in a discussion that required immediate answers, because London was burning and we felt the urge to contribute, to do our bit, as researchers.

So after all, the conclusion is not that it is research with Internet and big data that is to be conducted from now on. The era of big data reinforces a tradition of data use that had already begun long ago, and was already strong, without giving up the methods, tools and analytical solutions that have been used previously. So I see an evolution of two things in parallel – traditional and modern, small and big – with convergence and synergies that I hope, will soon materialize.

I am going to soon give another such workshop. It will take place on Tuesday, 21 May in the afternoon and on Wednesday, 22 May in the morning at the University of Hamburg, Germany, and it will be one of the many workshops preceding the 2013 Sunbelt conference (for those who aren’t familiar with it, it is the main international venue for experts of social network analysis). It will follow pretty much the same structure as at Greenwich, but based on past experience, I will shorten the theoretical introduction and dedicate more time to network metrics and measures, and their practical calculation and visualisation on the computer (with Gephi). This will make the workshop more interactive, while allowing enough time for participants to become familiar with formal concepts they may never have heard before. The other novelty is that the workshop will be taught jointly with Yasaman Sarabi, a PhD student at Greenwich who specializes in organisational network analysis. Together, we will be better able to support participants and help them with the software.

For those who have a particularly strong interest in social network analysis and cannot be content with just a one-day “taster”, I will also offer an intensive, two-week course on “Doing research with SNA“. It will be part of a Summer School organised at the University of Greenwich on 17 – 25 June 2013, just before the annual conference of the UK Social Networks Association. This course, also taught with Yasaman Sarabi, will review theories and measures, with computer applications (also using Gephi, but also UCINET and Netdraw); in addition, it will offer insight into, and hands-on experience in, research design, organising working groups in which participants set up and conduct a mini-research project, and then present their results. The objective is to help participants identify how they can integrate social network analysis into their own research, and how to reframe their questioning in order to allow for network concepts to play a role. The summer course targets PhD students and junior researchers as a priority, and (like all workshops I give) presupposes no preliminary knowledge of social network analysis, statistics, or computer programming.

More information on the Sunbelt workshop is available here.

More information on the Greenwich summer course is here.

The rationale is pretty much the same as usual. I have stressed many times how the recent rise of online social networking services (Facebook, LinkedIn, Twitter etc.) has drawn massive attention to the field of study of social network analysis (SNA). Yet social networks have always existed and are in fact a constant of human experience  – whether in the family, with friends, at school or on the workplace, to name but a few examples. Likewise, SNA already has a respectable history and has been successfully applied to study a wide variety of social contexts.

The workshop is aimed at those who are new to the field, and would like to better understand whether and how they can use it to enhance their own scholarly practice (whether it is research, teaching or consultancy). All social science backgrounds are welcome, and participants are assumed not to have any previous  knowledge of SNA (or statistics or software use, programming etc.). The goal of the workshop is to provide attendees with basic insight into what social network analysis is, and how it can be used in social science research, together with some hands-on experience of how to use network data and how to graphically represent networks, calculate key metrics, and perform some elementary analyses with Gephi, a powerful, though user-friendly, open-source software for visualizing and analyzing networks graphs.

More specifically, I will start with the fundamental principles of social network analysis and their grounding in social theories – including social science classics – moving then on to the broad range of their possible applications, with examples drawn from the literature. I will particularly insist on the substantial change of mindset that the network perspective requires with respect to standard social science approaches, due to its emphasis on relationships rather than attributes. I will also highlight uses of network-based reasoning to draw business and social policy recommendations.

I will then present network data, distinguishing type (ego and whole networks), format (edgelist, matrix), collection method (name generators and name interpreters, rosters, archives), and properties (one-mode, two-mode). I will focus on similarities and differences with respect to standard social science data, and discuss some of the opportunities and challenges arising from increasing availability of social network data from the web. I will illustrate the use of visualisation tools, showing how they can support network data interpretation, but also pointing to the limitations of graphs for analytical (rather than just descriptive) purposes.

I will then introduce basic measures of network composition and structure (density, centrality etc.), how they can be used to uncover important aspects of the social phenomenon under study, and how they can be represented graphically. I will briefly mention (but not detail) more complex statistical models of networks (ERGM, Siena).

I will use abundant examples from the literature, and will use my own research as an example, to enable participants to get a concrete sense of how SNA can be fruitfully integrated into social science research.Throughout the workshop, we will do exercises with test data sets in Gephi so that participants get a secure sense of their ability to handle data and derive conclusions from them. I will also provide references to key books, articles, software, websites and other resources for future use. There will be ample opportunities for questions and answers.

More information and registration forms are available from the Workshop website.

Here’s an excerpt:

This blog got about 11,000 views in 2012.

Click here to see the complete report.

Over three years, we have studied the structure, function and influence of the online social networks of persons with eating disorders, and potential consequences on their nutrition and health. Previous research on the pro-ana phenomenon simply looked at the contents of websites (typically only English-language ones), or sometimes at the effects of viewership on random samples of audiences (typically not actual users). We decided to move forwards and let users themselves speak; besides, for the very first time, we interrogated them not just on their health and their internet practices, but also their sociability. Indeed through a web-based survey with a special graphical application (see figure below for an example, and see this post for more detail), we reconstituted their entire personal networks of friends, family members, schoolmates, colleagues and acquaintances, both online and offline. In order not to limit the study the the English-speaking webosphere, we distributed the survey in two languages, English and French, and obtained responses from 284 persons.

We had three questions in mind when we started the study.

• First, is the Internet a refuge for people who feel uncomfortable with their social surroundings, who find it difficult to share their eating disorder experience?
• Second, are these persons trying to hurt themselves by visiting these sites -for example, are they aiming to learn (or to teach) tricks to limit their calorie intake, exercise more intensely, hide their weight loss from others?
• Third, does their use of websites and social media involve a rejection of established health norms, and perhaps of the entire health system?

The project results have brought us a few “surprises”. Well, to be sure, we as social scientists didn’t expect things to be as simply clear-cut as the media would present them. But what we found went well beyond what we could imagine at first.

To begin with, online networks are no refuge for isolated persons; most of users’ social relations are not online, but in everyday life. True, they often have smallish personal networks and do report some uneasiness about sharing their eating disorder experience with others, but there is no radical separation between the Internet and other social contexts such as the school, the workplace, or the family. Voices from the web are not the only influences on eating behaviours that users are subject to.

Concerning online interactions, we realize that our partcipants are motivated by two main needs: to obtain information and to obtain support. Information is essential for them to understand what they are facing, and how to cope with their problems. Here, Internet helps a lot: it is often through it that they could first put a name on their discomfort, realized they they were not alone in experiencing it, and learned “that one may recover, and that one may die“, as an interviewee put it. Some users go as far as exchanging and commenting scientific research articles about eating disorders.

They also look for support from peers on the web; but for their most serious problems, when their health is really at risk, they would rather activate their closest connections in everyday life (their partner, family members or close friends), rather than the members of an Internet forum. Here, the function of the Internet is not much to provide additional contacts, but to provide extra channels of communication with already known people, that facilitate and that can, if necessary, speed up interactions (for example, in case of an emergency).

Finally, we conclude that website users do not reject the health system at all -quite the opposite, they express an unmet demand for care. Although a large majority already have or had some form of care (see figure below), they would love to have more formal support for their daily life, when they are not so sick or so thin to be admitted to hospital, but still cannot regulate their food intake very well. They would love to receive guidance and information from health professionals. They would love to have easy access to care services -not involving travel and complex bureaucratic procedures for those who reside in rural areas, for example, or not involving the requirement to seek parental consent for those below 18. Demand for adequate care, we must admit, is much stronger than we initially expected.

There are, so far, two main implications of the study: first, we need to drop the “pro” from our discourse on ana-mia websites. There is no widespread attempt to prone anorexia, at least not with the radical flavour that is often phantasized in media accounts. This doesn’t mean ana-mia websites cannot be harmful, but this is often an unintentional effect, as the very fact of sharing experiences exposes users to the suffering of others, and may make recovery appear as a more distant possibility. On the other hand, strictly recovery-oriented websites can be just as harmful, as their often judgemental approach may put off some users, again making recovery less likely.

Second, what we have learned from our analysis of ana-mia websites suggests some guidelines for public-health interventions and campaigns exploiting the insight we have gained into the behavious and motivations of website users, to address their demand for additional care. Internet and social media appear as a promising tools in view of developing new solutions -so as to contribute a lot to the further improvement of the health of persons with living disorders.

The project ANAMIA has received funding from Agence National de la Recherche (ANR) for 2010-13. More details can be found here.

Two major issues appeared yesterday. One is the sustainability and durability of research projects originated as “just-in-time” ones. Rapid response papers are necessarily imperfect or incomplete, precisely because they need to be put together in such a short time; so they require integration in a longer term perspective, aiming at theoretical refinement and empirical validation. But over time, media enthusiasm may fade away, expected sources of funding may not materialize, data may be difficult or expensive to collect, and ethical committees may become more conservative in granting authorizations to projects such as these, which typically have strong political orientation (and implications).

The other issue is the availability of data. Internet firms and social networking services are understanding more and more clearly the economic value of data, and are more and more reluctant to give them away for free. This is creating increasing obstacles for research programmes that need these data as digital traces allowing to keep track of, and reconstitute, ongoing social change. There needs to be some strong political action to prevent this from happening .

The difficulties are clear, but the good thing is that the JITSO programme of research has a lot of freshness and willingness to go ahead, not least by exploring new modes of peer-reviewing and publication. Not all the methods or topics are new, of course, but the approach has finally become explicit, and as such, potentially recognised. I look forward to future editions of JITSO -though it’s not yet clear when, or where.

The slides of my keynote speech are available here.

With Antonio Casilli and Lise Mounier, two colleagues in our ANAMIA research project, we have a new peer-reviewed article on “Eliciting personal network data in web surveys through participant-generated sociograms”, which has been accepted for publication in the Field Methods journal, and is expected to come out in Vol. 26, issue 2, 2014.

We present an innovative method to collect personal network data in a web survey. Via a user-friendly flash applet, respondents can draw their own social networks of acquaintances, whether offline or online. These “ego-centered” networks display as targets whose centre represents the survey participant (ego).

We ask participants to draw around them their contacts. For example, asking them to report their online contacts, we prompt them to think of:

My online contacts… are people whom I have talked to and/or interacted with in the last six months, and whom I meet for example on discussion forums, blogs, email, MSN, social networks (Facebook, Last.fm etc.). At the centre of this target, it’s me. I shall place the others around me, the closest towards the centre and the others further away.

By clicking on a “+” button, participants can add new acquaintances (alters), drag and drop them around the target. They then indicate who they are (name, type of relationship, and gender) and to draw ties between them if they exist, or to group them together if they belong to a common social circle.

The data retrieved in this way enable us to calculate classical network metrics (size, density, tie strength etc.) and, if more than one target has been filled, to compare different networks: for example, establish similarities and differences between the online and offline personal networks of participants.

We have used this method to conduct a survey of users of websites, blogs and forums dedicated to eating disorders, in France and the UK.

A preliminary version of the article is available here. More information on the research project ANAMIA, for which the tool has been conceived, is available here.