Informacje - IBE PIB

How to analyze data from International Large-Scale Assessments (ILSA) using statistical software

This section describes available resources that support more experienced users in understanding the specifics of ILSA, effectively merging and analyzing datasets, and conducting advanced statistical analyses using different statistical software packages. We also demonstrate how to perform analyses using either open-source software (such as R) or commercial statistical software.

First, it is advisable to familiarize yourself with the available documentation, including technical reports from the studies and user manuals provided with the datasets. This will help you understand the study design, the sampling scheme, and how the sampling is reflected in the analytical weights. Pay particular attention to the use of variables relating to student achievements, as well as the rules for merging different datasets (e.g., datasets containing students’ data and teachers’ data).

Tools supporting statistical data analysis

Tools for analysing International Large-Scale Assessments (ILSA)

International Large-Scale Assessments require advanced statistical tools, which take into account complex sampling designs, replicate weights and plausible values.

The most convenient and easiest tool for beginner users is the IEA IDB Analyzer Software. Users of mainstream statistical software such as R, SPSS, Stata, or SAS can also analyze these data using additional packages or macros designed for International Large-Scale Assessments.

The table below presents a review of statistical software best suited for the analysis of data from such studies.

IEA IDB Analyzer

The IEA IDB Analyzer is a free tool developed by IEA for analyzing data from International Large-Scale Assessments (ILSA), such as those conducted by IEA, OECD, and other organizations. This software allows users to take into account the specific design of the study and perform analyses in different statistical packages, including SPSS, SAS, and R. The program requires a Windows operating system and the installation of the relevant statistical software.

The IEA IDB Analyzer features a graphical user interface that reads the data files from the user’s directory and generates syntax for merging and analyzing data in SPSS, SAS, or R. It consists of three main modules: the Merge Module, the Analysis Module, and a new module introduced in version 5.0, which enables the conversion of SPSS system files to R data files. The Merge Module generates syntax to merge files from different countries and levels (e.g., students, teachers, schools), while the Analysis Module generates syntax for analyzing these data. The generated code enables users to conduct statistical analyses, including calculating descriptive statistics, hypothesis testing, and running basic linear and logistic regression models.

More information about the IEA IDB Analyzer is available here.

Tutorial: "Analysis of ILSA data using the IEA IDB Analyzer" is available here.

Thematic analysis: "TIMSS case study analysis with the IEA IDB Analyzer" is available here.

R

R is an open-source statistical software environment offering advanced analytical tools, available through additional packages. Once loaded to the R environment, these packages extend its functionalities with new features. There are packages that, among other features, allow researchers to incorporate sampling design, analytic and replicate weights, as well as plausible values in analyses, as well as to conduct cross-country comparisons. The packages vary in functionality and complexity, allowing users to tailor the tools to their individual needs.

Recommended packages:

IEA IDB Analyzer

Creates and exports scripts for data merging and basic data analysis from studies.

Intsvy

A tool for analyzing data from PISA, TIMSS, PIRLS, ICILS and PIAAC studies – enables descriptive statistics and modeling suitable for less complex analyses.

Rrepest

A tool developed by the OECD for the analysis of ILSA data.

EdSurvey

A tool specifically designed for the analysis of ILSA data.

BIFIEsurvey

A versatile tool developed to analyze data from various studies with complex sampling designs, replicate weights, and plausible values.

SPSS

SPSS is a popular commercial statistical software widely used in social science research. It does not directly support the analysis of data containing plausible values and replicate weights. To incorporate these, additional macros or the IEA IDB Analyzer are required.

More information about the IEA IDB Analyzer is available here.

Tutorial: "Analysis of ILSA data using the IEA IDB Analyzer" is available here.

Stata

Stata is an advanced, commercial statistical software package valued for its intuitiveness and versatility in social science research analyses.

The svyset function allows users to specify sampling designs, weights, and replication methods. Stata does not directly support plausible values, which are essential for ILSA data analysis. However, they can be handled indirectly by working with imputed datasets or by installing additional packages.

Recommended packages can be installed by typing the following command in the console: ssc install package_name:

repest

Predefined study designs
Analysis of data with replicate weight
High flexibility in multilevel analyses

pv

A dedicated module for working with plausible values
Supports multiple variable sets in complex studies

pisatools

A dedicated package for analysis of PISA data
Built-in support for plausible values and replicate weights (BRR)
Enables calculation of descriptive statistics, regression and decomposition analyses

Tutorial: Analysis of ILSA data using the Stata repest package is available here.

SAS

SAS is an advanced, commercial statistical software for analyzing data in social and educational research.

The SURVEY procedures allow users to handle data from complex sampling designs. Plausible values can be incorporated using procedures for imputed data, and macros can be applied to obtain averaged results.

Mplus

Mplus is a commercial statistical software that includes psychometric functionalities and multilevel modeling features useful for ILSA data analyses. Plausible values can be incorporated using procedures for imputed data.

How to Analyze Data from International Large-Scale Assessments (ILSA)?
TUTORIALS

Do you want to analyse data from international large-scale assessments (ILSAs)? Here you will find practical information compiled for people who want to conduct such analyses independently, including those who are just becoming familiar with the specifics of these assessments. We describe the most popular software packages for analysing ILSA data and how to use them in practice.

In the tutorials, we present:

an overview of each software/package, including key functionalities as well as strengths and weaknesses;
how to install the software/package;
the international large-scale assessments whose data can be analysed using the software/package;
information on where to download data for particular assessments;
information on data structure, variable names, and merging datasets/files;
examples of specific data analyses with code (e.g., means, frequency distributions, correlations, linear regression, logistic regression, percentiles);
methods for data visualisation.

All tutorials are available in HTML, Word, and PDF formats.

Analysis of ILSA data using the IEA IDB Analyzer

The IEA IDB Analyzer is a free tool developed by the IEA for working with data from international large-scale assessments conducted by the IEA and the OECD. It does not require advanced programming skills. The software automatically generates scripts in SPSS, SAS, or R, taking into account the specific features of these studies (including the sampling design and the computations required for correct analysis of results). The tool also facilitates merging data from different files, for example combining students’ responses with information about their schools or teachers. This means that, with just a few clicks, you can prepare a dataset that allows you to examine how students’ results differ depending on selected school characteristics.

Analysis of ILSA data using the R intsvy package

The intsvy package in R is a tool for users who want to analyse data from international large-scale assessments (ILSAs) such as PISA, TIMSS, PIRLS, ICILS, or PIAAC. It streamlines work with large datasets and helps users move more quickly from raw files to results, without having to manually perform many repetitive steps. With intsvy, you can efficiently compute, for example, means, percentages, and percentiles, as well as produce simple analyses of relationships and create charts.

Analysis of ILSA data using the R Rrepest package

The Rrepest package in R was developed based on the Repest module in Stata. It was designed to facilitate the analysis of data from international large-scale assessments conducted by the OECD and the IEA and to speed up work with large datasets. The tool automatically applies key rules required for these studies (including appropriate weighting and the estimation of measurement uncertainty), making it easier to obtain correct results without performing many steps manually.

Analysis of ILSA data using the Stata repest package

Repest is a Stata package developed by OECD analysts that supports the analysis of data from international large-scale assessments by automating many of the steps required to obtain correct results. This makes it easier to work with large datasets and to move more quickly from raw files to outputs, without manually performing repetitive calculations. It is also a good solution for users who are just starting to work with international assessment data and want to base their analyses on well-established procedures.

How to Analyze Data from International Large-Scale Assessments (ILSA)?
THEMATIC ANALYSES

Do you want to prepare specific analyses using data from international large-scale assessments?

In the Thematic Analyses section, we:

provide step-by-step guidance on how to prepare and analyse data from selected assessments using different statistical software/packages;
present specific code and analysis outputs;
explain how to interpret the results;
focus on key variables commonly used in analyses (e.g., gender, country, socio-economic status, location, and students’ achievement in specific domains).

TIMSS case study analysis with the IEA IDB Analyzer

In this analysis, we present the process of preparing and analysing a TIMSS 2023 dataset using the IEA IDB Analyzer. This tool was developed for users who do not have advanced programming skills. It enables users to easily generate ready-to-run scripts in SPSS, SAS, or R syntax.

From this tutorial you will learn:

how to compare students’ achievement in mathematical reasoning and across selected countries, by gender,
how to analyse the relationship between the school location and students’ achievement in mathematical reasoning,
how to analyse the relationship between students’ socio-economic status and students’ achievement in biology.

SSES case study analysis with Rrepest

In this analysis, we present the process of preparing and analysing data from the SSES study conducted in 2019, which measures social and emotional skills. The analyses are carried out using the R package Rrepest.

From this tutorial you will learn:

how to compare results on students’ self-control across selected cities participating in the study;
how to compare students’ results by gender and age group (cohort);
how to analyse the relationship between socio-economic status and students’ level of empathy.

PISA case study analysis with intsvy

In this analysis, we present the process of preparing and analysing data from PISA 2022 using the intsvy package in R. The analysis was developed for R users who are looking for a tool that automatically supports international educational assessments and enables easy visualisation of analytical results.

From this tutorial you will learn:

how to compare students’ reading literacy achievement across selected participating countries by gender,
how to analyse differences in students’ science achievement depending on the student’s school location, as well as by student gender;
how to analyse the relationship between a student’s family socio-economic status and being absent from school for more than three months due to problem behaviour.

Analysis of ILSA data using the IEA IDB Analyzer

Analysis of ILSA data using the R intsvy package

Analysis of ILSA data using the R Rrepest package

Analysis of ILSA data using the Stata repest package

Where to find the data and research instruments from International Large-Scale Assessments (ILSA) in education?

Data from International Large-Scale Assessments in education are readily accessible to a wide range of users. They can be browsed and analyzed using interactive tools, which also allow customized reports and visualizations to be created to meet specific research needs. More advanced analyses, however, require the use of statistical software.

Unit-level data from the studies conducted by IEA and OECD are publicly available on those organizations’ websites, however they are copyright protected. Downloading the datasets typically requires reading and accepting the terms and conditions of use. Sometimes users may also need to provide contact information for themselves and their institution, and indicate the studies in which they plan to use the data. In some cases, additional forms must be completed. To support the use of these resources, online data browsers have been developed. These tools are easy to navigate and do not require advanced statistical skills.

International Large-Scale Assessments usually consist of tests, which measure participants’ skills, and questionnaires for gathering contextual information. Questionnaires for different respondent groups (e.g., students, teachers, principals) are typically published and openly available. The test instruments, however, are not publicly accessible, since the studies are conducted in cycles and rely on repeated measures to track trends over time. The only exceptions are sample items, which are published after each cycle and made available in reports and on the study organizers’ websites.

As with the data, the research instruments are the intellectual property of IEA and OECD and are copyright protected. When using these instruments – whether in their source versions or in various language adaptations – additional rules and requirements may apply.

Learn more

strona testowa

Where to find international datasets?

The OECD PIAAC website provides:

Datasets from each PIAAC cycle, containing the full set of responses from respondents;
Technical documentation, user guides, codebooks, and variable information;
Source and national versions of the questionnaires.

Available data formats: SAS, SPSS, and CSV, along with a Stata script for importing the data from the CSV file.

National items

Countries participating in International Large-Scale Assessments may – in agreement with the organizers – include a small number of questions in the questionnaire that are of particular national relevance. These items are usually placed at the end of the questionnaire. Datasets are provided below with respondents’ answers to the country-specific questions from the most recent cycle (PIAAC 2023). The files can be merged with the publicly available international databases by using the common identifier (respondent ID).

It is a simple way to enrich the analyses with information that is particularly relevant in Poland and to interpret the results more effectively. We encourage you to download the dataset and see what additional insights it can provide.

How to analyse study data?

Do you want to analyse PIAAC data? Here you will find practical information compiled for people who want to carry out such analyses independently, including those who are just becoming familiar with the specifics of international large-scale assessments (ILSA). We describe the most popular software and packages for analysing ILSA data and how to use them in practice. We also provide step-by-step guidance on how to conduct analyses—from where to download the data, through information on data structure and variable names, to examples of specific analyses with ready-to-run analysis scripts and visualisations.

Tutorials

Analysis of ILSA data using the IEA IDB Analyzer

Analysis of ILSA data using the R intsvy package

Analysis of ILSA data using the Stata repest package

Analysis of ILSA data in R Using the Rrepest Package

Useful links

Where to find international datasets?

The OECD SSES website provides:

Datasets from each SSES study cycles, containing the full set of responses from students, parents, school principals, and teachers;
Technical documentation, user guides, codebooks, and variable information;
Source and national versions of the questionnaires.

Available data formats: STATA and SPSS and for SSES 2023, also and CSV.

How to analyse study data?

Do you want to analyse SSES data? Here you will find practical information compiled for people who want to carry out such analyses independently, including those who are just becoming familiar with the specifics of international large-scale assessments (ILSA). We describe the most popular software and packages for analysing ILSA data and how to use them in practice. We also provide step-by-step guidance on how to conduct analyses—from where to download the data, through information on data structure and variable names, to examples of specific analyses with ready-to-run analysis scripts and visualisations.

Tutorials

Analysis of ILSA data using the Stata repest package

Analysis of ILSA data in R Using the Rrepest Package

Thematic Analyses

SSES case study analysis with Rrepest

Przydatne linki

Datasets and technical documentation

SSES data and methodology

Technical reports

SSES 2023

SSES 2019

Questions? Please contact us.

Study description

Study website

Where to find international datasets?

The OECD TALIS website provides:

Datasets from each TALIS cycle, containing the full set of responses from teachers and school principals;
Technical documentation, user guides, codebooks, and variable information;
Source and national versions of the questionnaires.

Available data formats: SAS, SPSS, and, depending on the study cycle, also STATA and CSV.

National items

How to analyse study data?

Do you want to analyse TALIS data? Here you will find practical information compiled for people who want to carry out such analyses independently, including those who are just becoming familiar with the specifics of international large-scale assessments (ILSA). We describe the most popular software and packages for analysing ILSA data and how to use them in practice. We also provide step-by-step guidance on how to conduct analyses—from where to download the data, through information on data structure and variable names, to examples of specific analyses with ready-to-run analysis scripts and visualisations.

Tutorials

Analysis of ILSA data using the IEA IDB Analyzer

Analysis of ILSA data using the R intsvy package

Analysis of ILSA data using the Stata repest package

Analysis of ILSA data in R Using the Rrepest Package

Useful links

Datasets and technical documentation

TALIS data and methodology

Technical reports

TALIS 2018 Technical Report

TALIS 2013 Technical Report

TALIS 2008 Technical Report

User guides for the International Database

TALIS 2018 and TALIS Starting Strong 2018 User Guide

TALIS 2013 User Guide for the International Database

Data browsers

International Data Explorer (NCES)

Website on International Large-Scale Assessments in education

ILSA Gateway

Questions? Please contact us.

Study description

Study website

Where to find international datasets?

IEA Data and Tools Repository provides:

Datasets from each ICILS cycle containing the complete set of responses from students, school principals and teachers from countries participating in the study;
Technical documentation, user guides, codebooks, and variable information;
Source versions of the questionnaires and a description of the national adaptations applied.

Available data formats: SAS and SPSS.

National items

How to analyse study data?

Do you want to analyse ICILS data? Here you will find practical information compiled for people who want to carry out such analyses independently, including those who are just becoming familiar with the specifics of international large-scale assessments (ILSA). We describe the most popular software and packages for analysing ILSA data and how to use them in practice. We also provide step-by-step guidance on how to conduct analyses—from where to download the data, through information on data structure and variable names, to examples of specific analyses with ready-to-run analysis scripts and visualisations.