Developing a Measure of Women’s Agency in Tunisia : Machine Learning and Generative Artificial Intelligence Approaches

Overview & Synopsis

Women’s agency — their ability to define and act on their goals — is often the target of policies and programs, both as a means of reducing other gender inequities and as an end in and of itself. Measuring agency is complex, and variation in how the concept is operationalized contributes to difficulty in synthesizing evidence on the topic. At the same time, standard measures that are used in large-scale survey programs are often treated as universal and compared across widely differing settings. This paper investigates the extent to which a single scale for women’s agency can be used across contexts. The study replicates and builds on a recent study in India that used qualitative data and machine learning techniques to develop a five-question measure of women’s agency. This paper applies a variety of machine learning and, in a new contribution, generative artificial intelligence techniques and data from qualitative in-depth interviews and quantitative surveys from the same sample of women in Tunisia to identify what questions best measure overall agency. The main finding is that context matters. Overlap between the questions that performed well in India and in Tunisia is minimal. The best agency questions selected by machine learning in one context (for example, a rural area) do not perform well in other contexts (for example, an urban area). Researchers may need to generate and use context-specific agency measures to measure agency accurately. Although machine learning approaches perform well against a benchmark qualitative agency score, generative artificial intelligence approaches do not. Current, readily available generative artificial intelligence tools do not appear to have the capabilities needed to make this exercise more effective or less time-consuming.