Methodologically Sound or Suspect?
Ipsos, a firm focusing on global market research and public opinion, has published a survey on local views towards Section 377A of the Penal Code and views towards same-sex relationships, which it conducted from late May to early June 2022.
According to the latest Ipsos survey, 44% of respondents support the law, whereas 20% of respondents opposed the law. The survey was conducted among 500 Singaporeans and permanent residents aged 18 years old and above, between 25 May and 2 June. As reported by the Straits Times, “quotas” on age, gender and ethnicity “ensured the composition of respondents reflected Singapore’s overall population distribution”.
The survey reportedly also found that 51% of respondents agree that same-sex couples can successfully raise children, just like other parents. About 49% of respondents agree that same-sex couples should have the same rights to adopt children as heterosexual couples.
Methodologically Sound or Suspect?
Are the results of the Ipsos survey methodologically sound or suspect? Here are some of our critiques of the survey.
Opacity of the Survey Methodology
The lack of transparency of how and what questions were formulated is a curiosity.
As of the time of writing, the original report or results of the Ipsos survey have not been posted on the Ipsos website. Apart from news reports which likely reflect the information that was fed to news outlets via an Ipsos media release, the public has no way of directly reviewing the results of the survey. There is likewise, scant information on survey methodology and no information on who commissioned the survey which could give clarity as to what the objectives of the survey were.
As such, there is currently no way to assess if survey questions were formulated during survey design with an eye for neutrality, or if respondent screening reflected a bias of any sort. With the lack of such transparency, it is useful for members of the public to approach such data with eyes open.
Online, and Small Sample Size
The next critique is that Ipsos has conducted its survey online with a small sample size. The last time that Ipsos conducted a survey regarding views on Section 377A was in 2018, where it found that more than half (55%) of all respondents supported the law, while 12% indicated that they opposed it.
Both the 2018 and 2022 surveys were conducted online, which means that they would only have been able to sample the views of individuals who had access to the internet and were willing and able to spend time to complete the survey. A survey method like this would exclude those who are not proficient in English, as well as the less tech-savvy in Singapore, such as the elderly, persons with disabilities and less privileged groups.
Moreover, the previous Ipsos survey had a sample size which was one and a half times that of the recent one, with 750 Singaporean Citizens and Permanent Residents and conducted over the period from July to August 2018. The 2022 iteration only surveyed 500 respondents.
Both these Ipsos surveys pale in comparison in sample size and survey method to the survey conducted by the Institute of Policy Studies (IPS), published in 2019. The survey sampled a listing of 5,000 random household addresses obtained from the Department of Statistics, and successfully interviewed 3,000 respondents. In addition to the main sample, an additional 1,000 Indians and Malays were also surveyed to provide a booster sample. The data obtained from both these sampling methods have been combined and subsequently weighted to resemble the demographics of the national population on race and age.
For a universe as large as Singapore’s population, n=500 may be statistically representative. But relative to other government surveys which routinely involve between 1000 to 2000 respondents, the Ipsos survey does not provide as much confidence in the generalisability of the findings.
Representation: Weighting vs Quota
Next, it is important for any survey to be representative of the population being sampled. As a general rule, it should be broadly representative of various groups within the population.
One way of ensuring proper representation is by “weighting” the data after the data has been collected. For example, if there is an overrepresentation of under-35s in a survey, the results collected from under-35s will be discounted by a certain percentage and moderated to fit the proportion of under-35s in the population.
Another way is to impose “quotas”. To do so, the population is divided into separate groups in the survey design stage, and certain fixed numbers of respondents are allowed for each group. For example, a survey of n=1,000 people may impose a quota of only 500 responses from males, so that an equal number of females can be allowed to respond. Quota sampling requires availability of precise data over the whole population in order to set the quotas without bias.
This makes the 2018 and 2022 surveys somewhat difficult to compare fairly. Although the 2018 survey “weighted” the data it collected by “age, gender and ethnicity”, the 2022 survey reported imposed “quotas” according to the same categories.
Combined with the fact that the 2022 survey was conducted online with a small sample size of n=500, this may lend itself to a ‘fastest fingers first’ situation so that the first respondents of each age, gender and ethnicity would have their responses recorded while slower respondents would not be able to respond. Results may be compromised especially if interest groups rally their followers to fill up the survey.
Overrepresentation of LGBTQ-identified Persons?
Finally, there is a curious statistic according to an infographic from Ipsos published on Wake Up Singapore and Yahoo News. According to the infographic, “12% of Singaporeans” identify as LGBTQ+ (lesbian, gay, bisexual, transgender, queer, etc.). This is a strange overrepresentation of individuals who identify as LGBTQ+ reflected in the survey.
Previously, The Singapore LGBT encyclopaedia Wiki estimated, based on data from “developed countries”, that people who identify as LGBT in Singapore “would range from 3 to 5%”. (The page has since been updated to include the latest Ipsos survey results.)
A Gallup poll surveying LGBT identification in the United States, which was published in 2022, found that 7.1% of U.S. adults identified as LGBT. The Gallup survey was based on aggregated 2021 data, encompassing interviews with more than 12,000 U.S. adults.
A YouGov survey of eight western countries (Spain, United States, Great Britain, France, Germany, Sweden, Italy and Denmark), published in 2021, found that between 5% to 10% of the populations there identified as LGBTQ+.
Singapore is clearly a more conservative society compared to all of the above countries in question, apart from its laws and policies on matters like marriage and family. But even if one is prepared to ignore these factors, the number of respondents who identified as LGBTQ+ in the Ipsos survey still remains unusually large.
What then explains the over-representation of LGBTQ identification in the survey results? Were the results of the Ipsos survey skewed by interest groups or some methodological error? If so, what does this say about the rest of the conclusions of this survey?
If however, we were to grant that the results of the survey are accurate, then perhaps data showing that 12% of respondents identify as LGBTQ+ may reflect the phenomenon of self-identification because of the politics of inclusiveness, or other such social contagions such as Rapid Onset Gender Dysphoria. Either way, there is cause for concern.
Why Do the Survey Results Matter?
Surveys like the REACH and Ipsos surveys have come at a time when the Government has said that it is considering “the best way forward” on Section 377A, and is consulting with different groups.
At the same time, even as it dismissed the latest round of legal challenges against Section 377A, Singapore’s apex court has emphasised that “it is Parliament, and not the courts, that is best placed to devise a pluralistic vision that accommodates divergent interests.”
The stakes have been raised particularly because the Government has previously said that its position on Section 377A “will have to keep pace with the changes in society and how society sees these issues”. Back in 2007, Section 377A had been retained on the basis that the “majority” of Singapore society “find homosexual behaviour offensive and unacceptable”, thereby linking Section 377A with societal attitudes.
In Minister Shanmugam’s March 2022 speech, he noted that “social attitudes towards homosexuality have gradually shifted” since 2007, and said that: “Policies need to evolve to keep abreast of such changes in views. And legislation needs to evolve to support updated policies.”
A week later, the Minister added that if there are changes in the law, every Ministry will have to work through the potential impact and consequences, and ascertain what needs to be done, in line with our society’s values.
These statements from the Government expressly linking Section 377A to societal values and views make the stakes particularly high for those who are either for or against the repeal of Section 377A, including everything that it represents or symbolises to either perspective.
It is no surprise that the results of the Ipsos survey has been welcomed by Pink Dot, which expressed its “hope” that “many of the overdue changes that we wish to see will finally be implemented in earnest, and we remain ready to engage in dialogue to support our policymakers with these changes”.
Conversely, the Ipsos survey has been met with scepticism from other quarters. One user in the public Facebook group “We Are Against Pink Dot in Singapore” wrote that “if Shanmugam is basing his assessment because of this 500 suspicious participants, something fishy is going on.”
Given the questionable methodology and limited sample size, the survey results are likely not going to be very representative at all of Singaporeans’ actual views on Section 377A.
When it comes to formulating policy, closed-door dialogues with experts and stakeholders may be more useful in articulating and addressing concerns, as well as formulating policy in a comprehensive and systematic manner.
