Highlights from the OONI Run 2.0 Demonstration and Ask-Me-Anything session
On October 30, 2024, Localization Lab hosted an OONI Run Demonstration and Ask-Me-Anything event following the release of OONI Run version 2.0. The event provided a platform for the project developer to showcase the tool, highlight the changes made, discuss the app's creation, and explore its various use cases concerning internet freedom and digital rights.
Understanding OONI Run 2.0
Elizaveta Yachmeneva, OONI's community manager, shared insights into the application's functionality and the reasons behind the upgrade from the first to the second version. The presentation is available to view here.
What is OONI Run?
OONI Run is a website that facilitates coordinated testing using OONI Probe. The latest release, OONI Run 2.0, supports community efforts to detect website blocks. Users can create an OONI Run link to test a list of websites of their choice. This feature is particularly valuable for the Localization Lab community, enabling them to test websites in various countries and languages. It's also crucial for digital rights activists, researchers, and defenders, as it allows them to gather evidence of censorship.
Highlights of OONI Run 2.0
Rapid response: Enables quick testing of websites during political events or protests.
Censorship research: Allows users to create and share lists of websites for testing.
Monitor your site: Empower users to monitor the blocking of their website globally.
Updated UI: Features an updated user interface for improved user experience.
Test settings: Provides individual test settings on each card for enhanced control.
About OONI
Founded in 2012, the Open Observatory of Network Interference (OONI) is a non-profit free software project dedicated to empowering decentralized efforts in documenting internet censorship worldwide.
Key takeaways from the event:
OONI Run 2.0 is a user-friendly tool that provides real-time open data, making it a valuable asset for anyone interested in internet censorship. Its ability to facilitate website testing in different countries and languages is crucial for the Localization Lab community and other stakeholders invested in internet freedom and digital rights.
Participants gained a practical understanding of how to generate links on the OONI Run platform and utilize OONI Probe for testing.
The event fostered a discussion on best practices for using the tool, allowing participants to ask questions and provide feedback on existing features.
The session emphasized the importance of OONI Run 2.0 in supporting community efforts to detect website blocks and promote internet freedom.
Questions and Answers
We had some time to answer questions from participants during the Q&A segment. Below are some of the questions asked by community members;
Q: Have we heard any feedback/stories from communities that OONI works with that OONI Run has been a ton of help for?
A: Initially, our friends from Venezuela Inteligente gave us the idea for OONI Run. In previous years, they shared Raspberry PIs with their community of volunteers across Venezuela to run our software, OONI Probe, for measuring various forms of internet censorship. The problem though was that OONI Probe would only run at specific times per day, and it would only test certain predetermined lists of websites. As a result, when temporary blocks would suddenly emerge – involving websites that were not already included in these predetermined lists – those blocks would not get measured. This raised the need to have the flexibility to more dynamically determine which websites get tested, and to have the ability to enable a distributed community of OONI Probe users to easily and quickly test such websites.
Q: What kind of information are users such as technologists and researchers looking for when they use OONI Run?
A: Over the past 7 years, we have seen community members use OONI Run leading up to and during elections, or in response to emergent new blocks during protests or other political events. Sinar Project, for example, used OONI Run to coordinate an OONI censorship measurement campaign in Malaysia to encourage the testing of election-related websites during the country’s 2022 general election.
India’s Software Freedom Law Centre (SFLC.in) embedded OONI Run widgets into their India Internet Shutdown Tracker project website, encouraging long-term OONI Probe testing of news media, blogs, human rights, and political websites in India. Venezuela Inteligente coordinated with OONI Probe volunteers in different regions of Venezuela to measure (and respond to) temporary website blocks that emerged in correlation with political events. Human rights advocates from the global #KeepItOn campaign used OONI Run leading up to and during political events (such as elections and protests) around the world to collect censorship measurement data in support of their advocacy efforts.
Community members have also used OONI Run for research purposes. Digital Security Lab Ukraine, for example, coordinated with OONI Probe volunteers in different regions of Ukraine to examine how internet censorship differs from region to region (and network to network) in the country. OTF Information Controls research fellows, Igor Valentovitch and Ksenia Ermoshina, investigated and compared internet censorship in Russia and Crimea during Russia’s 2018 presidential election by coordinating with OONI Probe communities for the testing of their custom lists of websites.
Q: So if we want to run a test from the test list on our phones, how do we do it?
A: If you only want to measure the blocking of websites from the test list, you can do so through the following steps:
1. Tap the Websites card in the Dashboard of your OONI Probe mobile app.
2. Tap Run in the Websites card.
OONI Probe will automatically determine which country-specific list to pick for testing based on the country you’re running OONI Probe from. For example, if you run OONI Probe in Brazil, you will test websites from the global test list and the Brazilian test list. If you travel to Germany and run the OONI Probe, it will test the websites from the global and German test lists.
But if you’re running OONI Probe from a country that doesn’t have a country-specific test list yet (because it hasn’t yet been created), you will only test websites from the global list.
Q: What is the reason for having an expiration date from previous experience or learnings from the previous OONI Run version? What informed adding the expiration date?
A: OONI Run links are stored indefinitely. They are only marked as expired, but the link itself is never deleted. When a link expires it's not editable anymore, but people can still see it inside of their app until they manually remove it.
This is done to ensure their history and maintain the transparency of what has been tested. At the same time, we hope that this will limit the number of installed but outdated links that are being tested automatically, and therefore taking testing capacity from our users.
Q: How do organizations usually use the Test lists?
A: URL testing lists intended to help in testing URL censorship. They have content representing a wide range of categories at the local and regional levels, and content in local languages. In countries where Internet censorship has been reported, the local lists also include many of the sites that are alleged to have been blocked.
These test lists are machine-readable files and are mostly used by network measurement projects to detect the blocking of websites in different countries and regions.
Q: What has been the most interesting use case and/ or result you’ve heard from partners regarding the previous version of OONI Run?
A: We have some examples of the use cases collected here: https://ooni.org/support/ooni-censorship-measurement-campaigns#examples-of-ooni-censorship-measurement-campaigns
This is one of the most interesting comparative studies done with the use of the OONI Run tool: https://www.opentech.fund/news/exploring-online-media-filtering-during-2018-russian-presidential-elections/
You can learn more about the design of this study in this video: https://www.youtube.com/watch?v=nIn-r6JxdEM.
Q: What happens if I run the tests while connected to a VPN?
A: We advise against running OONI Probe over a VPN because you wouldn’t be measuring your network. You would, instead, be measuring the network provided by your VPN, which is probably based in a different country and is uncensored.
To capture internet censorship (as experienced by a local internet user), you need to turn off your VPN (or any other circumvention software) before running OONI Probe tests.
Q: Does OONI store my personal data when I run a test? My IP address, or anything?
A: OONI Probe was built with the privacy and security of its users in mind, but it’s not a privacy tool. We therefore try to limit data collection to types of data that are necessary for measuring various forms of internet censorship.
By default, OONI Probe collects:
The date and time of a measurement (such as “2019-10-30, 14:00 UTC”)
Country code (such as “IT” for Italy)
Information about your network: This includes the AS Number (for example, “AS30722” if a test was performed on Vodafone Italia) and the type of network you are on (wifi or mobile).
Network measurement data (which varies depending on the specific test performed)
Q: Does OONI Probe collect your IP address?
A: We might unintentionally collect your IP address and other personally identifiable information if this is included in the HTTP headers or other metadata of measurements. This data might be collected if the websites OONI Probe is testing employ tracking technologies or contain custom content. We take measures to remove IP addresses and other potentially personally identifiable information from our database of collected measurements to reduce the risk to you.
Learn more about our data practices through the OONI Data Policy: https://ooni.org/about/data-policy/
Q: Is there a maximum number of websites that a list can hold?
A: Test lists serve the purpose of monitoring when policies change – what’s most likely to be blocked or unblocked. While test lists may include some websites that are known to be blocked (and that is useful for detecting censorship techniques adopted by ISPs), most sites are not censored locally when they are added to test lists.
The aim of using the test list methodology is to not only identify censorship but to also confirm the accessibility of sites. Unlike blocklists (which can include thousands of URLs depending on a country), each test list is usually limited to up to 1,000 sites (due to the aforementioned bandwidth constraints).
Q: I can't run the link. Somehow the button "Install New Link" is excessively on the top and it is not possible to press it. It seems to be some sort of app visualization problem.
A: Please get in touch with our team, you can either reach out to Elizaveta on Slack or send me an email (elizaveta@openobservatory.org). This is a new tool, we launched it only 10 days ago and there may still be some bugs that we haven’t noticed while testing.
We would appreciate it if you share with us any unexpected behavior of the application, ideally with screenshots and the model of the device from which you tried to run the test.
Q: What are some best practices when running a test using an OONI Run link? Can we help come up with a list of these and share it with community members?
A: We have a user guide for those who want to create an OONI Run link and coordinate measurement campaigns: https://ooni.org/support/ooni-run/
The most important best practices to consider while creating an OONI Run link are:
Think about the number of URLs you are including in the list of URLs for testing. There is no limit to the number of URLs to include in the OONI Run link. However, the length of the list affects the time and amount of traffic it will take for the OONI Probe to run the test. Depending on your goals, context, and audience, you may want to limit the list of URLs to a few links to allow users to run tests faster and limit the amount of data they would need to spend for testing.
Check the URL formatting. To ensure that each URL is typed accurately, please copy-paste it from a browser search field. Important things to keep in mind while adding URLs to the list:
Is the website on HTTP or HTTPS? If the latter, please add an extra s after HTTP.
Does the domain include www? If so, please include it.
If the website is on HTTPS (e.g. https://www.hrw.org/), you don’t need to specify a webpage (e.g., https://www.hrw.org/publications), since ISPs usually cannot limit blocking to a specific webpage when it’s hosted on HTTPS. Rather, they would have to block access to the entire website.
While running the tests, the most important things to keep in mind are to repeat the test as many times as possible, and on as many networks as possible.
To collect hard evidence of the blocking, ideally, we need to have a stable flow of measurements before, during, and after the blocking. If we don’t have measurements before the blocking, we cannot state that it started only on the date when we received the first measurement. In the same way, if we have measurements only from one network, we cannot be confident that the blocking affects other networks – censorship can differ from one provider to another, sometimes some Internet Service Providers do not implement blocking at all.
Q: Would it be helpful to add a “why we don’t activate a VPN during a test run” there too, so the user understands the implications?
A: We advise against running OONI Probe over a VPN because you wouldn’t be measuring your network. You would, instead, be measuring the network provided by your VPN, which is probably based in a different country and is uncensored.
To capture internet censorship (as experienced by a local internet user), you need to turn off your VPN (or any other circumvention software) before running OONI Probe tests.
Q: Have you considered adding behavior to the iOS/Android apps to detect if a VPN is active on the device and then display a warning/message discouraging VPN usage with OONI? Educating the user might reduce the rate of VPNs in the results.
A: Yes, we have such behavior implemented. You can find the relevant setting in the `Settings > Advanced > Warn when VPN is in use` section of the app. By default, this setting is on, and the application shows a warning message to the users trying to run the tests while their VPN service is on.
Q: I assume running OONI with VPNs is undesirable in general, not just in the case of out-of-country VPNs since you're more interested in eyeball networks. Correct?
A: Yes, this is correct. As we want to capture internet censorship as experienced by a local internet user, ideally we would prefer any VPN service to be turned off.
Make OONI Run 2.0 accessible to your community! Join our localization effort. Contact Chido (chido@localizationlab.org) or Dianne (dianne@localizationlab.org) and help us translate OONI Run into your language.
Join the OONI community in advocating for a more open and free internet by:
Running OONI Probe: https://ooni.org/install/
Contributing to test lists: https://test-lists.ooni.org/
Analyzing OONI data: https://ooni.org/data/
Participating in community discussions: https://slack.ooni.org
Enrolling in the online course: https://advocacyassembly.org/en/courses/63/