Privacy

The Information Commissioner's Office (ICO) is set for an overhaul to drive greater innovation and growth in the UK's data sector and better protect the public from major data threats, under planned reforms announced by the Digital Secretary Oliver Dowden.

One year on from the publication of the National Data Strategy, the government has today launched a wide-ranging consultation on proposed changes to the UK's data landscape. As part of this, a new governance model is planned for the ICO, including an independent board and chief executive to mirror the governance structures of other regulators such as the Competition and Markets Authority (CMA), Financial Conduct Authority (FCA) and Ofcom.

This follows the selection of John Edwards as the government's preferred candidate as the new Information Commissioner, who is currently serving as the New Zealand Privacy Commissioner.

Now that we have left the EU, the government wants to create a pro-growth and trusted data regime that unleashes data's power across the economy and society, for the benefit of British citizens and British businesses.

The reforms outlined in this consultation will:

• Cement our position as a science superpower, simplifying data use by researchers and developers of AI and other cutting edge technologies.

• Build on the unprecedented and life-saving use of data to tackle the COVID-19 pandemic.

• Secure the UK's status as a global hub for the free and responsible flow of personal data - complementing our ambitious agenda for new trade deals and data partnerships with some of the world's fastest growing economies.

• Reinforce the responsibility of businesses to keep personal information safe, while empowering them to grow and innovate.

• Ensure that the ICO remains a world-leading regulator, enabling people to use data responsibly to achieve economic and social goals.

Encrypted-email company ProtonMail sells itself on its privacy features, promising to let uses take control of their personal data.

But it had been legally obliged to collect data from an account said to be linked to a climate activist arrested by French police.

ProtonMail's website previously claimed encrypted emails cannot be shared with third parties. And as well as offering end-to-end encryption, it did not, by default, keep any IP [Internet Protocol] logs which can be linked to your anonymous email account.

ProtonMail has now removed the claim from the front page of its website, which it said it would update to clarify its obligations in cases of criminal prosecution. Its privacy policy now says: If you are breaking Swiss law, ProtonMail can be legally compelled to log your IP address as part of a Swiss criminal investigation.

The company also publishes reports of the requests for information it receives. And last year, it received more than 3,500 requests for assistance from Swiss authorities.

The company said it stood with activists and suggested those seeking anonymity also use The Onion Router (Tor) network, which hides users IP addresses under several layers of security.

The British Government plans to sweep away inane parts of the EU's data laws in a move that would put an end to pointless web cookie consent banners and red tape.

In the first major regulatory reforms since Brexit, Culture Secretary Oliver Dowden has set out proposals which he says will turbocharge the UK's digital economy and allow data to be used more flexibly.

In particular Dowden signalled the reforms would cut down on cookie banners, used by websites to secure consent for storing personal data when using their websites, arguing that many of them were pointless. The cookie banners achieve little beyond dangerously training internet users to mindlessly tick boxes when asked, just to make the damn things go away.

Ministers also intend to shake up the Information Commissioners Office and have poached John Edwards, New Zealand's current privacy commissioner, to head up the data censor and oversee the new-look regime.

Describing the reforms as the data dividend of Brexit, Dowden said a new British framework would be more proportionate, help cut costs for businesses, and enable greater innovation, which will drive growth and opportunities and jobs.

However, the move risks opening up a fresh schism with the European Commission, which believes GDPR has been highly influential in driving up data privacy standards across the world.

Dating app Tinder has confirmed plans to introduce ID verification for its users around the world.

The firm said it would begin by making the process voluntary, except where mandated by law, and would take into account expert recommendations and what documents are most appropriate in each country.

It could come into force by the end of the year.

Critics of the idea have argued it could leave whistleblowers exposed, particularly in authoritarian states, and restrict access to online services in countries where ID documents are not commonplace among the entire population.

Tracey Breeden, vice president of safety and social advocacy at Tinder's parent firm, Match Group, said feedback from experts and users would be a vital part of its approach in helping ease such fears:

We know that in many parts of the world and within traditionally marginalised communities, people might have compelling reasons that they can't or don't want to share their real-world identity with an online platform, she said.

Creating a truly equitable solution for ID verification is a challenging but critical safety project and we are looking to our communities as well as experts to help inform our approach.

Apple's new program for scanning images sent on iMessage steps back from the company's prior support for the privacy and security of encrypted messages. The program, initially limited to the United States, narrows the understanding of end-to-end encryption to allow for client-side scanning. While Apple aims at the scourge of child exploitation and abuse, the company has created an infrastructure that is all too easy to redirect to greater surveillance and censorship. The program will undermine Apple's defense that it can't comply with the broader demands.

For years, countries around the world have asked for access to and control over encrypted messages, asking technology companies to "nerd harder" when faced with the pushback that access to messages in the clear was incompatible with strong encryption. The Apple child safety message scanning program is currently being rolled out only in the United States.

The United States has not been shy about seeking access to encrypted communications, pressuring the companies to make it easier to obtain data with warrants and to voluntarily turn over data. However, the U.S. faces serious constitutional issues if it wanted to pass a law that required warrantless screening and reporting of content. Even if conducted by a private party, a search ordered by the government is subject to the Fourth Amendment's protections. Any "warrant" issued for suspicionless mass surveillance would be an unconstitutional general warrant. As the Ninth Circuit Court of Appeals has explained , "Search warrants . . . are fundamentally offensive to the underlying principles of the Fourth Amendment when they are so bountiful and expansive in their language that they constitute a virtual, all-encompassing dragnet[.]" With this new program, Apple has failed to hold a strong policy line against U.S. laws undermining encryption, but there remains a constitutional backstop to some of the worst excesses. But U.S constitutional protection may not necessarily be replicated in every country.

Apple is a global company, with phones and computers in use all over the world, and many governments pressure that comes along with that. Apple has promised it will refuse government "demands to build and deploy government-mandated changes that degrade the privacy of users." It is good that Apple says it will not, but this is not nearly as strong a protection as saying it cannot, which could not honestly be said about any system of this type. Moreover, if it implements this change, Apple will need to not just fight for privacy, but win in legislatures and courts around the world. To keep its promise, Apple will have to resist the pressure to expand the iMessage scanning program to new countries, to scan for new types of content and to report outside parent-child relationships.

It is no surprise that authoritarian countries demand companies provide access and control to encrypted messages, often the last best hope for dissidents to organize and communicate. For example, Citizen Lab's research shows that--right now--China's unencrypted WeChat service already surveils images and files shared by users, and uses them to train censorship algorithms. "When a message is sent from one WeChat user to another, it passes through a server managed by Tencent (WeChat's parent company) that detects if the message includes blacklisted keywords before a message is sent to the recipient." As the Stanford Internet Observatory's Riana Pfefferkorn explains , this type of technology is a roadmap showing "how a client-side scanning system originally built only for CSAM [Child Sexual Abuse Material] could and would be suborned for censorship and political persecution." As Apple has found , China, with the world's biggest market, can be hard to refuse. Other countries are not shy about applying extreme pressure on companies, including arresting local employees of the tech companies.

But many times potent pressure to access encrypted data also comes from democratic countries that strive to uphold the rule of law, at least at first. If companies fail to hold the line in such countries, the changes made to undermine encryption can easily be replicated by countries with weaker democratic institutions and poor human rights records--often using similar legal language, but with different ideas about public order and state security, as well as what constitutes impermissible content, from obscenity to indecency to political speech. This is very dangerous. These countries, with poor human rights records, will nevertheless contend that they are no different. They are sovereign nations, and will see their public-order needs as equally urgent. They will contend that if Apple is providing access to any nation-state under that state's local laws, Apple must also provide access to other countries, at least, under the same terms.

'Five Eyes' Countries Will Seek to Scan Messages

For example, the Five Eyes--an alliance of the intelligence services of Canada, New Zealand, Australia, the United Kingdom, and the United States-- warned in 2018 that they will "pursue technological, enforcement, legislative or other measures to achieve lawful access solutions" if the companies didn't voluntarily provide access to encrypted messages. More recently, the Five Eyes have pivoted from terrorism to the prevention of CSAM as the justification, but the demand for unencrypted access remains the same, and the Five Eyes are unlikely to be satisfied without changes to assist terrorism and criminal investigations too.

The United Kingdom's Investigatory Powers Act, following through on the Five Eyes' threat, allows their Secretary of State to issue " technical capacity notices ," which oblige telecommunications operators to make the technical ability of "providing assistance in giving effect to an interception warrant, equipment interference warrant, or a warrant or authorisation for obtaining communications data." As the UK Parliament considered the IPA, we warned that a "company could be compelled to distribute an update in order to facilitate the execution of an equipment interference warrant, and ordered to refrain from notifying their customers."

Under the IPA, the Secretary of State must consider "the technical feasibility of complying with the notice." But the infrastructure needed to roll out Apple's proposed changes makes it harder to say that additional surveillance is not technically feasible. With Apple's new program, we worry that the UK might try to compel an update that would expand the current functionality of the iMessage scanning program, with different algorithmic targets and wider reporting. As the iMessage "communication safety" feature is entirely Apple's own invention, Apple can all too easily change its own criteria for what will be flagged for reporting. Apple may receive an order to adopt its hash matching program for iPhoto into the message pre-screening. Likewise, the criteria for which accounts will apply this scanning, and where positive hits get reported, are wholly within Apple's control.

Australia followed suit with its Assistance and Access Act, which likewise allows for requirements to provide technical assistance and capabilities, with the disturbing potential to undermine encryption. While the Act contains some safeguards, a coalition of civil society organizations, tech companies, and trade associations, including EFF and--wait for it--Apple, explained that they were insufficient.

Indeed, in Apple's own submission to the Australian government, Apple warned "the government may seek to compel providers to install or test software or equipment, facilitate access to customer equipment, turn over source code, remove forms of electronic protection, modify characteristics of a service, or substitute a service, among other things." If only Apple would remember that these very techniques could also be used in an attempt to mandate or change the scope of Apple's scanning program.

While Canada has yet to adopt an explicit requirement for plain text access, the Canadian government is actively pursuing filtering obligations for various online platforms , which raise the spectre of a more aggressive set of obligations targeting private messaging applications.

Censorship Regimes Are In Place And Ready to Go

For the Five Eyes, the ask is mostly for surveillance capabilities, but India and Indonesia are already down the slippery slope to content censorship. The Indian government's new Intermediary Guidelines and Digital Media Ethics Code (" 2021 Rules "), in effect earlier this year, directly imposes dangerous requirements for platforms to pre-screen content. Rule 4(4) compels content filtering, requiring that providers "endeavor to deploy technology-based measures," including automated tools or other mechanisms, to "proactively identify information" that has been forbidden under the Rules.

India's defense of the 2021 rules , written in response to the criticism from three UN Special Rapporteurs , was to highlight the very real dangers to children, and skips over the much broader mandate of the scanning and censorship rules. The 2021 Rules impose proactive and automatic enforcement of its content takedown provisions, requiring the proactive blocking of material previously held to be forbidden under Indian law. These laws broadly include those protecting "the sovereignty and integrity of India; security of the State; friendly relations with foreign States; public order; decency or morality." This is no hypothetical slippery slope--it's not hard to see how this language could be dangerous to freedom of expression and political dissent. Indeed, India's track record on its Unlawful Activities Prevention Act , which has reportedly been used to arrest academics, writers and poets for leading rallies and posting political messages on social media, highlight this danger.

It would be no surprise if India claimed that Apple's scanning program was a great start towards compliance, with a few more tweaks needed to address the 2021 Rules' wider mandate. Apple has promised to protest any expansion, and could argue in court, as WhatsApp and others have, that the 2021 Rules should be struck down, or that Apple does not fit the definition of a social media intermediary regulated under these 2021 Rules. But the Indian rules illustrate both the governmental desire and the legal backing for pre-screening encrypted content, and Apple's changes makes it all the easier to slip into this dystopia.

This is, unfortunately, an ever-growing trend. Indonesia, too, has adopted Ministerial Regulation MR5 to require service providers (including "instant messaging" providers) to "ensure" that their system "does not contain any prohibited [information]; and [...] does not facilitate the dissemination of prohibited [information]". MR5 defines prohibited information as anything that violates any provision of Indonesia's laws and regulations, or creates "community anxiety" or "disturbance in public order." MR5 also imposes disproportionate sanctions, including a general blocking of systems for those who fail to ensure there is no prohibited content and information in their systems. Indonesia may also see the iMessage scanning functionality as a tool for compliance with Regulation MR5, and pressure Apple to adopt a broader and more invasive version in their country.

Pressure Will Grow

The pressure to expand Apple's program to more countries and more types of content will only continue. In fall of 2020, in the European Union, a series of leaked documents from the European Commission foreshadowed an anti-encryption law to the European Parliament, perhaps this year. Fortunately, there is a backstop in the EU. Under the e-commerce directive, EU Member States are not allowed to impose a general obligation to monitor the information that users transmit or store, as stated in the Article 15 of the e-Commerce Directive (2000/31/EC). Indeed, the Court of Justice of the European Union ( CJEU) has stated explicitly that intermediaries may not be obliged to monitor their services in a general manner in order to detect and prevent illegal activity of their users. Such an obligation will be incompatible with fairness and proportionality. Despite this, in a leaked internal document published by Politico, the European Commission committed itself to an action plan for mandatory detection of CSAM by relevant online service providers (expected in December 2021) that pointed to client-side scanning as the solution, which can potentially apply to secure private messaging apps, and seizing upon the notion that it preserves the protection of end-to-end encryption.

For governmental policymakers who have been urging companies to nerd harder, wordsmithing harder is just as good. The end result of access to unencrypted communication is the goal, and if that can be achieved in a way that arguably leaves a more narrowly defined end-to-end encryption in place, all the better for them.

All it would take to widen the narrow backdoor that Apple is building is an expansion of the machine learning parameters to look for additional types of content, the adoption of the iPhoto hash matching to iMessage, or a tweak of the configuration flags to scan, not just children's, but anyone's accounts. Apple has a fully built system just waiting for external pressure to make the necessary changes. China and doubtless other countries already have hashes and content classifiers to identify messages impermissible under their laws, even if they are protected by international human rights law. The abuse cases are easy to imagine: governments that outlaw homosexuality might require a classifier to be trained to restrict apparent LGBTQ+ content, or an authoritarian regime might demand a classifier able to spot popular satirical images or protest flyers.

Now that Apple has built it, they will come. With good intentions, Apple has paved the road to mandated security weakness around the world, enabling and reinforcing the arguments that, should the intentions be good enough, scanning through your personal life and private communications is acceptable. We urge Apple to reconsider and return to the mantra Apple so memorably emblazoned on a billboard at 2019's CES conference in Las Vegas: What happens on your iPhone, stays on your iPhone

   Apple intends to install software, initially on American iPhones, to scan for child abuse imagery, raising alarm among security researchers who warn that it will open the door to surveillance of millions of people’s personal devices.

The automated system would proactively alert a team of human reviewers if it believes illegal imagery is detected, who would then contact law enforcement if the material can be verified. The scheme will initially roll out only in the US.

According to people briefed on the plans, every photo uploaded to iCloud in the US will be given a safety voucher saying whether it is suspect or not. Once a certain number of photos are marked as suspect, Apple will enable all the suspect photos to be decrypted and, if apparently illegal, passed on to the relevant authorities.

The scheme seems to be a nasty compromise with governments to allow Apple to offer encrypted communication whilst allowing state security to see what some people may be hiding.

Alec Muffett, a security researcher and privacy campaigner who formerly worked at Facebook and Deliveroo, said Apple's move was tectonic and a huge and regressive step for individual privacy. Apple are walking back privacy to enable 1984, he said.

Ross Anderson, professor of security engineering at the University of Cambridge, said:

It is an absolutely appalling idea, because it is going to lead to distributed bulk surveillance of . . . our phones and laptops.

Although the system is currently trained to spot child sex abuse, it could be adapted to scan for any other targeted imagery and text, for instance, terror beheadings or anti-government signs at protests, say researchers. Apple's precedent could also increase pressure on other tech companies to use similar techniques.

And given that the system is based on mapping images to a hash code and then comparing that has code with those from known child porn images, then surely there is a chance of a false positive when an innocent image just happens to the map to the same hash code as an illegal image. That could surely have devastating consequences with police banging on doors at dawn accompanied by the 'there's no smoke without fire' presumption of guilt that exists around the scourge of child porn. An unlucky hash may then lead to a trashed life.

Apple's official blog post inevitably frames the new snooping capability as if it was targeted only at child porn but it is clear that the capability can be extended way beyond this narrow definition. The blog post states:

Child Sexual Abuse Material (CSAM) detection

To help address this, new technology in iOS and iPadOS* will allow Apple to detect known CSAM images stored in iCloud Photos. This will enable Apple to report these instances to the National Center for Missing and Exploited Children (NCMEC). NCMEC acts as a comprehensive reporting center for CSAM and works in collaboration with law enforcement agencies across the United States.

Apple's method of detecting known CSAM is designed with user privacy in mind. Instead of scanning images in the cloud, the system performs on-device matching using a database of known CSAM image hashes provided by NCMEC and other child safety organizations. Apple further transforms this database into an unreadable set of hashes that is securely stored on users' devices.

Before an image is stored in iCloud Photos, an on-device matching process is performed for that image against the known CSAM hashes. This matching process is powered by a cryptographic technology called private set intersection, which determines if there is a match without revealing the result. The device creates a cryptographic safety voucher that encodes the match result along with additional encrypted data about the image. This voucher is uploaded to iCloud Photos along with the image.

Using another technology called threshold secret sharing, the system ensures the contents of the safety vouchers cannot be interpreted by Apple unless the iCloud Photos account crosses a threshold of known CSAM content. The threshold is set to provide an extremely high level of accuracy and ensures less than a one in one trillion chance per year of incorrectly flagging a given account.

Only when the threshold is exceeded does the cryptographic technology allow Apple to interpret the contents of the safety vouchers associated with the matching CSAM images. Apple then manually reviews each report to confirm there is a match, disables the user's account, and sends a report to NCMEC. If a user feels their account has been mistakenly flagged they can file an appeal to have their account reinstated.

This innovative new technology allows Apple to provide valuable and actionable information to NCMEC and law enforcement regarding the proliferation of known CSAM. And it does so while providing significant privacy benefits over existing techniques since Apple only learns about users' photos if they have a collection of known CSAM in their iCloud Photos account. Even in these cases, Apple only learns about images that match known CSAM.

Expanding guidance in Siri and Search

Apple is also expanding guidance in Siri and Search by providing additional resources to help children and parents stay safe online and get help with unsafe situations. For example, users who ask Siri how they can report CSAM or child exploitation will be pointed to resources for where and how to file a report.

Siri and Search are also being updated to intervene when users perform searches for queries related to CSAM. These interventions will explain to users that interest in this topic is harmful and problematic, and provide resources from partners to get help with this issue.

These updates to Siri and Search are coming later this year in an update to iOS 15, iPadOS 15, watchOS 8, and macOS Monterey.*

Update: Apples photo scanning and snooping 'misunderstood'

13th August 2021. See article from cnet.com

Apple plans to scan some photos on iPhones, iPads and Mac computers for images depicting child abuse. The move has upset privacy advocates and security researchers, who worry that the company's newest technology could be twisted into a tool for surveillance and political censorship. Apple says those concerns are misplaced and based on a misunderstanding of the technology it's developed.

In an interview published Friday by The Wall Street Journal, Apple's software head, Craig Federighi, attributed much of people's concerns to the company's poorly handled announcements of its plans. Apple won't be scanning all photos on a phone, for example, only those connected to its iCloud Photo Library syncing system.

It's really clear a lot of messages got jumbled pretty badly in terms of how things were understood, Federighi said in his interview. We wish that this would've come out a little more clearly for everyone because we feel very positive and strongly about what we're doing.

 

 

Update: Apple offers slight improvements

14th August 2021. See article from theverge.com

The idea that Apple would be snooping on your device to detect child porn and nude mages hasn't gone down well with users and privacy campaigners. The bad publicity has prompted the company to offer an olive branch.

To address the possibility for countries to expand the scope of flagged images to be detected for their own surveillance purposes, Apple says it will only detect images that exist in at least 2 country's lists. Apple says it won't rely on a single government-affiliated database -- like that of the US-based National Center for Missing and Exploited Children, or NCMEC -- to identify CSAM. Instead, it will only match pictures from at least two groups with different national affiliations. The goal is that no single government could have the power to secretly insert unrelated content for censorship purposes, since it wouldn't match hashes in any other database.

Apple has also said that it would 'resist' requests from countries to expand the definition of images of interest. However this is a worthless reassurance when all it would take is a court order for Apple to be forced into complying with any requests that the authorities make.

Apple has also states the tolerances that will be applied to prevent false positives. It is alarming that innocent images can in fact generate a hash code that matches a child porn image. And to try and prevent innocent people from being locked up, Apple will now require 30 images to nave hashes matching illegal images before the images get investigated by Apple staff. Previously Apple had declined to comment on what the tolerance value will be.

Apple has announced impending changes to its operating systems that include new protections for children features in iCloud and iMessage. If you've spent any time following the Crypto Wars, you know what this means: Apple is planning to build a backdoor into its data storage system and its messaging system.

Child exploitation is a serious problem, and Apple isn't the first tech company to bend its privacy-protective stance in an attempt to combat it. But that choice will come at a high price for overall user privacy. Apple can explain at length how its technical implementation will preserve privacy and security in its proposed backdoor, but at the end of the day, even a thoroughly documented, carefully thought-out, and narrowly-scoped backdoor is still a backdoor.

To say that we are disappointed by Apple's plans is an understatement. Apple has historically been a champion of end-to-end encryption, for all of the same reasons that EFF has articulated time and time again. Apple's compromise on end-to-end encryption may appease government agencies in the U.S. and abroad, but it is a shocking about-face for users who have relied on the company's leadership in privacy and security.

There are two main features that the company is planning to install in every Apple device. One is a scanning feature that will scan all photos as they get uploaded into iCloud Photos to see if they match a photo in the database of known child sexual abuse material (CSAM) maintained by the National Center for Missing & Exploited Children (NCMEC). The other feature scans all iMessage images sent or received by child accounts204that is, accounts designated as owned by a minor204for sexually explicit material, and if the child is young enough, notifies the parent when these images are sent or received. This feature can be turned on or off by parents.

When Apple releases these client-side scanning functionalities, users of iCloud Photos, child users of iMessage, and anyone who talks to a minor through iMessage will have to carefully consider their privacy and security priorities in light of the changes, and possibly be unable to safely use what until this development is one of the preeminent encrypted messengers.

Apple Is Opening the Door to Broader Abuse

We've said it before, and we'll say it again now: it's impossible to build a client-side scanning system that can only be used for sexually explicit images sent or received by children. As a consequence, even a well-intentioned effort to build such a system will break key promises of the messenger's encryption itself and open the door to broader abuses.

All it would take to widen the narrow backdoor that Apple is building is an expansion of the machine learning parameters to look for additional types of content, or a tweak of the configuration flags to scan, not just children's, but anyone's accounts. That's not a slippery slope; that's a fully built system just waiting for external pressure to make the slightest change. Take the example of India, where recently passed rules include dangerous requirements for platforms to identify the origins of messages and pre-screen content. New laws in Ethiopia requiring content takedowns of misinformation in 24 hours may apply to messaging services. And many other countries204often those with authoritarian governments204have passed similar laws. Apple's changes would enable such screening, takedown, and reporting in its end-to-end messaging. The abuse cases are easy to imagine: governments that outlaw homosexuality might require the classifier to be trained to restrict apparent LGBTQ+ content, or an authoritarian regime might demand the classifier be able to spot popular satirical images or protest flyers.

We've already seen this mission creep in action. One of the technologies originally built to scan and hash child sexual abuse imagery has been repurposed to create a database of terrorist content that companies can contribute to and access for the purpose of banning such content. The database, managed by the Global Internet Forum to Counter Terrorism (GIFCT), is troublingly without external oversight, despite calls from civil society. While it's therefore impossible to know whether the database has overreached, we do know that platforms regularly flag critical content as terrorism, including documentation of violence and repression, counterspeech, art, and satire.

Image Scanning on iCloud Photos: A Decrease in Privacy

Apple's plan for scanning photos that get uploaded into iCloud Photos is similar in some ways to Microsoft's PhotoDNA. The main product difference is that Apple's scanning will happen on-device. The (unauditable) database of processed CSAM images will be distributed in the operating system (OS), the processed images transformed so that users cannot see what the image is, and matching done on those transformed images using private set intersection where the device will not know whether a match has been found. This means that when the features are rolled out, a version of the NCMEC CSAM database will be uploaded onto every single iPhone. The result of the matching will be sent up to Apple, but Apple can only tell that matches were found once a sufficient number of photos have matched a preset threshold.

Once a certain number of photos are detected, the photos in question will be sent to human reviewers within Apple, who determine that the photos are in fact part of the CSAM database. If confirmed by the human reviewer, those photos will be sent to NCMEC, and the user's account disabled. Again, the bottom line here is that whatever privacy and security aspects are in the technical details, all photos uploaded to iCloud will be scanned.

Make no mistake: this is a decrease in privacy for all iCloud Photos users, not an improvement.

Currently, although Apple holds the keys to view Photos stored in iCloud Photos, it does not scan these images. Civil liberties organizations have asked the company to remove its ability to do so. But Apple is choosing the opposite approach and giving itself more knowledge of users' content.

Machine Learning and Parental Notifications in iMessage: A Shift Away From Strong Encryption

Apple's second main new feature is two kinds of notifications based on scanning photos sent or received by iMessage. To implement these notifications, Apple will be rolling out an on-device machine learning classifier designed to detect sexually explicit images. According to Apple, these features will be limited (at launch) to U.S. users under 18 who have been enrolled in a Family Account. In these new processes, if an account held by a child under 13 wishes to send an image that the on-device machine learning classifier determines is a sexually explicit image, a notification will pop up, telling the under-13 child that their parent will be notified of this content. If the under-13 child still chooses to send the content, they have to accept that the parent will be notified, and the image will be irrevocably saved to the parental controls section of their phone for the parent to view later. For users between the ages of 13 and 17, a similar warning notification will pop up, though without the parental notification.

Similarly, if the under-13 child receives an image that iMessage deems to be sexually explicit, before being allowed to view the photo, a notification will pop up that tells the under-13 child that their parent will be notified that they are receiving a sexually explicit image. Again, if the under-13 user accepts the image, the parent is notified and the image is saved to the phone. Users between 13 and 17 years old will similarly receive a warning notification, but a notification about this action will not be sent to their parent's device.

This means that if204for instance204a minor using an iPhone without these features turned on sends a photo to another minor who does have the features enabled, they do not receive a notification that iMessage considers their image to be explicit or that the recipient's parent will be notified. The recipient's parents will be informed of the content without the sender consenting to their involvement. Additionally, once sent or received, the sexually explicit image cannot be deleted from the under-13 user's device.

Whether sending or receiving such content, the under-13 user has the option to decline without the parent being notified. Nevertheless, these notifications give the sense that Apple is watching over the user's shoulder204and in the case of under-13s, that's essentially what Apple has given parents the ability to do.

It is also important to note that Apple has chosen to use the notoriously difficult-to-audit technology of machine learning classifiers to determine what constitutes a sexually explicit image. We know from years of documentation and research that machine-learning technologies, used without human oversight, have a habit of wrongfully classifying content, including supposedly sexually explicit content. When blogging platform Tumblr instituted a filter for sexual content in 2018, it famously caught all sorts of other imagery in the net, including pictures of Pomeranian puppies, selfies of fully-clothed individuals, and more. Facebook's attempts to police nudity have resulted in the removal of pictures of famous statues such as Copenhagen's Little Mermaid. These filters have a history of chilling expression, and there's plenty of reason to believe that Apple's will do the same.

Since the detection of a sexually explicit image will be using on-device machine learning to scan the contents of messages, Apple will no longer be able to honestly call iMessage end-to-end encrypted. Apple and its proponents may argue that scanning before or after a message is encrypted or decrypted keeps the end-to-end promise intact, but that would be semantic maneuvering to cover up a tectonic shift in the company's stance toward strong encryption.

Whatever Apple Calls It, It's No Longer Secure Messaging

As a reminder, a secure messaging system is a system where no one but the user and their intended recipients can read the messages or otherwise analyze their contents to infer what they are talking about. Despite messages passing through a server, an end-to-end encrypted message will not allow the server to know the contents of a message. When that same server has a channel for revealing information about the contents of a significant portion of messages, that's not end-to-end encryption. In this case, while Apple will never see the images sent or received by the user, it has still created the classifier that scans the images that would provide the notifications to the parent. Therefore, it would now be possible for Apple to add new training data to the classifier sent to users' devices or send notifications to a wider audience, easily censoring and chilling speech.

But even without such expansions, this system will give parents who do not have the best interests of their children in mind one more way to monitor and control them, limiting the internet's potential for expanding the world of those whose lives would otherwise be restricted. And because family sharing plans may be organized by abusive partners, it's not a stretch to imagine using this feature as a form of stalkerware.

People have the right to communicate privately without backdoors or censorship, including when those people are minors. Apple should make the right decision: keep these backdoors off of users' devices.

Apple has announced 2 new snooping capabilities (initial used for US users only) that will be added to its operating systems in the near future.

The first will be to analyse the content of pictures on users' devices sent in messages. Apple says that this system will only be used to inform parents when their under 12yo children attempt to send sexual content. No doubt Apple will come under pressure to scan images of all users for an ever expanding list of restrictions, eg terrorism, covid memes, copyrighted images etc.

The second scan is to match any photos being uploaded to Apple's iCloud Photo Library. If these images match a curated list of child abuse image hashes then Apple will decrypt flagged images and judge for themselves whether it is an illegal image, and then inform the police. Apple claims that they will avoid the life shattering possibility of a false positive by only investigating if several images match hashes.

Update: WhatsApp responds: A setback for people's privacy all over the world

7th August 2021. See article from dailymail.co.uk

The head of WhatsApp tweeted a barrage of criticism against Apple over plans to automatically scan iPhones and cloud storage for images of child abuse. It would see flagged owners reported to the police after a company employee has looked at their photos.

But WhatsApp head Will Cathcart said the popular messaging app would not follow Apple's strategy. His criticism adds to a stream of criticism of Apple's new system by privacy campaigners who say it is the start of an infrastructure for surveillance and censorship. Cathcart said:

I think this is the wrong approach and a setback for people's privacy all over the world.

Apple's system can scan all the private photos on your phone -- even photos you haven't shared with anyone. That's not privacy.

People have asked if we'll adopt this system for WhatsApp. The answer is no.

Instead of focusing on making it easy for people to report content that's shared with them, Apple has built software that can scan all the private photos on your phone -- even photos you haven't shared with anyone. That's not privacy.

We've had personal computers for decades and there has never been a mandate to scan the private content of all desktops, laptops or phones globally for unlawful content. It's not how technology built in free countries works..

Will this system be used in China? What content will they consider illegal there and how will we ever know? How will they manage requests from governments all around the world to add other types of content to the list for scanning? Can this scanning software running on your phone be error proof? Researchers have not been allowed to find out. Why not? How will we know how often mistakes are violating people's privacy? What will happen when spyware companies find a way to exploit this software? Recent reporting showed the cost of vulnerabilities in iOS software as is. What happens if someone figures out how to exploit this new system?, Cathcart listed as concerning questions.

There are so many problems with this approach, and it's troubling to see them act without engaging experts that have long documented their technical and broader concerns with this.