kouider

HTML5 · CSS3 · JavaScript · React · Angular · PHP · Laravel · Node.js · Docker · Python · WordPress    
← All posts

Your Notes Went to a Company You Never Heard Of. Then It Was Hacked.

Rows of grey server racks in a vacant office, with cables hanging loose after the company moved out

You asked the AI inside your notes app to tidy up a page. That question travelled to a company you have never heard of — and in October, someone broke into its computers.

What actually happened

Notion is an app people use for notes, documents and to-do lists, at work, at university or at home. Some of its features run on artificial intelligence: you ask for a summary of a long page and it writes one.

Notion does not own the machines that do that work. It rents them from a company called Fireworks AI, which sells computing time for AI programs the way a catering kitchen rents out its ovens. When you use the feature, your words travel to Fireworks and an answer comes back.

In October, an intruder got into a cloud provider — a company that stores and runs computers for other businesses — that hosted Fireworks’ research machines. On 24 October the intruder used that opening to reach the machines Fireworks uses to build and test its service, and ran a job there. Fireworks says it noticed the same day and called in outside investigators.

Fireworks says the intruder took information about customer accounts, including billing and shipping addresses and invoice history. For a limited number of accounts, stored data and parts of AI programs were also reached. Card numbers were not involved, because Fireworks says it does not keep them.

On 2 November, Fireworks warned Notion that a small set of data had been accessed. By the next morning, Notion had cut the connection, deleted what it had stored at Fireworks and stopped all traffic between them. On 6 November Fireworks said customer data might be involved. On 8 November it confirmed which data. On 9 November Notion began writing to the customers affected.

What was in it: transcripts of conversations people had with Notion AI through a feature called Insights — the questions typed in, and the answers that came back. Notion says fewer than one per cent of its workspaces were involved, the shared spaces where a team or a family keeps its pages.

Why it matters to you

Two things are worth taking from this, and neither is really about Notion.

The first is that the app you trust and the company holding your text are often not the same company. AI features are usually bought in from a specialist, so your words can pass through two or three businesses before an answer comes back. You agreed to the terms of one of them.

If a meeting note, a client list or a few lines about a health problem went through that feature, the sentence you wrote is now in the hands of someone you never dealt with.

The second is what people type into those boxes. On 11 November, Notion said it had cancelled some of its own access keys — long strings that let software log in to an account — after finding them inside the stolen conversations. Keys like that get into a chat box because a person pastes them there.

Notion has told affected customers to treat the data as exposed, and to change anything sensitive they typed into Notion AI during that period.

Rows of grey server racks in a vacant office, with cables hanging loose after the company moved out
Server racks in a vacated office in Toronto. AI features in everyday apps usually run on machines like these, owned by a different company than the app itself. Photo: Mike Beltzner, CC BY-SA 2.0.

What you can actually do

  • Treat an AI chat box like a postcard, not a sealed letter. Anyone who ends up holding it can read it.
  • Keep passwords, card and bank numbers, ID numbers and medical details out of every AI chat, whichever app it sits in.
  • If you have ever pasted a password into one, change that password today and switch on two-step login.
  • If you use Notion, search your inbox for a message from Notion dated 9 November or later. Notion says it contacted every affected workspace, so no message means yours was not among them.
  • Open the app yourself instead of following a link in any email about this. A breach is a useful excuse for a fake security notice.
  • Go through the AI settings of the apps you use and switch off the features you never touch. Each one you turn off is text that stays on your own account.
  • Keep medical, money and family notes somewhere you control — a paper notebook, or a file on your own device — rather than in a chat window.

Common misunderstandings

Myth: I have nothing worth stealing

The intruders were not after your shopping list. Fireworks says they wanted access to AI programs so they could sell that computing power to other people. Your conversations happened to be in the room.

Myth: If it cannot be traced to me, I am fine

Notion calls most of the stolen transcripts unattributable, meaning they carry no label saying who wrote them or which workspace they came from. That is a statement about labels, not about content. A conversation can hold your name, your employer, your child’s school or a password, and none of that needs a label to be useful to a stranger.

Myth: The app I trust is the company holding my data

Most apps rent their AI from someone else. That is how smaller companies can afford modern features, so it is not a scandal in itself. But it means one break-in can reach the customers of many products at once. This is the second breach at Fireworks in two months; the first was disclosed on 9 October.

What to watch for next

Expect email. Breach notices are public, and scammers read them too. A message that says it is contacting you about the Notion incident and asks you to confirm a password or a card number is not from Notion, whatever the logo looks like.

Expect the same story elsewhere. AI features are being added to email, calendars, photo apps and shopping apps, and each one usually involves a partner company running the program behind it. The pattern will come back.

The one thing to remember

Your words leave the app. Treat every AI chat box as a place that other people’s computers can read, and keep passwords and card numbers out of it.

Image: “388 King – server room” by Mike Beltzner, licensed CC BY-SA 2.0, via Wikimedia Commons.