PingCAP, which builds the distributed database TiDB and runs TiDB Cloud, disclosed on 19 October that an unauthorised party used stolen service account credentials to reach internal systems and copy data and source code.

The company's central reassurance is that it found no evidence that data stored in TiDB itself was affected. That appears to be true, and it is narrower than it reads.

The dates do the work

Access ran from 9 September to 7 October. PingCAP detected the activity on 7 October and began disabling the affected credentials that day. Disclosure followed on 19 October.

Twenty-eight days of access, discovered on day twenty-eight. The detection did not shorten the intrusion; it ended it. An external incident response firm then reported 4 vulnerabilities on 12 and 13 October, all of which PingCAP says were fixed by 14 October.

What "no customer data" leaves out

The exported material, as reported from the 19 October disclosure, includes TiDB Cloud account details and business records — and SQL editor text and errors. The SQL editor is the web console where a customer types queries against their own database, which makes that last item the contents of their questions.

Sit with what a typed query contains. Table names, column names, the shape of the schema, and very often literal values in a WHERE clause: an account number somebody was chasing, an email address they were debugging, an identifier that means something.

The database was not reached. The questions people asked it were. For some customers those are close to the same thing, and the distinction is one only the customer can evaluate, because only the customer knows what they typed.

The account details run to user names, email addresses, organisation and project associations and cluster identifiers. The business records run to customer contacts, contract terms, orders, pricing, payment terms and billing configuration.

The API key table

One exported table held encrypted TiDB Cloud API key values. PingCAP says testing showed that one investigator-owned encrypted value could not be used directly as an API key, and that as of the 19 October disclosure it was still assessing whether other usable credentials were exposed.

Read that carefully. One value was tested, and the assessment is ongoing.

Encrypted is doing a lot of work in that sentence, and nobody outside the company can check it. Rotate the keys. The cost of rotating is an afternoon; the cost of being wrong about the encryption is everything the key reaches.

What the attacker claims

An extortion group has claimed a far larger theft, including a figure of about 10.6 terabytes and a replication feed pulling fresh internal data every six hours for twenty-two days. None of that is verified, and the outlet that reported it said so plainly.

The group did not claim to have taken TiDB databases themselves, which is at least consistent with PingCAP's account. Treat the rest as an assertion by a party with an interest in it being believed.

A separate notice from PingCAP's Japanese subsidiary, reported on 21 October, puts up to around 50,000 personal information records in Japan among the leaked data, and says no credit card information was exposed.

What to do

  • Rotate TiDB Cloud API keys, and do not wait for a finding that says you had to.
  • Review access logs and permissions on your clusters for the September and October window.
  • Assume the contents of your SQL editor history are known, and work out what that actually exposed in your case. Nobody else can do that for you.
  • If your contract terms and pricing are commercially sensitive, those were in the business records, and that is a different conversation with a different department.
  • Check whether anything in your infrastructure authenticates to TiDB Cloud with a long-lived key that nobody has rotated since it was issued.

What is not established

  • Whether any exported API key is usable. One was tested and the assessment continues.
  • How the service account credentials were stolen.
  • Whether the four vulnerabilities found in October were used during the intrusion. PingCAP says it is still investigating.
  • Whether the attacker's claims about volume and persistence are true.
  • What the gap between detection on 7 October and disclosure on 19 October was spent on.
  • Whether any customer has been harmed. No downstream incident has been reported.

PingCAP's own write-up could not be retrieved while this was written, and everything above is drawn from the reporting of it.