Should staff put customer data into ChatGPT, Copilot or Claude?

In December 2023, Victoria's Department of Families, Fairness and Housing reported a privacy incident involving a child-protection worker using ChatGPT to draft a court report. The Victorian privacy regulator later found that the worker had entered names and risk-assessment information into the tool. The report also contained inaccurate information that understated risks to the child. The department accepted the findings and received a compliance notice requiring remedial action.^{[1, 2]}

For a business owner, this raises two questions: what customer data are staff giving AI, and what are they doing with its answers?

This is analysis of Australian privacy requirements and provider documentation, not legal advice. Product information was checked on 13 November 2026; today's products differ from those in the 2023 incident.

Should staff use personal AI accounts for work?

My recommendation is to require company-provided, managed AI accounts for work and put that rule in writing. Give staff an approved tool and examples of permitted use, so they can get useful work done within clear limits.

A paid personal subscription is still a personal account. Your business needs to control access, settings and what happens when an employee leaves. A staff member turning off model training does not establish those controls.

A starting policy to adapt: Use only company-approved AI tools and work accounts for business tasks. Do not enter work information into personal AI accounts. Follow the approved data categories below, and obtain approval before uploading customer records. Check AI-generated content before using it, and report accidental uploads or sharing immediately to the designated manager.

What protection does a business AI account provide?

Business accounts can provide organisational controls and stronger contractual protections. They still need configuring, and a promise not to train models on your information does not mean the provider never stores it.

ChatGPT Business and Enterprise exclude business content from model training by default. Microsoft says Copilot Chat prompts and responses under work-account protection do not train foundation models. Claude's commercial services also exclude content by default, with permission and feedback exceptions.^{[3, 4, 5]} Check that staff actually select the work account or workspace.

Separate three questions: training asks whether information improves models; retention asks how long it remains stored; access asks who or what can retrieve it. Chat history, provider access and organisational audit records may still exist when training is excluded.^{[6, 4]}

What is zero data retention, and can my business get it?

Zero data retention (ZDR) is an arrangement under which a provider does not retain the prompts and responses covered by that arrangement, subject to its exceptions. The service still processes the information to answer; coverage depends on the product, model, features and agreement.

ZDR differs from switching off training or deleting chats after a set period. For example, ChatGPT Temporary Chat excludes conversations from training but may retain them for up to 30 days for safety.^[7]

Get written confirmation of coverage for chats, uploads, saved memories and connected services. Ask what safety records or account metadata remain, which exceptions apply, and whether your own software stores copies. ZDR cannot fix excessive file permissions, a stolen staff login, or an employee sharing an answer with the wrong person.

What data should staff be allowed to enter?

Start with public business material and invented examples in approved work tools. Require approval for identifiable customer information, and keep highly sensitive records and access secrets out of routine chat. The table below is my suggested starting policy, not a universal legal classification.

For a hypothetical payment reminder, ask for a template with placeholders, then add the real name and amount in your own system. Remove identifying details before uploading; asking the chatbot to anonymise the original gives it the information first. Removing names alone may leave people identifiable from the remaining details.

What should I configure in the company account?

Assign an owner to configure and review the service before staff use it with customer information. Treat account protection, stored data, connected apps and output checking as separate controls.

1. Manage staff access. Give each person a work login, require multi-factor authentication, restrict administrator rights and remove access when they leave.
2. Check training and feedback. Confirm the business terms apply. Disable optional data sharing or feedback that permits content reuse where available; explain the remaining rules to staff.
3. Set retention deliberately. Choose the shortest period consistent with business and legal needs. Check uploads, projects and saved memories separately. Confirm ZDR in writing where required.
4. Limit sharing and connections. Restrict public links and unnecessary apps. Review permissions before connecting mailboxes or drives; web search and external agents can have different data-handling terms.^[13]
5. Check before acting. Require a person to verify facts and recipients before sending customer messages or making decisions. Storage controls would not have corrected the inaccurate court report in the opening case.
6. Keep evidence and a response route. Record approved uses, settings, terms and review dates. Name someone to receive incident reports, contain exposure and assess notification duties. Avoid creating extra copies of customer data just to document approval.

For wider access, see connecting AI assistants to company files. These controls also sit within the broader guide to defending against AI attacks.

What are the Australian privacy obligations?

A company account or ZDR arrangement does not itself establish permission to upload customer information. Where the Commonwealth Privacy Act applies, the business still needs to meet its privacy obligations. The opening case involved separate Victorian public-sector law.

The Australian Privacy Principles address collection notices (APP 5), permitted purposes and exceptions (APP 6), overseas disclosure (APP 8), accuracy (APP 10.2), and security and disposal (APP 11).^[14] The OAIC recommends keeping personal information, especially sensitive information, out of publicly available generative AI tools as best practice.^[15]

Many businesses with annual turnover of $3 million or less fall outside the Act's general coverage, but exceptions apply, including health-service providers. Check the OAIC's small-business checklist rather than deciding from turnover alone.^[16] Customer contracts and confidentiality commitments also need checking before approving a use.