What AI Cannot Do in Business Software

Knowing what AI cannot do is more valuable than knowing what it can. The capabilities are widely marketed. The limitations are quietly omitted. For business software, understanding these limitations prevents costly mistakes and misplaced trust.

AI cannot guarantee arithmetic accuracy. Large language models generate plausible text, not verified calculations. A model asked to calculate tax on £45,000 of income might produce the correct answer — or it might produce a convincing but incorrect one. The problem is that you cannot tell the difference by looking at the output. Financial calculations must use deterministic engines — explicit formulas with verified arithmetic — not probabilistic models that might round differently on Tuesday than they did on Monday.

AI cannot interpret law authoritatively. Tax law, employment law, data protection law — these are complex, nuanced, and context-dependent. AI can summarise legal provisions in accessible language. It can flag potential issues based on keyword matching. But it cannot provide the authoritative interpretation that a qualified professional can. Using AI to determine whether a specific expense is allowable for tax purposes is risky because the answer depends on facts and context that the AI may not fully understand.

AI cannot maintain state reliably. Business processes often involve sequential steps where the output of one step is the input to the next. A customer onboarding process, a filing workflow, a multi-step calculation. AI models process each interaction independently — they do not inherently track where you are in a process or ensure that steps are completed in the correct order. Workflows that require reliable state management need explicit process engines, not AI conversations.

AI cannot audit itself. When a business system produces an output — a tax calculation, a financial report, a compliance submission — the output must be traceable to its inputs and the logic that produced it. AI models are opaque by nature. You can see the input and the output but not the reasoning between them. This makes AI unsuitable for any output that may need to be explained to a regulator, an auditor, or a court.

AI cannot handle novel situations reliably. AI excels at pattern matching within its training data. It fails when confronted with situations that are significantly different from anything it has seen. In business software, novel situations are common — an unusual transaction, a new type of expense, a regulatory change. A system that handles routine cases well but fails unpredictably on edge cases is dangerous, because users may not recognise when they have hit an edge case.

AI cannot replace domain expertise. It can augment expertise — helping an accountant work faster, helping a business owner understand their data better. But it cannot substitute for the knowledge, judgement, and experience that come from working in a domain for years. An AI that helps a qualified accountant review transactions is useful. An AI that encourages a non-accountant to make tax decisions without professional advice is irresponsible.

AI cannot maintain consistency across interactions. The same question asked differently may produce different answers. The same data processed at different times may produce different categorisations. This variability is inherent in probabilistic models and is usually acceptable for suggestion and assistance tasks. It is unacceptable for tasks requiring consistency — recurring reports, regulatory submissions, financial calculations.

The appropriate role for AI in business software is clear: assistance with routine tasks, pattern recognition in data, natural language interaction for queries and search, and content generation for communication. The inappropriate role is equally clear: definitive calculations, authoritative compliance guidance, critical decision-making, and any output where inconsistency or error has financial or legal consequences.

At neart.ai, these boundaries are explicit in our architecture. AI enhances the user experience. Controlled logic governs the outputs. The line between the two is clearly drawn and rigorously maintained.