From rules to learning: machine learning, deep learning and generative AI in plain English

Finance has been automating its work for decades. An Excel IF formula, an ERP rule that matches bank lines to invoices, a workflow that sends any payment above a limit to the finance director: all of these are automation, and all of them follow rules that a person wrote down. What is new is software that writes its own rules by studying examples. That shift sounds technical, but it changes the questions you must ask as a professional. You can read a rule and decide whether it is right. You cannot read a learned model in the same way. You can only test what it does.

This lesson maps the territory: rules, machine learning, deep learning and generative AI, with what each is good at, how it fails and how you check it. When a vendor calls a product "AI-powered", you will be the person in the room who asks: which kind, trained on what, and tested how?

Rules: the automation you already know

A rules-based system does exactly what someone told it to do. Take a bank reconciliation matching rule: treat a bank line and a ledger line as a match when the amounts are equal and the dates are no more than three days apart. With the bank amount in B2, the ledger amount in E2 and the two dates in C2 and F2, the Excel version is =IF(AND(B2=E2,ABS(C2-F2)<=3),"Match","Review"). Excel stores dates as day numbers, so subtracting one date from another gives the gap in days.

Rules are transparent (an auditor can read and test them), predictable (the same input always gives the same output) and cheap to run. Their weakness is that the world is messier than any rule. The same supplier appears as "Kanem Packaging Ltd", "KANEM PKG" and "Kanem Packaging Limited". A narration reads "part pymt inv 4471 & 4472". A customer settles three invoices with one transfer, less a deduction for damaged goods. Each exception needs another rule, until the rule book becomes impossible to maintain. That is where learning systems become attractive.

Machine learning: rules learned from examples

Machine learning turns the process around. Instead of writing the rule, you give the computer many examples together with the right answer, and an algorithm finds the patterns that connect them.

Suppose Ardena Foods Plc, our fictional Nigerian food manufacturer, has 24 months of supplier invoice lines already coded to the general ledger. Each line has a supplier, a description, an amount, a cost centre and the account code a person chose. A machine learning model studies those examples and learns, for instance, that diesel from a particular supplier charged to the Kaduna factory almost always goes to factory power and fuel. Given a new line, it suggests a code with a confidence score between 0 and 1. The team can accept high-confidence suggestions after sample review and send low-confidence ones to a person.

There are two main families:

• Supervised learning uses labelled examples: past data where the right answer is known. Classification predicts a category: which account code, will this borrower default, is this transfer fraudulent. Regression predicts a number: next month's sales, or how many days a customer will take to pay.
• Unsupervised learning has no labels. It finds structure by itself: grouping customers by payment behaviour, or flagging journal entries that look unlike all the others. Much anomaly detection works this way, and we use it in module 9.

A third family, reinforcement learning, learns by trial and error from rewards. A form of it, using feedback from human reviewers, is part of how chat assistants are trained.

The discipline that makes it work: testing on unseen data

A model can memorise its training examples the way a student can memorise last year's exam answers. This is called overfitting, and it produces a model that looks brilliant on old data and fails on new data. The protection is simple and non-negotiable: hold back data the model never sees during training, and measure performance only on that.

In finance, hold back a later period rather than a random sample. If you train on January 2024 to March 2026 and test on April to June 2026, you are testing what you actually need, which is performance on the future. A random sample lets the model peek, because the same customers, suppliers and seasonal patterns appear on both sides of the split.

Two more consequences follow. First, a model learns whatever is in its labels, including inconsistent coding and any bias in past lending decisions. Second, models age. When prices, exchange rates or customer behaviour shift, a model trained on the old world drifts, so monitoring after go-live is part of the job.

Deep learning: many layers, learned features

Deep learning is a kind of machine learning that uses neural networks: many layers of simple calculations, each passing its results to the next. With enough layers and examples, the network learns useful features by itself. Shown thousands of scanned invoices, it learns which marks on the page are the supplier name, the date and the total, without anyone describing those things to it.

Deep learning powers the tools that read documents, transcribe results calls and understand language. It needs large amounts of data and computing power, so finance teams usually buy it inside products rather than build it. It is also harder to explain: you can measure how often it is right, but not easily say why it produced one particular output. That matters when the output is a credit decision about a person, or a figure an auditor will test.

Generative AI: models that produce new content

Generative AI produces new content: text, tables, images and code. The large language models behind assistants such as ChatGPT, Claude, Copilot or Gemini are deep learning models trained on vast amounts of text to predict what comes next, and then trained further to follow instructions. Lesson 3 opens them up properly.

The key difference for you is how you test them. A classification model gives an answer you can score against a known label: the account code was right or wrong. A generative model writes open-ended output with no single right answer, so you check it against source documents and rules, as with the variance commentary in lesson 1. You can also ask a generative assistant to pick an account code, which needs no training data, but it must then be tested like any other model.

Worked example

Tamari Microfinance Bank, our fictional microfinance bank, is offered a fraud detection model for mobile transfers. The vendor's slide says the model is "99% accurate". Tamari's risk team asks to run it on one month of past transfers that the model was never trained on, a month in which every fraud case has since been confirmed by investigation.

Step 1: build the confusion matrix

A confusion matrix sets what the model said against what actually happened.

The four cells are: 20 frauds caught, 100 genuine customers flagged by mistake (false positives), 5 frauds missed (false negatives), and 9,875 genuine transfers correctly left alone. Before anything else, check the arithmetic: rows and columns both add to 10,000.

Step 2: calculate the measures

Accuracy is close to the vendor's claim. Now look at what it hides.

Step 3: compare with doing nothing

Imagine a "model" that never flags anything. It is right about all 9,975 genuine transfers and wrong about the 25 frauds, so its accuracy is 9,975 ÷ 10,000 = 99.75%. That is higher than the real model, and it catches no fraud at all. When the events you care about are rare, as fraud, loan default and material misstatement usually are, accuracy flatters. Recall and precision tell you what you need to know.

Step 4: look at value and at the threshold

The 25 frauds totalled ₦18.4 million. The 20 caught were worth ₦14.9 million and the 5 missed ₦3.5 million, so the model caught 81.0% of fraud by value, slightly better than its 80.0% by count. Always ask for both.

The model flags a transfer when its score passes a threshold, and the threshold is a choice:

Every alert needs an analyst. At 15 minutes an alert, 120 alerts is 30 hours of review a month and 310 alerts is 77.5 hours, and a lower threshold also blocks more genuine customers. The AI practitioner produces this table. Choosing the row is a risk appetite decision for management, weighing missed fraud against review cost and customer friction, documented with reasons.

Prompt templates

You are a model risk reviewer at a [microfinance bank / manufacturer / asset manager].
A vendor has described a [fraud detection / credit scoring / account coding] model as follows:
"""
[paste the vendor's description and any performance figures]
"""
Task: list the questions we must ask before we rely on this model.
Cover: the training data and period; how the labels were confirmed; the test period and
whether it was held out from training; recall and precision at the threshold we would use;
performance by value as well as by count; how individual outputs can be explained;
and how drift will be monitored after go-live.
Format: a numbered list, each question followed by one line on why it matters.
Do not assume any performance figure that is not stated above.

Use this before a meeting with an AI vendor, or when reviewing a model built in-house.

For each finance task below, say whether the best fit is a rules engine, a machine learning
model or a generative AI assistant, and explain why in one sentence. Then name the main risk
and one control for that risk.
Tasks:
[list 5 to 10 tasks from your own work]
Format: a table with columns Task | Best fit | Why | Main risk | Control.
If a task needs a combination, say which part each approach does.

Use this when deciding where automation belongs in your own process.

Check it

• Confirm the test data was held out from training and comes from a later period than the training data.
• Rebuild the confusion matrix and check that rows and columns add to the total number of cases.
• Recalculate accuracy, recall and precision yourself, and compare accuracy with the do-nothing baseline.
• Ask for results by value as well as by count.
• Find out who confirmed the labels (fraud, default, correct account code) and how.
• Agree how performance will be monitored after go-live, and what result triggers a review.

Lecturer's note: The mistake to avoid is accepting a single headline accuracy figure. In finance, the events that matter most are rare, so a model can be highly "accurate" while missing the very cases you bought it to find. Ask for the confusion matrix at the threshold you will actually operate, on a later period the model never saw, and judge it against the cost of a missed case and the cost of a false alarm. That judgement belongs to you, not to the vendor.

Key points

• Rules are written by people and can be read and audited; learned models can only be tested.
• Supervised learning learns from labelled examples; unsupervised learning finds structure without labels; deep learning uses many-layered neural networks; generative AI produces new content.
• A model is only as good as its labels, must be tested on later data it never saw, and needs monitoring after go-live.
• Accuracy misleads when events are rare: use recall, precision, value-weighted results and a do-nothing baseline.
• Choosing a threshold is a business judgement about the cost of misses against the cost of false alarms.

Practice task

Tamari tests a loan default model on 2,000 loans from a held-out later period. In that period 160 loans defaulted. The model predicted default for 210 loans, of which 120 actually defaulted. Build the confusion matrix, calculate accuracy, recall and precision, compare accuracy with a model that predicts no defaults at all, and write a note of no more than 100 words to the head of credit. A good answer finds 120 true positives, 90 false positives, 40 missed defaulters and 1,750 true negatives; accuracy of 93.5% against a do-nothing baseline of 92.0%; recall of 75.0% and precision of 57.1%. The note points out that 40 defaulters were missed, asks what those loans were worth, and asks which threshold produced these results before recommending anything.