Project overview
The lender managed a growing SME loan book using overnight batch scoring and spreadsheet-based reporting to the regulator. Decisions lagged market conditions by days and the reporting process consumed a team for a week every month.
CodeQube designed and delivered a real-time risk platform combining streaming data ingestion, a governed feature store, ML scoring models and an audit-ready reporting layer.
Business problem
Where the client started.
Risk scores were refreshed once a day; intraday events such as missed payments or bureau alerts were not reflected until the next morning.
Model logic was embedded in SQL stored procedures with no versioning, testing or explainability.
Monthly regulatory reports were assembled manually from six systems, with frequent reconciliation errors.
Auditors could not trace how a given decision had been reached.
Proposed solution
What we proposed, and why.
A Kafka-based streaming backbone ingesting payment events, bureau updates and account changes in real time.
A feature store and model registry (MLflow) with versioned gradient-boosted scoring models and SHAP-based explainability per decision.
A rules engine layered on top of ML scores so credit policy remained under business control.
An immutable decision ledger and automated reporting pipeline producing regulator-format outputs on demand.
Architecture
How the system is put together.
Event streams land in Kafka and are processed by Flink jobs that compute features and trigger scoring. Scores and explanations are written to an append-only decision ledger in PostgreSQL and replicated to Snowflake for reporting.
Technology stack
Backend, frontend, data and cloud.
Challenges faced
What got in the way, and how we dealt with it.
Challenge
The regulator required every automated decision to be explainable to the customer and auditable for seven years.
Resolution
Persisted SHAP values and model version alongside each score in an append-only ledger; built a decision-replay tool for auditors.
Implementation details
How delivery was sequenced.
- 1
Assessment & target architecture (months 1–2)
Data source inventory, rule extraction, model baseline and an architecture approved by the client’s risk committee.
- 2
Streaming foundation (months 3–4)
Kafka, CDC connectors and Flink jobs computing the first 60 features with data quality checks.
- 3
Scoring & rules (months 5–7)
Model training pipeline, registry, scoring API, rules engine and decision ledger; shadow-run against legacy scoring.
- 4
Console & reporting (months 8–10)
Risk analyst console, regulatory dashboards and automated report generation; parallel run and regulator sign-off.
Non-functional requirements
Security and scalability considerations.
Security
- Field-level encryption for PII with KMS-managed keys; tokenised identifiers throughout the analytics layer.
- Role-based access with SSO and step-up authentication for policy changes.
- Full audit trail on model deployments, rule changes and data access, retained for seven years.
Scalability
- Flink parallelism scales with Kafka partitions; the platform was load-tested to 20× current event volume.
- Scoring service autoscaled on EKS with p99 latency under 120ms.
- Snowflake separation of storage and compute keeps reporting workloads from affecting real-time processing.
Results & business impact
What it delivered.
Business impact
- Early-delinquency detection improved by 3.5 days on average, allowing earlier intervention.
- Risk team redeployed from report assembly to portfolio analysis.
- The platform now underpins two additional lending products launched since go-live.
More case studies