By Darwin Palma, Lead Data Engineer at Vizorfin
“Can you show me where this number came from?”
That question — asked by a regulator, an auditor, or an internal compliance officer — is the moment where most fund operations teams discover whether their data infrastructure can support what’s being asked.
For funds with automated pipelines, the answer takes seconds. For funds with manual operations, the answer can take weeks.
This article is about data lineage: what it means, why regulators expect it, and how to build it.
What Data Lineage Actually Means
Data lineage is the ability to trace any data point — a number in a report, a field in a filing — back to its original source, through every transformation, validation, and aggregation step along the way.
It answers four questions:
- Where did this data come from? (Source system, file, or API)
- How was it transformed? (Calculation rules, validation logic, aggregation)
- Who or what touched it? (Automated pipeline, human edit, approval)
- When did this happen? (Timestamp and sequence)
If you can answer those four questions for every number in every filing, you have data lineage. If you can’t, you don’t.
Why Regulators Expect It
Regulators aren’t asking for data lineage out of curiosity. They’re asking because:
- Accuracy verification: They need to confirm that reported figures are correct.
- Fraud detection: They look for patterns where data is manipulated between source and filing.
- Audit reproducibility: They want to confirm that the same inputs produce the same outputs.
- Accountability: They need to know who is responsible for each step.
Under the SEC Marketing Rule, FCA rules, ASIC’s INFO 269, and equivalent regulations in Canada and New Zealand, the ability to demonstrate data integrity is a baseline expectation — not a nice-to-have.
Why Manual Operations Can’t Deliver It
In a spreadsheet-based process, data lineage is reconstructed after the fact. That means:
- Someone has to trace which cells reference which other cells
- Manual edits leave no trail unless version history is enabled (and even then, it’s limited)
- Multiple contributors create ambiguity about who changed what
- Copy-paste workflows break the chain of traceability
In practice, most manual operations can produce approximate lineage. Very few can produce defensible lineage.
What Automated Lineage Looks Like
In an automated pipeline, data lineage is built into the system:
- Every ingestion event is logged with source, timestamp, and checksum
- Every transformation is versioned with the rule that produced it
- Every validation is recorded with pass/fail status
- Every report is linked to the data that produced it
When a regulator asks “where did this number come from?”, the answer is a query, not a project.
The Technical Components
Building data lineage into fund operations requires:
- Ingestion logging: Every source file, API call, or manual input is recorded with metadata.
- Transformation versioning: Every rule applied to the data is stored with the version active at the time.
- Validation checkpoints: Data quality checks are logged, not just executed.
- Report-to-source mapping: Every output field is linked to its input fields.
- Immutable audit log: Records cannot be altered after the fact.
These are not exotic requirements. They’re standard data engineering practices. The challenge is that most fund operations teams have never been asked to build them.
A Practical Starting Point
If you’re evaluating your current data lineage capability, start with these questions:
- Can you trace a single field in your last regulatory filing back to its source?
- How long would it take to produce that trace?
- Would the answer be the same if you ran the query six months from now?
- Could you produce lineage for every field in every filing?
If the answers are uncertain, data lineage is a gap worth addressing — not because a regulator is asking today, but because they will eventually.
Where Vizorfin Fits
Vizorfin’s platform is built around traceability. Every data point ingested is logged, versioned, and linked to its downstream uses. Our architecture is designed so that “where did this number come from?” is a query, not a project.
Important: Vizorfin is a financial technology company. We are not a registered investment adviser, broker-dealer, or financial services licensee in any jurisdiction. Our software is a decision-support tool and should not be the sole basis for financial or compliance decisions. Individual results will vary based on your specific circumstances, data environment, and implementation. You should consult with qualified financial, legal, and tax professionals regarding your specific situation.
Risk Warning: The use of AI-powered financial automation tools involves risks. No specific financial outcome is guaranteed. You should not rely solely on automated systems for financial decision-making.
If you’d like to see how Vizorfin handles data lineage.
This content is for educational and informational purposes only and does not constitute financial, investment, legal, or tax advice.