Columnar and Idempotent Architecture for Financial Risk Analysis: Design and Evaluation with Apache Parquet and Business Intelligence

Abstract

This paper presents a portfolio-oriented financial analytics architecture that prioritizes operational simplicity, reproducibility, and low cost. The complete workflow relies on three pillars: (i) collaborative repositories for raw data ingestion; (ii) a modular ETL process in Python/Pandas that normalizes data, validates quality rules, calculates performance and risk indicators, and generates Apache Parquet columnar artifacts ready for consumption; and (iii) Microsoft Power BI as a highly memory-efficient interactive visualization layer. The system is designed to be idempotent: the same input produces the same results, accompanied by footprints (hashes) and metadata that facilitate traceability. We describe how price calendar densification is managed, how portfolio and index are compared via rebasing, and the construction of high-interpretive-value DAX metrics (cumulative returns, 21D rolling volatility, ticker contribution, maximum drawdown, and seasonality). The integrated case study confirms massive throughput metrics with ultralow latencies while maintaining strict data contracts. Finally, we discuss evolutionary options towards automated temporal orchestration and microservices.