The Data Silo Problem: When Your Data Lives Everywhere
Six months ago, I inherited a mess. The company’s data was split across three completely separate systems with no bridge between them. User profiles sat in a MySQL database owned by the web team. Financial transactions lived in a PostgreSQL cluster — chosen specifically for ACID compliance. And roughly 200GB of raw clickstream logs were parked in a MinIO bucket as Parquet files, untouched and growing daily.
Then the marketing team dropped this on my desk: “What’s the total spend of users who clicked on our summer campaign banner?”
A simple question. Three separate data sources. No shared infrastructure.
I had three options, all of them bad:
- Write a Python script to fetch data from all three sources, join them in memory with Pandas, and hope the 16GB RAM server doesn’t choke.
- Build an Airflow ETL pipeline to consolidate everything into a single warehouse — a solid long-term solution, but two to three days of setup work for one ad-hoc question.
- Export CSVs and stitch them together in Excel. I’ve seen this approach end careers.
Why Cross-Database Queries Are Genuinely Hard
Traditional databases are designed to be self-contained. MySQL has no idea PostgreSQL exists. Neither can natively read a Parquet file sitting in object storage.
The real bottleneck is the compute layer. Each engine speaks its own dialect, uses its own storage format, and builds its own execution plan. Joining data across these boundaries almost always means moving data to wherever the compute lives — and that’s where the slowness and cost pile up.
ETL vs. Data Federation: Picking the Right Tool
I evaluated three common approaches before committing to anything:
- ETL (Extract, Transform, Load): Solid for scheduled reporting. But it introduces latency — you’re always querying yesterday’s snapshot, not live data. For a one-off analysis, building a pipeline is massive overkill.
- PostgreSQL Foreign Data Wrappers (FDW): Useful when Postgres is already your hub, but performance degrades noticeably when joining across multiple remote sources at once.
- Trino (formerly PrestoSQL): A distributed SQL query engine that stores nothing. It connects directly to each data source, fetches only the rows it needs, and executes the join in its own worker memory.
Trino won. For ad-hoc, multi-source queries against live data, nothing else came close. It treats every source — MySQL, Postgres, MinIO — as a “catalog,” so you can query across all of them with a single SQL statement.
Setting Up the Trino Environment with Docker
Docker is the fastest way to get this running. The entire stack — Trino, MySQL, Postgres, MinIO, and a Hive Metastore — can be up in under five minutes.
Trino’s configuration lives in .properties files under the /etc/trino/catalog/ directory. One file per data source.
1. Catalog Configuration Files
PostgreSQL Connector (postgres.properties):
connector.name=postgresql
connection-url=jdbc:postgresql://postgres-server:5432/finance_db
connection-user=admin
connection-password=secret_pass
MySQL Connector (mysql.properties):
connector.name=mysql
connection-url=jdbc:mysql://mysql-server:3306/user_db
connection-user=analyst
connection-password=another_secret
MinIO/S3 Connector (minio.properties):
Trino reads object storage through the Hive connector. This requires a Metastore — either Hive Metastore (HMS) or AWS Glue — to track table schemas and file locations. For local setups, a standalone HMS container works fine.
connector.name=hive
hive.s3.endpoint=http://minio-server:9000
hive.s3.aws-access-key=minio_admin
hive.s3.aws-secret-key=minio_password
hive.s3.path-style-access=true
hive.metastore.uri=thrift://metastore:9083
2. Preparing Your Raw Data
Not all your data will already be in a database. For quick CSV-to-JSON conversions before uploading files to MinIO, I’ve been using toolcraft.app/en/tools/data/csv-to-json. It runs entirely in the browser — nothing gets sent to a server — which matters when handling sensitive data. Handy for cleaning up small datasets before they land in MinIO for Trino to query.
The Query That Answers Three Databases at Once
With catalogs configured and Trino running, the query is just plain SQL. No new syntax to learn. The only difference is the three-part catalog.schema.table naming convention.
Here’s the exact query I ran for the marketing team’s request:
SELECT
u.username,
u.email,
SUM(t.amount) as total_spent,
l.campaign_id
FROM mysql.user_db.users u
JOIN postgres.finance_db.transactions t ON u.id = t.user_id
JOIN minio.logs.campaign_clicks l ON u.id = l.user_id
WHERE l.campaign_name = 'Summer_2024'
GROUP BY u.username, u.email, l.campaign_id
HAVING SUM(t.amount) > 100
ORDER BY total_spent DESC;
That single query hits three different network protocols, handles three different storage formats, and merges everything in Trino’s distributed memory. On our setup it returned roughly 4,200 rows in about 8 seconds. No ETL. No intermediate tables.
Performance: What I Learned the Hard Way
Trino is powerful, but federated queries have failure modes that don’t show up in single-source setups. Here’s what bit me early on.
Predicate Pushdown: Make Sure It’s Actually Happening
Trino tries to push your WHERE filters down to the source database. When it works, MySQL handles the filtering locally and only returns the matching rows. When it doesn’t, Trino pulls the entire table across the network first. On a 50-million-row table, that’s the difference between a 3-second query and a 45-minute one.
Run EXPLAIN ANALYZE on every slow query. Look at the Input rows vs Output rows in the connector scan stage. If input rows match your full table size, pushdown failed and you need to rethink the filter condition.
Large Cross-Catalog Joins Kill Performance Fast
Joining a 100M-row Postgres table with a 100M-row MySQL table means Trino has to shuffle enormous amounts of data to its worker nodes over the network. It works — but slowly.
The practical fix: keep your largest table in fast columnar storage (Parquet on MinIO), and apply tight filters on the smaller JDBC-backed tables. Let the columnar scan do the heavy lifting, not the network.
Network Latency is the Hidden Multiplier
Trino fetches data in real-time. Every millisecond of latency between Trino and your data sources compounds across millions of rows. Deploy Trino in the same VPC as the databases it queries. A Trino cluster in AWS us-east-1 talking to a MySQL server in a Singapore colocation facility will be painfully slow — regardless of how many worker nodes you add.
The Result: Analysis Instead of Data Plumbing
Before Trino, most of my time went to moving data — writing scripts, debugging pipelines, waiting for ETL jobs to finish. The actual analysis was almost an afterthought.
That ratio flipped. PostgreSQL, MySQL, and MinIO are now peer catalogs in a single SQL interface. Questions that used to take a full day to set up take minutes to answer. The data stays where it lives. Trino brings the compute to it.

